-
Notifications
You must be signed in to change notification settings - Fork 115
Simplify Network Broker config generation instructions #2418
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -36,36 +36,26 @@ The Semgrep Network Broker is available to Enterprise tier users. | |||||
|
|
||||||
| Ensure that you are logged in to the server where you want to run Semgrep Network Broker. Complete the following steps while logged in to that server. | ||||||
|
|
||||||
| ### Create the config file | ||||||
|
|
||||||
| ### 1. Create the config file | ||||||
| <Tabs | ||||||
| defaultValue="current" | ||||||
| values={[ | ||||||
| {label: 'v0.25.0 and later', value: 'current'}, | ||||||
| {label: 'v0.24.0 and earlier', value: 'legacy'} | ||||||
| ]} | ||||||
| > | ||||||
|
|
||||||
| <TabItem value='current'> | ||||||
|
|
||||||
| Create a `config.yaml` file similar to the following snippet, or copy a starting config from the Semgrep AppSec Platform at **Settings > Broker**. The steps required to generate values for the placeholders `SEMGREP_LOCAL_ADDRESS`, `YOUR_PRIVATE_KEY`, and `YOUR_BASE_URL`, as well as the scopes required for the access tokens, are provided in subsequent steps of this guide. | ||||||
|
|
||||||
| ```yaml | ||||||
| inbound: | ||||||
| wireguard: | ||||||
| localAddress: SEMGREP_LOCAL_ADDRESS | ||||||
| privateKey: YOUR_PRIVATE_KEY | ||||||
| peers: | ||||||
| - endpoint: wireguard.semgrep.dev:51820 | ||||||
| allowlist: [] | ||||||
| gitlab: | ||||||
| baseUrl: YOUR_BASE_URL | ||||||
| token: GITLAB_PAT | ||||||
| ``` | ||||||
|
|
||||||
| 1. Log in to the Semgrep AppSec platform and navigate to **Settings > Broker**. | ||||||
| 2. Copy the starting config into a `config.yaml` file. It should look similar to this: | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| ```yaml | ||||||
| inbound: | ||||||
| wireguard: | ||||||
| localAddress: SEMGREP_LOCAL_ADDRESS | ||||||
| peers: | ||||||
| - endpoint: wireguard.semgrep.dev:51820 | ||||||
| ``` | ||||||
| </TabItem> | ||||||
| <TabItem value='legacy'> | ||||||
|
|
||||||
| :::note | ||||||
| Semgrep recommends that users running Network Broker v0.24.0 or earlier to upgrade to v0.25.0 or later. This enables the use of a simplified config file. | ||||||
| ::: | ||||||
|
|
@@ -95,55 +85,51 @@ The `publicKey` value should be entered precisely as shown in the example: | |||||
| 4EqJwDZ8X/qXB5u3Wpo2cxnKlysec93uhRvGWPix0lg= | ||||||
| ``` | ||||||
|
|
||||||
| #### Add your local address to the config | ||||||
| 1. Convert your organization ID to hexadecimal. The organization ID is found in Semgrep AppSec Platform under [**Settings > General > Identifiers**](https://semgrep.dev/orgs/-/settings/general/identifiers) in Semgrep AppSec Platform. This is sometimes also called a deployment ID. You can use a tool such as [Decimal to Hexadecimal converter](https://www.rapidtables.com/convert/number/decimal-to-hex.html) to perform the conversion if needed. | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| 2. Embed the resulting hexadecimal value in the string `fdf0:59dc:33cf:9be8:0:ORGANIZATION_ID:0:1`, replacing `ORGANIZATION_ID` with the value. | ||||||
| 3. Update the `localAddress` field of `config.yaml`, replacing `SEMGREP_LOCAL_ADDRESS` with the string you generated in Step 2. | ||||||
|
|
||||||
| ```yaml | ||||||
| inbound: | ||||||
| wireguard: | ||||||
| localAddress: fdf0:59dc:33cf:9be8:0:ORGANIZATION_ID:0:1 | ||||||
| ``` | ||||||
| </TabItem> | ||||||
| </Tabs> | ||||||
|
|
||||||
| #### Multiple configuration files | ||||||
| You can overlay multiple configuration files on top of each other by passing multiple `-c` arguments: | ||||||
|
|
||||||
| ```console | ||||||
| semgrep-network-broker -c config1.yaml -c config2.yaml -c config3.yaml | ||||||
| ``` | ||||||
|
|
||||||
| Note that arrays are replaced, while maps are merged. | ||||||
|
|
||||||
| ### Generate a keypair | ||||||
|
|
||||||
| The broker requires a WireGuard keypair to establish a secure connection. To generate your private key to replace `YOUR_PRIVATE_KEY` in the config template: | ||||||
|
|
||||||
| 1. Determine the [Network Broker version](https://github.com/semgrep/semgrep-network-broker/pkgs/container/semgrep-network-broker) you want to use. The format should be similar to `v0.22.0`. Most users should use the latest version, especially when setting up the broker for the first time. | ||||||
| 1. Run the following command in the CLI to generate your private key, replacing the placeholder with the Network Broker version number: | ||||||
| <pre class="language-console"><code>docker run ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> genkey</code></pre> | ||||||
| 1. Run the following command in the CLI to generate your public key, replacing the placeholders with your private key generated in the previous step and the Network Broker version number: | ||||||
|
|
||||||
| <pre class="language-console"><code>echo <span className="placeholder">"YOUR_PRIVATE_KEY"</span> | sudo docker run -i ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> pubkey</code></pre> | ||||||
|
|
||||||
| :::info Key sharing | ||||||
| Your public key is safe to share. Do **not** share your private key with anyone, including Semgrep. | ||||||
| ::: | ||||||
|
|
||||||
| ### Update the config with the keypair | ||||||
|
|
||||||
| 1. Update the `config.yaml` file by replacing `YOUR_PRIVATE_KEY` with the value of your private key. | ||||||
| 1. Add your public key to the Semgrep AppSec Platform: | ||||||
| ### 2. Generate and store your private key | ||||||
| * Network Broker requires a WireGuard keypair to establish a secure connection. To generate your private key: | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| 1. Determine the [Network Broker version](https://github.com/semgrep/semgrep-network-broker/pkgs/container/semgrep-network-broker) you want to use. The format should be similar to `v0.36.0`. Most users should use the latest version, especially when setting up the broker for the first time. | ||||||
| 2. Run the following command in your terminal to generate your private key, replacing the placeholder with the Network Broker version number: | ||||||
| <pre class="language-console"><code>docker run ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> genkey</code></pre> | ||||||
| 3. Store your private key in the config file by adding a `privateKey` field under `wireguard` section with its value set to the key you just generated. | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| ### 3. Share your public key with Semgrep | ||||||
| :::info Key sharing | ||||||
| Your public key is safe to share. **Never** share your private key with anyone, including Semgrep. | ||||||
| ::: | ||||||
| 1. Run the following command in your terminal to generate your public key, replacing the placeholders with your private key generated in the previous step and the Network Broker version number: | ||||||
| <pre class="language-console"><code>echo <span className="placeholder">"YOUR_PRIVATE_KEY"</span> | sudo docker run -i ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> pubkey</code></pre> | ||||||
|
|
||||||
| ### 4. Add your public key to the Semgrep AppSec Platform | ||||||
| 1. Log in to Semgrep AppSec Platform. | ||||||
| 2. Navigate to **Settings** > **Broker**. | ||||||
| 3. Paste your public key and click **Add Public Key**. | ||||||
|  | ||||||
|
|
||||||
| ### Update the config with your SCM information | ||||||
| ### 5. Update the config with your SCM information | ||||||
| 1. Update the `config.yaml` by replacing the SCM information containing `YOUR_BASE_URL` with your SCM and its base URL for Azure DevOps, GitHub, GitLab, or Bitbucket Data Center. | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| Update the `config.yaml` by replacing the SCM information containing `YOUR_BASE_URL` with your SCM and its base URL for Azure DevOps, GitHub, GitLab, or Bitbucket Data Center. | ||||||
|
|
||||||
| <Tabs | ||||||
| defaultValue="gh" | ||||||
| values={[ | ||||||
| {label: 'Azure DevOps', value: 'ado'}, | ||||||
| {label: 'Bitbucket', value: 'bb'}, | ||||||
| {label: 'GitHub', value: 'gh'}, | ||||||
| {label: 'GitLab', value: 'gl'}, | ||||||
| ]} | ||||||
| > | ||||||
| <Tabs | ||||||
| defaultValue="gh" | ||||||
| values={[ | ||||||
| {label: 'Azure DevOps', value: 'ado'}, | ||||||
| {label: 'Bitbucket', value: 'bb'}, | ||||||
| {label: 'GitHub', value: 'gh'}, | ||||||
| {label: 'GitLab', value: 'gl'}, | ||||||
| ]} | ||||||
| > | ||||||
|
|
||||||
| <TabItem value='ado'> | ||||||
|
|
||||||
|
|
@@ -199,23 +185,21 @@ Semgrep recommends providing the access token when you [connect the source code | |||||
| </TabItem> | ||||||
| </Tabs> | ||||||
|
|
||||||
| ### Add your local address to the config | ||||||
| ### 6. Start the Network Broker | ||||||
| i. Run the following command to start Semgrep Network Broker with your completed configuration file: | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| 1. Convert your organization ID to hexadecimal. The organization ID is found in Semgrep AppSec Platform under [**Settings > General > Identifiers**](https://semgrep.dev/orgs/-/settings/general/identifiers) in Semgrep AppSec Platform. This is sometimes also called a deployment ID. You can use a tool such as [Decimal to Hexadecimal converter](https://www.rapidtables.com/convert/number/decimal-to-hex.html) to perform the conversion if needed. | ||||||
| 2. Embed the resulting hexadecimal value in the string `fdf0:59dc:33cf:9be8:0:ORGANIZATION_ID:0:1`, replacing `ORGANIZATION_ID` with the value. | ||||||
| 3. Update the `localAddress` field of `config.yaml`, replacing `SEMGREP_LOCAL_ADDRESS` with the string you generated in Step 2. | ||||||
| <pre class="language-console"><code>sudo docker run -d -it --rm -v $(pwd):/emt ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> -c /emt/config.yaml</code></pre> | ||||||
|
|
||||||
| ```yaml | ||||||
| inbound: | ||||||
| wireguard: | ||||||
| localAddress: fdf0:59dc:33cf:9be8:0:ORGANIZATION_ID:0:1 | ||||||
| ``` | ||||||
|
|
||||||
| ### Start the broker | ||||||
| #### Multiple configuration files | ||||||
| You can overlay multiple configuration files on top of each other by passing multiple `-c` arguments: | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| ```console | ||||||
| semgrep-network-broker -c config1.yaml -c config2.yaml -c config3.yaml | ||||||
| ``` | ||||||
|
|
||||||
| Run the following command to start Semgrep Network Broker with your completed configuration file: | ||||||
| Note that arrays are replaced, while maps are merged. | ||||||
|
|
||||||
| <pre class="language-console"><code>sudo docker run -d -it --rm -v $(pwd):/emt ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> -c /emt/config.yaml</code></pre> | ||||||
|
|
||||||
| ## Check Semgrep Network Broker logs | ||||||
|
|
||||||
|
|
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.