Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 56 additions & 72 deletions docs/semgrep-ci/network-broker.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,36 +36,26 @@ The Semgrep Network Broker is available to Enterprise tier users.

Ensure that you are logged in to the server where you want to run Semgrep Network Broker. Complete the following steps while logged in to that server.

### Create the config file

### 1. Create the config file
<Tabs
defaultValue="current"
values={[
{label: 'v0.25.0 and later', value: 'current'},
{label: 'v0.24.0 and earlier', value: 'legacy'}
]}
>

<TabItem value='current'>

Create a `config.yaml` file similar to the following snippet, or copy a starting config from the Semgrep AppSec Platform at **Settings > Broker**. The steps required to generate values for the placeholders `SEMGREP_LOCAL_ADDRESS`, `YOUR_PRIVATE_KEY`, and `YOUR_BASE_URL`, as well as the scopes required for the access tokens, are provided in subsequent steps of this guide.

```yaml
inbound:
wireguard:
localAddress: SEMGREP_LOCAL_ADDRESS
privateKey: YOUR_PRIVATE_KEY
peers:
- endpoint: wireguard.semgrep.dev:51820
allowlist: []
gitlab:
baseUrl: YOUR_BASE_URL
token: GITLAB_PAT
```

1. Log in to the Semgrep AppSec platform and navigate to **Settings > Broker**.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Log in to the Semgrep AppSec platform and navigate to **Settings > Broker**.
1. Sign in to Semgrep AppSec platform and navigate to **Settings > Broker**.

2. Copy the starting config into a `config.yaml` file. It should look similar to this:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. Copy the starting config into a `config.yaml` file. It should look similar to this:
2. Create a `config.yaml` file that contains the following starting configuration:

```yaml
inbound:
wireguard:
localAddress: SEMGREP_LOCAL_ADDRESS
peers:
- endpoint: wireguard.semgrep.dev:51820
```
</TabItem>
<TabItem value='legacy'>

:::note
Semgrep recommends that users running Network Broker v0.24.0 or earlier to upgrade to v0.25.0 or later. This enables the use of a simplified config file.
:::
Expand Down Expand Up @@ -95,55 +85,51 @@ The `publicKey` value should be entered precisely as shown in the example:
4EqJwDZ8X/qXB5u3Wpo2cxnKlysec93uhRvGWPix0lg=
```

#### Add your local address to the config
1. Convert your organization ID to hexadecimal. The organization ID is found in Semgrep AppSec Platform under [**Settings > General > Identifiers**](https://semgrep.dev/orgs/-/settings/general/identifiers) in Semgrep AppSec Platform. This is sometimes also called a deployment ID. You can use a tool such as [Decimal to Hexadecimal converter](https://www.rapidtables.com/convert/number/decimal-to-hex.html) to perform the conversion if needed.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Convert your organization ID to hexadecimal. The organization ID is found in Semgrep AppSec Platform under [**Settings > General > Identifiers**](https://semgrep.dev/orgs/-/settings/general/identifiers) in Semgrep AppSec Platform. This is sometimes also called a deployment ID. You can use a tool such as [Decimal to Hexadecimal converter](https://www.rapidtables.com/convert/number/decimal-to-hex.html) to perform the conversion if needed.
1. Convert your organization ID to hexadecimal. The organization ID is found in Semgrep AppSec Platform under [**Settings > General > Identifiers**](https://semgrep.dev/orgs/-/settings/general/identifiers). This value is sometimes called a deployment ID. You can use a tool such as [Decimal to Hexadecimal converter](https://www.rapidtables.com/convert/number/decimal-to-hex.html) to perform the conversion if needed.

2. Embed the resulting hexadecimal value in the string `fdf0:59dc:33cf:9be8:0:ORGANIZATION_ID:0:1`, replacing `ORGANIZATION_ID` with the value.
3. Update the `localAddress` field of `config.yaml`, replacing `SEMGREP_LOCAL_ADDRESS` with the string you generated in Step 2.

```yaml
inbound:
wireguard:
localAddress: fdf0:59dc:33cf:9be8:0:ORGANIZATION_ID:0:1
```
</TabItem>
</Tabs>

#### Multiple configuration files
You can overlay multiple configuration files on top of each other by passing multiple `-c` arguments:

```console
semgrep-network-broker -c config1.yaml -c config2.yaml -c config3.yaml
```

Note that arrays are replaced, while maps are merged.

### Generate a keypair

The broker requires a WireGuard keypair to establish a secure connection. To generate your private key to replace `YOUR_PRIVATE_KEY` in the config template:

1. Determine the [Network Broker version](https://github.com/semgrep/semgrep-network-broker/pkgs/container/semgrep-network-broker) you want to use. The format should be similar to `v0.22.0`. Most users should use the latest version, especially when setting up the broker for the first time.
1. Run the following command in the CLI to generate your private key, replacing the placeholder with the Network Broker version number:
<pre class="language-console"><code>docker run ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> genkey</code></pre>
1. Run the following command in the CLI to generate your public key, replacing the placeholders with your private key generated in the previous step and the Network Broker version number:

<pre class="language-console"><code>echo <span className="placeholder">"YOUR_PRIVATE_KEY"</span> | sudo docker run -i ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> pubkey</code></pre>

:::info Key sharing
Your public key is safe to share. Do **not** share your private key with anyone, including Semgrep.
:::

### Update the config with the keypair

1. Update the `config.yaml` file by replacing `YOUR_PRIVATE_KEY` with the value of your private key.
1. Add your public key to the Semgrep AppSec Platform:
### 2. Generate and store your private key
* Network Broker requires a WireGuard keypair to establish a secure connection. To generate your private key:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Network Broker requires a WireGuard keypair to establish a secure connection. To generate your private key:
The Network Broker requires a WireGuard keypair to establish a secure connection. To generate your private key:

1. Determine the [Network Broker version](https://github.com/semgrep/semgrep-network-broker/pkgs/container/semgrep-network-broker) you want to use. The format should be similar to `v0.36.0`. Most users should use the latest version, especially when setting up the broker for the first time.
2. Run the following command in your terminal to generate your private key, replacing the placeholder with the Network Broker version number:
<pre class="language-console"><code>docker run ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> genkey</code></pre>
3. Store your private key in the config file by adding a `privateKey` field under `wireguard` section with its value set to the key you just generated.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
3. Store your private key in the config file by adding a `privateKey` field under `wireguard` section with its value set to the key you just generated.
3. Store your private key in the config file by adding a `privateKey` field under `wireguard` section with its value set to the key you generated.


### 3. Share your public key with Semgrep
:::info Key sharing
Your public key is safe to share. **Never** share your private key with anyone, including Semgrep.
:::
1. Run the following command in your terminal to generate your public key, replacing the placeholders with your private key generated in the previous step and the Network Broker version number:
<pre class="language-console"><code>echo <span className="placeholder">"YOUR_PRIVATE_KEY"</span> | sudo docker run -i ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> pubkey</code></pre>

### 4. Add your public key to the Semgrep AppSec Platform
1. Log in to Semgrep AppSec Platform.
2. Navigate to **Settings** > **Broker**.
3. Paste your public key and click **Add Public Key**.
![Screenshot of Semgrep AppSec Platform's Network Broker page](/img/scp-broker.png#md-width)

### Update the config with your SCM information
### 5. Update the config with your SCM information
1. Update the `config.yaml` by replacing the SCM information containing `YOUR_BASE_URL` with your SCM and its base URL for Azure DevOps, GitHub, GitLab, or Bitbucket Data Center.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Update the `config.yaml` by replacing the SCM information containing `YOUR_BASE_URL` with your SCM and its base URL for Azure DevOps, GitHub, GitLab, or Bitbucket Data Center.
Update the `config.yaml` by replacing the SCM information containing `YOUR_BASE_URL` with your SCM and its base URL for Azure DevOps, GitHub, GitLab, or Bitbucket Data Center.


Update the `config.yaml` by replacing the SCM information containing `YOUR_BASE_URL` with your SCM and its base URL for Azure DevOps, GitHub, GitLab, or Bitbucket Data Center.

<Tabs
defaultValue="gh"
values={[
{label: 'Azure DevOps', value: 'ado'},
{label: 'Bitbucket', value: 'bb'},
{label: 'GitHub', value: 'gh'},
{label: 'GitLab', value: 'gl'},
]}
>
<Tabs
defaultValue="gh"
values={[
{label: 'Azure DevOps', value: 'ado'},
{label: 'Bitbucket', value: 'bb'},
{label: 'GitHub', value: 'gh'},
{label: 'GitLab', value: 'gl'},
]}
>

<TabItem value='ado'>

Expand Down Expand Up @@ -199,23 +185,21 @@ Semgrep recommends providing the access token when you [connect the source code
</TabItem>
</Tabs>

### Add your local address to the config
### 6. Start the Network Broker
i. Run the following command to start Semgrep Network Broker with your completed configuration file:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
i. Run the following command to start Semgrep Network Broker with your completed configuration file:
Run the following command to start Semgrep Network Broker with your completed configuration file:


1. Convert your organization ID to hexadecimal. The organization ID is found in Semgrep AppSec Platform under [**Settings > General > Identifiers**](https://semgrep.dev/orgs/-/settings/general/identifiers) in Semgrep AppSec Platform. This is sometimes also called a deployment ID. You can use a tool such as [Decimal to Hexadecimal converter](https://www.rapidtables.com/convert/number/decimal-to-hex.html) to perform the conversion if needed.
2. Embed the resulting hexadecimal value in the string `fdf0:59dc:33cf:9be8:0:ORGANIZATION_ID:0:1`, replacing `ORGANIZATION_ID` with the value.
3. Update the `localAddress` field of `config.yaml`, replacing `SEMGREP_LOCAL_ADDRESS` with the string you generated in Step 2.
<pre class="language-console"><code>sudo docker run -d -it --rm -v $(pwd):/emt ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> -c /emt/config.yaml</code></pre>

```yaml
inbound:
wireguard:
localAddress: fdf0:59dc:33cf:9be8:0:ORGANIZATION_ID:0:1
```

### Start the broker
#### Multiple configuration files
You can overlay multiple configuration files on top of each other by passing multiple `-c` arguments:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You can overlay multiple configuration files on top of each other by passing multiple `-c` arguments:
You can overlay multiple configuration files by passing multiple `-c` arguments:


```console
semgrep-network-broker -c config1.yaml -c config2.yaml -c config3.yaml
```

Run the following command to start Semgrep Network Broker with your completed configuration file:
Note that arrays are replaced, while maps are merged.

<pre class="language-console"><code>sudo docker run -d -it --rm -v $(pwd):/emt ghcr.io/semgrep/semgrep-network-broker:<span className="placeholder">VERSION_NUMBER</span> -c /emt/config.yaml</code></pre>

## Check Semgrep Network Broker logs

Expand Down