From 2e8cb32d351c3d24a677a071f1ded2ec39b52ca5 Mon Sep 17 00:00:00 2001 From: Vinicius Dalpiccol Date: Fri, 29 Aug 2025 21:24:39 +0200 Subject: [PATCH 1/3] adds azure storage examples and environment variable specifications --- deployment/overview.mdx | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/deployment/overview.mdx b/deployment/overview.mdx index 6ddc875..900ea26 100644 --- a/deployment/overview.mdx +++ b/deployment/overview.mdx @@ -51,3 +51,36 @@ these, for example: * `file:///my/local/filesystem` * `/my/local/filesystem` * `gs://my-gcs-bucket` +* `https://mystorageaccount.blob.core.windows.net/my-bucket/path` +* `https://mystorageaccount.dfs.core.windows.net/my-bucket/path` +* `abfs://my-container@mystorageaccount.dfs.core.windows.net/path` + +Authentication with the underlying storage provider is usually expected through environment variables corresponding +to the cloud provider: + +| Provider | Environment Variable | Description | +| ----- | ----------- | ------- | +| AWS | `AWS_ACCESS_KEY_ID` | access_key_id | +| AWS | `AWS_SECRET_ACCESS_KEY` | secret_access_key | +| AWS | `AWS_DEFAULT_REGION` | Region | +| AWS | `AWS_ENDPOINT` | Endpoint | +| AWS | `AWS_SESSION_TOKEN` | Token | +| AWS | `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` | | +| AWS | `AWS_CONTAINER_CREDENTIALS_FULL_URI` | | +| AWS | `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE` | | +| AWS | `AWS_ALLOW_HTTP` | Set to "true" to permit HTTP connections without TLS | +| AWS | `AWS_REQUEST_PAYER` | Set to "true" to permit operations on requester-pays buckets | +| Azure | AZURE_STORAGE_ACCOUNT_NAME | Storage account name | +| Azure | AZURE_STORAGE_ACCOUNT_KEY | Storage account master key | +| Azure | AZURE_STORAGE_ACCESS_KEY | Alias for AZURE_STORAGE_ACCOUNT_KEY | +| Azure | AZURE_STORAGE_CLIENT_ID | Client id for service principal authorization | +| Azure | AZURE_STORAGE_CLIENT_SECRET | Client secret for service principal authorization | +| Azure | AZURE_STORAGE_TENANT_ID | Tenant id used in oauth flows | +| GCS | `GOOGLE_SERVICE_ACCOUNT` | Location of service account file | +| GCS | `GOOGLE_SERVICE_ACCOUNT_PATH` | (Alias) Location of service account file | +| GCS | `SERVICE_ACCOUNT` | (Alias) Location of service account file | +| GCS | `GOOGLE_SERVICE_ACCOUNT_KEY` | JSON serialized service account key | +| GCS | `GOOGLE_BUCKET` | Bucket name | +| GCS | `GOOGLE_BUCKET_NAME` | (Alias) bucket name | +| R2 | `R2_ACCESS_KEY_ID` | access_key_id | +| R2 | `R2_SECRET_ACCESS_KEY` | secret_access_key | \ No newline at end of file From 2ef016673f18d1595f210e9728b9eb71c1100345 Mon Sep 17 00:00:00 2001 From: Vinicius Dalpiccol Date: Fri, 29 Aug 2025 21:27:42 +0200 Subject: [PATCH 2/3] remove signs --- deployment/overview.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/deployment/overview.mdx b/deployment/overview.mdx index 900ea26..4695677 100644 --- a/deployment/overview.mdx +++ b/deployment/overview.mdx @@ -65,9 +65,9 @@ to the cloud provider: | AWS | `AWS_DEFAULT_REGION` | Region | | AWS | `AWS_ENDPOINT` | Endpoint | | AWS | `AWS_SESSION_TOKEN` | Token | -| AWS | `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` | | -| AWS | `AWS_CONTAINER_CREDENTIALS_FULL_URI` | | -| AWS | `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE` | | +| AWS | `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` | https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html | +| AWS | `AWS_CONTAINER_CREDENTIALS_FULL_URI` | https://docs.aws.amazon.com/sdkref/latest/guide/feature-container-credentials.html | +| AWS | `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE` | https://docs.aws.amazon.com/sdkref/latest/guide/feature-container-credentials.html | | AWS | `AWS_ALLOW_HTTP` | Set to "true" to permit HTTP connections without TLS | | AWS | `AWS_REQUEST_PAYER` | Set to "true" to permit operations on requester-pays buckets | | Azure | AZURE_STORAGE_ACCOUNT_NAME | Storage account name | From f59d95443290307eaa1d6fd545d187cce9209878 Mon Sep 17 00:00:00 2001 From: Vinicius Dalpiccol Date: Fri, 29 Aug 2025 21:28:18 +0200 Subject: [PATCH 3/3] codeblocks around azure env vars --- deployment/overview.mdx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/deployment/overview.mdx b/deployment/overview.mdx index 4695677..810f12e 100644 --- a/deployment/overview.mdx +++ b/deployment/overview.mdx @@ -70,12 +70,12 @@ to the cloud provider: | AWS | `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE` | https://docs.aws.amazon.com/sdkref/latest/guide/feature-container-credentials.html | | AWS | `AWS_ALLOW_HTTP` | Set to "true" to permit HTTP connections without TLS | | AWS | `AWS_REQUEST_PAYER` | Set to "true" to permit operations on requester-pays buckets | -| Azure | AZURE_STORAGE_ACCOUNT_NAME | Storage account name | -| Azure | AZURE_STORAGE_ACCOUNT_KEY | Storage account master key | -| Azure | AZURE_STORAGE_ACCESS_KEY | Alias for AZURE_STORAGE_ACCOUNT_KEY | -| Azure | AZURE_STORAGE_CLIENT_ID | Client id for service principal authorization | -| Azure | AZURE_STORAGE_CLIENT_SECRET | Client secret for service principal authorization | -| Azure | AZURE_STORAGE_TENANT_ID | Tenant id used in oauth flows | +| Azure | `AZURE_STORAGE_ACCOUNT_NAME` | Storage account name | +| Azure | `AZURE_STORAGE_ACCOUNT_KEY` | Storage account master key | +| Azure | `AZURE_STORAGE_ACCESS_KEY` | Alias for AZURE_STORAGE_ACCOUNT_KEY | +| Azure | `AZURE_STORAGE_CLIENT_ID` | Client id for service principal authorization | +| Azure | `AZURE_STORAGE_CLIENT_SECRET` | Client secret for service principal authorization | +| Azure | `AZURE_STORAGE_TENANT_ID` | Tenant id used in oauth flows | | GCS | `GOOGLE_SERVICE_ACCOUNT` | Location of service account file | | GCS | `GOOGLE_SERVICE_ACCOUNT_PATH` | (Alias) Location of service account file | | GCS | `SERVICE_ACCOUNT` | (Alias) Location of service account file |