diff --git a/spec/openapi.yaml b/spec/openapi.yaml index d598744f..3cc46627 100644 --- a/spec/openapi.yaml +++ b/spec/openapi.yaml @@ -3,10 +3,12 @@ info: contact: name: API Specification (based on STAC) url: http://data.geo.admin.ch/api/stac/v1/ - description: >- + description: | This is an OpenAPI definition of the API to query and access federal geodata on data.geo.admin.ch. The API is based on the core SpatioTemporal Asset Catalog API specification [STAC](http://stacspec.org) and adds two extensions for extended searching possibilities. + + Find tutorials, examples and best practices around the STAC API in the [Tech Docs](https://docs.geo.admin.ch/download-data/stac-api/overview.html). title: The SpatioTemporal Asset Catalog API for data.geo.admin.ch version: 1.0.0 servers: diff --git a/spec/static/assets/service-stac-upload-process.dia b/spec/static/assets/service-stac-upload-process.dia deleted file mode 100644 index c6436821..00000000 Binary files a/spec/static/assets/service-stac-upload-process.dia and /dev/null differ diff --git a/spec/static/assets/service-stac-upload-process.svg b/spec/static/assets/service-stac-upload-process.svg deleted file mode 100644 index 73f5fad6..00000000 --- a/spec/static/assets/service-stac-upload-process.svg +++ /dev/null @@ -1,755 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/spec/static/spec/v1/openapi.yaml b/spec/static/spec/v1/openapi.yaml index 4c39ca18..1d97a4c0 100644 --- a/spec/static/spec/v1/openapi.yaml +++ b/spec/static/spec/v1/openapi.yaml @@ -1958,8 +1958,12 @@ info: contact: name: API Specification (based on STAC) url: http://data.geo.admin.ch/api/stac/v1/ - description: >- - This is an OpenAPI definition of the API to query and access federal geodata on data.geo.admin.ch. The API is based on the core SpatioTemporal Asset Catalog API specification [STAC](http://stacspec.org) and adds two extensions for extended searching possibilities. + description: | + This is an OpenAPI definition of the API to query and access federal geodata on + data.geo.admin.ch. The API is based on the core SpatioTemporal Asset Catalog API specification + [STAC](http://stacspec.org) and adds two extensions for extended searching possibilities. + + Find tutorials, examples and best practices around the STAC API in the [Tech Docs](https://docs.geo.admin.ch/download-data/stac-api/overview.html). title: The SpatioTemporal Asset Catalog API for data.geo.admin.ch version: 1.0.0 servers: diff --git a/spec/static/spec/v1/openapitransactional.yaml b/spec/static/spec/v1/openapitransactional.yaml index a90eb2f5..11ef0f96 100644 --- a/spec/static/spec/v1/openapitransactional.yaml +++ b/spec/static/spec/v1/openapitransactional.yaml @@ -2796,8 +2796,12 @@ info: contact: name: API Specification (based on STAC) url: http://data.geo.admin.ch/api/stac/v1/ - description: >- - This is an OpenAPI definition of the API to query and access federal geodata on data.geo.admin.ch. The API is based on the core SpatioTemporal Asset Catalog API specification [STAC](http://stacspec.org) and adds two extensions for extended searching possibilities. + description: | + This is an OpenAPI definition of the API to query and access federal geodata on + data.geo.admin.ch. The API is based on the core SpatioTemporal Asset Catalog API specification + [STAC](http://stacspec.org) and adds two extensions for extended searching possibilities. + + Find tutorials, examples and best practices around the STAC API in the [Tech Docs](https://docs.geo.admin.ch/download-data/stac-api/overview.html). title: The SpatioTemporal Asset Catalog API for data.geo.admin.ch version: 1.0.0 servers: @@ -2812,372 +2816,23 @@ tags: name: STAC - name: Data Management description: | - Metadata management requests. Theses requests are used to create, update or delete the STAC - metadata. - - *NOTE: these requests require authentication as described in [here](#tag/Authentication).* - - ## Supported Media Type - - List of the current supported media types: - - | Media Type | Description | File Extension(s) | - | ---------- | ----------- | ----------------- | - | application/vnd.apache.parquet | Apache Parquet column-oriented data file format | .parquet | - | application/x.ascii-grid+zip | Zipped ESRI ASCII raster format (.asc) | .zip | - | application/x.ascii-xyz+zip | Zipped XYZ (.xyz) | .zip | - | application/x.e00+zip | Zipped e00 | .zip | - | application/x.geotiff+zip | Zipped GeoTIFF | .zip | - | image/tiff; application=geotiff | GeoTIFF | .tiff, .tif | - | application/x.tiff+zip | Zipped TIFF | .zip | - | application/x.png+zip | Zipped PNG | .zip | - | application/x.jpeg+zip | Zipped JPEG | .zip | - | application/vnd.google-earth.kml+xml | KML | .kml | - | application/vnd.google-earth.kmz | Zipped KML | .kmz | - | application/x.dxf+zip | Zipped DXF | .zip | - | application/gml+xml | GML | .gml, .xml | - | application/x.gml+zip | Zipped GML | .zip | - | application/vnd.las | LIDAR | .las | - | application/vnd.laszip | Zipped LIDAR | .laz, .zip | - | application/x.vnd.las+zip | Zipped LAS | .zip | - | application/vnd.laszip+copc | Cloud Optimized Point Cloud (COPC) | .zip, .laz | - | application/x.shapefile+zip | Zipped Shapefile | .zip | - | application/x.filegdb+zip | Zipped File Geodatabase | .zip | - | application/x.filegdbp+zip | Zipped File Geodatabase (ArcGIS Pro) | .zip | - | application/x.ms-access+zip | Zipped Personal Geodatabase | .zip | - | application/x.ms-excel+zip | Zipped Excel | .zip | - | application/x.tab+zip | Zipped Mapinfo-TAB | .zip | - | application/x.tab-raster+zip | Zipped Mapinfo-Raster-TAB | .zip | - | application/x.csv+zip | Zipped CSV | .zip | - | text/csv | CSV | .csv | - | application/geopackage+sqlite3 | Geopackage | .gpkg | - | application/x.geopackage+zip | Zipped Geopackage | .zip | - | application/geo+json | GeoJSON (does not support auto HTTP compression !) | .json, .geojson | - | application/x.geojson+zip | Zipped GeoJSON | .zip | - | application/x.interlis; version=2.3 | Interlis 2 | .xtf, .xml | - | application/x.interlis+zip; version=2.3 | Zipped XTF (2.3) | .zip | - | application/x.interlis; version=2.4 | Interlis 2 | .xtf | .xml | - | application/x.interlis+zip; version=2.4 | Zipped XTF (2.4) | .zip | - | application/x.interlis; version=1 | Interlis 1 | .itf | - | application/x.interlis+zip; version=1 | Zipped ITF | .zip | - | image/tiff; application=geotiff; profile=cloud-optimized | Cloud Optimized GeoTIFF (COG) | .tiff, .tif |` - | application/pdf | PDF | .pdf | - | application/x.pdf+zip | Zipped PDF | .zip | - | application/json | JSON | .json | - | application/x.json+zip | Zipped JSON | .zip | - | application/x-netcdf | NetCDF | .nc | - | application/x.netcdf+zip | Zipped NetCDF | .zip | - | application/xml | XML | .xml | - | application/x.xml+zip | Zipped XML | .zip | - | application/vnd.mapbox-vector-tile | mbtiles | .mbtiles | - | text/plain | Text | .txt | - | text/x.plain+zip | Zipped text | .zip | - | application/x.dwg+zip | Zipped DWG | .zip | - | application/zip | Generic Zip File | .zip | - | image/tiff | TIFF | .tiff, .tif | - | image/jpeg | JPEG | .jpeg, .jpg | - | image/png | PNG | .png | - | application/vnd.sqlite3 | sqlite | .sqlite | - | application/grib | GRIB/GRIB2 | .grib, .grib2 | + The endpoints in this section create, update or delete STAC metadata. + + See the [Tech Docs](https://docs.geo.admin.ch/download-data/stac-api/overview.html) for more details + on authentication and supported media types. - name: Asset Upload Management description: | - Asset files are uploaded via the STAC API using the API requests described in this chapter. - - - [Authentication](#section/Authentication) - - [Compression](#section/Compression) - - [Example](#section/Example) - - ***IMPORTANT NOTES:*** - - *Files bigger than 10 MB should use compression, - see [Compression](#section/Compression)* - - ## Basic Steps to upload assets - - To upload an assets file there are generally three steps involved: - -
    -
  1. Create new upload process
    2. -
  2. Upload parts/files
    3. -
  3. Complete upload process
    4. -
- - Any file that is larger than 5 GB must be split into multiple parts. A file part must be at - least 5 MB except for the last one and at most 5 GB, otherwise the complete operation will fail. - If the file is less that 5 GB, you will only upload a single part, but must still start and complete the process as with multiple parts. - - (1) Use the [create new upload](#tag/Asset-Upload-Management/operation/createAssetUpload) request to start a new upload. - It will return a list of urls. - - (2) Use the urls to [upload asset file part](#tag/Asset-Upload-Management/operation/uploadAssetFilePart). Do this for each file part. You may also upload multiple parts in parallel. - - (3) Once all parts have been uploaded, execute the [complete the upload](#tag/Asset-Upload-Management/operation/completeMultipartUpload) request. - The new asset file be available to users after you have successfully completed the upload. - - If any errors happen during the upload process, you can [abort the upload](#tag/Asset-Upload-Management/operation/abortMultipartUpload) - and restart the process. - - Multiple parts (2) can be upload concurrently, but you cannot create a new upload (1) while another is still in progress. - If you wish to start a new upload you must first complete or abort any existing upload that is in progress. - Use [list uploads](#tag/Asset-Upload-Management/operation/getAssetUploads) to see if any uploads - are in progress. - - ## Detailed example of asset upload - - This describes the process in more detail with focus on automated recurring uploads and error handling. - - ### Glossary - - - Abort Upload: abort an upload in progress by sending an abort request to the service - - Cancel Upload: cancel the upload iteration without sending an abort request to the service - - ### Recurrent upload - - If you have recurrent asset uploads, you need to have a proper error handling otherwise the uploads might get stuck. Asset uploads operation are not stateless but statefull therefore the error handling is important. Here below is a simple practical example on which errors to handle in case of recurrent asset upload. - - Note this example is only recommended if the upload is recurrent (for example every hour). The number of retries below depends on the upload frequency, if the upload frequency is daily then you might want to have at least 3 retries with some exponential backoff time between each retries, in opposite if the upload is done at high frequency you might skip the retries and simply cancel the upload, using the next upload iteration as retry. - - 1. Create Asset Upload - - ```text - POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads - ``` - - - `201 OK` => Continue to step 2. - - `400 Bad Request` - - Response is `{"description": "Upload already in progress", "code": 400}` => Abort the upload - - To do so first get the `upload_id` of the `in-progress` upload via - - ```text - GET https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads?status=in-progress - ``` - - - Then using this id abort the upload - - ```text - POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/abort - ``` - - - Then restart the step 1. - - Another `400 Bad Request` => Cancel upload - - Your request is not correct, analyze your request and correct it before retrying the step 1. - - - `500 Internal Server Error` => Cancel upload - - This is generally an application crash and should be notify to the service administrator, a retry would usually be useless, simply cancel the upload. - - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry - - Service is momentarily not available, wait a short amount of time and retry step 1. the amount of time to wait and the number of retries depends on the upload rate, but a minimum wait time of 100ms is recommended. - 2. Upload the parts via the presigned URL - - ```text - PUT {presigned_url} - ``` - - - `200 OK` => Continue to step 3. - - `400 Bad Request` => Abort upload + The endpoints in this section manage the process of uploading assets to the STAC catalog. - Abort upload using the current `upload_id` and contact service administrator. - - ```text - POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/abort - ``` - - - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry - - Retry step 2. with a short wait time (min 100ms). - - 3. Complete the upload - - ```text - POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/complete - ``` - - - `200 OK` => Upload successful - - `400 Bad Request` => Cancel upload - - Your request is invalid/incorrect, you need to cancel the upload script and verify its correctness. - - `500 Internal Server Error` => Cancel upload - - This is generally an application crash and should be notify to the service administrator, a retry would usually be useless, simply cancel the upload. - - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry - - Service is momentarily not available, wait a short moment (100ms), then retry the request. - - The following figure shows the flow of a multipart upload process. - ![diagram](/api/stac/static/assets/service-stac-upload-process.svg) - - - ## Authentication - - POST/PUT requests require authentication as described in [here](#tag/Authentication). - - ## Compression - - Files between *1 MB* and *10 MB* are automatically compressed during download using *gzip* or - *br* based on the `Accept-Encoding` header of the request. But note that this compression is - only done for standard Media Type (see [File types that CloudFront compresses](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ServingCompressedFiles.html#compressed-content-cloudfront-file-types)). - - It is highly recommended to upload files bigger than *10 MB* using - a compressed media type (see [Supported Media Type](#section/Supported-Media-Type)) ! - If this not possible (as e.g. for json directly usd in a browser application), the file should - be either compressed upfront (see below) or split in smaller files. - - ### Upfront Compression using `content_encoding` - - In the case where you have a file bigger than 10 MB that you can't split into multiple files - or pack into a compressed media type, then you can use the upfront compression method together - with the `content_encoding` parameter. - - For this you need to first compress the file using gzip or br compression algorithm and then - use the `content_encoding` parameter in the [Create Asset's multipart upload](#tag/Asset-Upload-Management/operation/createAssetUpload) - - ***NOTES:*** - - In this case the file will be always delivered compressed, which means that the client that - download the file needs to be compatible with the HTTP Compression algorithm defined in - `Content-Encoding` header. - - - ## Example - - ```python - import os - import hashlib - from base64 import b64encode - - import requests - import multihash - - # variables - scheme = 'https' - hostname = 'data.geo.admin.ch' - collection = 'ch.swisstopo.pixelkarte-farbe-pk200.noscale' - item = 'smr200-200-4-2016' - asset = 'smr200-200-4-2016-2056-kgrs-10.tiff' - asset_path = f'collections/{collection}/items/{item}/assets/{asset}' - user = os.environ.get('STAC_USER', 'unknown-user') - password = os.environ.get('STAC_PASSWORD', 'unknown-password') - - with open('smr200-200-4-2016-2056-kgrs-10.tiff', 'rb') as fd: - data = fd.read() - - checksum_multihash = multihash.to_hex_string(multihash.encode(hashlib.sha256(data).digest(), 'sha2-256')) - md5 = b64encode(hashlib.md5(data).digest()).decode('utf-8') - - # 1. Create a multipart upload - response = requests.post( - f"{scheme}://{hostname}/api/stac/v1/{asset_path}/uploads", - auth=(user, password), - json={ - "number_parts": 1, - "md5_parts": [{ - "part_number": 1, - "md5": md5 - }], - "file:checksum": checksum_multihash - } - ) - upload_id = response.json()['upload_id'] - - # 2. Upload the part using the presigned url - response = requests.put(response.json()['urls'][0]['url'], data=data, headers={'Content-MD5': md5}) - etag = response.headers['ETag'] - - # 3. Complete the upload - response = requests.post( - f"{scheme}://{hostname}/api/stac/v1/{asset_path}/uploads/{upload_id}/complete", - auth=(user, password), - json={'parts': [{'etag': etag, 'part_number': 1}]} - ) - ``` - - See https://aws.amazon.com/premiumsupport/knowledge-center/data-integrity-s3/ for other examples on how to compute the base64 MD5 of a part. + See the [Tech Docs](https://docs.geo.admin.ch/download-data/stac-api/overview.html) for examples and + more details on authentication and compression. - name: Collection Asset Upload Management description: | - Collection Asset files are uploaded via the STAC API using the API requests described in this chapter. - - The flow of the requests is the same as for assets that belong to features, which is described under [Asset Upload Management](#tag/Asset-Upload-Management) - - [Basic steps to upload assets](#section/Basic-Steps-to-upload-assets) - - [Detailed example of asset upload](#section/Detailed-example-of-asset-upload) - - [Authentication](#section/Authentication) - - [Compression](#section/Compression) - - [Example](#section/Example) - - name: Authentication - description: | - All write requests require authentication. There are currently two types of supported authentications: - - * [Session authentication](#section/Session-authentication) - * [JSON Web Token authentication](#section/JSON-Web-Token-authentication) - - ## Session authentication - - When using the browsable API the user can simply use the admin interface for logging in. - The service issues a session cookie which the browser can use to authenticate itself - and perform write requests. This authentication method is only intended - for web browsers users of the admin interface. Non-browser clients and - API endpoints are not guaranteed to work with session authentication. - - ## JSON Web Token authentication - - The user authenticates with a JSON Web Token (JWT) passed in the - `Authorization` header with the `Bearer` HTTP authentication scheme as - described in - [RFC 6750](https://datatracker.ietf.org/doc/html/rfc6750#section-2.1). - - Here is an example using curl, assuming the JWT is `123456`: - - ``` - curl --request POST \ - --url https://data.geoadmin.ch/api/stac/v1/collections/ch.swisstopo.swisstlmregio/items/swisstlmregio-2020/assets \ - --header 'Authorization: Bearer 123456' \ - --header 'Content-Type: application/json' \ - --data '{ - "id": "fancy_unique_id", - "item": "swisstlmregio-2020", - "title": "My title", - "type": "application/x.filegdb+zip", - "description": "My description", - "proj:epsg": 2056, - "file:checksum": "12200ADEC47F803A8CF1055ED36750B3BA573C79A3AF7DA6D6F5A2AED03EA16AF3BC" - }' - ``` - - Tokens are obtained by requesting them from the - [Amazon Cognito InitiateAuth API](https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_InitiateAuth.html). - - Here is an example using curl and jq: - - ``` - curl --request POST \ - --url https://cognito-idp.eu-central-1.amazonaws.com/ \ - --header 'Content-Type: application/x-amz-json-1.1' \ - --header 'X-Amz-Target: AWSCognitoIdentityProviderService.InitiateAuth' \ - --data '{ - "AuthFlow": "USER_PASSWORD_AUTH", - "AuthParameters": { - "PASSWORD": "I_love_Minnie_Mouse", - "USERNAME": "MickeyMouse" - }, - "ClientId": "CLIENT_ID" - }' | jq -r .AuthenticationResult.AccessToken - ``` - - The `CLIENT_ID` value needs to be substituted for the correct client - identifier which you should receive along with your username and password. - - Notice the response from `InitiateAuth` is a JSON document. The token used - to authenticate against the STAC API is the `AccessToken`. There are cases - where the response will not contain that token (e.g. if the password must - be updated or a second factor is required for authentication). It is the - responsibility of the client to handle these cases. - [AWS provides an SDK](https://aws.amazon.com/developer/tools/) which may - make this easier. - - The access token is only valid for a certain duration (as per - the `AuthenticationResult.ExpiresIn` field in the response). You need to - refresh it periodically, either by obtaining a new JWT or by - [using the refresh token](https://docs.aws.amazon.com/cognito/latest/developerguide/amazon-cognito-user-pools-using-the-refresh-token.html). - The refresh token is normally valid for a longer time period. + The endpoints in this section manage the process of uploading collection assets to the STAC catalog. + The flow of the requests is the same as for assets that belong to features. + + See the [Tech Docs](https://docs.geo.admin.ch/download-data/stac-api/overview.html) for examples and + more details on authentication and compression. paths: /: get: diff --git a/spec/transaction/tags.yaml b/spec/transaction/tags.yaml index 229b2ca4..7db2b22b 100644 --- a/spec/transaction/tags.yaml +++ b/spec/transaction/tags.yaml @@ -5,371 +5,22 @@ tags: - name: STAC - name: Data Management description: | - Metadata management requests. Theses requests are used to create, update or delete the STAC - metadata. + The endpoints in this section create, update or delete STAC metadata. - *NOTE: these requests require authentication as described in [here](#tag/Authentication).* - - ## Supported Media Type - - List of the current supported media types: - - | Media Type | Description | File Extension(s) | - | ---------- | ----------- | ----------------- | - | application/vnd.apache.parquet | Apache Parquet column-oriented data file format | .parquet | - | application/x.ascii-grid+zip | Zipped ESRI ASCII raster format (.asc) | .zip | - | application/x.ascii-xyz+zip | Zipped XYZ (.xyz) | .zip | - | application/x.e00+zip | Zipped e00 | .zip | - | application/x.geotiff+zip | Zipped GeoTIFF | .zip | - | image/tiff; application=geotiff | GeoTIFF | .tiff, .tif | - | application/x.tiff+zip | Zipped TIFF | .zip | - | application/x.png+zip | Zipped PNG | .zip | - | application/x.jpeg+zip | Zipped JPEG | .zip | - | application/vnd.google-earth.kml+xml | KML | .kml | - | application/vnd.google-earth.kmz | Zipped KML | .kmz | - | application/x.dxf+zip | Zipped DXF | .zip | - | application/gml+xml | GML | .gml, .xml | - | application/x.gml+zip | Zipped GML | .zip | - | application/vnd.las | LIDAR | .las | - | application/vnd.laszip | Zipped LIDAR | .laz, .zip | - | application/x.vnd.las+zip | Zipped LAS | .zip | - | application/vnd.laszip+copc | Cloud Optimized Point Cloud (COPC) | .zip, .laz | - | application/x.shapefile+zip | Zipped Shapefile | .zip | - | application/x.filegdb+zip | Zipped File Geodatabase | .zip | - | application/x.filegdbp+zip | Zipped File Geodatabase (ArcGIS Pro) | .zip | - | application/x.ms-access+zip | Zipped Personal Geodatabase | .zip | - | application/x.ms-excel+zip | Zipped Excel | .zip | - | application/x.tab+zip | Zipped Mapinfo-TAB | .zip | - | application/x.tab-raster+zip | Zipped Mapinfo-Raster-TAB | .zip | - | application/x.csv+zip | Zipped CSV | .zip | - | text/csv | CSV | .csv | - | application/geopackage+sqlite3 | Geopackage | .gpkg | - | application/x.geopackage+zip | Zipped Geopackage | .zip | - | application/geo+json | GeoJSON (does not support auto HTTP compression !) | .json, .geojson | - | application/x.geojson+zip | Zipped GeoJSON | .zip | - | application/x.interlis; version=2.3 | Interlis 2 | .xtf, .xml | - | application/x.interlis+zip; version=2.3 | Zipped XTF (2.3) | .zip | - | application/x.interlis; version=2.4 | Interlis 2 | .xtf | .xml | - | application/x.interlis+zip; version=2.4 | Zipped XTF (2.4) | .zip | - | application/x.interlis; version=1 | Interlis 1 | .itf | - | application/x.interlis+zip; version=1 | Zipped ITF | .zip | - | image/tiff; application=geotiff; profile=cloud-optimized | Cloud Optimized GeoTIFF (COG) | .tiff, .tif |` - | application/pdf | PDF | .pdf | - | application/x.pdf+zip | Zipped PDF | .zip | - | application/json | JSON | .json | - | application/x.json+zip | Zipped JSON | .zip | - | application/x-netcdf | NetCDF | .nc | - | application/x.netcdf+zip | Zipped NetCDF | .zip | - | application/xml | XML | .xml | - | application/x.xml+zip | Zipped XML | .zip | - | application/vnd.mapbox-vector-tile | mbtiles | .mbtiles | - | text/plain | Text | .txt | - | text/x.plain+zip | Zipped text | .zip | - | application/x.dwg+zip | Zipped DWG | .zip | - | application/zip | Generic Zip File | .zip | - | image/tiff | TIFF | .tiff, .tif | - | image/jpeg | JPEG | .jpeg, .jpg | - | image/png | PNG | .png | - | application/vnd.sqlite3 | sqlite | .sqlite | - | application/grib | GRIB/GRIB2 | .grib, .grib2 | + See the [Tech Docs](https://docs.geo.admin.ch/download-data/stac-api/overview.html) for more details + on authentication and supported media types. - name: Asset Upload Management description: | - Asset files are uploaded via the STAC API using the API requests described in this chapter. - - - [Authentication](#section/Authentication) - - [Compression](#section/Compression) - - [Example](#section/Example) - - ***IMPORTANT NOTES:*** - - *Files bigger than 10 MB should use compression, - see [Compression](#section/Compression)* - - ## Basic Steps to upload assets - - To upload an assets file there are generally three steps involved: - -
    -
  1. Create new upload process
    2. -
  2. Upload parts/files
    3. -
  3. Complete upload process
    4. -
- - Any file that is larger than 5 GB must be split into multiple parts. A file part must be at - least 5 MB except for the last one and at most 5 GB, otherwise the complete operation will fail. - If the file is less that 5 GB, you will only upload a single part, but must still start and complete the process as with multiple parts. - - (1) Use the [create new upload](#tag/Asset-Upload-Management/operation/createAssetUpload) request to start a new upload. - It will return a list of urls. - - (2) Use the urls to [upload asset file part](#tag/Asset-Upload-Management/operation/uploadAssetFilePart). Do this for each file part. You may also upload multiple parts in parallel. - - (3) Once all parts have been uploaded, execute the [complete the upload](#tag/Asset-Upload-Management/operation/completeMultipartUpload) request. - The new asset file be available to users after you have successfully completed the upload. - - If any errors happen during the upload process, you can [abort the upload](#tag/Asset-Upload-Management/operation/abortMultipartUpload) - and restart the process. - - Multiple parts (2) can be upload concurrently, but you cannot create a new upload (1) while another is still in progress. - If you wish to start a new upload you must first complete or abort any existing upload that is in progress. - Use [list uploads](#tag/Asset-Upload-Management/operation/getAssetUploads) to see if any uploads - are in progress. - - ## Detailed example of asset upload - - This describes the process in more detail with focus on automated recurring uploads and error handling. - - ### Glossary - - - Abort Upload: abort an upload in progress by sending an abort request to the service - - Cancel Upload: cancel the upload iteration without sending an abort request to the service - - ### Recurrent upload - - If you have recurrent asset uploads, you need to have a proper error handling otherwise the uploads might get stuck. Asset uploads operation are not stateless but statefull therefore the error handling is important. Here below is a simple practical example on which errors to handle in case of recurrent asset upload. - - Note this example is only recommended if the upload is recurrent (for example every hour). The number of retries below depends on the upload frequency, if the upload frequency is daily then you might want to have at least 3 retries with some exponential backoff time between each retries, in opposite if the upload is done at high frequency you might skip the retries and simply cancel the upload, using the next upload iteration as retry. - - 1. Create Asset Upload - - ```text - POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads - ``` - - - `201 OK` => Continue to step 2. - - `400 Bad Request` - - Response is `{"description": "Upload already in progress", "code": 400}` => Abort the upload - - To do so first get the `upload_id` of the `in-progress` upload via - - ```text - GET https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads?status=in-progress - ``` - - - Then using this id abort the upload - - ```text - POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/abort - ``` - - - Then restart the step 1. - - Another `400 Bad Request` => Cancel upload - - Your request is not correct, analyze your request and correct it before retrying the step 1. - - - `500 Internal Server Error` => Cancel upload - - This is generally an application crash and should be notify to the service administrator, a retry would usually be useless, simply cancel the upload. - - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry - - Service is momentarily not available, wait a short amount of time and retry step 1. the amount of time to wait and the number of retries depends on the upload rate, but a minimum wait time of 100ms is recommended. - 2. Upload the parts via the presigned URL - - ```text - PUT {presigned_url} - ``` - - - `200 OK` => Continue to step 3. - - `400 Bad Request` => Abort upload - - Abort upload using the current `upload_id` and contact service administrator. - - ```text - POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/abort - ``` - - - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry - - Retry step 2. with a short wait time (min 100ms). - - 3. Complete the upload - - ```text - POST https://data.geo.admin.ch/api/stac/v1/collections/{collection}/items/{item}/assets/{asset}/uploads/{upload_id}/complete - ``` + The endpoints in this section manage the process of uploading assets to the STAC catalog. - - `200 OK` => Upload successful - - `400 Bad Request` => Cancel upload + See the [Tech Docs](https://docs.geo.admin.ch/download-data/stac-api/overview.html) for examples and + more details on authentication and compression. - Your request is invalid/incorrect, you need to cancel the upload script and verify its correctness. - - `500 Internal Server Error` => Cancel upload - - This is generally an application crash and should be notify to the service administrator, a retry would usually be useless, simply cancel the upload. - - `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout` => Retry - - Service is momentarily not available, wait a short moment (100ms), then retry the request. - - The following figure shows the flow of a multipart upload process. - ![diagram](/api/stac/static/assets/service-stac-upload-process.svg) - - - ## Authentication - - POST/PUT requests require authentication as described in [here](#tag/Authentication). - - ## Compression - - Files between *1 MB* and *10 MB* are automatically compressed during download using *gzip* or - *br* based on the `Accept-Encoding` header of the request. But note that this compression is - only done for standard Media Type (see [File types that CloudFront compresses](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ServingCompressedFiles.html#compressed-content-cloudfront-file-types)). - - It is highly recommended to upload files bigger than *10 MB* using - a compressed media type (see [Supported Media Type](#section/Supported-Media-Type)) ! - If this not possible (as e.g. for json directly usd in a browser application), the file should - be either compressed upfront (see below) or split in smaller files. - - ### Upfront Compression using `content_encoding` - - In the case where you have a file bigger than 10 MB that you can't split into multiple files - or pack into a compressed media type, then you can use the upfront compression method together - with the `content_encoding` parameter. - - For this you need to first compress the file using gzip or br compression algorithm and then - use the `content_encoding` parameter in the [Create Asset's multipart upload](#tag/Asset-Upload-Management/operation/createAssetUpload) - - ***NOTES:*** - - In this case the file will be always delivered compressed, which means that the client that - download the file needs to be compatible with the HTTP Compression algorithm defined in - `Content-Encoding` header. - - - ## Example - - ```python - import os - import hashlib - from base64 import b64encode - - import requests - import multihash - - # variables - scheme = 'https' - hostname = 'data.geo.admin.ch' - collection = 'ch.swisstopo.pixelkarte-farbe-pk200.noscale' - item = 'smr200-200-4-2016' - asset = 'smr200-200-4-2016-2056-kgrs-10.tiff' - asset_path = f'collections/{collection}/items/{item}/assets/{asset}' - user = os.environ.get('STAC_USER', 'unknown-user') - password = os.environ.get('STAC_PASSWORD', 'unknown-password') - - with open('smr200-200-4-2016-2056-kgrs-10.tiff', 'rb') as fd: - data = fd.read() - - checksum_multihash = multihash.to_hex_string(multihash.encode(hashlib.sha256(data).digest(), 'sha2-256')) - md5 = b64encode(hashlib.md5(data).digest()).decode('utf-8') - - # 1. Create a multipart upload - response = requests.post( - f"{scheme}://{hostname}/api/stac/v1/{asset_path}/uploads", - auth=(user, password), - json={ - "number_parts": 1, - "md5_parts": [{ - "part_number": 1, - "md5": md5 - }], - "file:checksum": checksum_multihash - } - ) - upload_id = response.json()['upload_id'] - - # 2. Upload the part using the presigned url - response = requests.put(response.json()['urls'][0]['url'], data=data, headers={'Content-MD5': md5}) - etag = response.headers['ETag'] - - # 3. Complete the upload - response = requests.post( - f"{scheme}://{hostname}/api/stac/v1/{asset_path}/uploads/{upload_id}/complete", - auth=(user, password), - json={'parts': [{'etag': etag, 'part_number': 1}]} - ) - ``` - - See https://aws.amazon.com/premiumsupport/knowledge-center/data-integrity-s3/ for other examples on how to compute the base64 MD5 of a part. - name: Collection Asset Upload Management description: | - Collection Asset files are uploaded via the STAC API using the API requests described in this chapter. - - The flow of the requests is the same as for assets that belong to features, which is described under [Asset Upload Management](#tag/Asset-Upload-Management) - - [Basic steps to upload assets](#section/Basic-Steps-to-upload-assets) - - [Detailed example of asset upload](#section/Detailed-example-of-asset-upload) - - [Authentication](#section/Authentication) - - [Compression](#section/Compression) - - [Example](#section/Example) - - - name: Authentication - description: | - All write requests require authentication. There are currently two types of supported authentications: - - * [Session authentication](#section/Session-authentication) - * [JSON Web Token authentication](#section/JSON-Web-Token-authentication) - - ## Session authentication - - When using the browsable API the user can simply use the admin interface for logging in. - The service issues a session cookie which the browser can use to authenticate itself - and perform write requests. This authentication method is only intended - for web browsers users of the admin interface. Non-browser clients and - API endpoints are not guaranteed to work with session authentication. - - ## JSON Web Token authentication - - The user authenticates with a JSON Web Token (JWT) passed in the - `Authorization` header with the `Bearer` HTTP authentication scheme as - described in - [RFC 6750](https://datatracker.ietf.org/doc/html/rfc6750#section-2.1). - - Here is an example using curl, assuming the JWT is `123456`: - - ``` - curl --request POST \ - --url https://data.geoadmin.ch/api/stac/v1/collections/ch.swisstopo.swisstlmregio/items/swisstlmregio-2020/assets \ - --header 'Authorization: Bearer 123456' \ - --header 'Content-Type: application/json' \ - --data '{ - "id": "fancy_unique_id", - "item": "swisstlmregio-2020", - "title": "My title", - "type": "application/x.filegdb+zip", - "description": "My description", - "proj:epsg": 2056, - "file:checksum": "12200ADEC47F803A8CF1055ED36750B3BA573C79A3AF7DA6D6F5A2AED03EA16AF3BC" - }' - ``` - - Tokens are obtained by requesting them from the - [Amazon Cognito InitiateAuth API](https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_InitiateAuth.html). - - Here is an example using curl and jq: - - ``` - curl --request POST \ - --url https://cognito-idp.eu-central-1.amazonaws.com/ \ - --header 'Content-Type: application/x-amz-json-1.1' \ - --header 'X-Amz-Target: AWSCognitoIdentityProviderService.InitiateAuth' \ - --data '{ - "AuthFlow": "USER_PASSWORD_AUTH", - "AuthParameters": { - "PASSWORD": "I_love_Minnie_Mouse", - "USERNAME": "MickeyMouse" - }, - "ClientId": "CLIENT_ID" - }' | jq -r .AuthenticationResult.AccessToken - ``` - - The `CLIENT_ID` value needs to be substituted for the correct client - identifier which you should receive along with your username and password. - - Notice the response from `InitiateAuth` is a JSON document. The token used - to authenticate against the STAC API is the `AccessToken`. There are cases - where the response will not contain that token (e.g. if the password must - be updated or a second factor is required for authentication). It is the - responsibility of the client to handle these cases. - [AWS provides an SDK](https://aws.amazon.com/developer/tools/) which may - make this easier. + The endpoints in this section manage the process of uploading collection assets to the STAC catalog. + The flow of the requests is the same as for assets that belong to features. - The access token is only valid for a certain duration (as per - the `AuthenticationResult.ExpiresIn` field in the response). You need to - refresh it periodically, either by obtaining a new JWT or by - [using the refresh token](https://docs.aws.amazon.com/cognito/latest/developerguide/amazon-cognito-user-pools-using-the-refresh-token.html). - The refresh token is normally valid for a longer time period. + See the [Tech Docs](https://docs.geo.admin.ch/download-data/stac-api/overview.html) for examples and + more details on authentication and compression.