Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .agents/skills/lint/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Use this skill when:
## Execution Steps

1. **`make format`** — auto-formats all Kotlin files with ktlint. Fixes the vast majority of Kotlin style issues.
2. **`./mvnw checkstyle:check`** — reports Checkstyle violations for Java and XML files. Fix violations manually.
2. **`make lint`** — reports ktlint and Checkstyle violations. Fix Checkstyle (Java/XML) violations manually.
3. **`make sort`** — run this if any `pom.xml` was added or modified during this task.
4. Fix any remaining violations manually (see resources below for style guides).
5. **`make install`** — confirm all linting CI gates pass before finishing.
Expand All @@ -37,7 +37,7 @@ Use this skill when:
- Missing trailing newline
- Line too long (max 120 characters)

To check without modifying files: `./mvnw ktlint:check`
To check without modifying files: `make lint`

### Checkstyle (Java / XML)

Expand All @@ -50,7 +50,7 @@ Fix these manually:
## Completion Criteria

- [ ] `make format` run (Kotlin auto-fixed)
- [ ] `./mvnw checkstyle:check` passes with no violations
- [ ] `make lint` passes with no violations
- [ ] `make sort` run if any `pom.xml` was modified
- [ ] `make install` passes all CI gates

Expand Down
2 changes: 1 addition & 1 deletion .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ All third-party contributions to this project must be accompanied by a signed co

**Prerequisites:**
- Java 25 (compile/run; JVM target 17 with Kotlin language/api 2.2, per Spring Boot 4.x guidance)
- Maven 3.9+ (use the included `./mvnw` wrapper)
- Maven 3.9+ (use make targets)
- Docker (for Docker build and integration tests)

**Build and verify:**
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/maven-ci-and-prb.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2017-2020 Adobe.
# Copyright 2017-2026 Adobe.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -52,4 +52,4 @@ jobs:
distribution: 'oracle'
cache: 'maven'
- name: Build with Maven
run: ./mvnw -B -V -Dstyle.color=always clean verify
run: make verify
2 changes: 1 addition & 1 deletion .github/workflows/maven-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,4 +87,4 @@ jobs:
ADOBE_BOT_GITHUB_PASSWORD: ${{ secrets.ADOBE_BOT_GITHUB_PASSWORD }}
SONATYPE_USERNAME: ${{ secrets.S3MOCK_AFRANKEN_SONATYPE_USERNAME }}
SONATYPE_PASSWORD: ${{ secrets.S3MOCK_AFRANKEN_SONATYPE_PASSWORD }}
run: ./mvnw -B -V release:prepare release:perform
run: make release
62 changes: 7 additions & 55 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,59 +65,7 @@ See `dto/ListBucketResult.kt` for a representative example. XML names must match

## Storage

Filesystem layout:
```
<root>/<bucket>/bucketMetadata.json
<root>/<bucket>/<uuid>/binaryData
<root>/<bucket>/<uuid>/objectMetadata.json
<root>/<bucket>/<uuid>/<version-id>-binaryData # versioning
<root>/<bucket>/<uuid>/<version-id>-objectMetadata.json # versioning
<root>/<bucket>/multiparts/<upload-id>/multipartMetadata.json
<root>/<bucket>/multiparts/<upload-id>/<part>.part
```

**`bucketMetadata.json`** fields (`BucketMetadata`):

| Field | Type | Notes |
|---|---|---|
| `name` | `String` | Bucket name |
| `creationDate` | `String` | ISO-8601 timestamp |
| `bucketRegion` | `String` | AWS region string |
| `objects` | `Map<String, UUID>` | key → object UUID mapping |
| `versioningConfiguration` | `VersioningConfiguration?` | null until versioning is configured |
| `objectLockConfiguration` | `ObjectLockConfiguration?` | null until object lock is enabled |
| `bucketLifecycleConfiguration` | `BucketLifecycleConfiguration?` | null until lifecycle rules are set |
| `objectOwnership` | `ObjectOwnership?` | null until ownership is set |
| `bucketInfo` | `BucketInfo?` | bucket type/data-redundancy info |
| `locationInfo` | `LocationInfo?` | bucket location info |
| `path` | `Path` | filesystem path to the bucket folder (not serialized for cross-host use) |

**`objectMetadata.json`** fields (`S3ObjectMetadata`):

| Field | Type | Notes |
|---|---|---|
| `id` | `UUID` | object identity (matches the folder name) |
| `key` | `String` | S3 object key |
| `size` | `String` | content length as string |
| `contentType` | `String?` | MIME type |
| `etag` | `String?` | ETag value |
| `modificationDate` | `String` | formatted date string |
| `lastModified` | `Long` | epoch millis |
| `dataPath` | `Path` | path to the `binaryData` file |
| `userMetadata` | `Map<String, String>?` | `x-amz-meta-*` headers |
| `storeHeaders` | `Map<String, String>?` | headers persisted verbatim (e.g. `Content-Encoding`) |
| `encryptionHeaders` | `Map<String, String>?` | SSE headers |
| `tags` | `List<Tag>?` | object tags |
| `checksumAlgorithm` | `ChecksumAlgorithm?` | CRC32 / SHA-256 / etc. |
| `checksum` | `String?` | computed checksum value |
| `checksumType` | `ChecksumType?` | FULL\_OBJECT or COMPOSITE |
| `storageClass` | `StorageClass?` | STANDARD, GLACIER, etc. |
| `owner` | `Owner` | object owner |
| `legalHold` | `LegalHold?` | WORM legal hold status |
| `retention` | `Retention?` | WORM retention mode + until-date |
| `policy` | `AccessControlPolicy?` | ACL policy |
| `versionId` | `String?` | non-null when versioning is enabled |
| `deleteMarker` | `Boolean` | true for versioned delete markers |
Filesystem layout and metadata schemas (`BucketMetadata`, `S3ObjectMetadata` fields): see **[server/AGENTS.md](server/AGENTS.md) § Storage Schema**.

## Configuration

Expand Down Expand Up @@ -161,7 +109,11 @@ make install # Full build
make skip-docker # Skip Docker
make test # Unit tests only
make integration-tests # Run integration tests
make format # Format Kotlin code (ktlint)
make integration-test-class CLASS=BucketIT # Run one IT class (or CLASS=BucketIT#methodName)
make format # Format Kotlin code (ktlint, auto-fix)
make lint # Check style without auto-fixing (ktlint + Checkstyle)
make typecheck # Compile all modules without running tests
make check # lint + typecheck + test combined
make run # Run S3Mock from source (Spring Boot)
make sort # Sort POM files (sortpom)
```
Expand All @@ -173,7 +125,7 @@ Use the **`lint` skill** to fix formatting and verify style gates (ktlint + Chec
All PRs and pushes are validated by the `maven-ci-and-prb.yml` GitHub Actions workflow.

**Required gates** (all must pass before merge):
1. Compilation and build (`./mvnw clean install`)
1. Compilation and build (`make verify`)
2. Unit tests (`*Test.kt` in each module)
3. Integration tests (`*IT.kt` against Docker container)
4. ktlint (Kotlin code style)
Expand Down
2 changes: 1 addition & 1 deletion INVARIANTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,4 +79,4 @@ A task is not complete until all of the following are true:
- Unit tests cover the new or modified logic (`*Test.kt` in the module the change was made)
- Integration tests cover the observable HTTP/S3 behavior (`*IT.kt` in `integration-tests/`)
- `CHANGELOG.md` has an entry under the current version for any user-facing bug fix or feature
- `make format` passes (ktlint + Checkstyle)
- `make lint` passes (ktlint + Checkstyle)
12 changes: 11 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

# Agents: run only make targets listed here. No direct shell commands.

.PHONY: build verify install skip-docker format fmt lint typecheck check integration-tests run test test-class sort help
.PHONY: build verify install skip-docker format fmt lint typecheck check integration-tests integration-test-class run test test-class sort release help
.DEFAULT_GOAL := build

build: verify
Expand Down Expand Up @@ -58,6 +58,11 @@ test-class:
integration-tests:
./mvnw -B -V -Dstyle.color=always verify -pl integration-tests

# Run a single integration test class or method: make integration-test-class CLASS=BucketIT
# To run a specific method: make integration-test-class CLASS=BucketIT#shouldCreateBucket
integration-test-class:
./mvnw -B -V -Dstyle.color=always verify -pl integration-tests -Dit.test=$(CLASS)

# Master validation target: lint + typecheck + unit tests.
check: lint typecheck test

Expand All @@ -67,6 +72,9 @@ run:
sort:
./mvnw -B -V -Dstyle.color=always com.github.ekryd.sortpom:sortpom-maven-plugin:sort

release:
./mvnw -B -V release:prepare release:perform

help:
@echo ""
@echo "Usage: make <target>"
Expand All @@ -88,9 +96,11 @@ help:
@echo " test Unit tests only (server/ module)"
@echo " test-class Run one test class: make test-class CLASS=BucketServiceTest"
@echo " integration-tests Integration tests against a live Docker container"
@echo " integration-test-class Run one integration test: make integration-test-class CLASS=BucketIT"
@echo ""
@echo "Development"
@echo " run Run S3Mock from source via Spring Boot"
@echo " sort Sort POM files with sortpom"
@echo " release Prepare and perform a Maven release (CI use)"
@echo " help Show this message"
@echo ""
15 changes: 11 additions & 4 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,17 @@ graph TD
TC -->|path-style HTTP/HTTPS| CONN
JU5 -->|in-process| CONN
TNG -->|in-process| CONN
CONN --> KF --> Controller
Controller --> Service
Service --> Store
Store --> FS
CONN --> KF
KF --> BC & OC & MC & FC
KF --> KMS
BC --> BS
OC --> OS
MC --> MS
OS --> BKS
BS --> BKS
OS --> OBS
MS --> MPS
BKS & OBS & MPS --> FS
```

## Data Flow
Expand Down
12 changes: 6 additions & 6 deletions docs/SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ How to build, configure, and run S3Mock locally.

## Prerequisites

| Tool | Minimum version | Notes |
|---|---|---|
| Tool | Minimum version | Notes |
|---|---|------------------------------------------------|
| JDK | 25 | Build toolchain only — bytecode targets JDK 17 |
| Maven | 3.9+ | Or use the included `./mvnw` wrapper |
| Docker | Any recent version | Required for integration tests and `make run` |
| Maven | 3.9+ | Use make targets |
| Docker | Any recent version | Required for integration tests and `make run` |

Verify:
```bash
Expand Down Expand Up @@ -78,8 +78,8 @@ make check # lint + typecheck + unit tests

For a specific integration test:
```bash
./mvnw verify -pl integration-tests -Dit.test=BucketIT
./mvnw verify -pl integration-tests -Dit.test=BucketIT#shouldCreateBucket
make integration-test-class CLASS=BucketIT
make integration-test-class CLASS=BucketIT#shouldCreateBucket
```

## Running Validation
Expand Down
7 changes: 3 additions & 4 deletions docs/TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,8 @@ See **[docs/KOTLIN.md](KOTLIN.md)** for Kotlin naming conventions (backtick test
```bash
make test # Unit tests only
make integration-tests # All integration tests
./mvnw verify -pl integration-tests -Dit.test=BucketIT # Specific class
./mvnw verify -pl integration-tests -Dit.test=BucketIT#shouldCreateBucket # Specific method
./mvnw test -pl server -DskipDocker # Skip Docker
make integration-test-class CLASS=BucketIT # Specific class
make integration-test-class CLASS=BucketIT#shouldCreateBucket # Specific method
```

> Integration tests require Docker to be running.
Expand All @@ -118,7 +117,7 @@ make integration-tests # A
- **Docker not running**: Run `docker info` — if it fails, Docker is not running; escalate to the human rather than debugging the test failure
- **Port conflict**: Check `lsof -i :9090`
- **Flaky test**: Look for shared state or ordering dependencies
- **Compilation error**: Run `./mvnw clean install -DskipDocker -DskipTests` first
- **Compilation error**: Run `make typecheck` first

## Checklist

Expand Down
15 changes: 11 additions & 4 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,10 @@
<url>https://github.com/adobe/S3Mock/tree/main</url>
</scm>


<properties>

<archunit.version>1.4.2</archunit.version>

<aws-kotlin.version>1.6.76</aws-kotlin.version>

<aws-v2.version>2.44.8</aws-v2.version>
Expand Down Expand Up @@ -228,6 +229,12 @@
<artifactId>xmlunit-assertj3</artifactId>
<version>${xmlunit-assertj3.version}</version>
</dependency>
<dependency>
<groupId>com.tngtech.archunit</groupId>
<artifactId>archunit-junit5</artifactId>
<version>${archunit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>digital.pragmatech.testing</groupId>
<artifactId>spring-test-profiler</artifactId>
Expand All @@ -238,7 +245,6 @@
</dependencyManagement>

<build>
<sourceDirectory>src/main/kotlin</sourceDirectory>
<pluginManagement>
<plugins>
<plugin>
Expand Down Expand Up @@ -431,8 +437,8 @@
<requireJavaVersion>
<version>${java.version}</version>
</requireJavaVersion>
<dependencyConvergence />
<requireUpperBoundDeps />
<dependencyConvergence></dependencyConvergence>
<requireUpperBoundDeps></requireUpperBoundDeps>
</rules>
</configuration>
</plugin>
Expand Down Expand Up @@ -662,6 +668,7 @@
<artifactId>central-publishing-maven-plugin</artifactId>
</plugin>
</plugins>
<sourceDirectory>src/main/kotlin</sourceDirectory>
</build>

<profiles>
Expand Down
56 changes: 56 additions & 0 deletions server/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,62 @@ synchronized(lockStore[id]!!) {
- Never acquire more than one lock in a single call path — there is no established ordering, so taking two locks risks deadlock.
- Do not introduce `ReentrantLock`, `ReadWriteLock`, or other lock types — the existing `synchronized`/`Any()` pattern is intentional and consistent throughout all stores.

## Storage Schema

Filesystem layout:
```
<root>/<bucket>/bucketMetadata.json
<root>/<bucket>/<uuid>/binaryData
<root>/<bucket>/<uuid>/objectMetadata.json
<root>/<bucket>/<uuid>/<version-id>-binaryData # versioning
<root>/<bucket>/<uuid>/<version-id>-objectMetadata.json # versioning
<root>/<bucket>/multiparts/<upload-id>/multipartMetadata.json
<root>/<bucket>/multiparts/<upload-id>/<part>.part
```

**`bucketMetadata.json`** fields (`BucketMetadata`):

| Field | Type | Notes |
|---|---|---|
| `name` | `String` | Bucket name |
| `creationDate` | `String` | ISO-8601 timestamp |
| `bucketRegion` | `String` | AWS region string |
| `objects` | `Map<String, UUID>` | key → object UUID mapping |
| `versioningConfiguration` | `VersioningConfiguration?` | null until versioning is configured |
| `objectLockConfiguration` | `ObjectLockConfiguration?` | null until object lock is enabled |
| `bucketLifecycleConfiguration` | `BucketLifecycleConfiguration?` | null until lifecycle rules are set |
| `objectOwnership` | `ObjectOwnership?` | null until ownership is set |
| `bucketInfo` | `BucketInfo?` | bucket type/data-redundancy info |
| `locationInfo` | `LocationInfo?` | bucket location info |
| `path` | `Path` | filesystem path to the bucket folder (not serialized for cross-host use) |

**`objectMetadata.json`** fields (`S3ObjectMetadata`):

| Field | Type | Notes |
|---|---|---|
| `id` | `UUID` | object identity (matches the folder name) |
| `key` | `String` | S3 object key |
| `size` | `String` | content length as string |
| `contentType` | `String?` | MIME type |
| `etag` | `String?` | ETag value |
| `modificationDate` | `String` | formatted date string |
| `lastModified` | `Long` | epoch millis |
| `dataPath` | `Path` | path to the `binaryData` file |
| `userMetadata` | `Map<String, String>?` | `x-amz-meta-*` headers |
| `storeHeaders` | `Map<String, String>?` | headers persisted verbatim (e.g. `Content-Encoding`) |
| `encryptionHeaders` | `Map<String, String>?` | SSE headers |
| `tags` | `List<Tag>?` | object tags |
| `checksumAlgorithm` | `ChecksumAlgorithm?` | CRC32 / SHA-256 / etc. |
| `checksum` | `String?` | computed checksum value |
| `checksumType` | `ChecksumType?` | FULL\_OBJECT or COMPOSITE |
| `storageClass` | `StorageClass?` | STANDARD, GLACIER, etc. |
| `owner` | `Owner` | object owner |
| `legalHold` | `LegalHold?` | WORM legal hold status |
| `retention` | `Retention?` | WORM retention mode + until-date |
| `policy` | `AccessControlPolicy?` | ACL policy |
| `versionId` | `String?` | non-null when versioning is enabled |
| `deleteMarker` | `Boolean` | true for versioned delete markers |

## Testing

See **[docs/TESTING.md](../docs/TESTING.md)** for the full strategy. Service and store tests use `@SpringBootTest` with `@MockitoBean`; controller tests use `@WebMvcTest` with `@MockitoBean` and `BaseControllerTest`. Always extend the appropriate base class (`ServiceTestBase`, `StoreTestBase`, `BaseControllerTest`).
Expand Down
14 changes: 9 additions & 5 deletions server/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -101,14 +101,18 @@
<artifactId>jackson-module-kotlin</artifactId>
</dependency>
<dependency>
<groupId>digital.pragmatech.testing</groupId>
<artifactId>spring-test-profiler</artifactId>
<groupId>com.code-intelligence</groupId>
<artifactId>jazzer-junit</artifactId>
<scope>test</scope>
</dependency>
<!-- Test Dependencies -->
<dependency>
<groupId>com.code-intelligence</groupId>
<artifactId>jazzer-junit</artifactId>
<groupId>com.tngtech.archunit</groupId>
<artifactId>archunit-junit5</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>digital.pragmatech.testing</groupId>
<artifactId>spring-test-profiler</artifactId>
<scope>test</scope>
</dependency>
<dependency>
Expand Down
Loading
Loading