Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,24 @@
# Changelog

## [0.3.1] - 2026-05-21

### Added
- `pos3` console-script entry point with `ls`, `download`, `upload` subcommands
([#10](https://github.com/Positronic-Robotics/pos3/issues/10)).
- `pos3 ls <prefix> [-r] [--profile NAME]` lists objects, one full `s3://` URL
per line on stdout.
- `pos3 download <url> [--local PATH] [--delete] [--exclude PATTERN]... [--profile NAME]`
prints only the resulting local path to stdout; progress and logs go to
stderr, so `data_dir=$(pos3 download s3://bucket/dataset/)` is safe.
- `pos3 upload <url> [--local PATH] [--delete] [--exclude PATTERN]... [--profile NAME]`
is one-shot (no background loop or interval). Source defaults to the cache
path `pos3 download` would have produced; errors if the source doesn't
exist.
- `--delete` defaults OFF for both `download` and `upload` (the Python API
defaults to `True`; the CLI is more conservative for interactive use).
- `--profile` is supported alongside the URL form `s3://<profile>@bucket/...`;
the URL form wins on conflict, matching the Python precedence.

## [0.3.0] - 2026-05-19

### Added
Expand Down
33 changes: 33 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,39 @@ Lists files/objects in a directory or S3 prefix.

**Returns**: List of full S3 URLs or local paths.

## CLI

`pos3` ships a small command-line interface for the most common one-shot
operations. After `uv pip install pos3` (or `pip install pos3`), `pos3` is on
your `$PATH`:

```bash
# List objects (one full s3:// URL per line on stdout)
pos3 ls s3://bucket/dataset/
pos3 ls -r s3://bucket/dataset/

# Download an S3 prefix or object into the cache (or a custom --local path).
# Only the resulting local path is written to stdout — progress and logs go
# to stderr — so it's safe to capture in a shell variable:
data_dir=$(pos3 download s3://bucket/dataset/)

# One-shot upload. Source defaults to the same cache path `pos3 download`
# would have produced; --local overrides. Errors if the source doesn't exist.
pos3 upload s3://bucket/results/ --local ./out/
```

All three subcommands accept `--profile NAME`. The URL form
`s3://<profile>@bucket/...` takes precedence over `--profile` on conflict
(matching the Python API).

`--delete` defaults to **OFF** for both `download` and `upload`, even though
the Python API defaults to `True`. CLI defaults are conservative for
interactive shell use; pass `--delete` explicitly to mirror file removals.

The CLI is one-shot only — no background sync, no `pos3 sync` subcommand.
Use the Python `pos3.mirror()` context manager when you need an interval-based
loop or bi-directional sync over a job's lifetime.

## Comparison with Libraries

Why use `pos3` instead of other Python libraries?
Expand Down
135 changes: 135 additions & 0 deletions pos3/cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
"""Command-line interface for pos3.

Three subcommands — ``ls``, ``download``, ``upload`` — each running inside a
short-lived ``pos3.mirror()`` context. ``upload`` is one-shot: no background
interval, no sync loop.

CLI defaults intentionally differ from the Python API: ``--delete`` defaults
to OFF for both ``download`` and ``upload``, because the API's ``True``
default is too destructive for interactive shell use. ``download`` writes
only the resulting local path to stdout so the output is safe to capture in
``$(pos3 download ...)``; progress bars and logs go to stderr.
"""

from __future__ import annotations

import argparse
import sys
from pathlib import Path

from . import _is_s3_path, _require_active_mirror, download, ls, mirror, upload
from .profiles import _resolve_profile, _url_profile


def _build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="pos3", description="pos3 command-line interface.")
subparsers = parser.add_subparsers(dest="command", required=True)

def add_profile(p: argparse.ArgumentParser) -> None:
p.add_argument(
"--profile",
metavar="NAME",
help="Named pos3 profile. Overridden by the URL form s3://<profile>@bucket/key.",
)

def add_transfer_args(p: argparse.ArgumentParser, local_help: str) -> None:
p.add_argument("--local", metavar="PATH", help=local_help)
p.add_argument(
"--delete",
action="store_true",
help="Delete files not present at the other end. Defaults to OFF in the CLI.",
)
p.add_argument(
"--exclude",
metavar="PATTERN",
action="append",
default=None,
help="Glob pattern to skip. May be passed multiple times.",
)
add_profile(p)

p_ls = subparsers.add_parser("ls", help="List objects under a prefix.")
p_ls.add_argument("prefix", help="S3 prefix (s3://bucket/key) or local path.")
p_ls.add_argument("-r", "--recursive", action="store_true", help="List subdirectories recursively.")
add_profile(p_ls)

p_dl = subparsers.add_parser("download", help="Download an S3 prefix or object to a local path.")
p_dl.add_argument("url", help="Source S3 URL (s3://bucket/key).")
add_transfer_args(p_dl, local_help="Destination path. Defaults to the cache path.")

p_up = subparsers.add_parser("upload", help="One-shot upload of a local path to S3.")
p_up.add_argument("url", help="Destination S3 URL (s3://bucket/key).")
add_transfer_args(p_up, local_help="Source path. Defaults to the cache path.")

return parser


def _cmd_ls(args: argparse.Namespace) -> int:
with mirror(show_progress=False):
for item in ls(args.prefix, recursive=args.recursive, profile=args.profile):
print(item)
return 0


def _cmd_download(args: argparse.Namespace) -> int:
with mirror(show_progress=True):
local_path = download(
args.url,
local=args.local,
delete=args.delete,
Comment on lines +102 to +105
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject non-S3 source URLs in download

_cmd_download forwards args.url directly to download() without validating the s3:// scheme, but download() treats non-S3 inputs as local passthrough (_Mirror.download returns the resolved path and does no transfer). As a result, pos3 download bucket/path exits 0 and prints a local path even though nothing was downloaded, which can silently break shell pipelines that rely on success status and stdout path.

Useful? React with 👍 / 👎.

exclude=args.exclude,
profile=args.profile,
)
print(str(local_path))
return 0


def _cmd_upload(args: argparse.Namespace) -> int:
with mirror(show_progress=True):
mirror_obj = _require_active_mirror()
if args.local:
source = Path(args.local).expanduser().resolve()
else:
# Resolve the same profile precedence (URL > --profile > context default)
# the upload() call will use, so the cache path we check matches the one
# upload() would target.
profile: str | None = args.profile
if _is_s3_path(args.url):
url_profile = _url_profile(args.url)
if url_profile is not None:
profile = url_profile
effective_profile = _resolve_profile(profile) or mirror_obj.options.default_profile
source = mirror_obj.options.cache_path_for(args.url, effective_profile)
if not source.exists():
print(f"pos3 upload: source path does not exist: {source}", file=sys.stderr)
return 1
upload(
args.url,
local=source,
Comment on lines +146 to +148
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject non-S3 destination URLs in upload

When --local is provided, _cmd_upload never validates that args.url is an S3 URL before calling upload(). For non-S3 destinations, upload() takes the local-path passthrough branch (_Mirror.upload), creates that local directory, and returns success without any remote upload. This makes commands like pos3 upload ./dst --local ./src report success while performing no S3 operation.

Useful? React with 👍 / 👎.

interval=None,
delete=args.delete,
exclude=args.exclude,
profile=args.profile,
)
return 0


_COMMANDS = {
"ls": _cmd_ls,
"download": _cmd_download,
"upload": _cmd_upload,
}


def main(argv: list[str] | None = None) -> int:
parser = _build_parser()
args = parser.parse_args(argv)
try:
return _COMMANDS[args.command](args)
except ValueError as exc:
print(f"pos3 {args.command}: {exc}", file=sys.stderr)
return 1


if __name__ == "__main__":
sys.exit(main())
5 changes: 4 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "pos3"
version = "0.3.0"
version = "0.3.1"
description = "S3 Simple Sync - Make using S3 as simple as using local files"
readme = "README.md"
requires-python = ">=3.11"
Expand All @@ -27,6 +27,9 @@ dependencies = [
Homepage = "https://github.com/Positronic-Robotics/pos3"
Repository = "https://github.com/Positronic-Robotics/pos3"

[project.scripts]
pos3 = "pos3.cli:main"

[project.optional-dependencies]
dev = [
"pytest>=7.0",
Expand Down
Loading
Loading