Skip to content

[codex] experimental python 3.14t feature server #6334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
franciscojavierarceo wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

[codex] experimental python 3.14t feature server #6334

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,5 @@ env
offline_build/
!offline_build/arrow
!offline_build/ibis
ui/node_modules
sdk/python/feast/ui/node_modules
14 changes: 14 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -683,6 +683,15 @@ build-feature-server-docker: ## Build Feature Server Docker image
-f sdk/python/feast/infra/feature_servers/multicloud/Dockerfile \
$(if $(filter true,$(DOCKER_PUSH)),--push,--load) sdk/python/feast/infra/feature_servers/multicloud

push-feature-server-experimental-docker: ## Push experimental Python 3.14 free-threaded Feature Server Docker image
docker push $(REGISTRY)/feature-server:$(VERSION)-python314t-experimental

build-feature-server-experimental-docker: ## Build experimental Python 3.14 free-threaded Feature Server Docker image
docker buildx build $(if $(DOCKER_PLATFORMS),--platform $(DOCKER_PLATFORMS),) \
-t $(REGISTRY)/feature-server:$(VERSION)-python314t-experimental \
-f sdk/python/feast/infra/feature_servers/multicloud/Dockerfile.experimental \
$(if $(filter true,$(DOCKER_PUSH)),--push,--load) .

push-feature-transformation-server-docker: ## Push Feature Transformation Server Docker image
docker push $(REGISTRY)/feature-transformation-server:$(VERSION)

Expand Down Expand Up @@ -733,6 +742,11 @@ build-feature-server-dev: ## Build Feature Server Dev Docker image
-t feastdev/feature-server:dev \
-f sdk/python/feast/infra/feature_servers/multicloud/Dockerfile.dev --load .

benchmark-feature-server-experimental: ## Benchmark stable vs experimental feature server containers
uv run python infra/scripts/benchmark_feature_server_experimental.py \
--baseline-image $(REGISTRY)/feature-server:$(VERSION) \
--experimental-image $(REGISTRY)/feature-server:$(VERSION)-python314t-experimental

build-feature-server-dev-docker: ## Build Feature Server Dev Docker image
docker buildx build $(if $(DOCKER_PLATFORMS),--platform $(DOCKER_PLATFORMS),) \
-t $(REGISTRY)/feature-server:$(VERSION) \
Expand Down
12 changes: 12 additions & 0 deletions docs/project/development-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,18 @@ make compile-protos-python
docker build -t docker-whale -f ./sdk/python/feast/infra/feature_servers/multicloud/Dockerfile .
```

### Building the experimental Python 3.14 free-threaded feature server image
```sh
make build-feature-server-experimental-docker

# Optional: compare it to the stable image with the sample local repo
make benchmark-feature-server-experimental
```

This experimental image is currently aimed at early compatibility and performance testing for the Python feature server. The benchmark target compares the stable and experimental containers against the sample local repo and is currently most representative for SQLite-backed serving experiments.

In the current SQLite benchmark run, the experimental `3.14t` image reached `157.01 req/s` versus `187.65 req/s` for the stable image, while improving p95 latency from `274.72 ms` to `262.61 ms`. Treat that as an early data point rather than a universal result: the free-threaded stack is still evolving, and dependency behavior can outweigh any no-GIL gains on a given workload.

### Code Style and Linting
Feast Python SDK and CLI codebase:
- Conforms to [Black code style](https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html)
Expand Down
28 changes: 27 additions & 1 deletion docs/reference/feature-servers/python-feature-server.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,32 @@ Key performance options:
- Use HTTP health checks instead of TCP for better application-level monitoring
- Consider horizontal pod autoscaling based on request latency metrics

### Experimental Python 3.14 Free-Threaded Container

Feast also ships an experimental feature-server container build for Python 3.14 free-threaded (`3.14t`) runtimes. This image is intended for benchmarking and early compatibility validation, not as the default production image.

Today, the experimental image is best treated as a SQLite-oriented serving experiment. The online-serving startup path has been trimmed to avoid eagerly importing `pandas`, which makes it a better fit for free-threaded serving benchmarks, but the broader Feast runtime and some offline-oriented paths still rely on dependencies that are not fully optimized for no-GIL execution yet.

```bash
# Build the stable image
make build-feature-server-docker

# Build the experimental no-GIL image
make build-feature-server-experimental-docker

# Compare the two images against the same sample feature repo
make benchmark-feature-server-experimental
```

The benchmark target writes a JSON report to `feature-server-experimental-benchmark.json` with throughput and latency comparisons for the stable and experimental containers. The current harness uses the sample local feature repo and is most useful for side-by-side SQLite serving comparisons. Because the free-threaded Python ecosystem is still evolving, treat the result as a workload-specific experiment and validate dependency compatibility before promoting it further.

On the current SQLite benchmark run that seeded this experiment, the experimental `3.14t` image delivered lower throughput but slightly better tail latency than the stable image:

- Stable image: `187.65 req/s`, `152.17 ms` mean latency, `274.72 ms` p95 latency, `35` failed requests
- Experimental image: `157.01 req/s`, `183.21 ms` mean latency, `262.61 ms` p95 latency, `31` failed requests

That works out to about `16%` lower throughput for the experimental image, alongside about `4.6%` better p95 latency on this workload. In other words, the current no-GIL image is not yet a clear performance win for SQLite online serving, but it is now easy to benchmark and iterate on as dependencies improve.

## Deploying as a service

See [this](../../how-to-guides/running-feast-in-production.md#id-4.2.-deploy-feast-feature-servers-on-kubernetes) for an example on how to run Feast on Kubernetes using the Operator.
Expand Down Expand Up @@ -536,4 +562,4 @@ The [PyTorch NLP template](https://github.com/feast-dev/feast/tree/main/sdk/pyth

## How to configure Authentication and Authorization ?

Please refer the [page](./../../../docs/getting-started/concepts/permission.md) for more details on how to configure authentication and authorization.
Please refer the [page](./../../../docs/getting-started/concepts/permission.md) for more details on how to configure authentication and authorization.
Loading
Loading