Feature request: model caching setup and switch to local storage model loading path if the model cache exist in specified dir

[wwvela]

Problem

When streaming models from object storage (S3/GCS/Azure) in Kubernetes environments, every pod startup requires a full network download — even when multiple pods on the same node load the same model. This wastes network bandwidth and increases cold start latency, especially during scale-up events where multiple pods start simultaneously on the same node.

Proposed Solution

Add a write-through local cache to the runai-model-streamer. When RUNAI_STREAMER_CACHE_DIR is set, tensor data streamed from object storage is simultaneously written to a local directory (typically NVMe-backed). Subsequent pods on the same node read directly from local storage instead of re-downloading from S3 which could improve the performance greatly.

Use Cases

1. Horizontal scale-up: First pod downloads from S3, second+ pods on the same node load from local NVMe at full disk speed
2. Pod restarts: Crashed/restarted pods on the same node load instantly from cache
3. Multi-tenant nodes: Different models cached side-by-side, each served from local storage on repeat access

Expected Performance Impact

Scenario	Without Cache	With Cache (1st pod - cache miss)	With Cache (2nd pod - cache hit)
22GB model, g6e (TP=4)	8.5s	8.3s (~same without cache)	6.9s (19% faster via NVME loading path)
64GB model, p5 (TP=4)	15.7s	15.7s (~same without cache)	7.6s (52% faster via NVME loading path)
220GB model, p5 (TP=8)	33.6s	29.4s (~same without cache)	9.6s (71% faster via NVME loading path)

[wwvela]

I created a pull request for this feature and did some experiment testing by pulling model from s3 bucket with different model size and instance type. Post the test outcome in the pull request description.