Skip to content

feat(cache): filesystem hardening with hex sharding and size limits #1125

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vibegui wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat(cache): filesystem hardening with hex sharding and size limits #1125

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c236751
feat(cache): filesystem hardening with hex sharding, size limits, and…
vibegui Mar 18, 2026
947e388
refactor(cache): move LRU observability to PR #1124
vibegui Mar 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions runtime/caches/cacheWriteWorker.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
// Worker thread for cache write operations.
// Offloads SHA1 hashing, buffer combining, and FS writes from the main event loop.

const CACHE_MAX_ENTRY_SIZE = parseInt(
Deno.env.get("CACHE_MAX_ENTRY_SIZE") ?? "2097152", // 2 MB
) || 2097152;

const textEncoder = new TextEncoder();

const initializedDirs = new Set<string>();
Expand Down Expand Up @@ -57,6 +61,12 @@ function generateCombinedBuffer(
return buf;
}

function shardedPath(cacheDir: string, key: string): string {
const l1 = key.substring(0, 2);
const l2 = key.substring(2, 4);
return `${cacheDir}/${l1}/${l2}/${key}`;
}

// --- Message handler ---

export interface CacheWriteMessage {
Expand Down Expand Up @@ -85,8 +95,12 @@ self.onmessage = async (e: MessageEvent<CacheWriteMessage>) => {
// Combine into single buffer
const buffer = generateCombinedBuffer(body, headersBytes);

// Write to filesystem
const filePath = `${cacheDir}/${cacheKey}`;
if (buffer.length > CACHE_MAX_ENTRY_SIZE) return;

// Write to filesystem (with hex sharding for directory distribution)
const filePath = shardedPath(cacheDir, cacheKey);
const dir = filePath.substring(0, filePath.lastIndexOf("/"));
ensureCacheDir(dir);
await Deno.writeFile(filePath, buffer);
} catch (err) {
console.error("[cache-write-worker] error:", err);
Expand Down
67 changes: 62 additions & 5 deletions runtime/caches/fileSystem.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,45 @@ import {
const FILE_SYSTEM_CACHE_DIRECTORY =
Deno.env.get("FILE_SYSTEM_CACHE_DIRECTORY") ?? "/tmp/deco_cache";

const CACHE_MAX_ENTRY_SIZE = parseInt(
Deno.env.get("CACHE_MAX_ENTRY_SIZE") ?? "2097152", // 2 MB
) || 2097152;

// Warn when write rate exceeds this many writes per minute.
// High write rates usually indicate bots, missing cache keys, or very short TTLs.
const CACHE_WRITE_RATE_WARN = parseInt(
Deno.env.get("CACHE_WRITE_RATE_WARN") ?? "500",
) || 500;

// --- Write rate tracking ---
let writeCount = 0;
let writeWindowStart = Date.now();

function trackWriteRate(key: string) {
const now = Date.now();
if (now - writeWindowStart > 60_000) {
writeWindowStart = now;
writeCount = 0;
}
writeCount++;
if (writeCount === CACHE_WRITE_RATE_WARN) {
logger.warn(
`fs_cache: high write rate — ${writeCount} writes in the last minute. ` +
`Latest key: ${key}. ` +
`Consider increasing CACHE_MAX_AGE_S or reviewing loader cacheKey functions. ` +
`Adjust threshold with CACHE_WRITE_RATE_WARN (current: ${CACHE_WRITE_RATE_WARN}/min).`,
);
}
}

const initializedShardDirs = new Set<string>();

function shardedPath(cacheDir: string, key: string): string {
const l1 = key.substring(0, 2);
const l2 = key.substring(2, 4);
return `${cacheDir}/${l1}/${l2}/${key}`;
}

// Reuse TextEncoder instance to avoid repeated instantiation
const textEncoder = new TextEncoder();

Expand Down Expand Up @@ -106,7 +145,7 @@ function createFileSystemCache(): CacheStorage {
if (
FILE_SYSTEM_CACHE_DIRECTORY && !existsSync(FILE_SYSTEM_CACHE_DIRECTORY)
) {
await Deno.mkdirSync(FILE_SYSTEM_CACHE_DIRECTORY, { recursive: true });
await Deno.mkdir(FILE_SYSTEM_CACHE_DIRECTORY, { recursive: true });
}
isCacheInitialized = true;
} catch (err) {
Expand All @@ -118,11 +157,25 @@ function createFileSystemCache(): CacheStorage {
key: string,
responseArray: Uint8Array,
) {
if (responseArray.length > CACHE_MAX_ENTRY_SIZE) {
// Evict any existing entry so stale data doesn't stay pinned on disk.
deleteFile(key).catch(() => {});
return;
}
if (!isCacheInitialized) {
await assertCacheDirectory();
}
const filePath = `${FILE_SYSTEM_CACHE_DIRECTORY}/${key}`;

trackWriteRate(key);
const filePath = shardedPath(FILE_SYSTEM_CACHE_DIRECTORY, key);

@cubic-dev-ai cubic-dev-ai Bot Mar 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Oversized-entry eviction only targets the new sharded path, leaving legacy flat-path cache files undeleted.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At runtime/caches/fileSystem.ts, line 169:

<comment>Oversized-entry eviction only targets the new sharded path, leaving legacy flat-path cache files undeleted.</comment>

<file context>
@@ -118,11 +157,25 @@ function createFileSystemCache(): CacheStorage {
-    const filePath = `${FILE_SYSTEM_CACHE_DIRECTORY}/${key}`;
-
+    trackWriteRate(key);
+    const filePath = shardedPath(FILE_SYSTEM_CACHE_DIRECTORY, key);
+    const dir = filePath.substring(0, filePath.lastIndexOf("/"));
+    if (!initializedShardDirs.has(dir)) {
</file context>
Fix with Cubic

const dir = filePath.substring(0, filePath.lastIndexOf("/"));
if (!initializedShardDirs.has(dir)) {
try {
await Deno.mkdir(dir, { recursive: true });
initializedShardDirs.add(dir);
} catch {
// transient failure — don't mark initialized so next write retries mkdir
}
}
await Deno.writeFile(filePath, responseArray);
return;
}
Expand All @@ -132,8 +185,12 @@ function createFileSystemCache(): CacheStorage {
await assertCacheDirectory();
}
try {
const filePath = `${FILE_SYSTEM_CACHE_DIRECTORY}/${key}`;
const filePath = shardedPath(FILE_SYSTEM_CACHE_DIRECTORY, key);
const fileContent = await Deno.readFile(filePath);
if (fileContent.length > CACHE_MAX_ENTRY_SIZE) {
Deno.remove(filePath).catch(() => {});
return null;
}
return fileContent;
} catch (_err) {
const err = _err as { code?: string };
Expand All @@ -151,7 +208,7 @@ function createFileSystemCache(): CacheStorage {
await assertCacheDirectory();
}
try {
const filePath = `${FILE_SYSTEM_CACHE_DIRECTORY}/${key}`;
const filePath = shardedPath(FILE_SYSTEM_CACHE_DIRECTORY, key);
await Deno.remove(filePath);
return true;
} catch (err) {
Expand Down
54 changes: 53 additions & 1 deletion runtime/caches/lrucache.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,19 @@
import { LRUCache } from "npm:lru-cache@10.2.0";
import { ValueType } from "../../deps.ts";
import { logger } from "../../observability/otel/config.ts";
import { meter } from "../../observability/otel/metrics.ts";

@coderabbitai coderabbitai Bot Mar 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Address pipeline failure: file not formatted.

The CI pipeline reports that this file fails deno fmt --check. Please run deno fmt to fix formatting.

🧰 Tools
🪛 GitHub Actions: ci

[error] 1-1: deno fmt --check found not formatted file: runtime/caches/lrucache.ts.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@runtime/caches/lrucache.ts` around lines 1 - 4, The file fails formatting
checks; run the Deno formatter and commit the changes (or apply equivalent
formatting) so `deno fmt --check` passes; specifically reformat the import block
and surrounding code in the module that imports LRUCache, ValueType, logger, and
meter (the exports/definitions around LRUCache and any functions or classes in
lrucache.ts) to match Deno's style guidelines and ensure the file is
syntactically unchanged except for formatting.

import {
assertCanBeCached,
assertNoOptions,
baseCache,
createBaseCacheStorage,
} from "./utils.ts";

const lruEvictionCounter = meter.createCounter("lru_cache_eviction", {
unit: "1",
valueType: ValueType.DOUBLE,
});

// keep compatible with old variable name
const CACHE_MAX_SIZE = parseInt(
Deno.env.get("CACHE_MAX_SIZE") ?? Deno.env.get("MAX_CACHE_SIZE") ??
Expand All @@ -30,17 +38,61 @@ const cacheOptions = (cache: Cache) => (
maxSize: CACHE_MAX_SIZE,
ttlAutopurge: CACHE_TTL_AUTOPURGE,
ttlResolution: CACHE_TTL_RESOLUTION,
dispose: async (_value: boolean, key: string) => {
dispose: async (_value: boolean, key: string, reason: string) => {
lruEvictionCounter.add(1, { reason });
await cache.delete(key);
},

@coderabbitai coderabbitai Bot Mar 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🌐 Web query:

lru-cache npm dispose callback signature reason parameter

💡 Result:

The dispose callback in the lru-cache npm package (current version 11.2.7 as of 2026-03-13) has the signature (value: V, key: K, reason: DisposeReason) => void. It is provided via the dispose or disposeAfter options when creating an LRUCache instance. The reason parameter is a string literal union type DisposeReason with the following possible values: - 'evict': Item evicted because it is the least recently used and the cache is full. - 'set': A new value was set, overwriting the old value. - 'delete': Item explicitly deleted via delete, clear, or set with undefined. - 'expire': Item removed due to exceeding its TTL. - 'fetch': A fetchMethod operation returned undefined or was aborted. Example usage: const cache = new LRUCache({ max: 500, dispose: (value, key, reason) => { console.log(Disposing ${key} (${reason}):, value); // cleanup logic here } }); Note: The dispose callback is called before the item is fully removed from the cache. Use disposeAfter for post-removal cleanup. In older versions (< v7), the signature was (key, value) without reason.

Citations:

Fix the dispose callback signature: it must be synchronous, not async.

The lru-cache library (v10.2.0 and later) expects the dispose callback to have the signature (value: V, key: K, reason: DisposeReason) => void where reason is one of 'evict', 'set', 'delete', or 'expire'. The current implementation incorrectly marks the callback as async. Remove the async keyword and the await on cache.delete(key) to match the expected synchronous signature. If post-removal cleanup is needed, use the disposeAfter option instead.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@runtime/caches/lrucache.ts` around lines 41 - 44, The dispose callback for
the LRU must be synchronous: remove the async keyword from the dispose function
on the LRU options (the dispose handler currently declared as dispose: async
(_value: boolean, key: string, reason: string) => { ... }) and remove the await
before cache.delete(key) so the function returns void; keep the
lruEvictionCounter.add(1, { reason }) call but perform synchronous
cache.delete(key) (or defer any asynchronous cleanup to disposeAfter) to match
the required signature and allowed DisposeReason values.

}
);

const lruSizeGauge = meter.createObservableGauge("lru_cache_keys", {
description: "number of keys in the LRU cache",
unit: "1",
valueType: ValueType.DOUBLE,
});

const lruBytesGauge = meter.createObservableGauge("lru_cache_bytes", {
description: "total bytes tracked by the LRU cache",
unit: "bytes",
valueType: ValueType.DOUBLE,
});

// deno-lint-ignore no-explicit-any
const activeCaches = new Map<string, LRUCache<string, any>>();

lruSizeGauge.addCallback((observer) => {
for (const [name, lru] of activeCaches) {
observer.observe(lru.size, { cache: name });
}
});
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

// Warn when LRU disk usage exceeds this fraction of CACHE_MAX_SIZE.
// At this point the LRU is evicting aggressively and disk is nearly full.
const LRU_DISK_WARN_RATIO = parseFloat(
Deno.env.get("LRU_DISK_WARN_RATIO") ?? "0.9",
);

lruBytesGauge.addCallback((observer) => {
for (const [name, lru] of activeCaches) {
observer.observe(lru.calculatedSize, { cache: name });
const ratio = lru.calculatedSize / CACHE_MAX_SIZE;
if (ratio >= LRU_DISK_WARN_RATIO) {
logger.warn(
`lru_cache: disk usage for cache "${name}" is at ` +
`${Math.round(lru.calculatedSize / 1024 / 1024)}MB / ` +
`${Math.round(CACHE_MAX_SIZE / 1024 / 1024)}MB (${Math.round(ratio * 100)}%). ` +
`LRU is evicting aggressively. Consider increasing CACHE_MAX_SIZE or reducing CACHE_MAX_AGE_S.`,
);
}
}
});

function createLruCacheStorage(cacheStorageInner: CacheStorage): CacheStorage {
const caches = createBaseCacheStorage(
cacheStorageInner,
(_cacheName, cacheInner, requestURLSHA1) => {
const fileCache = new LRUCache(cacheOptions(cacheInner));
activeCaches.set(_cacheName, fileCache);
return Promise.resolve({
...baseCache,
delete: async (
Expand Down
Loading