erofs: add LRU dentry cache to bound memory usage#13199
Conversation
Reference counting: parent holds ref on each child in childMap. Lookup returns a ref'd dentry. Dentries with refs==1 (only parent's ref) and no children enter the LRU cache. Eviction removes from parent's childMap and DecRefs the dentry. Cache limit is 1000.
Could you explain this? benchmarks are regressing with caching? Or improving because lower-is-better for latency benchmarks. What are these microbenchmarks? |
Sorry for the unclear description. The baseline is the current code (no eviction - all dentries cached forever). Throughput drops because the patch adds ref counting on every path step and evicts dentries beyond the 1000 limit. Without this patch, memory grows unbounded. Microbenchmarks are find/read/stat workloads. Parallel find: -2%. Over-cache random read: -5%. In-cache stat: -8%. Over-cache stat: -22%. |
2 participants
Fixes #13195
The erofs filesystem never evicts cached dentries, leading to unbounded memory growth (4 MB -> 2.3 GB after walking 1.5M files).
Add an LRU dentry cache modeled after fsimpl/gofer. Dentries with refs == 1 (only the parent's reference) and no children are placed on an LRU list and evicted when the cache exceeds 1000 entries. Path resolution functions now follow ref-on-return semantics.
Microbenchmarks show up to ~20% regression.