-
-
Notifications
You must be signed in to change notification settings - Fork 35.5k
vfs: add minimal node:vfs subsystem #63115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mcollina
wants to merge
25
commits into
nodejs:main
Choose a base branch
from
mcollina:vfs-minimal
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 17 commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
adb398b
vfs: add minimal node:vfs subsystem
mcollina ce68078
test: add VFS API tests adapted from mount-based tests
mcollina 78cbe8f
test: add VFS unit tests for VirtualDir, file handles, and provider base
mcollina 6f07670
test: adapt more VFS tests to direct API
mcollina f44f6fb
test: add VFS callback, stream, watch, and real-provider async tests
mcollina 43c5876
test: cover remaining VFS API edge cases
mcollina 2395931
test: push every VFS file to >=95% line coverage
mcollina d2c6d4e
test: push branch coverage to 95%+
mcollina 2d78d3a
test: rename and split VFS test files into topic-based names
mcollina bb3beae
vfs: gate node:vfs behind --experimental-vfs flag
mcollina a55b8d8
vfs: fix lint errors
mcollina 6a9b3af
vfs: import getVirtualFd eagerly in streams
mcollina afb36cf
doc: trim vfs docs to match the minimal API
mcollina bf7f7f7
vfs: register node:vfs as an experimental builtin in CI metadata
mcollina b703045
test: stabilize test-vfs-watch-directory
mcollina a487356
Apply suggestions from code review
mcollina 3709515
module: exclude node:vfs from builtinModules when flag is disabled
mcollina c76d3d3
Apply suggestions from code review
mcollina 939365a
vfs: use posix paths on windows
mcollina 56589c8
Apply suggestions from code review
mcollina e947d94
Apply suggestions from code review
mcollina 1f47d56
vfs: drop RealFileHandle.fd getter
mcollina 3858f68
vfs: fix regressions from code-review changes
mcollina a58b46b
fixup typo
mcollina 82cde0f
vfs: add TODO to reuse FileHandle for async ops
mcollina File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,307 @@ | ||
| # Virtual File System | ||
|
|
||
| <!--introduced_in=REPLACEME--> | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| > Stability: 1 - Experimental | ||
|
|
||
| <!-- source_link=lib/vfs.js --> | ||
|
|
||
| The `node:vfs` module provides an in-memory virtual file system with an | ||
| `fs`-like API. It is useful for tests, fixtures, embedded assets, and other | ||
|
mcollina marked this conversation as resolved.
Outdated
|
||
| scenarios where you need a self-contained file system without touching the | ||
| actual file-system. | ||
|
|
||
| To access it: | ||
|
|
||
| ```mjs | ||
| import vfs from 'node:vfs'; | ||
| ``` | ||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
| ``` | ||
|
|
||
| This module is only available under the `node:` scheme, and only when Node.js | ||
| is started with the `--experimental-vfs` flag. | ||
|
|
||
| ## Basic usage | ||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
|
|
||
| const myVfs = vfs.create(); | ||
| myVfs.mkdirSync('/dir', { recursive: true }); | ||
| myVfs.writeFileSync('/dir/hello.txt', 'Hello, VFS!'); | ||
|
|
||
| console.log(myVfs.readFileSync('/dir/hello.txt', 'utf8')); // 'Hello, VFS!' | ||
| ``` | ||
|
|
||
| `vfs.create()` returns a [`VirtualFileSystem`][] instance backed by a | ||
| [`MemoryProvider`][] by default. The instance exposes synchronous, | ||
| callback-based, and promise-based file system methods that mirror the | ||
| shape of the [`node:fs`][] API. All paths are POSIX-style and absolute | ||
| (starting with `/`). | ||
|
|
||
| ## `vfs.create([provider][, options])` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * `provider` {VirtualProvider} The provider to use. **Default:** | ||
| `new MemoryProvider()`. | ||
| * `options` {Object} | ||
| * `emitExperimentalWarning` {boolean} Whether to emit the experimental | ||
| warning when the instance is created. **Default:** `true`. | ||
| * Returns: {VirtualFileSystem} | ||
|
|
||
| Convenience factory equivalent to `new VirtualFileSystem(provider, options)`. | ||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
|
|
||
| // Default in-memory provider | ||
| const memoryVfs = vfs.create(); | ||
|
|
||
| // Explicit provider | ||
| const realVfs = vfs.create(new vfs.RealFSProvider('/tmp/sandbox')); | ||
| ``` | ||
|
|
||
| ## Class: `VirtualFileSystem` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| A `VirtualFileSystem` wraps a [`VirtualProvider`][] and exposes a | ||
| `node:fs`-like API. Each instance maintains its own file tree. | ||
|
|
||
| ### `new VirtualFileSystem([provider][, options])` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * `provider` {VirtualProvider} The provider to use. **Default:** | ||
| `new MemoryProvider()`. | ||
| * `options` {Object} | ||
| * `emitExperimentalWarning` {boolean} Whether to emit the experimental | ||
| warning. **Default:** `true`. | ||
|
|
||
| ### `vfs.provider` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * {VirtualProvider} | ||
|
|
||
| The provider backing this VFS instance. | ||
|
|
||
| ### `vfs.readonly` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * {boolean} | ||
|
|
||
| `true` when the underlying provider is read-only. | ||
|
|
||
| ### File system methods | ||
|
mcollina marked this conversation as resolved.
Outdated
|
||
|
|
||
| `VirtualFileSystem` implements the following methods, with the same | ||
| signatures as their [`node:fs`][] counterparts: | ||
|
|
||
| #### Synchronous methods | ||
|
mcollina marked this conversation as resolved.
Outdated
|
||
|
|
||
| * `existsSync(path)` | ||
| * `statSync(path[, options])` | ||
| * `lstatSync(path[, options])` | ||
| * `readFileSync(path[, options])` | ||
| * `writeFileSync(path, data[, options])` | ||
| * `appendFileSync(path, data[, options])` | ||
| * `readdirSync(path[, options])` | ||
| * `mkdirSync(path[, options])` | ||
| * `rmdirSync(path)` | ||
| * `unlinkSync(path)` | ||
| * `renameSync(oldPath, newPath)` | ||
| * `copyFileSync(src, dest[, mode])` | ||
| * `realpathSync(path[, options])` | ||
| * `readlinkSync(path[, options])` | ||
| * `symlinkSync(target, path[, type])` | ||
| * `accessSync(path[, mode])` | ||
| * `rmSync(path[, options])` | ||
| * `truncateSync(path[, len])` | ||
| * `ftruncateSync(fd[, len])` | ||
| * `linkSync(existingPath, newPath)` | ||
| * `chmodSync(path, mode)` | ||
| * `chownSync(path, uid, gid)` | ||
| * `utimesSync(path, atime, mtime)` | ||
| * `lutimesSync(path, atime, mtime)` | ||
| * `mkdtempSync(prefix)` | ||
| * `opendirSync(path[, options])` | ||
| * `openAsBlob(path[, options])` | ||
| * File-descriptor ops: `openSync`, `closeSync`, `readSync`, `writeSync`, | ||
| `fstatSync` | ||
| * Streams: `createReadStream`, `createWriteStream` | ||
| * Watchers: `watch`, `watchFile`, `unwatchFile` | ||
|
|
||
| #### Callback-style asynchronous methods | ||
|
mcollina marked this conversation as resolved.
Outdated
|
||
|
|
||
| `readFile`, `writeFile`, `stat`, `lstat`, `readdir`, `realpath`, `readlink`, | ||
| `access`, `open`, `close`, `read`, `write`, `rm`, `fstat`, `truncate`, | ||
| `ftruncate`, `link`, `mkdtemp`, `opendir`. Each takes a Node.js-style | ||
| callback `(err, ...result)`. | ||
|
mcollina marked this conversation as resolved.
Outdated
|
||
|
|
||
| #### Promise methods | ||
|
mcollina marked this conversation as resolved.
Outdated
|
||
|
|
||
| `vfs.promises` exposes the promise-based variants: | ||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
|
|
||
| async function example() { | ||
| const myVfs = vfs.create(); | ||
| await myVfs.promises.writeFile('/file.txt', 'hello'); | ||
| const data = await myVfs.promises.readFile('/file.txt', 'utf8'); | ||
| return data; | ||
| } | ||
| example(); | ||
| ``` | ||
|
|
||
| The promise namespace mirrors `fs.promises` and includes `readFile`, | ||
| `writeFile`, `appendFile`, `stat`, `lstat`, `readdir`, `mkdir`, `rmdir`, | ||
| `unlink`, `rename`, `copyFile`, `realpath`, `readlink`, `symlink`, | ||
| `access`, `rm`, `truncate`, `link`, `mkdtemp`, `chmod`, `chown`, `lchown`, | ||
| `utimes`, `lutimes`, `open`, `lchmod`, and `watch`. | ||
|
|
||
| ## Class: `VirtualProvider` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| The base class for all VFS providers. Subclasses implement the essential | ||
| primitives (`open`, `stat`, `readdir`, `mkdir`, `rmdir`, `unlink`, | ||
| `rename`, ...) and inherit default implementations of the derived | ||
| methods (`readFile`, `writeFile`, `exists`, `copyFile`, `access`, ...). | ||
|
mcollina marked this conversation as resolved.
Outdated
|
||
|
|
||
| ### Capability flags | ||
|
|
||
| * `provider.readonly` {boolean} **Default:** `false`. | ||
| * `provider.supportsSymlinks` {boolean} **Default:** `false`. | ||
| * `provider.supportsWatch` {boolean} **Default:** `false`. | ||
|
|
||
| ### Creating custom providers | ||
|
|
||
| ```cjs | ||
| const { VirtualProvider } = require('node:vfs'); | ||
|
|
||
| class StaticProvider extends VirtualProvider { | ||
| get readonly() { return true; } | ||
|
|
||
| statSync(path) { /* ... */ } | ||
| openSync(path, flags) { /* ... */ } | ||
| readdirSync(path, options) { /* ... */ } | ||
| // ... | ||
| } | ||
| ``` | ||
|
|
||
| The base class throws `ERR_METHOD_NOT_IMPLEMENTED` for any primitive | ||
| that has not been overridden, and rejects writes from a `readonly` | ||
| provider with `EROFS`. | ||
|
|
||
| ## Class: `MemoryProvider` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| The default in-memory provider. Stores files, directories, and symbolic | ||
| links in a `Map`-backed tree, supports symlinks (`supportsSymlinks === | ||
| true`), and supports watching (`supportsWatch === true`). | ||
|
|
||
| ### `memoryProvider.setReadOnly()` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| Locks the provider into read-only mode. Subsequent writes through any | ||
| [`VirtualFileSystem`][] using this provider throw `EROFS`. There is no | ||
| way to revert the provider to writable. | ||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
|
|
||
| const provider = new vfs.MemoryProvider(); | ||
| const myVfs = vfs.create(provider); | ||
| myVfs.writeFileSync('/seed.txt', 'initial'); | ||
|
|
||
| provider.setReadOnly(); | ||
|
|
||
| myVfs.writeFileSync('/x.txt', 'fail'); // throws EROFS | ||
| ``` | ||
|
|
||
| ## Class: `RealFSProvider` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| A provider that wraps a real file system directory and exposes its | ||
|
mcollina marked this conversation as resolved.
Outdated
|
||
| contents through the VFS API. All VFS paths are resolved relative to | ||
| the root and verified to stay inside it; symbolic links resolving | ||
| outside the root are rejected. | ||
|
|
||
| ### `new RealFSProvider(rootPath)` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * `rootPath` {string} The absolute file system path to use as the root. | ||
| Must be a non-empty string. | ||
|
mcollina marked this conversation as resolved.
Outdated
|
||
|
|
||
| ```cjs | ||
| const vfs = require('node:vfs'); | ||
|
|
||
| const realVfs = vfs.create(new vfs.RealFSProvider('/tmp/sandbox')); | ||
| realVfs.writeFileSync('/file.txt', 'hello'); // writes /tmp/sandbox/file.txt | ||
| ``` | ||
|
|
||
| ### `realFSProvider.rootPath` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| * {string} | ||
|
|
||
| The resolved absolute path used as the root. | ||
|
|
||
| ## Implementation details | ||
|
|
||
| ### `Stats` objects | ||
|
|
||
| VFS `Stats` objects are real instances of [`fs.Stats`][] (or | ||
| [`fs.BigIntStats`][] when `{ bigint: true }` is requested). Their | ||
| fields use synthetic but stable values: | ||
|
|
||
| * `dev` is `4085` (the VFS device id). | ||
| * `ino` is monotonically increasing per process. | ||
| * `blksize` is `4096`. | ||
| * `blocks` is `Math.ceil(size / 512)`. | ||
| * Times default to the moment the entry was created/last modified. | ||
|
|
||
| [`MemoryProvider`]: #class-memoryprovider | ||
| [`VirtualFileSystem`]: #class-virtualfilesystem | ||
| [`VirtualProvider`]: #class-virtualprovider | ||
| [`fs.BigIntStats`]: fs.md#class-fsbigintstats | ||
| [`fs.Stats`]: fs.md#class-fsstats | ||
| [`node:fs`]: fs.md | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.