Pooria/1024 railtracks folder

docs/advanced_usage/config.md

@@ -22,7 +22,7 @@ Configuration parameters follow a specific precedence order, allowing you to ove
 - **`context`** (`Dict[str, Any]`): Global context variables for execution
 - **`broadcast_callback`** (`Callable`): Callback function for broadcast messages
 - **`prompt_injection`** (`bool`): Automatically inject prompts from context variables
-- **`save_state`** (`bool`): Save execution state to `.railtracks` directory
+- **`save_state`** (`bool`): Save execution state to the `.railtracks` data directory (see [Data directory resolution](#data-directory-railtracks) below)
 
 ## Default Values
 
@@ -146,6 +146,19 @@ with rt.session(
         "Debug this workflow",
     )
 ```
+## Data directory (`.railtracks`)
+
+When `save_state` is enabled, railtracks resolves the `.railtracks` data directory using the following priority order:
+
+1. **`RAILTRACKS_HOME` environment variable** — set this to the **parent directory** where `.railtracks` should live. Useful for CI environments or shared storage locations.
+   ```bash
+   export RAILTRACKS_HOME=/path/to/project-root   # .railtracks is created inside here
+   ```
+2. **Upward directory traversal** — walks up from the current working directory until it finds an existing `.railtracks` folder. This means running scripts from any subdirectory of your project will always resolve to the same directory, as long as you have run `railtracks init` from the project root once.
+3. **Fallback to `cwd()`** — if no `.railtracks` directory is found in any parent, one is created in the current working directory. A warning is emitted to prompt you to run `railtracks init` from the intended project root.
+
+The same resolution logic applies to sessions, evaluations, and the visualizer, so all data always lands in one consistent location.
+
 ## Important Notes
 
 - `rt.set_config()` must be called **before** any agent execution

docs/evaluations/quickstart.md

@@ -1,6 +1,6 @@
 # Evaluations
 
-Evaluations in `railtracks` are a useful tool to analyze, aggregate, and finally visualize agent runs invoked previously. Sessions are automatically stored in `.railtracks/data/sessions`, so evaluations can be run at any time after invoking your agent.
+Evaluations in `railtracks` are a useful tool to analyze, aggregate, and finally visualize agent runs invoked previously. Sessions are automatically stored in `.railtracks/data/sessions`, so evaluations can be run at any time after invoking your agent. Railtracks locates the `.railtracks` directory by walking up from your current working directory, so the right data is found regardless of which subdirectory you run from.
 
 ## Evaluation Definition
 ```python

docs/evaluations/visualization.md

@@ -1,6 +1,6 @@
 # Visualization
 
-After running evaluations, results are automatically saved to `.railtracks/data/evaluations`. The built-in visualizer lets you explore these results locally with no sign up required.
+After running evaluations, results are automatically saved to `.railtracks/data/evaluations`. Railtracks locates this directory by walking up from your current working directory, so results are always written to the same place regardless of where you run your scripts from. The built-in visualizer lets you explore these results locally with no sign up required.
 
 !!! tip "Setting up the visualizer"
     See [Observability → Visualization](../observability/agenthub/local.md) for installation and setup instructions.

docs/observability/agenthub/local.md

@@ -23,7 +23,16 @@ Railtracks comes with a built-in visualization tool that runs locally with **no
     railtracks viz
     ```
 
-This will create a `.railtracks` directory in your current working directory setting up the web app in your web browser
+This will create a `.railtracks` directory at your project root and open the web app in your browser. Once initialised, railtracks will find that directory automatically, even if you run your agents from a subdirectory, by walking up the folder tree until it locates `.railtracks`.
+
+!!! tip "Running from multiple directories?"
+    Run `railtracks init` once from your project root (the same level as your `.git` folder). All subsequent agent runs across the project will resolve to that single `.railtracks` directory regardless of which subdirectory they are launched from.
+
+    If you need a fixed location outside your project (e.g. a shared drive or CI environment), set the `RAILTRACKS_HOME` environment variable to the **parent directory** where `.railtracks` should live:
+    ```bash
+    export RAILTRACKS_HOME=/path/to/my/project   # .railtracks is created inside here
+    ```
+    `RAILTRACKS_HOME` always takes priority over directory traversal.
 
 
 <div class="rt-video-container">

packages/railtracks/src/railtracks/_session.py

@@ -1,17 +1,16 @@
 import inspect
 import json
-import os
 import time
 import uuid
 import warnings
 from functools import wraps
-from pathlib import Path
 from typing import Any, Callable, Coroutine, Dict, ParamSpec, Tuple, TypeVar, overload
 
 from railtracks.exceptions.messages.exception_messages import (
     ExceptionMessageKey,
     get_message,
 )
+from railtracks.paths import resolve_railtracks_home
 
 from .context.central import (
     delete_globals,
@@ -167,8 +166,7 @@ def __enter__(self):
     def __exit__(self, exc_type, exc_val, exc_tb):
         if self.executor_config.save_state:
             try:
-                railtracks_home = os.environ.get("RAILTRACKS_HOME", ".railtracks")
-                railtracks_dir = Path(railtracks_home)
+                railtracks_dir = resolve_railtracks_home()
                 sessions_dir = railtracks_dir / "data" / "sessions"
                 sessions_dir.mkdir(
                     parents=True, exist_ok=True

packages/railtracks/src/railtracks/cli/__init__.py

@@ -31,9 +31,10 @@
 
 from colorama import Fore, Style
 
+from railtracks.paths import resolve_railtracks_home
+
 from .constants import (
     DEFAULT_PORT,
-    UI_VERSION_FILE,
     cli_directory,
     cli_name,
     latest_ui_url,
@@ -111,32 +112,34 @@ def is_port_in_use(port):
 
 def create_railtracks_dir():
     """Create .railtracks directory if it doesn't exist and add to .gitignore"""
-    railtracks_dir = Path(cli_directory)
+    railtracks_dir = resolve_railtracks_home()
     if not railtracks_dir.exists():
         print_status(f"Creating {cli_directory} directory...")
-        railtracks_dir.mkdir(exist_ok=True)
-        print_success(f"Created {cli_directory} directory")
-
-    gitignore_path = Path(".gitignore")
-    if gitignore_path.exists():
-        with open(gitignore_path) as f:
-            gitignore_content = f.read()
-
-        if cli_directory not in gitignore_content:
-            print_status(f"Adding {cli_directory} to .gitignore...")
-            with open(gitignore_path, "a") as f:
-                f.write(f"\n{cli_directory}\n")
-            print_success(f"Added {cli_directory} to .gitignore")
+        railtracks_dir.mkdir(parents=True, exist_ok=True)
+        print_success(f"Created {railtracks_dir}")
+
+        gitignore_path = railtracks_dir.parent / ".gitignore"
+        if gitignore_path.exists():
+            with open(gitignore_path) as f:
+                gitignore_content = f.read()
+
+            if cli_directory not in gitignore_content:
+                print_status(f"Adding {cli_directory} to .gitignore...")
+                with open(gitignore_path, "a") as f:
+                    f.write(f"\n{cli_directory}\n")
+                print_success(f"Added {cli_directory} to .gitignore")
+        else:
+            print_status("Creating .gitignore file...")
+            with open(gitignore_path, "w") as f:
+                f.write(f"{cli_directory}\n")
+            print_success(f"Created .gitignore with {cli_directory}")
     else:
-        print_status("Creating .gitignore file...")
-        with open(gitignore_path, "w") as f:
-            f.write(f"{cli_directory}\n")
-        print_success(f"Created .gitignore with {cli_directory}")
+        print_status(f"Using existing {railtracks_dir}")
 
 
 def get_stored_ui_version():
     """Get the stored UI version (ETag) from disk"""
-    version_file = Path(UI_VERSION_FILE)
+    version_file = resolve_railtracks_home() / ".ui_version"
     try:
         if version_file.exists():
             return version_file.read_text().strip()
@@ -147,7 +150,7 @@ def get_stored_ui_version():
 
 def save_ui_version(version: str):
     """Save the UI version (ETag) to disk"""
-    version_file = Path(UI_VERSION_FILE)
+    version_file = resolve_railtracks_home() / ".ui_version"
     try:
         version_file.write_text(version)
     except Exception:
@@ -177,7 +180,7 @@ def check_for_ui_update():
 def download_and_extract_ui():
     """Download the latest frontend UI and extract it to .railtracks/ui"""
     ui_url = latest_ui_url
-    ui_dir = Path(f"{cli_directory}/ui")
+    ui_dir = resolve_railtracks_home() / "ui"
 
     print_status("Downloading latest frontend UI...")
 

packages/railtracks/src/railtracks/cli/constants.py

@@ -2,7 +2,6 @@
 
 cli_name = "railtracks"
 cli_directory = ".railtracks"
-UI_VERSION_FILE = f"{cli_directory}/.ui_version"
 DEFAULT_PORT = 3030
 
 # TODO: Once we are releasing to PyPi change this to the release asset instead

packages/railtracks/src/railtracks/cli/viz_server.py

@@ -10,15 +10,17 @@
 from fastapi import FastAPI
 from fastapi.responses import FileResponse, JSONResponse
 
-from .constants import DEFAULT_PORT, cli_directory
+from railtracks.paths import resolve_railtracks_home
+
+from .constants import DEFAULT_PORT
 from .io import print_error, print_status, print_success, print_warning
 
 app = FastAPI()
 
 
 def get_railtracks_dir() -> Path:
     """Get the .railtracks directory path"""
-    return Path(cli_directory)
+    return resolve_railtracks_home()
 
 
 def get_data_dir(subdir: str) -> Path:
@@ -93,7 +95,7 @@ async def serve_ui_or_404(full_path: str):
     if full_path.startswith("api/"):
         return JSONResponse(content={"error": "Not Found"}, status_code=404)
 
-    ui_dir = Path(f"{cli_directory}/ui")
+    ui_dir = get_railtracks_dir() / "ui"
     ui_file = ui_dir / full_path
     if ui_file.exists() and ui_file.is_file():
         return FileResponse(str(ui_file))
@@ -116,7 +118,7 @@ def start(self):
         self.running = True
 
         print_success(f"🚀 railtracks server running at http://localhost:{self.port}")
-        print_status(f"📁 Serving files from: {cli_directory}/ui/")
+        print_status(f"📁 Serving files from: {get_railtracks_dir() / 'ui'}")
         print_status("📋 API endpoints:")
         print_status("   GET  /api/evaluations - Get all evaluation JSON files")
         print_status("   GET  /api/sessions - Get all session JSON files")

packages/railtracks/src/railtracks/evaluations/utils.py

@@ -1,10 +1,9 @@
 import json
-from pathlib import Path
 from typing import Any
 
-from .result import EvaluationResult
+from railtracks.paths import resolve_railtracks_home
 
-EVALS_DIR = Path(".railtracks/data/evaluations")
+from .result import EvaluationResult
 
 
 def payload(evaluation_result: EvaluationResult) -> dict[str, Any]:
@@ -14,8 +13,9 @@ def payload(evaluation_result: EvaluationResult) -> dict[str, Any]:
 
 def save(results: list[EvaluationResult]):
     """Save evaluation results to disk."""
+    evals_dir = resolve_railtracks_home() / "data" / "evaluations"
     for result in results:
-        fp = EVALS_DIR / f"{result.evaluation_id}.json"
+        fp = evals_dir / f"{result.evaluation_id}.json"
         fp.parent.mkdir(parents=True, exist_ok=True)
         if fp.exists():
             raise Exception(

packages/railtracks/src/railtracks/paths.py

@@ -0,0 +1,34 @@
+import os
+import warnings
+from pathlib import Path
+
+_DIRNAME = ".railtracks"
+
+
+def resolve_railtracks_home() -> Path:
+    """Return the .railtracks directory path.
+
+    Resolution order:
+    1. RAILTRACKS_HOME env var — .railtracks is created inside this directory.
+    2. Walk up from cwd() looking for an existing .railtracks directory.
+    3. Fall back to cwd()/.railtracks with a UserWarning.
+    """
+    env = os.environ.get("RAILTRACKS_HOME")
+    if env:
+        return Path(env) / _DIRNAME
+
+    current = Path.cwd()
+    for directory in [current, *current.parents]:
+        candidate = directory / _DIRNAME
+        if candidate.is_dir():
+            return candidate
+
+    fallback = current / _DIRNAME
+    warnings.warn(
+        f"No {_DIRNAME!r} directory found in '{current}' or any parent directory. "
+        f"Data will be written to '{fallback}'. "
+        f"Run 'railtracks init' from your project root to set a permanent location.",
+        UserWarning,
+        stacklevel=2,
+    )
+    return fallback

packages/railtracks/tests/conftest.py

@@ -255,13 +255,19 @@ def disable_persistence_for_tests():
 def allow_persistence():
     """
     Fixture to allow session persistence for specific tests that need to verify persistence.
-
+
+    Also unsets RAILTRACKS_HOME so that tests relying on monkeypatch.chdir are not
+    redirected to an externally configured directory.
+
     Usage:
         def test_something(allow_persistence):
             # Session persistence will be enabled for this test
             pass
     """
+    previous_home = os.environ.pop("RAILTRACKS_HOME", None)
     os.environ["RAILTRACKS_ALLOW_PERSISTENCE"] = "1"
     yield
     os.environ.pop("RAILTRACKS_ALLOW_PERSISTENCE", None)
+    if previous_home is not None:
+        os.environ["RAILTRACKS_HOME"] = previous_home
 

packages/railtracks/tests/unit_tests/cli/test_cli.py

@@ -71,12 +71,15 @@ def setUp(self):
         """Set up temporary directory for testing"""
         self.test_dir = tempfile.mkdtemp()
         self.original_cwd = os.getcwd()
+        self._original_railtracks_home = os.environ.pop("RAILTRACKS_HOME", None)
         os.chdir(self.test_dir)
 
     def tearDown(self):
         """Clean up temporary directory"""
         os.chdir(self.original_cwd)
         shutil.rmtree(self.test_dir)
+        if self._original_railtracks_home is not None:
+            os.environ["RAILTRACKS_HOME"] = self._original_railtracks_home
 
     @patch('railtracks.cli.print_status')
     @patch('railtracks.cli.print_success')
@@ -166,11 +169,13 @@ def setUp(self):
         """Set up test environment"""
         self.test_dir = tempfile.mkdtemp()
         self.original_cwd = os.getcwd()
+        self._original_railtracks_home = os.environ.pop("RAILTRACKS_HOME", None)
         os.chdir(self.test_dir)
 
         # Create .railtracks directory
-        railtracks_dir = Path(os.environ.get("RAILTRACKS_HOME", ".railtracks"))
-        railtracks_dir.mkdir()
+        from railtracks.paths import resolve_railtracks_home
+        railtracks_dir = resolve_railtracks_home()
+        railtracks_dir.mkdir(parents=True, exist_ok=True)
 
         # Create test JSON files in root
         self.test_files = {
@@ -192,6 +197,8 @@ def tearDown(self):
         """Clean up test environment"""
         os.chdir(self.original_cwd)
         shutil.rmtree(self.test_dir)
+        if self._original_railtracks_home is not None:
+            os.environ["RAILTRACKS_HOME"] = self._original_railtracks_home
 
     def test_get_evaluations_empty(self):
         """Test /api/evaluations endpoint with no data directory"""
@@ -412,13 +419,16 @@ class TestUIVersionTracking(unittest.TestCase):
     def setUp(self):
         self.test_dir = tempfile.mkdtemp()
         self.original_cwd = os.getcwd()
+        self._original_railtracks_home = os.environ.pop("RAILTRACKS_HOME", None)
         os.chdir(self.test_dir)
         # Create .railtracks dir so the version file path is valid
         Path(".railtracks").mkdir()
 
     def tearDown(self):
         os.chdir(self.original_cwd)
         shutil.rmtree(self.test_dir)
+        if self._original_railtracks_home is not None:
+            os.environ["RAILTRACKS_HOME"] = self._original_railtracks_home
 
     # --- get_stored_ui_version ---
 
@@ -523,14 +533,16 @@ def test_print_update_available_contains_update_command(self, mock_print):
         printed_text = mock_print.call_args[0][0]
         self.assertIn('railtracks update', printed_text)
 
-    # --- UI_VERSION_FILE derivation ---
+    # --- version file location ---
 
-    def test_ui_version_file_derived_from_cli_directory(self):
-        """UI_VERSION_FILE is derived from cli_directory, not hardcoded separately"""
-        import railtracks.cli as cli_module
+    def test_ui_version_file_inside_railtracks_home(self):
+        """Version file is stored inside the resolved railtracks home directory"""
+        from railtracks.paths import resolve_railtracks_home
+        version_file = resolve_railtracks_home() / ".ui_version"
+        save_ui_version("test-etag")
         self.assertTrue(
-            cli_module.UI_VERSION_FILE.startswith(cli_module.cli_directory),
-            "UI_VERSION_FILE should start with cli_directory so they stay in sync",
+            version_file.exists(),
+            f"Version file not found at expected location: {version_file}",
         )
 
     # --- temp file cleanup on failure ---