mongodb · BigMarcio · Apr 20, 2026 · Apr 24, 2026 · Apr 24, 2026 · Apr 27, 2026
diff --git a/migration/mongosync_insights/CONFIGURATION.md b/migration/mongosync_insights/CONFIGURATION.md
@@ -55,7 +55,7 @@ All configuration can be set using `export` commands before running the applicat
 
 | Variable | Default | Description |
 |----------|---------|-------------|
-| `MI_ERROR_PATTERNS_FILE` | `lib/error_patterns.json` _(auto-detected)_ | Path to a custom error patterns JSON file used during log analysis to detect common errors (e.g., oplog rollover, timeouts, verifier mismatches) |
+| `MI_ERROR_PATTERNS_FILE` | `lib/error_patterns.json` _(auto-detected)_ | Path to a custom error patterns JSON file used during log analysis to detect common errors (e.g., oplog rollover, timeouts, verifier mismatches). Each entry may include an optional `recommendation` string, shown in the Errors tab when a line matches that pattern. |
 
 ### UI Customization
 
@@ -77,7 +77,7 @@ All configuration can be set using `export` commands before running the applicat
 
 | Variable | Default | Description |
 |----------|---------|-------------|
-| `MI_SECURE_COOKIES` | `true` | Enable secure cookies (requires HTTPS) |
+| `MI_SECURE_COOKIES` | mirrors `MI_SSL_ENABLED` (`false` when HTTPS is off) | Enable secure cookies (set `true` when using HTTPS) |
 | `MI_SESSION_TIMEOUT` | `3600` | Session timeout in seconds (1 hour default) |
 | `MI_SSL_ENABLED` | `false` | Enable HTTPS/SSL in Flask application |
 | `MI_SSL_CERT` | `/etc/letsencrypt/live/your-domain/fullchain.pem` | Path to SSL certificate file |
@@ -90,7 +90,7 @@ All configuration can be set using `export` commands before running the applicat
 
 ### Connection String Validation
 
-> **Note**: For connection string handling information, see [VALIDATION.md](VALIDATION.md)
+> **Note**: For connection string handling information, see [CONNECTION_STRING.md](CONNECTION_STRING.md)
 
 ---
 
@@ -295,7 +295,7 @@ export MI_LOG_FILE=/var/log/mongosync-insights/insights.log
 
 - **[README.md](README.md)** - Getting started and installation guide
 - **[HTTPS_SETUP.md](HTTPS_SETUP.md)** - Enable HTTPS/SSL for secure deployments
-- **[VALIDATION.md](VALIDATION.md)** - Connection string validation, sanitization, and error handling
+- **[CONNECTION_STRING.md](CONNECTION_STRING.md)** - Connection string formats, security, and troubleshooting
 
 ### License
 

diff --git a/migration/mongosync_insights/VALIDATION.md → ...n/mongosync_insights/CONNECTION_STRING.md b/migration/mongosync_insights/VALIDATION.md → ...n/mongosync_insights/CONNECTION_STRING.md
diff --git a/migration/mongosync_insights/HTTPS_SETUP.md b/migration/mongosync_insights/HTTPS_SETUP.md
@@ -486,7 +486,7 @@ sudo systemctl restart apache2
 | `MI_SSL_KEY` | `/etc/letsencrypt/live/your-domain/privkey.pem` | Path to SSL private key file |
 | `MI_PORT` | `3030` | Port to run the application on (use 443 for HTTPS) |
 | `MI_HOST` | `127.0.0.1` | Host to bind to (use 0.0.0.0 for all interfaces) |
-| `MI_SECURE_COOKIES` | `true` | Enable secure cookies (requires HTTPS) |
+| `MI_SECURE_COOKIES` | mirrors `MI_SSL_ENABLED` (`false` when HTTPS is off) | Enable secure cookies (set `true` when using HTTPS) |
 
 ### Example Configurations
 

diff --git a/migration/mongosync_insights/LOG_VERBOSITY.md b/migration/mongosync_insights/LOG_VERBOSITY.md
@@ -0,0 +1,104 @@
+# Log Verbosity and Plot Coverage
+
+Mongosync Insights reads the structured JSON log lines that mongosync writes to its log files. The amount of data available to plot depends directly on the **verbosity level** used when mongosync was running.
+
+## Verbosity Hierarchy
+
+Mongosync uses a cumulative verbosity system. Each level includes all levels below it:
+
+| Level | Severity order | How to enable |
+|-------|---------------|---------------|
+| `error` | lowest verbosity | Always present |
+| `warn`  | ↑ | Always present |
+| `info`  | ↑ | Default — no flags needed |
+| `debug` | ↑ | `--verbosity 1` |
+| `trace` | highest verbosity | `--verbosity 2` |
+
+To set verbosity when starting mongosync:
+
+```bash
+# default (info and above)
+mongosync --cluster0 "..." --cluster1 "..."
+
+# enable debug logs
+mongosync --cluster0 "..." --cluster1 "..." --verbosity 1
+
+# enable trace logs
+mongosync --cluster0 "..." --cluster1 "..." --verbosity 2
+```
+
+## Effect on Plots and Features
+
+The table below lists every chart and panel in Mongosync Insights, the minimum verbosity required, and the exact log pattern that feeds it.
+
+### Global Migration Metrics
+
+| Chart / Feature | Min. verbosity | Source log pattern |
+|----------------|---------------|--------------------|
+| Mongosync Phases (scatter + table) | `info` (default) | `"Starting ... phase"` \| `"Commit handler called"` |
+| Lag Time (seconds) | `info` (default) | `"Replication progress"` → `lagTimeSeconds` |
+| Est. Source Oplog Time Remaining | `info` (default) | `"Replication progress"` → `estimatedOplogTimeRemaining` |
+| Ping Latency — src & dst | `info` (default) | `"Operation duration stats"` → `sourcePingLatencyMs` / `destinationPingLatencyMs` |
+| Average Source CRUD Event Rate | `info` (default) | `"Average Source CRUD events rate"` → `srcCRUDEventsPerSec` |
+
+> **Note:** `Average Source CRUD events rate` is only emitted by **standalone mongosync**. It is not logged during Live Import runs.
+
+### Collection Copy Metrics
+
+| Chart / Feature | Min. verbosity | Source log pattern |
+|----------------|---------------|--------------------|
+| Partition Init Progress (time series) | `info` (default) | `"Creating a single/initial partition for..."` |
+| Partition Init Summary — doc count & sampler | `info` (default) | `"Pre-sampling information"` |
+| Partition Init Summary — **partition count & duration** | **`debug` (`--verbosity 1`)** | `"Persisted a new partition after sampling"` |
+| Data Copied Over Time | `info` (default) | `"sent response"` → `body.progress.collectionCopy.estimatedCopiedBytes` |
+| Estimated Total & Copied (bar) | `info` (default) | `"sent response"` → `body.progress.collectionCopy` |
+| Partitions Copied (time series + bar) | `info` (default) | `"Completed writing X / Y partitions to destination cluster"` |
+
+### CEA Metrics
+
+| Chart / Feature | Min. verbosity | Source log pattern |
+|----------------|---------------|--------------------|
+| Change Events Applied | `info` (default) | `"Replication progress"` → `totalEventsApplied` |
+| Events Rate per Second | `info` (default) | `"Replication progress"` → `eventApplicationRatePerSecond` |
+| CEA Source Read — avg / max / ops | `info` (default) | `"Operation duration stats"` → `CEASourceRead` |
+| CEA Destination Write — avg / max / ops | `info` (default) | `"Operation duration stats"` → `CEADestinationWrite` |
+
+### Indexes Metrics
+
+| Chart / Feature | Min. verbosity | Source log pattern |
+|----------------|---------------|--------------------|
+| Index Built Over Time | `info` (default) | `"sent response"` → `body.progress.indexBuilding.indexesBuilt` |
+| Total and Index Built (bar) | `info` (default) | `"sent response"` → `body.progress.indexBuilding.totalIndexesToBuild` |
+
+### Verifier Metrics
+
+| Chart / Feature | Min. verbosity | Source log pattern |
+|----------------|---------------|--------------------|
+| Source Verifier Lag Time | `warn` (automatic) | Field `verifierSrcLagTimeSeconds` present |
+| Destination Verifier Lag Time | `warn` (automatic) | Field `verifierDstLagTimeSeconds` present |
+
+> **Note:** Verifier lag lines are emitted by mongosync at `warn` level independently of the `--verbosity` flag. No extra configuration is needed — they appear automatically whenever the live verifier is active.
+
+### Info Tabs (non-chart panels)
+
+| Panel | Min. verbosity | Source log pattern |
+|-------|---------------|--------------------|
+| Version Info | `info` (default) | `"Version info"` |
+| Start Options | `info` (default) | `"Received request"` filtered by `uri=/api/v1/start` |
+| Mongosync Options | `info` (default) | `"Mongosync Options"` |
+| Hidden Flags | `info` (default) | `"Mongosync HiddenFlags"` |
+| Natural Order Collections | `info` (default) | `reason` field → `"Selected for natural order collection reads"` |
+
+> **Note:** Version Info, Mongosync Options, and Hidden Flags are startup log lines. If you upload a **rotated or partial log file** that was captured mid-migration (i.e., after mongosync already rotated its initial log), these panels will be empty. The data is present in the earlier rotated log files, not the current one.
+
+## Summary
+
+| Verbosity | Charts and panels available |
+|-----------|----------------------------|
+| Default (`info`, no flags) | Everything **except** Partition Init partition count & duration columns |
+| `--verbosity 1` (`debug`) | Full coverage, including Partition Init partition count & duration |
+| `--verbosity 2` (`trace`) | No additional plots currently — `trace`-level lines are not yet extracted by Mongosync Insights |
+
+## Recommendation
+
+For a complete analysis, capture mongosync logs with at least `--verbosity 1`. This adds one additional log line per partition created (`"Persisted a new partition after sampling"`), which is low volume and has negligible performance impact. All other charts work at the default verbosity level.
diff --git a/migration/mongosync_insights/PACKAGING.md b/migration/mongosync_insights/PACKAGING.md
@@ -43,7 +43,7 @@ dist/mongosync-insights-<version>-1.x86_64.rpm
 Copy the RPM to the target machine (USB, SCP, etc.) and install:
 
 ```bash
-sudo rpm -i mongosync-insights-0.8.0.18-1.x86_64.rpm
+sudo rpm -i mongosync-insights-0.8.2.8-1.x86_64.rpm
 ```
 
 No internet access is required. No additional packages need to be installed.