RdmpCohortBuildHealthBoardBreakdown plugin package (9.2.3)

RdmpCohortBuildHealthBoardBreakdown/INSTALL.md

@@ -0,0 +1,56 @@
+# RdmpCohortBuildHealthBoardBreakdown plugin (RDMP 9.2.3)
+
+Reproduces the Cohort Builder's per-set / per-container count tree (the FinalCount and cumulative
+running totals shown as UNION/INTERSECT/EXCEPT are applied) **split by Scottish health board**, plus
+an unfiltered baseline. Saved as a long-format CSV.
+
+Built against the **released RDMP 9.2.3**. Do not use on a different major.minor RDMP.
+
+## How it works (cache-only, cross-server safe)
+
+It builds the cohort **once** (populating the query cache), then recomposes every count point from the
+cached per-set identifier tables and splits each by `SHARE_Demography.Region` with one GROUP BY per
+node. It never re-runs the source catalogues per board, and never touches the source servers after the
+single build — only the query-cache server (which is why the demography catalogue must be on the same
+server as the query cache).
+
+## Requirements
+
+- The cohort identification configuration must have a **query caching server** configured (the
+  breakdown works only on cached results; it refuses otherwise).
+- `SHARE_Demography` (with a `Region` health-board cipher column and a CHI IsExtractionIdentifier
+  column) must be on the **same SQL server as the query cache** (the command checks and refuses if not).
+
+## Install
+
+**GUI:** RDMP desktop → Plugins node → *Add Plugin* (or drag `RdmpCohortBuildHealthBoardBreakdown.rdmp`
+onto it) → restart RDMP. **Or** drop the `.rdmp` next to `rdmp.exe` /
+`ResearchDataManagementPlatform.exe`.
+
+Confirm (CLI): `rdmp.exe cmd ListSupportedCommands` lists `ExportCohortBuildHealthBoardBreakdown`.
+
+## Use
+
+**GUI:** right-click a Cohort Identification Configuration → *Export ... Build Health Board Breakdown*
+→ choose a CSV path.
+
+**CLI:**
+```
+rdmp.exe cmd ExportCohortBuildHealthBoardBreakdown CohortIdentificationConfiguration:<id> out.csv "SHARE_Demography" "Region"
+```
+Args after the CIC are optional (defaults: `<cic>-build-healthboard.csv`, `SHARE_Demography`, `Region`).
+
+## Output (long format, board-grouped)
+
+Columns: `Board, Node, Order, Type, Name, Container, SetOperation, FinalCount, CumulativeCount`. The
+`Unfiltered` tree first (RDMP's own numbers), then each health board's full tree (boards partition the
+cohort, 1 patient ↔ 1 board), then an `Unknown` board (patients with no / unmapped region). `FinalCount`
+is the node's own count; `CumulativeCount` is the running total within the parent container (blank for
+the first child, as in the UI). Boards (+ Unknown) reconcile to the unfiltered total at every node.
+
+## Validation
+
+Verified end-to-end on a deterministic synthetic fixture (top EXCEPT over an inclusion INTERSECT minus
+four exclusion sets, cohort partitioned across 3 boards): every national and per-board FinalCount and
+cumulative was asserted cell-by-cell, the unfiltered column equals RDMP's own CohortCompiler counts, and
+the boards sum to national at every node.

RdmpCohortBuildHealthBoardBreakdown/README.md

@@ -0,0 +1,42 @@
+# RdmpCohortBuildHealthBoardBreakdown (RDMP 9.2.3 plugin)
+
+Reproduces the Cohort Builder's per-set / per-container count tree (the `FinalCount` and cumulative
+running totals shown as UNION/INTERSECT/EXCEPT are applied) **split by Scottish health board**, plus an
+unfiltered national total, and writes it to a wide CSV.
+
+This folder is a self-contained package: the ready-to-install plugin, install/usage notes, and the source.
+
+## Contents
+
+| Path | What it is |
+|---|---|
+| `RdmpCohortBuildHealthBoardBreakdown.rdmp` | the built plugin (drop into RDMP / add via the Plugins node) |
+| `INSTALL.md` | install + usage (GUI right-click and CLI) |
+| `src/` | plugin source (command, report, health-board lookup, UI hook, csproj, nuspec) |
+
+## How it works (in one paragraph)
+
+It builds the national cohort once (which populates RDMP's query cache), then recomposes every count
+point purely from the cached per-set identifier tables and splits each by `SHARE_Demography.Region`
+with one `GROUP BY` per node, all boards at once. No per-board rebuild, and no hits on the source
+catalogues after the single build (so it is cross-server safe). Requires a query-caching server, and
+the demography catalogue on the same server as the cache.
+
+## Output
+
+Wide CSV: one row per count point (name once), a `Metric` column (Final + Cumulative), a `Total`
+column (RDMP's national number), one column per Scottish board, then `Other` (present non-Scottish /
+unmapped region codes) and `NotKnown` (not in demography / null region). The column header is repeated
+above a `% of final cohort` row. Boards + Other + NotKnown reconcile to Total on every row.
+
+## Validation
+
+Verified end-to-end against a deterministic synthetic fixture (top EXCEPT over an inclusion INTERSECT
+minus four exclusion sets, the cohort partitioned across 3 boards): every national and per-board
+`FinalCount` / cumulative is asserted cell-by-cell, the unfiltered column equals RDMP's own
+`CohortCompiler` counts, and the boards sum to national at every node.
+
+## Build from source (optional)
+
+`src/` builds against RDMP 9.2.3 (`Rdmp.Core`, `Private=false`). Package the resulting DLL + nuspec into
+a `.rdmp` zip (`<nuspec>` at root, DLL under `lib/net10.0/`).

RdmpCohortBuildHealthBoardBreakdown/src/CohortBuildHealthBoardBreakdownPluginUserInterface.cs

@@ -0,0 +1,24 @@
+// Surfaces the cohort build health board breakdown in the RDMP desktop GUI (right-click a cohort
+// identification configuration). The same command class is auto-discovered for the CLI
+// (`rdmp cmd ExportCohortBuildHealthBoardBreakdown`), so one definition serves both.
+
+using System.Collections.Generic;
+using Rdmp.Core;
+using Rdmp.Core.CommandExecution;
+using Rdmp.Core.CommandExecution.AtomicCommands;
+using Rdmp.Core.Curation.Data.Cohort;
+
+namespace RdmpCohortBuildHealthBoardBreakdown;
+
+public class CohortBuildHealthBoardBreakdownPluginUserInterface : PluginUserInterface
+{
+    public CohortBuildHealthBoardBreakdownPluginUserInterface(IBasicActivateItems itemActivator) : base(itemActivator)
+    {
+    }
+
+    public override IEnumerable<IAtomicCommand> GetAdditionalRightClickMenuItems(object o)
+    {
+        if (o is CohortIdentificationConfiguration cic)
+            yield return new ExecuteCommandExportCohortBuildHealthBoardBreakdown(BasicActivator, cic);
+    }
+}

RdmpCohortBuildHealthBoardBreakdown/src/CohortBuildHealthBoardBreakdownReport.cs

@@ -0,0 +1,174 @@
+// Copyright (c) The University of Dundee 2024-2024
+// This file is part of the Research Data Management Platform (RDMP).
+// RDMP is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+// RDMP is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+// You should have received a copy of the GNU General Public License along with RDMP. If not, see <https://www.gnu.org/licenses/>.
+
+using System.Collections.Generic;
+using System.Globalization;
+using System.IO;
+using System.Linq;
+using System.Text;
+
+namespace Rdmp.Core.CohortCreation;
+
+/// <summary>
+/// Projects a cohort build's count tree (the per-set / per-container <c>FinalCount</c> and cumulative
+/// running totals shown in the Cohort Builder) split by health board into a WIDE CSV: one row per
+/// (count-point × metric), the container/set name written once, a <c>Total</c> column (RDMP's own
+/// national count), one column per Scottish health board, an <c>Other</c> column (present non-Scottish /
+/// unmapped region codes) and a <c>NotKnown</c> residual (patients not in demography / NULL region).
+/// A bottom <c>% of final cohort</c> row gives each board's share of the final national cohort.
+/// Boards + Other + NotKnown reconcile to Total on every row.
+/// </summary>
+public static class CohortBuildHealthBoardBreakdownReport
+{
+    public const string OtherColumn = "Other";
+    public const string NotKnownColumn = "NotKnown";
+    public const string PercentMetric = "% of final cohort";
+
+    /// <summary>One count point of the build tree with its per-region counts (known boards only).</summary>
+    public sealed class NodeBreakdown
+    {
+        public int Seq { get; init; }
+        public string Type { get; init; } = "";
+        public string Name { get; init; } = "";
+
+        /// <summary>Parent container name (empty for the root).</summary>
+        public string Container { get; init; } = "";
+
+        public string SetOperation { get; init; } = "";
+        public int DisplayOrder { get; init; }
+
+        /// <summary>RDMP's own count for this node (the unfiltered/national total).</summary>
+        public int FinalUnfiltered { get; init; }
+
+        /// <summary>RDMP's own cumulative within the parent container; null if not applicable.</summary>
+        public int? CumulativeUnfiltered { get; init; }
+
+        /// <summary>Region cipher → final count (every present code; GROUP BY Region result).</summary>
+        public IReadOnlyDictionary<string, int> FinalByRegion { get; init; } = new Dictionary<string, int>();
+
+        /// <summary>Region cipher → cumulative count; null when this node has no cumulative.</summary>
+        public IReadOnlyDictionary<string, int> CumulativeByRegion { get; init; }
+    }
+
+    /// <summary>The Total / per-board / Other / NotKnown counts for one node+metric.</summary>
+    public sealed class Buckets
+    {
+        public int Total { get; init; }
+
+        /// <summary>Region cipher → count (mapped Scottish boards only).</summary>
+        public IReadOnlyDictionary<string, int> Boards { get; init; } = new Dictionary<string, int>();
+
+        /// <summary>Sum of present region codes that are NOT one of the 15 Scottish boards.</summary>
+        public int Other { get; init; }
+
+        /// <summary>Total − boards − Other = not-in-demography + NULL region.</summary>
+        public int NotKnown { get; init; }
+    }
+
+    /// <summary>
+    /// Splits one node's region counts into Total / mapped-boards / Other / NotKnown. <paramref name="byRegion"/>
+    /// is the GROUP BY Region result (every present code); <paramref name="total"/> is RDMP's own count.
+    /// </summary>
+    public static Buckets Split(int total, IReadOnlyDictionary<string, int> byRegion)
+    {
+        var boards = new Dictionary<string, int>(System.StringComparer.OrdinalIgnoreCase);
+        var other = 0;
+        foreach (var (code, n) in byRegion)
+            if (HealthBoardLookup.Resolve(code).Node == HealthBoardLookup.UnknownNode)
+                other += n; // present but not a Scottish board (non-Scottish / unmapped)
+            else
+                boards[code] = n;
+
+        return new Buckets
+        {
+            Total = total,
+            Boards = boards,
+            Other = other,
+            NotKnown = total - boards.Values.Sum() - other
+        };
+    }
+
+    /// <summary>The ordered mapped boards that appear anywhere (column order: node then name).</summary>
+    private static List<HealthBoard> BoardColumns(IEnumerable<NodeBreakdown> nodes) =>
+        nodes
+            .SelectMany(n => n.FinalByRegion.Keys.Concat(n.CumulativeByRegion?.Keys ?? Enumerable.Empty<string>()))
+            .Select(HealthBoardLookup.Resolve)
+            .Where(b => b.Node != HealthBoardLookup.UnknownNode)
+            .GroupBy(b => b.Region, System.StringComparer.OrdinalIgnoreCase)
+            .Select(g => g.First())
+            .OrderBy(b => b.Node, System.StringComparer.OrdinalIgnoreCase)
+            .ThenBy(b => b.Name, System.StringComparer.OrdinalIgnoreCase)
+            .ToList();
+
+    /// <summary>Builds the wide CSV (data rows per node+metric, then a % of final cohort row).</summary>
+    public static string ToCsv(IReadOnlyList<NodeBreakdown> nodes)
+    {
+        var ordered = nodes.OrderBy(n => n.Seq).ToList();
+        var boards = BoardColumns(ordered);
+
+        var header = new List<string> { "Order", "Type", "Name", "Container", "SetOperation", "Metric", "Total" };
+        header.AddRange(boards.Select(b => b.Name));
+        header.Add(OtherColumn);
+        header.Add(NotKnownColumn);
+
+        var sb = new StringBuilder();
+        sb.AppendLine(string.Join(",", header.Select(Escape)));
+
+        foreach (var n in ordered)
+        {
+            AppendCountRow(sb, n, boards, "Final", Split(n.FinalUnfiltered, n.FinalByRegion));
+            if (n.CumulativeUnfiltered.HasValue && n.CumulativeByRegion != null)
+                AppendCountRow(sb, n, boards, "Cumulative",
+                    Split(n.CumulativeUnfiltered.Value, n.CumulativeByRegion));
+        }
+
+        // bottom: % of final cohort (root node's Final), after a blank separator
+        var root = ordered.FirstOrDefault(n => string.IsNullOrEmpty(n.Container)) ?? ordered.FirstOrDefault();
+        if (root != null && root.FinalUnfiltered > 0)
+        {
+            sb.AppendLine();
+            sb.AppendLine(string.Join(",", header.Select(Escape))); // repeat header so % aligns to each board
+            var b = Split(root.FinalUnfiltered, root.FinalByRegion);
+            double Pct(int v) => v * 100.0 / b.Total;
+            var cells = new List<string> { "", "", PercentMetric, "", "", PercentMetric, Fmt(100.0) };
+            cells.AddRange(boards.Select(bd => Fmt(Pct(b.Boards.TryGetValue(bd.Region, out var v) ? v : 0))));
+            cells.Add(Fmt(Pct(b.Other)));
+            cells.Add(Fmt(Pct(b.NotKnown)));
+            sb.AppendLine(string.Join(",", cells.Select(Escape)));
+        }
+
+        return sb.ToString();
+    }
+
+    private static void AppendCountRow(StringBuilder sb, NodeBreakdown n, List<HealthBoard> boards,
+        string metric, Buckets b)
+    {
+        var cells = new List<string>
+        {
+            n.DisplayOrder.ToString(CultureInfo.InvariantCulture),
+            n.Type, n.Name, n.Container, n.SetOperation, metric,
+            b.Total.ToString(CultureInfo.InvariantCulture)
+        };
+        cells.AddRange(boards.Select(bd =>
+            (b.Boards.TryGetValue(bd.Region, out var v) ? v : 0).ToString(CultureInfo.InvariantCulture)));
+        cells.Add(b.Other.ToString(CultureInfo.InvariantCulture));
+        cells.Add(b.NotKnown.ToString(CultureInfo.InvariantCulture));
+        sb.AppendLine(string.Join(",", cells.Select(Escape)));
+    }
+
+    public static void WriteCsv(string path, IReadOnlyList<NodeBreakdown> nodes) =>
+        File.WriteAllText(path, ToCsv(nodes));
+
+    private static string Fmt(double d) => d.ToString("0.0", CultureInfo.InvariantCulture);
+
+    private static string Escape(string field)
+    {
+        field ??= "";
+        if (field.Contains(',') || field.Contains('"') || field.Contains('\n') || field.Contains('\r'))
+            return $"\"{field.Replace("\"", "\"\"")}\"";
+        return field;
+    }
+}