Skip to content

refactor(table): split the 5.3k-line service.ts god-file into per-concern modules#5069

Merged
waleedlatif1 merged 8 commits into
stagingfrom
refactor/split-table-service
Jun 15, 2026
Merged

refactor(table): split the 5.3k-line service.ts god-file into per-concern modules#5069
waleedlatif1 merged 8 commits into
stagingfrom
refactor/split-table-service

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

Decompose lib/table/service.ts (5,324 lines / 58 exports) into per-concern modules, following the established lib/knowledge/ pattern (subdomain service.ts files, plain exported functions, drizzle inline — no repository/manager layer) while keeping the @/lib/table barrel stable.

service.ts: 5,324 → 651 lines (now the pure table-entity module: root CRUD + the shared withLockedTable/addTableColumnsWithTx primitives). New modules:

  • rows/ordering.ts, rows/executions.ts — leaf foundations (position/order-key, workflow-group result rows)
  • rows/service.ts — row CRUD + query
  • columns/service.ts — schema/column management
  • jobs/service.tstableJobs state machine + export jobs
  • workflow-groups/service.ts — workflow-group management (lazy backfill-runner cycle-breaker preserved)
  • tx.ts, import-data.ts — shared tx helpers; import-job data ops

Built in 7 dependency-ordered, individually-green commits (leaves first).

Type of Change

  • Refactor (no functional change)

Testing

  • Pure code-motion, verbatim bodies. Three independent adversarial auditors confirmed every one of the ~60 moved symbols is byte-identical to origin/staging, no cycles, public surface preserved, all consumers + test mocks repointed correctly, no duplication, no shims.
  • 339 table + consumer tests pass (228 table; copilot/user-table + API routes); typecheck clean; biome clean (52 files); strict check:api-validation + monorepo-boundary pass.
  • Dead helpers carried into the new modules were dropped (countTablescreateTable has its own inline count check; unused buildOrderedRowValues); over-exported file-local symbols restored to private so the public barrel surface is unchanged.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…m service.ts

Move the row position/fractional-ordering internals to rows/ordering.ts,
the row-execution (workflow-group result) internals to rows/executions.ts,
and the shared tx-timeout helpers to tx.ts. Pure code-motion — verbatim
bodies, identical behavior. service.ts: 5324 -> 4442 lines.
Move insert/update/upsert/delete/replace/batch row writes, queryRows/
getRowById reads, and findRowMatches into rows/service.ts. Verbatim
bodies; consumers repointed; @/lib/table barrel re-exports the new module
so callers are unchanged. service.ts: 4442 -> 2788 lines.
…e.ts

Move add/rename/delete column ops and column-type/constraint updates into
columns/service.ts. addTableColumnsWithTx stays in service.ts (table-creation
primitive) to avoid a cycle. Verbatim bodies; barrel re-exports the module.
service.ts: 2788 -> 2149 lines.
…service.ts

Move tableJobs reads/mapping, the job lifecycle state machine, and export-job
queries into jobs/service.ts. service.ts imports latestJob* one-way for table
metadata enrichment (no cycle). Import-data orchestration helpers stay in
service.ts for now. Verbatim bodies. service.ts: 2149 -> 1791 lines.
…ps/service.ts

Move add/update/delete workflow groups + outputs and pruneStale into
workflow-groups/service.ts, preserving the dynamic backfill-runner import
(cycle-breaker). Verbatim bodies; no cycle. service.ts: 1791 -> 851 lines.
Move bulk insert, schema setup, and append/replace import operations into
import-data.ts (consumed by import-runner + import route). service.ts is now
the pure table-entity module (root CRUD + shared lock/column primitives).
Verbatim bodies; no cycle. service.ts: 851 -> 664 lines (5324 at start).
…t-split

Un-export DerivedJobFields/JOB_PROJECTION/mapJobRow (file-local in jobs/service.ts,
were private pre-split) so they no longer leak into the @/lib/table barrel. Remove
dead code carried into the new modules: countTables (createTable does its own inline
count check) and the unused buildOrderedRowValues/OrderedRowValue pair.
@vercel

vercel Bot commented Jun 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jun 15, 2026 9:27pm

Request Review

@cursor

cursor Bot commented Jun 15, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Large surface-area refactor across table CRUD, jobs, imports, and workflow execution paths; risk is import/mock miswiring rather than new logic, but regressions would affect core data and background job flows.

Overview
This refactor decomposes the table service god-file into subdomain modules (mirroring lib/knowledge/): rows/service (CRUD/query), rows/ordering and rows/executions (position keys and workflow exec sidecar), columns/service, jobs/service, import-data, plus a slim service.ts for table definition CRUD and shared withLockedTable / schema helpers.

Call sites across table API routes, v1 APIs, background workers (workflow-column-execution, resume-execution), copilot tools, and runners now import from the specific module instead of a single @/lib/table/service. The @/lib/table barrel re-exports the new services so external imports stay stable; Vitest mocks were split to match (e.g. job routes mock @/lib/table/jobs/service, row routes mock @/lib/table/rows/service).

No intended behavior change—logic is moved verbatim into the new files with dependency ordering preserved (e.g. column ops still use withLockedTable from the root service).

Reviewed by Cursor Bugbot for commit a5a2b7e. Configure here.

@waleedlatif1 waleedlatif1 changed the title refactor(table): split the 5.3k-line service.ts god-file into per-concern modules (no behavior change) refactor(table): split the 5.3k-line service.ts god-file into per-concern modules Jun 15, 2026
@greptile-apps

greptile-apps Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR decomposes the monolithic lib/table/service.ts (5,324 lines, 58 exports) into six per-concern modules: rows/service.ts, rows/ordering.ts, rows/executions.ts, columns/service.ts, jobs/service.ts, workflow-groups/service.ts, plus shared tx.ts and import-data.ts helpers. The barrel (@/lib/table) is updated to re-export all six new modules, and every consumer is repointed to the correct leaf module.

  • Code-motion only: function bodies are verbatim copies; the dependency graph is a valid DAG (backfill-runner cycle preserved via lazy dynamic import); no logic was altered beyond two relative → absolute dynamic-import strings.
  • Consumers updated: all API routes, background runners, copilot tools, and test mocks are repointed to the new module paths.
  • Three previously-private helpers (EMPTY_JOB_FIELDS, latestJobForTable, latestJobsForTables) are now exported from the barrel as a side-effect of export * from '@/lib/table/jobs/service'; this is additive and harmless but technically widens the public surface beyond what the PR description claims.

Confidence Score: 5/5

Safe to merge — pure code-motion with no logic changes, a valid acyclic dependency graph, and all consumers correctly repointed.

Every moved function body is verbatim from the original file. The dependency graph is a strict DAG with the backfill-runner cycle preserved via lazy dynamic import. No consumer imports a symbol missing from the new barrel or sub-module paths. The only difference from the stated intent is that three previously-private helpers become reachable via the barrel, which is additive and cannot regress existing callers.

The new barrel (index.ts) is the only place worth a second glance to confirm intentional surface exposure of the three newly-public job helpers.

Important Files Changed

Filename Overview
apps/sim/lib/table/index.ts Barrel updated to re-export all six new sub-modules. Three previously-private helpers (EMPTY_JOB_FIELDS, latestJobForTable, latestJobsForTables) are now public via the barrel as a side-effect.
apps/sim/lib/table/service.ts Trimmed from 5,324 to 651 lines; retains table root CRUD and the shared withLockedTable/addTableColumnsWithTx primitives. withLockedTable promoted to exported so sub-modules can consume it.
apps/sim/lib/table/rows/service.ts New 1,676-line module holding all row CRUD and query operations. No import from service.ts — correct leaf in the DAG.
apps/sim/lib/table/jobs/service.ts New 383-line module for the tableJobs state machine. Imports pendingDeleteMask from rows/service, never from service.ts — correct leaf placement.
apps/sim/lib/table/columns/service.ts New 668-line module for column schema mutations. Correctly imports withLockedTable from service.ts.
apps/sim/lib/table/workflow-groups/service.ts New 964-line module for workflow-group management. Preserves lazy dynamic import of backfill-runner to break the cycle; path updated from relative to absolute alias.
apps/sim/lib/table/import-data.ts New 200-line module for bulk-import data ops. Clean dependency chain: batchInsertRowsWithTx from rows/service, addTableColumnsWithTx from service.
apps/sim/lib/table/tx.ts New 50-line shared tx helpers (setTableTxTimeouts, scaledStatementTimeoutMs). Correctly not re-exported via the barrel.
apps/sim/lib/table/rows/ordering.ts New 557-line module for row position/order-key internals. Not re-exported via barrel — correct, internal module.
apps/sim/lib/table/rows/executions.ts New 298-line module for tableRowExecutions read/write internals. Not re-exported via barrel — correct, internal module.
apps/sim/app/api/table/[tableId]/import/route.test.ts Test mocks split into three targeted vi.mock calls (import-data, jobs/service, rows/service). All 27 tests pass.
apps/sim/app/api/table/[tableId]/import/route.ts importAppendRows/importReplaceRows imported directly from @/lib/table/import-data for test-mockability; all other symbols remain on the barrel import.
apps/sim/lib/copilot/tools/server/table/user-table.ts Imports split across four modules (service, columns/service, rows/service, workflow-groups/service). Pure mechanical repointing.
apps/sim/background/workflow-column-execution.ts Four dynamic imports of getRowById/updateRow repointed from @/lib/table/service to @/lib/table/rows/service. Correct split.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    barrel["@/lib/table (index.ts)\n— public barrel —"]
    service["service.ts\nroot CRUD + withLockedTable\naddTableColumnsWithTx"]
    rows_svc["rows/service.ts\ninsertRow / updateRow / deleteRow\nqueryRows / findRowMatches"]
    cols_svc["columns/service.ts\naddTableColumn / renameColumn\ndeleteColumn / updateColumnType"]
    jobs_svc["jobs/service.ts\nmarkTableJobRunning / releaseJobClaim\nselectExportRowPage / getTableJob"]
    wfg_svc["workflow-groups/service.ts\naddWorkflowGroup / updateWorkflowGroup\npruneStaleWorkflowGroupOutputs"]
    import_data["import-data.ts\nbulkInsertImportBatch\nimportAppendRows / importReplaceRows"]
    rows_ord["rows/ordering.ts internal\nacquireRowOrderLock / selectRowIdPage"]
    rows_exec["rows/executions.ts internal\nloadExecutionsByRow / applyExecutionsPatch"]
    tx["tx.ts internal\nsetTableTxTimeouts / scaledStatementTimeoutMs"]
    backfill["backfill-runner.ts\n— lazy dynamic import —"]
    barrel --> service
    barrel --> rows_svc
    barrel --> cols_svc
    barrel --> jobs_svc
    barrel --> wfg_svc
    barrel --> import_data
    service --> jobs_svc
    cols_svc --> service
    wfg_svc --> service
    import_data --> service
    import_data --> rows_svc
    jobs_svc --> rows_svc
    rows_svc --> rows_ord
    rows_svc --> rows_exec
    cols_svc --> rows_exec
    wfg_svc --> rows_exec
    service --> tx
    rows_svc --> tx
    cols_svc --> tx
    rows_ord --> tx
    wfg_svc --> tx
    wfg_svc -. "await import()" .-> backfill
    backfill --> jobs_svc
    backfill --> rows_svc
Loading

Reviews (3): Last reviewed commit: "refactor(table): use absolute imports th..." | Re-trigger Greptile

…TSDoc

Normalize all relative imports under lib/table to absolute @/lib/table/...
per the project import rule (the split modules and a few pre-existing
holdouts), and relocate the getTableById TSDoc that had drifted above
applyColumnOrderToSchema. Comment/import-only; zero behavior change.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit a5a2b7e. Configure here.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

Re: the partial @/lib/table/rows/service mock in import/route.test.ts — confirmed safe. The import route imports exactly one symbol from rows/service: dispatchAfterBatchInsert (called at route.ts:287), and the mock provides precisely that. The route's other @/lib/table barrel imports are releaseJobClaim (jobs/service), sanitizeName/validateMapping, and types — none from rows/service, and it calls no row CRUD/query function. So the mock nulls nothing the test path exercises (27/27 pass). It's also a test-only mock, so it has no production/integration impact regardless.

This whole PR is mechanically verified pure code-motion: of the 93 functions in the original service.ts, 90 are token-identical in the new modules, 1 (updateWorkflowGroup) differs only by two dynamic-import() path strings resolving to the same modules, and 2 were intentionally-removed dead helpers. All 14 non-function symbols are byte-identical.

Re-triggering on the latest commit (a5a2b7e) — the summary footer was still on 059b682.

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit a5a2b7e. Configure here.

@waleedlatif1 waleedlatif1 merged commit 39d0b56 into staging Jun 15, 2026
15 checks passed
@waleedlatif1 waleedlatif1 deleted the refactor/split-table-service branch June 15, 2026 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant