Skip to content

Refresh test schema to match current platform schema - #16

Merged
rowanseymour merged 2 commits into
mainfrom
refresh_testdb_schema
Aug 5, 2026
Merged

Refresh test schema to match current platform schema#16
rowanseymour merged 2 commits into
mainfrom
refresh_testdb_schema

Conversation

@rowanseymour

@rowanseymour rowanseymour commented Aug 5, 2026

Copy link
Copy Markdown
Member

The hand written testdb.sql used by the test suite had drifted from the platform schema it's meant to mirror. Since it's only a subset of that schema, drift is easy to miss — but a column that's misnamed, wrongly typed, or has the wrong nullability means the tests can pass against a shape that doesn't exist in production.

Refreshed it against the current schema. The rule is now stated at the top of the file: it stays a minimal subset (only the tables and columns archiver reads from or deletes from), but any column that is present must match its real definition exactly.

Changes

Renamed or removed upstream:

  • auth_user no longer exists — replaced with users_user (usernameemail)
  • msgs_msg.quick_replies varchar(64)[]quickreplies jsonb
  • added flows_flowstartcount, which DeleteFlowStarts deletes from but which was missing entirely
  • dropped the DROP TABLE lines for channels_channellog (no longer exists) and ivr_call (never created here)

Nullability, two of which were inverted:

  • flows_flowrun.session_uuid is NOT NULL, flows_flowrun.results is nullable
  • msgs_msg.next_attempt, msgs_broadcast.is_active, contacts_contact.created_by_id and modified_by_id are all nullable

Types:

  • uuid rather than varchar(36) on channels_channel, flows_flow, contacts_contactgroup, msgs_label (contacts_contact.uuid is genuinely still a varchar)
  • bigint/bigserial on archives_archive.id and rollup_id, flows_flowrun.id, msgs_label.id, msgs_msg_labels.*, msgs_broadcast.schedule_id, msgs_broadcastmsgcount.count and the m2m table ids
  • msgs_msg.attachments is varchar(2048)[]
  • name lengths on orgs_org, channels_channel, flows_flow, contacts_contactgroup

Also added the non-null columns needed to keep the fixture coherent (msgs_label.org_id, flows_flowstart.flow_id/uuid/start_type/status/modified_on, msgs_broadcast.base_language, msgs_broadcastmsgcount.is_squashed) plus the foreign keys on archives_archive.org_id and rollup_id.

Coverage for orphaned flow start deletion

The reason the missing flows_flowstartcount table went unnoticed is that its delete path never ran: the fixture's only flow start had a run pointing at it, so the NOT EXISTS guard in DeleteFlowStarts never selected anything and the DELETE from flows_flowstartcount statement never executed.

Added flow start #2 for org 3 — old enough to be past the retention threshold, with no runs pointing at it — along with its contact, group and count rows, and an org 3 contact group for it to reference. TestArchiveOrgRuns now asserts all four rows are deleted while start #1, which still has a run, survives.

Test plan

  • go test -p=1 ./... passes.
  • Verified the schema refresh by diffing information_schema.columns for the loaded test schema against the real one — every column present matches on name, type, length and nullability.
  • Verified the new assertions are meaningful by flipping the flows_flowstartcount count expectation to 1 and confirming TestArchiveOrgRuns fails, so the delete really is executing rather than the row never having existed.
  • Fixture data is otherwise unchanged apart from values for the newly required columns, so the archive content and hash assertions are untouched.

The hand written test schema had drifted from the actual platform schema. Columns which are present now match
their real definitions in name, type and nullability:

 * auth_user is now users_user
 * msgs_msg.quick_replies is now quickreplies jsonb
 * add flows_flowstartcount which we delete from when deleting orphaned starts
 * fix nullability of flows_flowrun.session_uuid, flows_flowrun.results and msgs_msg.next_attempt
 * use uuid type for uuid columns which are no longer varchar
 * use bigint/bigserial for ids which are no longer integer
 * fix varchar lengths on name and attachment columns
 * add missing non-null columns and foreign keys
 * remove drops for tables which are never created
Comment thread testdb.sql
@claude

claude Bot commented Aug 5, 2026

Copy link
Copy Markdown

Review Summary

Solid, well-documented change — this is a schema-only refresh of the testdb.sql fixture to match the real temba schema, with clear rationale in the description. I verified the main risks and found no correctness issues:

  • Renamed/retyped columns don't break queries. quick_repliesquickreplies is safe: no Go code references either name (the message archive query at archives/messages.go reads only attachments, which stays varchar(...)[] and works with unnest). The auth_userusers_user rename is reflected in the msgs_msg.created_by_id FK.
  • New NOT NULL columns have no conflicting fixture data. msgs_optin, msgs_broadcastmsgcount, and flows_flowstartcount aren't seeded, and the flows_flowstart / msgs_broadcast / msgs_label inserts were updated for their new required columns.
  • FK & create ordering are correctflows_flowstartcount is created after flows_flowstart and dropped via CASCADE.
  • DeleteFlowStarts now has a table to delete from (flows_flowstartcount), which was previously missing entirely.

One gap (also acknowledged in the PR notes): the newly-added flows_flowstartcount table and the DeleteFlowStarts delete path are still never exercised by the tests, since the fixture's only flow start has a run attached. Adding an orphaned start would both close that coverage gap and protect the new table from silent future drift — flagged inline.

No blocking issues.

The fixture's only flow start had a run attached, so the orphan deletion path in DeleteFlowStarts never
executed and the tables it deletes from were never validated. Add an old start with no runs, along with its
contact, group and count rows, and assert they're all deleted while a start which still has a run is not.
@rowanseymour
rowanseymour merged commit 61d86eb into main Aug 5, 2026
3 checks passed
@rowanseymour
rowanseymour deleted the refresh_testdb_schema branch August 5, 2026 22:29
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 5, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant