fix: keep Case 3 filter adoption in memory, never write it back - #1042
Open
ydixken wants to merge 2 commits into
Open
fix: keep Case 3 filter adoption in memory, never write it back#1042ydixken wants to merge 2 commits into
ydixken wants to merge 2 commits into
Conversation
The exclude work directory is now opened by a list command, by a filterless clone and by a clone filtered on extensions alone, each of which used to overwrite its stored filters and leave the next clone --filters exclude.ini failing with a filtering mismatch (dimitri#1038). Both clones fork workers that re-enter the catalog setup check, which is the path the fix has to keep working.
A command that opens a work directory created with filters, but is invoked without them, adopted the catalog's stored filter type and then wrote its own serialization back to source.db. That serialization carries the type and nothing else, so the stored pattern lists were lost and every later --filters command in that directory failed with a filtering mismatch. This was never confined to a handful of commands: any filter-checked command run without the stored filters reaches case 3, and a plain list tables exits 0 and prints its usual output while truncating the record. The write-back was there so that forked workers, which inherit the adopted type with empty relation lists and whatever extension filtering their parent had, would not hit the case 2 conflict; case 2 now recognizes that adopted signature and lets them through, which leaves the stored record immutable for every filterless caller at once. list progress skips the filter check outright, as the other filter-agnostic commands already do. Fixes dimitri#1038
ydixken
added a commit
to ydixken/pgcopydb-operator
that referenced
this pull request
Aug 21, 2026
The two filed pgcopydb issues now point at the PRs that fix them, dimitri/pgcopydb#1042 and dimitri/pgcopydb#1041. PostgreSQL's 2026-08-13 minor releases (CVE-2026-6471) load only the output plugins named in output_plugin_libraries, which defaults to pgoutput and test_decoding. A source on those releases refuses spec.follow.plugin: wal2json until an admin adds it and reloads, so the prerequisites carry the rule and troubleshooting carries the new error string.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Any filter-checked command invoked without --filters on a filtered work dir (Case 3: list progress, list table-parts, compare, or clone when the filters file is simply not passed) adopts the stored filter type, then persists its own re-serialization via catalog_update_filters. That serialization comes from the command's empty SourceFilters, so the pattern arrays are dropped:
{"type":"SOURCE_FILTER_TYPE_EXCL","exclude-schema":["audit"]}degrades to{"type":"SOURCE_FILTER_TYPE_EXCL"}. Every laterclone --filterson that work dir (or its live workers) then fails the setup comparison and exits 12; one barelist progresspoisons the work dir for good.The write-back exists so forked workers, which inherit the adopted type with empty relation lists, pass the comparison on re-entry (1089e6c). This patch keeps that property without mutating the catalog: the stored record becomes immutable, and the comparison recognizes the adopted signature, the stored type plus whatever extension filtering the parent carried (extension filters contribute no filter type, so an extension-only --filters file also adopts). list progress is additionally marked filter-agnostic (skipFilterCheck) like the sentinel commands.
Regression: tests/filtering now runs list table-parts, list progress, a filterless clone, and an extension-only clone between a filtered clone and its --resume rerun, asserting the stored filtering survives each; tests/pagila-standby keeps covering the filterless-clone worker re-entry path.
Fixes #1038.