Add spreadsheets for browsing the problem database - #8
Open
anirudhabspk wants to merge 3 commits into
Open
Conversation
Makes 9,385 rows sortable and filterable without running anything. Generated, never authored: every cell is copied from a record or is one subtraction of two published bounds. Two tiers. database/index/all.csv covers every row with only the columns that mean the same thing in every family. One CSV per family adds that family's gap, with the unit written into the column name. The split is the point. A gap of 2 in linear codes counts Hamming distance and a gap of 179 in constant weight codes counts codewords, so a single sortable column holding both invites a meaningless comparison, and GitHub's CSV viewer sorts one column at a time. Inside a family file every row shares the unit, so sorting by gap is correct by construction. Naming the column gap_blocks rather than gap plus a separate unit column makes the mistake structurally impossible. Existence families carry no gap column rather than 3,107 blanks. No difficulty rating and no importance score. The gap says how wide the window between the best construction and the best proof still is, which helps pick a target but is not hardness: a wide gap can equally mean nobody has a strong bound argument yet. Constant weight codes sit at a median gap of 91% of the record for that reason, and their source says its upper bounds are weak. Tests re-derive the interesting column instead of trusting the generator. All 6,247 gaps are recomputed from records.json, with 0 mismatches and 31 intentional blanks for the Ramsey pairs too large to search. has_verifier is re-derived per row, the general table is asserted to carry no gap column, existence families to carry none either, and generation is checked deterministic. CI fails if the committed index is stale. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The definition was buried below the file list. It now opens with the point: the index is a view, not a layer. It adds no information, every cell is copied or is one subtraction, and it can be regenerated or deleted at will. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The old version opened with "a view, not a layer" and said the index "adds no information", which does not tell a reader what the files are. It now says they are spreadsheets, one row per problem, and explains why the numbers can be trusted without using database jargon. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
anirudhabspk
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
The database has 9,385 open maths problems in it. Right now the only way to look
through them is to open JSON files by hand.
This adds spreadsheets. Click any file in
database/index/on GitHub and you geta table you can sort and search.
Base branch is
feat/database-scale(#5). Merge order is #3, #4, #5, then this.This replaces #7, which also included some Erdős prize and Lean linking work that
we decided to drop.
How to check it yourself
Open
database/index/covering-design.csvand sort bygap_blocks. The rows witha gap of 1 are the problems where one better answer would settle the question.
Then run:
Where the numbers come from
A script reads the database and writes these files. Every number is either copied
straight out of a problem's record, or is one bound subtracted from another.
Nothing is guessed or scored.
The tests check this rather than take my word for it. They recompute all 6,247
gaps from the original records and compare them. There are no mismatches. 31 rows
are left blank on purpose, for the Ramsey problems that are too large for the
checker to search.
Why there are 8 files instead of 1
all.csvlists all 9,385 problems and has no gap column.The other 7 files each cover one family of problems and include that family's gap.
The gap has to be split up because it is measured in different things. For linear
codes a gap of 2 means 2 units of Hamming distance. For constant weight codes a
gap of 179 means 179 codewords. If both sat in one sortable column you could
order them together, and the result would tell you nothing. GitHub also sorts one
column at a time, so it cannot group by family first.
The unit is part of the column name, so nobody can mistake
gap_blocksforgap_codewords.Two families have no gap column at all. A difference set either exists or it does
not, so there are no bounds to narrow, and a gap column there would be 3,107
empty cells.
What "gap" means
The distance between the best answer anyone has published and the best limit
anyone has proved. A gap of 1 means one step from solved.
It is not a difficulty rating. A wide gap can mean the problem is far from
solved. It can also mean nobody has found a good way to prove a limit yet.
Constant weight codes have a median gap of 91 percent of the record for the
second reason, and that source says its own limits are weak. So sorting by gap to
find easy problems would mislead you.
What I left out on purpose
There is no difficulty rating and no subject category. Both need a judgement
call, and I wanted every number in these files to be something a reviewer can
check.
I also left out the 5,963 problems from #6. They have no family and no gap
recorded, so including them would add thousands of empty cells.
Files
scripts/build_index.py, the generator, with a--checkmodedatabase/index/, 8 generated tables and a READMEtests/test_index.py, 8 tests