Two-column pages: columns are detected but markdown is emitted in raster order (lines interleaved) #219

Summary

On newspaper-style two-column pages, columns are detected (pages_with_columns is populated) but the markdown is still emitted in raster order: each output line is the left-column line concatenated with the right-column line. The README documents "Automatic detection of newspaper-style columns, sequential reading order" — detection works, the sequential reordering does not appear to be applied.

On some real-world two-column pages the same layout is instead classified as a table (pages_with_tables), and the cells come out with the columns interleaved. Both symptoms destroy reading order.

• Version: pdf-inspector 0.2.6 (Python binding), Python 3.12.5, macOS 26.5 arm64
• PDF type: text_based, no OCR needed

Reproduction

No sample file needed — this script builds the PDF (pip install fpdf2):

from fpdf import FPDF

COL_W, COL_X = 85, (15, 110)
BODY = ("Paciente do sexo masculino, 62 anos, comparece ao servico com quadro de "
        "dispneia progressiva ha tres semanas, associada a edema de membros "
        "inferiores e ortopneia. Ao exame, estertores bibasais e turgencia jugular.")
ALTS = ["A) Iniciar diuretico de alca por via intravenosa.",
        "B) Solicitar ecocardiograma transtoracico.",
        "C) Prescrever betabloqueador em dose plena.",
        "D) Indicar cateterismo cardiaco de urgencia."]

pdf = FPDF(format="A4")
pdf.set_auto_page_break(False)

n = 1
for _ in range(2):                      # 2 pages
    pdf.add_page()
    for x in COL_X:                     # 2 columns per page
        pdf.set_xy(x, 20)
        pdf.set_font("helvetica", "B", 10)
        pdf.multi_cell(COL_W, 5, f"{n} - ", align="L")
        pdf.set_x(x)
        pdf.set_font("helvetica", "", 10)
        pdf.multi_cell(COL_W, 5, BODY + " A conduta mais adequada e:", align="J")
        for a in ALTS:
            pdf.set_x(x)
            pdf.multi_cell(COL_W, 5, a, align="L")
        n += 1

pdf.output("two-column-exam.pdf")

import pdf_inspector
r = pdf_inspector.extract_pages_markdown("two-column-exam.pdf")
print(r.pages_with_columns)   # [1, 2]
print(r.pages[0].markdown)

Actual output (page 1)

## 1 -2 -

Paciente do sexo masculino, 62 anos, comparece ao Paciente do sexo masculino, 62 anos, comparece ao servico com quadro de dispneia progressiva ha tres servico com quadro de dispneia progressiva ha tres semanas, associada a edema de membros inferiores semanas, associada a edema de membros inferiores e ortopneia. Ao exame, estertores bibasais e e ortopneia. Ao exame, estertores bibasais e turgencia jugular. A conduta mais adequada e: turgencia jugular. A conduta mais adequada e:

A) Iniciar diuretico de alca por via intravenosa. A) Iniciar diuretico de alca por via intravenosa.
B) Solicitar ecocardiograma transtoracico. B) Solicitar ecocardiograma transtoracico.
C) Prescrever betabloqueador em dose plena. C) Prescrever betabloqueador em dose plena.
D) Indicar cateterismo cardiaco de urgencia. D) Indicar cateterismo cardiaco de urgencia.

Both items collapse into ## 1 -2 -, and every body line is the left line + the right line glued together.

Expected output

Column 1 fully, then column 2:

## 1 -

Paciente do sexo masculino, 62 anos, ... A conduta mais adequada e:

A) Iniciar diuretico de alca por via intravenosa.
...

## 2 -

Paciente do sexo masculino, 62 anos, ... A conduta mais adequada e:

A) Iniciar diuretico de alca por via intravenosa.
...

pdftotext -layout (poppler) keeps the two columns separable on the same files.

Real-world impact

Measured on a corpus of 26 two-column Brazilian medical residency exam PDFs (text-based, no OCR). A regex parser that splits questions on the N - marker at line start recovers all items from pdftotext -layout, but from extract_pages_markdown the split dies at the first affected page:

The other 21 files (single-column, or two-column pages that happened not to trigger it) parse identically to poppler once markdown syntax is normalized, and the answer-key grids actually come out better as markdown tables. So this is the one blocker for using pdf-inspector as the extractor in that pipeline.

In the SUS-SP 2021 case the offending page went down the table path instead, producing rows like:

|24 - 25 -|respectivamente:|B) Indicar toracoscopia diagnostica. C) Indicar drenagem toracica. ...

i.e. two item markers in one cell, and alternatives from different items merged into a third — same root cause, different renderer.

Notes

Happy to test a patch against the 26-file corpus. A public escape hatch would also help even before the ordering is fixed — the Python binding currently exposes only pages, so there is no way to opt out of table detection or to request a layout-preserving text mode. The Rust PdfOptions / ProcessMode mentioned in the README are not reachable from Python 0.2.6.