updated starmath writer

[jdpipe]

a new PR based on the jdpipe-master plus changes to address comments from #285

to generate the visual comparison doc/pdf files (eg starmath-review.pdf) use the following:

./tools/make-starmath-review.py --output-dir /tmp/starmath-review && xdg-open /tmp/starmath-review/starmath-review.odt

this change uses the 'native' representation as the base for tests, and includes all tex-to-native tests from the existing test suite in addition to the new ones earlier provided just for starmath.

there are a number of limitations in starmath exposed. the main issue is around \mathcal and \mathbb and \mathfrak which don't exist 'easily' in LO. my solution was to request the user to select 'Latin Modern Math' as their custom 'serif' font in LO Math, and then provide these chars as unicode in the equation. converted equations include a comment telling users to do that.

[jdpipe]

all tests seem to be passing now -- some changes required to overcome an issue with API changes in the typst-symbols package, which was blocking the build for this PR. I patched for that, but if you have a better/different solution for that issue, please use that instead.

[jgm]

You should be able to simply merge/rebase the commit to texmath that handles the typst-symbols changes.

[jgm]

there are a number of limitations in starmath exposed. the main issue is around \mathcal and \mathbb and \mathfrak which don't exist 'easily' in LO. my solution was to request the user to select 'Latin Modern Math' as their custom 'serif' font in LO Math, and then provide these chars as unicode in the equation. converted equations include a comment telling users to do that.

Do these work currently in LibreOffice with pandoc's current MathML output?

[jdpipe]

there are a number of limitations in starmath exposed. the main issue is around \mathcal and \mathbb and \mathfrak which don't exist 'easily' in LO. my solution was to request the user to select 'Latin Modern Math' as their custom 'serif' font in LO Math, and then provide these chars as unicode in the equation. converted equations include a comment telling users to do that.

Do these work currently in LibreOffice with pandoc's current MathML output?

I glad you asked... I learned something.

By testing 'standard' pandoc/LO on a separate/clean Raspberry Pi system, I found that the existing MathML-based behaviour for texmath is already good specifically with these fonts. The default fonts (Liberation Serif etc) in LO appear to provide the missing symbols via font fallback behaviour (to Noto Sans Math, as confirmed using gucharmap). The following LaTeX code

\mathbb{A} \mathcal{A} \mathfrak{A}

can be used directly with pdflatex to get

Meanwhile, un-patched pandoc converts this (via MathML) into the perfectly serviceable Starmath code:

{ nitalic 𝔸 nitalic 𝓐 nitalic 𝕬 }

This could be only slightly improved by rendering as

nitalic {𝔸𝓐𝕬}
or perhaps
nitalic {𝔸 𝓐 𝕬}

or some variants on that theme. A bit of testing with LibreOffice in Raspberry Pi (standard/default fonts, no tweaks or special fonts added) gave me:

Whereas LO with Ubuntu 24.04 (with whatever added fonts I happened to have, and with the 'serif' font set for Starmath set to 'Latin Modern Math'):

So clearly if we choose 'nitalic {𝔸𝓐𝕬}' then we get some glitchy behaviour in Raspberry Pi with the standard font (ie's the fallback to Noto Sans Math which seems to be the issue). But actually even when that glitch is not there, I felt the symbols were a bit too close. LO is not as nuanced with symbols spacing as Latex is. The best compromise seems to be

nitalic {𝔸 𝓐 𝕬}

rather than

nitalic {𝔸𝓐𝕬}

and I felt this is also better than the current pandoc behaviour, which is
{ nitalic 𝔸 nitalic 𝓐 nitalic 𝕬 }

But having decided on the 'nitalic + unicode' approach, it's all much of a muchness. It appears that Ubuntu (or at least my local install) has got better font fallbacks, and more work is probably needed on math fonts for LO in general. But the unicode-encoding of \mathcal and \mathbb and \mathfrak (etc) is all we need for now, we don't need to require the user to install/select a special font as I had earlier been suggesting. That is because fonts can selectively fall back to other fonts where specific glyphs are missing, and that is already working correctly here in this specific case, at least on the Linux systems I checked.

It is nevertheless true that the Latin Modern Math font is markedly better on character spacing that the default fallback fonts, in both cases. I feel that there's probably a valid bug here to be reported to font-config or whatever the relevant project is...

[jdpipe]

@jgm i think the rebase is done now... hopefully the build is clean...

[jgm]

As for the linux/stack build failure...be sure to include .github/workflows/ci.yml in your rebase; I have removed the stack build from master...