Make OCaml<->Coq string conversion in the extraction drivers linear-time and stack-safe - #2408
Open
JasonGross wants to merge 1 commit into
Open
Make OCaml<->Coq string conversion in the extraction drivers linear-time and stack-safe#2408JasonGross wants to merge 1 commit into
JasonGross wants to merge 1 commit into
Conversation
Scrutineer finding #2510: in the extracted OCaml synthesis binaries
(src/StandaloneOCamlMain.v) and the js_of_ocaml web build
(src/StandaloneJsOfOCamlMain.v), every input line (argv element, stdin
line, --hints-file line) was converted to a Coq string by
String.string_of_list_ascii
(List.map (fun n:nat => string_get s n) (List.seq 0 (string_length s)))
where the index is a Peano nat (no ExtrOcamlNatInt is in use) and the
int<->nat coercions nat_of_int / int_of_nat are inserted on every
character. This costs L(L+1)/2 recursive calls and 2L stack frames for
a line of length L: a 1e5-character hints-file line took 12.5 s and a
1e6-character line died immediately with "Fatal error: exception Stack
overflow". The reverse conversion (string_of_Coq_string, used for
file names and, in the web build, for all output) was quadratic for the
same reason (List.nth with a nat_of_int-coerced index per character),
and output went through the non-tail-recursive list_ascii_of_string.
Fix: implement the conversions directly as OCaml primitives via
Extract Constant. string_to_Coq_string walks the string from the end
with a tail-recursive accumulator; string_of_Coq_string uses a Buffer;
output uses a new fprintf_Coq_string primitive (List.iter output_char,
then flush). All are linear-time and use constant stack. The
int inductive, string_length/string_get/string_init, and the
nat_of_int/int_of_nat coercions are removed so the pattern cannot be
reintroduced by accident. Lines read from a channel are converted
with a single tail-recursive fold (Coq_strings_of_rev_lines) instead of
List.map (List.rev_append ...), and the js_of_ocaml driver's
js_to_list_map uses a tail-recursive map, so that very many lines do
not overflow the stack either.
The generated code is unchanged: the rebuilt fiat_crypto binary
produces byte-identical output for the Makefile.examples targets that
were checked, and the same error message as before on an invalid
hints file.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Rbn2gww3MGhvrh52fNjpD
JasonGross
marked this pull request as ready for review
September 2, 2026 03:41
JasonGross
enabled auto-merge (squash)
September 2, 2026 03:41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes scrutineer finding #2510 (CWE-407: quadratic-time, linear-stack conversion of every input line in the extraction IO glue).
Bug
In
src/StandaloneOCamlMain.v(all 26 extracted OCaml synthesis binaries) andsrc/StandaloneJsOfOCamlMain.v(the js_of_ocaml web build), every OCaml string coming from argv, stdin, or a--hints-filewas turned into a CoqString.stringbyString.string_of_list_ascii (List.map (fun n:nat => string_get s n) (List.seq 0 (string_length s))). Coqnatis extracted as unary Peano numbers here (noExtrOcamlNatInt), and thenat_of_int/int_of_natcoercions were inserted on every character, so converting one line of length L costs L(L+1)/2 recursive calls and ~2L stack frames. The reverse directionstring_of_Coq_string(file names; in the web build, all output) was quadratic for the same reason viaList.nthwith a coerced index, and output went through the non-tail-recursivelist_ascii_of_string. A hints file with a single long line is a well-formed input thatfiat-amd64/gentest.pyfeeds to the binary frommake.Measured with the installed binary (
unsaturated-solinas curve25519 64 5 '2^255 - 19' carry_mul ... --hints-file <one line of N 'a's>, default 8 MB stack):Fatal error: exception Stack overflowA 100,000-character argv element likewise took 11.3 s.
Fix
Implement the conversions directly as OCaml primitives with
Extract Constant, in both drivers:string_to_Coq_string: walk the string from the end with a tail-recursive accumulator (linear, constant stack).string_of_Coq_string:Buffer+List.iter(linear, constant stack).fprintf_Coq_stringprimitive for output (List.iter output_charthenflush), replacing per-characterPrintf.fprintf "%c%!"overlist_ascii_of_string. Flushing is now once per string instead of once per character.fold_leftover the reversed line list (Coq_strings_of_rev_lines) instead ofList.map f (rev_append ...); the js driver'sjs_to_list_mapuses a tail-recursive map, so very many lines do not overflow the stack either.intinductive,string_length/string_get/string_init, and thenat_of_int/int_of_natcoercions are removed entirely so the pattern cannot be reintroduced silently.fprintf_charis removed as dead.The Coq-side models (the
OCamlPrimitivesmodule that keeps coqchk from reporting axioms) are updated accordingly. The Haskell driver was never affected. Alternatives considered:Require ExtrOcamlNatInt(would changenatextraction for the whole program, which is far more invasive and not obviously safe), or bounding line length on read (rejects valid inputs and leaves the quadratic behaviour below the bound).Verification
Overlay-compiled against the installed library (Rocq 9.4+alpha, OCaml 4.14.2):
coqcofsrc/StandaloneOCamlMain.v,src/StandaloneJsOfOCamlMain.v,src/Bedrock/Standalone/StandaloneOCamlMain.v,src/Bedrock/Standalone/StandaloneJsOfOCamlMain.v,src/Rewriter/PerfTesting/StandaloneOCamlMain.v: all succeed.src/ExtractionOCaml/fiat_crypto.v,unsaturated_solinas.v,word_by_word_montgomery.v, andsrc/ExtractionJsOfOCaml/fiat_crypto.v; the extracted OCaml contains nonat_of_int/int_of_nat. Built the three OCaml binaries with theMakefile.standaloneocamlfind ocamlopt -package unix -w -20 -gcommands.fiat_crypto, same invocation as above (8 MB stack):--hints-file -on stdinThe stdout of the 100,000 case (the "Error while parsing assembly: Line 1: aaa…" report) is byte-identical between the old and new binaries after normalising the binary path.
make -f Makefile.examples fiat-c/src/curve25519_64.c fiat-c/src/p256_64.c fiat-rust/src/poly1305_32.rs fiat-go/64/p256/p256.gowith the rebuilt separate binaries: all four regenerated files are byte-identical to the committed ones (git statusclean).fiat-amd64/fiat_curve25519_carry_mul/seed0000000059842304_ratio12536.asm) through old and newfiat_crypto: exit 0 in both, byte-identical stdout after normalising the binary path.Remaining / not verified
Stack overflowunder an 8 MB stack, but now inside Coq code downstream of the driver:Crypto.Util.Strings.String.split_newlines(List.flat_map (split ...)) called fromAssembly.Parse.parse_Lines. With theulimit -S -s 1048576the README already recommends, that line completes (parse error reported) in 111 s. That is the assembly parser, a separate component from this finding's IO glue; not addressed here.coqc+ extraction:js_of_ocamlis not installed in this environment, so the extractedsrc/ExtractionJsOfOCaml/fiat_crypto.mlwas not compiled. The two new primitives use onlyStdlib.String/Stdlib.Buffer/Stdlib.Listand are identical to the ones exercised in the native build.🤖 Generated with Claude Code
https://claude.ai/code/session_016Rbn2gww3MGhvrh52fNjpD