Description
While working with the Cactus codebase, I encountered three related bugs/limitations in the FFI bindings and tool call parsing logic:
1. Invalid JSON generated during single-quoted parameter parsing in tool calls
LiquidAI models (LFM2/LFM2.5) format tool calls as Python-like arguments (e.g., my_tool(param='val')).
- The issue: In
cactus-engine/src/chat_tools.h, the parser function py_literal_to_json translates escaped single quotes (\') literally as \' inside the resulting double-quoted JSON string (producing "hello \'world\"").
- Why it fails: Per RFC 8259,
\' is an invalid escape sequence in JSON. Standard JSON parsers (e.g. C++ picojson and python's json module) fail with a syntax error, causing tool execution to abort.
- Proposed fix: Modify the escape sequence parser to unescape
\' to ' in the double-quoted JSON string since single quotes do not require escaping.
2. RuntimeError when retrieving documents > 4KB from index
- The issue: The vector database supports indexing documents up to 65,535 bytes (
Index::validate_documents). However, the Python FFI binding cactus_index_get uses a hardcoded buffer limit _INDEX_DOC_BUF_SIZE = 4096.
- Why it fails: When retrieving a document that exceeds 4KB, the C++ code returns
-1 (buffer too small), causing the Python wrapper to raise RuntimeError: Failed to get from index.
- Proposed fix: Bump
_INDEX_DOC_BUF_SIZE to 65536 in the Python FFI bindings to accommodate the maximum supported document size.
3. RAG query response buffer size limitation (64KB)
- The issue: The Python binding
cactus_rag_query allocates a 65536 byte buffer.
- Why it fails: RAG queries returning multiple chunks (up to
top_k = 5) can easily exceed 64KB, causing query failures.
- Proposed fix: Increase the RAG query buffer to
1 << 20 (1MB), matching the completion buffer size.
Verification
- Created a new C++ test suite
cactus-engine/tests/test_chat_tools.cpp to verify parsing logic.
- Verified that all tests compile and pass successfully.
I have these changes ready on a local branch. Please let me know if you would like me to submit a Pull Request!
Description
While working with the Cactus codebase, I encountered three related bugs/limitations in the FFI bindings and tool call parsing logic:
1. Invalid JSON generated during single-quoted parameter parsing in tool calls
LiquidAI models (
LFM2/LFM2.5) format tool calls as Python-like arguments (e.g.,my_tool(param='val')).cactus-engine/src/chat_tools.h, the parser functionpy_literal_to_jsontranslates escaped single quotes (\') literally as\'inside the resulting double-quoted JSON string (producing"hello \'world\"").\'is an invalid escape sequence in JSON. Standard JSON parsers (e.g. C++picojsonand python'sjsonmodule) fail with a syntax error, causing tool execution to abort.\'to'in the double-quoted JSON string since single quotes do not require escaping.2.
RuntimeErrorwhen retrieving documents > 4KB from indexIndex::validate_documents). However, the Python FFI bindingcactus_index_getuses a hardcoded buffer limit_INDEX_DOC_BUF_SIZE = 4096.-1(buffer too small), causing the Python wrapper to raiseRuntimeError: Failed to get from index._INDEX_DOC_BUF_SIZEto65536in the Python FFI bindings to accommodate the maximum supported document size.3. RAG query response buffer size limitation (64KB)
cactus_rag_queryallocates a65536byte buffer.top_k = 5) can easily exceed 64KB, causing query failures.1 << 20(1MB), matching the completion buffer size.Verification
cactus-engine/tests/test_chat_tools.cppto verify parsing logic.I have these changes ready on a local branch. Please let me know if you would like me to submit a Pull Request!