Skip to content

README.md

Latest commit

 

History

History
53 lines (37 loc) · 4.83 KB

README.md

File metadata and controls

53 lines (37 loc) · 4.83 KB
Raw
title Literate GenAI project

Literate programming has a long history, going back to Donald Knuth's foundational works from 1984. His work was refined or simplified by Norman Ramsey in 1994 to introduce the NOWEB style of references to deconstruct programs into human understandable chunks.

Both Knuth's and Ramsey's systems are no longer much in use. However, literate programming is still done in the form of notebooks, for which there are numerous systems available. Chief among those: Mathematica, Jupyter, Observable, Marimo, Pluto.jl, etc. Each of those comes with their own (often language specific) file formats. The most natural extension of Knuth's ideas is actually using NOWEB syntax in combination with Markdown. Entangled is a tool that makes literate programming easier and more scalable by keeping Markdown files and tangled code synchronized during the development process (Hidding 2023). More importantly, Entangled provides a natural syntax in which NOWEB is combined with Markdown.

With the increasing influence of GenAI1, the broad tendency is to store knowledge contexts into a collection of Markdown files for LLMs to work with. Again we see the use of Markdown as a universal container for human-readable knowledge. This alone could be incentive enough to drive further research into combining literate programming techniques with the use of GenAI. We can pose several questions.

Where it was Knuth's original goal to make programmers explain to human beings what it is we want the computer to do, we can now use this technique to make computers explain to humans what it is what they want other, arguably more deterministic omputers to do.

  1. (guide) Is there a workflow in which the initial documentation acts as a prompt for an LLM to complete the literate code for us?
  2. (guide) Can we use literate programming techniques to force LLMs to explain their code in a human readable and reproducable way?
  3. (guide) Are LLMs able to deconstruct legacy source into literate code so we understand its function better?
  4. (engineering) Can we integrate Entangled into an LLM harness, so that its literate response can be checked for correctness, possibly against type checking compilers and/or sets of unit tests?
  5. (strategy) Can literate programming increase the quality of generated PRs with regards to readability, correctness and accountability?

Project Plan

  • Available hours: 400-500 (by Elena's account)
  • Work with 4/5 engineers during October-December 2026. Unconfirmed: Johan (lead), Ole, Leon, Victor (has experience in GenAI use). Possible involvement of GenAI team: Carlos, Faruk and Ou.
  1. Establish models to use. Victor (or someone with equivalent GenAI experience) teaches the team how to add a Markdown knowledge base to a prompt, either through its CLI or an agent enabled editor like Zed. Outcome: a Markdown file that users can attach to a prompt to teach the LLM literate programming. Such a text could also be attached to an AGENTS.md file for project incorporation.
  2. Establish a workflow where we demonstrate uses by prompting a problem statement. We could score outcomes on fraction of code vs. proze, code size versus a hand-crafted reference implementation, generic readability for review (jury score).
  3. Use identical prompts with and without literate programming enabled (by either applying or leaving out the literate programming prompt injection). Can we discover changes in the quality of produced PRs?
  4. Use entries from the Obfuscated C Coding Competition to try to explain how they work. Or we use some obfuscated Fortran from past projects.
  5. Establish a human knowledge base on how to integrate Entangled into an existing LLM harness.

Project outcomes:

  • A markdown file for prompt injection and AGENT.md inclusion.
  • Blog posts and entries into the eScience guide.
  • Depending on results: position paper in a journal on open science practices.
  • Improved stability of the Entangled tool.

Entangled Fact sheet

  • Started in 2018, as a Haskell application.
  • Ported to Python in 2023.
  • Published in ieee eScience 2023: Hidding, Johan. "Entangled, a Bidirectional System for Sustainable Literate Programming." 2023 IEEE 19th International Conference on e-Science (e-Science). IEEE, 2023.
  • 103 stars on Github for the Python implementation (as of 01-07-2026).
  • Numerous user statements available.
  • Over 13k downloads per month from PyPi, see PyPiStats. For comparison (Jupyter Lab: 46M, Marimo: 2.3M).

Resources

Footnotes

Footnotes

  1. We use the term GenAI here to distinguish from the wider field of AI, understanding GenAI to be the application of Large Language Models (LLM) to generate, proze, code or both. ↩