|
| 1 | +<p align="center"> |
| 2 | + <h1 align="center">System Prompt Open</h1> |
| 3 | +</p> |
| 4 | + |
| 5 | +<h3 align="center">Extracted System Prompts from Frontier LLMs</h3> |
| 6 | + |
| 7 | +<p align="center"> |
| 8 | + <a href="https://arxiv.org/abs/2601.21233"><img src="https://img.shields.io/badge/arXiv-2601.21233-b31b1b.svg"></a> |
| 9 | + <a href="https://github.com/x-zheng16/System-Prompt-Open/stargazers"><img src="https://img.shields.io/github/stars/x-zheng16/System-Prompt-Open" alt="Stars"></a> |
| 10 | + <a href="https://github.com/x-zheng16/System-Prompt-Open/network/members"><img src="https://img.shields.io/github/forks/x-zheng16/System-Prompt-Open" alt="Forks"></a> |
| 11 | + <a href="https://github.com/x-zheng16/System-Prompt-Open/issues"><img src="https://img.shields.io/github/issues/x-zheng16/System-Prompt-Open" alt="Issues"></a> |
| 12 | +</p> |
| 13 | + |
| 14 | +<h3 align="center"> |
| 15 | + <a href="https://x-zheng16.github.io/System-Prompt-Open/">Website</a> | |
| 16 | + <a href="https://arxiv.org/abs/2601.21233">Paper</a> | |
| 17 | + <a href="https://github.com/x-zheng16/JustAsk">JustAsk Code</a> |
| 18 | +</h3> |
| 19 | + |
| 20 | +--- |
| 21 | + |
| 22 | +> **We asked. They answered.** |
| 23 | +> |
| 24 | +> An open database of system prompts extracted from **40+ commercial LLMs** using [JustAsk](https://github.com/x-zheng16/JustAsk), a self-evolving code agent framework that autonomously discovers extraction strategies through interaction alone. |
| 25 | +
|
| 26 | +## Key Results |
| 27 | + |
| 28 | +| Metric | Value | |
| 29 | +|:-------|:------| |
| 30 | +| Models tested | **41** black-box commercial LLMs + code agents | |
| 31 | +| Verified accuracy | **85--95%** on Claude Code (against leaked source) | |
| 32 | +| Extraction success | **100%** (consistency score >= 0.7) across all 41 models | |
| 33 | +| Supervision required | **Zero** -- fully autonomous UCB-based skill evolution | |
| 34 | + |
| 35 | +## Claude Code Verification |
| 36 | + |
| 37 | +Claude Code's source was leaked via `.map` file in the npm registry (March 2026). |
| 38 | +We compared it against our JustAsk extractions from January 2026 -- **two months before the leak**. |
| 39 | + |
| 40 | +| Agent | Accuracy | Gap | |
| 41 | +|:------|:--------:|:----| |
| 42 | +| Explore Subagent | **95%** | Only missed `pip install` in bash restrictions | |
| 43 | +| Plan Subagent | **93%** | Minor output format embellishment | |
| 44 | +| General-Purpose Subagent | **90%** | Missed completeness directive | |
| 45 | +| Main Agent | **85%** | Missed 2 entire sections | |
| 46 | + |
| 47 | +## Gallery |
| 48 | + |
| 49 | +Browse extracted system prompts interactively: |
| 50 | + |
| 51 | +**[x-zheng16.github.io/System-Prompt-Open](https://x-zheng16.github.io/System-Prompt-Open/)** |
| 52 | + |
| 53 | +49 entries covering: |
| 54 | +- **Claude Code** (4 agents, verified against source) |
| 55 | +- **Code Agent CLIs** (Gemini CLI, Cursor, Copilot, Perplexity) |
| 56 | +- **41 commercial LLMs** (OpenAI, Anthropic, Google, Meta, DeepSeek, xAI, and more) |
| 57 | + |
| 58 | +## How It Works |
| 59 | + |
| 60 | +JustAsk formulates system prompt extraction as an online exploration problem: |
| 61 | + |
| 62 | +1. **UCB Skill Selection** -- balances exploitation of effective skills with exploration of uncertain alternatives |
| 63 | +2. **Hierarchical Skill Space** -- 14 low-level atomic probes + 14 high-level multi-turn orchestration strategies |
| 64 | +3. **Self-Evolving Rules** -- learns from every interaction, accumulates long-term extraction knowledge |
| 65 | +4. **Consistency Validation** -- cross-skill agreement provides extrinsic reward signal |
| 66 | + |
| 67 | +``` |
| 68 | +UCB(Ci) = success_rate(Ci) + c * sqrt(ln(N) / ni) |
| 69 | + ^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^ |
| 70 | + exploitation exploration bonus |
| 71 | +``` |
| 72 | + |
| 73 | +## Citation |
| 74 | + |
| 75 | +```bibtex |
| 76 | +@article{zheng2026justask, |
| 77 | + title={Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs}, |
| 78 | + author={Zheng, Xiang and Wu, Yutao and Huang, Hanxun and Li, Yige |
| 79 | + and Ma, Xingjun and Li, Bo and Jiang, Yu-Gang and Wang, Cong}, |
| 80 | + journal={arXiv preprint arXiv:2601.21233}, |
| 81 | + year={2026} |
| 82 | +} |
| 83 | +``` |
| 84 | + |
| 85 | +## License |
| 86 | + |
| 87 | +MIT |
0 commit comments