Skip to content

Commit 05f43a0

Browse files
x-zheng16claude
andcommitted
add README with badges, key results, and gallery links
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 1d3af8d commit 05f43a0

1 file changed

Lines changed: 87 additions & 0 deletions

File tree

README.md

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
<p align="center">
2+
<h1 align="center">System Prompt Open</h1>
3+
</p>
4+
5+
<h3 align="center">Extracted System Prompts from Frontier LLMs</h3>
6+
7+
<p align="center">
8+
<a href="https://arxiv.org/abs/2601.21233"><img src="https://img.shields.io/badge/arXiv-2601.21233-b31b1b.svg"></a>
9+
<a href="https://github.com/x-zheng16/System-Prompt-Open/stargazers"><img src="https://img.shields.io/github/stars/x-zheng16/System-Prompt-Open" alt="Stars"></a>
10+
<a href="https://github.com/x-zheng16/System-Prompt-Open/network/members"><img src="https://img.shields.io/github/forks/x-zheng16/System-Prompt-Open" alt="Forks"></a>
11+
<a href="https://github.com/x-zheng16/System-Prompt-Open/issues"><img src="https://img.shields.io/github/issues/x-zheng16/System-Prompt-Open" alt="Issues"></a>
12+
</p>
13+
14+
<h3 align="center">
15+
<a href="https://x-zheng16.github.io/System-Prompt-Open/">Website</a> &nbsp;|&nbsp;
16+
<a href="https://arxiv.org/abs/2601.21233">Paper</a> &nbsp;|&nbsp;
17+
<a href="https://github.com/x-zheng16/JustAsk">JustAsk Code</a>
18+
</h3>
19+
20+
---
21+
22+
> **We asked. They answered.**
23+
>
24+
> An open database of system prompts extracted from **40+ commercial LLMs** using [JustAsk](https://github.com/x-zheng16/JustAsk), a self-evolving code agent framework that autonomously discovers extraction strategies through interaction alone.
25+
26+
## Key Results
27+
28+
| Metric | Value |
29+
|:-------|:------|
30+
| Models tested | **41** black-box commercial LLMs + code agents |
31+
| Verified accuracy | **85--95%** on Claude Code (against leaked source) |
32+
| Extraction success | **100%** (consistency score >= 0.7) across all 41 models |
33+
| Supervision required | **Zero** -- fully autonomous UCB-based skill evolution |
34+
35+
## Claude Code Verification
36+
37+
Claude Code's source was leaked via `.map` file in the npm registry (March 2026).
38+
We compared it against our JustAsk extractions from January 2026 -- **two months before the leak**.
39+
40+
| Agent | Accuracy | Gap |
41+
|:------|:--------:|:----|
42+
| Explore Subagent | **95%** | Only missed `pip install` in bash restrictions |
43+
| Plan Subagent | **93%** | Minor output format embellishment |
44+
| General-Purpose Subagent | **90%** | Missed completeness directive |
45+
| Main Agent | **85%** | Missed 2 entire sections |
46+
47+
## Gallery
48+
49+
Browse extracted system prompts interactively:
50+
51+
**[x-zheng16.github.io/System-Prompt-Open](https://x-zheng16.github.io/System-Prompt-Open/)**
52+
53+
49 entries covering:
54+
- **Claude Code** (4 agents, verified against source)
55+
- **Code Agent CLIs** (Gemini CLI, Cursor, Copilot, Perplexity)
56+
- **41 commercial LLMs** (OpenAI, Anthropic, Google, Meta, DeepSeek, xAI, and more)
57+
58+
## How It Works
59+
60+
JustAsk formulates system prompt extraction as an online exploration problem:
61+
62+
1. **UCB Skill Selection** -- balances exploitation of effective skills with exploration of uncertain alternatives
63+
2. **Hierarchical Skill Space** -- 14 low-level atomic probes + 14 high-level multi-turn orchestration strategies
64+
3. **Self-Evolving Rules** -- learns from every interaction, accumulates long-term extraction knowledge
65+
4. **Consistency Validation** -- cross-skill agreement provides extrinsic reward signal
66+
67+
```
68+
UCB(Ci) = success_rate(Ci) + c * sqrt(ln(N) / ni)
69+
^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^
70+
exploitation exploration bonus
71+
```
72+
73+
## Citation
74+
75+
```bibtex
76+
@article{zheng2026justask,
77+
title={Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs},
78+
author={Zheng, Xiang and Wu, Yutao and Huang, Hanxun and Li, Yige
79+
and Ma, Xingjun and Li, Bo and Jiang, Yu-Gang and Wang, Cong},
80+
journal={arXiv preprint arXiv:2601.21233},
81+
year={2026}
82+
}
83+
```
84+
85+
## License
86+
87+
MIT

0 commit comments

Comments
 (0)