|
| 1 | +# Contributing to ISC-Bench |
| 2 | + |
| 3 | +Thanks for helping us grow the ISC evidence base. This document describes the two main ways to contribute: submitting a new ISC trigger, and contributing code, templates, or documentation. |
| 4 | + |
| 5 | +## Submit an ISC Trigger |
| 6 | + |
| 7 | +We accept community reproductions of Internal Safety Collapse across new models, domains, and task settings. |
| 8 | + |
| 9 | +### Workflow |
| 10 | + |
| 11 | +| Step | What to do | |
| 12 | +|:--|:--| |
| 13 | +| 1. **Trigger ISC** | Pick any [template](templates/) and run it via API (OpenRouter, direct API, etc.). API-based testing is strongly preferred for reproducibility. | |
| 14 | +| 2. **Collect evidence** | Save the model output, share link, or API log. Include the template name and the model API ID. | |
| 15 | +| 3. **Submit the case** | [Open an Issue](https://github.com/wuyoscar/ISC-Bench/issues/new?template=isc-submission.md&title=%5BISC%5D+Model+Name) using the ISC submission template. We handle redaction before publishing. | |
| 16 | + |
| 17 | +### Which Templates to Use |
| 18 | + |
| 19 | +- **`aiml_*` templates** are the recommended entry point for general testing. They are well-understood, broadly applicable, and safe to share. |
| 20 | +- **Cross-domain templates** (biology, chemistry, epidemiology, pharmacology, clinical genomics) are intended for qualified researchers. Public anchors are intentionally weakened; each template includes guidance for more controlled evaluation. |
| 21 | + |
| 22 | +### What Counts as a Valid ISC Case |
| 23 | + |
| 24 | +- No jailbreak, adversarial prompt optimization, or obfuscation |
| 25 | +- The task frames a legitimate professional workflow |
| 26 | +- The model generates harmful content as a functional requirement of completing the task |
| 27 | +- The same request would normally be refused if asked directly |
| 28 | +- The run is reproducible (API log, share link, or agent transcript) |
| 29 | + |
| 30 | +### Attribution |
| 31 | + |
| 32 | +Confirmed reproductions appear on the leaderboard and in [`community/`](community/) with your GitHub handle credited. If you prefer to remain anonymous, note that in the issue. |
| 33 | + |
| 34 | +## Contribute Code, Templates, or Documentation |
| 35 | + |
| 36 | +Other contributions are welcome too: |
| 37 | + |
| 38 | +- **New templates**: follow the existing schema in [`templates/README.md`](templates/README.md). Keep the TVD structure intact; do not strengthen the harmful payload beyond what the task requires. |
| 39 | +- **Experiment code**: see [`experiment/`](experiment/). Preserve reproducibility; avoid adding hidden dependencies. |
| 40 | +- **Docs**: fixes to README, SKILL.md, or template guides are always welcome. |
| 41 | + |
| 42 | +### Pull Request Checklist |
| 43 | + |
| 44 | +- [ ] Changes are scoped to a single topic |
| 45 | +- [ ] Existing file formats and naming conventions are preserved |
| 46 | +- [ ] Any behavioral change is reflected in the relevant README or docs |
| 47 | +- [ ] No secrets, `.env` files, or personal credentials committed |
| 48 | +- [ ] If the change affects templates, the TVD pattern is preserved |
| 49 | + |
| 50 | +## Safety Boundary |
| 51 | + |
| 52 | +ISC-Bench is an academic safety research repository. Please preserve that framing in any contribution: |
| 53 | + |
| 54 | +- Do not strengthen harmful examples beyond what the task requires |
| 55 | +- Do not add content that reads like operational misuse guidance |
| 56 | +- Prefer mild, benchmark-style examples and reproducible research framing |
| 57 | + |
| 58 | +## Questions |
| 59 | + |
| 60 | +Open a [discussion](https://github.com/wuyoscar/ISC-Bench/discussions) or reach out via the issue tracker. |
0 commit comments