Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
60 lines (39 loc) · 3.11 KB

CONTRIBUTING.md

File metadata and controls

60 lines (39 loc) · 3.11 KB

Contributing to ISC-Bench

Thanks for helping us grow the ISC evidence base. This document describes the two main ways to contribute: submitting a new ISC trigger, and contributing code, templates, or documentation.

Submit an ISC Trigger

We accept community reproductions of Internal Safety Collapse across new models, domains, and task settings.

Workflow

Step What to do
1. Trigger ISC Pick any template and run it via API (OpenRouter, direct API, etc.). API-based testing is strongly preferred for reproducibility.
2. Collect evidence Save the model output, share link, or API log. Include the template name and the model API ID.
3. Submit the case Open an Issue using the ISC submission template. We handle redaction before publishing.

Which Templates to Use

  • aiml_* templates are the recommended entry point for general testing. They are well-understood, broadly applicable, and safe to share.
  • Cross-domain templates (biology, chemistry, epidemiology, pharmacology, clinical genomics) are intended for qualified researchers. Public anchors are intentionally weakened; each template includes guidance for more controlled evaluation.

What Counts as a Valid ISC Case

  • No jailbreak, adversarial prompt optimization, or obfuscation
  • The task frames a legitimate professional workflow
  • The model generates harmful content as a functional requirement of completing the task
  • The same request would normally be refused if asked directly
  • The run is reproducible (API log, share link, or agent transcript)

Attribution

Confirmed reproductions appear on the leaderboard and in community/ with your GitHub handle credited. If you prefer to remain anonymous, note that in the issue.

Contribute Code, Templates, or Documentation

Other contributions are welcome too:

  • New templates: follow the existing schema in templates/README.md. Keep the TVD structure intact; do not strengthen the harmful payload beyond what the task requires.
  • Experiment code: see experiment/. Preserve reproducibility; avoid adding hidden dependencies.
  • Docs: fixes to README, SKILL.md, or template guides are always welcome.

Pull Request Checklist

  • Changes are scoped to a single topic
  • Existing file formats and naming conventions are preserved
  • Any behavioral change is reflected in the relevant README or docs
  • No secrets, .env files, or personal credentials committed
  • If the change affects templates, the TVD pattern is preserved

Safety Boundary

ISC-Bench is an academic safety research repository. Please preserve that framing in any contribution:

  • Do not strengthen harmful examples beyond what the task requires
  • Do not add content that reads like operational misuse guidance
  • Prefer mild, benchmark-style examples and reproducible research framing

Questions

Open a discussion or reach out via the issue tracker.