Skip to content

Commit b12fa7d

Browse files
authored
Merge pull request #72 from wuyoscar/update/2026-03-31
feat: 40 confirmed models, README overhaul, 84 templates, 40 community cases
2 parents c39651a + 907d2a6 commit b12fa7d

115 files changed

Lines changed: 5100 additions & 616 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CHANGELOG.md

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,37 @@
22

33
All notable updates to ISC-Bench are documented here.
44

5-
## 2026-03-29 (latest)
5+
## 2026-04-01 (latest)
6+
7+
### New ISC Triggers
8+
- **GPT-4.1** (Rank 68) — agentic detoxify + single-turn cyber_scapy + compbio_cobrapy
9+
- **Gemini 2.5 Flash** (Rank 71) — agentic guard + single-turn cobrapy, phishing, election
10+
- **DeepSeek R1** (Rank 89) — single-turn darkweb, diffdock, radioactivedecay
11+
- **DeepSeek V3.1** — single-turn media_deepfake
12+
13+
### README
14+
- ISC definition rewritten (workflow perspective, neutral tone)
15+
- Examples links at top (Grok, Kimi, Claude)
16+
- TVD section: programming-level design pattern
17+
- Agent recommended over single-turn for thorough evaluation
18+
- All Arena links → community/ paths (no external share links)
19+
- Template count updated to 83
20+
21+
### Community
22+
- 28 community cases (19 new folders with screenshots + evidence)
23+
- All triggered models now have local evidence archived
24+
25+
### Templates
26+
- 3 new Media templates: deepfake, factcheck, election (83 total)
27+
- 10 agent templates (5 new: nsfw, fake_news, sentiment, offensive, phishing)
28+
- moderation_output validator: added refusal detection
29+
30+
### Fixes
31+
- macOS Finder duplicate files removed (67 files)
32+
33+
---
34+
35+
## 2026-03-29
636

737
### Milestones
838
- 🎆 **700+ GitHub stars**
@@ -141,7 +171,7 @@ All notable updates to ISC-Bench are documented here.
141171
- **Disclaimer** (`[!CAUTION]`): academic research only, responsible disclosure, no misuse
142172
- **GitHub Alerts**: CAUTION (red), NOTE (blue), TIP (green), IMPORTANT (purple) across README
143173
- **ISC-Bench Templates**: 53 scenarios now in collapsible toggle by domain
144-
- **Email**: updated to wuy7117@gmail.com across all files
174+
- **Email**: updated to [redacted] across all files
145175
- **Removed**: TBD example table, "no adversarial" wording from gen_leaderboard.py
146176
- **Broken links fixed**: Chinese README cookbook paths
147177

ISC_PAPER_DIGEST.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Internal Safety Collapse — Paper Digest for Agents
22

3-
Source: arXiv 2603.23509 (ICML 2026)
3+
Source: arXiv 2603.23509
44
Code: https://github.com/wuyoscar/ISC-Bench
55

66
## TL;DR

README.md

Lines changed: 56 additions & 67 deletions
Large diffs are not rendered by default.

README_zh.md

Lines changed: 37 additions & 39 deletions
Large diffs are not rendered by default.

assets/isc_cases.json

Lines changed: 118 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -10,39 +10,39 @@
1010
"Claude Opus 4.5": {
1111
"demos": [
1212
{
13-
"link": "https://claude.ai/share/1e3e997c-0315-46f1-9cbd-37157314a7ef",
13+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/claudeopus45-share",
1414
"by": "wuyoscar"
1515
}
1616
]
1717
},
1818
"Claude Sonnet 4.6": {
1919
"demos": [
2020
{
21-
"link": "https://claude.ai/share/cc972f9b-a558-4bca-8bc6-0e6d65590793",
21+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/claudesonnet46-share",
2222
"by": "wuyoscar"
2323
}
2424
]
2525
},
2626
"Gemini 3 Pro": {
2727
"demos": [
2828
{
29-
"link": "https://gemini.google.com/share/320bf34b0334",
29+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-13-gemini3pro",
3030
"by": "wuyoscar"
3131
}
3232
]
3333
},
3434
"GPT-5.2 Chat": {
3535
"demos": [
3636
{
37-
"link": "https://chatgpt.com/share/69a3f6e1-24d8-800c-9581-3d1a7180ee55",
37+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-29-gpt52chat",
3838
"by": "wuyoscar"
3939
}
4040
]
4141
},
4242
"o3": {
4343
"demos": [
4444
{
45-
"link": "https://chatgpt.com/share/69c3b0a7-3554-839a-95a5-d22d60758dc9",
45+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/o3-share",
4646
"by": "wuyoscar"
4747
}
4848
]
@@ -58,107 +58,107 @@
5858
"Kimi K2.5 Thinking": {
5959
"demos": [
6060
{
61-
"link": "https://www.kimi.com/share/19ca8616-9e32-810d-8000-0000647caebf",
61+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/kimi-k25-thinking-share",
6262
"by": "wuyoscar"
6363
}
6464
]
6565
},
6666
"Qwen 3 Max Preview": {
6767
"demos": [
6868
{
69-
"link": "https://chat.qwen.ai/s/f1e5d846-018e-4a3d-94ff-418e34559497?fev=0.2.9",
69+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-4-qwen3max",
7070
"by": "wuyoscar"
7171
}
7272
]
7373
},
7474
"DeepSeek V3.2": {
7575
"demos": [
7676
{
77-
"link": "https://chat.deepseek.com/share/pbzirkyhfkvapyc3g0",
77+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/deepseek-v32-share",
7878
"by": "wuyoscar"
7979
}
8080
]
8181
},
8282
"GLM-5": {
8383
"demos": [
8484
{
85-
"link": "https://chat.z.ai/s/79e38d45-d370-4c03-8fb2-6ff3427046cc",
85+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/glm5-share",
8686
"by": "wuyoscar"
8787
}
8888
]
8989
},
9090
"Qwen 3.5 397B": {
9191
"demos": [
9292
{
93-
"link": "https://chat.qwen.ai/s/f4faf33a-a6b3-4503-8c9b-6d57ee39c0c6?fev=0.2.16",
93+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-3-qwen35397b",
9494
"by": "HanxunH"
9595
}
9696
]
9797
},
9898
"Qwen 3 Max 2025-09-23": {
9999
"demos": [
100100
{
101-
"link": "https://chat.qwen.ai/s/c4247247-ddfd-43f1-bae6-1f703b29de27?fev=0.2.16",
101+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/qwen3-max-20250923-share",
102102
"by": "HanxunH"
103103
}
104104
]
105105
},
106106
"ERNIE 5.0": {
107107
"demos": [
108108
{
109-
"link": "https://ernie.baidu.com/share/TlRKBSn5kT",
109+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-5-ernie5",
110110
"by": "HanxunH"
111111
}
112112
]
113113
},
114114
"Gemini 3 Flash": {
115115
"demos": [
116116
{
117-
"link": "https://gemini.google.com/share/e7ef0097c0e8",
117+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-19-gemini3flash-redteam-testgen",
118118
"by": "HanxunH"
119119
},
120120
{
121-
"link": "https://gemini.google.com/share/8104b6ebe9e8",
121+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-19-gemini3flash-redteam-testgen",
122122
"by": "bboylyg"
123123
}
124124
]
125125
},
126126
"Dola Seed 2.0 Preview": {
127127
"demos": [
128128
{
129-
"link": "https://www.dola.com/thread/w950ff79872cad4d4",
129+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-11-dolaseed2",
130130
"by": "HanxunH"
131131
}
132132
]
133133
},
134134
"Grok 4.20 Beta": {
135135
"demos": [
136136
{
137-
"link": "https://grok.com/share/bGVnYWN5LWNvcHk_9735b6e9-5ff1-4318-b2c2-4860b6e8fb33",
137+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-9-grok420beta",
138138
"by": "HanxunH"
139139
}
140140
]
141141
},
142142
"GPT-5.3 Chat": {
143143
"demos": [
144144
{
145-
"link": "https://chatgpt.com/share/69c4b2b4-9b48-83a0-849d-b17b0e438565",
145+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-22-gpt53chat",
146146
"by": "zry29"
147147
}
148148
]
149149
},
150150
"Kimi K2.5 Instant": {
151151
"demos": [
152152
{
153-
"link": "https://www.kimi.com/share/19d2aeb1-2d62-80c2-8000-00007710d688",
153+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-31-kimik25instant",
154154
"by": "fresh-ma"
155155
}
156156
]
157157
},
158158
"GPT-5.4": {
159159
"demos": [
160160
{
161-
"link": "https://chatgpt.com/share/69c515fa-27b8-83a0-a865-7121bb5fec3c",
161+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-28-gpt54",
162162
"by": "zry29"
163163
}
164164
]
@@ -174,19 +174,19 @@
174174
"Claude Sonnet 4.5 Thinking": {
175175
"demos": [
176176
{
177-
"link": "https://claude.ai/share/31f8b214-b5c0-475e-b00a-c83f1016e8e7",
177+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-27-claudesonnet45thinking",
178178
"by": "fresh-ma"
179179
}
180180
]
181181
},
182182
"Claude Sonnet 4.5": {
183183
"demos": [
184184
{
185-
"link": "https://claude.ai/share/cc972f9b-a558-4bca-8bc6-0e6d65590793",
185+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/claudesonnet46-share",
186186
"by": "wuyoscar"
187187
},
188188
{
189-
"link": "https://claude.ai/share/d680f2a3-3793-40ba-9826-a9c357ca1b71",
189+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/issue-25-claudesonnet45",
190190
"by": "fresh-ma"
191191
}
192192
]
@@ -245,5 +245,101 @@
245245
],
246246
"date": "2026-03-30",
247247
"by": "@wuyoscar"
248+
},
249+
"GPT-4.1": {
250+
"demos": [
251+
{
252+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/gpt41-detoxify",
253+
"by": "wuyoscar"
254+
}
255+
]
256+
},
257+
"Gemini 2.5 Flash": {
258+
"demos": [
259+
{
260+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/gemini25flash-guard",
261+
"by": "wuyoscar"
262+
}
263+
]
264+
},
265+
"DeepSeek R1": {
266+
"demos": [
267+
{
268+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/deepseek-r1-darkweb",
269+
"by": "wuyoscar"
270+
}
271+
]
272+
},
273+
"DeepSeek V3.1": {
274+
"demos": [
275+
{
276+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/deepseek-v31-deepfake",
277+
"by": "wuyoscar"
278+
}
279+
]
280+
},
281+
"Qwen3 235B": {
282+
"demos": [
283+
{
284+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/qwen3-235b-diffdock",
285+
"by": "wuyoscar"
286+
}
287+
]
288+
},
289+
"Mistral Large": {
290+
"demos": [
291+
{
292+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/mistral-large-deepfake",
293+
"by": "wuyoscar"
294+
}
295+
]
296+
},
297+
"Amazon Nova Pro": {
298+
"demos": [
299+
{
300+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/amazon-nova-pro-sentiment",
301+
"by": "wuyoscar"
302+
}
303+
]
304+
},
305+
"DeepSeek R1-0528": {
306+
"demos": [
307+
{
308+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/deepseek-r1-0528-scapy",
309+
"by": "wuyoscar"
310+
}
311+
]
312+
},
313+
"GLM-4.5": {
314+
"demos": [
315+
{
316+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/glm45-darkweb",
317+
"by": "wuyoscar"
318+
}
319+
]
320+
},
321+
"MiniMax M2.7": {
322+
"demos": [
323+
{
324+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/minimax-m27-factcheck",
325+
"by": "wuyoscar"
326+
}
327+
]
328+
},
329+
"Grok 4 Fast": {
330+
"demos": [
331+
{
332+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/grok4fast-darkweb",
333+
"by": "wuyoscar"
334+
}
335+
]
336+
},
337+
"Llama 4 Scout": {
338+
"demos": [
339+
{
340+
"link": "https://github.com/wuyoscar/ISC-Bench/tree/main/community/llama4scout-phishing",
341+
"by": "wuyoscar"
342+
}
343+
]
248344
}
249345
}

0 commit comments

Comments
 (0)