[Threat Intel 6/11] Enforce the indicator read contract with a filtered alias by stephmilovic · Pull Request #287202 · elastic/kibana

stephmilovic · 2026-08-25T22:31:32Z

Summary

Adds the filtered per-space indicator alias over .threat-intel-indicators. The alias enforces space_id and precision ioc_tier filters so Indicator Match rules do not rely on query-time discipline alone.

Where this sits

#	PR	Depends on	Status
1	#287197 workflow gate correction + fixtures	nothing	✅ merged
2	#287198 contracts, constants, shared libs	nothing	✅ merged
3	#287199 content parsing	#287198	closed
4	#287200 SSRF-guarded HTTP client	#287198	✅ merged
5	#287201 index templates, seeding, inference features	#287198, #287200	✅ merged
6	#287202 indicator alias ← this PR	#287198, #287201	✅ merged
7	#287203 IOC extraction	#287198, #287200	✅ merged
8	#287204 LLM services, routes, source catalog API	#287198, #287200, #287202, #287203	✅ merged
9	#287205 RSS adapter	#287198, #287200, #287203, #287204	✅ merged
10	#287206 remaining adapters, dispatcher, fetch_source step	#287198, #287200, #287203, #287205	✅ merged
11	#287207 promote/scrub tasks and plugin wiring	#287198–#287206	✅ merged
12	#287479 generator fixture article URLs	nothing	✅ merged

#287479 came first in the merge train and is already on main. The numbered series (#287197–#287207) stacks on top in dependency order. #287199 (content parsing) was closed; IOC extraction ships its own section-header classifier.

Scope review follow-up

Preserves the corrected .threat-intel-indicators namespace (not .kibana-threat-intel-indicators).

Product boundary

The alias narrows what a reader sees. It does not grant read access by itself. threatIntelSupplyEnabled stays off until direct-index cross-space isolation is hardened.

To test

node scripts/jest --config x-pack/solutions/security/plugins/security_solution/server/threat_intel/jest.config.js setup/indicator_alias.test.ts

PR developed with Cursor + Auto

…ed alias

…ant them The doc comment presented the filtered alias as making an index safe for an Indicator Match rule to point at. That holds for the two things the alias filters, space and tier, and says nothing about the third thing standing between a rule and the data. An Indicator Match rule reads its threat index as the rule's own API key, which inherits the author's Elasticsearch roles rather than their Kibana feature privileges (`create_threat_signals.ts` uses `asCurrentUser` plus `openPointInTime`). Nothing in this plugin grants an index privilege on the indicator index or its aliases, so a rule pointed at one fails for every non-superuser until an operator grants `read` by hand. That is a gap in the pipeline rather than a property of the alias, and the alias is where an operator meets it, so it is documented here. Whether it stays a documented manual grant or becomes a feature-privilege change is a product decision, not one to settle in a comment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…space can grant its per-space aliases to Detection Engine rules. This updates the alias module to match, and rewrites the access note to say where the grant now comes from rather than describing it as an unfilled gap. The test derives the index name from the constant instead of hardcoding it, so it follows any future change. One property is pinned separately: the alias must stay outside `.kibana`. That is what makes the grant possible, and nothing else in the suite would catch a name drifting back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

infra-vault-gh-plugin-prod · 2026-09-02T20:39:34Z

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

infra-vault-gh-plugin-prod · 2026-09-02T20:39:36Z

Pinging @elastic/security-solution (Team: SecuritySolution)

stephmilovic · 2026-09-02T21:21:32Z

@elasticmachine merge upstream

aliasFilterMatches compared only the first space_id and ioc_tier terms clauses and never asserted the clause count, so a stored filter with a third terms clause or a duplicate space_id was reported as matching and never repaired. It now requires exactly the two recognized terms clauses, classified independently so an empty or duplicate clause cannot be chosen ahead of the real one. Co-authored-by: Cursor <cursoragent@cursor.com>

kibanamachine · 2026-09-03T15:35:25Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: a4e599d
Build duration: 63 mins
Storybooks Preview

Failed CI Steps

FTR Configs #84

Metrics [docs]

✅ unchanged

Test Failures

[job] [logs] FTR Configs #84 / console app text input with a data URI in the load_from query loads the data from the URI

History

💛 Build #495836 was flaky 8fa5748

…ed alias (elastic#287202)

…og API (#287204) ## Summary Adds LLM enrichment services and Threat Intelligence HTTP routes, including the fixed-catalog source list and enable/disable API. Manual report creation and provenance URLs go through the shared HTTP/HTTPS normalizer. Also wires the routes, inference features, and one-time bootstrap into `plugin.ts`, gated by `threatIntelSupplyEnabled`. This wiring is interim: it exists so the enrichment eval suite (see Evaluation below) has routes to call ahead of the full pipeline wiring. `#287207` replaces it with a proper `wiring.ts` module; that PR's description has a rebase note covering the `plugin.ts` conflict resolution. ## Where this sits | # | PR | Depends on | Status | |---|---|---|---| | 1 | #287197 workflow gate correction + fixtures | nothing | ✅ merged | | 2 | #287198 contracts, constants, shared libs | nothing | ✅ merged | | 3 | #287199 content parsing | #287198 | closed | | 4 | #287200 SSRF-guarded HTTP client | #287198 | ✅ merged | | 5 | #287201 index templates, seeding, inference features | #287198, #287200 | ✅ merged | | 6 | #287202 indicator alias | #287198, #287201 | ✅ merged | | 7 | #287203 IOC extraction | #287198, #287200 | ✅ merged | | 8 | #287204 LLM services, routes, source catalog API **← this PR** | #287198, #287200, #287202, #287203 | 👀 ready | | 9 | #287205 RSS adapter | #287198, #287200, #287203, #287204 | draft | | 10 | #287206 remaining adapters, dispatcher, fetch_source step | #287198, #287200, #287203, #287205 | draft | | 11 | #287207 promote/scrub tasks and plugin wiring | #287198–#287206 | draft | | 12 | #287479 generator fixture article URLs | nothing | ✅ merged | | 13 | #289343 Scout API tests for the deterministic routes | #287204 | draft | | 14 | #289345 enrichment eval suite | #287204 | draft | #287479 came first in the merge train and is already on `main`. The numbered series (#287197–#287207) stacks on top in dependency order. #287199 (content parsing) was closed; IOC extraction ships its own section-header classifier. ## Scope review follow-up - Shared `normalizeProvenanceUrl` helper (HTTP/HTTPS only, strips credentials, bounded length). - Source create/delete routes removed. Updates may change only `enabled`. - Lists and mutations reject catalog IDs outside the approved set. - Ensures the per-space indicator alias on `list_sources` (idempotent). ## Evaluation A `@kbn/evals` suite exercises all four enrichment stages against the BlackHat demo pack article text, posting each input directly to the internal route and scoring the structured response. It runs as a tracked baseline (scores recorded, no build-failing thresholds yet). Latest full run, EIS Claude Sonnet 4.6 as model and judge, 1 repetition, 20 examples: | Stage | Evaluator | Score | |---|---|---| | assess_relevance | IsIntelligenceMatch | 1.00 | | assess_relevance | RelevanceShapeValid | 1.00 | | classify_severity | SeverityExactMatch | 0.83 | | classify_severity | SeverityWithinOneLevel | 1.00 | | enrich_taxonomy | CategoryRecall | 0.80 | | enrich_taxonomy | RegionRecall | 1.00 | | extract_diamond | DiamondNoIocLeak | 1.00 | | extract_diamond | DiamondSignalCount | 1.00 | | extract_diamond | criteria (LLM judge, majority vote) | 1.00 | The suite ships in #289345, a separate eval-only PR that stacks on this one (it needs the routes and the flag-gated wiring added here). The four deterministic routes are covered separately by Scout API tests in #289343. Any prompt or calibration fix the evals surface lands by amending this PR, not the eval PR. ## To test ``` node scripts/jest --config x-pack/solutions/security/plugins/security_solution/server/threat_intel/jest.config.js routes/list_sources.test.ts services/provenance_url.test.ts ``` _PR developed with Cursor + Auto_ --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Jon Wålstedt <jon.walstedt@elastic.co> Co-authored-by: Cursor <cursoragent@cursor.com>

…ed alias (elastic#287202)

…d the flag (#287207) ## Summary Wires Threat Intelligence behind `threatIntelSupplyEnabled`: bootstrap, managed workflow installation, promote/scrub tasks, and route registration. Promote mirrors extracted IOCs into `.threat-intel-indicators` for Indicator Match rules. ## Where this sits | # | PR | Depends on | Status | |---|---|---|---| | 1 | #287197 workflow gate correction + fixtures | nothing | ✅ merged | | 2 | #287198 contracts, constants, shared libs | nothing | ✅ merged | | 3 | #287199 content parsing | #287198 | closed | | 4 | #287200 SSRF-guarded HTTP client | #287198 | ✅ merged | | 5 | #287201 index templates, seeding, inference features | #287198, #287200 | ✅ merged | | 6 | #287202 indicator alias | #287198, #287201 | ✅ merged | | 7 | #287203 IOC extraction | #287198, #287200 | ✅ merged | | 8 | #287204 LLM services, routes, source catalog API | #287198, #287200, #287202, #287203 | ✅ merged | | 9 | #287205 RSS adapter | #287198, #287200, #287203, #287204 | ✅ merged | | 10 | #287206 remaining adapters, dispatcher, fetch_source step | #287198, #287200, #287203, #287205 | ✅ merged | | 11 | #287207 promote/scrub tasks and plugin wiring **← this PR** | #287198–#287206 | ready for review | | 12 | #287479 generator fixture article URLs | nothing | ✅ merged | | 13 | #289343 Scout API tests for the deterministic routes | #287204 | ready for review | | 14 | #289345 enrichment eval suite | #287204 | ready for review | | 15 | #290144 read APIs, readiness, space-keyed attribution | #287207 | draft | #287479 came first in the merge train and is already on `main`. The numbered series (#287197–#287205) is merged; #287206 and #287207 now sit directly on `main`. #287199 (content parsing) was closed; IOC extraction ships its own section-header classifier. ## Scope review follow-up - Installs the managed workflows with the right space topology: `attribute_alerts_to_reports` installs per space (it queries `.alerts-security.alerts-*` and writes per-space hit totals, so a global install would clobber every space's totals onto one shared doc), while `ingest_threat_feeds` and `enrich_threat_report` install once globally. Alert analysis and threat intel share a single `ready()` call so neither install set is dropped, and spaces created after boot are reconciled on the promote task. All three ship `enabled: false`, and enrich routes its HTTP calls through a fixed space (`default`) instead of the install-time `workflow.spaceId` (which is `'*'` globally). - Ensures the default-space indicator alias on start. - Chunks promotion bulk writes and treats HTTP 408/500 as retryable bulk failures. - Sanitizes provenance and extracted IOC reference URLs during promotion. - Documents direct-index cross-space isolation as the blocker to enabling the feature. ## Bootstrap fix folded in from #289343 `seed_default_sources.ts` sorted the legacy-source disable scan on `_id`, which Elasticsearch rejects with `illegal_argument_exception: Fielddata access on the _id field is disallowed` unless `indices.id_field_data.enabled` is set, so it failed bootstrap on a stock cluster. Found this while getting #289343's Scout suite green and moved the fix here since this PR is the one wiring up bootstrap and already needs `security-threat-hunting` review, so it isn't adding a reviewer #289343 wouldn't otherwise need. ## Product boundary `threatIntelSupplyEnabled` defaults to false. Do not enable until direct-index cross-space isolation is hardened or the administrator trust model is explicitly accepted. ## To test ``` node scripts/jest --config x-pack/solutions/security/plugins/security_solution/server/threat_intel/jest.config.js wiring.test.ts tasks/promote_threat_indicators.test.ts node scripts/jest x-pack/solutions/security/plugins/security_solution/server/workflows/ node scripts/jest src/platform/packages/shared/kbn-workflows/managed/definitions/threat_intel/ ``` _PR developed with Cursor + Auto + Sonnet 5 + Opus 4.8_ --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Jon Wålstedt <jon.walstedt@elastic.co> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

stephmilovic force-pushed the threat-intel-4c-indicator-alias branch 9 times, most recently from c41bcff to 2a154c6 Compare August 26, 2026 20:45

stephmilovic mentioned this pull request Aug 27, 2026

[Threat Intel] Move the indicator index out of the .kibana namespace #287479

Merged

jonwalstedt force-pushed the threat-intel-4c-indicator-alias branch from cf6961d to fafd6a5 Compare August 28, 2026 19:59

stephmilovic force-pushed the threat-intel-4c-indicator-alias branch 2 times, most recently from 2b74889 to 8b76a42 Compare September 2, 2026 00:13

stephmilovic and others added 2 commits September 2, 2026 14:33

feat(threat-intel): enforce the indicator read contract with a filter…

07a7a4c

…ed alias

stephmilovic force-pushed the threat-intel-4c-indicator-alias branch from 8b76a42 to e17f264 Compare September 2, 2026 20:38

stephmilovic marked this pull request as ready for review September 2, 2026 20:39

stephmilovic requested a review from a team as a code owner September 2, 2026 20:39

Merge branch 'main' into threat-intel-4c-indicator-alias

8fa5748

jonwalstedt reviewed Sep 3, 2026

View reviewed changes

Comment thread ...ck/solutions/security/plugins/security_solution/server/threat_intel/setup/indicator_alias.ts Outdated

Comment thread ...ck/solutions/security/plugins/security_solution/server/threat_intel/setup/indicator_alias.ts

jonwalstedt approved these changes Sep 3, 2026

View reviewed changes

stephmilovic merged commit 74ac798 into elastic:main Sep 3, 2026
44 of 45 checks passed

juliaElastic pushed a commit to juliaElastic/kibana that referenced this pull request Sep 4, 2026

[Threat Intel 6/11] Enforce the indicator read contract with a filter…

0fccdd0

…ed alias (elastic#287202)

This was referenced Sep 4, 2026

[Threat Intel] Scout API tests for the deterministic enrichment routes #289343

Merged

[Threat Intel] Enrichment eval suite for the LLM-backed routes #289345

Merged

Supplementing pushed a commit to Supplementing/kibana that referenced this pull request Sep 8, 2026

[Threat Intel 6/11] Enforce the indicator read contract with a filter…

372d6d0

…ed alias (elastic#287202)

Navigation Menu

Uh oh!

[Threat Intel 6/11] Enforce the indicator read contract with a filtered alias - #287202

[Threat Intel 6/11] Enforce the indicator read contract with a filtered alias#287202
stephmilovic merged 5 commits into
elastic:mainfrom
stephmilovic:threat-intel-4c-indicator-alias

stephmilovic commented Aug 25, 2026 •

edited by jonwalstedt

Loading

Uh oh!

infra-vault-gh-plugin-prod Bot commented Sep 2, 2026

Uh oh!

infra-vault-gh-plugin-prod Bot commented Sep 2, 2026

Uh oh!

stephmilovic commented Sep 2, 2026

Uh oh!

Uh oh!

Uh oh!

kibanamachine commented Sep 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

stephmilovic commented Aug 25, 2026 • edited by jonwalstedt Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Where this sits

Scope review follow-up

Product boundary

To test

Uh oh!

infra-vault-gh-plugin-prod Bot commented Sep 2, 2026

Uh oh!

infra-vault-gh-plugin-prod Bot commented Sep 2, 2026

Uh oh!

stephmilovic commented Sep 2, 2026

Uh oh!

Uh oh!

Uh oh!

kibanamachine commented Sep 3, 2026

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Test Failures

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

stephmilovic commented Aug 25, 2026 •

edited by jonwalstedt

Loading