[AlertZero] Adding managed action workflows for issuing isolate-host, kill-process, suspend-process response actions

[ymao1]

Summary

Adding AlertZero managed system workflows for running 3 response actions: isolate-host, kill-process, suspend-process. The workflows dispatch the response action via a kibana.request step and retrieves the action ID from the API response, then polls for completion and returns a final status. Action metadata should conform to the specs merged in #290705. No error handling in these workflows as errors should bubble up and be handled in a common manner by the AlertZero proposal framework (#291038)

To Verify

1. Start ES and Kibana with the following FF:

xpack.alertzero.enabled: true
xpack.agenticInvestigations.enabled: true
uiSettings.overrides:
  workflows:ui:showManagedWorkflows: true

2. Run setup script node x-pack/solutions/security/plugins/security_solution/scripts/endpoint/run_endpoint_agent.js. This requires you to have Docker and Multipass
3. Get IDs of endpoint(s) that were enrolled and the PID or entity_id of some processes on the endpoint.
4. Try running the manual workflows to test the steps:

[kibanamachine]

Selected for Libra review

This PR was selected for Libra review as part of the temporary 50% trial.

To opt out permanently, remove the reviewer:libra label. It will not be added again to this PR.

[kibanamachine]

Libra found 2 issues.

Generated by Libra

[kibanamachine]

Libra found 1 issue.

Generated by Libra

[semd]

Could we create all the defend action workfows inside one directory (I used managed/definitions/alertzero/actions/defend/* in my PR) and so we can assign the defend-workflows team ownership to the whole directory instead of each workflow individually?

[ymao1]

Updated in 87f1fe7 and added the pre-flight privileges probe in 5302e96

[kibanamachine]

Libra found 1 issue.

Generated by Libra

[kibanamachine]

The new test suite never executes any of the three workflows; it only schema-validates and inspects parsed YAML, so regressions in runtime expression rendering, on-failure handling, loop behavior, or failure propagation can still pass while these destructive actions dispatch or report outcomes incorrectly. Add a workflow-engine/integration test with a mocked kibana.request that covers the preflight denial, pending-to-success polling, and wasSuccessful: false paths.

WorkflowSchema.safeParse(parse(yaml)) proves structural validity only. The remaining cases similarly search parsed step objects and compare strings rather than running the workflow control flow.

[tomsonpl]

Thanks for putting this together @ymao1 / @semd 🙇

Mostly questions here, since you two know the engine far better than I do. Context for where I'm coming from: I did a PoC of osquery in workflows a while back (#267972, still draft) and we spent a while on the "dispatch, then wait for completion" problem in the osquery-workflows-extensions RFC — so this is the same shape of thing I've been stuck on, not a critique of your approach.

Would createPollServerStepDefinition make sense here, or is YAML the right call?

Genuine question rather than a suggestion. createPollServerStepDefinition (workflows_extensions/server/step_registry/types.ts:313) looks like it covers what the while + wait loop is doing — start/poll, persisted state, ceilings, fixed/exponential policy — and evals is using it (evals/server/workflows/steps.ts:289). But you'd both know better than me whether it's a fit for managed workflows specifically, or whether keeping the poll visible in YAML is deliberate (inspectability of a high-impact action seems like a decent reason on its own).

The one thing that made me wonder: as I read it, durable suspend here depends on wait: 10s being above the in-process threshold in handle_execution_delay.ts, whereas the poll step forces task scheduling from attempt 2 regardless of interval. Is that right, and if so is it worth a comment pinning why 10s — or am I misreading how the nested wait behaves inside while?

For context on why I care about the answer rather than the outcome: the osquery runQueryAndWait verbs are still unbuilt, so whatever pattern you land on is probably the one I should follow. Happy to just adopt whatever you decide.

Two things I couldn't work out from reading

• emit_result reads steps.poll_status.output.data.*, but poll_status lives inside the while body (same in action_kill_process.yaml). Are loop-body outputs still addressable from outside after the loop completes? I saw an output_eviction suite in the engine tests and couldn't convince myself either way — if they're reclaimed, the four outputs would come back empty, and presumably you'd have hit that in testing already.
• The probe recovers the HTTP status via slice: 0, 8 compared against "HTTP 404". Is there a structured status on the kibana.request error you could read instead, or is string-matching the only option today?

On the license question from the thread

@dasansol92 — your update looks right to me, and I think there are three gates in play:

• canIsolateHost → Platinum+ (authz.ts:156)
• canKillProcess / canSuspendProcess → Enterprise (authz.ts:158-159)
• canAccessEndpointActionsLogManagement → canReadActionsLogManagement && isPlatinumPlusLicense (authz.ts:149), where canReadActionsLogManagement is itself Enterprise-gated (authz.ts:148)

If that's right, the probe seems to be earning its keep for isolate specifically — Platinum can isolate but can't read the action log, so a license check alone wouldn't catch it — while for kill/suspend it's closer to redundant since those are already Enterprise. Does that match how you read it?

Not for this PR — maybe worth noting on elastic/security-team#19292

source: workflows doesn't look expressible today. RESPONSE_ACTION_TYPE is ['automated', 'manual'] and it's inferred rather than stored — fetch_action_requests.ts:236-252 maps manual → no data.alert_id, automated → has one. A workflow-issued action carries no alert_id, so in Response Actions History it'd look identical to an analyst clicking Isolate. The approver's identity is captured but not the mechanism, which might matter for AlertZero attribution later. Entirely separate from this PR — just flagging in case it's not already tracked.

---

One caveat on the above: I read action_isolate_host.yaml and action_kill_process.yaml closely but skimmed action_suspend_process.yaml, and I haven't run anything locally — so treat the two questions above as "I couldn't tell from reading" rather than "this is broken."

[ymao1]

@tomsonpl Thanks for reviewing!

Would createPollServerStepDefinition make sense here, or is YAML the right call?

The workflows added in this PR perform all the steps of dispatching the response action and polling for completion using built-in workflow steps. I have also prototyped creating a custom workflow step for these actions in #290370 that uses createPollServerStepDefinition. I believe there are product and technical questions to work through on that PR (one custom workflow step vs one per action was raised as an example) so I've converted that PR back to draft for discussion. Once the custom workflow steps are created using createPollServerStepDefinition, we can update these managed workflows in this PR to use them and remove a lot of the logic. My preference would be to move all of the polling logic inside the custom workflow step, that is managed and owned by the team who knows the logic best. The 10s polling interval was not based on any previous knowledge of how long to wait for response actions to execute and I'm happy to take feedback as to polling interval and max duration for this managed workflow.

emit_result reads steps.poll_status.output.data.*, but poll_status lives inside the while body (same in action_kill_process.yaml). Are loop-body outputs still addressable from outside after the loop completes? I saw an output_eviction suite in the engine tests and couldn't convince myself either way — if they're reclaimed, the four outputs would come back empty, and presumably you'd have hit that in testing already.

Yes, loop body outputs are addressable from the outside after the loop completes. Eviction does not occur between the last poll iteration and the emit_result because they occur in the same task execution.

The probe recovers the HTTP status via slice: 0, 8 compared against "HTTP 404". Is there a structured status on the kibana.request error you could read instead, or is string-matching the only option today?

I will look into whether we can simplify this against a status code

If that's right, the probe seems to be earning its keep for isolate specifically — Platinum can isolate but can't read the action log, so a license check alone wouldn't catch it — while for kill/suspend it's closer to redundant since those are already Enterprise. Does that match how you read it?

If it's not required for kill/suspend process, I'm happy to remove this extra step.

[ymao1]

@tomsonpl Followups:

The probe recovers the HTTP status via slice: 0, 8 compared against "HTTP 404". Is there a structured status on the kibana.request error you could read instead, or is string-matching the only option today?

Here is the output of the probe_action_details step when there is an error. There is a JSON blob but it's stringified so we would still need to perform string manipulation to slice and then JSON parse the string. I think matching against HTTP 404 is sufficient for now.

{
  "error": {
    "type": "Error",
    "message": "HTTP 403: {\"statusCode\":403,\"error\":\"Forbidden\",\"message\":\"Endpoint authorization failure\"}"
  }
}

If that's right, the probe seems to be earning its keep for isolate specifically — Platinum can isolate but can't read the action log, so a license check alone wouldn't catch it — while for kill/suspend it's closer to redundant since those are already Enterprise. Does that match how you read it?

It looks like isolate host, process operations and response actions history are separate subfeature privileges underneath security so it seems possible there are custom roles that allow access to process operations without access to response actions history, regardless of license level, so keeping the pre-flight probe seems like the safer option. Please let me know if I've misunderstood this

[kibanamachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 68b4f03
• Build duration: 51 mins

Failed CI Steps

• FTR Configs #124
• Scout Lane #21 - stateful-classic / default
• Scout Lane #21 - stateful-classic / default
• Scout Lane #65 - serverless-observability_complete / default

Metrics [docs]

✅ unchanged

Test Failures

• [job] [logs] FTR Configs #124 / Detection Engine - Exception workflows APIs @serverless @serverlessQA @ess rule exceptions execution creating rules with exceptions should be able to execute against an exception list that does include valid case sensitive entries and get back 0 alerts
• [job] [logs] Scout Lane #65 - serverless-observability_complete / default / local-serverless-observability_complete - Synthetics manage rules privilege - lets a read user with can_manage_rules enable default alerting
• [job] [logs] Scout Lane #21 - stateful-classic / default / local-stateful-classic - Synthetics manage rules privilege - lets a read user with can_manage_rules enable default alerting
• [job] [logs] Scout Lane #21 - stateful-classic / default / local-stateful-classic - Synthetics manage rules privilege - lets a read user with can_manage_rules enable default alerting

History

• 💛 Build #502399 was flaky 5b34384
• 💔 Build #502388 failed 9099960
• 💛 Build #502136 was flaky 5c022c1

cc @ymao1

[paul-tavares]

LGTM 👍

[paul-tavares]

Just FYI: kill-descendants is a 9.6 feature.