Working a simulated SOC case with a bounded AI step

I use AI agents to take the slow, repetitive parts out of security orchestration, automation and response (SOAR) work, so my attention goes to the decision. This entry shows which tasks I hand to an agent, what each one produces, and the gate that stops an agent from acting on its own.

The case is simulated. The events, the playbook run and the model call are sample evidence built on the same simulated day as the rest of the console. I have not measured time saved, so there is no speed claim here; the method for measuring one is at the end.

What I hand to an agent

• Gathering and normalising evidence. Windows events and firewall records arrive in different shapes. The agent maps them into one schema so they can be compared at all.
• Grouping duplicates. Seven alerts about the same account and machine are one observation, not seven.
• Drafting the case note. A first draft with the facts, the open questions and the evidence references, which I then edit.
• Recommending a next step. A bounded choice with an uncertainty attached, never an action.
• Keeping the record honest. Outside this case, I keep my lab handoff notes with an assistant, and every fact is tagged confirmed, historical, proposed or unknown. That discipline is why this site can say which statuses come from a record.

What I do not hand over: closing a case, changing an account, isolating a machine, or deciding that two observations are the same incident.

The question

Seven failed logons hit the file server FS-01 within a few minutes, all for one account and all from the workstation WS-02. In the same quarter hour, Sophos denied WS-02 five times on its way to a SOC dashboard. Is this an attack, a mistake, or two unrelated things that happen to share a workstation?

The pipeline, stage by stage

{"ts":"2026-11-07T08:14:00Z","source":"windows","event":"logon_failed","host":"fs-01","account":"LAB\\student01","src_host":"ws-02","sub_status":"0xC000006A"}
{"ts":"2026-11-07T08:15:00Z","source":"windows","event":"logon_failed","host":"fs-01","account":"LAB\\student01","src_host":"ws-02","sub_status":"0xC000006A"}
… 10 more records

{
  "case": "SOC-0142",
  "choices": [
    "request context",
    "analyst review",
    "candidate for closure"
  ],
  "chosen": "request context",
  "uncertainty": "medium",
  "evidence": [
    "E1",
    "E3",
    "E4"
  ],
  "model": "Jev (simulated call)"
}

Where the agent saves time, and where I check the evidence

The pattern is the same in every row: the agent saves the typing and the sorting, and I keep the judgement and the verification. The last row stays empty on purpose.

The replay

What the playbook does

1. Collect and deduplicate. The 4625 events and the Sophos records arrive as separate alerts. The playbook folds the logon failures into one case and attaches account, endpoint, time and source context.
2. Ask a bounded question. It asks TypeSafe’s Jev to choose between typed options: request context, analyst review, or candidate for closure. The answer comes back with an uncertainty, and both are stored next to the evidence that produced them.
3. Apply explicit rules. Ambiguity, missing context, a timeout or conflicting evidence goes to a person. A recommendation to close only prompts an analyst. Disabling an account or isolating a host needs explicit approval.
4. Record the decision. The case keeps its ID, the evidence references, the playbook version, the model identifier, the recommendation, the human action and the outcome.

Why the firewall deny stays separate

The playbook can put the logon events and the firewall record in front of me. Jev can recommend where the case goes next. I still have to explain why a firewall deny belongs in the same case, or why I would leave it separate. Here it targets a different host and port, so I link it as its own observation and keep it out of the evidence for a password attack.

Try the investigation yourself

The terminal is a training room in the style of TryHackMe: fixed commands over the simulated data, and a decision at the end. It is deliberately separate from the replay so the guided version does not blur into the evidence.

How I would measure the time saved

Run the same five-case set twice: once by hand, once with the pipeline, starting a timer at the first alert and stopping at a disposition I would sign. Keep the cases, the fixture and the scoring rules public, count only my time, and report both medians with the spread. This page publishes no speed number, because none has been measured that way.

The Kali Red and Blue page shows where test activity like this comes from, and how Kali Blue’s evidence feeds the same kind of case.

What I learned

Working through SOC-0142 changed what I ask an agent for. I stopped asking it for answers and started asking it for well-sorted evidence and a bounded suggestion. The step that matters most, deciding whether the firewall deny is part of the story, is exactly the one the agent cannot make for me, because it depends on why WS-02 was talking to that dashboard.

Takeaway: let an agent gather, normalise, group and draft. Before signing anything, check one raw event against the agent’s version of it.

Further reading: TypeSafe’s introduction to Jev · Windows event 4625