The label AIOps for Security has settled, over the past year, on a wave of tools that score security alerts with machine learning, investigate suspicious emails with language models, and — in a few products — close tickets without waiting for a person. Unlike most technology names a hospital IT team encounters, this one carries no standards body and no founding paper to cite. It is editorial shorthand that stuck because procurement needed something to call the category. That vacuum is exactly why beliefs harden into facts before anyone checks them. Below are six beliefs we keep hearing in hospital IT corridors, each set against what the published evidence actually shows.
The stakes are not abstract. A hospital’s security queue resembles a differential diagnosis: most abnormal findings are benign, the dangerous findings are rare, and the cost of a wrong call is asymmetric. Tools that promise to automate the queue are therefore making claims about miss rates, not merely about workload. Those claims deserve the same scrutiny we apply to a diagnostic test’s sensitivity and specificity before it changes a care pathway.
Belief one: an AIOps for Security flag means an attack is underway
Anomaly detection sits at the core of the category, usually sold as behaviour analytics — UEBA, in vendor shorthand. The underlying assumption is that unusual equals hostile. Operational experience says otherwise: the majority of anomalies trace back to mundane causes, a system change nobody logged, a new hire whose access pattern looks odd for two weeks, a scheduled job that quietly moved its window. Rare events are also fiendishly hard to score, a point researchers have pressed for over a decade; Sommer and Paxson put it plainly in 2010 when they noted that machine learning stayed “rarely employed” in production detection, precisely because wrong answers cost so much more than blanks. A false positive wastes an analyst’s evening. A false negative admits an intruder. Treating every flag as an attack over-weights the first error and quietly ignores the second.
Belief two: the automation percentage is the scoreboard
“Percent automated” is the number vendors lead with, and the worst number to manage toward. The moment a team is measured on it, every incentive points toward closing cases fast rather than closing them correctly. Nobody tracks what the machine got wrong, because the wrongness is invisible by design — the ticket simply never reached a human. The evidence-led alternative is to measure what the system misses, per category, against blind labels, and to publish that number internally with the same prominence as the workload saved. Automation percentage measures throughput. Miss rate measures safety. Only one of them belongs on a dashboard that governs patient-facing risk, and it is telling that the teams who publish both numbers usually discover the second one makes the first look trivial.
Belief three: accuracy earned elsewhere applies here
Vendor case studies quote accuracy figures from environments with different mail flow, different base rates, different adversaries. None of it transfers, because detection performance is a function of the data it was measured on. Gartner has a name for the softer version of this problem — “AI washing”, where capability claims outrun the evidence behind them. There is also a reliable tell: when a published score sits near-perfect on a genuinely hard problem, that usually signals bias in the evaluation, not brilliance in the model. Security machine-learning research has documented the pattern repeatedly; the TESSERACT line of work between 2019 and 2022 showed F1 scores around 0.99 deflating once spatial and temporal biases in the test setup were corrected.
Belief four: one pooled score tells the whole story
Aggregate accuracy is where weak systems hide. Pool every alert category together and a tool can post a comfortable 1% overall error rate while failing one category outright — the reference below documents cloud-identity misses running at 42.3% while the phishing error rate sat at 9.1%. Averaging across categories buries the failure exactly where the consequences concentrate. The fix is unglamorous: report every metric per category, and never let a blended number into a governance conversation. If a vendor cannot produce per-category rates on data shaped like yours, you are not evaluating a system; you are evaluating a sales team.
Belief five: a spotless record proves an AIOps for Security agent is safe
This one fools careful people, because it dresses itself up as diligence. Suppose an agent closed 1,845 cases last quarter and your assurance program sampled 5% of those closes — roughly ninety-odd tickets — and found zero misses. Intuition says clean bill of health. Arithmetic says the sample still bounds the true miss rate near 38%. The rule of three does the work here: zero errors in n cases bounds the underlying rate at roughly 3/n, and a small n leaves an enormous ceiling. Certifying a 2% miss rate demands 236 cases once one miss appears in your sample; two misses push the demand to 313. Random spot-checks of closed tickets are weak assurance by construction. What works is adversarial instead: plant canary cases — the common prescription is fifty per quarter — and treat three closed canaries as grounds to withdraw the closing privilege immediately, with no appeals process.
Belief six: prompt injection is a filtering bug someone will patch
Agents that read inbound email are reading attacker-controlled content, and the UK’s National Cyber Security Centre has been direct about what that means: a model placed in this position becomes what their guidance calls an “inherently confusable deputy” — attentive, eager, and unreliable — and prompt injection may never be fully mitigated. The guidance, published in December 2025, explicitly rejects the analogy to SQL injection — you cannot treat this as ordinary input validation, and deny-lists lose to infinite rephrasing. The workable controls are deterministic rather than clever: restrict what the agent may touch, keep its permissions at least privilege, and never let an email-reading agent hold an outbound channel worth hijacking.
What to believe instead about AIOps for Security
None of this argues against the category; it argues against believing it wholesale. The pragmatic stance borrows from clinical practice — grant autonomy in grades, matched to evidence, and keep a human evaluating the machinery’s output, a point MITRE’s guidance makes plainly. In day-to-day terms that means:
- Count the base rate per alert category before scoring any vendor claim, because rarity caps precision no matter how good the model is.
- Shadow every candidate with blind labels before it touches a live decision, and keep those labels sealed from the tool’s own training loop.
- Set miss gates in writing per action class — recommending, auto-closing, acting — and retire the privilege loudly when a canary trip breaks a gate.
- Re-test after every material change, because a green light signed off in March says nothing about the adversary you face in October.
The first quarter of that discipline is modest and cheap: export your team’s past close decisions and pick two categories to measure properly; run the base-rate arithmetic and start scoring candidates in shadow with blind labels; then set the gate values in writing, plant the first canary batch, and put a cost figure next to the hours the queue consumed. Maturity, in this field, is measured by how quickly a claimed number gets replaced by one you computed yourself.
For a structured map of the whole field — the autonomy levels, the evidence gates, the evaluation methods — xOps School maintains a reference page on AIOps for Security that we hand to hospital IT leads who want the detail without the brochure language. Teams already running monitoring stacks will find the operational side familiar; our earlier walkthrough of AIOps observability, from monitoring through automated remediation, covers that adjacent ground. The telehealth follow-up model we described is also a useful mental template here — it is, in effect, supervised delegation with scheduled review, which is precisely how a cautious team should hand decisions to software; the same pattern sits behind remote follow-up care in cardiology. And if the skills gap is the real blocker, practical automation training remains the shortest route to a team that can run these evaluations itself.
Six beliefs, six corrections. The category will mature — evaluation discipline is already dragging it there, one embarrassing benchmark at a time — but the burden of proof sits with the system that wants to act, never with the queue it wants to close.