Skip to main content
PluginsThreat-Classifier
ML Task · Threat regression

Threat-Classifier regression

Fuses GuardDuty, SaaS telemetry, and analyst feedback into an adaptive regression bundle. Scorecards publish directly into Security Hub and EventBridge so incidents stay explainable and approvals remain immutable.

Threat intelligence regressionLatency SLA: 250ms P95Auto-tuned retraining
Latency P95236 msMeasured across 4 VPC endpoints with concurrency guards and warm pool rotation.
Analyst override drop↓ 72%Overrides fell from 89/day to 25/day once the feedback loop landed in production.
SOC throughput lift× 3.4Triage runbooks close in a single pass with score explanations embedded in tickets.
Precision lift against analyst feedback

Analyst adjudications land in under 15 minutes, keeping regression weights honest. Blue area shows lift over the legacy scoring model while dots track human feedback.

Threat-Classifier regression precision vs baselineSprint 0Sprint 1Sprint 2Sprint 3Sprint 4Sprint 50.400.500.600.700.80Sprint cadencePrecision (↑ better)
Adaptive regression liftLegacy scoring envelopeAnalyst override trend
Week 0
Shadow scoring launched
Mirrors GuardDuty severity, emitting suppressed findings into Security Hub for analyst-only review.
Week 2
Feedback captured as data
Analyst adjudications are streamed through EventBridge within 12 minutes to hydrate retraining windows.
Week 4
Drift alarm triggered
RMSE threshold breach auto-quarantines stale enrichers and spins a SageMaker retrain in isolation.
Week 6
Blue/green promotion
New weights graduate through canary accounts with immutable manifests and dual approval in CodePipeline.
Week 8
Explainability shipped to execs
Regression deck narrates feature lift, override rate, and residual risk for security leadership.

Explainable triage, not black-box lifts

Daily regression reports pair precision lift with override deltas so SOC leads can defend actions in incident postmortems before promoting new weights.

  • Top lift contributor
    +18% from SaaS session linking
    Okta risk signals fused with GuardDuty findings suppressed MFA fatigue false positives during red-team spikes.
  • Counter-signal quarantined
    -6% from stale asset tags
    Deprecated CMDB tags triggered auto-curation to quarantine the feature before it polluted the next retrain window.

Promotion stays inside IaC guardrails

Every retrain writes an immutable manifest to the registry; CodePipeline gates demand drift diffs plus approval before SageMaker endpoints flip to the new bundle.

  • Guardrail
    Manifest hash verification
    Deployment halts if the feature manifest hash diverges from the approved artifact, protecting downstream IaC rollbacks.

Latency budgets stay predictable

Concurrency windows and warm pool orchestration keep P95 under 250ms even as we blend SaaS enrichers, analyst overrides, and managed threat feeds.

  • Observation
    EventBridge fan-out throttling
    Adaptive concurrency halved queue spikes during chaos drills without dropping recall or breaking cost envelopes.