Trace

Hack the North 2026's CSE challenge, Log & Order, supplied 180,800 Apache access-log lines from August 2025 to March 2026, with seven useful fields and no labels. Somewhere in March sat one real multi-stage attack: a credential brute force from an unfamiliar IP, a CSRF probe, a privilege escalation and an account takeover that exfiltrated data, 22 lines in all.

The background traffic was seeded with decoys (off-hours activity, weekend traffic, 403 probes, isolated failed logins), so simple rules drown in false positives. The brief was tooling a security analyst could actually use to find the incident and document it.

• Turned every log line into 15 causal, behavioural features rather than raw fields: how surprising this IP is for this user, how unusual the method and path template are, per-user denial rates on sensitive resources, recent 401s, off-hours flags. Statistics are fit on the training months only, and each row sees only the rows before it.
• Benchmarked a zoo of detectors (KMeans, Gaussian mixtures, Isolation Forest, One-Class SVM, LOF, HBOS, ECOD, COPOD, PCA, autoencoders, DeepSVDD and a no-ML weighted-surprisal rule) against a PyTorch autoencoder ensemble: seven seeded models with error-weighted reconstruction scores, rank-averaged.
• Chose hyperparameters and the alert threshold without touching the test month. Synthetic attacks mimicking the tactics were injected into the February validation window across five seeds; the 22 hand-labelled March rows were scored exactly once, with 1,000× bootstrap confidence intervals.
• Added a hybrid triage step: the Gaussian mixture's top-60 shortlist goes to GPT-5 together with each user's learned permission profile, which promotes confirmed lines and demotes cleared ones. Blind LLM baselines saw raw log windows with no profiles, for a fair comparison.
• Shipped an analyst dashboard, plain HTML, CSS and JavaScript on Vercel in front of a Flask API on Railway: upload logs, pick a detector, get candidate investigations grouped in time with five minutes of context either side per IP and account, explanation tags on every line, analyst notes, and an evidence report export with an executive assessment and one section per episode.

The autoencoder ensemble reached 0.918 PR-AUC and the Gaussian mixture 0.909, each surfacing 20 of the 22 attack lines in its top 50; the two are statistically tied, and the mixture trains in under a second on a CPU. Moving that same model from raw fields to the behavioural features lifted it from 0.61 to 0.91, which was the project's main lesson: the representation mattered more than the algorithm.

Blind GPT-5 and Claude baselines found most of the attack but flagged every authorised read of a confidential file, 225 and 614 false positives. Detector-guided triage removed every false positive at $0.08 a run. Trace took first place in the CSE challenge at Hack the North 2026.