Tech

Machine learning as an on-box inspector

xorappsec uses ML on the box. It does not call a remote AI API. Every request is decoded, turned into features, and scored inside the xorappsec process. This page is the illustrated version of docs/architecture and docs/ml.

Three inspection phases — Decode → indicators → contextual fusion. Complementary engines run first and only Prevent short-circuits.

The four tasks in one binary

Phase 1 — decode

File: crates/xorsec-inspect/src/decode.rs. URL, HTML entities, hex, Base64, JWT claims, JSON/XML flatten, multipart, and ${…} lookup expansion (including ${::-j}). XML external entities are never fetched. If a payload is still opaque after this step, later phases cannot see it.

Phase 2 — indicators

File: indicators.rs plus ngram.rs. Families (SQLi, XSS, RCE, JNDI, Spring, …) are scores, not drops. Dual NORM+SIG trigrams boost a family when structural patterns survive letter substitution. Origin/Referer/Host/Cookie are skipped so a browser Origin is not treated as SSRF.

Phase 3 — context

File: ml.rs. Two models plus fusion:

Supervised — 32-d logistic. Feature 31 is always 1.0 (bias). Public basic weights ship in the binary; --supervised-model loads a replacement.
Unsupervised — per-asset URL baselines (templated paths so /users/42 shares a model with /users/99) plus an Isolation Forest fitted on benign traffic only.
Reputation — EWMA per source identity.

score = (0.72·supervised + 0.14·unsupervised + 0.08·(1-reputation) + 0.06·indicator)
        × false_detection × crowd
if indicator ≥ 0.93: score = max(score, 0.93)   # JNDI / Spring class
if supervised ≥ 0.90: score = max(score, 0.78)

Those floors are why a Kindergarten asset still blocks decoded Log4Shell.

Learning levels

Level	Typical gate	Recommendation
Kindergarten → High School	volume + time	Keep learning
Bachelor	~5k+ requests	Review tuning suggestions
Graduate	mature, no trusted sources yet	Prevent critical
Master / PhD	trusted sources + tuning	Prevent high and above

Trusted sources are not an allow-list. They accelerate the benign baseline after minNumOfSources distinct identities agree.

What stays local

Request bodies and keys never leave the process except what you enable in a log trigger. Training of the supervised vector happens in the private xorappsec-ml lab and is loaded as a JSON artifact. Online unsupervised state is data/unsupervised_model.json on the box.

Operator ML notes Architecture docs