!

Security Insight

Fighting the Slopdemic: Are AI‑generated junk reports slowing you down?

Architecture/Engines

Scoring engines

Three engines vote on every report. Each one looks at the body through a different lens, returns a 0–100 score, and is combined as a weighted sum. This page is the landing point for the Score Fusion stage of the pipeline — a single index of every engine, what it sees, what it can't see, and where it's known to misfire.

3 engines · weights sum to 100%
Engine 2 gates Engine 3 below substance 30
AVRI rubric layer sits behind E2 + E3
Three luminous glass pillars side by side — cyan (Linguistic Analysis), violet (Substance Analysis), amber (CWE Coherence) — beams from the top of each converging into a single point of white light above.
How the composite is built
composite = 0.05 × (100 − E1) + 0.60 × E2 + 0.35 × E3*
where E3* = E3 capped at 42 if E2 < 30, else E3

E1 (AI Authorship) is inverted — a high score means the prose looks machine-generated, which lowers the composite. E2 (Substance) carries the most weight because its signals are nearly impossible to fabricate. E3 (CWE Coherence) is gated by E2 so that naming the right CWE in an evidence-free report can't lift the composite into borderline territory.

Engine 1
AI Authorship
5%
0.05 of the composite

Detects machine-generated prose patterns — burstiness, vocabulary, repetitive scaffolding.

What it measures
  • Sentence-length burstiness vs. expert baseline
  • Vocabulary richness and rare-word density
  • Stock LLM scaffolding phrases ('It is important to note that…')
  • Inverted in the composite: high score → likely AI-written → penalised
What it can't see
  • A human reporter who writes blandly will look LLM-ish
  • An LLM polishing genuine technical content can score low
  • Non-native English writing is intentionally not penalised
Known false-positive patterns
  • Generated security templates from internal tooling
  • Reports translated through a model from another language
  • Highly repetitive but legitimate disclosures (mass-assignment lists)
Worked exampleRED
GPT-style write-up · no PoC
Burstiness 0.41 (baseline 0.78), 12 stock scaffolding hits, vocabulary entropy in the bottom decile.
→ 78 → contributes (100 − 78) × 0.05 = 1.1 to composite
Engine 2
Technical Substance
60%
0.60 of the composite — the dominant signal

Counts concrete, verifiable artifacts — code blocks, file paths, line numbers, real URLs, repro steps.

What it measures
  • codeEvidence (35%) — language-tagged fenced code that's actually copy-pasteable
  • references (30%) — file paths, line numbers, endpoints, version numbers
  • reproducibility (20%) — explicit steps, environment, real working URLs
  • pocIntegrity (10%) — does the claimed PoC actually exist in the body?
  • claimEvidence (5%) — ratio of asserted claims to evidence backing them
What it can't see
  • A bug whose evidence is binary or visual (a video repro, a memory dump)
  • Theoretical / design-flaw reports where there is no line number to point at
  • Any vulnerability that can only be described prose-first (cryptographic protocol flaws)
Known false-positive patterns
  • Hardware bugs and side-channel research often lack file paths
  • Race conditions where the only evidence is a timing trace
  • Reports that link to a private repo the engine cannot read
Worked exampleRED
Polished slop · CWE-89 cited · no PoC
0 code blocks · 0 file paths · 0 line numbers · 2 placeholder URLs · claim:evidence 11.0
→ 17 → contributes 17 × 0.60 = 10.2 and trips the substance gate
Engine 3
CWE Coherence
35%
0.35 of the composite, gated to ≤ 42 when Engine 2 is below 30

Checks whether the body actually describes the CWE the title claims.

What it measures
  • Term-frequency classifier across the full CWE catalogue
  • Soft citation: which CWE the body actually sounds like
  • Parent ↔ child relationships count as soft matches, not mismatches
  • Gated by Engine 2 — naming the right CWE is cheap when there is no evidence
What it can't see
  • Chained bugs that legitimately cross CWE boundaries (SSRF → RCE)
  • Vendor-specific taxonomies that don't map cleanly to CWE language
  • Reports too short for the classifier to commit to any CWE — abstains rather than guesses
Known false-positive patterns
  • A genuine cross-CWE chain where the title picks one link and the body emphasises another
  • Family vs. specific drift (CWE-79 cited, CWE-80 described)
  • In-house bug taxonomies translated into CWE numbers post-hoc
Worked exampleRED
Wrong CWE · genuine bug
Title claims CWE-89 (SQLi). Body, repro, and PoC all describe CWE-77 (command injection).
→ raw 31 → no gate fires → contributes 31 × 0.35 = 10.9; soft citation flags CWE-77
Rubric Layer
AVRI — Adaptive Vulnerability Rubric Inference

Family-aware rubric layer that sits behind Engines 2 and 3. Picks a weakness family first, then judges the report against the rubric for that family rather than one flat checklist.