Pattern Detection

Pattern Detection

When to use this skill

• The main job is finding repeated shapes or suspicious irregularities, not fully explaining or fixing them yet.
• The user needs a reusable scan, shortlist, grouped candidate set, or anomaly triage across code, logs/events, telemetry, or metric tables.
• The prompt is really asking "what patterns keep showing up?", "what looks off?", or "what should we inspect first?" even if it never says "pattern detection".
• The user needs confidence notes, false-positive risks, or the fastest validation step, not a raw wall of hits.
• The request mentions patterns, anomalies, outliers, repeated failures, suspicious spikes, odd cohorts, rule packs, signatures, code smells, anti-patterns, fraud signals, or recurring issue families.

Do not use this skill as the main workflow when:

• The real job is root-cause incident reconstruction from logs or traces → log-analysis
• The real job is decision-ready KPI explanation, experiment analysis, or stakeholder reporting → data-analysis
• The real job is repo tracing, ownership lookup, or metric-definition search → codebase-search
• The real job is security remediation, policy design, or vulnerability hardening → security-best-practices or code-review
• The real job is alert tuning, telemetry coverage, or incident operations → monitoring-observability

Core idea

pattern-detection should behave like a detection packet router, not a giant bag of regexes.

1. Normalize the prompt into one primary detection packet. 2. Pick the cheapest evidence mode that can surface trustworthy candidates. 3. Return grouped findings with confidence and false-positive notes. 4. Give one fastest validation step. 5. Route out aggressively once the bottleneck becomes diagnosis, explanation, remediation, or operations.

Read these support docs before choosing the packet:

• references/intake-packets-and-route-outs.md
• references/detection-modes.md
• references/confidence-and-false-positive-checklist.md

Instructions

Step 1: Normalize the request

Convert the prompt into this intake shape first:

pattern_detection_packet:
  primary_packet: text-prefilter | structural-code-rule | log-event-pattern | metric-anomaly
  evidence_shape: code | configs | logs | events | telemetry | tables | time-series | mixed | unknown
  detection_goal: repeated-shape | risky-structure | noisy-family | suspicious-spike | drift | outlier | rule-draft | shortlist
  grouping_key: file-path | symbol-shape | error-family | event-name | build-version | environment | browser-device | cohort | metric-segment | unknown
  trust_risk: broad-match | parser-drift | instrumentation-change | denominator-gap | seasonality | sample-size | none | mixed | unknown
  route_after: stay-here | log-analysis | data-analysis | codebase-search | security-best-practices | monitoring-observability | code-review

Choose one primary packet for the run. If two are plausible, pick the one that reduces uncertainty fastest.

Step 2: Choose the packet

Packet rules:

• Prefer text-prefilter when the fastest win is to narrow the search space cheaply.
• Prefer structural-code-rule when plain text matching is too noisy or misses true code shape.
• Prefer log-event-pattern when volume, spread, parser quality, or cohort grouping matters more than single-line reading.
• Prefer metric-anomaly when the user cares about spikes, drift, cohorts, or time-window irregularities.

Step 3: Narrow the evidence before claiming a pattern

Apply at least one narrowing move before interpreting results:

• limit by file/path/language scope
• limit by time window, deploy window, or experiment window
• group by one stable key: message family, event name, exception class, browser/device, build, cohort, metric segment
• separate one noisy source from a broad multi-source spread
• compare a suspicious window with a baseline window
• note whether parser, schema, or instrumentation drift could dominate the signal

Useful heuristics by packet:

• text-prefilter → counts + representative examples before deeper claims
• structural-code-rule → describe the shape in words before drafting a rule
• log-event-pattern → separate volume from spread and isolate cohort/environment differences
• metric-anomaly → include denominator, baseline, and segment context before calling something abnormal

Step 4: Surface grouped candidates, not a raw hit list

Use this order: 1. Candidate family — what repeated shape or anomaly exists? 2. Why it matters — correctness, reliability, security, cost, UX, or player/product impact 3. Confidence — high / medium / low 4. False-positive risk — what could make it misleading? 5. Next check — the fastest validation or route-out

Do not silently turn detection into a full diagnosis, code rewrite, dashboard memo, or alert rollout.

Step 5: Use packet-specific heuristics

For text-prefilter

• Use it only as a cheap narrowing pass.
• Capture counts and representative examples, not every hit.
• Explicitly note when syntax/context is too important for plain text alone.
• Escalate to structural-code-rule or log-event-pattern when the match quality is too noisy.

For structural-code-rule

• Describe the code shape in words before reaching for tooling.
• Group matches by family, not by file order.
• Call out likely exclusions and false positives.
• Route actual remediation or security hardening outward once the family is identified.

For log-event-pattern

• Normalize one grouping unit first: error family, event name, exception class, browser, build, region, feature flag, tenant.
• Separate repeated fallout from the likely primary family.
• Flag parser or instrumentation drift explicitly.
• Route root-cause reconstruction to log-analysis once the user needs the actual incident story.

For metric-anomaly

• Name baseline vs comparison window.
• Compare absolute and relative change with denominator context.
• Test whether the anomaly is broad or concentrated in one segment.
• Flag seasonality, low sample size, or instrumentation changes before implying business meaning.
• Route deeper explanation, recommendations, or experiment narration to data-analysis.

Step 6: Return one compact detection brief

Default response shape:

## Detection brief
- Packet: text-prefilter | structural-code-rule | log-event-pattern | metric-anomaly
- Scope: [files / logs / rows / metrics / time window]
- Grouping key: [file family / symbol shape / error family / cohort / segment]
- Trust level: high | medium | low

## Candidate findings
1. [pattern family or anomaly]
   - Why it matters: ...
   - Confidence: high | medium | low
   - False-positive risk: ...
   - Next check: ...
2. ...

## Route-out
- stay here | log-analysis | data-analysis | codebase-search | security-best-practices | monitoring-observability | code-review

Keep it compact. The point is to leave the user with the smallest trustworthy shortlist, not a giant implementation memo.

Step 7: Route out aggressively

Switch when the next job is no longer first-pass detection:

• Root-cause incident or outage reconstruction → log-analysis
• KPI explanation, experiment readout, or business narrative → data-analysis
• Repo tracing, ownership lookup, or metric-definition hunting → codebase-search
• Security remediation or hardening design → security-best-practices
• Judgment-heavy patch / PR review → code-review
• Alert routing, telemetry coverage, incident ops, or monitoring policy → monitoring-observability

If the evidence is too thin: 1. mark confidence low 2. ask for the smallest missing anchor only if required: scope, time window, grouping key, or baseline 3. do not pretend certainty from one noisy excerpt or thin sample

Examples

Example 1: Repeated risky code shape

Prompt:

Scan this repo for repeated risky error-handling patterns and tell me what to inspect first.

Good response shape:

• choose text-prefilter or structural-code-rule
• group matches by family
• include confidence and false-positive notes
• route remediation to review/refactor/security skills instead of fixing everything in place

Example 2: KPI spike triage

Prompt:

We have a KPI spike in a CSV export. Is this pattern-detection or data-analysis?

Good response shape:

• choose metric-anomaly for first-pass detection
• name baseline, denominator, and trust checks
• route explanation and recommendations to data-analysis

Example 3: Gameplay telemetry outliers

Prompt:

Look for suspicious gameplay telemetry outliers after yesterday's update.

Good response shape:

• choose log-event-pattern or metric-anomaly
• segment by build, cohort, region, or item/class
• include instrumentation caveats
• stay in detection mode instead of promising a full balance fix

Example 4: Alert or anomaly?

Prompt:

We keep getting noisy anomaly alerts on one metric; should I debug the threshold, inspect the pattern, or write a dashboard summary?

Good response shape:

• start with metric-anomaly to classify whether there is a real suspicious window or just alert noise
• call out seasonality / low-history / denominator risks
• route alert tuning to monitoring-observability and KPI explanation to data-analysis

Best practices

1. Start with the unit of evidence and grouping key, not the tool name. 2. Use the cheapest trustworthy packet before escalating. 3. Always include confidence, false-positive risk, and one next check. 4. Group findings into candidate families, not raw hit lists. 5. Distinguish detection from diagnosis, remediation, explanation, and alert operations. 6. Treat parser, instrumentation, seasonality, and denominator issues as first-class caveats. 7. Prefer reusable rule thinking when the same pattern is likely to recur.

References

• ripgrep guide
• ast-grep introduction
• Semgrep pattern syntax
• CodeQL overview
• Datadog regex/grok parsing guide
• OpenSearch anomaly dashboards
• scikit-learn outlier detection

Confidence and False-Positive Checklist

Use this checklist before presenting a finding as meaningful.

Confidence questions

• Is the signal repeated or just a one-off?
• Is the coverage window large enough to compare?
• Is the grouping key stable (event name, browser, build, cohort, function shape)?
• Does the pattern persist across more than one segment or example?
• Would a human reviewer likely agree the matches belong to the same family?

Common false-positive sources

• broad regex or string matching
• parser/grok failures
• instrumentation drift after a release
• one noisy cohort dominating the counts
• duplicated rows / repeated retries / replayed events
• seasonality or launch timing misread as abnormal behavior
• code matches that share text but not structure

Default labels

High confidence

• strong structure or grouping
• multiple supporting examples
• low ambiguity

Medium confidence

• signal is plausible, but one major caveat remains

Low confidence

• thin data, weak baseline, broad matching, or likely data/parser issues

Minimum follow-up per finding

Every finding should include: 1. one sentence on why it matters 2. one sentence on what could make it misleading 3. one fast validation step

Example phrasing

• Confidence: medium — repeated across three mobile Safari cohorts, but the post-release window is only 24 hours.
• False-positive risk: event instrumentation changed in the same deployment.
• Next check: compare with server-side purchase success and browser/version split.

Pattern Detection Modes

Use this reference when the user asks for "pattern detection" but the evidence type is unclear.

1. Text prefilter

Use when you need a cheap narrowing pass.

Typical evidence:

• source files
• configs
• log lines
• event payload fragments

Good outputs:

• candidate files
• repeated strings
• suspicious substrings to inspect next

Pitfalls:

• brittle regex
• misses syntax/context
• easy to overcount duplicates

2. Structural code rule

Use when the pattern is code-shaped and syntax matters.

Typical evidence:

• repeated call shapes
• unsafe API usage
• migration candidates
• copy-pasted branch/error patterns

Good outputs:

• rule idea in words
• grouped matches by family
• exclusions / false-positive notes

Pitfalls:

• rule too broad
• tries to solve remediation inside the same pass

3. Log / event pattern

Use when the signal appears across repeated logs or events.

Typical evidence:

• repeated errors
• browser/build-specific failures
• noisy retry loops
• suspicious gameplay or product events

Good outputs:

• grouped clusters
• affected environments/cohorts
• count + spread context

Pitfalls:

• parser drift
• one noisy source dominating the signal
• forgetting environment/build segmentation

4. Metric anomaly

Use when the pattern lives in aggregates.

Typical evidence:

• KPI spikes/drops
• funnel drop-offs
• retention anomalies
• spend / conversion swings
• economy/progression outliers

Good outputs:

• baseline vs comparison window
• absolute + relative delta
• suspicious segments
• confidence + caveats

Pitfalls:

• weak denominator context
• seasonality ignored
• instrumentation gaps mistaken for behavior change

Quick routing rules

• Need root-cause incident reconstruction? → log-analysis
• Need stakeholder-ready metric explanation? → data-analysis
• Need ownership / call-site tracing / schema lookup? → codebase-search
• Need remediation or hardening plan? → security-best-practices or code-review

Intake Packets and Route-Outs

Use this packet chooser before doing any detection work.

Packet chooser

Minimal intake prompts

• Scope: files, services, routes, cohorts, environments, builds, time window
• Grouping key: symbol shape, message family, event name, browser/device, cohort, metric segment
• Decision after detection: investigate root cause, create a rule, explain KPI movement, harden security, or tune monitoring

Fast route-outs

• Need the actual outage or incident story? → log-analysis
• Need KPI explanation, recommendations, or experiment readout? → data-analysis
• Need call-site tracing or metric-definition lookup? → codebase-search
• Need remediation or hardening design? → security-best-practices or code-review
• Need alert thresholds, telemetry coverage, or incident operations? → monitoring-observability

Portable output rule

Every packet should end with: 1. grouped candidate findings 2. confidence and false-positive risk 3. one fastest validation step 4. one route-out if the bottleneck moved