Admiralty System

Admiralty System for CTI

The Admiralty System (NATO AJP-2.1) is the gold standard for assessing intelligence. The British Royal Navy developed it in the early 20th century. NATO intelligence communities now use it worldwide, and it is gaining ground in cyber threat intelligence.

It rates two things, separately:

• Source Reliability (A to F): how trustworthy the origin is
• Information Credibility (1 to 6): how trustworthy the data is, independent of the source

The combined output is an alphanumeric code such as A1, B3, F6.

Core rule: assess source and information SEPARATELY

This is the single most common mistake. A source who is reliable on malware analysis is not automatically reliable on geopolitics. A piece of information can be true from a sketchy source, or false from a usually reliable one. Always rate them independently, never in lockstep.

Second common mistake: confusing "two vendors reported it" with two independent sources. If both vendors pull from the same original dataset (the same breach forum post, the same scan, the same leak), that is ONE source. Independent corroboration means independent collection.

Source Reliability Scale (A to F)

Attributes to weigh:

• History of accuracy on this specific topic (not other topics)
• Proximity to the event (firsthand, secondhand, hearsay)
• Technical skill in the domain
• Motivation (financial gain, reputation, ideology, deception)
• Identity corroboration (links to known infrastructure or aliases)
• Forum reputation score, post count, join date
• Administrator or moderator status on the forum
• Documented links to other known threat actors or breaches

Information Credibility Scale (1 to 6)

Attributes to weigh:

• Internal logical consistency
• Fit with prior known facts
• Independent corroboration (genuinely independent, not echoed)
• Technical plausibility (does the claimed attack chain actually work)
• Presence of sample data, hashes, or other concrete proof
• Time elapsed since the event
• Whether the data could be recycled from earlier breaches

Workflow

When asked to assess a source or information pair, follow this sequence:

1. Extract structured metadata

For each post, report, blog, or claim, pull out these fields:

• Case name
• Forum or platform
• Post content (what is being claimed)
• Date of post
• Author handle
• Communication identifiers (XMPP, Telegram, email, Tox)
• Associated handles or known aliases
• Forum section
• Post link (defanged)
• Author first seen date
• Author involvement in other cases
• Author forum reputation
• Author observed in other forums

This structured extraction forces you to look at the evidence before you score anything.

2. Rate the source (A to F)

Walk through the source attributes. Justify the letter with one or two sentences of reasoning. If the user is tracking the same case across multiple posts, rate each post's source independently. Ratings can change as a track record builds.

3. Rate the information (1 to 6)

Walk through the information attributes. Has anyone independently corroborated this? Is the technical claim plausible? Is the sample data verifiable? Could it be recycled data from an older breach? Justify the number with one or two sentences.

4. Recommend next steps

For low-confidence ratings (E5, F6, D4, and similar), suggest specific verification actions:

• Cross-check on other forums or marketplaces
• Pull a sample of the data and validate against known schemas
• Check the original infrastructure (login endpoints, archived snapshots, Wayback, FOFA)
• Compare timestamps with public incident announcements
• Hash-check leaked samples against known prior leaks to detect recycled data
• Look for admin endorsement or community vouching

5. Suggest SATs when relevant

When you are looking at a developing case with multiple posts over time, recommend two structured analytic techniques:

• Timeline: showing how source and information ratings evolve over each post
• Link chart: showing relationships between aliases, infrastructure, and forums, with each edge colour-coded as confirmed, partially validated, or pending validation

These externalise your reasoning and let peers challenge it. They are also what a stakeholder needs to follow the analysis.

6. Always flag alternative hypotheses

Before locking in an assessment, list at least two competing explanations:

• The actor really did breach the company
• The actor is recycling old or unrelated data
• The actor is aggregating multiple smaller leaks and rebranding them as one
• The data is from a third-party processor, not the named company
• The post is a reputation play with no real data behind it
• The post is misinformation or disinformation

The Admiralty rating should reflect the most likely hypothesis AFTER you consider the alternatives, not before.

Output Format

Always produce a structured assessment table. Use this template:

| Field | Value |
|-------|-------|
| Case Name | ... |
| Forum / Platform | ... |
| Post Content | ... |
| Date of Post | ... |
| Author | ... |
| Communication Identifiers | ... |
| Associated With | ... |
| Forum Section | ... |
| Post Link | ... |
| Author First Seen | ... |
| Author Involved in Other Cases | ... |
| Author Forum Reputation | ... |
| Author Observed in Other Forums | ... |
| Source Reliability | X (label) |
| Information Credibility | Y (label) |
| Reasoning (source) | ... |
| Reasoning (information) | ... |
| Alternative Hypotheses | ... |
| Recommended Next Steps | ... |

When tracking the same case over time, add a final section:

• Timeline of ratings: post 1 (date, rating), post 2 (date, rating), and so on
• Observed trend: for example, rating climbing from F6 to B2 as corroboration arrives

Hard constraints

• NEVER conflate the source rating with the information rating. They are independent axes.
• NEVER count two reports using the same underlying data as independent corroboration.
• NEVER give an A rating casually. A is reserved for sources with a long, well-documented, topic-specific track record.
• When unsure, prefer F or 6 over guessing. "Cannot be judged" is a valid and honest rating.
• ALWAYS justify the rating with a short written reason. A bare letter-number score is not enough.
• ALWAYS preserve the system as defined (NATO AJP-2.1). Do not invent custom variants. Otherwise teams cannot compare assessments and the system collapses into noise.

When to flag uncertainty in the output

If the rating is D, E, F, 4, 5, or 6, wrap the assessment in caveats:

• "We assess this to be [rating] based on available information, with the following limitations..."
• "This rating may change as the case develops"
• "Pending [verification step], this assessment should be treated as preliminary"

This protects the reader from treating a preliminary rating as a confirmed fact, and it protects you when the case evolves.

Examples

Example 1: New dark web forum post

Input: A new user "SpidermanData" on Exploit[.]in posts a sale for 560M Ticketmaster user records at $500,000. Account has 2 posts and joined yesterday.

Output: Source = F (reliability cannot be judged, no track record). Information = 6 (truth cannot be judged, no sample provided, no corroboration). Next steps: monitor for sample drop, cross-check BreachForums, watch for admin endorsement, check XMPP identifier against known actor inventories.

Example 2: Established threat actor reposts the claim

Input: ShinyHunters (BreachForums administrator, reputation score 1,087, known for Tokopedia, AT&T, Santander breaches) reposts the same Ticketmaster claim with folder size data showing 1.3TB.

Output: Source = B (usually reliable, strong track record on breach claims, admin status). Information = 3 (possibly true, structural metadata consistent with a real breach, but no public sample yet). Next steps: pull the folder structure and compare against known Ticketmaster export schemas.

Example 3: Vendor report citing a single dark web source

Input: Vendor X publishes a report titled "Biggest supply chain hack of 2025" based on a BreachForums post from a new actor "rose87168".

Output: The vendor source rating depends on the vendor's own track record (rate it separately). The underlying claim source remains the new actor (F). The vendor putting their badge on the claim does NOT upgrade the original source rating. Information credibility depends on what the vendor independently validated versus what they just repeated. Build a link chart and colour each claim: confirmed (green), partially validated (yellow), pending validation (red). Push back on any claim that sits in yellow or red but is presented as fact.

Example 4: Trusted vendor with multi-source corroboration

Input: A top-tier CTI vendor with a long, accurate track record on this threat actor publishes a report on an actively exploited zero-day. Three independent vendors confirm the same exploit chain from their own telemetry.

Output: Source = A (completely reliable, long topic-specific history). Information = 1 (confirmed by multiple independent sources, each with their own collection). Note: even at A1, state the limitations and the assessment date, because the situation can shift.

Notes on terminology

• "Information" in this context means the data and claims being conveyed, not the platform or channel.
• "Source" means the originator of the claim, which can be a person, a forum account, a system, or a sensor.
• A system or sensor is still a source, and logs can still be tampered with. Cross-check internal and external logs for consistency before trusting either.
• Treat each post by the same actor as a separate observation. A B-rated actor can drop an F-rated post if the specific claim is implausible.

Bundled Resources

This skill ships with two extra files. Load them only in the specific situations below, not on every assessment.

references/edge-cases.md

Read this file when the situation does not cleanly match the four examples in SKILL.md. Covers six tricky scenarios:

• Two vendors with contradictory attributions on the same campaign
• Sensor or honeypot data (system as source)
• Aggregated feed data where origin cannot be traced
• Law enforcement attribution with sealed evidence
• Insider leaks where motivation is hostile but information is accurate
• Recycled-data detection in breach claims (Australian and US examples)

If you are rating any of the above, read the file before producing your assessment.

assets/case-tracking-template.csv

Use this file when the user is tracking the same case across multiple posts over time. It is a CSV with all assessment fields as columns and one example row populated with the SpidermanData Ticketmaster post.

When the user says they want to track a case, hand them a copy of this file populated with rows for each post observed so far. This mirrors the Freddy Murstad / Sean O'Connor workflow: one row per post, ratings change as the case develops, the timeline of ratings becomes the audit trail.

Admiralty System: Edge Cases

Load this file when the main SKILL.md examples do not cleanly cover the situation. These six scenarios are the most common edge cases in CTI and OSINT work where a naive A-F / 1-6 read leads to the wrong answer.

1. Two vendors with contradictory attributions on the same campaign

Scenario: Mandiant attributes a campaign to APT29. Kaspersky attributes the same campaign to a different actor, "TURLA-adjacent". Both publish technical reports with overlapping IoCs.

Why it is tricky: Two A-rated sources contradicting each other does not average out to B. The information credibility does not collapse to 4 just because there is disagreement. The two claims are different, not opposed.

How to rate:

• Rate each vendor as a source independently. Both can be A.
• Rate each claim independently. Mandiant's claim "this is APT29" is one piece of information. Kaspersky's claim "this is TURLA-adjacent" is a separate piece of information.
• Both can be A3 (possibly true, plausible from a reliable source, no independent corroboration of the attribution itself).
• The overlap in IoCs is A1 (confirmed by independent sources). The attribution layered on top is not.

Pitfall: Do not let the disagreement push you to F or 6. The disagreement is signal, not noise. It tells you the attribution layer is not yet confirmed even when the technical evidence is solid.

2. Sensor or honeypot data (system as source)

Scenario: A honeypot in your sensor network captures exploitation attempts and the captured payload matches a published CVE. The honeypot report is fully automated, no human involved.

Why it is tricky: A system feels objective and tempts you to rate it A1. But systems can be poisoned, misconfigured, or compromised, and they also have a track record like any source.

How to rate:

• Rate the sensor as a source. How long has it been deployed? Have its captures been corroborated against external feeds? Has it ever produced false positives? A long-running honeypot with externally validated captures can be B or A. A new sensor with no track record is C or D.
• Rate the information separately. Does the captured payload validate against the published CVE? Can you reproduce the exploitation in a lab? Has another sensor in another region captured the same payload?
• Cross-check internal logs against external logs. If the sensor reports an exploit attempt but the upstream firewall does not see the matching traffic, the sensor is potentially compromised.

Pitfall: Treating system output as automatically A1 because it is "machine generated". The machine is still a source. Rate it like one.

3. Aggregated feed data (origin cannot be traced)

Scenario: A commercial threat feed pushes 200 IoCs per day. The feed aggregates from 30 upstream sources. You cannot tell which upstream provided which IoC.

Why it is tricky: You cannot rate the source because you do not know what the source actually is.

How to rate:

• The aggregator is the source you can see. Rate the aggregator on their history of accuracy and false-positive rate. This is typically C or D for most commercial feeds because they do not filter.
• The information cannot be rated higher than the weakest upstream. If you do not know the upstreams, the information is at best 3 (possibly true) and often 4 (doubtful).
• Insist on provenance from the feed vendor. Any vendor unwilling to disclose upstream sources should drop a full letter grade.

Pitfall: Trusting feed data because the vendor is well-known. The vendor's reputation does not transfer to the underlying claims if the vendor cannot trace them.

4. Law enforcement attribution with sealed evidence

Scenario: The FBI or AFP publicly attributes a campaign to a named actor or country, citing classified or sealed evidence not available to you.

Why it is tricky: You are being asked to trust a high-reputation source on information you cannot independently verify. The institutional history of the source is strong but the specific evidence is invisible.

How to rate:

• Source rating reflects the institutional track record. FBI on cyber attribution is typically B or A depending on the case. Note: track records vary by case type. They are stronger on financial cybercrime than on state-sponsored attribution.
• Information rating cannot rise above what you can verify. Without access to the sealed evidence, the information is at best 3 (possibly true) and usually 4 (doubtful from external view).
• An indictment is stronger than a press release because it carries legal weight and can be challenged in court. Treat indictments as 2 or 3. Treat press releases as 3 or 4.

Pitfall: Letting the institutional weight inflate the information rating. Source and information are separate axes. An A source making an unverifiable claim is still A6 if you genuinely cannot judge the claim.

5. Insider leak (hostile motivation, technically accurate information)

Scenario: A disgruntled employee leaks internal documents from a company. The documents are technically accurate but the leaker's motivation is revenge, financial gain, or political damage.

Why it is tricky: The motivation tempts you to rate the source low (D or E), but the documents themselves are real.

How to rate:

• Source rating reflects reliability, which is about whether the information they provide is accurate, not whether their motivation is pure. A leaker with verifiable insider access whose documents have been validated as authentic is C or B. A leaker whose past leaks have been falsified or selectively edited is E.
• Information rating reflects whether the data is true. If the documents check out (metadata, internal references, formatting match), they can be 2 or 1 even from a hostile source.
• Flag the motivation in your reasoning. A B2 from a hostile leaker is still B2, but stakeholders need to know the leak is selective by design and not a complete view.

Pitfall: Conflating motivation with reliability. Reliability is about accuracy. Motivation is context for interpreting what the source chose to share and what they may have withheld.

6. Recycled-data detection (especially Australian and US breach claims)

Scenario: An actor on BreachForums claims a fresh breach against a major Australian retailer. The data sample includes 2M records with name, email, phone, DOB.

Why it is tricky: Australia has had several large public breaches in recent years (Optus 2022, Medibank 2022, Latitude 2023, Dymocks 2023, MediSecure 2024). US has more. Repackaged data from these is a common scam. The sample data will look real because it IS real, just not from where the actor claims.

How to rate:

• Hash-check the sample against known prior leaks before assigning any rating. Use Have I Been Pwned, internal CTI databases, and previous samples you have collected.
• If the sample matches a known prior breach in part or whole, source rating drops to E and information rating drops to 5.
• If hash-check is clean, proceed with the normal workflow. Source rating depends on actor track record. Information rating depends on whether the data structure matches what the claimed victim would actually export.

Pitfall: Assuming "the data looks real" is positive evidence the breach is real. Recycled data is real data. The question is whether it is FROM the claimed victim.

Cross-cutting rules

• When in doubt between two letters or numbers, pick the lower one. The system is designed to flag uncertainty honestly, not to look confident.
• Always re-rate every post in a case independently, even when the actor is the same. Source reputation can lift the floor, but each post stands on its own.
• The four colour states from the SKILL.md workflow (confirmed green, partially validated yellow, pending red) apply to claims, not to whole reports. A single report can contain green, yellow, and red claims at the same time. Break it apart.