Confidence scoring matters to CTI teams because it decides how a feed enters real workflows. When analysts search for confidence scoring, they usually want one thing: a clear way to judge whether an extracted indicator is strong enough to ingest, review, suppress, or escalate.
This article is about confidence scoring in scam-focused threat intelligence. Active Defense turns scammer conversations on Telegram and by e-mail into a feed of scored indicators, including crypto wallets, bank details, phone numbers, messenger accounts, links, gift cards, payment handles, and file and photo hashes. Customers get their own feed with STIX 2.1 export, so those scores can travel into CTI platforms instead of staying trapped in analyst notes.
What Confidence Scoring Means in a Threat Feed
Confidence scoring is a numeric or categorical expression of how strongly a feed producer believes an extracted item matches the claimed indicator type and is worth analyst attention. In practice, that score tells you how much trust to place in the extraction step itself, not what action you must take next.
That distinction matters because raw threat feeds rarely contain only clean, high-signal data. They often mix obvious indicators with partial strings, messy formatting, forwarded account names, image-derived artifacts, and context that is unclear at first pass. Confidence gives analysts a way to triage that mix without treating every observable the same.
It also helps to separate confidence from other concepts that CTI teams use every day:
- Confidence asks: how likely is this to be the indicator type the feed says it is?
- Severity asks: how serious is the related activity?
- Risk asks: how relevant is it to our environment?
- Impact asks: what would happen if it were used against us?
So a highly confident wallet, phone number, or URL may still be low priority in one workflow. A lower-confidence item may still deserve review if it appears in a sensitive case or near other meaningful evidence.
This is especially important in scam-focused intelligence, where observables often arrive through unstructured chat, forwarded messages, images, voice notes, and attachments rather than neat machine-generated logs. In a CTI platform, confidence scoring can drive ingestion rules, enrichment logic, analyst review queues, and automated thresholds.
A useful scoring model should also be consistent enough that analysts can compare indicators within the same feed, even if each organization sets different cutoffs for action. For a broader view of what makes a usable feed beyond scoring alone, see what should be included in a threat-intelligence feed.
How Good Confidence Scores Are Produced
Good confidence scoring starts with evidence that can survive ingestion into a CTI platform. Analysts do not just need a score; they need a score that reflects how the indicator was recognized in the first place, especially when extraction comes from scam chats rather than clean system logs.
In Active Defense, every received message is parsed for indicators, and each extracted item gets a confidence score from two inputs:
- a structural check
- an LLM
That matters because scam messages mix valid payment details with fragments, pasted examples, forwarding artifacts, and low-quality text. A structural check tests whether a candidate fits the expected form of the claimed indicator type. Depending on type, that can include format rules and checksum-style properties. The LLM then looks at message context: whether the value appears to be an actual payment target, contact point, or file artifact, rather than stray text.
The supported indicator types are broad enough for real scam operations:
- crypto wallets: BTC, ETH, TRON, SOL
- URLs
- e-mail addresses
- phone numbers
- messenger accounts
- bank details: IBAN, SWIFT
- gift cards
- payment handles
- file and photo hashes
The source record also matters. Every message is stored the moment it arrives, so later deletion by the scammer does not remove the original input used for extraction. Files, voice messages, and photos are preserved by SHA-256, and photos also get a perceptual hash. That gives scoring and later review a stable base, which is one reason CTI software needs to handle provenance and reprocessing well.
For teams evaluating feed quality, the practical test is whether scores can travel with enough underlying structure to support ingestion policy. That is central to what an IOC feed must support. You can also compare how these outputs appear in weekly scam briefings and monthly scam data reports.
How Analysts Should Use Confidence Scoring in a CTI Platform
Confidence scoring matters most when it drives an operational decision. If it sits on the indicator record as a decorative field, it adds little value. In a CTI platform, the score should help decide what enters automation, what waits for review, and what stays out of production workflows until more evidence appears.
A practical model is simple:
- Auto-ingest high-confidence indicators into normal enrichment and correlation flows.
- Send medium-confidence indicators to an analyst queue for review, enrichment, or case-by-case approval.
- Suppress, sandbox, or hold low-confidence indicators until they are corroborated by later collection or internal sightings.
Thresholds should be tuned by indicator type, not applied as one universal rule. The operational meaning of confidence is different for a crypto wallet, a phone number, and a perceptual photo match. A score that is strong enough for one type may be too weak for another, especially where formatting rules are loose or reuse patterns are indirect.
It is also worth preserving the producer's original confidence score. If your team wants a second layer of assessment, add an internal score, status, or disposition rather than overwriting source confidence. That keeps provenance intact and makes it easier to compare feed behavior over time.
For prioritization, confidence works best alongside other signals such as recency, sighting volume, actor linkage, and internal relevance. A scored indicator may still be stale, false-positive, or simply unimportant to your environment. Confidence should shape correlation logic, but it should not replace analyst judgment.
Relationship context can also improve triage. In Active Defense, chats that share an indicator, a reused photo, or a forwarded account are linked to one actor. That gives analysts context beyond the single observable and can help explain why an item deserves attention.
Each customer gets their own feed with STIX 2.1 export, which helps teams carry confidence metadata into CTI tooling and downstream workflows.
What Makes Confidence Scores Trustworthy Enough to Operationalize
Confidence scoring is trustworthy enough to operationalize when the score is tied to preserved evidence, stable handling of source material, and output that can be regenerated later. Analysts importing feeds into a CTI platform need more than a number. They need to know the record behind that number will still exist when they review a case, tune thresholds, or reprocess older data.
Three things matter here:
- the original source is retained at arrival
- extracted artifacts stay tied to that source
- the customer feed can be rebuilt from stored history
In Active Defense, every message is stored the moment it arrives. If a scammer later uses “delete for everyone,” that does not remove the stored copy. That matters in scam collections because payment details, contact points, and links are often removed after delivery. A confidence score is more useful when the evidence behind it is still available for analyst review.
Media preservation also affects trust. Files, voice messages, and photos are kept by SHA-256, and photos also get a perceptual hash. That gives CTI teams a way to review the exact artifact received and to compare image reuse across conversations without relying on text extraction alone.
Operational trust also depends on whether the feed fits review and testing before rollout. Teams can request a free feed sample if they qualify, or start with a 90-day pilot for up to 500 conversations at 1,900 EUR net, credited to the subscription afterwards.
One more point: replies to scammers are drafted by AI and approved by a human operator by default. That keeps collection tied to supervised handling before new source material enters the feed.
Common Questions About Confidence Scoring
Does confidence scoring measure extraction quality or threat level?
Confidence scoring answers a narrower question: how likely an extracted value is to be the indicator type the feed says it is. It does not, by itself, tell you how harmful the activity is or how urgent your response should be.
That distinction helps when your CTI platform mixes many workflows. A scored IBAN, wallet, or messenger account can be well formed and contextually plausible without being important to your environment.
Why do some indicator types need different score thresholds?
Because the evidence is different.
A BTC wallet, an IBAN, and a SHA-256 file hash have stronger structural patterns than a payment handle or a messenger account name. A photo perceptual hash also behaves differently from a URL or e-mail address. If you use one universal cutoff, you flatten those differences and make the feed less useful.
A practical setup is to tune by type, for example:
- stricter automation for structured financial and hash data
- analyst review for looser identifiers
- separate handling for perceptual photo matches
What should travel with confidence into a CTI platform?
The score is more useful when it arrives with the indicator type and enough surrounding context to support downstream rules.
For scam-focused feeds, that can include whether the item is a crypto wallet, bank detail, phone number, messenger account, link, gift card, payment handle, or file or photo hash. In Active Defense, chats that share an indicator, a reused photo, or a forwarded account are linked to one actor, which gives the indicator more operational context once imported.
How does confidence scoring relate to collection quality?
Collection quality affects how much trust analysts can place in a scored feed over time.
In Active Defense, replies to scammers are drafted by AI and approved by a human operator by default. Every message is stored the moment it arrives, so a later “delete for everyone” does not remove that copy. Each customer gets their own feed with STIX 2.1 export, and the feed can be rebuilt from stored history if needed.
Closing: Confidence Scoring Is Useful When It Is Actionable
Confidence scoring is useful when it improves ingestion and triage decisions. If it does not help analysts decide what to ingest, review, suppress, enrich, or escalate, it is just decoration on an indicator record.
In Active Defense, scammer conversations on Telegram and by e-mail become a feed of scored indicators. Those indicators include relationship context too: chats that share an indicator, a reused photo, or a forwarded account are linked to one actor, and each customer gets their own delivery.
For CTI teams, the practical value comes from the combination:
- scored indicators
- preserved source history
- actor linkage
- export formats that fit operational workflows
When comparing feeds, inspect sample data, look at how scoring behaves across different indicator types, and check how cleanly the data maps into your CTI platform. For a broader evaluation framework, read Cyber Intelligence Feeds for Scam Indicators.