“Human in the loop AI” usually means people want a plain answer to how AI and human review work together in real operations. For fraud and threat-intelligence teams, that question is not about generic automation. It is about whether AI can engage scammers safely enough to produce usable intelligence. At Active Defense, AI decoy personas keep scammers talking on Telegram and by e-mail, while a human operator approves every reply by default. This article explains what that model means, why it matters in scammer engagement, how it becomes a threat-intelligence feed, and what to evaluate. Active Defense never hacks back.
What human in the loop AI means here
In plain terms, human in the loop AI means the system does part of the work, but a person reviews or approves important steps before anything is sent or acted on. That is a familiar idea in many workflows. In scammer engagement, it matters more because an outbound message does not just complete a task. It can change the direction of the conversation, influence what the scammer reveals next, and create operational risk if it goes out without review.
Here, the model is straightforward: each reply is drafted by AI and approved by a human operator before it is sent. Human-in-the-loop is the default. Replies are then sent with typing pauses and in short messages, which matches how chat conversations usually unfold.
For fraud and threat-intelligence teams, the benefit is practical. AI speeds up drafting, while a person keeps control over judgment and the final outbound message. That is different from full autonomy. A fully autonomous agent decides and sends on its own. This system is better understood as supervised execution: AI prepares, a human decides.
That distinction matters because the goal is not just more replies. The goal is sustained, believable engagement that keeps a scammer responding long enough to reveal more about the actor and the payment path. Teams that work with scam indicators already know how much downstream value can depend on conversation quality, whether they later review wallet screening, bank account checks, payment fraud detection, gift card payment chains, or false positives in feeds.
Why teams engage scammers on purpose
Teams engage scammers on purpose because every extra exchange can produce more operational detail. A longer conversation can prompt the scammer to reveal payment routes, contact points, reused assets, or other clues that support investigations and detection. In practice, the value is not the chat itself. It is what the chat can hand over for analysis.
From scammer messages, Active Defense extracts:
- crypto wallets
- bank details
- phone numbers
- messenger accounts
- links
- gift cards
- payment handles
- file and photo hashes
Each indicator gets a confidence score. That helps analysts decide what to review or action first instead of treating all extracted data as equally reliable.
The next step is linking. When chats share an indicator, a reused photo, or a forwarded account, they can be linked to one actor. That is more useful than reading isolated conversations one by one, especially because fraud operations often reuse infrastructure across many contacts. Repeated details across chats can help teams spot patterns faster, whether they are reviewing domain reputation checks, reverse face search work, or signs of caller ID spoofing.
Each customer gets their own feed with STIX 2.1 export, so the result is not just a conversation log but something a threat-intelligence program can ingest and work from. For readers who want more examples from live collection, see the weekly scam briefings and the monthly scam data reports.
How the workflow works from first contact to feed
The workflow starts when a new conversation reaches a decoy persona. From there, the sequence is simple: first contact is assessed, a reply is drafted, a human operator reviews it, the message is sent, the scammer answers, and the resulting material is turned into intelligence.
A classifier checks every new chat at first contact. On messengers, if the classifier is unsure, the chat is left alone. That matters because the earliest decision shapes whether the system should engage at all, rather than treating uncertainty as a reason to start talking.
Once a conversation is active, each outbound reply is drafted by AI and approved by a human operator before it is sent. Replies are sent with typing pauses and in short messages, which fits the pacing of chat-based exchanges.
On the inbound side, every message the decoy receives is stored the moment it arrives. If a scammer later uses “delete for everyone,” that does not remove the stored copy. Photos, voice messages, and files are retained as part of the record. Files and voice items are kept by SHA-256, and photos are also kept with a perceptual hash.
After collection, the analysis step turns raw conversation data into usable output. Indicators are extracted from every message and scored using structural checks and an LLM. That includes payment details, contact points, and artifact hashes from the material the scammer sends over. The scoring helps teams prioritize review and follow-up without treating extracted indicators as verified by default.
The result is a workflow that connects engagement, preservation, and analysis into one intelligence pipeline.
What good human oversight changes in practice
In scammer engagement, human oversight is not just a safety check before a message goes out. It changes the quality of the conversation itself. A person can reject a draft reply that feels weak, off-tone, or simply unhelpful for the next step in the exchange.
That matters because good engagement depends on judgment, not just response speed. In practice, an operator can assess questions such as:
- does this reply fit the persona?
- is it likely to keep the scammer talking?
- does it avoid steering the exchange in the wrong direction?
Short, paced replies can make a chat feel more natural in messenger environments. Human review helps keep that pacing coherent across the conversation, so the exchange does not drift or become inconsistent from one message to the next.
The division of work is clear. AI handles drafting speed and consistency. The human operator remains accountable for the final outbound message.
That oversight also affects intelligence quality downstream. Better replies can lead to better disclosures from the scammer, which means more useful material for fraud and threat-intelligence teams. That is especially important when teams later assess scored indicators, reduce false positives in scam indicator feeds, review domain reputation in scam investigations, or compare reused images through reverse face search in scam profile reviews.
The boundary matters too: Active Defense never hacks back. Human oversight is applied to deception and collection through conversation, not to intrusive action.
How to evaluate a vendor or pilot in this category
Treat evaluation as an operations review, not a demo review. The main question is whether the service gives your team material it can route into existing fraud and threat-intelligence work without rebuilding the process around the vendor.
A practical checklist:
- what the customer-specific feed contains
- whether STIX 2.1 export is available
- whether stored history can support rebuilding the feed later
- how actor linking is done across chats
- whether files, voice messages, and photos are preserved as artifacts
- what commercial starting point is offered
- how quickly the vendor answers an inquiry
For this category, feed structure matters as much as conversation handling. Teams usually need extracted indicators, actor links, and preserved artifacts in a form they can review, compare, and pass into their own systems.
Ask for the onboarding details too. Active Defense offers a free feed sample for qualified leads. It also offers a 90-day pilot for up to 500 conversations at 1,900 EUR net, credited to the subscription afterwards. Every enquiry gets an answer within 24 hours with a one-page architecture brief.
If deployment location is part of your review, note that the console and API run on Azure in Germany.
A good pilot should let your team judge output quality, feed usability, and fit with analyst workflows before a broader rollout.
Questions readers ask about human in the loop AI
Where does human review sit in the workflow?
It sits on the outbound step. Active Defense drafts each reply with AI, and a human operator approves it before it is sent by default.
That setup makes the review point clear: AI prepares the next message, but a person decides whether it should leave the system.
What happens at the very start of a new chat?
A classifier checks every new conversation at first contact.
On messengers, if the classifier is unsure, the chat is left alone. That is different from treating uncertainty as a reason to engage.
How are replies sent so the exchange feels like a real chat?
Replies go out with typing pauses and in short messages.
For teams evaluating human in the loop AI, that detail matters because messenger conversations are shaped by pacing as much as by wording.
What is preserved besides plain text?
The record is broader than a transcript. Every message is stored the moment it arrives.
It also keeps:
- photos
- voice messages
- files
Files and voice messages are kept by SHA-256. Photos are also kept with a perceptual hash.
What does the intelligence output look like?
The output is a customer-specific feed rebuilt from stored history. Each customer gets their own feed with STIX 2.1 export.
From scammer messages, Active Defense extracts:
- crypto wallets
- bank details
- phone numbers
- messenger accounts
- links
- gift cards
- payment handles
- file and photo hashes
Each indicator gets a confidence score.
Can a team try it before a subscription?
Yes. Active Defense offers a free feed sample for qualified leads.
It also offers a 90-day pilot for up to 500 conversations at 1,900 EUR net, credited to the subscription afterwards.
Console and API run on Azure in Germany.
Human in the loop AI is useful when the output is usable
Human in the loop AI is only worth using in scammer engagement if the result fits real fraud and threat-intelligence work. A chat that goes nowhere is not the point. The value is a feed your team can review, compare, and move into existing workflows.
For that, look for output that is:
- customer-specific
- rebuildable from stored history
- structured for sharing in STIX 2.1
- based on indicators scored for confidence
- enriched with links between chats tied to one actor
Active Defense turns scammer conversations on Telegram and by e-mail into that kind of output. Teams can start with a free feed sample for qualified leads or a 90-day pilot for up to 500 conversations at 1,900 EUR net, credited to the subscription afterwards.