Quick answer: A fraud detection system scores payments, logins and account events in real time, combining rules, supervised machine learning, behavioural analytics and network (graph) analysis to block or hold suspicious activity before money leaves. In 2026 the design brief for Australian banks has widened from unauthorised card fraud to authorised scam payments and the mule accounts that receive them, driven by the Scams Prevention Framework and the industry Scam-Safe Accord.
For twenty years a bank fraud detection system had one job: stop transactions the customer did not make. Card-not-present fraud, account takeover, cheque and application fraud. The customer was the victim and the bank's loss was clear.
Scams broke that model. In an authorised push payment scam the customer makes the payment, passes every authentication step and often argues with the bank when it tries to intervene. The money lands in a mule account at another institution and is gone in minutes over the New Payments Platform. Detection now has to work on both sides of the payment, inbound and outbound, and across institutions. This guide explains how a modern fraud detection system is built, how AI and machine learning fit, how scams and mule-account detection extend the architecture, and what the Australian regulatory frame demands. It is written for heads of fraud and financial crime, fraud technology leads, CROs and the payments and data teams who build with them.
How does a fraud detection system work?
A fraud detection system is a decisioning pipeline sitting in the payment and digital channel flows. In sequence:
- Event ingestion. Payments (card, NPP, BPAY, direct entry, international), logins, device and session telemetry, profile changes, new payee registrations and open-banking data arrive as events, ideally in a streaming layer rather than batch.
- Enrichment. Each event is joined to customer history, device reputation, counterparty history, consortium intelligence and third-party signals (telco, email and phone age, sanctions and PEP screening where relevant).
- Feature computation. Velocity counts, deviations from baseline behaviour, time-since-last-event, geolocation plausibility, payee novelty and hundreds of other features are computed in milliseconds.
- Scoring. Rules, machine-learning models and behavioural models each produce a score or reason; a decision layer combines them against policy.
- Decision and action. Approve, step-up authentication, hold for review, warn the customer, block, or report. Latency budgets are typically well under a second for card and real-time payments.
- Case management and feedback. Alerts flow to analysts; outcomes (confirmed fraud, false positive, customer-confirmed genuine) flow back as labels to retrain models and tune rules.
The quality of step 6 determines everything upstream. Models are only as good as the labelled outcomes they learn from.
Rules, machine learning and behavioural analytics: what each layer does
- Rules — Strength: Transparent, fast to deploy, regulator-friendly · Weakness: High false positives, brittle, easy for fraudsters to probe · Typical use: Hard policy (sanctions, limits), known typologies, new-attack stopgaps
- Supervised ML (gradient boosting, neural nets) — Strength: Learns subtle patterns across hundreds of features · Weakness: Needs labelled data; blind to novel attacks; explainability work required · Typical use: Card, account-takeover and application fraud scoring
- Unsupervised and anomaly detection — Strength: Finds novel behaviour without labels · Weakness: Noisy; needs analyst triage · Typical use: Emerging typologies, insider risk
- Behavioural biometrics and session analytics — Strength: Detects coercion, remote access tools and bots in real time · Weakness: Privacy and consent considerations; vendor-dependent · Typical use: Scam-in-progress detection, account takeover
- Graph and network analytics — Strength: Reveals rings, shared devices, funnel accounts · Weakness: Compute-heavy; needs good entity resolution · Typical use: Mule detection, application fraud rings, bust-out
- Consortium and intelligence sharing — Strength: Sees what one bank cannot · Weakness: Data governance and legal basis · Typical use: Mule account intelligence, scam payee lists
The modern system is not "rules versus AI". It is all six layers orchestrated, with rules providing the guardrails and the AI providing the lift.
How is AI used in fraud detection?
Three ways, in increasing order of maturity:
- Scoring models. Supervised models trained on confirmed fraud outcomes remain the workhorse. The practical work is in feature engineering, class imbalance handling, drift monitoring and explainability (reason codes for analysts and, under Australian consumer law and the Privacy Act's automated decision-making transparency rules from December 2026, for customers).
- Adaptive and real-time learning. Models that update features and thresholds continuously as fraud patterns shift, with champion-challenger controls so a bad update cannot silently increase losses.
- Generative AI and agents in fraud operations. Large language models summarise cases, draft analyst narratives and customer communications, and increasingly orchestrate investigation steps. Autonomy is normally bounded to low-risk actions; see agentic AI use cases in banking and insurance.
The same AI also powers the attacker: deepfake voice and video for impersonation, synthetic identities for application fraud, and LLM-written phishing at scale. Detection programmes should assume adversarial AI as a baseline threat.
Why scams and mule accounts change the design
Scam payments defeat classic detection because every signal says "genuine customer": correct credentials, known device, authorised payment. Detection has to look for the context of coercion and manipulation and at the destination of funds.
Outbound (the victim's bank). Signals include first-time payee, payment amount out of pattern, remote-access tool or screen-sharing active during the session, long call in progress on the customer's phone (where telco data-sharing exists), hesitation and typing-pattern changes captured by behavioural biometrics, and payee accounts flagged by consortium intelligence. Interventions escalate from tailored warnings and friction (delays on first payments to new payees) to call-backs and holds.
Inbound (the receiving bank). This is mule account detection. A money mule is an account holder who receives and forwards illicit funds, knowingly or not. Signals include new or dormant accounts suddenly receiving multiple inbound payments from unrelated payers followed by rapid outbound transfers or cash withdrawals, shared devices or contact details across accounts, and inbound payments matching scam reports from other institutions. Graph analytics is the key technique: mules are rarely visible one account at a time.
Cross-institution. Australian banks exchange scam and mule intelligence through the Australian Financial Crimes Exchange and its Fraud Reporting Exchange, and the industry's Scam-Safe Accord committed banks to confirmation of payee, biometric checks for new accounts, limits and warnings on high-risk payments and faster intelligence sharing. Confirmation of payee, which checks the account name against the BSB and account number before a payment is made, removes a whole class of invoice and impersonation scams if customers heed the warning.
Singapore and New Zealand readers will recognise the same shift: mule accounts are a national priority in Singapore, where "money mule" is one of the most searched fraud terms, and New Zealand's banks have moved to confirmation of payee and shared liability discussions along similar lines.
What the Scams Prevention Framework requires of your detection system
The Scams Prevention Framework Act 2025 created an economy-wide regime, administered by the ACCC with sector regulators, under which designated sectors (banks first, with telecommunications and digital platforms) must take reasonable steps to prevent, detect, report, disrupt and respond to scams, with substantial civil penalties for breaches and external dispute resolution through AFCA. Sector codes set the detail.
For a fraud detection system that translates into evidence requirements:
- Prevent: customer warnings, friction on high-risk payments, confirmation of payee, controls on new-account opening against mule recruitment.
- Detect: scam-specific models and signals on outbound payments; mule detection on inbound funds; monitoring coverage across all payment rails including NPP and international.
- Report: timely reporting of actionable scam intelligence to the National Anti-Scam Centre and to other institutions; retained case records.
- Disrupt: holds, recalls and account restrictions executed within defined timeframes; participation in intelligence exchanges.
- Respond: complaint handling and reimbursement decisions that can be reconstructed and defended at AFCA.
Boards will want a line of sight from each of those verbs to a control, a metric and an owner.
Fraud detection versus AML transaction monitoring
They are cousins, not twins. Fraud detection is real-time, loss-preventing and customer-facing; it stops a payment. AML transaction monitoring is largely post-event, obligation-driven and regulator-facing; it detects patterns of money laundering and terrorism financing and produces suspicious matter reports to AUSTRAC. Mule accounts are where the two meet: a mule is both a fraud destination and a laundering vehicle, and the best programmes share entity resolution, graph infrastructure and case management across both teams. The AML side is covered in our guide to AML/CTF reforms, transaction monitoring and perpetual KYC.
How to reduce false positives without raising losses
- Use model scores to suppress and prioritise rule alerts rather than deleting rules.
- Invest in entity resolution so the same customer, device and counterparty are recognised across channels.
- Feed customer-confirmed genuine outcomes back as labels, not just confirmed fraud.
- Segment thresholds by customer risk and product; a first-home buyer's settlement payment and a daily $50 transfer need different treatment.
- Measure detection rate, false positive ratio, customer friction and analyst handling time together; optimising one in isolation moves loss elsewhere.
- Run champion-challenger on every model and rule change, with the fraud business owner approving promotion.
Buying versus building a fraud detection system
Most Australian institutions run a vendor decisioning platform with in-house models and rules on top. When evaluating fraud detection software for banks, test for: streaming latency under peak NPP load, ability to deploy your own models alongside vendor models, graph and consortium capability, explainability outputs, case-management integration, and the vendor's own resilience and data handling, because under APRA CPS 230 a fraud platform supporting payments is a material service provider.
Key takeaways
- A fraud detection system is a real-time decisioning pipeline; the feedback loop from analyst outcomes is its most important component.
- Rules, supervised ML, anomaly detection, behavioural analytics, graph analytics and consortium intelligence are layers to orchestrate, not alternatives.
- Scams require detection of coercion on the outbound side and mule accounts on the inbound side, across institutions.
- The Scams Prevention Framework turns prevent, detect, report, disrupt and respond into evidence requirements with Board-level accountability.
- Share entity resolution, graph infrastructure and case management between fraud and AML teams; mule accounts belong to both.
Join your peers at the Clutch Fraud and Financial Crime Technology Summits
Clutch Events runs free-to-attend, invite-curated, practitioner-led fraud and financial crime technology summits for senior fraud, financial crime, risk and technology leaders at Australian financial institutions:
- Sydney Financial Crime and Fraud Technology Summit 2026 — 29 October 2026
- Sydney Fraud and Financial Crime Technology Summit 2027 — 13 May 2027
- Melbourne Fraud and Financial Crime Technology Summit 2027 — 18 August 2027
See all upcoming Clutch events · More guides on Clutch Events Insights