This is a cache of https://www.elastic.co/observability-labs/blog/anomaly-detection-to-identify-fraud. It is a snapshot of the page at 2025-02-05T00:44:51.205+0000.
Using Anomaly Detection in Elastic Cloud to Identify Fraud — Elastic Observability Labs
Jonathan SimonKaren Mcdermott

Using Anomaly Detection in Elastic Cloud to Identify Fraud

Follow the step-by-step process of using Elastic Cloud’s anomaly detection to analyze example credit card transactions to detect potential fraud.

8 min read
Using Anomaly Detection in Elastic Cloud to Identify Fraud

Fraud detection is one of the most pressing challenges facing the financial services industry today. With the rise of digital payments, app-based banking, and online financial services, the volume and sophistication of fraudulent activity have grown significantly. In recent years, high-profile incidents like the $200 million credit card fraud scheme uncovered by the U.S. Department of Justice, which involved the creation of thousands of fake identities, have highlighted just how advanced fraud operations have become. These threats pose serious risks to financial institutions and their customers, making real-time fraud prevention an absolute necessity.

Elastic Cloud provides a powerful solution to meet these challenges. Its scalable, high-performance platform enables organizations to ingest and analyze all data types efficiently (from transactional data to customers’ personal information to claims data), delivering actionable insights that empower fraud prevention teams to detect anomalies and stop fraud before it occurs. From identifying unusual spending patterns to uncovering hidden threats, Elastic Cloud offers the speed and flexibility needed to safeguard assets in an increasingly digital economy.

In this blog, we’ll walk you through how Elastic Cloud can be used to identify fraud within credit card transactions—a key area of focus due to the high volume of data and the significant potential for fraudulent activity.

We’ll use a

Node.js
code example to generate an example set of credit card transactions. The generated transactions include a data anomaly similar to an anomaly that might occur as a result of fraudulent activity known as “Card Testing”, which is when a malicious actor tests to see if stolen credit card data can be used to make fraudulent transactions. We’ll then import the credit card transactions into an Elastic Cloud index and use Elastic Observability’s Anomaly Detection feature to analyze the transactions to detect potential signs of “Card Testing”.

Performing fraud detection with Elastic Cloud

Generate example credit card transactions

Begin the process by using a terminal on your local computer to run a Node.js code example that will generate some example credit card transaction data.

Within your terminal window, run the following git clone command to clone the Github repository containing the Node.js code example:

git clone https://github.com/elastic/observability-examples

Run the following cd command to change directory to the code example folder:

cd observability-examples/anomaly-detection

Run the following npm install command to install the code example’s dependencies:

npm install

Enter the following node command to run the code example which will generate a JSON file named transactions.ndjson containing 1000 example credit card transactions:

node generate-transactions.js 

Now that we've got some credit card transaction data, we can import the transactions into Elastic Cloud to analyze the data.

Import transactions data into an Elastic Cloud index

We’ll start the import process in Elastic Cloud. Create an Elastic Serverless project in which we can import and analyze the transaction data. Click Create project.

Click Next in the Elastic for Observability project type tile.

Click Create project.

Click Continue.

Select the Application tile.

Enter the text “Upload” into the search box.

Select the Upload a file tile.

Click Select or drag and drop a file.

Select the transactions.ndjson file on your local computer that was created from running the Node.js code example in a previous step.

Click Import.

Enter an Index name and click Import.

You’ll see a confirmation when the import process completes and the new index is successfully created.

Use Anomaly Detection to analyze credit card transactions

Anomaly Detection is a powerful tool that can analyze your data to find unusual patterns that would otherwise be difficult, if not impossible, to manually uncover. Now that we've got transaction data loaded into an index, let's use anomaly detection to analyze it. Click Machine learning in the navigation menu.

Select Anomaly Detection Jobs

Click Create anomaly detection job.

Select the Index containing the imported transactions as the data source of the anomaly detection job.

As mentioned above, one form of credit card fraud is called “Card Testing” where a malicious actor tests a batch of credit cards to determine if they are still valid.

We can analyze the transaction data in our index to detect fraudulent “Card Testing” by using the anomaly detection Population wizard. Select the Population wizard tile.

Click Use full data.

Click Next.

Click the Population field selector and select IPAddress.

Click the Add metric option.

Select Count(Event rate) as the metric to be added.

Click Next.

Enter a Job ID and click Next.

Click Next.

Click Create job.

Once the job completes, click View results.

You should see that an anomaly has been detected. It looks like a specific IP Address has been identified performing an exceedingly high number of transactions with multiple credit cards on a single day.

You can click the red highlighted segments in the timeline to see more details to assist you with evaluating possible remediation actions to implement.

In just a few steps, we were able to create a machine learning job that grouped all the transactions by the IP address that sent them and identified slices of time where one IP sent an unusually large number of requests compared to other IPs. Our fraudster!

Take the next step in fraud prevention

Fraud detection is an ongoing battle for organizations across industries, and the stakes are higher than ever. As digital payments, insurance claims, and online banking continue to dominate, the need for robust, real-time solutions to detect and prevent fraud is critical. In this blog, we demonstrated how Elastic Cloud empowers organizations to address this challenge effectively.

By using Elastic Cloud’s powerful capabilities, we ingested and analyzed a dataset of credit card transactions to detect potential fraudulent activity, such as “Card Testing.” From ingesting data into an Elastic index to leveraging machine learning-powered anomaly detection, this step-by-step process highlighted how Elastic Cloud can uncover hidden patterns and provide actionable insights to fraud prevention teams.

This example is just the beginning of what Elastic Cloud can do. Its scalable architecture, flexible tools, and powerful analytics make it an invaluable asset for any organization looking to protect their customers and assets from fraud. Whether it's detecting unusual spending patterns, identifying compromised accounts, or monitoring large-scale operations, Elastic Cloud provides the speed, precision, and efficiency financial services organizations need to stay one step ahead of fraudsters.

As fraud continues to evolve, so must the tools we use to combat it. Elastic Cloud gives you the power to meet these challenges head-on, enabling your institution to provide a safer, more secure experience for your customers.

Ready to explore more? View a guided tour of all the steps in this blog post or create an Elastic Serverless Observability project and start analyzing your data for anomalies today.

Related resources:

Share this article