Detecting Scam and Fraud

[tweetmeme source=”jeric40”

It was a big story in the news yesterday, how Norwegian Air Shuttle was scammed by their Competitor Cimber Sterling.  Norwegian had put out super bargains in the Danish market to attract new customers. Cimber employees bought huge number of tickets in the name of Anders And (Danish version of Donald Duck), Alotto Fagina (Character from Austin Power movies) and Bjørn Kjos, the CEO of Norwegian.  The result was that Norwegian had a tremendous amount of no shows.

The question arises, could this be avoided?

The simple answer is yes, it is quite simple.  With an automated Data Quality Solution, this whole scam would have been stopped in the making. I don’t know the full extent of the Cimber Scam, but here are some of my assumptions based on what has come out in the media.  Here are some red flags that should have been raised:

Red flag 1. “Non Real Life Subjects”
Data Quality solutions are set up to find names people use to trick you it can be:

–          Cartoon Characters like Donald Duck, Batman and Superman

–          Film Characters like Alotta Fagina and Clark Kent

–          Random letters.  Aaaaa Bbbbb and Eeeeff Ghhhh

The DQ solution would have cleansed out these orders.

Red flag 2. Multiple use of same credit card.
It is said that a Cimber Sterling employee used the same credit card to book several hundred ticket over short period of time. This should have raised concerns in the fraud detection team, and be manually checked.

They should also cross check information.  Is it Natural that their CEO will use a Danish Credit Card to book his flights?

Red flag 3. Duplicate Check
DQ solutions run duplicate checks.  Unless Cimber employees have a very vivid imagination in inventing names, I am sure they must have used some names over again.   This should have been caught while ordering.

The Cimber Employees might have used a very common way of committing fraud.  Use variations and typos of the names.  Unless you have an error tolerant solution, it is difficult to catch these. In this post, I have explained how the name Christopher Quist can be written in almost 400 natural versions.

Red flag 4. Sanction List
I just mention this, because I am sure Norwegian has a system for this because of legal requirements.  In DQ solutions we also check according to the EU sanction list, to see that terrorists like Osama Bin Laden and other non wanted individuals cannot purchase from you.

It is normal that when you get an order to check all above red flags.  Most red flag warnings will be handled by an automated process.  When there are dubious entries, it will go to a operator that will handle this manually.

A Data Quality Solution is a cheap way to insure your company against such scams as encountered by Norwegian.  I think we will see more stories like this in the news, since most companies have not have focus on Data Quality.  I have written more about this post.

Most companies are not aware of the high cost of bad data quality, I hope this scam will help rise the awareness.

Here is my post from yesterday about the scam

Here is an article in Norwegian about the scam.

The scam continues. Here is another article in Norwegian


10 Critical questions to ask when you implement CRM

I came across an interesting article called: Saving CRM: Creating a data quality program by Douglas Ross.  Mr Ross is Douglas Ross, VP & CTO, Western & Southern Financial Group, so it’s from a users point of view, and not a vendor.  This is a well pointed article I urge everybody to read.

The most interesting part of the article is the list of 10 Questions you should ask before you start the project.  I have listed them here.

1. Have the benefits of improved data quality been defined for and agreed upon by the senior executives in the business?
2. Has the organization defined architectural standards for data, the relationships between data items, and requirements for data usage including those levied by the audit, regulatory, and compliance areas?
3. Does the organization measure data quality and strive for continuous improvement using agreed upon metrics, scorecards, and dashboards?
4. Are data-entry personnel equipped with tools to help enter clean data into the target systems?
5. Are there formal data stewardship roles, and are the related processes well-defined?
6. Do the systems you intend to integrate all support a universal, immutable customer identifier?
7. Do the target systems support all the necessary data elements and the actual relationships between products, people, accounts, and employees?
8. Do the target systems cooperate with one another to maintain data integrity, or do they “fight it out” and overwrite one another’s information from time to time?

9. Has the organization undertaken a bulk cleanup project to cleanse or rationalize legacy data?
10. Does the IT organization understand the benefits of clean data in driving improved business results?

If you answered “no” to nine or more questions, you’re in the same boat with a lot of other organizations.

I hope these questions will be mandatory for future implementations.

Read the article here.

Introducing new Thoughts and Concepts of Data Quality Firewall

Data Quality Firewall (DQF)

I have lately worked on a project including the Data Quality Firewall of an international corporation.  In this process we have tried to see what is the best set-up og the DQF and where it should be placed.

Since there may be several definitions of a Data Quality Firewall, I use the definition in Wikipedia:

“A Data Quality Firewall is the use of software to protect a computer system from the entry of erroneous, duplicated or poor quality data. Gartner estimates that poor quality data causes failure in up to 50% of Customer relationship management systems.

Older technology required the tight integration of data quality software, whereas this can now be accomplished by loosely coupling technology in a service-oriented architecture. (SOA)”

The New Concept of the Data Quality Firewall:
The firewall will be set as a workflow process that will do all necessary checks and interpretation to allow only the correct and accurate data to be entered into the database. The workflow will be set up as an integrated process across different systems and databases, based on SOA.

The Data Quality Firewall can be set up at different places to serve different needs. We will also introduce new concepts in the Data Quality Firewall thinking:

A. “Backend Data Quality Firewall” the most common used today
B. “Frontend Data Quality Firewall” set up in the data entry phase
C. “Double Data Quality Firewall” which will ensure the best data quality.

The Data Quality Firewall could include processes like:

A detailed status may be created per record; the detailed status may be analyzed and summarized into a status overview.

All the correction could be made into a interactive report to the supervisor.

Backend Data Quality Firewall

This is the most common used Data Quality Firewall in the market today. The checking is not done in the Data Entry phase, but when the data is transferred from temporary databases to the Master Database.

Even though faulty data is entered in the data entry phase, the Backend Data Quality Firewall will be set up to prevent the irregular data is entered into the Master Database or other relevant databases. The workflows will be set up individually towards the customer, to optimize the firewall according to the nature of the data from each individual system, operator, web and customer service.

The reason for setting up the Backend Data Quality Firewall first is to put the protection as close to the Master Data as possible.

Frontend Data Quality Firewall

As mentioned, the reason for setting up the Backend Data Quality Firewall first is to put the protection as close to the essential data as possible. The challenge with this is that you put the firewall away from where the dirty data is created. The dirty and faulty data is often created in the data entry phase. The reason for this can be many:

  • Operatives cannot find the right customer and re-enters it
  • In a high commission based business, operatives fight for their customer. Customers can be entered with a twist in the name accidentally or on purpose. Either way the operatives will fight for the ownership and commission.
  • Inaccurate or incomplete data can be entered in required fields, just to move on to the next customer

If the Firewall is put in the data entry phase, the amount of dirty data will be drastically reduced. The Firewall will consist of the workflows individually set up to each center/country and FACT-Finder Address Search. The results will be:

  • With the error tolerant search with FACT-Finder it ensured that the operatives find the right customer instantly. This will save time in the search, no need to spend time on register the customer a new. Operatives will get higher job satisfaction and be able to increase the number of calls they can receive.
  • If an operative tries to enter a customer with a little twist in the name they will get a message saying “possible duplicate found, do you wish to continue”. From that window, the operative may jump directly to the found duplicate, to continue working with that identified duplicate.If the operative overrides this message, it can be difficult to argue that it was by accident. This will lead to less infight between operatives, higher job-satisfaction and less double commissioning.
  • Workflows will be set up to check the incomplete and inaccurate data.
  • A monitoring service can also be set up to see if the operatives use the tool available for them. If an operative overrides a duplicate with a “secure match” this action could be logged or sent to a data quality steward to check the quality of matching or the quality of work of the operative.

FACT-Finder Address Search will be implemented in such a way that it is an integrated part of the CRM/ERP system, either it is a self developed or if it comes from outside vendors like, SuperOffice, Microsoft CRM, Siebel or others.

With the Firewall set up in the data entry phase the data that will be sent from the CRM/ERP system will be considerably cleaner. To set up individual workflows will be easier and more secure if is optimized in the centers of Data Entry.

A Double or Multiple Firewall(s)

In the old days you set up more than one wall in the castle to protect yourself. Our idea is to put down a Data Quality Firewall where different needs to be addressed in different ways.

One could set up the 1st firewall in the frontend and optimize the workflows in the Firewall to deal with the challenges that comes in the data entry phase. The 1st Firewall can have interaction with the user and is therefore a powerful solution for data quality (the human factor) Challenges addressed will be:

Correct basic errors as

  • Finding the right customer
  • Incomplete data
  • Data in wrong field – Examples: First name in last name field and or mobile number in the fixed net field.
  • Right salutation and gender.
  • If a duplicate is entered in spite of the search function, the duplicate will be matched to the original record, with a notification that to a supervisor that there was put in a duplicate.

The 2nd firewall will be set up in the backend, and the workflow of this Firewall will be set to deal with the challenges that come in the transfer of the large amount of Data from the front end to the Master Database or optimized databases for CRM, ERP or other specialized system database. It will be working without interaction from users.

Focus of the 2nd Firewall:

  • Settling the “Echo Problem”
  • Building the Customer Hierarchies
  • Worldbase Matching
  • Advanced PAF cleansing and De-duping

The Double Firewall would be highly efficient and would provide the best ROI and results of the solutions.

If you have thoughts and ideas about these concepts, please feel free to contact me!

Put a Data Quality Backbone in your CRM system.

It has become widely accepted that Data Quality is the main reason for CRM failure:

“Without clean data, there is no CRM. Poor data quality can lead to serious business problems.” – is quoted from an Article called “Avoid the pitfalls of poor data quality” in

“If there is no standard of quality for data, a CRM system becomes useless” – is quoted fromblog called Poor Data Quality = CRM Failure based on this article in BTOBonline

“Dirty Data can jeopardize your CRM” – an article in

“most executives are oblivious to the data quality lacerations that are slowly bleeding their companies to death,” – is quoted from an article called Data quality: ‘The cornerstone of CRM‘ in Computer world

“The simplest systems can deliver amazing results if they operate on clean data.”  – is quoted from the article “The Bane of CRM – Data Quality

Gartner estimates that 50% of CRM implementations fail.

Here is an efficient way of securing your Data Quality Backbone:

Omikron Data Quality Server has through several implementations proved to be the backbone of several successful CRM systems. With it’s 2 core modules it secures that your operatives find their customers, and that the data stays clean and up to date. With SOA it’s a flexible and easy to integrate solution.

Module 1: Find your customers easy, fast and intelligent with FACT-Finder Address-Search

Most duplicates are created in the data entering phase, because the operators don’t find the existing customers. Most systems have search/duplicate checks, but they are not very efficient and return few hits. If you are able to find your customers, you have solved a major part of the problem.

FACT-Finder is Europe’s Market leading search- and navigation engine for online shops. We use the same technology in our FACT-Finder Address Search to search in your CRM system. Fact Finder Address-Search is:

  • Error tolerant
  • Quick
  • Language independent
  • Easy to navigate in the results
  • Easy to integrate

The illustration shows how the DQS works and communicate with CRM and ERP systems.

Module 2: Always a clean database with DQS Duplicate Check

DQS Duplicate Check will ensure that your database(s) are up to date. It can

  • cleanse your data for duplicates and can enrich them with reference data every day
  • match and consolidate the data across several databases and systems
  • monitor the data entry phase to detect where poor data quality is initiated

Additional Modules for Data Quality Server
Data Quality Server is module based. Here are some of the other modules:

  • Sanction List Comparison
  • E-mail verification
  • Telephone Number Plan verification
  • Gender Assignement
  • Postal Correction
  • Data Enrichement

You can read more about Omikron Data Quality Server in this electronic brochure or on the webpage.

Ten Mistakes To Avoid In Data Quality Management

In TDWI’s Best of Business Intelligence, Volume 5, they have a chapter on ten mistakes to avoid in Data Quality Management written by Arkady Maydanchik.  I will try to summarize the ten points. I strongly advise you to download the report

1.  Inadequate staffing of Data Quality teams

“The Data Quality teams must include both IT specialists and business users.”

2.  Hoping that Data will get better by itself

“One fundamental misconception is that data quality will improve by itself as a result of general IT advancements”

3.  Lack of Data Quality Assessment

“Typically, organizations underestimate of overestimate the quality of their data, and they rarely understand the impact of data quality on business projects. Assessment is the cornerstone of any Data Quality management program. It helps describe the stat of the data and advances understanding of how well the data supports various processes. Assessment also helps the business how much the data problems are costing it”

On a personal note.  We have performed many tests on customer and production databases to help businesses asses the temperature on their databases.

4.  Narrow Focus

“Data Quality has continually detoriated in the areas of HR, finance, product orders and sales, loans and accounts, patients and students, and myriad of other categories.

The main reason we fail adequately manage to quality in these categories is that this data’s structure is far more complex and does not allow for  a “one size fits all” solution

5.  Bad Metadata

“The greatest challenge in Data Quality management is that the actual content and structure of teh data are rarely understood”

6.  Ignoring Data Quality during Data conversions

“The quality of the data after conversion is directly proportional to the amount of time spent to analyze and profile the data and  uncover the true data content. Unfortunately, the common practice is to convert first and deal with the quality later”

7.  Winner-Loser Approach in Data Consolidation

“The correct approach to data consolidation is to view it in a similar light as data cleansing. We select one of the data sources as the primary data source for each data element, design a comprehensive set of tests comparing the data against other sources, and then use the additional data for “data cleansing”. Once the data in the primary data source is correct, we convert it to the target database”

8.  Ineadquate monitoring of Data interfaces

“The solution to inferface monitoring is to design programs operating between the source and target databases”

9.  Forgetting about Data decay

“In the age of numerous interfaces across systems, we have come to rely on changes made in one database to be propogated to all other databases. However, there are times when it does not happen. For instance, interfaces may ignore retroactive data corrections, or IT personnel may make changes using backdoor update query – which of course does not trigger any transactions in the downstream systems.

10. Poor organization of Data Quality Metadata

“Data Quality assessment projects routinely generate innumerable unstructured error reports with no effective way of summarizing and analyzing the findings. As a result, the value of the Data Quality initiatives is greatly diminished. In the worst cases, the projects are abondoned.

This is just a summary. I strongly urge you to download the full report.  You can also mail
me at, and I will send it to you.

Tip 1 to avoid Poor Data Quality – Easy and Efficient monitoring of data entries

Poor data quality can have different sources:

  • System inefficiencies. Neither your software or internal control system does not catch the poor data
  • We are only humans.  Poor data a entered for sevaral reasons. Typos,  we enter data in wrong fields,  enter 0000000 in postal or telephone fields or just fill in as little as you are allowed to do

Customers set up our Data Quality Server to do intelligent continuous cleansing across databases with intelligent search to find customers or adresses.  In theory this will result in clean master database(s), but because of the system inefficencies and the human factor, some poor data is still found.

We have a feature that can monitor the entries of data.  You can analyze why the poor data is entered, and then be corrected.  A mail is generated every day to a supervisor, that can analyse the result.  If it’s human error the person who have entered wrong/insufficient data will be informed so they have a chance to improve their quality.  If it is system ineffiencies this will be corrected.  With this monitoring service the amount of poor data entered is drastically reduced in a short time span.