Data Quality and Analytics – Optimize What Goes In, Benefit from What Comes Out

Generating accurate visualizations of the performance of a business takes accurate information. Businesses with multiple systems and large stores of data often struggle with:

• Duplicate, yet inconsistent information
• Out-of-date records
• Junk data, including falsified, or unverified content
• Missing information

If you can scrub your data before it gets into your business analytics system, you reduce the risk of setting the wrong impressions of your business performance. As powerful as on-premises and SaaS BI applications are, they are only as effective as what you “feed” them.

Creating and Maintaining Quality Data

It’s not enough to start out with clean data and hope for the best. It’s a long-term commitment.

To protect the data integrity and trustworthiness throughout its lifecycle, some good practices are to:

• Remember if you have a number of cloud apps, there will be a lot of data in motion. Establish consistent metadata across your different systems, and be just as thorough and consistent with data entry before you integrate with your reporting system.
• Schedule regular information audits – If some departments are entering information on mobile devices, there are more opportunities for typos and inaccurate data. Catch mistakes before executives are scheduled to review the latest reports.
• The entire organization should be committed to data quality. It’s not just an IT problem. If sales, marketing, finance and other teams share in the responsibility of data quality, business performance analysis will be more effective.
• Nominate “data stewards” across different departments to do regular data quality assessments. Set up a data quality committee to review progress.

Treat Your Data Like a Party Guest

You wouldn’t just let any random stranger into your house. You’d assess them, make sure they were reliable, and ask them to take off their dirty shoes before they joined your party, right? Treat your data like you would a party guest.

Data which tracks mud across your executive dashboard and wears a lampshade on its head spoils the party for everyone. Data visualization is like taking photos of your party guests, and data with integrity produces the best visuals.

Big Business Doesn’t Guarantee Big Data Success

A recent study by Price Waterhouse Coopers and Iron Mountain found most companies are failing at generating any strategic value from their data. The businesses in this study employed from 250 to over 2,500 employees, so you’d think they would have to be equipped with powerful business intelligence tools.

The study also found that businesses are good at capturing data, yet they don’t know how to effectively process it and generate the most value from it. Many experts say that “dirty” data is more dangerous than a lack of data. They say bad data can skew results, and your analytics program will fail.

Some leading data quality practices include:

• De-duplication – Identify copies of the same records, even when there are slight variations of the same data, such as Mike, Michael, Mikey etc. Unique identifiers like e-mail addresses are recommended as flags for identifying duplicates
• Refreshment – Ask customers to update their data, or employ data scientists to keep your information current and your analytics systems optimized
• Synchronization – Most businesses of all sizes have multiple applications to store and process data. Tools to sync and normalize data pay big dividends
• Cleansing – Bogus data can be detrimental to your analytics efforts, even if it was meant as a joke. Scrubbing out “Amanda Hugandkiss” or “Bob Loblaw” from your CRM system will improve your marketing campaigns and lead targeting efforts
• Enrichment – “Fleshing out” your data as you work with customers, suppliers, or as time passes will increase the quality of your reporting

Master Data Management for the Cloud

MDM solutions for on-premises software are powerful engines which extract, transform and load data between multiple systems. Cloud MDM solutions are gaining market share, both from traditional Big Data companies like Informatica, niche player Boomi (acquired by Dell in 2010) and open-source vendor Talend.

Integrating multiple data repositories to an analytics engine is risky. Leading SaaS solutions offer powerful API’s, and it isn’t the quality of the software itself which puts analytics projects at risk. Bad data sabotages many business intelligence initiatives. Up to 55 percent of BI projects fail because of bad data.

Data Quality Means All Hands on Deck

The IT department can’t take all of the responsibility for data integrity. Every employee within a company needs to play a role in ensuring better data:

• Sales and customer service agents
• Finance and operations employees
• Shop floor and logistics employees in manufacturing and distribution
• Engineers and product designers
• C-level executives and upper management

The old saying “garbage in, garbage out” is the simplest way to describe the risks associated with ignoring the challenge of data quality in tracking business performance. Companies that invest thousands of dollars into analytics solutions but allow dirty data to derail their programs shouldn’t blame their analytics system.

Whether you are a data scientist seeking tools to synchronize data across multiple systems or a CIO looking to ensure data can be relied upon to make strategic decisions on, make sure your data quality is addressed before it is fed into your analytics engine. Find ways to seek out, normalize or destroy rogue data.

As the growth of data accelerates in growing businesses, and expectations for performance analytics has hit real-time for some BI vendors, the importance of data quality increases. Address data integrity now, and don’t become like so many businesses that can’t generate any strategic value from their information assets.

Is addressing data quality a strategic focus for your business this year? What tools and/or strategies have you put in place to generate more value from your corporate data?