BI/DW Best Practice #4 - Organization wide participation to ensure data quality

The technologies don't intelligent alone. The people are the intelligent and the BI solution assists them. We should introduce the best practice in the organization to maintain their data quality.

There are usually multiple copies of similar data within an organization. For example, there are the copies of "Product" data in Point of Sales (POS) system, Supply Chain System, Marking Information System and also the ERP system. We always recommend to figure out and define a single ultimate source for each entity but not getting various copies from multiple sources. Once the source is defined, we push a data cleansing exercise in these ultimate sources (e.g. create the missing data). The organization should also raise data governance rules to maintain the source data. The most basic rule is that we prohibit the creation and modification of entities' data directly in the copies. If we define the ERP is the source of Product entity, all changes of product should be triggered from ERP. It means that we can't add or modify the product data in the Point of Sales system. Instead, POS team should request the product data change in ERP and propagate it to all downstream systems. This is the basic of the concept of single source of truth. With the source data under control, we can have reliable and consistent data in the BI/DW.

We also call the above policy as the data stewardship. Each source system owner take the ownership of the source entity and are responsible to manage its own data from creation, modification and distribution.