Monday, August 31, 2020

BI/DW Best Practice #1 - Principles

We always consider the business need, performance, capability to change and reality. I believe that they are the top design principles for a BI/DW project.  They also represent the corresponding quality referring to the traps that I discussed in my last article.


Business Need

The main purpose of BI/DW solution is to assist business decision making. We should understand the business processes and build reliable processes and data for user analysis. We should also support the users to switch into the new business procedures and practice.

Performance

Performance is the major measurement of BI/DW projects. The fast respond report queries and visualization helps the management respond to the market. It includes the fast respond queries, short downtime for data loading and low latency.

Capable to Change

Per my experience, "Change" is one of the major challenges of BI/DW projects. The user requirements may change. Change may come from wrong understanding of the business processes. The source systems can be changed (e.g. upgrade). And even come from the business and market.

If the architecture of our BI/DW solution is not ready to those potential change. Costly rework efforts are required to modify and maintain your BI/DW solution. (The hell of Rework) Surrogate key and decoupling design are good examples how we can better handle for future changes.

Reality

We may sometime have some great idea for the BI/DW project. But we may finally find that no such source data available. This is an example of reality. Usually, we can't ultimately fulfill all the principles all the ways. So, we are also talking about the trade off between those principles above. 

There may be some methodology which attain an ultimate performance. However, if it introduces a big efforts to rework for the future changes, think carefully and consider the trade off. It is why I am not recommending extraction by direct connecting to the source system. I may explain by an example later. 

No comments:

Post a Comment

Extract: Performance Tips

Below listed some common performance tips for extract queries. Extract required columns only and specify in the query, avoid select * Extrac...