Semantics data warehouse

Paradoxically, the goal of a Data Warehouse project is not to build a Data Warehouse. It is to answer business questions and evaluate hypothesis, predict the future. This is something that you cannot do without special handling of your source data. It is just not in such form you could apply any existing tool. And there is usually a lot of data. All over the place. Since 80’s the DW industry has insisted users that in order to solve these problems you have to build a Data Warehouse. A super database that merges and loads all sources in one place, under one master schema that captures the common domain view of all analysts and other users. Only when you have done this you can start querying the “new and clean” data, the industry “best practice” tells us. In order to build this Super Database, one need first to collect all possible questions that users will be asking the system. That will determine what data sources should be extracted and how the schema should be designed.

Over 30 years of history has shown that this method isn’t really working in most non-trivial cases. Because if you have no insight to your sources in first place (as you are looking for the answer to the questions of your business users), how then can you define a schema – a data model that represents their view of the domain and decide how to map from source to your schema? Count on long long and many meetings to coordinate your efforts between your users, your IT staff, your data modelers, your DW expert consultants, database vendor and who not. Everyone has stake in DW project.

In fact, you need just One question to get going and bootstrap your Data Warehouse – if you insist this still is what you want to build. One thing leads to another. There come 2nd and 3rd and N-th question and you do the same thing again and you provide answers to your users. You provide business value to your company. And one day you notice that – “Oh! – I probably have built something that could be called a Data Warehouse!”.