In moving into a new more senior role, I was asked to look into the Kimball method. There was not a great deal of time to fully immerse myself into this methodology, that I never really formally implemented, so I commenced with was the purchase of Kimball’s Data Warehouse Toolkit Classics:
As per the official Kimball Group site, the recommended order of reading this 3 volume set is to start with reading The Data Warehouse Lifecycle Toolkit, 2nd Edition. I was able to carve out some time and read the first 8 chapters of the book. What I was expecting versus what I encountered were two completely different things. I was expecting a treatise on mathematical theorems, proofs, and set theory. I was wrong to say the least.
Initial Thoughts on The Data Warehouse Lifecycle Toolkit
What I found in the first few chapters was a well written guide book on how to navigate the corporate landscape, and how to map out a Data Warehouse project in a corporate environment. The first few chapters discussed:
- Introducing the Kimball Method
- Project Management
- Requirements Collection
The book does not start getting technical until Chapter 4. Circling back to the first 3 chapters, there is a lot of good advice on how to deal with the various groups in an organization, how to conduct meetings and interviews with the stakeholders, staffing, communication, Project/Program Management, and scoping the project.
Early into chapter 2, I realized that this is not the typical, dense IT tome that I am very familiar with. There is a very distinct thread of humanization in these initial chapters. The authors are obviously well seasoned professionals that know how to deal with all levels of people in an organization.
As for the technical portion of the book, the high level points to note, for me at least, are the following:
- Technical Architecture Overview
- Back Room Architecture
- Presentation Server Architecture
- Front Room Architecture
- Infrastructure
- Metadata
- Security
- Dimensional Modelling
- Physical Database Design and Performance
It is around chapter 6 where things start to get really technical. I have indeed worked with Data Warehouses in my time but never in such a formal way that is outlined in the book. The key points that are noteworthy to me, are as follows, in no particular order:
Enterprise Data Warehouse Bus Architecture
According to the Kimball Group, this is
the technology- and database-independent bus architecture allows for incremental data warehouse and business intelligence (DW/BI) development.
Data Warehouse Bus ArchitectureIt is just a sensible, intuitive way to deal with the DW/BI project in manageable pieces.
Degenerate Dimensions
A Degenerate Dimension is simply a dimension key in a Fact table that does not have a corresponding dimension table. They are commonly assigned by the operational business process/source system.
Slowly Changing Dimensions
According to the wonderful folks at Oracle, a Slowly Changing Dimension is “a dimension that stores and manages both current and historical data over time in a data warehouse.” There are 3 types and they involve overwriting existing data, creating another dimension record, and creating a current value field.
There is much more to this concept of course.
Fact Table Granularity
This idea is simply the lowest level of information that you can store in a Fact table. All the analysis is done beforehand with the key stakeholders before you would even begin to get to the modelling stage before the granularity is decided upon.
Final Thoughts
I have been impressed by the tone and flow of the book so far. As stated before, there is a very human aspect to the book. There a logical flow and progression on how to conduct a proper DW/BI project. For the most part, it is technology agnostic. One of my overarching thoughts were that the discussion about database best practices could be condensed.
Also, a more ominous theme I picked up on is that there is no mention of modern, cloud based databases. These are not mentioned by virtue of the fact that the book The Data Warehouse Lifecycle Toolkit, 2nd Edition was published in 2008, which is 15 years after the time of this article.
My thoughts are that the best aspect of the Kimball Methodology can be maintained to a certain extent, but the method will have to be updated for a cloud based world to maintain relevance. I am not the only one who has thought about this, as you will find in this article and this article.