An Insight Into the Building of a Data Warehouse

Summary:A data warehouse is not an application; rather a tool that allows authorized individual's access in order to conduct performance-monitoring functions. Building a data warehouse involves creating a storage system that boasts large amounts of relevant data. The data is then stored in a definite manner so that one can maximize its convertibility into useful information. Thus, data warehouse consolidates data from varied sources or multiple databases in-house, as well as from external sources.

Building a data warehouse is a complex process and it involves a number of complicated steps. The first step toward building a data warehouse is to implement data warehouse architecture. Once the architecture is ready the data-warehousing architect locates all data elements necessary to support the data warehouse, followed by building of a dimensional model. Populating fact and dimension tables into the model is often time-consuming.

Here are a few steps involved in the process of building a data warehouse:

Transformation of the data for optimum usage: Before the transformation of the data, extraction of that data takes place. As the data is extracted from many different sources, it needs to be processed in several ways. They are:

•Integration: In this process, common fields used in data structures from different sources are reconciled, both structurally and content-wise. During this differing measurements are transformed and fields of differing lengths and formats are negotiated as well.

•Condensation: This is when data is condensed during extraction and before loading into the warehouse.

•Stabilization: This is when frequency of data change can be found out. The stabilization data can be grouped together by attribute, depending on the attribute's stability.

•Normalization: Data warehouse is a high I/O environment. When data is normalized, there are occurrences of related items in different locations. This isn't I/O efficient. To avoid such inefficiency instances where the number of occurrences of a particular data item is stable enough are to be identified so as to grab it with a single I/O.

Equip the system with superior tools: The data warehouse system needs to be equipped with a number of important software. They are:

•Extract-Transform-Load (ETL): This software is the heart of a data warehouse. It will help you extract the data from operational systems, transform the data according to business requirements and then load it into the data warehouse.

•Online Analytical Processing (OLAP): Online Analytical Processing (OLAP) is a software technology that helps users in gaining an insight into data through fast, consistent, interactive access to a wide variety of information that has been transformed from raw data to reflect the real dimensionality of the enterprise. OLAP functionality is characterized by dynamic multi-dimensional analysis of consolidated data that supports users in analytical and navigational activities.

•Data mining and EIS: Data mining is a technology employed to map any correlations in a significant volume of data of the information system in order to detect any trends. It is often supported by artificial intelligence techniques to show hidden links between data. An EIS (Executive Information System) is a tool, which makes it possible to organize, analyze and determine indicators to create border tables.