Do you really need a data warehouse?
Consider what a data warehouse is for. It basically allows you to do four things: it supports data mining and predictive analytics; it allows you to query and report on your data in a conventional manner; it supports more complex, analytical queries; and it can be used as the foundation for multi-dimensional, slice and dice style capabilities.
Now, data mining and predictive analytics is complex and expensive stuff and if you want to do this sort of thing then you certainly need a data warehouse. However, it is arguable that you don't need a data warehouse if you want to do any or all of the other things mentioned.
I have written before about the advantages of using column-based databases on low-cost hardware for complex, analytic queries, and I will not go over that ground again. Similarly, it is no secret that you do not need a data warehouse to support standard BI tools with OLAP capability. In this article I want to turn my attention to conventional reporting and queries, which form the bread and butter functionality of data warehouses. Do you actually need a data warehouse for this? No, you might equally use a report warehouse.
What is a report warehouse? Well, it's a store for report-based information.
Consider your ERP applications. And all your other applications for that matter. All of these applications will have many pre-built and canned reports, and possibly customised reports as well, which are produced on a regular basis for the benefit of various members of staff. One of the options that is common to all of these reports is that they can be written to a print file.
Now imagine that you had software that could read that print file, extract all the data therein, and then allow you to combine that data with related data from other reports in any convenient way, to filter report data so that you could make the output more suitable for individual users (each with their own view of the data), re-sort the data where necessary, and to re-assemble data according to your own specifications. And, of course, suppose that you could store this information so that you could combine not just contemporaneous data but information over time. And do that all without requiring any programming. That should give you an idea of what a report warehouse does.
Of course, a report warehouse won't be anything like as big as a data warehouse, it won't be as complex, it won't (as a project) be as prone to failure, and it will cost a great deal less. It is also, and this is a key point, designed so that end users can extract the data that they need, rather than the business analysts and statisticians that are typically served by data warehouse functionality.
Where you can get a report warehouse?
The answer is that you can get one from Datawatch Corporation, which offers the Monarch family of products that range from desktop capabilities that allow end users to redefine and filter existing reports, to a full-blown reporting warehouse. For smaller companies that do not need, or cannot afford, a full-blown data warehouse, this is a serious alternative to that approach, particularly when combined with some of the other technologies mentioned above. Equally, for larger companies that do require a data warehouse, a report warehouse can be seen as a useful complementary solution that will reduce the load on the data warehouse and which is much more amenable to end-user access.
Copyright 2004. Originally published by IT-Director.com, reprinted with permission. IT-Director.com provides IT decision makers with free daily e-mails containing news analysis, member-only discussion forums, free research, technology spotlights and free on-line consultancy. To register for a free e-mail subscription, click here.
For more information:
- Looking for free research? Browse our comprehensive White Papers section by topic, author or keyword.
- Are you tired of technospeak? The Web Services Advisor column uses plain talk and avoids the hype.
- For insightful opinion and commentary from today's industry leaders, read our Guest Commentary columns.
- Hey Codeheads! Start benefiting from these time-saving XML Developer Tips and .NET Developer Tips.
- Visit our huge Best Web Links for Web Services collection for the freshest editor-selected resources.
- Visit Ask the Experts for answers to your Web services, SOAP, WSDL, XML, .NET, Java and EAI questions.
- Couldn't attend one of our Webcasts? Don't miss out. Visit our archive to watch at your own convenience.
- Choking on the alphabet soup of industry acronyms? Visit our helpful Glossary for the latest lingo.
- Discuss this article, voice your opinion or talk with your peers in the SearchWebServices Discussion Forums.