New Qlikview video, about the in-memory advantage

• 

  Andrew said,

  I have seen the video, and it’s fairly interesting. Should generate a lot of interest in the product. Hmm, however are you saying that you cannot build well integrated apps without a data warehouse?? I’m fairly new to this qlikview and datawarehousing area. Can you elaborate on what you said? Thanks

• 

  Gilles said,

  Hi Andrew, thanks for your comment! Qlikviews marketing department likes to make you believe that Qlikview is the solution for all your problems. While Qlikview provides fairly comprehensive scripting possibilities to perform data integration it is not an ETL tool. We like to position Qlikview on top of a datawarehouse as (another) frontend tool.

  You are able to build well integrated apps with qlikview, but when it comes to maintenance, metadata, multisource integration, lineage, history treatment it falls a bit short. No doubt that qlikview is an excellent product, but my opinion is that you should use it as a frontend tool, not as your datawarehouse replacement.

  Regards,
  Gilles

• 

  Martin said,

  Hi Gilles,
  I agree that QV can be used as front-end tool on an existing data warehouse, BUT it can very well be used as full-replacement of a DWH as well.
  QV does have extensive scripting possibilities for ETL and with the QVD-technology you have enough options to build a sound and solid BI architecture.
  The companies/people that believe this will have the advantages and benefits of QV.
  The ones that don’t believe it, and still stick to their ‘real’ DWHs (probably the more conservative IT-departments), will miss these and will keep on suffering from the traditional OLAP-route (and keep on costing their companies a lot of money for BI-solutions the users have to keep on waiting for)…

• 

  Gilles said,

  Hi Martin,

  Thanks for your comment! This is an ongoing discussion between qlikview fanatics and BI specialists. I agree with you on the extensive scripting possibilities for ETL en QVD technology, but I dissagree on the sound and solid BI architecture. To explain this a little bit more, I’d like to borrow some parts of Ronald Damhof’s reaction in a “discussion” with another Qlikview fanatic. This exactly describes my opinion on using Qlikview for your “sound and solid BI architecture:

  In my opinion data should be stored as (vendor) neutral and open as possible in order to serve as many functionalities as possible. It’s called – begin agile for requirements you may not know in advance and creating a sustainable architecture for your data. If might even be a possiblity that an organisation does not want to lock her data into one single tool……..aint that realistic?

  Has it ever occured to you that there might be several uses/functionalities for the same data? Use/functionality that is not offered by QV (you will probably respond now that QV can do it all…..). Storing the data in a proprietary format aint gonna help this a lot….Or is the storage solution offered by QV open for other vendor’s?

  Something I also worry about is the effect Excel had on data proliferation. QV could lead to same issues as Excel. Data lying around everywhere…

  And I am not talking about my doubts concerning the storage of (semantic) compliant data across all possible outputs……

  And I am not talking about the extreme privacy regulations in government that screams for datamanagement and sophisticated security architecture accompanying your data as well as your BI.

  And I am not talking about metadata management control. A pillar where any sound data management architecture must be build upon.

  And I am not talking about large enterprise deployment of data and business Intelligence. The word ‘scale’ jumps to mind; extreme amounts of data, users, usage and business process complexity. You do not ‘buy’ a tool to accomodate this….you architect an environment.

  I am also not talking about the fact that many (especially) large size companies NEVER allow direct access to transational systems. And right they are! So you gotta build some kind of datastore and refresh it periodically. With increasing amounts of data this is by sure an architectural challenge.

  Another thingie that keeps bothering me; especially in large firms standaardisation is a goal in itself, this is mainly because of maintainability reasons. Tools need to have a level of integration with existing hardware, existing scheduling software, existing version control software, existing configuration management software, existing identity management sofware and so on…..How does QV compare to others on these integrational aspects? Somehow I got some doubts….

  Again,QV has earned it’s rightfull place in the BI front-end tooling arena. But to claim that DWH and ETL is just a ‘trick’ of vendors to make money,that’s not very nice.

• 

  Martin said,

  Hi Gilles,
  My statement is that QV will do (be sufficient) for 90% of the cases/companies that need a BI-solution. For them there is no need to look for more/other solutions: better spend their time and money on business solutions rather than technical perfectness….
  I understand your plee for the ideal world, but isn’t that an utopia? When you like to have a vendor neutral / open DWH we can also store the data in csv-files so that any application can use it. But I guess that will have some other (like performance) drawbacks….

• 

  Gilles said,

  Martin, these points are not about technical perfectness, it’s about sound & solid DWH/BI architecture. And a solid & sound DWH architecture shouldn’t be a Utopia! Sure, Qlikview can solve quite a lot of problems, sure, qlikview makes people smile (especially business users), but you should use qlikview where it’s good at and that’s not building datawarehouses.

  Cheers,
  Gilles

• 

  william said,

  This discussion stays interesting. To position Qlikview as yet another front end tool is a little belittling. Qlikview provides an unique platform to deliver smart, quick and valuable business solutions, explaining the success of Qlikview and the high satisfaction among customers worldwide.

  Whether you need a DWH or not is completely dependent of the solution needed. Qlikview is definitely no substitute for a DWH in big enterprises for all reasons Gilles mentioned above however, while I do believe scalability etc has top priority, there are enough companies, branches looking for smart solutions to answer specific questions / needs a classic DWH can’t provide (at least not within budget / time Qlikview can). We’re proving it every day )

• 

  Scott said,

  Interesting conversation and this is the subject of an upcoming blog I’ve written for our website. The data world is not a tidy place, and there’s no tidy answer to this. For everyone who asserts you always need a data warehouse, I can (and have) found IT folks who will concede that the majority of data warehouse implementations fail to deliver the business value anticipated or expected when they were started. But I would never use that fact to assert one shouldn’t bother with a warehouse.

  I CAN point to folks replacing their data warehouse with QlikView (I did at Pergo) or this gentleman: http://histalk2.com/2010/05/03/readers-write-5310/. I know hundreds of companies (most mid-size, yes) who have avoided the need to ever build a data warehouse by deploying QlikView. And yes, in (typically) larger companies QlikView is a fantastic vehicle for the “information delivery” or “last mile” of a data warehouse architecture. Can you say *poof* to indexing, cube building, data mart building, aggregations and summarization for the most part, etc.?

  In a given situation, there may be valid technical or business drivers for a data warehouse, regardless of the presence of QlikView. If so, build it or use it. If not, I see no reason to race to build one. And believe me, we build them all the time for clients. The existence of QlikView in a company doesn’t limit one in any way to build one later if your situation changes, either.

• 

  Vizubi said,

  I think that there are two main issues here. The first is the issue of using QlikView to do ETL directly from transactional systems. The second is the issue of using QlikView as the only repository of that data.

  Regarding the use of QlikView for ETL, I have the following thoughts:

  - QlikView scripting language allows you to do this in a feature rich way.
  - The language is quite complex
  - For highly heterogenous data coming from multiple systems or if the data structure of the source data is complex, creating a functional ETL process in QlikView is expensive in terms of resources and maintaining such a structure in my opinion would be very costly.

  Conclusion: QlikView for ETL could be an appropriate solution for organizations with relatively simple and relatively stable data structures, but the cost rises rapidly with the complexity of the source data.

  Regarding the use of QlikView for data storage I have the following thoughts:

  - Data stored in various QlikView formats (qvd, qvw) is not accessible using other tools
  - Exporting data from QlikView format is not a particularly functional option; possible but not particularly efficient (in the time = $ sense)

  - I fully agree that you’ll never know what you might want to do with your data and having to re-create the whole ETL process because your data is not readily accessible is clearly a big risk.

  Conclusion: Storing data exclusively in QlikView represents a big risk, but QlikView is more then a frontend for a traditional DWH.

• 

  Gilles said,

  OK, to conclude this conversation, and correct me if I’m wrong, we all think qlikview is a great product!

  Looking back at my post, where I claimed you DO NEED a datawarehouse, that might be a little bit exaggerated. It is not a “DO NEED” in all cases

  We all think that Qlikview in some cases can be sufficient to provide the required solution. This can be to provide the “last mile” as Scott states in an environment with an existing DWH, or with relatively small or departmental deployments, or with relatively simple and stable data structures.

  We also agree that there are solid reasons to not have qlikview as your DWH/ETL replacement, vizubi and Gilles state that storing data exclusively in qlikview represents a big risk. Also maintenance costs will rise with complexity of your data structures!

  Thanks,
  Gilles

Add A Comment