Real-time data warehousing : data integration framework.

• Main Author: Kyaw, Ye Lin.
• Other Authors: Vivekanand Gopalkrishnan
• Format: Theses and Dissertations
• Language: English
• Published: 2010
• Subjects: DRNTU::Engineering::Computer science and engineering::Information systems::Database management
• Online Access: http://hdl.handle.net/10356/42312
• Institution: Nanyang Technological University

A data warehouse primarily provides intelligent business facilities such as analytical processing, decision making and data mining. Design and implementation of data warehouses has been well studied and supported by many commercial offerings. However increasing demands for real-time systems fuels the need for changes in existing data warehouse designs and frameworks. One such change required to realise a real-time approach is to redefine the process of synchronization between transactional data and the data warehouse. Current loading processes are typically carried out by using the methods of insert, batch or bulk loading which can deteriorate the real-time performance. Though bulk loading is faster, it needs additional preprocessing of the data, which is not cost effective particularly when transferring large amounts of data. It is thought that real-time data integration can improve the process of frequently loading small amounts of data. This dissertation discusses the performance and functionalities of existing data integration tools along with their associated problems and drawbacks. Current trends in real-time data integration are also analysed and a new real-time data integration framework is introduced. Implementations of the well known real-time data integration method are also presented, as are possible extensions for implementing or introducing the new real-time data integration implementation method to address other problems. An in-depth empirical analysis demonstrates the validity of the proposed approach.