Business Intelligence in Cloud

Cloud computing based on pay-as-you-go hardware infrastructure like Azure, Amazon or Google offers business intelligence users new data warehouse options that bring unlimited scalability without traditional data center overhead and budget constraints. It can provide end to end platform for BI applications for storing and analyzing data. High scalable storage in cloud can be used to create large data warehouses enabling storage of data in Terabytes simultaneously utilizing the on-demand compute power to analyze data.

In my view, there are three main constraints to BI adoption in this new era of analytic data management for business intelligence:

• Effort required in consolidating and cleaning enterprise data: Prior to integrating data within the system it is required to detect and correct (delete) corrupt or inaccurate data from the store. This process will involve removing any typos, validating data definitions against the destination datawarehouse and correcting values against a known list of entities.
• Cost of BI technology: This requires well developed infrastructure in terms of high end data center servers and software for ETL (Extraction, transformation and loading) process. Developing BI applications means additional overheads on IT infrastructure leading to more financial commitment for driving business.
• Performance impact on existing infrastructure / inadequate IT infrastructure: Based on the business processes, rules and attributes defined by the business, data required for business intelligence is extracted from legacy and transaction systems, cleansed, validated and loaded into a dedicated database, known as a data warehouse. Developing BI applications using existing IT infrastructure can have a performance impact on the existing applications as it requires heavy processing during ETL processes and may be sometimes inadequate for BI processes.

Cloud computing is potentially an answer to two of these problems- second and third one. Its compute power enables organizations to analyze terabytes of data faster using BI applications more economically than ever before and is delivered in an on-demand basis. Cloud customers “rent” dedicated servers and the enterprise need to house, secure, and manage them.

Advantages of Transforming BI in Cloud

Cloud computing is transforming the economics of BI and opens up the opportunity for smaller enterprises to compete using the insight that BI provides. Cloud-based analytics will impact BI by:

• Easier evaluation of Technology: Cloud enables software companies to make new technology available to evaluators on a self-service basis, avoiding the need to download and set up free software downloads or acquire hardwares fitting to the technology.
• Increased short-term ad-hoc analysis: Where short term needs (weeks or months) for BI is required, cloud services are ideal. A data mart can be created in a few hours or days, used for the necessary period, and then cancel the cloud cluster, leaving behind no redundant hardware or software licenses. The cloud makes short term projects very economical.
• Increased flexibility: Due to the avoidance of long term financial commitments, individual business units will have the flexibility to fund more data mart projects. This is ideal for proof of concept, and ad-hoc analytic data projects on-demand. This agility enables isolated business units to respond to BI needs faster than their competitors and increase the quality of their strategy setting and execution.
• Drive data warehousing in MB markets: Medium-size businesses often have very large volumes of data for analysis, yet only a few IT resource at their disposal to analyze tons of terabytes of historical data to fine tune market strategies. Cloud-based analytics can enable such businesses to warehouse and analyze terabytes of data in spite of these resource constraints.
• Drive the analytic SaaS market: Companies that collect economic, market, advertising, scientific, and other data and then offer customers the ability to analyze it online will be able to bring their solutions to market with much less risk and cost by utilizing cloud infrastructures during the early stages of growth.

Growth Considerations

As data volumes grow, for analytic cloud projects to succeed they will require an architecture that is designed to function efficiently in elastic, hosted computing environments. At a minimum, such application must include the following architectural features:

• “Scale-out” shared-nothing architecture: To handle changing analytic workloads as elastically as the cloud. Auto-scaling of Virtual Machine (VM) can be used to proved necessary compute power required during heavy workload and an efficient  algorithm need’s to be worked out in order to auto-scale in VM’s when not required.
• Aggressive data storage: Cloud provides an appropriate infrastructure for storing large amount of data at low cost. No further additional overheads are required to store data on cloud thus helping achieve manpower savings for operations like data backup and server maintenance. For example in case of Windows Azure, Table Storage is designed to be massively scalable and a typical Azure Table can contain billions of records amassing to Terabytes of data. Blob Storage provides a means to store unstructured data much in the same way that would store a bunch of images on the File System of a server. Blobs can be mounted as XDrives on the Virtual Machine instance where a particular service is running and accessed exactly like a file system would.
• Automatic replication and failover: This will provide high availability in the cloud. In case of Windows Azure, data is stored on 3 nodes to enhance both access speeds and reduce data redundancy.

Challenges

While the current environment hints at several of the aforementioned advantages of the technology, they also bring into focus the challenges that need to be overcome in order to make BI on Cloud model work. Moving data to the cloud is an expensive proposition due to high network cost invloved in the movement of existing data from on-premise to cloud. Storing data in the cloud can be troublesome as data is core and proprietary to enterprises also BI components as services are not availabe from BI vendors.