We offer a wide array of solutions in the area of Data Analytics Services.
Big Data Solutions
Since 2000, there has been a huge upsurge in the data generated every day in the business world and this volume is more than ever in the history of humankind. How can we Analyze and Visualize this data? Apart from the Volumes, the Variety, the Velocity with which it flows in is paramount. This “VVV” data has to be managed, enriched and an insight gained into it.
ERP, SCM, CRM are classic examples of highly structured data stored in an RDBMS such as Oracle and SQL Server. Web logs, Social interactions and feeds contributed to data volumes moving from gigabytes to terabytes. RFID, GPS navigation, aircraft information moved this data to Petabytes.
When data that is generated is structured, relational data models/stores are ideal for storing and retrieving data. In contrast non-relational stores are suited for non-structured data while analysis is carried out programmatically. Modern data platforms must support both types of data equally well.
Big data needs to be managed, enriched and an insight gained into it in order to make use of this huge data available
Data Management: Regardless of whether the data is relational, non-relational or streaming data, the need is to monitor and manage it without having to worry about scale, performance, security and availability. Structured data management is supported by platforms such as SQL Server engine. Unstructured data is supported in platforms such as HDinsights which is 100 % open source implementation.
Examples of a few third party providers helping further opening up possibilities with Bigdata are Karmasphere, Datameer, & Hstreaming.
Data Discovery is made easy through data recommendation. It is important for a value based service to recommend data sets that add value to choices being made.
Example: if you consider a Customer dataset, Dunn and Bradstreet is recommended which has credit information. You can connect and combine data from hundreds of trusted data providers. Example: US census bureau.
This helps in combining personal data with organizational data with community and finally world data to enhance value for the base data considered.
Visualization and Analysis offered by PowerView and PowerPivot along with SSRS, PPS and Microsoft office shared through sharepoint collaboration offers excellent visualization
With big data gaining in a big way, offerings are available with rapidly increasing features
Data Integration Solutions
Data integration involves combining data residing in variegated sources and providing users with a integrated and unified view of this data. This process becomes significant in a variety of situations, which include both commercial (when two similar companies need to merge their databases) and scientific (combining research results from different bioinformatics repositories, for example) domains. Data integration has become the focus of extensive theoretical work, and numerous open problems remain unsolved.
A system designer constructs a mediated schema against which users can run queries. The virtual database interfaces with the source databases via wrapper code if required.
Data integration involves combining data from several disparate sources, which are stored using various technologies and provide a unified view of the data. Data integration becomes increasingly important in cases of merging systems of two companies or consolidating applications within one company to provide a unified view of the company’s data assets. The later initiative is often called a data warehouse.
Probably the most well-known implementation of data integration is building an enterprise’s data warehouse. The benefit of a data warehouse enables a business to perform analyses based on the data in the data warehouse. This would not be possible to do on the data available only in the source system. The reason is that the source systems may not contain corresponding data, even though the data are identically named, they may refer to different entities.
Data Integration Areas
Data integration is a term covering several distinct sub-areas such as:
Data Migration is the process of transferring data from one system to another while changing the storage, database or application. In reference to the ETL (Extract-Transform-Load) process, data migration always requires at least Extract and Load steps.
Typically data migration occurs during an upgrade of existing hardware or transfer to a completely new system. Examples include: migration to or from hardware platform; upgrading a database or migrating to new software; or company-mergers when the parallel systems in the two companies need to be merged into one. There are three main options to accomplish data migration:
- Merge the systems from the two companies into a brand new one
- Migrate one of the systems to the other one.
- Leave the systems as they are but create a common view on top of them – a data warehouse.
Master Data Management
The data that is available in the different systems within an organization such as a CRM system may not be in the standardized or homogeneous format. Master data management offers solutions to homogenize the data across different platforms.
In business, master data management (MDM) comprises the processes, governance, policies, standards and tools that consistently define and manage the critical data of an organization to provide a single point of reference. Master data management (MDM) is a comprehensive method of enabling an enterprise to link all of its critical data to one file, called a master file that provides a common point of reference. When properly done, master data management streamlines data sharing among personnel and departments.
- Data Cleansing is offered through SQL Server Integration Services (SSIS), Data Quality Services (DQS) and data governance through Master Data Services (MDS). Predictive analysis is possible through mining algorithms in SSAS.
Data mining is defined as extracting the required information from a huge set of data. In other words we can say that data mining is mining the knowledge from data. This information can be used for any of the following applications (though not limited to the below mentioned purposes)−
- Market Analysis
- Fraud Detection
- Customer Retention
- Production Control
- Science Exploration
Business Intelligence (BI) is a set of tools supporting the transformation of raw data into useful information which can support decision making. Business Intelligence provides reporting functionality, tools for identifying data clusters, support for data mining techniques, business performance management and predictive analysis.
The aim of Business Intelligence is to support decision making. In fact, BI tools are often called Decision Support Systems (DSS) or fact-based support systems as they provide business users with tools to analyze their data and extract information.
Business Intelligence tools often source the data from data warehouses. The reason is straightforward: a data warehouse already has data from various production systems within an enterprise; the data is cleansed, consolidated, conformed and stored in one location. Because of this BI tools are able to concentrate on analyzing the data.
Techniques Used in BI
When data is stored as a set or matrix of numbers, it is precise but difficult to interpret. For example, are sales going up, down or holding steady? When looking at more than one dimension of the data, this becomes even harder. Hence the visualization of data in charts is a convenient way to immediately understand how to interpret the data.
Data mining is a computer supported method to reveal previously unknown or unnoticed relations among data entities. Data mining techniques are used in a myriad of ways: shopping basket analysis, measurement of products consumers buy together in order to promote other products; in the banking sector, client risk assessment is used to evaluate whether the client is likely to pay back the loan based on historical data; in the insurance sector, fraud detection based on behavioral and historical data; in medicine and health, analysis of complications and/or common diseases may help to reduce the risk of cross infections.
Design, schedule and generation of the performance, sales, reconciliation and savings reports is an area where BI tools help business users. Reports output by BI tools efficiently gather and present information to support the management, planning and decision making process. Once the report is designed it can be automatically send to a predefined distribution list in the required form presenting daily/weekly/monthly statistics.
Time-series Analysis Including (Predictive Techniques)
Nearly all data warehouses and all enterprise data have a time dimension. For example, product sales, phone calls, patient hospitalizations, etc. It is extremely important to reveal the changes in user behavior in time, relation between products, or changes in sale contracts based on marketing promotion. Based on the historical data, we may also endeavor to predict future trends or outcomes.
On-line Analytical Processing (OLAP)
OLAP is best known for the OLAP-cubes which provide a visualization of multidimensional data. OLAP cubes display dimensions on the cube edges (e.g. time, product, customer type, customer age etc.). The values in the cube represent the measured facts (e.g. value of contracts, number of sold products etc.). The user can navigate through OLAP cubes using drill-up, -down and -across features. The drill-up functionality enables the user to easily zoom out to more coarse-grained details. Conversely, drill-down displays the information with more details. Finally, drilling-across means that the user can navigate to another OLAP cube to see the relations on another dimension(s). All the functionality is provided in real-time.
Statistical analysis uses the mathematic foundations to qualify the significance and reliability of the observed relations. The most interesting features are distribution analysis, confidence intervals (for example for changes in user behaviors, etc). Statistical analysis is used for devising and analyzing the results from data mining.
Popular Business Intelligence Tools
- Oracle Enterprise BI Server
- SAP Business Objects Enterprise
- SAP NetWeaver BI
- SAS Enterprise BI Server
- Microsoft BI platform
- IBM Cognos Series 8
- Board Management IntelligenceToolkit
- BizzScore Suite
- Oracle Hyperion System