APPLIED BIG DATA: STORAGE, INTEGRATION, QUALITY, PARALLEL AND DISTRIBUTED COMPUTING

The book begins with mass data storage and data processing. Data storage systems and advanced clustered storage methods are presented. Structured and unstructured data, compatibility issues, data quality, data security and cluster processing are considered. Big data analytics, its steps, capabilities, data sources and use cases are also developed. Big data in the cloud through cloud services is also discussed. Distributed Computing, its operation, models, advantages, disadvantages and uses are then discussed. In addition, parallel computing is discussed with its evolution, types, advantages, disadvantages, design patterns, memory architectures, synchronisation, languages, models and uses. Examples of parallel and distributed computing are presented using MATLAB's Parallel Computing tool and Apache Spark. Distributed storage systems are also developed and distributed database management systems are analysed, presenting examples with SQL Server. High availability, fault tolerance and their implementation on the web and in the cloud are also addressed. It also discusses massive computing tools in Big Data ecosystems with a focus on Hadoop, Mapreduce, Hadoop Distribute File System and Hadoop Common Components (Pig, Hive, Flume, Oozie, Hbase, Sqoop, Mahout and others). This is followed by job automation and examples developed with SQL Server. The Hadoop ecosystem of Apache Ambari is also presented. Analytics techniques (essentially Data Mining and Business Intelligence) and Big Data go hand in hand for the optimal exploitation of information. Data quality and integrity in Big Data processes and the movement of data between clusters are then addressed. As an example, the copying and movement of databases between servers in SQL Server is developed. It ends with the import and integration of data. Read more