MY WORK DIARY

  Apache Spark Getting Started Explore the basics of Apache Spark, an analytics engine used for big data processing. It's an open source, cluster computing framework built on top of Hadoop. Discover how it allows operations on data with both its own library methods and with SQL, while delivering great performance. Learn the characteristics, components, and functions of Spark, Hadoop, RDDS, the spark session, and master and worker notes. Install PySpark. Then, initialize a Spark Context and Spark DataFrame from the contents of an RDD and a DataFrame. Configure a DataFrame with a map function. Retrieve and transform data. Finally, convert Spark and Pandas DataFrames and vice versa. Table of Contents Course Overview Introduction to Spark and Hadoop Resilient Distributed Datasets (RDDs) RDD Operations Spark DataFrames Spark Architecture Spark Installation Working with RDDs Creating DataFrames from RDDs Contents of a DataFrame The SQLContext The map() Function of an RDD Accessing the Co

  Azure Data Fundamentals: Azure Analytics Workloads Azure Synapse Analytics is a limitless analytics service that brings together data warehousing and big data analytics. In this course, you will learn about analytics workloads, including Azure Synapse Analytics, Azure Synapse SQL pool, Data Warehouse Units. You'll also learn about the difference between transactional and analytic workloads and batch and real time data processing. You'll use Azure Portal and Azure PowerShell to create a Synapse SQL pool and Azure Data Lake Analytics. You'll learn about data warehousing workloads and when to use a data warehouse solution. Finally, you'll learn about the different Azure Data Lake Analytics. This course is one in a series that prepares learners for the Microsoft Azure Data Fundamentals (DP-900) exam. Table of Contents Course Overview Azure Synapse Analytics Architecture Using the Azure Synapse SQL Pool Data Warehouse Units Transactional and Analytic Workloads Batch and Re