When will I have access to the lectures and assignments?

To access the course materials, assignments and to earn a Certificate, you will need to purchase the Certificate experience when you enroll in a course. You can try a Free Trial instead, or apply for Financial Aid. The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.

What will I get if I subscribe to this Certificate?

When you enroll in the course, you get access to all of the courses in the Certificate, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.

Data Processing, Exploratory Analysis and Visualization

This course is part of Microsoft Big Data Management and Analytics Professional Certificate

Instructor: Microsoft

Included with

Learn more

5 modules

Gain insight into a topic and learn the fundamentals.

Intermediate level

Recommended experience

3 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

5 modules

Gain insight into a topic and learn the fundamentals.

Intermediate level

Recommended experience

3 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

Skills you'll gain

Details to know

Shareable certificate

Add to your LinkedIn profile

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

logos of Petrobras, TATA, Danone, Capgemini, P&G and L'Oreal

Build your Data Analysis expertise

This course is part of the Microsoft Big Data Management and Analytics Professional Certificate

When you enroll in this course, you'll also be enrolled in this Professional Certificate.

Learn new concepts from industry experts
Gain a foundational understanding of a subject or tool
Develop job-relevant skills with hands-on projects
Earn a shareable career certificate from Microsoft

There are 5 modules in this course

This course introduces distributed computing frameworks and big data visualization techniques. Learners will explore MapReduce, work with Apache Spark, implement transformations with PySpark, and use Spark SQL for large-scale analysis. The course concludes with building compelling dashboards and reports using Power BI for actionable business insights.

By the end of this course, you will be able to: - Explain distributed computing and MapReduce concepts - Process large datasets using Apache Spark and PySpark - Apply Spark SQL for advanced queries and transformations - Create dashboards and visualizations using Power BI Tools & Software: Apache Spark, PySpark, Azure Databricks, Power BI Skills: Distributed computing, Data analysis, PySpark, Spark SQL, Data visualization

Module details

Distributed Computing and MapReduce Concepts explores the foundational principles that enable modern organizations to process massive datasets that have outgrown the limits of single-machine computing. Through real-world examples, visual walkthroughs, hands-on labs, and guided design activities, you'll examine how data is broken into parallel tasks and executed across clusters of machines, how the Map, shuffle, and Reduce phases work together, and how common MapReduce patterns—such as counting, filtering, joining, and aggregation—solve practical big data problems efficiently and at scale.

What's included

6 videos3 readings8 assignments

6 videosTotal 36 minutes

The Scale Challenge in Modern Computing4 minutes
Visualizing Distributed Processing Workflows7 minutes
Simplifying Complex Problems with MapReduce5 minutes
Tracing MapReduce Execution Flow8 minutes
MapReduce Patterns in Production Systems5 minutes
Implementing MapReduce Patterns8 minutes

3 readingsTotal 30 minutes

Distributed Computing Principles for Big Data10 minutes
MapReduce Programming Model Deep Dive10 minutes
Essential MapReduce Patterns and Algorithms10 minutes

8 assignmentsTotal 240 minutes

Distributed Computing Analysis30 minutes
Distributed Computing Concepts Assessment30 minutes
MapReduce Algorithm Design30 minutes
MapReduce Execution Tracing30 minutes
MapReduce Programming Model Assessment30 minutes
MapReduce Solution Design30 minutes
MapReduce Patterns and Applications Assessment30 minutes
Distributed Computing and MapReduce Mastery Graded Quiz30 minutes

Apache Spark Architecture and Fundamentals provides a comprehensive introduction to the distributed processing engine that revolutionized big data analytics by overcoming traditional MapReduce limitations. Through real-world examples, visual walkthroughs, hands-on labs, and guided design activities, you'll examine Spark's core components, including the driver, executors, and cluster manager, explore how in-memory processing delivers dramatic performance improvements, and learn to configure and manage Spark clusters and applications for efficient large-scale data processing.

What's included

7 videos3 readings9 assignments

7 videosTotal 40 minutes

Spark's Revolution in Big Data Processing4 minutes
Spark Cluster Setup and Configuration – Part 16 minutes
Spark Cluster Setup and Configuration – Part 25 minutes
Inside Spark's Intelligent Execution Engine5 minutes
Analyzing Spark Execution Plans8 minutes
The Foundation of Spark's Power5 minutes
Hands-on RDD Programming7 minutes

3 readingsTotal 30 minutes

Apache Spark Architecture Deep Dive10 minutes
Spark Execution Model and Optimization10 minutes
RDD Programming Model and Operations10 minutes

9 assignmentsTotal 270 minutes

Spark Environment Setup30 minutes
Spark Architecture Components Assessment30 minutes
Spark Job Analysis30 minutes
Spark Execution Analysis30 minutes
Spark Execution Optimization Assessment30 minutes
RDD Operations and Lineage30 minutes
RDD Programming Practice30 minutes
RDD Programming Concepts Assessment30 minutes
Apache Spark Fundamentals Mastery Graded Quiz30 minutes

Data Processing with PySpark RDDs and DataFrames focuses on practical data processing using PySpark's Python API for Apache Spark. Through real-world examples, visual walkthroughs, hands-on labs, and guided design activities, you'll implement data processing operations using both RDDs and DataFrames, develop transformation pipelines, apply common data cleaning and preparation techniques, and optimize PySpark code for better performance across enterprise-scale big data scenarios.

What's included

6 videos3 readings10 assignments

6 videosTotal 37 minutes

Python Meets Big Data with PySpark4 minutes
PySpark Development Workflow9 minutes
DataFrames: Structured Big Data Made Simple4 minutes
DataFrame Operations and Schema Management8 minutes
Advanced Analytics with PySpark Transformations5 minutes
Building Complex Transformation Pipelines7 minutes

3 readingsTotal 30 minutes

PySpark Development Environment and Best Practices10 minutes
PySpark DataFrame Programming Guide10 minutes
Advanced PySpark DataFrame Operations10 minutes

10 assignmentsTotal 300 minutes

PySpark Environment Setup30 minutes
PySpark Development Environment30 minutes
PySpark Development Fundamentals Assessment30 minutes
DataFrame Schema and Operations30 minutes
DataFrame Data Cleaning Pipeline30 minutes
DataFrame Operations and Schema Assessment30 minutes
Advanced Transformation Patterns30 minutes
Complex Analytics Pipeline30 minutes
Advanced Transformation Techniques Assessment30 minutes
PySpark Data Processing Mastery Graded Quiz30 minutes

Advanced Data Processing with Spark SQL introduces Spark SQL as a powerful interface for structured data processing in distributed environments. Through real-world examples, visual walkthroughs, hands-on labs, and guided design activities, you'll master SQL operations at scale, from basic queries to complex analytical operations, learn to create and manage temporary views and tables, and optimize query performance for production workloads that would overwhelm traditional database systems.

What's included

6 videos3 readings10 assignments

6 videosTotal 35 minutes

SQL at Scale with Spark SQL4 minutes
Spark SQL Environment and Basic Queries7 minutes
Enterprise Analytics with Advanced Spark SQL5 minutes
Implementing Complex Analytical Queries7 minutes
Optimizing Spark SQL for Production Performance5 minutes
Query Performance Analysis and Tuning7 minutes

3 readingsTotal 30 minutes

Spark SQL Architecture and Programming Model10 minutes
Advanced Spark SQL Operations and Optimization10 minutes
Spark SQL Performance Tuning and Optimization10 minutes

10 assignmentsTotal 300 minutes

Spark SQL Views and Queries30 minutes
Spark SQL Environment Setup30 minutes
Spark SQL Fundamentals Assessment30 minutes
Complex SQL Query Development30 minutes
Advanced Analytics Pipeline30 minutes
Advanced SQL Operations Assessment30 minutes
Query Optimization Practice30 minutes
Comprehensive Performance Optimization30 minutes
Query Optimization Techniques Assessment30 minutes
Spark SQL Advanced Processing Mastery Graded Quiz 30 minutes

Data Visualization for Big Data with Power BI introduces comprehensive visualization techniques specifically designed for big data environments using Microsoft Power BI. Through real-world examples, visual walkthroughs, hands-on labs, and guided design activities, you'll learn to connect Power BI to various big data sources, create effective visualizations for large datasets, build interactive dashboards that enable self-service analytics, and implement best practices for handling performance challenges when visualizing massive datasets.