When will I have access to the lectures and assignments?

To access the course materials, assignments and to earn a Certificate, you will need to purchase the Certificate experience when you enroll in a course. You can try a Free Trial instead, or apply for Financial Aid. The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.

What will I get if I subscribe to this Certificate?

When you enroll in the course, you get access to all of the courses in the Certificate, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.

Data Analytics and Machine Learning for Big Data

This course is part of Microsoft Big Data Management and Analytics Professional Certificate

Instructor: Microsoft

Included with

Learn more

5 modules

Gain insight into a topic and learn the fundamentals.

Intermediate level

Recommended experience

3 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

5 modules

Gain insight into a topic and learn the fundamentals.

Intermediate level

Recommended experience

3 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

What you'll learn

- Manage big data storage and pipelines with Azure services.
- Process and analyze large datasets using Apache Spark and Databricks.

Skills you'll gain

Tools you'll learn

Apache Spark

Details to know

Shareable certificate

Add to your LinkedIn profile

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

logos of Petrobras, TATA, Danone, Capgemini, P&G and L'Oreal

Build your Data Analysis expertise

This course is part of the Microsoft Big Data Management and Analytics Professional Certificate

When you enroll in this course, you'll also be enrolled in this Professional Certificate.

Learn new concepts from industry experts
Gain a foundational understanding of a subject or tool
Develop job-relevant skills with hands-on projects
Earn a shareable career certificate from Microsoft

There are 5 modules in this course

This advanced course teaches machine learning and AI techniques for big data systems. Learners will build end-to-end ML pipelines with PySpark ML, implement supervised and unsupervised models, and apply NLP techniques at scale. The course also explores deep learning, distributed training, and integrating Generative AI into big data workflows.

By the end of this course, you will be able to: - Implement ML pipelines using PySpark ML - Build supervised, unsupervised, and recommendation models - Apply NLP and text analytics to large datasets -Integrate Generative AI and LLMs with big data systems Tools & Software: PySpark ML, PyTorch, TensorFlow, Azure Machine Learning, Azure OpenAI Service Skills: Machine learning, NLP, Deep learning, Generative AI, Model evaluation

Module details

Machine learning appears quite different when data exceeds the capacity of a single system. In this section, learners explore the foundational ideas behind machine learning in big data environments and how familiar approaches change at scale. You will examine supervised and unsupervised learning, regression and classification problems, and the practical challenges that arise with massive datasets—such as scalability, distributed computing, and the need to adapt algorithms for large-scale processing.

What's included

6 videos3 readings7 assignments

6 videosTotal 29 minutes

Machine Learning Transforms Big Data into Business Intelligence4 minutes
ML Problem Classification and Business Mapping7 minutes
Data Quality Drives ML Success at Scale4 minutes
Distributed Data Preparation Workflows6 minutes
Rigorous Evaluation Prevents ML Disasters at Scale4 minutes
Implementing Scalable Model Evaluation5 minutes

3 readingsTotal 30 minutes

Machine Learning Fundamentals for Big Data Environments10 minutes
Big Data ML Preparation Techniques10 minutes
ML Model Evaluation for Big Data Systems10 minutes

7 assignmentsTotal 210 minutes

ML Fundamentals for Big Data Mastery30 minutes
Machine Learning Problem Analysis30 minutes
ML Fundamentals for Big Data Assessment30 minutes
ML Data Preparation Pipeline30 minutes
Data Preparation for ML at Scale Assessment30 minutes
Scalable Model Evaluation30 minutes
Model Evaluation at Scale Assessment30 minutes

A practical foundation for building scalable machine learning solutions using PySpark ML in big data environments. The content focuses on designing and implementing end-to-end machine learning pipelines with transformers and estimators, while developing regression, classification, and clustering models that scale across distributed systems. Emphasis is placed on real-world implementation and informed platform selection for enterprise deployments using Azure Databricks, Microsoft Fabric, and Azure HDInsight, ensuring solutions are both technically robust and operationally viable at scale.

What's included

6 videos3 readings10 assignments

6 videosTotal 36 minutes

Democratizing Machine Learning at Enterprise Scale4 minutes
PySpark ML Pipeline Development Across Platforms10 minutes
Supervised Learning Success Stories in Enterprise Big Data5 minutes
Supervised Learning Model Development6 minutes
Recommendation Systems Drive Business Growth4 minutes
Building Scalable Recommendation Systems8 minutes

3 readingsTotal 30 minutes

PySpark ML Architecture and Platform Comparison10 minutes
Supervised Learning Algorithms for Big Data10 minutes
Unsupervised Learning and Recommendation Systems10 minutes

10 assignmentsTotal 300 minutes

PySpark ML Implementation Mastery30 minutes
ML Pipeline Component Development30 minutes
ML Platform Comparison and Pipeline Creation30 minutes
PySpark ML Platform Fundamentals Assessment30 minutes
Supervised Learning Implementation30 minutes
Supervised Learning Model Development30 minutes
Supervised Learning at Scale Assessment30 minutes
Recommendation System Implementation30 minutes
Recommendation System Development30 minutes
Unsupervised Learning and Recommendations Assessment30 minutes

Large-scale text analytics introduces the challenges and techniques required to process and analyze unstructured text at enterprise scale using distributed computing frameworks. The focus is on applying natural language processing (NLP) techniques in scalable architectures to support text classification, sentiment analysis, and entity and relationship extraction across massive text corpora. Emphasis is placed on practical, production-oriented approaches for handling high-volume text data, with integration of Azure Cognitive Services to enhance accuracy, scalability, and operational efficiency in real-world analytics solutions.

What's included

6 videos3 readings10 assignments

6 videosTotal 39 minutes

Unlocking Value from Unstructured Text at Scale5 minutes
Building Scalable Text Processing Pipelines9 minutes
Advanced NLP Drives Business Intelligence5 minutes
Implementing Advanced NLP at Scale7 minutes
Production-Scale Text Classification Transforms Business Operations4 minutes
Building Production Text Classification Systems8 minutes

3 readingsTotal 30 minutes

Distributed Text Processing Techniques10 minutes
Advanced NLP Techniques for Big Data10 minutes
Scalable Text Classification Architectures10 minutes

10 assignmentsTotal 300 minutes

Text Analytics and NLP Mastery30 minutes
Text Preprocessing Pipeline Development30 minutes
Scalable Text Preprocessing Design30 minutes
Text Processing at Scale Assessment30 minutes
Advanced NLP Implementation and Monitoring30 minutes
NLP System Architecture Design30 minutes
Advanced NLP Techniques Assessment30 minutes
Text Classification System Development30 minutes
Text Classification System Implementation30 minutes
Text Classification at Scale Assessment30 minutes

Deep Learning for Big Data introduces the fundamentals of deep learning and advanced architectures specifically adapted for big data environments. Students will learn to implement neural networks for big data applications, apply transfer learning techniques with pre-trained models, and scale deep learning training across distributed clusters using modern frameworks and optimization techniques.

What's included

6 videos3 readings10 assignments

6 videosTotal 31 minutes

Deep Learning Revolutionizes Big Data Analytics5 minutes
Neural Network Implementation in Big Data Frameworks5 minutes
Advanced Architectures Transform Complex Data Analysis6 minutes
CNN and RNN Implementation at Scale5 minutes
Distributed Deep Learning Enables Breakthrough Scale4 minutes
Implementing Distributed Deep Learning Training5 minutes

3 readingsTotal 30 minutes

Deep Learning Architectures for Big Data10 minutes
Advanced Deep Learning Architectures for Scale10 minutes
Distributed Deep Learning Training Strategies10 minutes

10 assignmentsTotal 300 minutes

Deep Learning for Big Data Mastery30 minutes
Neural Network Implementation30 minutes
Neural Network for Big Data Classification30 minutes
Deep Learning Fundamentals Assessment30 minutes
Advanced Architecture Implementation30 minutes
Deep Learning Architecture Design30 minutes
Advanced Deep Learning Architectures Assessment30 minutes
Distributed Training Implementation and Management30 minutes
Distributed Deep Learning Training30 minutes
Distributed Deep Learning Training Assessment30 minutes

Generative AI and Big Data Integration explores how generative AI transforms big data analytics by enabling intelligent, natural language–driven workflows at scale. You will learn how foundation models and large language models integrate with distributed data pipelines to automate insights, enhance analytics, and power modern data applications. Through hands-on labs, you will implement LLM integration, apply fine-tuning for domain-specific use cases, and design production-ready GenAI solutions for real-world big data scenarios.