1. Why Data Preparation Matters

Data preparation is the most time-consuming part of any ML project — typically 60–80% of the total effort. Poor data leads to poor models regardless of algorithm choice. This lesson covers the key concepts, techniques, and AWS services for preparing data for ML.


2. Data Preparation Pipeline on AWS

ML Data Preparation Pipeline:

1. COLLECT
   Sources: S3, RDS, DynamoDB, Kinesis, APIs, third-party
   Tools:   DMS, DataSync, Kinesis Firehose, Glue ETL

2. EXPLORE & PROFILE
   Understand: distributions, missing values, outliers, correlations
   Tools:   SageMaker Data Wrangler, Glue DataBrew, Athena

3. CLEAN
   Fix: missing values, duplicates, invalid data, inconsistencies
   Tools:   Data Wrangler, DataBrew, Glue ETL, SageMaker Processing

4. TRANSFORM & ENGINEER
   Create: new features, encode categories, normalize, scale
   Tools:   Data Wrangler, SageMaker Processing, Glue ETL

5. LABEL
   Annotate: label data for supervised learning
   Tools:   SageMaker Ground Truth

6. SPLIT
   Divide: training set (70-80%), validation set (10-15%), test set (10-15%)

7. STORE & VERSION
   Store: S3 (Parquet), Feature Store (reusable features)
   Tools:   SageMaker Feature Store, S3 versioning

3. Data Types for ML

4. Data Cleaning Techniques

Missing Values Strategy The right approach depends on: how much data is missing (if <5% rows, removing is OK), the pattern (random vs systematic), and the feature importance. Imputation preserves data volume but introduces assumptions. Flagging missing values as a separate indicator can itself be informative for the model.

5. Feature Engineering

Feature engineering transforms raw data into features that better represent the underlying problem, improving model accuracy. It is often the single most impactful step in ML.

Important Warning Normalization vs Standardization: Use normalization (Min-Max) when features have bounded ranges, and you need values in [0,1] (e.g., neural networks, KNN). Use standardization (Z-Score) when features have different units/scales, and the algorithm assumes normal distribution (e.g., linear regression, SVM). Many tree-based models (XGBoost, Random Forest) do NOT require scaling.

6. Data Labeling


SageMaker Ground Truth

Ground Truth is a data labeling service that builds high-quality training datasets using human labelers, pre-trained ML models, or a combination (active learning).

7. Data Splitting

Split your dataset into three parts to properly evaluate model performance.

Critical Rule NEVER use the test set during training or hyperparameter tuning. The test set must remain completely unseen until final evaluation. If you tune based on test results, your evaluation is biased and doesn’t reflect real-world performance. Validation set = tuning. Test set = final report card.

8. AWS Data Preparation Services Comparison

9. Data Quality & Bias

Data Quality Dimensions

Bias in Training Data

  1. Selection bias: training data doesn’t represent the real population (e.g., only data from one region)
  2. Measurement bias: data collection method introduces systematic errors (e.g., surveys only from tech-savvy users)
  3. Label bias: Labelers introduce subjective bias in annotations
  4. Historical bias: data reflects past discrimination (e.g., biased hiring data perpetuates bias)
  5. AWS tool: SageMaker Clarify detects pre-training bias in datasets and post-training bias in models

10. Data Formats for SageMaker

Exam Tip Data Prep: "80% of ML time" = data preparation. "Visual feature engineering" = Data Wrangler. "No-code data cleaning" = DataBrew. "Label training data" = Ground Truth. "Active learning" = Ground Truth auto-labels + humans. "Reuse features" = Feature Store. "Detect data bias" = Clarify. "SageMaker-optimized format" = RecordIO-Protobuf. "Fine-tuning data format" = JSONL. NEVER use a test set for tuning. Training/Validation/Test = 70/15/15.