Skip to content

Data: The Foundation of Machine Learning

This tutorial introduces the essential role of data in machine learning and walks through the key phases of preparing data for modeling. You'll understand why data quality matters more than algorithm choice and learn the systematic approach to handling data from collection to model-ready format.

Estimated time: 40 minutes

Why This Matters

Problem statement:

The most sophisticated machine learning algorithm is worthless without good data.

Models learn patterns from data, so if your data is biased, incomplete, or messy, your predictions will be unreliable regardless of model complexity. The phrase "garbage in, garbage out" is fundamental, because poor data quality leads directly to poor model performance.

Practical benefits: Understanding data workflows enables you to spot quality issues early, choose appropriate preprocessing strategies, and build robust ML systems that work in production.

Professional context: Industry surveys consistently show that data scientists spend 60-80% of their time on data preparation rather than modeling. Companies with strong data infrastructure outperform competitors regardless of algorithm sophistication. The most impactful ML practitioners aren't necessarily those who know the fanciest algorithms; they're the ones who understand their data deeply, handle it correctly, and ensure quality throughout the pipeline.

Data Overview

Core Concepts

What is Data in ML Context?

In machine learning, data consists of examples (also called samples, observations, or rows) with features (attributes, variables, or columns) and optionally labels (target values for supervised learning).

Example: Predicting house prices

  • Sample: One house
  • Features: Square footage, number of bedrooms, location, age, etc.
  • Label: Sale price (what we're trying to predict)

Types of Data

Structured vs Unstructured:

  • Structured: Organized in tables with defined columns (databases, spreadsheets)
  • Unstructured: No predefined format (text, images, audio, video)

Numerical vs Categorical:

  • Numerical: Quantitative values (age: 25, temperature: 18.5°C)
  • Categorical: Discrete categories (color: red/blue/green, city: Tirana/Durrës)

Time-series: Data points ordered by time (stock prices, sensor readings, web traffic)

Data Quality Dimensions

Completeness: Are all expected values present? Missing data can bias results.

Accuracy: Do values reflect reality? Measurement errors propagate through models.

Consistency: Do values follow expected formats and rules? Inconsistencies cause processing failures.

Relevance: Do features relate to your prediction task? Irrelevant features add noise.

The Data Pipeline: Key Phases

The machine learning data pipeline is a series of steps that transform raw data into a format suitable for training models. Think of it as a factory assembly line where raw materials become finished products.

Data Pipeline

Phase 1: Data Collection

Purpose: Gather data from various sources to create your dataset.

Common sources:

  • Databases: SQL/NoSQL databases (MySQL, PostgreSQL, MongoDB)
  • APIs: Web services providing data programmatically
  • Web scraping: Extracting data from websites
  • Files: CSV, JSON, Excel files
  • Sensors/IoT: Real-time streaming data
  • User input: Forms, surveys, clicks

Key considerations:

  • Representative samples: Data should reflect the real-world distribution you'll encounter
  • Bias awareness: Who collected this data? What populations are missing?
  • Volume: Do you have enough samples for your task? Rule of thumb: at least 10 samples per feature

How you collect data determines what patterns your model can learn.

If your training data only includes customers from one region, your model won't work well for other regions.

Example: Building a spam detector

  • Good collection: Emails from diverse users, timeframes, and domains
  • Poor collection: Only emails from one company's inbox over one week

Phase 2: Data Exploration (EDA)

Purpose: Understand what you have before making changes.

Key questions:

  • What does the data look like? (first few rows)
  • What types are the features? (numerical, categorical, dates)
  • What are the distributions? (histograms, box plots)
  • Are there obvious outliers or anomalies?
  • What patterns exist? (correlations, trends)
  • What's missing? (null values, empty fields)

Tools: Summary statistics (mean, median, std), visualizations (histograms, scatter plots), correlation matrices

ML connection: Exploration guides your modeling choices. If features are highly correlated, some algorithms may struggle. If classes are imbalanced, accuracy might be misleading. If distributions are skewed, you might need transformations.

Real example: Exploring customer data reveals that 95% are from age 25-35. This tells you your model might not work well for older customers—you need more diverse data or age-specific models.

Phase 3: Data Cleaning

Purpose: Fix or remove problems that would cause errors or bias.

Common tasks:

Handling missing values:

  • Remove: Delete rows/columns with missing data (if small percentage)
  • Impute: Fill with mean, median, mode, or predicted values
  • Flag: Create indicator variable showing missingness

Dealing with duplicates:

  • Identify exact or near-duplicates
  • Remove or merge duplicate records

Fixing inconsistencies:

  • Standardize formats (dates: YYYY-MM-DD, names: Title Case)
  • Correct typos and data entry errors
  • Resolve conflicting values

Handling outliers:

  • Detect using statistical methods (Z-score, IQR)
  • Decide: Remove, cap, or keep based on domain knowledge

ML connection: Dirty data creates noisy patterns that models learn incorrectly. One extreme outlier can skew an entire regression line. Missing values handled poorly can introduce bias or cause crashes during training.

Data Cleaning

Phase 4: Data Transformation

Purpose: Convert data into forms that algorithms can process effectively.

Common transformations:

Scaling and normalization:

  • Min-max scaling: Squash values to range
  • Standardization: Center at 0 with std dev of 1
  • Why: Prevents features with large values from dominating distance-based algorithms

Encoding categorical variables:

  • Label encoding: Assign numbers (red=0, blue=1, green=2)
  • One-hot encoding: Create binary columns for each category
  • Why: Most ML algorithms require numerical input

Feature engineering:

  • Combining features: Create "price per square foot" from price and area
  • Extracting components: Pull day-of-week from datetime
  • Binning: Convert continuous age to age groups
  • Why: New features can capture relationships better than original data

ML connection: Proper scaling is critical for gradient descent (used in neural networks) and distance-based algorithms (k-NN, k-means). Without it, features with larger scales dominate, leading to poor performance.

Example: In predicting house prices

  • Feature 1: Square footage (500-5000)
  • Feature 2: Number of bedrooms (1-5)

Without scaling, square footage dominates. After standardization, both contribute equally.

Phase 5: Data Splitting

Purpose: Separate data to train models and evaluate them fairly.

Standard split:

  • Training set (70-80%): Used to train the model
  • Validation set (10-15%): Used to tune hyperparameters and select models
  • Test set (10-15%): Used only once at the end for final evaluation

Why split? A model evaluated on training data appears better than it actually is (overfitting). Testing on unseen data reveals true performance.

ML connection: Without proper splitting, you can't trust performance metrics. A model with 99% accuracy on training data might have 60% on new data. The test set simulates how your model will perform in production.

Critical rule: NEVER look at test set during development. It's your final "exam"—use it only once.

Data Splitting

Phase 6: Data Storage & Versioning

Purpose: Organize datasets for reproducibility and collaboration.

Best practices:

  • Keep raw data separate: Never modify original files
  • Version datasets: Track changes like you track code (DVC, Git LFS)
  • Document transformations: Record every step applied
  • Use consistent naming: data_raw.csv, data_cleaned.csv, data_train.csv

ML connection: When model performance changes, you need to know if it's due to code changes or data changes. Versioning enables debugging, auditing, and reproducing results months later.

Common Data Challenges

Insufficient Data

Problem: Too few samples for algorithms to learn patterns effectively.

Impact: High variance, poor generalization, overfitting to noise.

Solutions: Data augmentation, transfer learning, collect more data.

Imbalanced Data

Problem: Unequal representation of classes (95% normal, 5% fraud).

Impact: Models predict majority class for everything, achieving high accuracy but missing minority class entirely.

Solutions: Resampling, cost-sensitive learning, different metrics (F1-score, not accuracy).

High Dimensionality

Problem: Too many features relative to samples (curse of dimensionality).

Impact: Models require exponentially more data, distance metrics become meaningless, increased computation.

Solutions: Feature selection, dimensionality reduction (PCA), regularization.

Data Drift

Problem: Production data distribution differs from training data over time.

Impact: Model performance degrades silently in production.

Solutions: Continuous monitoring, periodic retraining, drift detection systems.

Best Practices

Start with exploration: Look at your data before deciding on approaches. One hour of exploration saves ten hours of debugging.

Document everything: Record data sources, transformations, and decisions. Your future self will thank you.

Version control datasets: Treat data like code. Track changes and tag versions used for specific models.

Check for bias early: Examine demographic representation, temporal coverage, and geographic diversity before investing in modeling.

Validate continuously: Don't wait until the end to check data quality. Validate at each pipeline stage.

Keep raw data untouched: Always maintain original data. Apply transformations in code, never by editing files directly.

Test pipeline with small samples: Verify your entire workflow on small data before processing millions of records.

Quick Reference: Data Pipeline Phases

Phase Key Question Tools/Techniques Next Tutorial
Collection Where does data come from? APIs, databases, web scraping Data Collection
Exploration What does data look like? Summary stats, visualizations Pandas, Visualization
Cleaning What's wrong with data? Handle missing, duplicates, outliers Data Wrangling
Transformation How to prepare for models? Scaling, encoding, feature engineering Pandas, NumPy
Splitting How to evaluate fairly? Train/val/test split, cross-validation Pandas
Storage How to organize and track? File structures, versioning, documentation Data Collection

Summary & Next Steps

Key accomplishments: You've learned why data quality determines ML success, understood the systematic data pipeline workflow, recognized common data challenges and their impacts, and connected proper data handling to model performance and production readiness.

Critical insights:

  • 80/20 rule in practice: Most ML work is data preparation, not modeling
  • Quality over quantity: Small, clean datasets often outperform large, messy ones
  • Domain knowledge matters: Understanding context helps make better data decisions
  • Iteration is normal: Data work is cyclical—exploration reveals cleaning needs, cleaning enables better exploration

Connections to previous topics:

  • Statistics: Use mean, variance, distributions to understand data patterns
  • Probability: Sampling strategies affect what patterns models can learn
  • Linear algebra: Data is organized as matrices (rows = samples, columns = features)

What's next in this section:

Pandas Essentials: Load, filter, transform, and aggregate data efficiently

Visualization: Create plots to explore patterns and communicate insights

Data Wrangling: Handle missing values, merge datasets, reshape data structures

Data Collection: APIs, web scraping, and database queries for gathering data

SQL & NoSQL: Query structured and unstructured data stores

Each tutorial builds hands-on skills for specific pipeline phases. By the end, you'll have a complete workflow from raw data to model-ready datasets.

External resources:

Remember: Great models start with great data. Master the pipeline, and you'll build better ML systems than those who chase fancy algorithms with messy data.