RugvAI Labs

Testing with pytest: Beyond Basic assert

pytest runs the test suites at NASA, CERN, FastAPI, and Anthropic. Mastering fixtures, parametrize, and conftest is the difference between "my code works on my laptop" and "my code ships to a million users with confidence." This is the most leveraged 30 minutes you'll spend in this entire track.

Learning Objectives

After this lesson, you will be able to:

Write parametrized tests with @pytest.mark.parametrize to eliminate test duplication and get per-case failure reports
Use fixtures for reusable setup and teardown, choosing the right scope (function, module, session) for each resource
Share fixtures across test files with conftest.py and understand fixture discovery order
Patch external dependencies and environment variables using monkeypatch to isolate units under test
Test exceptions correctly with pytest.raises and floating-point equality with pytest.approx
Measure and interpret test coverage with pytest-cov, and understand why 100% coverage is not the goal

Prerequisites

Before You Start

Review

Why

Why This Matters

Every ML library, every production Python service, every open-source project uses pytest -- when you look at PyTorch, scikit-learn, FastAPI, or Airflow on GitHub, their test directories all use pytest. Knowing pytest is not optional for production Python work
The type-hints-testing lesson teaches assert x == y — that is fine for a script -- but it does not scale. Five tests become fifty. Testing every edge case means copying the same function call ten times with slightly different inputs. pytest's parametrize, fixtures, and plugins solve this at the framework level
pytest finds bugs that assert-based tests miss -- specifically: (a) which inputs cause failures (assert tells you the assertion failed, pytest shows you which parametrized case); (b) floating-point comparisons (assert 0.1 + 0.2 == 0.3 silently fails); (c) whether exceptions are raised with the right message (not just any exception)

What Do You Think?

You have a function test_add_numbers() that does assert add(2, 2) == 4. pytest runs it. The test fails. What does pytest display that plain python test_file.py would NOT show you?

#Why pytest Over unittest

Before pytest, Python testing meant inheriting from unittest.TestCase:

pythonreference · read-only

# unittest style — verbose, but you'll see this in legacy code
import unittest

class TestTemperatureConverter(unittest.TestCase):
    def test_freezing(self):
        self.assertEqual(celsius_to_fahrenheit(0), 32)

    def test_boiling(self):
        self.assertEqual(celsius_to_fahrenheit(100), 212)

    def test_negative(self):
        self.assertAlmostEqual(celsius_to_fahrenheit(-40), -40)

if __name__ == "__main__":
    unittest.main()

pythonreference · read-only

# pytest style — just functions, standard assert
def test_freezing():
    assert celsius_to_fahrenheit(0) == 32

def test_boiling():
    assert celsius_to_fahrenheit(100) == 212

def test_negative():
    assert celsius_to_fahrenheit(-40) == pytest.approx(-40)

Feature	unittest	pytest
Boilerplate	Class + methods	Just functions
Assertion style	self.assertEqual, assertIn, ...	Standard assert
Failure messages	Generic	Shows actual values
Parametrize	Manual or ddt plugin	Built-in decorator
Fixtures	setUp / tearDown	@pytest.fixture (composable)
Plugin ecosystem	Minimal	1000+ plugins

pytest is also fully compatible with unittest-style tests — you can run both with pytest.

#Basic pytest

#File and Function Naming

pytest discovers tests by convention:

pytest Naming Conventions

Pattern	Alternative	What pytest collects
test_*.py	*_test.py	Test files
test_*()	test_* methods	Test functions/methods

pythonrunnable cell

# test_statistics.py

def mean(values: list[float]) -> float:
    """Calculate the arithmetic mean."""
    if not values:
        raise ValueError("Cannot compute mean of empty list")
    return sum(values) / len(values)


def test_mean_basic():
    assert mean([1, 2, 3]) == 2.0

def test_mean_single_element():
    assert mean([42]) == 42.0

def test_mean_with_negatives():
    assert mean([-1, 1]) == 0.0

Run with: pytest test_statistics.py -v

test_statistics.py::test_mean_basic          PASSED
test_statistics.py::test_mean_single_element PASSED
test_statistics.py::test_mean_with_negatives PASSED

#@pytest.mark.parametrize: Kill the Duplication

This is the single most useful pytest feature for ML code. Instead of writing 10 near-identical test functions, you write one parametrized test.

Before: 5 separate test functions

pythonreference · read-only

# Repetitive — the pattern is the same, only inputs and expected change
def test_celsius_to_fahrenheit_freezing():
    assert celsius_to_fahrenheit(0) == 32

def test_celsius_to_fahrenheit_boiling():
    assert celsius_to_fahrenheit(100) == 212

def test_celsius_to_fahrenheit_body_temp():
    assert celsius_to_fahrenheit(37) == pytest.approx(98.6, abs=0.1)

def test_celsius_to_fahrenheit_freezer():
    assert celsius_to_fahrenheit(-18) == pytest.approx(0, abs=0.5)

def test_celsius_to_fahrenheit_negative_crossover():
    assert celsius_to_fahrenheit(-40) == -40

After: 1 parametrized test

pythonreference · read-only

import pytest

@pytest.mark.parametrize("celsius,expected_f", [
    (0,    32),
    (100,  212),
    (37,   pytest.approx(98.6, abs=0.1)),
    (-18,  pytest.approx(0,    abs=0.5)),
    (-40,  -40),
])
def test_celsius_to_fahrenheit(celsius, expected_f):
    assert celsius_to_fahrenheit(celsius) == expected_f

pytest generates a separate test ID for each case:

test_temp.py::test_celsius_to_fahrenheit[0-32]       PASSED
test_temp.py::test_celsius_to_fahrenheit[100-212]    PASSED
test_temp.py::test_celsius_to_fahrenheit[37-...]     PASSED
test_temp.py::test_celsius_to_fahrenheit[-18-...]    PASSED
test_temp.py::test_celsius_to_fahrenheit[-40--40]    PASSED

If case 3 fails, you see exactly which input caused it — not just "the parametrized test failed."

#Multiple Parameters, Named IDs

pythonreference · read-only

@pytest.mark.parametrize("a,b,expected", [
    (2, 3, 5),
    (0, 0, 0),
    (-1, 1, 0),
    (100, -50, 50),
], ids=["positive", "zeros", "cancel", "mixed"])
def test_add(a, b, expected):
    assert add(a, b) == expected

Named IDs make the output more readable:

test_math.py::test_add[positive]  PASSED
test_math.py::test_add[zeros]     PASSED

#Indirect Parametrize (Advanced)

pythonreference · read-only

@pytest.mark.parametrize("model_name", ["linear", "tree", "forest"])
def test_model_fits(model_name, sample_dataset):
    model = build_model(model_name)
    model.fit(sample_dataset["X"], sample_dataset["y"])
    assert model.score(sample_dataset["X"], sample_dataset["y"]) > 0.5

#Fixtures: Reusable Setup and Teardown

Fixtures provide test dependencies — database connections, sample data, temporary files — in a clean, composable way. They replace setUp/tearDown from unittest.

pythonreference · read-only

import pytest
import tempfile
import os

# ── Simple fixture: shared sample data ──────────────────────────────────────
@pytest.fixture
def sample_scores():
    """Provide a standard list of test scores for multiple tests."""
    return [72, 85, 91, 68, 79, 88, 95, 62, 74, 83]

# ── Fixture with teardown using yield ───────────────────────────────────────
@pytest.fixture
def temp_csv_file():
    """Create a temporary CSV file, yield its path, then clean up."""
    with tempfile.NamedTemporaryFile(mode='w', suffix='.csv', delete=False) as f:
        f.write("name,score\nAlice,92\nBob,78\nCarol,85\n")
        filepath = f.name
    yield filepath            # ← test runs here with filepath available
    os.unlink(filepath)       # ← teardown: runs after the test finishes

# ── Using fixtures in tests ──────────────────────────────────────────────────
def test_mean_calculation(sample_scores):
    result = mean(sample_scores)
    assert result == pytest.approx(79.7, abs=0.1)

def test_median_calculation(sample_scores):
    result = median(sample_scores)
    assert result == pytest.approx(81.0, abs=0.1)

def test_csv_loading(temp_csv_file):
    import pandas as pd
    df = pd.read_csv(temp_csv_file)
    assert len(df) == 3
    assert "score" in df.columns

#Fixture Scope: Controlling Reuse

pythonreference · read-only

# function scope (default): fixture runs fresh for every test function
@pytest.fixture
def fresh_database_connection():
    conn = create_connection(":memory:")
    conn.execute("CREATE TABLE users ...")
    yield conn
    conn.close()

# session scope: fixture runs ONCE for the entire test session
# Use for expensive setup: loading models, connecting to external services
@pytest.fixture(scope="session")
def ml_model():
    """Load an ML model once and share it across all tests in the session."""
    print("\nLoading ML model (expensive -- only happens once)...")
    model = load_model("model_weights.pkl")
    yield model
    # No teardown needed for a loaded model

# module scope: fixture runs once per test file
@pytest.fixture(scope="module")
def database_with_seed_data():
    db = create_test_database()
    db.seed(sample_users=100, sample_orders=500)
    yield db
    db.teardown()

#conftest.py: Shared Fixtures

Fixtures in conftest.py are automatically available to all test files in the same directory and below — no import needed:

conftest.py: Shared Fixtures

tests/

conftest.py — shared fixtures for all tests/

test_models.py

test_features.py

integration/

conftest.py — shared fixtures for integration tests only

test_pipeline.py

pythonreference · read-only

# tests/conftest.py
import pytest
import pandas as pd

@pytest.fixture(scope="session")
def sample_dataframe():
    """A standard sample DataFrame available to all test files."""
    return pd.DataFrame({
        "user_id": range(1, 101),
        "feature_a": [float(i) * 1.5 for i in range(100)],
        "feature_b": [i % 3 for i in range(100)],
        "label": [i % 2 for i in range(100)],
    })

#pytest.raises: Testing Exceptions

pythonreference · read-only

import pytest

def divide(a: float, b: float) -> float:
    if b == 0:
        raise ValueError("Cannot divide by zero")
    return a / b

# ── Basic exception check ────────────────────────────────────────────────────
def test_divide_by_zero_raises():
    with pytest.raises(ValueError):
        divide(10, 0)

# ── Check the exception message ──────────────────────────────────────────────
def test_divide_by_zero_message():
    with pytest.raises(ValueError, match="Cannot divide by zero"):
        divide(10, 0)

# ── Check the exception type is specific ────────────────────────────────────
def test_divide_by_zero_is_value_error_not_generic():
    with pytest.raises(ValueError):  # NOT just Exception
        divide(10, 0)

# ── Capture the exception to inspect it ─────────────────────────────────────
def test_divide_by_zero_exception_details():
    with pytest.raises(ValueError) as exc_info:
        divide(10, 0)
    assert "zero" in str(exc_info.value).lower()

# ── Parametrize exception tests ──────────────────────────────────────────────
@pytest.mark.parametrize("value,error_type,error_match", [
    (None,  TypeError,  "must be"),
    (-1,    ValueError, "must be positive"),
    (1001,  ValueError, "must not exceed 1000"),
])
def test_validate_input_errors(value, error_type, error_match):
    with pytest.raises(error_type, match=error_match):
        validate_input(value)

#pytest.approx: Floating-Point Comparisons

This is essential for any numerical or ML code.

pythonreference · read-only

# This FAILS — floating-point arithmetic is not exact
assert 0.1 + 0.2 == 0.3       # AssertionError! (0.1+0.2 = 0.30000000000000004)

# This PASSES — pytest.approx handles the floating-point imprecision
assert 0.1 + 0.2 == pytest.approx(0.3)

# Control the tolerance
assert 3.14159 == pytest.approx(3.14, abs=0.01)      # absolute tolerance
assert 1234.56 == pytest.approx(1234, rel=1e-3)       # relative tolerance (0.1%)

# Works with lists and numpy arrays
import numpy as np
assert np.array([1.0, 2.0, 3.0]) == pytest.approx([1.0, 2.0, 3.0])

# ML model accuracy check
accuracy = model.score(X_test, y_test)
assert accuracy == pytest.approx(0.92, abs=0.02)  # 92% ± 2%

pythonreference · read-only

# Real example: testing a normalization function
def normalize(values: list[float]) -> list[float]:
    """Normalize values to mean=0, std=1."""
    import statistics
    mean = statistics.mean(values)
    std = statistics.stdev(values)
    return [(x - mean) / std for x in values]

def test_normalize_mean_is_zero():
    result = normalize([1.0, 2.0, 3.0, 4.0, 5.0])
    assert sum(result) / len(result) == pytest.approx(0.0, abs=1e-10)

def test_normalize_std_is_one():
    import statistics
    result = normalize([1.0, 2.0, 3.0, 4.0, 5.0])
    assert statistics.stdev(result) == pytest.approx(1.0, rel=1e-6)

#Markers: Organizing and Filtering Tests

pythonreference · read-only

import pytest

# Mark slow tests — skip in fast CI, run manually before release
@pytest.mark.slow
def test_train_full_model():
    model = train_on_full_dataset()      # takes 5 minutes
    assert model.accuracy > 0.90

# Mark integration tests — require external services
@pytest.mark.integration
def test_database_connection():
    conn = connect_to_production_db()
    assert conn.ping() is True

# Mark expected failures (known bug, open ticket)
@pytest.mark.xfail(reason="Bug #1234: edge case in normalization")
def test_normalize_all_same_values():
    result = normalize([5.0, 5.0, 5.0])  # std is 0 — division by zero
    assert result == [0.0, 0.0, 0.0]

# Skip a test entirely
@pytest.mark.skip(reason="GPU not available in CI")
def test_gpu_training():
    ...

toml

[tool.pytest.ini_options]
markers = [
    "slow: marks tests as slow (run with -m slow)",
    "integration: marks tests requiring external services",
]

Run specific groups:

bash

pytest -m "not slow"           # skip slow tests in CI
pytest -m slow                  # run only slow tests
pytest -m "integration and not slow"
pytest -v -k "temperature"      # run tests with "temperature" in their name

#monkeypatch: Patching the World Around Your Code

monkeypatch is a built-in pytest fixture that lets you replace attributes, environment variables, and dictionary values for the duration of a single test. It always restores the original value afterwards, so tests remain isolated.

#Patching Environment Variables

pythonrunnable cell

import os

def get_api_url() -> str:
    """Read the API base URL from the environment."""
    return os.environ.get("API_URL", "https://api.production.example.com")

def test_api_url_from_environment(monkeypatch):
    """Test that the function reads from the environment."""
    monkeypatch.setenv("API_URL", "https://api.staging.example.com")
    assert get_api_url() == "https://api.staging.example.com"
    # After the test, API_URL is restored to its original value

def test_api_url_default(monkeypatch):
    """Test the default value when the environment variable is not set."""
    monkeypatch.delenv("API_URL", raising=False)  # remove it if it exists
    assert get_api_url() == "https://api.production.example.com"

#Patching Functions and Methods

pythonreference · read-only

import time

def measure_latency(fn, *args) -> float:
    """Measure how long fn(*args) takes in seconds."""
    start = time.time()
    fn(*args)
    return time.time() - start

def test_measure_latency(monkeypatch):
    """Test measure_latency without actually sleeping."""
    call_count = {"n": 0}

    # Replace time.time with a controlled version
    fake_times = iter([100.0, 100.5])  # start=100.0, end=100.5

    def fake_time():
        return next(fake_times)

    monkeypatch.setattr(time, "time", fake_time)

    latency = measure_latency(lambda: None)
    assert latency == pytest.approx(0.5, abs=1e-9)

#Patching a Class Method

pythonrunnable cell

# database.py
class UserRepository:
    def get_user(self, user_id: int) -> dict:
        """Fetch a user from the real database."""
        # ... real DB call here ...
        raise NotImplementedError("Would hit the real DB")

# Code under test
def get_user_display_name(repo: UserRepository, user_id: int) -> str:
    user = repo.get_user(user_id)
    return f"{user['first_name']} {user['last_name']}"

# test_service.py
def test_get_user_display_name(monkeypatch):
    """Test display name formatting without hitting the database."""
    def fake_get_user(self, user_id):
        return {"first_name": "Alice", "last_name": "Smith", "id": user_id}

    monkeypatch.setattr(UserRepository, "get_user", fake_get_user)

    repo = UserRepository()
    result = get_user_display_name(repo, 42)
    assert result == "Alice Smith"

Intuition

Build Your Intuition

monkeypatch is a time machine for dependencies. Your unit under test calls os.environ["API_KEY"], time.time(), or requests.get(). You cannot control the real environment, real clock, or real internet in a unit test. monkeypatch lets you freeze time, inject values, and replace external calls with controlled doubles — all without modifying a single line of production code.

The key difference from production mocking libraries (like unittest.mock): monkeypatch is procedural and pytest-native. It does not create mock objects with call-count tracking. For simple attribute replacement and environment patching, monkeypatch is the cleanest tool. For spy-style assertions ("was this function called with these arguments?"), reach for unittest.mock.patch or pytest-mock's mocker.patch.

#Coverage with pytest-cov

Coverage tells you which lines of your source code are executed by your tests. It does not tell you whether your tests are good — but it tells you which code is NOT tested at all.

bash

# Install
pip install pytest-cov

# Run tests with coverage
pytest --cov=src --cov-report=term-missing

# Generate HTML report (open htmlcov/index.html)
pytest --cov=src --cov-report=html

# Fail if coverage drops below 80%
pytest --cov=src --cov-fail-under=80

Example output:

Name                     Stmts   Miss  Cover   Missing
------------------------------------------------------
src/statistics.py           24      3    88%    45, 67-68
src/models/linear.py        48      8    83%    102-109
src/pipeline.py             91     21    77%    67-87
------------------------------------------------------
TOTAL                      163     32    80%

Interpreting coverage

Line 45 missed. The code at line 45 was never executed. It might be an error handler or edge case that has no test.
Lines 67-68 missed. A two-line block never ran. Often an else branch or a raise statement.
80% is the common target for production services. 100% is unrealistic for ML code (some paths require specific hardware/data).

Watch Out

Common Mistake

High coverage does not mean good tests. You can get 100% coverage with tests that never check return values:

pythonreference · read-only

def test_nothing():
    mean([1, 2, 3])  # executes the code, asserts nothing

Coverage is a floor (untested code is definitely a risk), not a ceiling (covered code is not necessarily correct). Use coverage to find blind spots, not to grade test quality.

#Putting It Together: The Full Code Playground

pythonplayground.py · Pyodide

import pytest
import math
from typing import Union

# ── Module Under Test: A simple statistics library ─────────────────────────
def mean(values: list[float]) -> float:
  """Calculate the arithmetic mean."""
  if not values:
      raise ValueError("Cannot compute mean of empty list")
  return sum(values) / len(values)

def median(values: list[float]) -> float:
  """Calculate the median value."""
  if not values:
      raise ValueError("Cannot compute median of empty list")
  sorted_vals = sorted(values)
  n = len(sorted_vals)
  mid = n // 2
  if n % 2 == 0:
      return (sorted_vals[mid - 1] + sorted_vals[mid]) / 2
  return sorted_vals[mid]

def std_dev(values: list[float]) -> float:
  """Calculate the population standard deviation."""
  if len(values) &lt; 2:
      raise ValueError("Standard deviation requires at least 2 values")
  m = mean(values)
  variance = sum((x - m) ** 2 for x in values) / len(values)
  return math.sqrt(variance)

def normalize(values: list[float]) -> list[float]:
  """Z-score normalize: mean=0, std=1."""
  m = mean(values)
  s = std_dev(values)
  return [(x - m) / s for x in values]

# ── TASK: Convert these 5 raw assert tests into proper pytest ───────────────
# Problems with the current tests:
# 1. They all use raw assert with no parametrize
# 2. No fixture for shared test data
# 3. No exception testing with pytest.raises
# 4. Floating-point comparisons need pytest.approx

# CURRENT (bad) tests:
def test_v1_mean():
  assert mean([1, 2, 3]) == 2.0
  assert mean([10, 20, 30]) == 20.0
  assert mean([0, 0, 0]) == 0.0
  assert mean([-1, 0, 1]) == 0.0
  assert mean([5]) == 5.0

def test_v1_std_dev():
  assert round(std_dev([2, 4, 4, 4, 5, 5, 7, 9]), 2) == 2.0
  assert round(std_dev([1, 2, 3]), 5) == round(math.sqrt(2/3), 5)

# YOUR TASK: Rewrite these below using proper pytest patterns
# ────────────────────────────────────────────────────────────────────────────

# 1. Create a fixture with sample data shared across tests
@pytest.fixture
def scores():
  # TODO: return [2, 4, 4, 4, 5, 5, 7, 9]
  pass

# 2. Parametrize the mean tests (5 cases → 1 parametrized test)
@pytest.mark.parametrize("values,expected", [
  # TODO: fill in the 5 cases from test_v1_mean
])
def test_mean_parametrized(values, expected):
  # TODO: assert using pytest.approx where appropriate
  pass

# 3. Test that mean() raises ValueError on empty list
def test_mean_empty_raises():
  # TODO: use pytest.raises
  pass

# 4. Test std_dev using the fixture and pytest.approx
def test_std_dev_classic(scores):
  # TODO: assert std_dev(scores) == pytest.approx(2.0)
  pass

# 5. Test that normalize() produces mean≈0 and std≈1
def test_normalize_properties(scores):
  result = normalize(scores)
  # TODO: assert mean(result) ≈ 0 and std_dev(result) ≈ 1 with pytest.approx
  pass

Tests · Run pytest -v to see each parametrized case with its ID. Verify test_mean_empty_raises catches the ValueError. Verify test_normalize_properties passes with pytest.approx tolerance.

#Quick Check

Quick Check1 / 4

You have 7 test functions, each calling the same function with a different input. What is the pytest way to clean this up?

Interactive Lab

Watch pytest discover, collect, and run tests — see fixture scope and parametrize in action

pytest Visualizer

pytest is the foundation of quality Python engineering. The patterns here — parametrize, fixtures, raises, approx — appear in every production Python codebase. Next: Python's Data Model — how dunder methods power the ==, in, len(), and for syntax you use in every test.

Your Reflection

Continue Learning

Discussion

Testing with pytest: Beyond Basic assert

After this lesson, you will be able to:

Before You Start

#Why pytest Over unittest

#Basic pytest

#File and Function Naming

#@pytest.mark.parametrize: Kill the Duplication

#Multiple Parameters, Named IDs

#Indirect Parametrize (Advanced)

#Fixtures: Reusable Setup and Teardown

#Fixture Scope: Controlling Reuse

#conftest.py: Shared Fixtures

#pytest.raises: Testing Exceptions

#pytest.approx: Floating-Point Comparisons

#Markers: Organizing and Filtering Tests

#monkeypatch: Patching the World Around Your Code

#Patching Environment Variables

#Patching Functions and Methods

#Patching a Class Method

#Coverage with pytest-cov

Interpreting coverage

#Putting It Together: The Full Code Playground

#Quick Check

Deep Dive

Deep Dive

Related Lessons