RugvAI Labs

Production Python: Logging, Config, CLI & Best Practices

Production code is 10% feature logic, 90% logging, retries, config, and graceful shutdown. Stripe's Python workers log structured JSON that flows into Splunk and Datadog. Typer (built on Click) is how every modern Python CLI — including pip and ruff — gets built. This lesson is the "ops" layer that turns prototypes into services.

Learning Objectives

After this lesson, you will be able to:

Use Python's logging module with levels, handlers, formatters, and structured JSON output
Manage application configuration with .env files, pydantic-settings, and the 12-factor app methodology
Build command-line tools using argparse, click, and typer for ML workflows
Structure a Python project with the src layout, pyproject.toml, and pre-commit hooks
Package and distribute a Python library using setuptools and wheels
Distinguish production code from scripts and apply best practices for reliability

Why

Why This Matters

Scripts that work on your laptop break in production -- they crash silently, expose secrets in logs, ignore errors, and cannot be configured without editing source code. Production Python is the difference between a prototype that impresses in a demo and a system that runs reliably for months
Every ML team needs tooling -- training scripts, data pipelines, evaluation harnesses, deployment tools. These are all CLI applications. Knowing argparse/click/typer means you can build tools your entire team uses, not just Jupyter notebooks that only you can run
Logging is how you debug production systems -- you cannot attach a debugger to a server running in the cloud. Structured logging with proper levels is how you find out what went wrong at 3 AM when a model serving endpoint starts returning bad predictions

Prerequisites

Before You Start

Review

#Why Production Code Differs from Scripts

A script is something you run once. Production code is something that runs for months, is maintained by a team, and must be reliable at 3 AM when nobody is watching.

Aspect	Script	Production Code
Error handling	`print("Error!")` and crash	Structured logging, graceful degradation, retry logic
Configuration	Hardcoded paths and values	Environment variables, config files, secrets management
Output	`print()` everywhere	Logging with levels (DEBUG, INFO, WARNING, ERROR)
Dependencies	`pip install` globally	Virtual environments, pinned versions, lockfiles
Testing	"It worked when I ran it"	Automated tests, CI/CD, coverage checks

#Logging -- The Right Way to Track What Your Code Does

#Why Not print()?

pythonreference · read-only

# This is what most beginners write
print("Loading model...")
print(f"Model loaded, accuracy: {accuracy}")
print("ERROR: prediction failed!")
print("DEBUG: input features:", features)

Problems with print():

No severity levels -- you cannot filter warnings from debug output
No timestamps -- when did the error happen?
No destination control -- everything goes to stdout. You cannot send errors to a file and info to console
No structured data -- log aggregation tools (Datadog, ELK, CloudWatch) cannot parse print output
Cannot disable in production -- you have to delete or comment out print statements

#Python's logging Module

pythonreference · read-only

import logging

# Create a logger for this module
logger = logging.getLogger(__name__)

# Set the minimum level to display
logging.basicConfig(level=logging.INFO)

# Use different levels for different importance
logger.debug("Feature vector: %s", features)      # development only
logger.info("Model loaded successfully")            # normal operation
logger.warning("GPU not available, using CPU")      # something unexpected
logger.error("Prediction failed: %s", str(e))       # something broke
logger.critical("Database connection lost")          # system is down

#Logging Levels Hierarchy

DEBUG    (10) -- Detailed diagnostic information. Only in development.
INFO     (20) -- Confirmation that things are working as expected.
WARNING  (30) -- Something unexpected but not broken. Default level.
ERROR    (40) -- Something failed. The operation could not complete.
CRITICAL (50) -- The system itself is failing. Immediate attention needed.

If you set the level to WARNING, only WARNING, ERROR, and CRITICAL messages appear. DEBUG and INFO are silently discarded. This is how you control verbosity without editing code.

#Formatters and Handlers

A handler determines where log messages go (console, file, network). A formatter determines what the message looks like.

pythonrunnable cell

import logging
import sys

def setup_logging(level: str = "INFO") -> logging.Logger:
    """Configure logging with console and file handlers."""
    logger = logging.getLogger("ml_service")
    logger.setLevel(getattr(logging, level.upper()))

    # Console handler -- human-readable format
    console_handler = logging.StreamHandler(sys.stdout)
    console_handler.setLevel(logging.INFO)
    console_fmt = logging.Formatter(
        "%(asctime)s | %(levelname)-8s | %(name)s | %(message)s",
        datefmt="%Y-%m-%d %H:%M:%S",
    )
    console_handler.setFormatter(console_fmt)

    # File handler -- captures everything including DEBUG
    file_handler = logging.FileHandler("app.log")
    file_handler.setLevel(logging.DEBUG)
    file_fmt = logging.Formatter(
        "%(asctime)s | %(levelname)-8s | %(name)s:%(lineno)d | %(message)s",
    )
    file_handler.setFormatter(file_fmt)

    logger.addHandler(console_handler)
    logger.addHandler(file_handler)

    return logger

# Usage
logger = setup_logging("DEBUG")
logger.info("Service started")
logger.debug("Config: %s", {"model": "v2", "batch_size": 32})
logger.error("Failed to load model: FileNotFoundError")

Console output:

2026-04-14 10:30:15 | INFO     | ml_service | Service started
2026-04-14 10:30:15 | ERROR    | ml_service | Failed to load model: FileNotFoundError

File output (includes DEBUG):

2026-04-14 10:30:15 | INFO     | ml_service:28 | Service started
2026-04-14 10:30:15 | DEBUG    | ml_service:29 | Config: {'model': 'v2', 'batch_size': 32}
2026-04-14 10:30:15 | ERROR    | ml_service:30 | Failed to load model: FileNotFoundError

#Structured Logging (JSON)

For production systems that feed into log aggregation tools, JSON logging is essential:

pythonrunnable cell

import logging
import json
from datetime import datetime, timezone

class JSONFormatter(logging.Formatter):
    """Format log records as JSON for machine parsing."""
    
    def format(self, record: logging.LogRecord) -> str:
        log_data = {
            "timestamp": datetime.now(timezone.utc).isoformat(),
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
            "module": record.module,
            "line": record.lineno,
        }
        # Include exception info if present
        if record.exc_info and record.exc_info[0]:
            log_data["exception"] = self.formatException(record.exc_info)
        return json.dumps(log_data)

# Apply the JSON formatter
handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())
logger = logging.getLogger("ml_service")
logger.addHandler(handler)
logger.setLevel(logging.INFO)

logger.info("Prediction completed", extra={"model": "v2", "latency_ms": 42})

Output:

json

{"timestamp": "2026-04-14T17:30:15.123456+00:00", "level": "INFO", "logger": "ml_service", "message": "Prediction completed", "module": "main", "line": 12}

What Do You Think?

You set logging.basicConfig(level=logging.WARNING). Then you call logger.info('Model loaded'). What happens?

The answer is nothing happens. The logging level acts as a filter. Since INFO (level 20) is below the configured threshold of WARNING (level 30), the message is discarded. Only WARNING, ERROR, and CRITICAL messages pass through. This is the core mechanism for controlling verbosity -- change one number and all debug output disappears.

#Configuration -- Never Hardcode Anything

#The Problem with Hardcoded Values

pythonrunnable cell

# DO NOT DO THIS
DATABASE_URL = "postgresql://admin:secretpassword@prod-db.example.com/ml_data"
API_KEY = "sk-abc123xyz789"
MODEL_PATH = "/home/alice/models/classifier_v3.pkl"
BATCH_SIZE = 32

Problems:

Secrets in source code -- anyone with repo access sees the password
Environment-specific -- Alice's path does not exist on the production server
Cannot change without redeployment -- tuning batch size requires a code change, review, and deploy

#Environment Variables and .env Files

The 12-factor app methodology says: store config in the environment, not in code.

bash

# .env file (NEVER commit this to git)
DATABASE_URL=postgresql://admin:secretpassword@prod-db.example.com/ml_data
API_KEY=sk-abc123xyz789
MODEL_PATH=/opt/models/classifier_v3.pkl
BATCH_SIZE=32
LOG_LEVEL=INFO
DEBUG=false

pythonreference · read-only

# Load .env with python-dotenv
from dotenv import load_dotenv
import os

load_dotenv()  # reads .env file into os.environ

database_url = os.getenv("DATABASE_URL")
api_key = os.getenv("API_KEY")
batch_size = int(os.getenv("BATCH_SIZE", "32"))  # default to 32
debug = os.getenv("DEBUG", "false").lower() == "true"

#Pydantic Settings -- Type-Safe Configuration

pydantic-settings gives you validated, typed configuration with automatic .env loading:

pythonreference · read-only

# config.py
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import Field

class Settings(BaseSettings):
    """Application settings loaded from environment variables."""
    
    # Required (no default = must be set)
    database_url: str
    api_key: str
    
    # Optional with defaults
    model_path: str = "/opt/models/classifier.pkl"
    batch_size: int = Field(default=32, ge=1, le=1024)
    log_level: str = "INFO"
    debug: bool = False
    
    # Nested prefix: ML_MODEL_NAME, ML_VERSION, etc.
    model_name: str = "classifier"
    model_version: str = "1.0.0"
    
    # Pydantic v2 style. The old `class Config:` form still works but
    # raises PydanticDeprecatedSince20 and goes away in v3.
    model_config = SettingsConfigDict(
        env_file=".env",
        env_file_encoding="utf-8",
        case_sensitive=False,  # DATABASE_URL and database_url both work
    )

# Usage
settings = Settings()
print(settings.database_url)  # from .env or environment
print(settings.batch_size)    # validated as int, 1-1024

#The 12-Factor App Principles (Key Ones)

Factor	Principle	Python Implementation
III. Config	Store config in environment	`pydantic-settings`, `.env` files
IV. Backing services	Treat databases, queues as attached resources	Connection URLs in env vars
V. Build, release, run	Strictly separate build and run stages	Docker build vs docker run
XI. Logs	Treat logs as event streams	Log to stdout, let the platform collect
XII. Admin processes	Run admin tasks as one-off processes

#Secrets Management

pythonreference · read-only

# .gitignore -- CRITICAL: never commit secrets
.env
.env.local
.env.production
*.pem
credentials.json

For production, use a secrets manager instead of .env files:

AWS Secrets Manager -- boto3.client("secretsmanager").get_secret_value()
HashiCorp Vault -- hvac.Client().secrets.kv.v2.read_secret_version()
Docker/Kubernetes Secrets -- mounted as files or environment variables

#CLI Tools -- Building Command-Line Applications

#Why CLI Tools Matter for ML

ML workflows are inherently command-line driven:

bash

python train.py --model resnet50 --epochs 100 --lr 0.001 --data ./data/train
python evaluate.py --model-path ./checkpoints/best.pt --test-data ./data/test
python predict.py --input image.jpg --model-path ./models/classifier.pkl
python export.py --model-path ./models/best.pt --format onnx --output ./exported/

#argparse -- Built-In, No Dependencies

pythonrunnable cell

# train.py
import argparse

def main():
    parser = argparse.ArgumentParser(
        description="Train an ML model",
    )
    parser.add_argument(
        "--model", type=str, required=True,
        choices=["linear", "tree", "forest", "neural"],
        help="Model architecture to train",
    )
    parser.add_argument(
        "--epochs", type=int, default=10,
        help="Number of training epochs (default: 10)",
    )
    parser.add_argument(
        "--lr", "--learning-rate", type=float, default=0.001,
        help="Learning rate (default: 0.001)",
    )
    parser.add_argument(
        "--data", type=str, required=True,
        help="Path to training data directory",
    )
    parser.add_argument(
        "--verbose", "-v", action="store_true",
        help="Enable verbose output",
    )
    
    args = parser.parse_args()
    
    print(f"Training {args.model} for {args.epochs} epochs")
    print(f"Learning rate: {args.lr}")
    print(f"Data: {args.data}")
    if args.verbose:
        print("Verbose mode enabled")

if __name__ == "__main__":
    main()

bash

$ python train.py --model neural --epochs 50 --lr 0.01 --data ./data -v
Training neural for 50 epochs
Learning rate: 0.01
Data: ./data
Verbose mode enabled

$ python train.py --help
usage: train.py [-h] --model {linear,tree,forest,neural} [--epochs EPOCHS]
                [--lr LR] --data DATA [--verbose]

Train an ML model
...

#click -- Decorators Make It Cleaner

pythonreference · read-only

# train.py
import click

@click.command()
@click.option("--model", type=click.Choice(["linear", "tree", "forest", "neural"]),
              required=True, help="Model architecture")
@click.option("--epochs", default=10, show_default=True, help="Training epochs")
@click.option("--lr", default=0.001, show_default=True, help="Learning rate")
@click.argument("data_path", type=click.Path(exists=True))
@click.option("-v", "--verbose", is_flag=True, help="Verbose output")
def train(model: str, epochs: int, lr: float, data_path: str, verbose: bool):
    """Train an ML model on the given dataset."""
    click.echo(f"Training {model} for {epochs} epochs (lr={lr})")
    click.echo(f"Data: {data_path}")
    
    if verbose:
        click.echo("Verbose mode enabled")
    
    # Simulate training with a progress bar
    import time
    with click.progressbar(range(epochs), label="Training") as bar:
        for epoch in bar:
            time.sleep(0.1)  # simulate work
    
    click.secho("Training complete!", fg="green", bold=True)

if __name__ == "__main__":
    train()

#typer -- Modern, Type-Hint-Based (Recommended)

Typer is built on click but uses type hints instead of decorators -- the same philosophy as FastAPI:

pythonreference · read-only

# train.py
import typer
from pathlib import Path
from enum import Enum

app = typer.Typer(help="ML Training CLI")

class ModelType(str, Enum):
    linear = "linear"
    tree = "tree"
    forest = "forest"
    neural = "neural"

@app.command()
def train(
    model: ModelType = typer.Option(..., help="Model architecture"),
    epochs: int = typer.Option(10, help="Number of training epochs"),
    lr: float = typer.Option(0.001, help="Learning rate"),
    data_path: Path = typer.Argument(..., exists=True, help="Training data directory"),
    verbose: bool = typer.Option(False, "--verbose", "-v", help="Verbose output"),
):
    """Train an ML model on the given dataset."""
    typer.echo(f"Training {model.value} for {epochs} epochs (lr={lr})")
    typer.echo(f"Data: {data_path}")

@app.command()
def evaluate(
    model_path: Path = typer.Argument(..., exists=True, help="Path to trained model"),
    test_data: Path = typer.Argument(..., exists=True, help="Path to test data"),
):
    """Evaluate a trained model on test data."""
    typer.echo(f"Evaluating {model_path} on {test_data}")

@app.command()
def export(
    model_path: Path = typer.Argument(..., exists=True),
    output_format: str = typer.Option("onnx", help="Export format"),
):
    """Export a model to a portable format."""
    typer.echo(f"Exporting {model_path} to {output_format}")

if __name__ == "__main__":
    app()

bash

$ python train.py --help
Usage: train.py [OPTIONS] COMMAND [ARGS]...

ML Training CLI

Commands:
  train     Train an ML model on the given dataset.
  evaluate  Evaluate a trained model on test data.
  export    Export a model to a portable format.

$ python train.py train --model neural --epochs 50 ./data
Training neural for 50 epochs (lr=0.001)
Data: ./data

#Comparison

Feature	argparse	click	typer
Built-in	Yes	No (pip install)	No (pip install)
Syntax	Procedural	Decorators	Type hints
Progress bars	No	Yes	Yes
Colored output	No	Yes	Yes
Subcommands	Manual	@group	Automatic
Auto-completion	No	Yes

#Project Structure -- The src Layout

#The Recommended Python Project Structure

Professional Python Project Structure

my-ml-project/

src/

my_ml_project/

__init__.py

config.py — pydantic-settings

cli.py — typer CLI

models/ — ML model classes

data/ — data loading & processing

api/ — FastAPI endpoints

utils/ — shared utilities

tests/

conftest.py

test_models.py

test_api.py

test_cli.py

pyproject.toml — project metadata + tool config

.env — local environment variables (NOT committed)

.env.example — template showing required vars (committed)

.gitignore

.pre-commit-config.yaml

README.md

Dockerfile

#Why the src/ Layout?

The src/ layout prevents a subtle and common bug: accidentally importing your project from the working directory instead of the installed package. Without src/, running python from the project root imports from the local directory, which might differ from what is installed. With src/, you must install the package (pip install -e .) to import it, guaranteeing consistency.

#pyproject.toml -- The Single Source of Truth

toml

[project]
name = "my-ml-project"
version = "1.0.0"
description = "An ML prediction service"
requires-python = ">=3.11"
dependencies = [
    "fastapi>=0.110",
    "uvicorn>=0.29",
    "pydantic>=2.0",
    "pydantic-settings>=2.0",
    "scikit-learn>=1.4",
    "numpy>=1.26",
    "typer>=0.12",
]

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "pytest-cov>=5.0",
    "httpx>=0.27",
    "ruff>=0.4",
    "mypy>=1.10",
    "pre-commit>=3.7",
]

[project.scripts]
ml-train = "my_ml_project.cli:app"

[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-v --cov=src --cov-report=term-missing"

[tool.ruff]
line-length = 100
target-version = "py311"

[tool.ruff.lint]
select = ["E", "F", "W", "I", "UP", "B", "SIM"]

[tool.mypy]
python_version = "3.11"
strict = true

#Pre-commit Hooks

Pre-commit hooks run automatically before every git commit, catching issues before they enter the codebase:

yaml

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.4.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.6.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-added-large-files
        args: ['--maxkb=500']
      - id: detect-private-key

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.10.0
    hooks:
      - id: mypy
        additional_dependencies: [pydantic>=2.0]

Install and run:

bash

pip install pre-commit
pre-commit install          # installs hooks into .git/hooks/
pre-commit run --all-files  # run on all files (first time)
# Now hooks run automatically on every git commit

#Code Formatting: Ruff

Ruff replaces black, isort, flake8, and pyflakes in a single tool that runs 10-100x faster:

bash

pip install ruff

# Lint (find issues)
ruff check src/ tests/

# Lint and auto-fix
ruff check --fix src/ tests/

# Format (like black)
ruff format src/ tests/

Watch Out

Watch out

Fix: Replace every print() with the appropriate logging level: logger.debug() for development diagnostics, logger.info() for normal operation, logger.warning() for unexpected situations, logger.error() for failures. Configure the logging level per environment (DEBUG in development, INFO or WARNING in production).

Using print() for debugging in production code instead of logging

print() goes to stdout with no timestamp, no severity, no source location, and no way to filter or redirect. In production, print output is often lost entirely because nobody is watching stdout. Logging sends messages to configurable destinations (files, monitoring services, log aggregation platforms) with structured metadata that makes searching and alerting possible. The effort to switch from print to logging is small; the operational benefit is enormous.

Watch Out

Watch out

Fix: Pass variables as positional arguments to the logger: logger.error('Failed for user %s', user_id). This delays string formatting until the message is actually emitted.

Using f-strings or % formatting to embed variable values inside logger calls: logger.error(f'Failed for user {user_id}')

When you write `logger.error(f"Failed for user {user_id}")`, Python evaluates the f-string immediately — even if the log level is set to CRITICAL and the message will be discarded. For a tight loop that logs thousands of times, this means thousands of unnecessary string allocations.

With `logger.error("Failed for user %s", user_id)`, Python passes the arguments to the logging system, which only calls `% formatting` if the message will actually be emitted. At WARNING level, debug and info messages skip formatting entirely.

```python

SLOW: always formats the string, even if DEBUG is filtered out

logger.debug(f"Processing batch {batch_id} with {len(records)} records")

FAST: defers formatting; skips it if level > DEBUG

logger.debug("Processing batch %s with %d records", batch_id, len(records)) ```

For structured JSON logging (where you want key-value pairs, not formatted strings), use the `extra` parameter instead: `logger.info("Prediction completed", extra={"user_id": user_id, "latency_ms": latency})`. Log aggregation tools like Datadog and ELK can then query on individual fields.

#Packaging and Distribution

#Building a Package

Once your project has a proper pyproject.toml, you can build distributable packages:

bash

# Install build tools
pip install build

# Build the package (creates dist/ directory)
python -m build

This creates two files in dist/:

my_ml_project-1.0.0.tar.gz -- source distribution (sdist)
my_ml_project-1.0.0-py3-none-any.whl -- wheel (binary distribution)

#What Is a Wheel?

A wheel (.whl) is a pre-built package format that installs without needing to compile anything. It is faster than installing from source. Modern Python packaging strongly favors wheels.

bash

# Install from a local wheel
pip install dist/my_ml_project-1.0.0-py3-none-any.whl

# Install in editable mode for development
pip install -e ".[dev]"

#Publishing to PyPI

bash

# Install twine (the upload tool)
pip install twine

# Upload to TestPyPI first (for testing)
twine upload --repository testpypi dist/*

# Upload to real PyPI
twine upload dist/*

# Now anyone in the world can install your package
# pip install my-ml-project

#Entry Points -- Installable CLI Commands

The [project.scripts] section in pyproject.toml creates CLI commands that are available after installation:

toml

[project.scripts]
ml-train = "my_ml_project.cli:app"
ml-predict = "my_ml_project.predict_cli:app"

After pip install my-ml-project, users can run:

bash

ml-train --model neural --epochs 50 ./data
ml-predict --input image.jpg --model-path ./models/best.pkl

No python prefix needed. The commands are installed as executables in the virtual environment's bin/ directory.

#Version Management

pythonreference · read-only

# src/my_ml_project/__init__.py
__version__ = "1.0.0"

# Access programmatically
import my_ml_project
print(my_ml_project.__version__)

For automatic versioning from git tags, use setuptools-scm:

toml

# pyproject.toml
[tool.setuptools_scm]

bash

git tag v1.0.0
python -m build  # version is automatically 1.0.0

#Code Playground

pythonplayground.py · Pyodide

# Production Python Patterns
# Logging, Config, and CLI simulation

import logging
import json
import os
from datetime import datetime, timezone

# ── Section A: Proper Logging Setup ─────────────────────────────────────────
def setup_logging(level: str = "INFO", json_format: bool = False) -> logging.Logger:
  """Configure a production-ready logger."""
  logger = logging.getLogger("ml_service")
  logger.setLevel(getattr(logging, level.upper()))
  
  # Remove existing handlers to avoid duplicates
  logger.handlers.clear()
  
  handler = logging.StreamHandler()
  
  if json_format:
      # JSON formatter for production
      class JSONFmt(logging.Formatter):
          def format(self, record):
              return json.dumps({
                  "ts": datetime.now(timezone.utc).isoformat(),
                  "level": record.levelname,
                  "msg": record.getMessage(),
                  "module": record.module,
              })
      handler.setFormatter(JSONFmt())
  else:
      # Human-readable for development
      handler.setFormatter(logging.Formatter(
          "%(asctime)s | %(levelname)-8s | %(message)s",
          datefmt="%H:%M:%S",
      ))
  
  logger.addHandler(handler)
  return logger

# ── Section B: Type-Safe Configuration (simulating pydantic-settings) ───────
class AppConfig:
  """Simulated pydantic-settings configuration."""
  
  def __init__(self):
      self.model_path = os.getenv("MODEL_PATH", "/opt/models/default.pkl")
      self.batch_size = int(os.getenv("BATCH_SIZE", "32"))
      self.log_level = os.getenv("LOG_LEVEL", "INFO")
      self.debug = os.getenv("DEBUG", "false").lower() == "true"
      self.api_key = os.getenv("API_KEY", "dev-key-not-for-production")
      
      # Validation
      if self.batch_size &lt; 1 or self.batch_size > 1024:
          raise ValueError(f"BATCH_SIZE must be 1-1024, got {self.batch_size}")
  
  def __repr__(self):
      return (
          f"AppConfig(model_path={self.model_path!r}, "
          f"batch_size={self.batch_size}, "
          f"log_level={self.log_level!r}, "
          f"debug={self.debug})"
      )
  
  def safe_dict(self) -> dict:
      """Return config as dict with secrets masked."""
      return {
          "model_path": self.model_path,
          "batch_size": self.batch_size,
          "log_level": self.log_level,
          "debug": self.debug,
          "api_key": self.api_key[:4] + "****",  # mask the secret
      }

# ── Section C: CLI Argument Parsing (simulating argparse/typer) ─────────────
def parse_train_args(args: list[str]) -> dict:
  """Parse training arguments (simplified argparse simulation)."""
  import argparse
  parser = argparse.ArgumentParser(description="Train an ML model")
  parser.add_argument("--model", required=True, 
                     choices=["linear", "tree", "neural"])
  parser.add_argument("--epochs", type=int, default=10)
  parser.add_argument("--lr", type=float, default=0.001)
  parser.add_argument("--verbose", "-v", action="store_true")
  
  parsed = parser.parse_args(args)
  return vars(parsed)

# ── Demo: Put it all together ───────────────────────────────────────────────
print("=== Logging Demo ===")
logger = setup_logging("DEBUG", json_format=False)
logger.debug("Debug: initializing components")
logger.info("Info: service starting up")
logger.warning("Warning: GPU not available, falling back to CPU")
logger.error("Error: failed to connect to database")

print("\n=== JSON Logging Demo ===")
json_logger = setup_logging("INFO", json_format=True)
json_logger.info("Service started")
json_logger.error("Prediction failed for batch 42")

print("\n=== Configuration Demo ===")
config = AppConfig()
print(f"Config: {config}")
print(f"Safe dict (secrets masked): {config.safe_dict()}")

print("\n=== CLI Parsing Demo ===")
args = parse_train_args(["--model", "neural", "--epochs", "50", "--lr", "0.01", "-v"])
print(f"Parsed args: {args}")

# ── Logging Level Filter Demo ───────────────────────────────────────────────
print("\n=== Level Filtering Demo ===")
prod_logger = setup_logging("WARNING", json_format=False)
prod_logger.debug("This debug message is FILTERED OUT in production")
prod_logger.info("This info message is FILTERED OUT in production")
prod_logger.warning("This warning IS visible in production")
prod_logger.error("This error IS visible in production")

# CHALLENGES:
# 1. Add a RotatingFileHandler simulation that "rotates" after N messages
# 2. Add a validate() method to AppConfig that checks all required vars are set
# 3. Create a parse_evaluate_args() for: --model-path, --test-data, --output-format
# 4. Build a function that logs execution time of any function (decorator style)

Tests · Configure logging with different levels, parse CLI arguments, and validate configuration!

#Key Takeaways

Replace print() with logging -- use logging.getLogger(__name__) with appropriate levels (DEBUG, INFO, WARNING, ERROR). Configure handlers for console vs file output, and use JSON formatting for production log aggregation
Never hardcode configuration -- use environment variables and .env files for local development, pydantic-settings for type-safe validation, and secrets managers for production credentials. The same code should run in all environments with zero changes
Build CLI tools for ML workflows -- use argparse for zero-dependency scripts, click for decorator-style CLIs, or typer for type-hint-based CLIs (recommended). Entry points in pyproject.toml make your tools installable as system commands
Use the src layout -- src/my_project/ prevents accidental local imports and forces you to install your package properly. pyproject.toml is the single source of truth for project metadata, dependencies, and tool configuration
Pre-commit hooks catch issues before they enter git -- ruff for linting and formatting, mypy for type checking, and detect-private-key for secrets. Install once, runs automatically on every commit
-- creates distributable packages. publishes to PyPI. Entry points create installable CLI commands

Quick Check1 / 5

You set the logging level to WARNING. Which of these messages will appear in the output?

python -m build

twine upload

Your Reflection

Continue Learning

Discussion

Production Python: Logging, Config, CLI & Best Practices

After this lesson, you will be able to:

Before You Start

#Why Production Code Differs from Scripts

#Logging -- The Right Way to Track What Your Code Does

#Why Not print()?

#Python's logging Module

#Logging Levels Hierarchy

#Formatters and Handlers

#Structured Logging (JSON)

#Configuration -- Never Hardcode Anything

#The Problem with Hardcoded Values

#Environment Variables and .env Files

#Pydantic Settings -- Type-Safe Configuration

#The 12-Factor App Principles (Key Ones)

#Secrets Management

#CLI Tools -- Building Command-Line Applications

#Why CLI Tools Matter for ML

#argparse -- Built-In, No Dependencies

#click -- Decorators Make It Cleaner

#typer -- Modern, Type-Hint-Based (Recommended)

#Comparison

#Project Structure -- The src Layout

#The Recommended Python Project Structure

#Why the src/ Layout?

#pyproject.toml -- The Single Source of Truth

#Pre-commit Hooks

#Code Formatting: Ruff

#Packaging and Distribution

#Building a Package

#What Is a Wheel?

#Publishing to PyPI

#Entry Points -- Installable CLI Commands

#Version Management

#Code Playground

The Python Packaging Ecosystem: A Brief History

#Key Takeaways