AWS SageMaker, Azure ML, and Vertex AI each have their own dialects for the same operations — train a model, register it, serve it, monitor it. Pick wrong and you've locked in 3x your bill and months of migration cost. Here's the honest comparison: where each platform genuinely wins, and where they hide their pricing.
Learning Objectives
After this lesson, you will be able to:
Compare the big three cloud AI platforms (AWS SageMaker, Azure ML, GCP Vertex AI) for training, serving, and managing models
Pick the right cloud provider for your specific workload based on cost, features, and what your team already uses
Design a cloud setup for fine-tuning a foundation model using managed services from any of the three providers
Cloud platforms can feel overwhelming with their hundreds of services, but do not worry -- by the end of this lesson, you will know exactly which services matter for ML and how to choose between providers. Most of the decision comes down to practical factors, not feature checklists.
What Do You Think?
You need to fine-tune a 7B parameter model on 100K training examples. Which cloud service would you choose?
The answer is D. All three clouds can handle this workload. The deciding factors are rarely technical -- they are organizational. Do you have an existing AWS account with credits? Is your company on Microsoft Enterprise Agreement? Does your team have GCP experience? Start from your constraints, not from a feature comparison chart.
AWS holds ~32% of the cloud market and has the broadest AI service portfolio. Its strength is sheer breadth: if an AI service exists, AWS probably offers it.
SageMaker is AWS's end-to-end ML platform. Think of it as a managed Jupyter notebook that grew into an entire ML factory.
Core capabilities
Feature
What It Does
When to Use
SageMaker Studio
Managed IDE (JupyterLab) with built-in experiment tracking
Daily ML development
Training Jobs
Managed distributed training on p4d/p5 (A100/H100) instances
Fine-tuning, pre-training
Endpoints
Real-time inference with auto-scaling
Production model serving
Batch Transform
Batch inference on large datasets
Offline scoring, nightly jobs
Pipelines
ML workflow orchestration (DAGs)
Automated retraining
JumpStart
Try it! Go to the free tiers of all three cloud providers (AWS Free Tier, Azure Free Account, GCP Free Trial) and navigate to their AI/ML service pages. Compare how each one organizes its ML tools. You will immediately see the personality differences described above -- AWS's massive list, Azure's enterprise polish, and GCP's developer-friendly simplicity.
Training a model on SageMaker
pythonreference · read-only
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# SageMaker training job — the key abstraction is the Estimator
import sagemaker
from sagemaker.huggingface import HuggingFace
estimator = HuggingFace(
entry_point="train.py", # Your training script
instance_type="ml.p4d.24xlarge", # 8x A100 GPUs
instance_count=1,
transformers_version="4.37",
pytorch_version="2.1",
py_version="py310",
role=sagemaker.get_execution_role(),
hyperparameters={
"model_name": "meta-llama/Llama-2-7b-hf",
"epochs": 3,
"batch_size": 4,
"learning_rate": 2e-5,
},
)
# This launches a managed training job — you pay only while it runs
estimator.fit({"train": "s3://my-bucket/training-data/"})
Key insight: SageMaker abstracts away infrastructure. You write a training script, specify hardware, and point to data in S3. SageMaker provisions instances, runs training, saves the model artifact to S3, and shuts down the instances. No SSH, no Docker management, no idle GPU costs.
Bedrock is AWS's managed LLM service -- think "API gateway to foundation models." You do not train or host anything. You send prompts, get responses, and pay per token.
Available models: Claude (Anthropic), Llama (Meta), Titan (Amazon), Mistral, Stability AI, Cohere.
Why use Bedrock instead of calling APIs directly?
Data stays within your AWS VPC (privacy/compliance)
Azure holds ~23% of the cloud market and has a unique advantage: the deepest integration with OpenAI models. If your company uses Microsoft 365, Teams, or has an Enterprise Agreement, Azure is the path of least resistance.
Azure ML is Microsoft's end-to-end ML platform. Its key differentiator is tight integration with the Microsoft ecosystem (VS Code, GitHub, Active Directory).
Core capabilities
Feature
What It Does
When to Use
Workspace
Central hub for experiments, models, data, compute
All ML projects
Compute Instances
Managed VMs for development (NC, ND series)
Exploration, prototyping
Compute Clusters
Auto-scaling GPU clusters for training
Distributed training
Managed Endpoints
Real-time and batch inference with auto-scaling
Production serving
Pipelines
ML workflow orchestration (similar to SageMaker Pipelines)
Automated retraining
Model Registry
Training a model on Azure ML
pythonreference · read-only
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# Azure ML training job using the v2 SDK
from azure.ai.ml import MLClient, command, Input
from azure.identity import DefaultAzureCredential
ml_client = MLClient(
DefaultAzureCredential(),
subscription_id="your-sub-id",
resource_group_name="my-rg",
workspace_name="my-workspace",
)
# Define the training job
training_job = command(
code="./src", # Local training script directory
command="python train.py "
"--model_name ${{inputs.model_name}} "
"--epochs 3 --batch_size 4 --lr 2e-5",
inputs={
"model_name": "meta-llama/Llama-2-7b-hf",
},
environment="AzureML-pytorch-2.1-cuda12@latest",
compute="gpu-cluster", # NC A100 v4 cluster
instance_count=1,
)
# Submit — Azure ML provisions compute, runs training, saves outputs
returned_job = ml_client.jobs.create_or_update(training_job)
This is Azure's killer feature for AI. You get OpenAI models (GPT-4, GPT-4o, DALL-E, Whisper) hosted within Azure's infrastructure, with enterprise security.
Why use Azure OpenAI instead of OpenAI directly?
Data does not leave your Azure tenant (critical for regulated industries)
Virtual network integration, private endpoints
Content filtering and abuse monitoring built in
SLA guarantees (99.9% uptime)
Pay with existing Azure commitment
The enterprise pitch: "We want to use GPT-4, but our data cannot leave our infrastructure." Azure OpenAI is the answer.
GCP holds ~11% of the cloud market but punches above its weight in AI. Google invented the Transformer, built TensorFlow, and designed custom TPU hardware. If cutting-edge AI research matters to you, GCP often gets new capabilities first.
Vertex AI is Google's unified ML platform. Its differentiator is the deepest integration with Google's AI research -- Gemini models, TPU hardware, and research-grade tools.
Core capabilities
Feature
What It Does
When to Use
Workbench
Managed JupyterLab with GPU/TPU support
ML development
Training
Custom and AutoML training on GPUs or TPUs
Model training
Prediction
Online and batch prediction endpoints
Production serving
Pipelines
Kubeflow-based ML workflow orchestration
Automated ML workflows
Model Garden
Pre-trained model hub (Gemini, Llama, Stable Diffusion)
Foundation model deployment
Feature Store
Centralized feature management
Training a model on Vertex AI
pythonreference · read-only
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# Vertex AI custom training job
from google.cloud import aiplatform
aiplatform.init(project="my-project", location="us-central1")
# Define a custom training job
job = aiplatform.CustomContainerTrainingJob(
display_name="llama-7b-finetune",
container_uri="us-docker.pkg.dev/vertex-ai/training/pytorch-gpu.2-1:latest",
command=["python", "train.py"],
args=[
"--model_name", "meta-llama/Llama-2-7b-hf",
"--epochs", "3",
"--batch_size", "4",
"--lr", "2e-5",
],
)
# Run the job — Vertex provisions a2-highgpu-1g (A100) machines
model = job.run(
replica_count=1,
machine_type="a2-highgpu-1g", # 1x A100 GPU
accelerator_type="NVIDIA_TESLA_A100",
accelerator_count=1,
)
TPUs (Tensor Processing Units) are Google's custom AI accelerators. They are not GPUs -- they are purpose-built for matrix multiplication, the core operation in neural networks.
TPU generations
TPU Version
TFLOPS (BF16)
HBM
Best For
TPU v4
275
32 GB
Training medium models
TPU v5e
197
16 GB
Cost-efficient inference
TPU v5p
459
95 GB
Large model training
TPU v6e (Trillium)
918
32 GB
Next-gen training & inference
Why TPUs matter: For JAX/TensorFlow workloads, TPU v5p can be 2-3x more cost-effective than NVIDIA H100 for large-scale training. The catch: your code must be TPU-compatible (JAX preferred, PyTorch/XLA works but with caveats).
BigQuery ML lets you train models directly in SQL. No Python, no infrastructure management.
sql
-- Train a classification model in SQL
CREATE OR REPLACE MODEL `my_project.my_dataset.churn_model`
OPTIONS(
model_type='BOOSTED_TREE_CLASSIFIER',
input_label_cols=['churned'],
max_iterations=50
) AS
SELECT * FROM `my_project.my_dataset.customer_features`
WHERE split = 'train';
-- Predict on new data
SELECT * FROM ML.PREDICT(
MODEL `my_project.my_dataset.churn_model`,
(SELECT * FROM `my_project.my_dataset.customer_features` WHERE split = 'test')
);
When BigQuery ML shines: Quick prototyping, tabular data, teams with SQL skills but limited ML expertise. It will never match a custom PyTorch training loop, but for 80% of business ML use cases (churn prediction, demand forecasting, classification), it is remarkably effective.
The biggest mistake teams make is choosing a cloud based on a blog post or conference talk. The right cloud is determined by your constraints, not by feature lists. Use this decision tree:
Step 1: Where does your data already live?
If your data is in S3 (AWS), the egress cost to move it elsewhere is $0.09/GB. For 50TB, that is $4,500 just to move data. Start with AWS.
If your data is in BigQuery (GCP), it is already optimized for Vertex AI. Start with GCP.
If your data is in Azure Blob Storage, Azure ML can access it directly. Start with Azure.
Step 2: What models do you need?
Need GPT-4 with enterprise security and data residency? Azure is the only option.
Need Claude, Llama, and Mistral via a single API? AWS Bedrock.
Need Gemini with tight platform integration or TPU training? GCP.
Step 3: What does your team know?
A team with 3 years of AWS experience will ship 2-3x faster on AWS than on a "technically superior" platform they have never used. Do not underestimate the cost of context switching.
Step 4: What enterprise agreements exist?
A Microsoft Enterprise Agreement can give you 30-50% off Azure. An AWS Savings Plan or GCP Committed Use Discount can similarly reduce costs. Check with your finance team before choosing.
What Do You Think?
Your company uses Office 365, has 10TB in Azure Blob Storage, and wants to fine-tune GPT-4. Which cloud platform should you choose?
The answer is C. This is a textbook case of "follow the data." Your 10TB is already in Azure Blob Storage (moving it would cost ~$900 in egress fees alone). Your company is already paying for Office 365 (likely has an Enterprise Agreement with Microsoft discounts). And GPT-4 with enterprise security is only available through Azure OpenAI. Every constraint points to Azure. Choosing AWS or GCP here would mean paying more, moving data, and fighting organizational inertia -- all for no technical advantage.
Choosing a cloud based on a blog post -- Pick based on your organization's existing investments, compliance requirements, and team skills. Not on which cloud had the best marketing this quarter.
Ignoring data gravity -- If your 500TB data lake lives in S3, moving it to GCS costs $45,000+ in egress fees alone. Start your cloud decision from where your data lives.
Using managed services for everything -- SageMaker endpoints cost 2-3x more than self-managed inference on EC2. Managed services are worth it early on; self-managed saves money at scale.
Forgetting networking costs -- Cross-region data transfer, VPC peering, and egress fees add up silently. A model serving system that fetches embeddings from a different region can cost thousands in hidden networking fees.
Not negotiating -- At scale ($50K+/month), every cloud provider offers custom pricing. Enterprise Discount Programs (AWS), Enterprise Agreements (Azure), and Committed Use Discounts (GCP) can save 30-50%.
Data gravity drives cloud choice more than features. If your data already lives in AWS S3, egress fees and migration effort often outweigh any advantage a competing cloud's ML services offer; optimize within your existing ecosystem first
Each cloud has a distinct AI identity. AWS Bedrock for the broadest managed LLM selection, Azure OpenAI Service for enterprise GPT-4 access, GCP Vertex AI + TPUs for training-intensive JAX/TensorFlow workloads; pick the one that matches your stack
Cloud-agnostic tooling is the escape hatch. MLflow for tracking, Kubeflow for orchestration, and Docker for packaging keep you portable without sacrificing convenience; build on managed services but abstract the critical interfaces
Multi-cloud adds capability and complexity in equal measure. Best-of-breed (three clouds for three purposes) requires a dedicated platform team and cross-cloud egress costs; primary + overflow is usually the right balance for teams under 50 engineers
TPUs beat GPUs on cost-per-FLOP for specific workloads. At scale, TPU v5p can be 2-3× more cost-effective than H100 for large matrix multiplication in JAX; but PyTorch workloads belong on NVIDIA unless you can rewrite them
Pick the cloud where your data already lives, abstract the parts that should be portable (MLflow, Kubeflow, Docker), and accept lock-in only where the managed service genuinely beats the alternatives. Next up: AI History & Future -- the boom-bust pattern that determines which of these platforms still exists in five years and where to bet your career.
Pre-trained model hub (Llama, Stable Diffusion, etc.)