Self host Gemma 4: Deploy LLMs on Cloud Run GPUs // TRAIN BRAIN

Self host Gemma 4: Deploy LLMs on Cloud Run GPUs

GCP credit → https://goo.gle/handson-ep7-lab1
Lab → https://goo.gle/guardians
In this episode, we deploy Google's Gemma 4 model to Cloud Run two completely different ways, each with real trade-offs you need to understand before choosing one for production.
? Ollama — model baked into the container. Instant cold starts. Rebuild to update.
⚡ vLLM — model mounted from Cloud Storage via FUSE. Slower first boot, but swap models without redeploying.
Both use Cloud Run GPUs, scale to zero, and ship through automated CI/CD with Cloud Build.
We build both. You decide which fits. ?
? CI/CD with Cloud Build
?️ GPU accelerated serverless inference
? Baked in vs. decoupled model architecture
? Scale to zero
⚖️ Cold start speed vs. production agility
Chapters:
0:00 - Intro
6:08 - Getting started with Agentverse lab
7:57 - Laying the foundations of the citadel
16:07 - Forging the power core: Self hosted LLMs
28:02 - Forging the citadel's central core: Deploy vLLM
43:59 - Summary
More resources:
Cloud Run GPU documentation → https://goo.gle/4sEbTvG
Ollama documentation → https://goo.gle/3Qdi64w
vLLM documentation → https://goo.gle/4cvvxE9
Cloud Storage FUSE → https://goo.gle/4cQAb0V
Watch more Hands on AI → https://www.youtube.com/watch?v=qCBreTfjFHQ&list=PLIivdWyY5sqKnJOvP89yF8t9mWuzMTcbM
? Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech
#Gemma4 #CloudRun
Speakers: Ayo Adedeji, Annie Wang
Products Mentioned: Agent Development Kit, Gemini API, Cloud Run

Google Cloud Tech

Helping you build what's next with secure infrastructure, developer tools, APIs, data analytics and machine learning....

Self host Gemma 4: Deploy LLMs on Cloud Run GPUs

How to scale to 60 million requests (with ZERO ops team) using Cloud Run

AI agent long-term memory with memory bank

Mark your calendars for the AI Agent Clinic on April 15th!

Join our Googlers live on April 15th!

Orchestrating ML/AI workloads with TPUs on GKE

BigQuery Graph in 5'

How to add persistent memory to your AI agent

What are domain specific language models?

Cloud Run Networking explained (Updated!)

Fundamentals of ADK - Learning Path

Looker and AlloyDB: The ultimate stack for near real time operational business intelligence

How to authenticate Google Cloud Client Libraries

Reinforcement learning on TPU demo

Private routing to Google with Network Connectivity Center

Running a multi-agent AI architecture

How to get started with Google Cloud Client Libraries

BigQuery Migration Service: Validation and optimization

Build an AI agent with Gemini CLI and Agent Development Kit

Mainframe Connector demo series

Accelerate with AI debrief

Quickstart: Conversational Analytics with GCP Billing and Looker

Scaling your AI agent architecture with Cloud Run

Building AI agents that speak to each other

What are Google Cloud Client Libraries?

Architecting multi-agent systems

BigQuery Migration Service: SQL and data transfer

Can we build the ultimate AI co-founder in 72 hours with Gemini?

Reinforcement learning & fine-tuning on TPUs | The Agent Factory Podcast

Building a life saving MCP server on Cloud Run (Avalanche demo)

What is Cluster Director?

Serving open models on Vertex AI: The comprehensive developer's guide

How to evaluate agents in practice

Antigravity and Nano Banana Pro with Remik | The Agent Factory Podcast

How to build context systems for AI agents

Run MongoDB compatible apps on Firestore (Zero code changes)

Stop coding, start architecting: Google Antigravity + Cloud Run

[Demo] Network Security Integration with Palo Alto

How to build a financial analyst assistant with Vertex AI Studio & Gemini in under 10 minutes

The agent evaluation revolution

Agent sandbox and Pod snapshotting: Supercharging agents on GKE | The Agent Factory Podcast

Leveraging the Looker connector in Looker Studio

How to assess data lake and data warehouse migrations to BigQuery

Refining your vision: A guide to AI image editing

From text to vision: An intro to AI image generation

Evolving your story: A guide to AI video editing

Bringing ideas to life: An intro to AI video generation

Building with Gemini 3, AI Studio, Antigravity, and Nano Banana | The Agent Factory Podcast

Fine-tuning open LLMs on GKE: The implementation gap

Video avatar agent | The Agent Factory Podcast

Gemini CLI: Write and deploy a Cloud Run app in 5 minutes

Build ANYTHING with Gemini 3 | The Agent Factory Podcast

Building Your Own MCP Server with ADK

This AI agent runs on Cloud Run + NVIDIA GPUs

Scaling AI with Google Cloud's TPUs

Deploying scalable and reliable AI inference on Google Cloud

Serving AI models at scale with vLLM

AI workload orchestration options

AI/ML frameworks for cloud TPUs

Model types and performance bottlenecks

AI workload storage options

Connecting ADK Agents to MCP Servers

Use the Gemini CLI Jules and Observability extensions together

Introduction to Vertex AI Agent Engine

Power your AI agents with MCP tools on Google Cloud Run

Use the Gemini CLI Jules and security extensions to fix security vulnerabilities in the background

Use the Jules extension for Gemini CLI to fix multiple GitHub issues

Dataplex fundamentals: Aspects & glossaries

We tried to jailbreak our AI (and Model Armor stopped it)

Parallel bug fixing & unit testing with Jules and Observability extensions for Gemini CLI

How to fix security vulnerabilities with the Jules and security extensions for Gemini CLI

How to fix multiple GitHub issues at once using the Jules extension for Gemini CLI

The path to AI inferencing on GKE Part 1: Guided model research

Vibe coding with Google AI Studio | The Agent Factory

Is it possible to create a model agnostic prompt?

Building agentic RAG for e-commerce with ADK and Vector Search

Demo: Vibe coding a command line Markdown viewer with the Gemini CLI

Don't guess: How to benchmark your AI prompts

Identity and Access Management for Agents

ComfyUI on GKE for Genmedia solutions