How to evaluate AI applications // TRAIN BRAIN

How to evaluate AI applications

Vertex AI Evaluation Service Tutorial Notebooks →
https://goo.gle/4i7vdxl
How do developers know if their AI applications are working effectively? How can developers measure AI performance? In this episode of Real Terms for AI, Googlers Aja Hammerly and Jason Davenport delve into creating golden datasets, defining essential metrics, and utilizing tools to measure any AI application's performance.
Chapters:
0:00 - Welcome
0:34 - Evaluating models versus evaluating apps
1:31 - Grounding
2:17 - Sources of evaluation data
3:47 - Define metrics and evaluation
5:07 - Analyzing and understanding metrics
6:19 - Ongoing evaluation
7:48 - Summary
Watch more Real Terms for AI → https://goo.gle/AIwordsExplained
Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech
#GoogleCloud #GenerativeAI
Speakers: Aja Hammerly, Jason Davenport
Products Mentioned: Gemini, Cloud General, Vertex AI

Google Cloud Tech

Helping you build what's next with secure infrastructure, developer tools, APIs, data analytics and machine learning....

Google Cloud x MLB Hackathon - Building with Gemini Models

How to create Looker Studio Reports in Looker

How to use stable diffusion on Cloud Run

Your first workload with AI Hypercomputer

Deploy Gemma 2 with multiple LoRA adapters on GKE

Multimodal AI in action

How to prepare data for LLMs

Cloud migration insights from banking

Introduction to grounding with Gemini on Vertex AI

Fine-tuning open AI models using Hugging Face TRL

Run Hugging Function Face transformers on GPU enabled Cloud Run functions

Ollama and Cloud Run with GPUs

Cloud Run functions with Gemma 2 and Ollama

Running Diffusion with Cloud Run GPUs

Introduction to Gemini on Vertex AI

How do I know my AI app is working?

How to evaluate AI applications

Choosing between self-hosted GKE and managed Vertex AI to host AI models

How to autoscale a TGI deployment on GKE

Looker Conversational Analytics

RAG expansion for AI apps

Using RAG expansion to improve model speed and accuracy

Protecting sensitive data in AI apps

Learn Hybrid Search with Vertex AI Vector Search

Deploy HUGS on GKE with Hugging Face

New "task type" embedding from the DeepMind team improves RAG search quality

Quick Gemma 2 deployment with Hugging Face

What is an AI agent?

Intro to AI agents

RAG vs Model tuning vs Large prompt window

65K node Kubernetes AI Platform - A Reality

Semantic modeling for AI

Function calling for LLMs, what is it? ?

AI + your code: Function Calling

Deploy open models with TGI on Cloud Run

RAG with LangChain on Google Cloud

Looker's Chart Config Editor & Visualization Assistant

Advanced RAG techniques for better retrieval performance

Advanced RAG techniques for developers

Prompt engineering for developers

How to run anything on Google Axion Processors

Google Axion Processors, explained

GenAI is a game changer for podcasts ??

What are Hugging Face Deep Learning Containers?

Demystifying RAG for developers

How to use Retrieval Augmented Generation (RAG)

Architecting a RAG Podcast Summarizer #AI #Tech

Data insights with Looker for Google Workspace

Embeddings for AI/ML and RAG apps

Build RAG apps with embeddings

How to secure your cloud with VPC Service Controls

Leverage LLM strengths for your features ?

Deploy Hugging Face models from Vertex AI Model Garden

Introducing pipe syntax in BigQuery and Cloud Logging

Experience multimodal AI with manga ONE PIECE (ワンピース)

Gemini for Developers - Vertex AI

Architecting a healthcare and life sciences startup with Google Cloud

Cloud KMS Autokey best practices

Boost productivity with Gemini Code Assist's Multi-suggestion Feature

Meet AI’s multitool: Vector embeddings

What are text embeddings?

Visualize data with Looker Studio

Dataflow for IoT galactic research

How to customize Gemini Code Assist with your private code

Dataflow for real-time IoT analytics

Persistent runtimes for notebooks ❌⌛️? #ProTip

Modern generative AI use cases #shorts

Gemini for Developers - Security

A developer’s guide to LLMs

Gemini for Developers - RAG

Use GPUs in Cloud Run

Getting started with Vertex AI #shorts

How Volkswagen built its AI assistant pt. 2

Gemini at Work: Create a marketing campaign with Imagen 3

Gemini at Work: Extracting business-critical insights from video content

Dataflow: Real-time clickstream analytics

Dataflow for Real-time Clickstream Analytics

Gemini in action: How Volkswagen built its AI assistant pt. 1

How to automate resource encryption with Cloud KMS Autokey and Terraform

AI agents for developers productivity #shorts