Anchovy – Image Understanding AI

• Image captioning (primary and secondary captions)
• Object labeling and tagging
• OCR text extraction from images
• Multi-language support
• Used for accessibility features and content understanding

Orca – Core AI Processing Engine

• General-purpose AI processing service
• Text processing and generation
• Multi-modal AI tasks
• The main AI workhorse for various text-based features

Scanner – Smart Screen Analysis

• Screenshot analysis and object detection
• Contextual action suggestions based on screen content
• Google services integration (Calendar, Contacts, Docs, Sheets)
• Smart clipboard operations
• Enables productivity automation from screen content

Mahi – Document Intelligence

• Document summarization
• Text simplification and explanation
• Outline generation
• Interactive Q&A with conversation history
• Designed for reading comprehension and educational assistance

Walrus – Content Safety & Moderation

• Text and image content filtering
• Safety analysis for inappropriate content
• Multi-modal moderation
• Image processing and optimization
• Ensures content safety across AI features

Snapper – General AI Service Provider

• Generic AI request handling
• Flexible processing for miscellaneous AI tasks
• Handles AI tasks that don’t fit other specialized providers

SeaPen – Planned New Feature

• No provider found in the current Manta codebase

Following is the complete list of machine learning models in Chrome many of which are on your device. They are located in your User Data folder and you can easily check to see which ones you have as they are all in numbered folders.

C:\Users\{YOUR_USERNAME}\AppData\Local\Google\Chrome\User Data\optimization_guide_model_store

On-Device AI Models

Chrome uses numerous on-device machine learning models to enhance user experience, improve performance, and protect privacy. These models run locally on your device, ensuring fast responses and data privacy. Here’s a comprehensive list of all Chrome’s on-device AI models and their functions:

Language and Text Processing Models

Language Detection

Identifies the language of text content on web pages to enable translation features and language-specific optimizations.

Text Classifier

Performs smart text selection and entity extraction from web content, helping identify important information like addresses, phone numbers, and dates.

Text Embedder

Generates numerical representations of text for similarity comparisons and semantic understanding across various Chrome features.

Passage Embedder

Creates embeddings specifically for longer text passages, enabling better understanding of document content and context.

Phrase Segmentation

Breaks down sentences into meaningful phrases, improving text comprehension and natural language processing capabilities.

Text Safety

Evaluates text content for potentially harmful or inappropriate material to protect users from unsafe content.

Generalized Safety

A newer, more comprehensive safety model that replaces the basic text safety model with broader content protection capabilities.

Proofreader API

Powers spelling and grammar checking features to help users write better content across the web.

Writing Assistance API

Supports Chrome’s Writer and Rewriter features, helping users compose and improve their written content.

Page Analysis and Content Models

Page Topics (v1 and v2)

Analyzes web pages to determine the main topics and themes present in the content for better content recommendations and filtering.

Page Entities

Identifies specific entities (people, places, organizations, products) mentioned on web pages for enhanced understanding and features.

Page Visibility

Determines which UI elements should be visible on a page based on content and user context.

Visual Search Classification

Classifies and extracts searchable images from web pages, enabling visual search capabilities.

Education Classifier

Identifies educational content and resources on web pages for specialized handling and recommendations.

Security and Privacy Models

Client-Side Phishing Detection

Detects potential phishing websites directly on your device without sending URLs to external servers.

Client-Side Phishing Image Embedder

Analyzes images on web pages to identify visual phishing attempts and deceptive content.

Notification Content Detection

Classifies notification content to identify suspicious or potentially harmful messages.

Scam Detection

Identifies potential scam patterns in web content and user interactions.

Notification Permission Predictions

Predicts whether users are likely to accept notification permissions based on context and behavior.

Geolocation Permission Predictions

Estimates the likelihood of users granting location access to websites.

Geolocation Image Permission Relevance

Analyzes visual context to determine if location permission requests are relevant.

Notification Image Permission Relevance

Evaluates visual elements to assess the relevance of notification permission requests.

Permissions AI (Multiple Models)

Advanced models for intelligent permission request handling, including AIv4 models for desktop geolocation and notifications.

User Segmentation and Personalization Models

Segmentation: New Tab User

Identifies users who frequently use the new tab page for personalized experiences.

Segmentation: Share User

Recognizes users who regularly share content for optimized sharing features.

Segmentation: Voice User

Identifies users who prefer voice interactions for enhanced voice features.

Segmentation: Chrome Start Android (v1 and v2)

Segments Android users based on their Chrome start page usage patterns.

Segmentation: Query Tiles User

Identifies users who benefit from query tile suggestions.

Segmentation: Low User Engagement

Detects users with minimal Chrome engagement for targeted re-engagement strategies.

Segmentation: Feed User

Identifies users who actively engage with Chrome’s content feed.

Segmentation: Shopping User

Recognizes users interested in shopping for enhanced e-commerce features.

Segmentation: Search User

Identifies users who heavily rely on search functionality.

Segmentation: Device Switcher

Detects users who frequently switch between devices for continuity features.

Segmentation: Adaptive Toolbar

Customizes toolbar options based on user behavior and preferences.

Segmentation: Tablet Productivity User

Identifies tablet users focused on productivity tasks.

Segmentation: Bottom Toolbar

Determines which users would benefit from a bottom toolbar layout.

Segmentation: Desktop NTP Module

Personalizes Desktop New Tab Page modules based on user preferences.

Segmentation: Compose Promotion

Determines which users should see promotions for Chrome’s Compose feature.

Segmentation: FedCM User

Identifies users who would benefit from Federated Credential Management features.

Segmentation: iOS Default Browser Promo

Determines when to show default browser promotions to iOS users.

Segmentation: Metrics Clustering

Groups users based on usage metrics for better feature targeting.

Search and Navigation Models

Omnibox On-Device Tail Suggest

Provides intelligent autocomplete suggestions for URL bar queries without server calls.

Omnibox URL Scoring

Ranks and scores URL suggestions in the address bar for better predictions.

History Search

Enhances searching through browsing history with intelligent understanding.

History Query Intent

Understands the user’s intent when searching through their browsing history.

URL Visit Resumption Ranker

Ranks previously visited URLs for quick resumption of browsing sessions.

Preloading Heuristics

Predicts which links users are likely to click for speculative preloading.

Content Creation and Assistance

Compose

Powers on-device text composition assistance for various writing tasks.

Help Me Write

The AI writing assistant for short-form content creation (as discussed in the previous article).

Form and Field Processing

Autofill Field Classification

Identifies and classifies form fields for accurate autofill suggestions.

Password Manager Form Classification

Recognizes and categorizes password and login forms for secure credential management.

Module Ranking and Recommendations

New Tab Page History Clusters Module Ranking

Ranks grouped history items for display in the New Tab Page.

iOS Module Ranker

Determines the order and relevance of modules on iOS start pages.

Android Home Module Ranker

Optimizes the arrangement of modules on Android home screens.

Application and Installation

Web App Installation Promo

Determines when and how to promote Progressive Web App installations.

Contextual Page Action: Price Tracking

Identifies when to show price tracking options based on page content.

Media and Visual Processing

Camera Background Segmentation

Separates foreground from background in video streams for virtual backgrounds.

Performance Optimization

Painful Page Load Prediction

Predicts when a page load will be slow or resource-intensive for optimization.

Experimental and Validation

Model Validation

Tests and validates new model deployments and updates.

Segmentation Dummy

Enables data collection for various experimental features.

Experimental Embedder

Tests new embedding model architectures and approaches.

AI Features Security Notes

Chrome deeply integrates AI both in user-facing features like Gemini Live in Chrome , “Help me write” and Devtools assistants and in internal models that help block unwanted
notifications or improve page loading.

Chrome does not treat misleading, misaligned or unsafe model output as a
vulnerability. Please report such safety violations using in-product feedback
mechanisms.

Entering a prompt into an AI feature’s input surface causes inappropriate output?

Chrome AI features include guardrails to ensure that their output is safe and
reasonable but these guidelines do not form a security boundary. Any prompt that
causes these guidelines to be violated is not a security issue in Chrome. Use
in-product mechanisms to thumbs up / thumbs down results, or click on
‘send feedback’ to report other inappropriate content.

Entering a prompt into an AI feature’s input surface leaks the system prompt, or provides access to backend services?

For AI features implemented using a Google backend it is possible that some
prompted output could be a valid abuse report, but will not be considered to be
bugs in Chrome. These should be reported via the Google Abuse VRP
or Google VRP depending on the severity of the
issue.

Entering a prompt into an AI feature’s input surface causes information to leak, or actions to happen?

Chrome AI features trust what people using Chrome supply in input fields, audio
inputs, or other Chrome input surfaces. Tricking a user into entering a
malicious prompt (e.g. by copy/pasting from a site) is not considered to be a
security boundary as many people copy & paste text and urls as they use features
in Chrome.

Url paths, parameters or fragments can influence the output of Chrome AI features?

AI features may use urls when generating their output so it is expected that
page content will influence the output. Chrome AI features include mitigations
and filters to prevent harmful actions that result from operating on page
content. Controlling the AI output is, by itself, not a security issue, unless
some further harm to a user can be demonstrated.

Page content can influence the output of Chrome AI features?

AI features may use page content (including images and subframes) when
generating their output so it is expected that page content will influence the
output. Chrome AI features include mitigations and filters to prevent harmful
actions that result from operating on page content. Controlling the AI output
is, by itself, not a security issue, unless some further harm to a user can be
demonstrated.

Invisible page content can influence the output of Chrome AI features?

AI features may use page content including invisible content when generating
their output so it is expected that page content will influence the output.
Chrome AI features may detect, scrub, or deprioritize invisible content, but
failing to do so is not considered a security vulnerability as it is impossible
to do so in all cases.

I have an example of page content that results in Chrome AI features creating links that leak information if followed?

Chrome AI features take actions to limit what navigations are possible, and
require user action before following links that could leak information to
prevent scalable or targeted attacks. Web pages can already supply links or
cause redirections and navigation and causing a user to follow these, via an AI
feature, does not add a new attack surface.

I have an example of page content that results in Chrome AI features performing harmful actions?

Indirect prompt injections that result in unintended actions or leak information
may be considered security issues and should be reported through the Chrome
security tracker. Please create a recording from a fresh session that
demonstrates the issue, and upload all files used as part of the demonstration.
If a Gemini session is associated with your report, it will help us if you are
able to share the session from your activity page, and the version of the model
you are using.

I have an example of page content that results in XSS in the context of a Chrome AI feature?

Output surfaces should sanitize inputs and transformed outputs. Please create a
recording from a fresh session that demonstrates the issue, and upload all files
used as part of the demonstration. If a Gemini session is associated with your
report, it will help us if you are able to share the session from your activity
page, and the version of the model you are using. Note that directly injecting
code into a trusted surface via devtools does not demonstrate a vulnerability.

AI Generated Vulnerability reports

Should I ask an AI to Generate a Vulnerability Report for Chrome?

Simply asking an AI to identify a bug report in Chrome is unlikely to yield a
valid report. Before submitting a report generated by AI please ensure you have
done enough human work to validate that any issue is (a) in our threat model,
and (b) reachable in Chrome by constructing a POC, generating an ASAN trace,
recording the bug reproducing, or performing your own debugging.

AI is prone to hallucinations when asked to find security bugs and can generate
reports that repeat previously fixed issues, or describe general classes of bugs
without discovering a specific actionable issue. As the reports can be lengthy,
they take a lot of time for our security experts to process and understand
before closing. Submitting reports without doing some work yourself to validate
that an issue is actually present in Chrome harms our users by wasting the time
and resources of the Chrome security team.

Submitting multiple low-quality AI generated reports will be treated as spamming
and has lead to accounts being banned from our reporting systems.

AI can be used to accelerate developer workflows and may be useful when
understanding code or translating from one language to another. AI tools can be
helpful when searching for security vulnerabilities in Chrome, but remember that
additional work must be done to ensure that vulnerability reports are brief,
actionable, and reproducible. These must meet the prerequisites of a baseline security bug report before we can pass them to teams to be fixed.

Source: https://source.chromium.org/chromium/chromium/src/+/main:docs/security/faq.md