Gemma Dev

Name: Gemma Dev
Author: google-gemma

google-gemma/gemma-skills

340 installs
873 repo stars
Updated July 8, 2026
google-gemma/gemma-skills

gemma-dev is a Claude Code skill that spins up a local streaming Gradio chat UI on Google Gemma instruction-tuned models for developers who need quick model smoke-tests and interactive demos with Hugging Face transformer

About

gemma-dev is a local LLM prototyping skill that loads Google Gemma instruction-tuned models such as google/gemma-4-E2B-it through a Hugging Face transformers text-generation pipeline with streaming output. A Gradio chat interface uses TextIteratorStreamer and threaded generation for responsive token-by-token replies with conversation history support. Developers swap model_id to test other available Gemma checkpoints with device_map auto and dtype auto settings. Reach for gemma-dev when you need a fast local chat demo to validate Gemma model responses before wiring production inference.

Hugging Face text-generation pipeline with device_map auto and dtype auto
TextIteratorStreamer plus background Thread for token-by-token Gradio yields
Maps Gradio chat history into role/content messages for chat templates
GenerationConfig max_new_tokens=256 with swappable model_id (default google/gemma-4-E2B-it)
gr.ChatInterface scaffold titled Gemma Chatbot for one-file local demos

Gemma Dev by the numbers

340 all-time installs (skills.sh)
Ranked #2,149 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
Data as of Aug 5, 2026 (Skillselion catalog sync)

npx skills add https://github.com/google-gemma/gemma-skills --skill gemma-dev

Add your badge

Show developers this skill is listed on Skillselion. Paste this into your README.

[![Listed on Skillselion](https://skillselion.com/badge/skills/google-gemma/gemma-skills/gemma-dev.svg)](https://skillselion.com/skills/google-gemma/gemma-skills/gemma-dev)

Installs	340
repo stars	★ 873
Last updated	July 8, 2026
Repository	google-gemma/gemma-skills ↗

How do you run a local Gemma chat demo?

Spin up a local streaming Gradio chat UI on Google Gemma instruction-tuned models for quick model smoke-tests and demos.

Who is it for?

ML developers prototyping local Gemma instruction-tuned chat demos with Gradio and Hugging Face transformers pipelines.

Skip if: Production GPU serving at scale or teams standardized on hosted APIs like Vertex AI without local model runs.

When should I use this skill?

The user wants a local streaming Gradio chat on Gemma models for smoke-tests, demos, or quick instruction-tuned validation.

What you get

A running Gradio chat UI with streaming Gemma model responses and swappable Hugging Face model checkpoints.

running Gradio chat UI
streaming Gemma inference prototype

Files

SKILL.mdMarkdownGitHub ↗

Gemma Development Skill

1. Core Principle: Prioritize App Tooling

DO NOT generate raw PyTorch, TensorFlow, or transformers code unless the user explicitly asks for "Training," "Fine-tuning," or "Research." Always default to high-level frameworks, SDKs, and tooling optimized for application development.

2. Model Selection Guide

CRITICAL: Do not blindly default to gemma-3-1b-it. You must analyze the user's specific domain, technical constraints, and required input modalities to recommend the exact right fit. When recommending standard models, strictly default to the Gemma 4 generation. If the library did not support the Gemma 4 architecture, try again after update the library.

Core Gemma Models

All Gemma 4 models feature Thinking Mode, enabling advanced reasoning to process complex logic, math, and multi-step problems before generating a response.

Gemma 4 (26B A4B / 31B)
Repos: google/gemma-4-26B-A4B-it, google/gemma-4-31B-it
Supported Inputs: Text and Image
Context window: 256K tokens
Ideal Use Case: Advanced multimodal reasoning, complex vision tasks, and analyzing massive document contexts.
Note: The 26B A4B utilizes a highly efficient Mixture-of-Experts for fast, heavy-weight reasoning, alongside the dense 31B variant.
Gemma 4 (12B)
Repos: google/gemma-4-12B-it
Supported Inputs: Text, Image, Audio
Context window: 256K tokens
Ideal Use Case: Multimodal reasoning (including audio), inference in laptops, and consumer devices.
Gemma 4 (E2B / E4B)
Repos: google/gemma-4-E2B-it, google/gemma-4-E4B-it
Supported Inputs: Text, Image, Audio
Context window: 128K tokens
Ideal Use Case: Mobile NPU acceleration; on-device workflows explicitly requiring native audio processing alongside robust reasoning.

Legacy & Lightweight Models (Gemma 3)

Gemma 3 (4B / 12B / 27B)
Repos: google/gemma-3-4b-it, google/gemma-3-12b-it, google/gemma-3-27b-it
Supports Text and Image inputs with a 128K context window. Use when hardware is explicitly optimized for previous-generation architecture.
Gemma 3 (270M / 1B)
Repos: google/gemma-3-270m-it, google/gemma-3-1b-it
Supports Text-only inputs with a 32K context window. Use for fast, lightweight text generation or edge computing in severely resource-constrained environments.

Task-Specific Variants

Route users to purpose-built variants rather than forcing a standard model to perform highly specialized workflows.

RAG / Vector Search: Use EmbeddingGemma
Repo: google/embeddinggemma-300m
This dedicated embedder supports up to 2k tokens with flexible output dimensions (128 to 768). Fetch Generate embeddings for the best practice.
Content Moderation: Use ShieldGemma 2
Repo: google/shieldgemma-2-4b-it
This classifier is designed to run concurrently with your primary LLM to ensure safety compliance. Fetch ShieldGemma 2 model card for the best practice.

3. Deployment Workflows

Map the user's deployment goals to the correct tooling stack and best practices.

Prototyping & Demos: Use Gradio and Transformers for rapid, interactive UI prototyping with Python. Follow the [assets/gradio-app.py] best practice.
Web & Client Applications: Use transformers.js to run inference directly on-device or entirely in the browser. Follow the [assets/transformers-js-app.js] best practice. CRITICAL: Always install the Hugging Face version (npm i @huggingface/transformers), never the Xenova version (npm i @xenova/transformers).
Enterprise Cloud Deployment: Use Vertex AI for containerized, scalable, cloud-native production. Follow the [assets/vertex-ai-app.py] best practice. This requires setting the environment variables GOOGLE_CLOUD_PROJECT, GOOGLE_CLOUD_LOCATION (e.g., us-central1), and GOOGLE_CLOUD_ENDPOINT_ID.

4. Speed Up Inference with Multi-Token Prediction (MTP)

When the user needs faster inference or lower latency, recommend Multi-Token Prediction (MTP) — Gemma 4's built-in speculative decoding technique. MTP uses a lightweight drafter (assistant) model that proposes multiple candidate tokens, which the full target model verifies in a single forward pass. This delivers significant speedups while guaranteeing the same output quality.

Assistant Model Repos

Each Gemma 4 target model has a corresponding assistant model. The naming convention is <target-model-id>-assistant:

Repos:
google/gemma-4-E2B-it-assistant
google/gemma-4-E4B-it-assistant
google/gemma-4-12B-it-assistant
google/gemma-4-31B-it-assistant
google/gemma-4-26B-A4B-it-assistant

Fetch MTP overview and MTP with Transformers for the best practice.

5. Quantization-Aware Training (QAT)

For deployments requiring maximum efficiency with minimal quality compromise, Gemma offers official Quantization-Aware Training (QAT) models. Unlike standard Post-Training Quantization (PTQ) which compresses a fully trained model and can lead to quality degradation, QAT integrates quantization simulation into the training process itself.

Recommend QAT models based on the target deployment engine:

llama.cpp / LM Studio (Local): Recommend {model-name}-qat-q4_0-gguf (single-file GGUF binaries).
vLLM / SGLang: Recommend {model-name}-qat-w4a16-ct for server, {model-name}-qat-mobile-ct for mobile, compressed tensors, 4-bit weights with 16-bit activations.
Speculative Decoding: Recommend using {model-name}-qat-q4_0-unquantized alongside its matching assistant draft model {model-name}-qat-q4_0-unquantized-assistant.
Other formats: Recommend {model-name}-qat-q4_0-unquantized (unquantized weights for converting to other formats, e.g. MLX).
Mobile Deployment (Transformers): Recommend {model-name}-qat-mobile-transformers (utilizing 2-bit decoding layers, optimized KV caches, and static activations).

Official Hugging Face collections:

`collections/google/gemma-4-qat-q4_0`: Contains -unquantized/-assistant (E2B, E4B, 12B, 26B A4B, 31B), -gguf (E2B, E4B, 12B, 26B A4B, 31B), and -w4a16-ct (E2B, E4B, 12B, 31B).
`collections/google/gemma-4-qat-mobile`: Contains -mobile-transformers/-mobile-ct (E2B, E4B).

6. Documentation Lookup

When MCP is Installed (Preferred)

If the `search_documentation` tool (from the Google MCP server) is available, use it as your only documentation source:

1. Call search_documentation with your query 2. Read the returned documentation 3. Trust MCP results as source of truth for API details — they are always up-to-date.

[!IMPORTANT]

When MCP tools are present, never fetch URLs manually. MCP provides up-to-date, indexed documentation that is more accurate and token-efficient than URL fetching.

When MCP is NOT Installed (Fallback Only)

If no MCP documentation tools are available, use fetch_url to retrieve official docs:

1. Fetch the Index URL (https://ai.google.dev/gemma/docs/llms.txt) to discover available pages. 2. Fetch specific pages as needed. Key reference pages include:

import gradio as gr
from transformers import pipeline, TextIteratorStreamer, GenerationConfig
from threading import Thread

# Load the pipeline
# Replace "google/gemma-4-E2B-it" with other available models
model_id = "google/gemma-4-E2B-it"

pipe = pipeline(
    "text-generation",
    model=model_id,
    device_map="auto",
    dtype="auto",
)

def chat(message, history):
    messages = []

    # Add conversation history
    for msg in history:
        role = msg["role"]

        # Extract text from the content list (e.g. [{'text': 'hello', 'type': 'text'}])
        if isinstance(msg["content"], list):
            content_text = "".join([item["text"] for item in msg["content"] if item["type"] == "text"])
        else:
            content_text = msg["content"]

        messages.append({"role": role, "content": content_text})

    # Add current user message
    messages.append({"role": "user", "content": message})

    streamer = TextIteratorStreamer(pipe.tokenizer, skip_prompt=True, skip_special_tokens=True)
    config = GenerationConfig(max_new_tokens=256)
    thread = Thread(target=pipe, args=(messages,), kwargs=dict(
        generation_config=config,
        streamer=streamer
    ))
    thread.start()

    # Generate response
    generated_text = ""
    for new_text in streamer:
        generated_text += new_text
        yield generated_text

# Create the ChatInterface
demo = gr.ChatInterface(
    fn=chat,
    title="Gemma Chatbot",
    description="Ask Gemma anything!",
)

if __name__ == "__main__":
    demo.launch()

import { pipeline, TextStreamer } from '@huggingface/transformers';
import cliProgress from 'cli-progress';
import inquirer from 'inquirer';

let generator;

async function initializeGemma() {
    console.log('Initializing Gemma model...');
    const progressBar = new cliProgress.SingleBar({}, cliProgress.Presets.shades_classic);
    progressBar.start(100, 0);

    generator = await pipeline('text-generation', 'onnx-community/gemma-4-E2B-it-ONNX', {
        device: 'webgpu',
        dtype: 'q4',
        progress_callback: (progress) => {
            progressBar.update(progress.progress);
        },
    });

    progressBar.stop();
    console.log('Gemma model initialized!');
}

async function* generate(question) {
    const messages = [
        {role: 'user', content: question}
    ];

    const prompt = generator.tokenizer.apply_chat_template(messages, {
        tokenize:false,
        add_generation_prompt: true,
    });

    const streamer = new TextStreamer(generator.tokenizer, {
        skip_prompt: true, // Don't stream the user's prompt back
        skip_special_tokens: true,
    });

    await generator(prompt, {
        max_new_tokens: 256,
        streamer: streamer,
    });
}

async function main() {
    console.clear();
    await initializeGemma();

    while (true) {
        const { question } = await inquirer.prompt({
            type: 'input',
            name: 'question',
            message: "Ask Gemma anything:",
        });

        if (question.toLowerCase() === 'exit') {
            console.log('See you!');
            break;
        }

        console.log('\nGemma: ');

        for await (const chunk of generate(question)) {
            console.log(chunk);
        }

        console.log('\n');
    }
}

main().catch(err => {
    console.error('An error occurred:', err);
});

import os
from transformers import AutoTokenizer
from google.cloud import aiplatform

PROJECT_ID = os.environ.get("GOOGLE_CLOUD_PROJECT")
LOCATION = os.environ.get("GOOGLE_CLOUD_LOCATION")
ENDPOINT_ID = os.environ.get("GOOGLE_CLOUD_ENDPOINT_ID")

MODEL_ID = "google/gemma-4-31B-it"
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

def predict_gemma(project: str, endpoint_id: str, prompt: str, location: str = "us-central1"):
    # Initialize the Vertex AI client
    aiplatform.init(project=project, location=location)
    
    # Reference the deployed endpoint
    endpoint = aiplatform.Endpoint(endpoint_id)
    
    # Format the payload for Gemma 4
    instances = [{"prompt": prompt, "max_tokens": 1024}]
    
    # Generate prediction
    response = endpoint.predict(instances=instances)
    
    for prediction in response.predictions:
        print(prediction)

question = input("User: ")
messages = [
    {"role": "user", "content": question}
]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

predict_gemma(project=PROJECT_ID, location=LOCATION, endpoint_id=ENDPOINT_ID, prompt=prompt)

Related skills

Setup Matt Pocock SkillsScaffold the per-repo configuration that Matt Pocock’s engineering agent skills rely on so they understand the issue tracker, triage labels, and domain documentation la495k196k

Lark AppsConnect Claude, Cursor or custom agents directly to Lark (Feishu) for messaging, document automation, approval workflows and enterprise data access.402k

CavemanSlash token usage by roughly 75% while keeping every technical detail intact when working with Claude Code, Cursor or similar agents.391k94.7k

Lark Skill MakerQuickly turn any Lark/Feishu OpenAPI call or multi-step workflow into a reusable agent skill with its own SKILL.md.386k16k

Running Claude Code Via Litellm CopilotRun Claude Code at a fraction of the cost by routing requests through LiteLLM to the GitHub Copilot Chat API.270k72

Codex PetGenerate a complete Codex Pet spritesheet and metadata from one reference image without needing an OpenAI key or Codex Pro.246k10

FAQ

Which Gemma model does gemma-dev use by default?

gemma-dev defaults to google/gemma-4-E2B-it loaded through a Hugging Face text-generation pipeline. Developers replace model_id to test other available Gemma instruction-tuned checkpoints.

How does gemma-dev stream chat responses?

gemma-dev uses Gradio with TextIteratorStreamer and a background Thread on the transformers pipeline. Tokens stream token-by-token into the chat UI with conversation history preserved.

AI & Agent Buildingllmagents