Gemini Vision

Gemini Vision API Skill

This skill enables Claude to use Google's Gemini API for advanced image understanding tasks including captioning, classification, visual question answering, object detection, segmentation, and multi-image analysis.

Quick Start

Prerequisites

1. Get API Key: Obtain from Google AI Studio 2. Install SDK: pip install google-genai (Python 3.9+)

API Key Configuration

The skill checks for GEMINI_API_KEY in this order:

1. Process environment variable (recommended)

   export GEMINI_API_KEY="your-api-key"

2. Skill directory: .claude/skills/gemini-vision/.env

   GEMINI_API_KEY=your-api-key

3. Project directory: .env or .gemini_api_key in project root

Security: Never commit API keys to version control. Add .env to .gitignore.

Core Capabilities

Image Analysis

• Captioning: Generate descriptive text for images
• Classification: Categorize and identify image content
• Visual QA: Answer questions about image content
• Multi-image: Compare and analyze up to 3,600 images

Advanced Features (Model-Specific)

• Object Detection: Identify and locate objects with bounding boxes (Gemini 2.0+)
• Segmentation: Create pixel-level masks for objects (Gemini 2.5+)
• Document Understanding: Process PDFs with vision (up to 1,000 pages)

Supported Formats

• Images: PNG, JPEG, WEBP, HEIC, HEIF
• Documents: PDF (up to 1,000 pages)
• Size Limits:
• Inline: 20MB max total request size
• File API: For larger files
• Max images: 3,600 per request

Available Models

• gemini-2.5-pro: Most capable, segmentation + detection
• gemini-2.5-flash: Fast, efficient, segmentation + detection
• gemini-2.5-flash-lite: Lightweight, segmentation + detection
• gemini-2.0-flash: Object detection support
• gemini-1.5-pro/flash: Previous generation

Usage Examples

Basic Image Analysis

# Analyze a local image
python scripts/analyze-image.py path/to/image.jpg "What's in this image?"

# Analyze from URL
python scripts/analyze-image.py https://example.com/image.jpg "Describe this"

# Specify model
python scripts/analyze-image.py image.jpg "Caption this" --model gemini-2.5-pro

Object Detection (2.0+)

python scripts/analyze-image.py image.jpg "Detect all objects" --model gemini-2.0-flash

Multi-Image Comparison

python scripts/analyze-image.py img1.jpg img2.jpg "What's different between these?"

File Upload (for large files or reuse)

# Upload file
python scripts/upload-file.py path/to/large-image.jpg

# Use uploaded file
python scripts/analyze-image.py file://file-id "Caption this"

File Management

# List uploaded files
python scripts/manage-files.py list

# Get file info
python scripts/manage-files.py get file-id

# Delete file
python scripts/manage-files.py delete file-id

Token Costs

Images consume tokens based on size:

• Small (≤384px both dimensions): 258 tokens
• Large: Tiled into 768×768 chunks, 258 tokens each

Token Formula:

crop_unit = floor(min(width, height) / 1.5)
tiles = (width / crop_unit) × (height / crop_unit)
total_tokens = tiles × 258

Example: 960×540 image = 6 tiles = 1,548 tokens

Rate Limits

Limits vary by tier (Free, Tier 1, 2, 3):

• Measured in RPM (requests/min), TPM (tokens/min), RPD (requests/day)
• Applied per project, not per API key
• RPD resets at midnight Pacific

Best Practices

Image Quality

• Use clear, non-blurry images
• Verify correct image rotation
• Consider token costs when sizing

Prompting

• Be specific in instructions
• Place text after image for single-image prompts
• Use few-shot examples for better accuracy
• Specify output format (JSON, markdown, etc.)

File Management

• Use File API for files >20MB
• Use File API for repeated usage (saves tokens)
• Files auto-delete after 48 hours
• Clean up manually when done

Security

• Never expose API keys in code
• Use environment variables
• Add API key restrictions in Google Cloud Console
• Monitor usage regularly
• Rotate keys periodically

Error Handling

Common errors:

• 401: Invalid API key
• 429: Rate limit exceeded
• 400: Invalid request (check file size, format)
• 403: Permission denied (check API key restrictions)

Additional Resources

See the references/ directory for:

• api-reference.md: Detailed API methods and endpoints
• examples.md: Comprehensive code examples
• best-practices.md: Advanced tips and optimization strategies

Implementation Guide

When implementing Gemini vision features:

1. Check API key availability using the 3-step lookup 2. Choose appropriate model based on requirements:

• Need segmentation? Use 2.5+ models
• Need detection? Use 2.0+ models
• Need speed? Use Flash variants
• Need quality? Use Pro variants

3. Validate inputs:

• Check file format (PNG, JPEG, WEBP, HEIC, HEIF, PDF)
• Verify file size (<20MB for inline, >20MB use File API)
• Count images (max 3,600)

4. Handle responses appropriately:

• Parse structured output if requested
• Extract bounding boxes for object detection
• Process segmentation masks if applicable

5. Manage files efficiently:

• Upload large files via File API
• Reuse uploaded files when possible
• Clean up after use

Scripts Overview

All scripts support the 3-step API key lookup:

• analyze-image.py: Main script for image analysis, supports inline and File API
• upload-file.py: Upload files to Gemini File API
• manage-files.py: List, get metadata, and delete uploaded files

Run any script with --help for detailed usage instructions.

---

Official Documentation: https://ai.google.dev/gemini-api/docs/image-understanding

Gemini Vision Skill

Google Gemini API skill for advanced image understanding tasks.

Quick Start

1. Get API Key: https://aistudio.google.com/apikey 2. Install SDK: pip install google-genai 3. Set API Key:

   export GEMINI_API_KEY="your-api-key"

Skill Structure

gemini-vision/
├── SKILL.md                 # Main skill file (auto-loaded by Claude)
├── README.md               # This file
├── scripts/                # Helper scripts
│   ├── analyze-image.py   # Main analysis script
│   ├── upload-file.py     # File upload helper
│   └── manage-files.py    # File management (list/get/delete)
└── references/            # Detailed documentation
    ├── api-reference.md   # API methods and endpoints
    ├── examples.md        # Code examples
    └── best-practices.md  # Advanced tips and optimization

Usage

Invoke the Skill

# In Claude Code CLI
/gemini-vision

Once loaded, Claude will have access to all Gemini Vision capabilities.

Direct Script Usage

# Analyze single image
python scripts/analyze-image.py image.jpg "What's in this image?"

# Multiple images
python scripts/analyze-image.py img1.jpg img2.jpg "What's different?"

# Upload file
python scripts/upload-file.py large_image.jpg

# Manage files
python scripts/manage-files.py list
python scripts/manage-files.py get files/abc123
python scripts/manage-files.py delete files/abc123

API Key Configuration

The skill checks for GEMINI_API_KEY in this order:

1. Process environment (recommended)

   export GEMINI_API_KEY="your-key"

2. Skill directory: .claude/skills/gemini-vision/.env

   GEMINI_API_KEY=your-api-key

3. Project root: .env or .gemini_api_key

Capabilities

Basic Features

• Image captioning and description
• Visual question answering
• Image classification
• Multi-image analysis (up to 3,600 images)

Advanced Features

• Object Detection (Gemini 2.0+): Bounding boxes
• Segmentation (Gemini 2.5+): Pixel-level masks
• Document Understanding: PDF processing (up to 1,000 pages)

Supported Formats

• Images: PNG, JPEG, WEBP, HEIC, HEIF
• Documents: PDF

Models

• gemini-2.5-pro: Most capable, segmentation + detection
• gemini-2.5-flash: Fast and efficient (recommended)
• gemini-2.5-flash-lite: Lightweight, high volume
• gemini-2.0-flash: Object detection
• gemini-1.5-pro/flash: Previous generation

Documentation

• SKILL.md: Quick reference and common usage patterns
• references/api-reference.md: Complete API documentation
• references/examples.md: Comprehensive code examples
• references/best-practices.md: Production tips and optimization

Examples

Python

from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

with open('image.jpg', 'rb') as f:
    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[
            types.Part.from_bytes(f.read(), mime_type='image/jpeg'),
            'Describe this image'
        ]
    )

print(response.text)

JavaScript

import { GoogleGenAI } from "@google/genai";
import fs from "node:fs";

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });
const image = fs.readFileSync("image.jpg", { encoding: "base64" });

const response = await ai.models.generate({
  model: "gemini-2.5-flash",
  contents: [{
    parts: [
      { inline_data: { mime_type: "image/jpeg", data: image } },
      { text: "What's in this image?" }
    ]
  }]
});

console.log(response.text);

Resources

• Official Docs: https://ai.google.dev/gemini-api/docs/image-understanding
• API Reference: https://ai.google.dev/gemini-api/docs/reference
• Google AI Studio: https://aistudio.google.com
• Cookbook: https://github.com/google-gemini/cookbook

License

MIT

Gemini Vision API Reference

Complete API reference for Google's Gemini Vision API.

Base URL

https://generativelanguage.googleapis.com/v1beta

Authentication

All requests require an API key passed as a query parameter or header:

Query Parameter:

?key=YOUR_API_KEY

Header:

Authorization: Bearer YOUR_API_KEY

API Methods

1. Generate Content

Generate a response from the model with image input.

Endpoint:

POST /models/{model}:generateContent

Models:

• gemini-2.5-pro - Most capable, segmentation + detection
• gemini-2.5-flash - Fast and efficient, segmentation + detection
• gemini-2.5-flash-lite - Lightweight, segmentation + detection
• gemini-2.0-flash - Object detection support
• gemini-1.5-pro - Previous generation, high quality
• gemini-1.5-flash - Previous generation, fast

Request Body:

{
  "contents": [
    {
      "parts": [
        {
          "text": "What's in this image?"
        },
        {
          "inline_data": {
            "mime_type": "image/jpeg",
            "data": "base64_encoded_image_data"
          }
        }
      ]
    }
  ]
}

Response:

{
  "candidates": [
    {
      "content": {
        "parts": [
          {
            "text": "The image shows..."
          }
        ]
      },
      "finishReason": "STOP"
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 258,
    "candidatesTokenCount": 45,
    "totalTokenCount": 303
  }
}

Python Example:

from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

with open('image.jpg', 'rb') as f:
    image_bytes = f.read()

response = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[
        types.Part.from_bytes(
            data=image_bytes,
            mime_type='image/jpeg'
        ),
        'What is in this image?'
    ]
)

print(response.text)

2. Upload File

Upload a file for reuse across multiple requests.

Endpoint:

POST /upload/files

Request (Multipart):

Content-Type: multipart/form-data

file: <binary file data>

Response:

{
  "file": {
    "name": "files/abc123xyz",
    "displayName": "image.jpg",
    "mimeType": "image/jpeg",
    "sizeBytes": "524288",
    "createTime": "2025-10-26T10:30:00Z",
    "updateTime": "2025-10-26T10:30:00Z",
    "expirationTime": "2025-10-28T10:30:00Z",
    "sha256Hash": "...",
    "uri": "https://generativelanguage.googleapis.com/v1beta/files/abc123xyz",
    "state": "ACTIVE"
  }
}

Python Example:

client = genai.Client(api_key="YOUR_API_KEY")

uploaded_file = client.files.upload(file="image.jpg")

print(f"Uploaded: {uploaded_file.name}")
print(f"State: {uploaded_file.state}")

3. Get File

Retrieve metadata for an uploaded file.

Endpoint:

GET /files/{file_id}

Response:

{
  "name": "files/abc123xyz",
  "displayName": "image.jpg",
  "mimeType": "image/jpeg",
  "sizeBytes": "524288",
  "createTime": "2025-10-26T10:30:00Z",
  "updateTime": "2025-10-26T10:30:00Z",
  "expirationTime": "2025-10-28T10:30:00Z",
  "sha256Hash": "...",
  "uri": "https://generativelanguage.googleapis.com/v1beta/files/abc123xyz",
  "state": "ACTIVE"
}

Python Example:

file_info = client.files.get(name="files/abc123xyz")
print(f"File: {file_info.display_name}")
print(f"Size: {file_info.size_bytes} bytes")

4. List Files

List all uploaded files.

Endpoint:

GET /files

Query Parameters:

• pageSize (optional): Maximum number of files to return (default: 10)
• pageToken (optional): Token for pagination

Response:

{
  "files": [
    {
      "name": "files/abc123xyz",
      "displayName": "image.jpg",
      "mimeType": "image/jpeg",
      ...
    }
  ],
  "nextPageToken": "..."
}

Python Example:

files = client.files.list()
for file in files:
    print(f"{file.display_name}: {file.name}")

5. Delete File

Delete an uploaded file.

Endpoint:

DELETE /files/{file_id}

Response:

{}

Python Example:

client.files.delete(name="files/abc123xyz")
print("File deleted")

Request Parameters

Content Part Types

Text Part:

{
  "text": "Your prompt text"
}

Inline Image Data:

{
  "inline_data": {
    "mime_type": "image/jpeg",
    "data": "base64_encoded_data"
  }
}

File Reference:

{
  "file_data": {
    "mime_type": "image/jpeg",
    "file_uri": "https://generativelanguage.googleapis.com/v1beta/files/abc123"
  }
}

Supported MIME Types

• image/png - PNG images
• image/jpeg - JPEG images
• image/webp - WebP images
• image/heic - HEIC images (Apple)
• image/heif - HEIF images
• application/pdf - PDF documents

Rate Limits

Limits vary by billing tier:

RPM: Requests per minute TPM: Tokens per minute RPD: Requests per day

Error Codes

Token Calculation

Images consume tokens based on their dimensions:

Small Images (≤384px both dimensions):

• Cost: 258 tokens

Large Images:

• Images are tiled into 768×768 pixel chunks
• Each chunk costs 258 tokens

Formula:

crop_unit = floor(min(width, height) / 1.5)
tiles_x = width / crop_unit
tiles_y = height / crop_unit
total_tiles = tiles_x × tiles_y
total_tokens = total_tiles × 258

Examples:

• 960×540 image: 6 tiles = 1,548 tokens
• 1920×1080 image: 6 tiles = 1,548 tokens
• 3840×2160 image: 25 tiles = 6,450 tokens

Size Limits

• Inline data: 20MB total request size (including prompts)
• File API: For files larger than 20MB
• Max images per request: 3,600 files
• PDF pages: Up to 1,000 pages
• File retention: 48 hours after upload

Best Practices

API Key Security

• Use environment variables
• Never commit keys to version control
• Add API key restrictions in Google Cloud Console
• Rotate keys regularly

File Management

• Use File API for files >20MB
• Use File API for repeated usage (saves tokens)
• Delete files after use to free quota
• Files auto-delete after 48 hours

Performance Optimization

• Choose appropriate model (Flash vs Pro)
• Resize images to reduce token costs
• Batch requests when possible
• Implement caching for repeated queries

Error Handling

• Implement exponential backoff for rate limits
• Validate inputs before API calls
• Handle network errors gracefully
• Log errors for debugging

Additional Resources

• Official Documentation: https://ai.google.dev/gemini-api/docs
• API Reference: https://ai.google.dev/gemini-api/docs/reference
• Google AI Studio: https://aistudio.google.com
• Community Forum: https://discuss.ai.google.dev

Gemini Vision API - Best Practices

Advanced tips and optimization strategies for production use.

Table of Contents

1. API Key Management 2. Image Quality & Preparation 3. Prompt Engineering 4. Performance Optimization 5. Cost Optimization 6. Error Handling 7. File Management 8. Production Deployment

---

API Key Management

Security Best Practices

DO:

• ✅ Store API keys in environment variables
• ✅ Use secret management services (AWS Secrets Manager, Google Secret Manager)
• ✅ Add API key restrictions in Google Cloud Console
• ✅ Rotate keys regularly (every 90 days)
• ✅ Monitor usage for anomalies
• ✅ Use separate keys for dev/staging/prod

DON'T:

• ❌ Commit API keys to version control
• ❌ Hardcode keys in source code
• ❌ Share keys via email or Slack
• ❌ Use production keys in development
• ❌ Expose keys in client-side code
• ❌ Log API keys in error messages

Key Restrictions

Configure in Google Cloud Console:

1. Application restrictions:

• HTTP referrers (websites)
• IP addresses (servers)
• Android apps
• iOS apps

2. API restrictions:

• Limit to specific Google APIs
• Only enable Generative Language API

3. Usage quotas:

• Set daily request limits
• Alert on unusual activity

---

Image Quality & Preparation

Image Quality Guidelines

High-quality images produce better results:

• ✅ Clear, well-lit images
• ✅ Correct orientation
• ✅ Appropriate resolution (not too small)
• ✅ Minimal compression artifacts
• ✅ Focused subject matter

Avoid:

• ❌ Blurry or out-of-focus images
• ❌ Heavily compressed images
• ❌ Rotated or upside-down images
• ❌ Very dark or overexposed images
• ❌ Watermarked images (may confuse model)

Image Preprocessing

from PIL import Image, ImageEnhance

def prepare_image(input_path, output_path):
    """Prepare image for optimal Gemini analysis."""
    img = Image.open(input_path)

    # 1. Auto-rotate based on EXIF data
    try:
        from PIL.ExifTags import TAGS
        exif = img._getexif()
        if exif:
            orientation = exif.get(274)  # Orientation tag
            if orientation == 3:
                img = img.rotate(180, expand=True)
            elif orientation == 6:
                img = img.rotate(270, expand=True)
            elif orientation == 8:
                img = img.rotate(90, expand=True)
    except:
        pass

    # 2. Enhance contrast if needed
    enhancer = ImageEnhance.Contrast(img)
    img = enhancer.enhance(1.2)

    # 3. Convert to RGB (remove alpha channel)
    if img.mode in ('RGBA', 'LA', 'P'):
        background = Image.new('RGB', img.size, (255, 255, 255))
        if img.mode == 'P':
            img = img.convert('RGBA')
        background.paste(img, mask=img.split()[-1])
        img = background

    # 4. Save with good quality
    img.save(output_path, 'JPEG', quality=95)
    return output_path

Resolution Guidelines

For optimal token usage:

• Small images (≤384px): 258 tokens
• Keep important images at 768px max dimension
• Large images (>768px) get tiled, increasing cost

def optimize_resolution(image_path, target_size=768):
    """Resize to minimize tokens while keeping quality."""
    img = Image.open(image_path)
    w, h = img.size

    # Already optimal
    if max(w, h) <= target_size:
        return img

    # Calculate new dimensions
    if w > h:
        new_w = target_size
        new_h = int(h * target_size / w)
    else:
        new_h = target_size
        new_w = int(w * target_size / h)

    return img.resize((new_w, new_h), Image.LANCZOS)

---

Prompt Engineering

Effective Prompts

Be Specific:

# Vague ❌
"What's in the image?"

# Specific ✅
"List all vehicles visible in the image with their colors and approximate positions."

Specify Output Format:

# No format ❌
"Tell me about this product."

# With format ✅
"Analyze this product and return JSON with: name, category, colors, features, and estimated price range."

Use Context:

# No context ❌
"Is this good quality?"

# With context ✅
"You are a professional photographer. Evaluate this image's composition, lighting, and technical quality. Rate from 1-10 and explain."

Multi-Turn Conversations

For complex analysis, break into steps:

from google import genai

client = genai.Client(api_key="YOUR_API_KEY")

# Upload once
uploaded_file = client.files.upload(file="complex_scene.jpg")

# Step 1: Overview
response1 = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[uploaded_file, "Describe the overall scene."]
)

# Step 2: Details
response2 = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[uploaded_file, f"Based on this scene: {response1.text}\n\nNow identify all people and their activities."]
)

# Step 3: Analysis
response3 = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[uploaded_file, f"Given: {response2.text}\n\nWhat social dynamics are present?"]
)

Few-Shot Learning

Provide examples for better accuracy:

prompt = """I'll show you examples of 'high quality' vs 'low quality' product photos, then you'll evaluate a new photo.

HIGH QUALITY examples:
- Clean white background
- Sharp focus on product
- Even lighting, no harsh shadows
- Multiple angles shown

LOW QUALITY examples:
- Cluttered background
- Blurry or poor focus
- Harsh lighting or dark shadows
- Only one angle

Now evaluate this product photo and explain your rating."""

---

Performance Optimization

Model Selection

Choose the right model for your use case:

Guidelines:

• Prototype with Flash, upgrade to Pro if needed
• Use Flash-Lite for simple captioning/classification
• Use 2.0+ only when detection/segmentation needed
• Consider latency requirements

Parallel Processing

Process multiple images concurrently:

import concurrent.futures
from google import genai

def process_batch(image_paths, max_workers=10):
    """Process images in parallel."""
    client = genai.Client(api_key="YOUR_API_KEY")

    def process_one(path):
        with open(path, 'rb') as f:
            response = client.models.generate_content(
                model='gemini-2.5-flash',
                contents=[
                    types.Part.from_bytes(f.read(), mime_type='image/jpeg'),
                    'Caption this image.'
                ]
            )
        return {'path': path, 'result': response.text}

    with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor:
        results = list(executor.map(process_one, image_paths))

    return results

Caching Strategies

Cache results to avoid redundant API calls:

import hashlib
import json
from pathlib import Path

class GeminiCache:
    def __init__(self, cache_dir='.cache'):
        self.cache_dir = Path(cache_dir)
        self.cache_dir.mkdir(exist_ok=True)

    def _get_key(self, image_bytes, prompt, model):
        """Generate cache key from inputs."""
        content = image_bytes + prompt.encode() + model.encode()
        return hashlib.sha256(content).hexdigest()

    def get(self, image_bytes, prompt, model):
        """Get cached result if exists."""
        key = self._get_key(image_bytes, prompt, model)
        cache_file = self.cache_dir / f"{key}.json"

        if cache_file.exists():
            with open(cache_file, 'r') as f:
                return json.load(f)
        return None

    def set(self, image_bytes, prompt, model, result):
        """Save result to cache."""
        key = self._get_key(image_bytes, prompt, model)
        cache_file = self.cache_dir / f"{key}.json"

        with open(cache_file, 'w') as f:
            json.dump(result, f)

# Usage
cache = GeminiCache()

with open('image.jpg', 'rb') as f:
    image_bytes = f.read()

cached = cache.get(image_bytes, prompt, model)
if cached:
    print("Using cached result")
    result = cached
else:
    print("Calling API")
    result = analyze_image(image_bytes, prompt, model)
    cache.set(image_bytes, prompt, model, result)

---

Cost Optimization

Token Management

Minimize token usage:

1. Resize images to optimal dimensions (≤768px) 2. Use File API for repeated analysis (upload once, use many times) 3. Batch related questions in single request 4. Choose smaller models when appropriate

Calculate Costs Before Processing

def estimate_tokens(width, height):
    """Estimate token cost for image."""
    if width <= 384 and height <= 384:
        return 258

    crop_unit = min(width, height) // 1.5
    tiles_x = width / crop_unit
    tiles_y = height / crop_unit
    total_tiles = tiles_x * tiles_y

    return int(total_tiles * 258)

def estimate_cost(image_path, model='gemini-2.5-flash'):
    """Estimate API cost for image analysis."""
    from PIL import Image

    img = Image.open(image_path)
    tokens = estimate_tokens(img.width, img.height)

    # Add prompt tokens (estimate ~50)
    total_tokens = tokens + 50

    # Pricing (example, check current rates)
    rates = {
        'gemini-2.5-pro': 0.000125,  # per 1K tokens
        'gemini-2.5-flash': 0.0000375,
        'gemini-2.5-flash-lite': 0.00001875,
    }

    cost = (total_tokens / 1000) * rates.get(model, 0.0000375)
    return {
        'tokens': total_tokens,
        'cost_usd': round(cost, 6),
        'model': model
    }

# Use it
info = estimate_cost('large_image.jpg')
print(f"Estimated cost: ${info['cost_usd']} ({info['tokens']} tokens)")

Batch Processing Strategy

def smart_batch_process(image_paths, budget_usd=1.00):
    """Process images within budget."""
    total_cost = 0
    results = []

    for path in image_paths:
        est = estimate_cost(path)

        if total_cost + est['cost_usd'] > budget_usd:
            print(f"Budget exceeded. Processed {len(results)} images.")
            break

        # Process image
        result = process_image(path)
        results.append(result)
        total_cost += est['cost_usd']

    print(f"Total cost: ${total_cost:.4f}")
    return results

---

Error Handling

Comprehensive Error Handling

from google import genai
import time

class GeminiError(Exception):
    """Base exception for Gemini errors."""
    pass

class RateLimitError(GeminiError):
    """Rate limit exceeded."""
    pass

class InvalidAPIKeyError(GeminiError):
    """Invalid API key."""
    pass

def analyze_with_error_handling(image_bytes, prompt, model='gemini-2.5-flash', max_retries=3):
    """Robust image analysis with error handling."""
    client = genai.Client(api_key="YOUR_API_KEY")

    for attempt in range(max_retries):
        try:
            response = client.models.generate_content(
                model=model,
                contents=[
                    types.Part.from_bytes(data=image_bytes, mime_type='image/jpeg'),
                    prompt
                ]
            )
            return response.text

        except Exception as e:
            error_str = str(e)

            # Rate limit
            if '429' in error_str:
                if attempt < max_retries - 1:
                    wait_time = (2 ** attempt) * 2
                    print(f"Rate limited. Retrying in {wait_time}s...")
                    time.sleep(wait_time)
                    continue
                raise RateLimitError("Rate limit exceeded after retries")

            # Invalid API key
            elif '401' in error_str or '403' in error_str:
                raise InvalidAPIKeyError("Invalid or unauthorized API key")

            # Invalid request
            elif '400' in error_str:
                raise GeminiError(f"Invalid request: {error_str}")

            # Server error
            elif '500' in error_str or '503' in error_str:
                if attempt < max_retries - 1:
                    time.sleep(2)
                    continue
                raise GeminiError(f"Server error after retries: {error_str}")

            # Unknown error
            else:
                raise GeminiError(f"Unexpected error: {error_str}")

    raise GeminiError("Max retries exceeded")

---

File Management

Efficient File Upload Strategy

from google import genai
from datetime import datetime, timedelta

class FileManager:
    def __init__(self, api_key):
        self.client = genai.Client(api_key=api_key)
        self.uploaded_files = {}  # Track uploaded files

    def upload_if_needed(self, file_path, reuse_hours=24):
        """Upload file only if not already uploaded recently."""
        file_id = self.uploaded_files.get(file_path)

        if file_id:
            # Check if file still exists
            try:
                self.client.files.get(name=file_id)
                print(f"Reusing existing file: {file_id}")
                return file_id
            except:
                # File expired, remove from tracking
                del self.uploaded_files[file_path]

        # Upload new file
        uploaded = self.client.files.upload(file=file_path)
        self.uploaded_files[file_path] = uploaded.name
        print(f"Uploaded new file: {uploaded.name}")
        return uploaded.name

    def cleanup(self):
        """Delete all tracked files."""
        for file_path, file_id in self.uploaded_files.items():
            try:
                self.client.files.delete(name=file_id)
                print(f"Deleted: {file_id}")
            except:
                pass
        self.uploaded_files.clear()

# Usage
fm = FileManager(api_key="YOUR_API_KEY")

# Upload once
file_id = fm.upload_if_needed('large_image.jpg')

# Use multiple times
for prompt in ['Caption this', 'What colors?', 'Any text?']:
    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[types.File(name=file_id), prompt]
    )
    print(response.text)

# Cleanup when done
fm.cleanup()

---

Production Deployment

Monitoring & Logging

import logging
from datetime import datetime

# Setup logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('gemini_api.log'),
        logging.StreamHandler()
    ]
)

def log_api_call(image_path, prompt, model, tokens, duration, success, error=None):
    """Log API call for monitoring."""
    log_data = {
        'timestamp': datetime.now().isoformat(),
        'image': image_path,
        'prompt_length': len(prompt),
        'model': model,
        'tokens': tokens,
        'duration_ms': duration * 1000,
        'success': success,
        'error': str(error) if error else None
    }

    if success:
        logging.info(f"API call succeeded: {log_data}")
    else:
        logging.error(f"API call failed: {log_data}")

    return log_data

Rate Limiting

from collections import deque
from time import time, sleep

class RateLimiter:
    def __init__(self, max_requests_per_minute=15):
        self.max_rpm = max_requests_per_minute
        self.requests = deque()

    def wait_if_needed(self):
        """Wait if rate limit would be exceeded."""
        now = time()

        # Remove requests older than 1 minute
        while self.requests and self.requests[0] < now - 60:
            self.requests.popleft()

        # If at limit, wait
        if len(self.requests) >= self.max_rpm:
            sleep_time = 60 - (now - self.requests[0])
            if sleep_time > 0:
                print(f"Rate limit approaching. Waiting {sleep_time:.1f}s...")
                sleep(sleep_time)

        self.requests.append(now)

# Usage
limiter = RateLimiter(max_requests_per_minute=15)

for image_path in image_paths:
    limiter.wait_if_needed()
    result = process_image(image_path)

Health Checks

def health_check():
    """Verify API is accessible."""
    try:
        client = genai.Client(api_key="YOUR_API_KEY")

        # Simple test request
        response = client.models.generate_content(
            model='gemini-2.5-flash',
            contents=['Hello, are you there?']
        )

        return {
            'status': 'healthy',
            'latency_ms': None,  # Measure if needed
            'timestamp': datetime.now().isoformat()
        }
    except Exception as e:
        return {
            'status': 'unhealthy',
            'error': str(e),
            'timestamp': datetime.now().isoformat()
        }

---

Additional Resources

• Official Docs: https://ai.google.dev/gemini-api/docs
• Pricing: https://ai.google.dev/gemini-api/docs/pricing
• Rate Limits: https://ai.google.dev/gemini-api/docs/rate-limits
• Community Forum: https://discuss.ai.google.dev
• Cookbook: https://github.com/google-gemini/cookbook

Gemini Vision API - Code Examples

Comprehensive code examples for common use cases.

Table of Contents

1. Basic Image Analysis 2. Multi-Image Analysis 3. Object Detection 4. Segmentation 5. Document Processing 6. File Upload & Management 7. Advanced Techniques

---

Basic Image Analysis

Analyze Local Image (Python)

from google import genai
from google.genai import types

# Initialize client
client = genai.Client(api_key="YOUR_API_KEY")

# Read image file
with open('path/to/image.jpg', 'rb') as f:
    image_bytes = f.read()

# Generate response
response = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[
        types.Part.from_bytes(
            data=image_bytes,
            mime_type='image/jpeg'
        ),
        'What is in this image?'
    ]
)

print(response.text)

Analyze Image from URL (Python)

from google import genai
from google.genai import types
import requests

# Initialize client
client = genai.Client(api_key="YOUR_API_KEY")

# Download image
image_url = "https://example.com/image.jpg"
image_bytes = requests.get(image_url).content

# Generate response
response = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[
        types.Part.from_bytes(
            data=image_bytes,
            mime_type='image/jpeg'
        ),
        'Describe this image in detail.'
    ]
)

print(response.text)

Analyze Image (JavaScript/Node.js)

import { GoogleGenAI } from "@google/genai";
import * as fs from "node:fs";

// Initialize client
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

// Read image file as base64
const base64Image = fs.readFileSync("path/to/image.jpg", {
  encoding: "base64",
});

// Generate response
const response = await ai.models.generate({
  model: "gemini-2.5-flash",
  contents: [
    {
      parts: [
        {
          inline_data: {
            mime_type: "image/jpeg",
            data: base64Image,
          },
        },
        {
          text: "What is in this image?",
        },
      ],
    },
  ],
});

console.log(response.text);

Analyze Image (REST/curl)

# Base64 encode the image
IMAGE_BASE64=$(base64 -w 0 image.jpg)

# Make API request
curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent?key=$GEMINI_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "contents": [
      {
        "parts": [
          {
            "inline_data": {
              "mime_type": "image/jpeg",
              "data": "'"$IMAGE_BASE64"'"
            }
          },
          {
            "text": "What is in this image?"
          }
        ]
      }
    ]
  }'

---

Multi-Image Analysis

Compare Two Images (Python)

from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

# Read both images
with open('image1.jpg', 'rb') as f1:
    img1_bytes = f1.read()

with open('image2.jpg', 'rb') as f2:
    img2_bytes = f2.read()

# Generate response
response = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[
        'What are the differences between these two images?',
        types.Part.from_bytes(data=img1_bytes, mime_type='image/jpeg'),
        types.Part.from_bytes(data=img2_bytes, mime_type='image/jpeg')
    ]
)

print(response.text)

Analyze Multiple Images (JavaScript)

import { GoogleGenAI } from "@google/genai";
import * as fs from "node:fs";

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const images = [
  fs.readFileSync("img1.jpg", { encoding: "base64" }),
  fs.readFileSync("img2.jpg", { encoding: "base64" }),
  fs.readFileSync("img3.jpg", { encoding: "base64" }),
];

const parts = [
  { text: "What is the common theme across these images?" },
  ...images.map(img => ({
    inline_data: { mime_type: "image/jpeg", data: img }
  }))
];

const response = await ai.models.generate({
  model: "gemini-2.5-flash",
  contents: [{ parts }],
});

console.log(response.text);

---

Object Detection

Detect Objects with Bounding Boxes (Python)

from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

with open('street_scene.jpg', 'rb') as f:
    image_bytes = f.read()

# Request object detection
response = client.models.generate_content(
    model='gemini-2.0-flash',  # 2.0+ required for detection
    contents=[
        types.Part.from_bytes(data=image_bytes, mime_type='image/jpeg'),
        'Detect all objects in this image and provide bounding boxes.'
    ]
)

print(response.text)

Parse Bounding Boxes (Python)

import re
import json

def parse_bounding_boxes(response_text):
    """Parse bounding box coordinates from response."""
    # Gemini returns boxes in format: [ymin, xmin, ymax, xmax]
    # Coordinates are in range [0, 1000]

    boxes = []

    # Look for patterns like: car [100, 200, 300, 400]
    pattern = r'(\w+)\s*\[(\d+),\s*(\d+),\s*(\d+),\s*(\d+)\]'
    matches = re.findall(pattern, response_text)

    for match in matches:
        label, ymin, xmin, ymax, xmax = match
        boxes.append({
            'label': label,
            'bbox': {
                'ymin': int(ymin) / 1000,  # Normalize to [0, 1]
                'xmin': int(xmin) / 1000,
                'ymax': int(ymax) / 1000,
                'xmax': int(xmax) / 1000
            }
        })

    return boxes

# Use it
boxes = parse_bounding_boxes(response.text)
print(json.dumps(boxes, indent=2))

Draw Bounding Boxes (Python)

from PIL import Image, ImageDraw

def draw_boxes(image_path, boxes, output_path='output.jpg'):
    """Draw bounding boxes on image."""
    img = Image.open(image_path)
    draw = ImageDraw.Draw(img)

    width, height = img.size

    for box in boxes:
        bbox = box['bbox']
        label = box['label']

        # Convert normalized coords to pixel coords
        x1 = int(bbox['xmin'] * width)
        y1 = int(bbox['ymin'] * height)
        x2 = int(bbox['xmax'] * width)
        y2 = int(bbox['ymax'] * height)

        # Draw rectangle
        draw.rectangle([x1, y1, x2, y2], outline='red', width=3)
        draw.text((x1, y1 - 10), label, fill='red')

    img.save(output_path)
    print(f"Saved to {output_path}")

# Use it
draw_boxes('street_scene.jpg', boxes)

---

Segmentation

Generate Segmentation Masks (Python)

from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

with open('portrait.jpg', 'rb') as f:
    image_bytes = f.read()

# Request segmentation
response = client.models.generate_content(
    model='gemini-2.5-flash',  # 2.5+ required for segmentation
    contents=[
        types.Part.from_bytes(data=image_bytes, mime_type='image/jpeg'),
        'Segment the person in this image and provide a pixel mask.'
    ]
)

print(response.text)

---

Document Processing

Extract Text from PDF (Python)

from google import genai

client = genai.Client(api_key="YOUR_API_KEY")

# Upload PDF
uploaded_file = client.files.upload(file="document.pdf")

# Process document
response = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[
        uploaded_file,
        'Extract all text from this PDF document.'
    ]
)

print(response.text)

Analyze Charts in Document (Python)

from google import genai

client = genai.Client(api_key="YOUR_API_KEY")

uploaded_file = client.files.upload(file="report.pdf")

response = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[
        uploaded_file,
        'Analyze all charts and graphs in this document. Provide the data values.'
    ]
)

print(response.text)

---

File Upload & Management

Upload and Reuse File (Python)

from google import genai

client = genai.Client(api_key="YOUR_API_KEY")

# Upload file once
uploaded_file = client.files.upload(file="large_image.jpg")
print(f"Uploaded: {uploaded_file.name}")

# Use multiple times
response1 = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[uploaded_file, 'Describe this image.']
)

response2 = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[uploaded_file, 'What colors are prominent?']
)

# Clean up
client.files.delete(name=uploaded_file.name)

List and Clean Up Old Files (Python)

from google import genai
from datetime import datetime, timedelta

client = genai.Client(api_key="YOUR_API_KEY")

# List all files
files = client.files.list()

# Delete files older than 24 hours
cutoff = datetime.now() - timedelta(hours=24)

for file in files:
    # Parse create_time and check age
    # Files auto-delete after 48 hours anyway
    print(f"Found: {file.display_name} ({file.name})")

    # Delete specific file
    # client.files.delete(name=file.name)

---

Advanced Techniques

Structured JSON Output (Python)

from google import genai
from google.genai import types
import json

client = genai.Client(api_key="YOUR_API_KEY")

with open('product.jpg', 'rb') as f:
    image_bytes = f.read()

response = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[
        types.Part.from_bytes(data=image_bytes, mime_type='image/jpeg'),
        '''Analyze this product image and return JSON with this structure:
        {
          "product_name": "...",
          "category": "...",
          "colors": ["...", "..."],
          "description": "...",
          "features": ["...", "..."]
        }'''
    ]
)

# Parse JSON from response
data = json.loads(response.text)
print(json.dumps(data, indent=2))

Few-Shot Learning (Python)

from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

# Provide examples first
with open('example1.jpg', 'rb') as f:
    ex1_bytes = f.read()

with open('example2.jpg', 'rb') as f:
    ex2_bytes = f.read()

with open('query.jpg', 'rb') as f:
    query_bytes = f.read()

response = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=[
        'Example 1: Modern style',
        types.Part.from_bytes(data=ex1_bytes, mime_type='image/jpeg'),
        'Example 2: Vintage style',
        types.Part.from_bytes(data=ex2_bytes, mime_type='image/jpeg'),
        'What style is this image?',
        types.Part.from_bytes(data=query_bytes, mime_type='image/jpeg')
    ]
)

print(response.text)

Batch Processing (Python)

from google import genai
from google.genai import types
from pathlib import Path
import concurrent.futures

client = genai.Client(api_key="YOUR_API_KEY")

def process_image(image_path):
    """Process a single image."""
    with open(image_path, 'rb') as f:
        image_bytes = f.read()

    response = client.models.generate_content(
        model='gemini-2.5-flash',
        contents=[
            types.Part.from_bytes(data=image_bytes, mime_type='image/jpeg'),
            'Caption this image in one sentence.'
        ]
    )

    return {
        'path': str(image_path),
        'caption': response.text
    }

# Process multiple images in parallel
image_dir = Path('images/')
image_files = list(image_dir.glob('*.jpg'))

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    results = list(executor.map(process_image, image_files))

for result in results:
    print(f"{result['path']}: {result['caption']}")

Error Handling with Retry (Python)

from google import genai
from google.genai import types
import time

client = genai.Client(api_key="YOUR_API_KEY")

def analyze_with_retry(image_bytes, prompt, max_retries=3):
    """Analyze image with exponential backoff retry."""
    for attempt in range(max_retries):
        try:
            response = client.models.generate_content(
                model='gemini-2.5-flash',
                contents=[
                    types.Part.from_bytes(data=image_bytes, mime_type='image/jpeg'),
                    prompt
                ]
            )
            return response.text

        except Exception as e:
            if '429' in str(e):  # Rate limit
                wait_time = (2 ** attempt) * 1  # Exponential backoff
                print(f"Rate limited. Waiting {wait_time}s...")
                time.sleep(wait_time)
            else:
                raise e

    raise Exception("Max retries exceeded")

# Use it
with open('image.jpg', 'rb') as f:
    result = analyze_with_retry(f.read(), "What is this?")
    print(result)

---

Tips & Best Practices

Optimizing Token Usage

from PIL import Image

def resize_for_gemini(image_path, max_dimension=768):
    """Resize image to minimize token cost."""
    img = Image.open(image_path)

    # If both dimensions ≤ 384, cost is only 258 tokens
    if max(img.size) <= 384:
        return img

    # Otherwise resize to max_dimension to control tiling
    ratio = max_dimension / max(img.size)
    new_size = tuple(int(dim * ratio) for dim in img.size)

    return img.resize(new_size, Image.LANCZOS)

# Use it
img = resize_for_gemini('large_image.jpg')
img.save('optimized.jpg')

Prompt Engineering

# Good: Specific and clear
prompt = "List all visible objects in this image as a bullet-pointed list."

# Better: Specify format and level of detail
prompt = """Analyze this image and provide:
1. Main subject (1 sentence)
2. Background elements (bullet list)
3. Colors (list of 3-5 dominant colors)
4. Overall mood (1 word)

Format as JSON."""

# Best: Include examples if needed
prompt = """Identify the architectural style of this building.
Examples:
- Gothic: pointed arches, flying buttresses
- Modern: clean lines, glass facades
- Victorian: ornate details, asymmetrical

Your answer:"""

---

For more examples and tutorials, visit:

• https://ai.google.dev/gemini-api/docs/image-understanding
• https://github.com/google-gemini/cookbook