Prompt Engineering

6. Gemini 3.1 Pro Thinking Levels: Cut API Costs 50-70% | Intermediate HIGH thinking is the default, billing at $12/M tokens. Most requests don't need it.

1. Route complexity: LOW (1,024 tokens) for classification/summarization, MEDIUM (8,192) for code review, HIGH (32,768) for complex debugging
2. Use thinking_config=types.ThinkingConfig(thinking_level="MEDIUM") in Google SDK
3. Never mix thinkingLevel and thinkingBudget — returns 400 error
4. Use gemini-3.1-pro-preview-customtools variant for agent workflows ($2/$12 per M tokens)
5. Monitor thinking tokens separately — HIGH can cost 63x more than LOW per request Source: Apiyi Guide

7. Optimize Prompt Caching by Provider | Advanced Full-context caching paradoxically INCREASES latency by 8.8% on GPT-4o. System-prompt-only is optimal.

1. Cache stable system prompt and tool definitions only — exclude dynamic tool results and conversation history
2. For Anthropic: explicit cache breakpoints after system prompt, 5-minute TTL (90% read savings)
3. Keep timestamps, user IDs, session data OUTSIDE cacheable prefix
4. For OpenAI: stable content first, dynamic last. DON'T cache full conversation context
5. Fixed function definitions — each tool definition change breaks the entire cached prefix Source: arXiv 2601.06007