GeminiLarge-context codeLevel 4: Multi-tool systems

How Do You Cut the Cost of Repeat Questions About a Large Codebase?

On the Gemini API, put the large code at the very start of each request and keep it identical, so you hit Gemini's implicit cache. Cached input tokens cost a fraction of the normal rate. This applies to the API only. In the app, repeat questions just use your plan's limits.

Prompt to copy
Keep the shared part first and identical in every request (Python, using the method names in Google's token docs; check the current SDK reference):

from google import genai

client = genai.Client()
codebase = open("[repo-dump.txt]").read()

for question in ["[question 1]", "[question 2]"]:
    interaction = client.interactions.create(
        model="gemini-3.8-flash",
        input=codebase + "\n\nBased on the code above: " + question,
    )
    print(interaction.usage.total_cached_tokens)
  • 3.8 Flash, full rate$0.75
  • 3.8 Flash, cached$0.075
  • Argon, full rate$2
  • Argon, cached$0.10
API price per million input tokens, at current rates. Source: Google's Gemini API pricing page, checked November 6, 2026.
  1. Dec 1, 2026Argon doubles$4 input, $0.20 cached
  2. Jan 1, 20273.8 Flash doubles$1.50 input, $0.15 cached
When those rates double. Source: Google's Gemini API pricing page, checked November 6, 2026.

Try it now

  1. 1Build one text file of your code, in a fixed order, so it comes out the same every time.
  2. 2Send two requests a short time apart, with that text first and only the question changing.
  3. 3Print total_cached_tokens from the usage object for each response.
  4. 4You'll know it worked when the second request shows a large cached count. A zero means your prompt starts differently, or the input is too small.

Use it when

When you ask many questions about the same large input in a short period: a code review tool, a script that checks each file against a style guide, or a chat over a big document.

Skip it when

One-off questions, or input under the 4,096 token minimum, where nothing gets cached. Don't count on it if your prompt starts with something that changes, such as a timestamp or the user's question, because that breaks the match from the first token.

Why it works

Google's caching docs say implicit caching is on by default for Gemini 2.5 and newer, with no setup. It needs a minimum input of 4,096 tokens on Gemini 4 Argon, 3.8 Flash, 3.6 Flash and 3.1 Pro Preview. Hits depend on a matching start to the prompt, so put big common content first and send similar requests within a short time. The price table shows the size of the saving. Gemini 3.8 Flash is $0.75 per million input tokens and $0.075 cached through December 31, 2026, then $1.50 and $0.15. Argon is $2 and $0.10 through November 30, then $4 and $0.20. The long context page says about 4x lower for Flash, which doesn't match that table, so go by the current price table.

Example

Arithmetic, not a Google figure, using 3.8 Flash rates through December 31. A 500,000 token code dump costs $0.375 in input per question at the full rate, and $0.0375 when fully cached. Ten questions come to $3.75 uncached and about $0.38 if all ten hit the cache, before output tokens. Cache hits aren't guaranteed, so measure your own total_cached_tokens. Explicit caching on the same model adds storage at $0.50 per million tokens per hour, or $0.25 an hour for this dump.

Going further

Explicit caching means creating and managing a cache object yourself. Google says it isn't supported in the Interactions API or for Gemini 4 Argon and later. Supported Gemini 3 and 2.5 models can use it through the generateContent API, so moving a pipeline to Argon can remove a cache you've built. Argon also has a rule for multi-turn chats: Gemini caches the full previous request only if the new one adds at least 2,050 tokens, counting output, thoughts and function results. Smaller additions re-process the last 2,050 tokens at the standard rate.

Common mistakes

  • Putting the question or a date at the top of the prompt, so no two requests start the same.
  • Budgeting on today's rate. Argon's prices double on December 1, 2026, and 3.8 Flash's on January 1, 2027.
  • Paying for explicit cache storage on a small or rarely used input. Storage bills by the hour whether you ask anything or not.
  • Reassembling the dump with files in a different order each time, which changes the start of the prompt.

Sources

Checked 6 November 2026. Products change often, so confirm details with the vendor.

Related

Beyond the course