Production-Ready Dual-Engine Handshake: Overcoming downloading States with Gemini Nano & Cloud Fallback

[joseph brewerton]

Hey Chrome AI Team, I wanted to share a real-world project I just built for a local parish website using the new ai.languageModel specification via AI studio.  I am not sure if it is of any use but thought someone might be interested in the approach?

Because our users are often on spotty mobile networks or older devices, I engineered a 'Local-First' controller. If Gemini Nano is ready, it runs 100% locally to save us server costs and protect user privacy. But if it's still 'downloading', or if it hits a glitch mid-chat, my code instantly switches over to a live streaming fallback using Gemini 3.5 Flash on the cloud so the text keeps typing smoothly.

I’ve attached the TypeScript controller below under the MIT license if you want to use it for your official examples or case studies. Thanks for the awesome API! 

 /** * Copyright (c) 2026 Joseph Brewerton / St Joseph's Fishponds * Licensed under the MIT License. * Permission is hereby granted to copy, modify, and distribute this software * provided the above copyright notice remains intact. */

/**

 * Local-First Dual-Engine Handshake Controller for Parish Web Assistant

 *

 * This controller orchestrates high-performance client-side progressive enhancement,

 * silently checking for native on-device browser AI capabilities using the modern Chrome

 * `ai.languageModel` specification.

 *

 * Handshake Flow:

 * 1. Checks if the browser supports window.ai.languageModel.

 * 2. Querying capabilities via `ai.languageModel.availability()`.

 *    - "available" -> Default to Gemini Nano local-first running on-device.

 *    - "downloadable" or "downloading" -> Gracefully bypass to prevent heavy bandwidth

 *      usage or UI stalling. Fall back to cloud API proxy (Gemini 3.5 Flash).

 *    - "unavailable" -> Gracefully fall back to cloud API proxy (Gemini 3.5 Flash).

 *

 * 3. Client-Side RAG (Local Context Enveloping):

 *    Weaves dynamic parish metadata (liturgy, mass hours, policies) cleanly into the system prompt.

 * 4. Resilient Streaming:

 *    Streams tokens dynamically via `.promptStreaming()`. On mid-session network or model failures,

 *    transparently swaps execution to cloud stream proxy so the user never sees an unresolved error.

 */

import { AIService } from './aiService';

import { GoogleGenAI } from '@google/genai';

export interface ParishContext {

  liturgicalSeason: string;

  liturgicalColor: string;

  weeklyMassTimes: string;

  officeHours: string;

  activeSafeguardingPolicy: string;

}

export type EngineType = 'Gemini Nano' | 'Gemini 3.5 Flash';

export interface HandshakeResult {

  supported: boolean;

  status: 'available' | 'downloadable' | 'downloading' | 'unavailable' | 'unsupported';

  engine: EngineType;

  message: string;

}

export interface ChatMessage {

  role: 'user' | 'model';

  parts: { text: string }[];

}

export interface StreamCallbacks {

  /**

   * Called whenever a new token or text chunk becomes available.

   * Chunks are typically cumulative in Chrome's promptStreaming API, but this callback delivers

   * either the full cumulative stream or deltas based on your implementation choice.

   */

  onChunk: (text: string) => void;

  /** Called when the full stream completes successfully. */

  onSuccess: (finalText: string, engineUsed: EngineType) => void;

  /** Called when an irrecoverable error or fallback is completed. */

  onFallbackStart?: (error: Error, fallbackMessage: string) => void;

  /** Called if both local and cloud fallback fail completely. */

  onError: (error: Error) => void;

}

export class LocalFirstController {

  private parishContext: ParishContext;

  private currentSession: any = null;

  constructor(context: ParishContext) {

    this.parishContext = context;

  }

  /**

   * Updates the parish context dynamically when state changes.

   */

  public updateContext(newContext: Partial<ParishContext>): void {

    this.parishContext = {

      ...this.parishContext,

      ...newContext

    };

  }

  /**

   * Inspects browser capabilities to establish local-first handshake.

   * Leverages the modern Chrome high-performance `ai.languageModel` specification.

   */

  public async performHandshake(): Promise<HandshakeResult> {

    if (typeof window === 'undefined') {

      return {

        supported: false,

        status: 'unsupported',

        engine: 'Gemini 3.5 Flash',

        message: 'Non-browser execution environment detected.'

      };

    }

    const anyWin = window as any;

    const ai = anyWin.ai || (anyWin.chrome && anyWin.chrome.ai);

    if (!ai || !ai.languageModel) {

      return {

        supported: false,

        status: 'unsupported',

        engine: 'Gemini 3.5 Flash',

        message: 'Browser does not support the modern ai.languageModel specifications.'

      };

    }

    try {

      // Modern Chrome Prompt API spec uses availability() returning "available", "downloadable", or "no" / "unavailable"

      const availability = await ai.languageModel.availability();

     

      switch (availability) {

        case 'available':

          return {

            supported: true,

            status: 'available',

            engine: 'Gemini Nano',

            message: 'On-device Gemini Nano is fully downloaded, warm, and ready.'

          };

        case 'downloadable':

        case 'downloading':

          return {

            supported: false, // Mark supported as false to enforce fast bypass

            status: availability as 'downloadable' | 'downloading',

            engine: 'Gemini 3.5 Flash',

            message: `On-device engine is ${availability}. Bypassing to cloud to avoid blocking client connection.`

          };

        case 'unavailable':

        default:

          return {

            supported: false,

            status: 'unavailable',

            engine: 'Gemini 3.5 Flash',

            message: 'On-device capabilities are currently unavailable.'

          };

      }

    } catch (err: any) {

      console.warn('[LocalFirstController] Handshake capability check failed:', err);

      return {

        supported: false,

        status: 'unavailable',

        engine: 'Gemini 3.5 Flash',

        message: `Failure checking capabilities: ${err.message || String(err)}`

      };

    }

  }

  /**

   * Generates a concise, highly-grounded systemic instruction set

   * that respects the on-device model's narrow token context boundary.

   */

  private generateSystemPrompt(): string {

    const {

      liturgicalSeason,

      liturgicalColor,

      weeklyMassTimes,

      officeHours,

      activeSafeguardingPolicy

    } = this.parishContext;

    return `You are "Parish Porter", a helpful on-device AI assistant for St Joseph's Catholic Church in Fishponds, Bristol, UK.

Current Liturgical State: Season: ${liturgicalSeason || "Ordinary Time"}, Color: ${liturgicalColor || "Green"}.

Mass Schedule:

${weeklyMassTimes || "Check our website for the updated times."}

Office Hours: ${officeHours || "Contact the parish admin office."}

Safeguarding Policy Summary:

${activeSafeguardingPolicy || "St Joseph's strictly abides by diocesan safeguarding standards."}

Rules:

- Give warm, reverent answers.

- Base questions primarily on the local context provided above.

- If unsure or information is absent, direct users to email or call the Parish Office.

- Be extremely succinct; fit replies within 2-4 sentences max to conserve token bandwidth.

- Use clean Markdown text.`;

  }

  /**

   * Executes a user query with a local-first streaming paradigm, backing up

   * transparently to the cloud proxy server upon any failure.

   */

  public async executeQueryWithStreaming(

    userMessage: string,

    history: ChatMessage[],

    callbacks: StreamCallbacks

  ): Promise<void> {

    const handshake = await this.performHandshake();

    if (handshake.supported && handshake.engine === 'Gemini Nano') {

      try {

        console.log('[LocalFirstController] Initializing native Gemini Nano session...');

        const anyWin = window as any;

        const ai = anyWin.ai || (anyWin.chrome && anyWin.chrome.ai);

       

        const systemPrompt = this.generateSystemPrompt();

       

        // Initialize a clean session with context enveloping

        this.currentSession = await ai.languageModel.create({

          systemPrompt: systemPrompt

        });

        // Format conversational history explicitly for Nano's strict sequence requirements

        const formattedPrompt = this.formatConversationForNano(history, userMessage);

       

        console.log('[LocalFirstController] Dispatching streaming prompt locally...');

        const stream = await this.currentSession.promptStreaming(formattedPrompt);

       

        let lastFullText = '';

       

        // Iterate through chunks over modern stream reader / async iterate

        if (typeof stream[Symbol.asyncIterator] === 'function') {

          for await (const chunk of stream) {

            lastFullText = chunk;

            callbacks.onChunk(chunk);

          }

        } else {

          // Fallback if returned object is standard stream API

          const reader = stream.getReader();

          try {

            while (true) {

              const { done, value } = await reader.read();

              if (done) break;

              if (value) {

                lastFullText = value;

                callbacks.onChunk(value);

              }

            }

          } finally {

            reader.releaseLock();

          }

        }

        // Clean up the session safely

        this.destroySession();

       

        callbacks.onSuccess(lastFullText, 'Gemini Nano');

        return;

      } catch (localError: any) {

        console.error('[LocalFirstController] On-device session streaming threw exception:', localError);

       

        this.destroySession();

       

        if (callbacks.onFallbackStart) {

          callbacks.onFallbackStart(

            localError,

            'Transparent fallback triggered. Streaming from Cloud Engine...'

          );

        }

       

        // Fall back gracefully to server-side Gemini 3.5 Flash

        await this.executeCloudFallback(userMessage, history, callbacks);

      }

    } else {

      // Direct pass to cloud if handshake bypassed local-first engine

      console.log(`[LocalFirstController] Bypassing on-device (${handshake.status}). Initializing cloud stream...`);

      await this.executeCloudFallback(userMessage, history, callbacks);

    }

  }

  /**

   * Orchestrates the cloud secondary fallback stream to the server API

   */

  private async executeCloudFallback(

    userMessage: string,

    history: ChatMessage[],

    callbacks: StreamCallbacks

  ): Promise<void> {

    try {

      console.log('[LocalFirstController] Contacting server-side Gemini 3.5 Flash API...');

     

      const apiKey = process.env.GEMINI_API_KEY;

      if (!apiKey) {

        throw new Error("GEMINI_API_KEY is not defined. Cloud fallback is disabled.");

      }

      // Initialize the Gemini SDK

      const ai = new GoogleGenAI({ apiKey });

      // Map chat messages back to standard cloud API structure

      const cloudHistory = history.map(m => ({

        role: m.role === 'user' ? 'user' : 'model',

        parts: m.parts

      }));

      const siteUrl = typeof window !== 'undefined' ? window.location.origin : '';

      const modelName = AIService.getModelName('flash');

      // Generate the response stream using the SDK

      const responseStream = await ai.models.generateContentStream({

        model: modelName,

        contents: [

          ...cloudHistory,

          { role: 'user', parts: [{ text: userMessage }] }

        ],

        config: {

          systemInstruction: `You are the "Parish Porter", a helpful and warm AI assistant for St Joseph's Catholic Church.

          Your goal is to assist parishioners and visitors with questions about Mass times, liturgical schedules, sacraments, parish news, and safeguarding.

         

          Access the current site content at ${siteUrl} to provide the most up-to-date and accurate information.

         

          Guidelines:

          - Be welcoming, patient, and reverent.

          - Use information from the site's newsletters, mass times, and sacrament pages.

          - If a question is about a specific date, check the Liturgical Calendar if possible.

          - If you are unsure or the information isn't on the site, politely direct them to contact the Parish Office.

          - Keep responses concise but informative.

          - Use Markdown for formatting (bold, lists, etc.).`,

          tools: [{

            urlContext: {

              urls: [

                siteUrl,

                `${siteUrl}/mass-times`,

                `${siteUrl}/newsletters`,

                `${siteUrl}/sacraments`,

                `${siteUrl}/parish-news`,

                `${siteUrl}/volunteer`,

                `${siteUrl}/daily-readings`

              ]

            }

          }]

        }

      });

      let fullText = '';

      for await (const chunk of responseStream) {

        const text = chunk.text || '';

        fullText += text;

        callbacks.onChunk(text);

      }

      callbacks.onSuccess(fullText, 'Gemini 3.5 Flash');

    } catch (cloudErr: any) {

      console.error('[LocalFirstController] Cloud engine failure:', cloudErr);

      callbacks.onError(cloudErr);

    }

  }

  /**

   * Clean up session reference securely

   */

  private destroySession(): void {

    if (this.currentSession) {

      try {

        if (typeof this.currentSession.destroy === 'function') {

          this.currentSession.destroy();

        }

      } catch (e) {

        console.warn('[LocalFirstController] Session destruction failed:', e);

      }

      this.currentSession = null;

    }

  }

  /**

   * Helper: Merges history items into a clean linear dialog sequence for Gemini Nano prompt consumption

   */

  private formatConversationForNano(history: ChatMessage[], currentMessage: string): string {

    const buffer: string[] = [];

   

    // Select maximum 6 historical dialog exchanges to control prompt bloat

    const recentHistory = history.slice(-6);

   

    for (const msg of recentHistory) {

      const prefix = msg.role === 'user' ? 'User: ' : 'Porter: ';

      const text = msg.parts?.[0]?.text || '';

      if (text.trim()) {

        buffer.push(`${prefix}${text}`);

      }

    }

   

    buffer.push(`User: ${currentMessage}`);

    buffer.push(`Porter: `);

   

    return buffer.join('\n\n');

  }

}

  

[Thomas Steiner]

Hi Joseph,

This looks AI coded, as it uses the old window.ai APIs. You may be interested in Modern Web Guidance, which teaches your LLM the new APIs. (Please consider testing your code before sending it to the mailing list. This code never ran on any recent versions of Chrome.)

Cheers,

Tom

--
You received this message because you are subscribed to the Google Groups "Chrome Built-in AI Early Preview Program Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chrome-ai-dev-previe...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chrome-ai-dev-preview-discuss/de9a926a-1742-43ed-ae31-901809a4196fn%40chromium.org.

--

Thomas Steiner, PhD—Developer Relations Engineer (blog.tomayac.com, toot.cafe/@tomayac)

Google Spain, S.L.U.

Torre Picasso, Pl. Pablo Ruiz Picasso, 1, Tetuán, 28020 Madrid, Spain

CIF: B63272603

Inscrita en el Registro Mercantil de Madrid, sección 8, Hoja M­-435397 Tomo 24227 Folio 25

----- BEGIN PGP SIGNATURE -----

Version: GnuPG v2.4.8 (GNU/Linux)

iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck

0fjumBl3DCharaCTersAttH3b0ttom.xKcd.cOm/1181.

----- END PGP SIGNATURE -----

[joseph brewerton]

I have put that in the text I did not claim to write it just thought is might be interesting ?

[joseph brewerton]

I have attached a working screenshot from localhost of the updated code, and the code altered to work with the new instructions.  I assume this is now a working example on a modern Chrome browser.  

Joe

[joseph brewerton]

Just an update it is now running both local and web Gemini in one box