The Architecture Decision That Made My Voice Agent App Surprisingly Easy to Migrate

I Migrated a Live Voice AI App in 4 Files; Here's Why the Architecture Made That Possible

The migration was supposed to be hard. Swapping out the real-time AI model powering a live voice interview platform mid-build — different protocol, different event model, different audio format, different tool-calling flow — that should be at least a week of rework.

It took me four files. One of them was a single line change.

I want to talk about why, because the reason isn't that the APIs are similar. They're not. The reason is a design decision I made early in the project that I wasn't even sure was worth the effort at the time.

What Truefit Is

Truefit is a real-time AI interview platform. You spin up an audio-video session, and an AI agent conducts the interview — asking questions tailored to the role, listening to the candidate's answers, handling follow-ups, and generating a structured hiring recommendation at the end.

The system is built on three core pieces: a WebRTC client in the browser, a backend that handles WebRTC signaling and WebSocket orchestration, and a real-time AI service that's doing the actual listening, responding, and evaluating. The original implementation used Google's Gemini Live (gemini-2.5-flash-native-audio-preview). The migration target is OpenAI Realtime (gpt-4o-realtime).

The honest reason for migrating: Gemini Live is still a preview model. It's unstable, subject to breaking changes, and not on a committed production roadmap. OpenAI Realtime is GA — versioned, stable, and significantly more battle-tested in the ecosystem. For a portfolio project I want to actually demonstrate, that matters.

The Design Decision That Made Everything Easy

Early in the build I adopted the ports and adapters pattern (also called hexagonal architecture) for how the different layers of Truefit talk to each other.

The idea is simple: you define an abstract interface — a "port" — that describes what you need a module to do, without specifying how it does it. Then any concrete implementation of that behavior is an "adapter" that satisfies the port's contract. The rest of the system only ever talks to the port, never to the adapter directly.

In Truefit, this looks like:

GeminiLiveAdapter          ← only file that imports google.genai
      ↑
LiveSessionPort            ← abstract interface (ABC) — open_session, send_audio,
      ↓                      send_image, send_tool_response, receive, close
LiveInterviewAgent         ← domain logic, calls port methods only
      ↓
InterviewConnection        ← WebSocket handler, knows nothing about Gemini or OpenAI
      ↓
AudioBridge                ← audio routing, knows nothing about the AI model

LiveInterviewAgent — which contains all the interview logic, orchestration, turn management, and tool handling — never imports GeminiLiveAdapter. It receives it as a constructor argument typed against LiveSessionPort. So does InterviewConnection. So does the factory function that wires everything together.

The result is that GeminiLiveAdapter is literally the only file in the entire codebase that touches the Gemini SDK. And when I need to switch providers, I'm not untangling dependencies across the system — I'm just writing a new adapter that satisfies the same interface.

What Actually Changed

Here's the full migration surface:

| File | Change |

|------|------|

Everything else — LiveInterviewAgent, InterviewConnection, AudioBridge logic, _AgentAudioTrack, all domain models, all frontend code — is completely unchanged. Zero modifications.

That's the ports-and-adapters pattern paying off in real time.

The Technical Diffs That Actually Mattered

The APIs are genuinely different under the hood. The adapter's job is to absorb those differences so nothing above it has to care. Two differences in particular were interesting to solve.

Audio Format

Gemini Live expects 16kHz mono PCM16 audio as input. OpenAI Realtime expects 24kHz. Both output 24kHz.

This sounds like a plumbing problem, but it's actually a one-line fix — because AudioBridge was written with a named constant:

_SAMPLE_RATE = 16_000  # becomes 24_000

The resampler that converts 48kHz WebRTC audio down to model input rate is parameterized against this constant. Change the constant, the whole pipeline adjusts. This is the kind of thing that feels over-engineered when you write it and exactly right when you need it.

OpenAI also encodes audio as base64 over WebSocket rather than raw bytes. That's absorbed entirely inside the adapter — the rest of the system still passes bytes around.

Tool Calling (The Most Significant Protocol Difference)

When the AI agent calls one of Truefit's tools — say, record_question or complete_interview — the two providers handle the response flow very differently.

Gemini: You send the result back in a single call. Done.

await self._session.send_tool_response(
    function_responses=types.FunctionResponse(
        name=name,
        response=result,
        id=call_id,
    )
)

OpenAI: It's a two-step flow. First you create a conversation item with the tool result. Then you send a separate response.create event to explicitly tell the model to continue from there.

# Step 1: deliver the result
await self._send({
    "type": "conversation.item.create",
    "item": {
        "type": "function_call_output",
        "call_id": call_id,
        "output": json.dumps(result),
    },
})

# Step 2: tell the model to resume
await self._send({"type": "response.create"})

The response.create step is easy to forget. If you omit it, the model just silently hangs — it has the result but it's waiting for the trigger to continue. This is the kind of protocol detail that costs hours if you miss it and five seconds if you know about it.

Because both adapters implement the same send_tool_response(*, call_id, name, result) signature from LiveSessionPort, LiveInterviewAgent calls it identically regardless of provider. The two-step OpenAI flow is invisible to everything above the adapter.

What We Also Built: Automatic Fallback

One thing that came out of this migration naturally was a FallbackLiveAdapter — a third adapter that wraps both the Gemini and OpenAI adapters behind a single interface.

When a session opens, it tries the primary provider first. If that fails — bad API key, network issue, model unavailable, timeout — it falls back to the secondary, automatically, without the rest of the system knowing anything changed. If both fail, it surfaces a clean error.

def get_live_adapter() -> LiveSessionPort:
    return create_live_adapter()
    # resolves to one of:
    # GeminiLiveAdapter                      (primary=gemini, fallback=none)
    # OpenAIRealtimeAdapter                  (primary=openai, fallback=none)
    # FallbackLiveAdapter(Gemini → OpenAI)   (primary=gemini, fallback=openai)
    # FallbackLiveAdapter(OpenAI → Gemini)   (primary=openai, fallback=gemini)

The fact that this was even possible — and that the agent layer required zero changes to support it — is the cleanest validation of the design. The port abstraction isn't just useful for swapping providers during a migration. It makes the system structurally extensible in ways you didn't have to plan for specifically.

What I'd Take Forward

A few things I'd carry into any future real-time AI integration:

Design for replaceability from the start. Provider APIs are unstable. Models get deprecated. New providers launch. If your domain logic is coupled to a specific SDK, you're one deprecation notice away from a painful refactor. If it's coupled to an abstract interface you control, you're one new adapter file away from being unblocked.

Named constants for format-sensitive values. Sample rate, audio format, timeout windows — anything that's tied to an external spec and might change. One constant, one place. The _SAMPLE_RATE change being a single line is the payoff for that discipline.

Read the protocol, not just the SDK. The OpenAI Realtime adapter talks raw WebSocket — no SDK abstraction. That's more code, but it also means I understand exactly what's happening on the wire. When something breaks, there's no SDK behavior to wonder about. And the two-step tool call flow is the kind of thing the SDK might silently handle for you — which means you might not know it exists until you hit it in production.

The interface is the long-term investment. LiveSessionPort is a handful of abstract methods. That's what I'll still have when providers 3, 4, and 5 come along.

Truefit is an ongoing project. This post documents decisions made during the Gemini -> OpenAI Realtime migration in May 2026.