# ADR-0017: audio — canonical speech synthesis + transcription interfaces

**Status:** Accepted — 2026-07-11

## Context

mort is growing agent tools that speak (TTS) and transcribe audio through a
llama-swap host whose upstreams expose the OpenAI `/v1/audio/speech` and
`/v1/audio/transcriptions` endpoints (kokoro, chatterbox, whisper.cpp). Like
image generation before it (ADR-0016), speech shares none of the chat
contract's message/tool/stream machinery, and majordomo had no speech surface
— an earlier migration doc explicitly scoped transcription out of the llm
package. The same reasoning that produced `imagegen` applies.

## Decision

- One new canonical **leaf package `audio`** holding both directions —
  synthesis and transcription — rather than two packages; they are one
  modality and will share future types (voice metadata, audio formats).
  Root re-exports mirror imagegen (`SpeechModel`, `SpeechProvider`,
  `SpeechRequest`, `SpeechResult`, `TranscriptionModel`, ...).
- Minimal v1 surface, imagegen conventions throughout (zero value = backend
  default, functional options + `Apply`, `Raw any` escape hatch):
  - `SpeechRequest{ Input; Voice; Format; Speed }` →
    `SpeechResult{ Audio []byte; MIME string; Raw }`;
    `SpeechModel.Speak(ctx, req, ...opts)`;
    `SpeechProvider.SpeechModel(id, ...)`.
  - `TranscriptionRequest{ Audio []byte; MIME; Filename; Language; Prompt }` →
    `TranscriptionResult{ Text string; Raw }`;
    `TranscriptionModel.Transcribe(ctx, req, ...opts)`;
    `TranscriptionProvider.TranscriptionModel(id, ...)`.
- **Bytes in/out, never URLs** — mirrors `llm.ImagePart`'s bytes-only
  contract; fetching is the caller's concern.
- Providers are split (`SpeechProvider` vs `TranscriptionProvider`) so a
  backend can implement either half; llamaswap implements both.
- First implementation: `provider/llamaswap` — `/v1/audio/speech` (JSON body,
  raw audio response; MIME from Content-Type with a format-based fallback),
  `/v1/audio/transcriptions` (multipart, `response_format=json`), plus
  `ListVoices(ctx, model)` (GET `/v1/audio/voices?model=`, tolerant of the
  string-list and object-list shapes upstreams use) as a llamaswap management
  method, not part of the canonical contract.
- Out of scope for v1 (designed-for, deferred): streaming synthesis,
  word-level timestamps/segments, translation, voice cloning inputs, and
  registry-level DSN resolution for audio models.

## Consequences

- Speech is provider-agnostic from day one; an OpenAI or Google speech
  backend implements the same interfaces.
- `SpeechResult.Audio` is a plain byte slice, so results flow into any file
  store or attachment pipeline without a majordomo dependency.
- The `audio` package name is the modality, not the direction; if music/sfx
  generation ever lands it has a home.