> For the complete documentation index, see [llms.txt](https://docs.avis.xyz/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.avis.xyz/api-reference/endpoints/audio-generation.md). # Audio Generation The audio module exposes two distinct products under `/audio`, plus a shared voice catalog and a shared read surface (poll/pending/history — jobs from both products come back together, distinguished by the `product` field). | Product | Submit endpoint | Model(s) | | ------------------------- | ---------------------------------------------- | ------------------------------ | | Text-to-Speech | `POST /api/v1/audio/text-to-speech` | `seed-tts-1.0`, `seed-tts-2.0` | | Generation With Reference | `POST /api/v1/audio/generation-with-reference` | `seed-audio-1.0-multilingual` | Both are async, job-based like video generation: submit returns `202 Accepted` with `{ generationId }`; poll `GET /api/v1/audio/generations/:generationId` until `status` is terminal (`succeeded` or `failed`). ### When to Use Which * **Text-to-Speech** generates audio from a fixed catalog voice (`voiceType`, picked from `GET /audio/voices`) — plain text in, that voice reading it out. No reference audio, image, or cloning support; also adds delivery controls Generation With Reference doesn't have: `pitch`, `silenceDuration`, `enableLanguageDetector`/`explicitLanguage`, markdown/emoji filtering. * **Generation With Reference** supports more reference types: it can do plain catalog-voice narration too (set `references: [{ speaker: ... }]`, or omit `references` entirely for text-only), plus clone a voice from a reference audio clip (`audioUrl`/`audioData`, up to 3 clips), condition generation on a reference image, and return word-level subtitles (`enableSubtitle`) — none of which Text-to-Speech can do. Use Text-to-Speech for straightforward "read this text in a catalog voice" with fine delivery control; use Generation With Reference for anything needing voice cloning, image conditioning, or subtitles. ### Voice Catalog `GET /api/v1/audio/voices` Lists the shared TTS voice catalog. Text-to-Speech selects a voice directly via `voiceType`; Generation With Reference can reference a `seed-tts-2.0` voice as a `speaker` inside `references[]`. #### Query Parameters | Field | Type | Required | Allowed values | Description | | ------------ | ------ | -------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------ | | `gender` | string | No | `male`, `female` | Filter by voice gender. | | `age` | string | No | `child`, `teenager`, `adult`, `middle-age` | Filter by voice age group. | | `category` | string | No | `General`, `Dubbing`, `Advertisement`, `AudioBook`, `Conversational`, `Entertainment`, `SocialMedia` | Filter by use-case category. | | `language` | string | No | BCP-47 code, e.g. `en-US` | Filter to voices supporting a specific language. | | `resourceId` | string | No | `seed-tts-1.0`, `seed-tts-2.0` | Filter to one TTS model version. | #### Response Fields | Field | Type | Description | | ------------------------------------ | --------- | ----------------------------------------------------------------------------------------------------------------------- | | `voices` | object\[] | Array of voice objects (see below). | | `total` | number | Total number of matching voices. | | `voices[].voiceType` | string | Voice identifier. Pass as `voiceType` in Text-to-Speech, or as `speaker` in Generation With Reference's `references[]`. | | `voices[].name` | string | Display name. | | `voices[].gender` | string | `male` or `female`. | | `voices[].age` | string | `child`, `teenager`, `adult`, or `middle-age`. | | `voices[].categories` | string\[] | Use-case categories. | | `voices[].languages` | object\[] | Supported languages: `{ language, text, flag }` (BCP-47 tag, sample text, flag emoji). | | `voices[].avatar` | string | Avatar image URL. | | `voices[].trialUrl` | string | Demo audio URL (mp3). | | `voices[].description` | string | Voice description. | | `voices[].resourceId` | string | `seed-tts-1.0` or `seed-tts-2.0`. | | `voices[].capabilities.contextTexts` | boolean | Whether this voice supports `contextTexts` (natural-language style prompt). `true` only for `seed-tts-2.0` voices. | ### Text-to-Speech `POST /api/v1/audio/text-to-speech` Synthesizes plain text with a catalog voice. Model (`seed-tts-1.0` vs `seed-tts-2.0`) is implied by the chosen `voiceType` — TTS 2.0 voices end in `_uranus_bigtts`. #### Request Fields | Field | Type | Required | Validation | Description | | ------------------------ | --------- | -------- | ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `voiceType` | string | Yes | Non-empty | Voice identifier. Use `GET /audio/voices` to list valid values. | | `text` | string | Yes | 1–1000 characters | Plain text to synthesize. SSML is not supported. | | `format` | string | No | `mp3`, `ogg_opus`, `pcm` | Output audio format. Defaults to `mp3`. `wav` is not offered — streamed WAV chunks each carry their own header, so the concatenated buffer would not be a valid WAV file; use `pcm` if you need uncompressed audio. | | `sampleRate` | integer | No | `8000`, `16000`, `22050`, `24000`, `32000`, `44100`, `48000` | Sampling rate in Hz. Defaults to `24000`. | | `speechRate` | integer | No | `-50` to `100`, default `0` | Speech speed. `0` = normal, `100` = 2× speed, `-50` = 0.5× speed. **Only takes effect when `pitch` is non-zero** — this is upstream (BytePlus) behavior; sending `speechRate` alone is accepted (`202`) but silently has no effect. | | `loudnessRate` | integer | No | `-50` to `100`, default `0` | Volume. `0` = normal, `100` = 2× volume, `-50` = 0.5× volume. | | `pitch` | integer | No | `-12` to `12`, default `0` | Pitch shift in semitones. Must be non-zero for `speechRate` to take effect. | | `silenceDuration` | integer | No | `0`–`30000`, default `0` | Silence in ms appended to the end of the audio. Applied to the last sentence only — useful for padding before concatenating multiple clips. | | `enableLanguageDetector` | boolean | No | default `false` | Auto-detect languages within mixed-language text, to avoid mispronouncing foreign words. | | `explicitLanguage` | string | No | `zh`, `zh-cn`, `en`, `ja`, `es`, `es-mx`, `id`, `pt-br`, `ko`, `it`, `de`, `fr` | Force a specific output language, overriding auto-detection. | | `disableMarkdownFilter` | boolean | No | default `false` | `false` (default): markdown is read literally (`**Hello**` → "star Hello star"). `true`: markdown is parsed and stripped (`**Hello**` → "Hello"). | | `disableEmojiFilter` | boolean | No | default `false` | `false` (default): emojis are stripped before synthesis. `true`: emojis are included. | | `contextTexts` | string\[] | No | Max 1 item. **TTS 2.0 only** | Natural-language style prompt to steer vocal style, e.g. `["Speak a little slower"]`. Only the first item is used; not billed. Rejected with `400` when `voiceType` is a TTS 1.0 voice. | #### Request Example ```json { "voiceType": "en_female_dacey_uranus_bigtts", "text": "Hello, welcome to our platform. We are glad to have you here.", "format": "mp3", "sampleRate": 24000, "loudnessRate": 10, "contextTexts": ["Speak warmly and at a relaxed pace"] } ``` Response — `202 Accepted`: ```json { "generationId": "gen_..." } ``` Poll `GET /api/v1/audio/generations/:generationId` (see Poll Job Status) for the result. ### Generation With Reference `POST /api/v1/audio/generation-with-reference` Generates audio from a natural-language prompt, optionally conditioned on a reference speaker voice, reference audio clip(s), or a reference image. #### Request Fields | Field | Type | Required | Validation | Description | | ---------------- | --------- | -------- | --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `model` | string | Yes | Must be `seed-audio-1.0-multilingual` | Generation-with-reference model to use. | | `textPrompt` | string | Yes | 1–3000 characters | Prompt describing the desired audio, or the text to synthesize. Reference audio items by position using `@Audio1`, `@Audio2`, `@Audio3`. | | `references` | object\[] | No | Max 3 items | Reference resources — see Reference Items below. Omit entirely for text-only generation. | | `format` | string | No | `mp3`, `wav`, `ogg_opus`, `pcm` | Output audio format. Defaults to `mp3`. Unlike Text-to-Speech, `wav` is supported here — this endpoint returns a single non-streaming buffer, so there's no chunk-header concatenation issue. | | `sampleRate` | integer | No | `8000`, `16000`, `24000`, `32000`, `44100`, `48000` | Sampling rate in Hz. | | `speechRate` | integer | No | `-50` to `100`, default `0` | Speech speed adjustment. | | `loudnessRate` | integer | No | `-50` to `100`, default `0` | Volume adjustment. | | `pitchRate` | integer | No | `-12` to `12`, default `0` | Pitch shift in semitones. | | `enableSubtitle` | boolean | No | default `false` | Enable word-level subtitles in the response. | #### Reference Items `references[]` accepts up to 3 items (enforced client-side). Each item should set exactly one field — this shape is **not validated client-side**; a malformed item (zero or multiple fields set, an unsupported mix) is rejected by BytePlus itself rather than a local `400`. | Field | Description | | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | | `speaker` | Speaker ID — a `seed-tts-2.0` voice from `GET /audio/voices`, or a cloned voice. Multiple `speaker` items are supported (e.g. multi-speaker dialogue). | | `audioUrl` | Remote audio URL. Reference by position in `textPrompt` as `@Audio1`, `@Audio2`, etc. | | `audioData` | Base64-encoded audio. Referenced the same way. | | `imageUrl` | Reference image URL (jpeg/png/webp). Per BytePlus, image references cannot be mixed with audio references. | | `imageData` | Base64-encoded reference image. Same restriction as `imageUrl`. | #### Request Examples Text-only generation: ```json { "model": "seed-audio-1.0-multilingual", "textPrompt": "A warm audiobook narrator reading: The sun was setting over the mountain ridge.", "format": "mp3" } ``` With a speaker voice: ```json { "model": "seed-audio-1.0-multilingual", "textPrompt": "Read this cheerfully: Welcome back! We have some exciting updates for you.", "references": [{ "speaker": "en_female_dacey_uranus_bigtts" }], "format": "mp3", "sampleRate": 24000 } ``` With reference audio: ```json { "model": "seed-audio-1.0-multilingual", "textPrompt": "In the style of @Audio1, read: The journey begins at dawn.", "references": [{ "audioUrl": "https://example.com/reference-voice.mp3" }], "format": "mp3" } ``` With subtitles: ```json { "model": "seed-audio-1.0-multilingual", "textPrompt": "Narrate this slowly and clearly: Chapter one. A new beginning.", "enableSubtitle": true, "format": "mp3" } ``` Response — `202 Accepted`: ```json { "generationId": "gen_..." } ``` ### Poll Job Status `GET /api/v1/audio/generations/:generationId` Shared across both products — use the `product` field to tell them apart. | Field | Type | Description | | ------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------- | | `generationId` | string | Generation ID. | | `product` | string | `text-to-speech` or `generation-with-reference`. | | `status` | string | `queued`, `processing`, `succeeded`, or `failed`. | | `audioUrl` | string | Signed audio download URL. Present when `succeeded`. | | `durationSeconds` | number | Post-processed (playable) audio duration in seconds. Generation-with-reference only. | | `originalDurationSeconds` | number | Model's original output duration — the billed duration. Generation-with-reference only. | | `providerUrl` | string | BytePlus's temporary audio URL (valid \~2h). Use `audioUrl` for the persisted copy. Generation-with-reference only. | | `subtitle` | object | Present when subtitles were requested. Generation-with-reference only — see below. | | `error` | string | Error message. Present when `status` is `failed`. | `subtitle` shape (present only when `enableSubtitle: true` was set on the request): | Field | Type | Description | | ------------------------------ | --------- | -------------------------------------------------------------------------------------------------------- | | `subtitle.text` | string | Full transcript text. | | `subtitle.sentences[]` | object\[] | Sentence-level segments: `{ text, startTime, endTime, words }`, times in ms from the start of the audio. | | `subtitle.sentences[].words[]` | object\[] | Word-level segments: `{ text, startTime, endTime }`. | Example response (Generation With Reference, succeeded, subtitles enabled): ```json { "generationId": "gen_6a6...", "product": "generation-with-reference", "status": "succeeded", "audioUrl": "https://cdn.example.com/assets/6a6.../audio.mp3", "durationSeconds": 12.34, "originalDurationSeconds": 12.34, "subtitle": { "text": "Hello. Stay close.", "sentences": [ { "text": "Hello.", "startTime": 0, "endTime": 1800, "words": [ { "text": "Hello", "startTime": 0, "endTime": 600 }, { "text": ".", "startTime": 600, "endTime": 1800 } ] } ] } } ``` Poll on an interval until `status` is `succeeded` or `failed`. ### Validation Notes * `contextTexts` on a Text-to-Speech request returns `400` when the selected `voiceType` is a TTS 1.0 voice. * `references[]` on a Generation With Reference request: capped at 3 items client-side (`400` above that); per-item field shape (exactly one field, image/audio mixing) is not validated client-side and is instead rejected by BytePlus. * An unknown `speaker` (not a `seed-tts-2.0` catalog voice) returns `400`. * Insufficient credits return `403`. * An unknown `voiceType` returns `400`. * SSML markup is not supported by the underlying provider on either product — plain text only. ### Discovering Models & Capabilities `GET /api/v1/ai/models?outputModalities=audio` filters the model catalog down to the audio models: `seed-tts-1.0`, `seed-tts-2.0` (Text-to-Speech), and `seed-audio-1.0-multilingual` (Generation With Reference). Each model's `capabilities.audio` object tells a client where to submit and where to fetch its voice catalog, so a model picker can be driven from this endpoint instead of hardcoding model IDs or submit routes. ``` GET /api/v1/ai/models?outputModalities=audio ``` The catalog `modelId` is the same value submitted elsewhere: it's the `model` field in `POST /audio/generation-with-reference`. Text-to-Speech doesn't take a `model` field at all — its model is implied by the chosen `voiceType`. #### `capabilities.audio` Fields | Field | Type | Description | | ------------------------ | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `endpoint` | string | Submit route (relative to `/api/v1`) for this model: `/audio/text-to-speech` or `/audio/generation-with-reference`. | | `voiceSource.endpoint` | string | Always `/audio/voices` — where to fetch the voice/speaker catalog. | | `voiceSource.resourceId` | string | `resourceId` to pass as `GET /audio/voices?resourceId=...` for this model. TTS models point at their own version (`seed-tts-1.0` / `seed-tts-2.0`); Generation With Reference points at `seed-tts-2.0`, since that's what its `speaker` reference field accepts. | | `modes` | string\[] | Generation modes auto-selected by which reference is supplied: `["text-only", "reference-audio", "reference-image"]`. Generation With Reference only; absent for TTS (single implicit mode). | | `languages` | string\[] | BCP-47 codes the model generates in. Generation With Reference only — absent for TTS models, whose supported languages are per-voice (see the voice catalog's `languages` field instead). | | `params` | object | Per-field bounds (enum values / numeric ranges), keyed by request field name — a machine-readable mirror of the Request Fields tables above. | #### Example Response ```json { "data": [ { "modelId": "seed-tts-2.0", "name": "Text To Speech 2.0", "providerIds": ["byteplus"], "inputModalities": ["text"], "outputModalities": ["audio"], "isActive": true, "capabilities": { "audio": { "endpoint": "/audio/text-to-speech", "voiceSource": { "endpoint": "/audio/voices", "resourceId": "seed-tts-2.0" }, "params": { "format": { "type": "enum", "values": ["mp3", "ogg_opus", "pcm"], "default": "mp3" }, "speechRate": { "type": "range", "min": -50, "max": 100, "default": 0 }, "pitch": { "type": "range", "min": -12, "max": 12, "default": 0 } } } } }, { "modelId": "seed-audio-1.0-multilingual", "name": "Dola-Seed-Audio 1.0", "providerIds": ["byteplus"], "inputModalities": ["text", "audio", "image"], "outputModalities": ["audio"], "isActive": true, "capabilities": { "audio": { "endpoint": "/audio/generation-with-reference", "voiceSource": { "endpoint": "/audio/voices", "resourceId": "seed-tts-2.0" }, "modes": ["text-only", "reference-audio", "reference-image"], "languages": [ "en", "zh", "ja", "ko", "es-MX", "id", "de", "pt-BR", "fr", "th", "vi", "ms", "fil", "it", "ru", "nl", "pl", "tr", "sv", "es" ], "params": { "format": { "type": "enum", "values": ["mp3", "wav", "ogg_opus", "pcm"], "default": "mp3" }, "sampleRate": { "type": "enum", "values": ["8000", "16000", "24000", "32000", "44100", "48000"], "default": "24000" }, "enableSubtitle": { "type": "bool", "default": false } } } } } ], "success": true, "status": 200, "timestamp": "2026-07-22T10:00:00.000Z" } ``` See `capabilities.params` Value Shapes for how each `params` entry's `type` (`enum`, `range`, `int`, `bool`, ...) determines its extra fields. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.avis.xyz/api-reference/endpoints/audio-generation.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.