Skip to content

Add AssemblyAI Universal Streaming API support and update changelog #70

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add AssemblyAI Universal Streaming API support and update changelog #70

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f827ee6
Update file
dev-docs-github-app-dev[bot] Jun 5, 2025
6046cde
Update file
dev-docs-github-app-dev[bot] Jun 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions fern/changelog/2025-04-15.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
1. **New GPT-4.1 Models Available**: You can now use `'gpt-4.1'`, `'gpt-4.1-mini'`, and `'gpt-4.1-nano'` as options for the `model` and `fallbackModels` with your [OpenAI models](https://api.vapi.ai/api#:~:text=OpenAIModel). These models may offer improved performance or features over previous versions.
---
title: API Enhancements - April 15, 2025
slug: changelog/2025-04-15
---

<Frame caption="New GPT-4.1 Models Available">
<img src="../static/images/changelog/gpt-4.1-models.png" alt="New GPT-4.1 Models Available" />
1. **AssemblyAI Universal Streaming API Support**: You can now configure AssemblyAI's Universal Streaming API to improve transcription accuracy and performance. Specify `AssemblyAITranscriber.useUniversalStreaming` to enable this enhanced streaming model for better real-time speech recognition.

2. **Transfer Tool Timeout Configuration**: Added timeout settings for call transfer operations. You can now configure `TransferTool.timeoutSeconds` to control how long the system waits during transfer attempts before timing out. This provides better control over call flow and prevents indefinite transfer delays.

3. **Analysis Plan Message Threshold**: Introduced `AnalysisPlan.messageThreshold` to control when call analysis is triggered based on the number of messages exchanged during a call. Set this threshold to ensure analysis only runs on substantive conversations, helping optimize processing resources and focus on meaningful interactions.

<Frame caption="Configure message threshold for call analysis">
<img src="../static/images/changelog/analysis-plan-message-threshold.png" alt="Analysis Plan Message Threshold" />
</Frame>

These enhancements provide better control over transcription quality, call transfer behavior, and analysis processing, allowing for more efficient and customizable voice AI implementations.
72 changes: 69 additions & 3 deletions fern/providers/transcriber/assembly-ai.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ subtitle: What is AssemblyAI?
slug: providers/transcriber/assembly-ai
---


**What is AssemblyAI?**

AssemblyAI is a leading provider of AI-driven speech recognition and understanding technologies. Their advanced models enable accurate transcription and analysis of audio data, facilitating applications across various industries.
Expand All @@ -29,6 +28,73 @@ AssemblyAI offers a comprehensive suite of AI-driven tools designed to meet dive

- AssemblyAI's real-time transcription feature enables sub-second latency conversion of speech to text, beneficial for live captioning, customer support, and interactive voice response systems, enhancing user experience and operational efficiency.

## Universal Streaming API Configuration

AssemblyAI now offers enhanced streaming capabilities through their Universal Streaming API. When using AssemblyAI as your transcriber with Vapi, you can enable this advanced streaming functionality to get even better performance and accuracy.

### Enabling Universal Streaming API

To enable the Universal Streaming API for your AssemblyAI transcriber, set the `enableUniversalStreamingApi` property to `true` in your assistant configuration:

```json
{
"transcriber": {
"provider": "assembly-ai",
"enableUniversalStreamingApi": true,
"model": "best",
"language": "en"
}
}
```

### Benefits of Universal Streaming API

When `enableUniversalStreamingApi` is enabled, you gain access to:

**Enhanced Accuracy**
- Improved transcription quality through AssemblyAI's latest Universal-2 model
- Better handling of diverse accents and speaking patterns
- Reduced word error rates across multiple languages

**Lower Latency**
- Optimized streaming pipeline for faster response times
- Reduced time-to-first-word in transcription results
- Better real-time conversation flow

**Advanced Features**
- Access to the latest AssemblyAI streaming features
- Improved punctuation and capitalization
- Better handling of background noise and audio quality variations

### Configuration Options

The Universal Streaming API works with all standard AssemblyAI transcriber options:

```json
{
"transcriber": {
"provider": "assembly-ai",
"enableUniversalStreamingApi": true,
"model": "best",
"language": "en",
"languageDetectionEnabled": true,
"keywords": ["product", "service", "support"]
}
}
```

### When to Use Universal Streaming API

Enable the Universal Streaming API when you need:
- **Maximum accuracy** for critical business conversations
- **Lowest possible latency** for real-time interactions
- **Multi-language support** with automatic language detection
- **Production-grade reliability** for high-volume applications

<Note>
The Universal Streaming API requires AssemblyAI API credentials to be configured in your Vapi dashboard under the "Transcriber Providers" section.
</Note>

**Use Cases for AssemblyAI**

AssemblyAI's versatile technology serves multiple industries, enhancing operations and delivering valuable insights:
Expand All @@ -39,7 +105,7 @@ AssemblyAI's versatile technology serves multiple industries, enhancing operatio

**Media And Content Creation**

- In the media sector, AssemblyAI’s speech-to-text solutions are used to transcribe interviews, podcasts, and video content. This makes content searchable, accessible, and easier to manage, enhancing the efficiency of media production workflows.
- In the media sector, AssemblyAI's speech-to-text solutions are used to transcribe interviews, podcasts, and video content. This makes content searchable, accessible, and easier to manage, enhancing the efficiency of media production workflows.

**Innovation and Research:**

Expand All @@ -51,4 +117,4 @@ AssemblyAI's versatile technology serves multiple industries, enhancing operatio

**Integrations and Compatibility**

- AssemblyAI offers a developer-friendly environment with RESTful API access, WebSocket support for real-time applications, SDKs for popular programming languages, detailed documentation and examples, ensuring seamless integration of speech recognition capabilities into existing systems.
- AssemblyAI offers a developer-friendly environment with RESTful API access, WebSocket support for real-time applications, SDKs for popular programming languages, detailed documentation and examples, ensuring seamless integration of speech recognition capabilities into existing systems.
Loading