Qwen Audio Agent: Decoupling Voice Conversation from Coding Tasks

Decoupled Architecture for Voice Coding

Qwen Audio Agent solves the blocking nature of traditional voice coding by implementing a dual-lane architecture. The system separates the voice runtime, which handles conversational state and interruptions, from the coding agent (such as Claude Code) that executes long-running tasks via the Agent Client Protocol (ACP). This design allows the voice layer to remain responsive to user queries or status checks while the backend agent processes refactoring or build tasks asynchronously. When a background task completes, the result is injected back into the active voice session without requiring the user to restart the interaction.

Interruption Handling and State Management

The system treats interruption as a state machine rather than a simple event. When the runtime detects user speech, it marks an interruption, emits an interrupt signal to suppress any audio or transcripts currently in flight, and immediately initiates a new turn. This approach minimizes the latency gap often found in voice assistants, where the system continues to output audio for a second or more after the user begins speaking.

Implementation and Limitations

The default configuration relies on Alibaba's DashScope API for optimized real-time voice performance, which requires a paid API key. While the project is open-source under the Apache 2.0 license, the documentation and default voice profiles are primarily in Chinese, requiring translation for non-native users. A fully local, offline mode is available for Apple Silicon users via an MLX backend and Hugging Face speech-to-text pipelines, though this path requires manual configuration and lacks the performance tuning of the cloud-based default.