IBM Granite Speech 4.1: 2B ASR models for speed vs features

Model Variants and Features

IBM released three Granite Speech 4.1 models, each around 2 billion parameters and optimized for edge deployment. The base model (Granite Speech 4.1 2B) supports transcription in seven languages (English, French, German, Spanish, Portuguese, Japanese) plus bidirectional speech translation to/from English. It includes punctuation, truecasing, and keyword biasing, where users pass a list of names, acronyms, or terms in the prompt to bias recognition.

The Plus variant (Granite Speech 4.1 2B Plus) supports five languages (dropping Japanese and translation). It adds speaker-attributed ASR (diarization with Speaker 1, Speaker 2 labels) and word-level timestamps with reported accuracy beating customized Whisper versions. It supports incremental decoding by passing previous transcript text as a prefix for chunked long audio, maintaining consistent speaker numbering across overlaps.

The NAR model (Granite Speech 4.1 2B NAR) prioritizes throughput. It uses a non-autoregressive architecture: a frozen CTC encoder generates a draft transcript, then an LLM-based editor applies bidirectional attention for copy, insert, delete, or replace operations to refine it.

Performance Benchmarks

The base model tops the Hugging Face Open ASR Leaderboard with a 5.33% word error rate (95% word accuracy) across diverse tasks and a real-time factor (RTFx) of 231 (processes nearly 4 minutes of audio per second of compute, or 1 hour in 16 seconds).

The Plus model has a higher word error rate but excels in timestamp accuracy over WhisperX. The NAR model claims 1820x RTF on batched H100 GPUs (1 hour of audio in 2 seconds), with word error rate close to the base despite lacking diarization, timestamps, translation, and keyword biasing.

Implementation and Usage

All models load via Hugging Face Transformers with an AutoProcessor. For diarization, change the prompt to request speaker labels. Keyword biasing uses a prompt like one listing specific terms. Incremental decoding passes prior text as prefix.

The demo runs on an RTX Pro 6000 Blackwell GPU using PyTorch with CUDA 13 and custom-compiled Flash Attention (required for NAR; may fail on T4). Code example:

import torch
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
processor = AutoProcessor.from_pretrained("ibm/granite-speech-4.1-2b")
model = AutoModelForSpeechSeq2Seq.from_pretrained("ibm/granite-speech-4.1-2b", torch_dtype=torch.bfloat16, device_map="auto")
# Transcribe with options like generate_kwargs={'keywords': ['term1', 'term2']}

Granite GitHub provides speculative decoding examples and fine-tuning notebooks from prior versions, usable for domain-specific accents or podcasts (e.g., court transcripts).