Skip to content

fix(frontend): preserve per-item lengths in Whisper batches - #3918

Open
8079-debug wants to merge 1 commit into
modelscope:mainfrom
8079-debug:fix/whisper-batch-lengths
Open

8079-debug wants to merge 1 commit into
modelscope:mainfrom
8079-debug:fix/whisper-batch-lengths

Conversation

@8079-debug

Copy link
Copy Markdown

Summary

  • Fix WhisperFrontend.forward() passing input_lengths[0] for every utterance instead of input_lengths[i].
  • Add 32 regression cases using real Torch STFT and Whisper Mel filters, covering unequal lengths in both orders, equal lengths, single-item batches, padding/trimming on and off, both output layouts, and list/tensor lengths.
  • Keep this focused on a one-line production change and its regression tests; no model, dependency, or CI configuration changes.

Type of change

  • Bug fix

Reproduction

For input lengths [1601, 4321] and a hop length of 160, the expected feature lengths are [10, 27]. The existing code reports [10, 10] because it reuses the first utterance's length. Reversing the batch similarly reports [27, 27] instead of [27, 10].

Validation

Tests were executed locally on Windows x64 / Python 3.13.3 using CPU tensors, with torch==2.14.1+cu126, numpy==2.2.6, openai-whisper==20250625, and pytest==9.2.0.

Red/green regression evidence

The same new test file was run against both versions:

  • Unmodified frontend from upstream commit e227484d98ca90aa829a0ae180731c9ee2e87e77, copied into an isolated temporary tree: 16 failed, 16 passed. All failures were the per-item length assertions for unequal-length batches; equal-length and single-item controls passed.
  • Fixed frontend: 32 passed, 0 skipped.
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -q -ra tests/test_whisper_frontend_batch_lengths.py

The tests also check output shape, finite feature values, and agreement between each batch item and its standalone frontend result.

  • Targeted regression suite executed with optional dependencies present.
  • python -m compileall -q funasr examples tests
  • python -m black --check tests/test_whisper_frontend_batch_lengths.py
  • git diff --check

User impact

Mixed-length Whisper frontend batches return each utterance's own feature length rather than repeating the first item's length. The Mel feature computation and existing padding/trimming behavior are unchanged.

Notes for reviewers

The test loads the frontend directly and stubs only the FunASR import root/registration, avoiding unrelated model imports. Torch and Whisper operations are real; no model weights or external inference services are used. The test requires pytest, torch, and openai-whisper; it skips if the optional runtime packages are absent.

These are local frontend regression results, not end-to-end ASR accuracy measurements or GPU inference validation. No CI workflow was changed.

Use each utterance's input length instead of the first item's so mixed-length batches report correct feature lengths. Cover batch ordering, padding, layout, and list/tensor length inputs with real frontend regressions.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant