Skip to content

fix(campplus): convert tensors before UMAP clustering - #3909

Open
carey-bk wants to merge 1 commit into
modelscope:mainfrom
carey-bk:fix/3908-cluster-tensor-numpy
Open

carey-bk wants to merge 1 commit into
modelscope:mainfrom
carey-bk:fix/3908-cluster-tensor-numpy

Conversation

@carey-bk

Copy link
Copy Markdown
Contributor

Summary

With at least 2,048 speaker embeddings and no oracle_num, CPU float32 tensors reach UMAP 0.6.0 with a PyTorch dtype and fail with TypeError: Cannot interpret 'torch.float32' as a data type.

Convert Tensor inputs with detach().cpu().numpy() in the automatic large-input branch before UMAP and centroid merging. Add a seeded, single-threaded regression using 2,048 × 192 synthetic embeddings for ndarray, Tensor, and Tensor with gradients. It checks the four-speaker partition and input values.

Related to #3908. Reporter workload validation remains outstanding.

Type of change

  • Bug fix

Validation

On macOS arm64, Python 3.12.14, PyTorch 2.10.0, NumPy 2.4.6, SciPy 1.17.1, scikit-learn 1.9.1, UMAP 0.6.0, numba 0.68.1 and pynndescent 0.7.0, with NUMBA_NUM_THREADS=1 OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1:

  • Before the fix, the new regression gives 2 failures and 1 pass. The ordinary Tensor hits the reported dtype TypeError, the gradient Tensor fails conversion, and the ndarray passes.
  • python -m pytest -q tests/test_cluster_backend.py tests/test_numpy_cluster_compatibility.py — 59 passed.
  • Adding tests/test_numpy_compatibility.py — 67 passed, 1 failed. The failure is test_frontend_applies_cmvn_and_preserves_padding, with librosa 0.11.0 reporting Window size mismatch: 512 != 400. It also fails on the unmodified baseline in the same environment.
  • python -m compileall -q funasr examples tests
  • git diff --check

UMAP layout and HDBSCAN default-change warnings remain visible. The standalone CPU reproduction completes with 2,048 integer labels matching the four synthetic speakers.

User impact

Automatic speaker clustering can complete with CPU float32 Tensor inputs when it reaches the UMAP branch.

Notes for reviewers

Unverified inputs and environments include GPU, bfloat16, real recordings, pretrained model accuracy, other UMAP/NumPy versions, and the reporter's Python 3.11.15 / PyTorch 2.15.0 combination.

AI assistance disclosure: OpenAI Codex assisted with the implementation, regression tests, local validation, and PR description.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant