Skip to content

fix(campplus): convert torch.Tensor to NumPy before passing to UMAP - #3911

Open
hansazilinon-art wants to merge 1 commit into
modelscope:mainfrom
hansazilinon-art:fix/cluster-backend-tensor-to-umap
Open

hansazilinon-art wants to merge 1 commit into
modelscope:mainfrom
hansazilinon-art:fix/cluster-backend-tensor-to-umap

Conversation

@hansazilinon-art

Copy link
Copy Markdown

Summary

ClusterBackend passes a torch.Tensor directly to UMAP. In UMAP 0.6.0, fit_transform inspects the input dtype and calls np.issubdtype(torch.float32, np.floating), which raises:

TypeError: Cannot interpret 'torch.float32' as a data type

This breaks CPU speaker diarization for recordings with at least 2,048 embeddings when no oracle_num is given.

Fixes #3908

Root cause

ClusterBackend.forward() in funasr/models/campplus/cluster_backend.py accepts embeddings from auto_model.py, which calls the backend with spk_embedding.cpu() (a tensor). All downstream clustering paths (spectral_cluster, kmeans_cluster, umap_hdbscan_cluster, merge_to_max_spks, merge_by_cos) expect a NumPy array.

Change

Normalize the input once at the start of forward():

X = X.detach().cpu().numpy() if isinstance(X, torch.Tensor) else np.asarray(X)

This is a no-op for callers that already pass NumPy arrays and only converts tensors, so it is safe for all existing call sites.

Testing

Reproduced with the synthetic 2,048 × 192 float32 embeddings from the issue report. With the fix, ClusterBackend.forward() returns (2048,) with labels [0, 1, 2, 3] instead of raising the TypeError. The NumPy input path is unchanged.

…UMAP

UMAP 0.6.0 calls np.issubdtype on the input dtype during fit_transform,
which raises TypeError when given a torch.Tensor directly. The fix ensures
the input is always a NumPy array by calling .detach().cpu().numpy() for
tensors, matching the type of embedding data from auto_model.forward().

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] ClusterBackend passes torch.Tensor to UMAP 0.6.0, causing a torch.float32 dtype TypeError

1 participant