Repository navigation
fix(campplus): convert torch.Tensor to NumPy before passing to UMAP - #3911
Open
hansazilinon-art wants to merge 1 commit into
Open
hansazilinon-art wants to merge 1 commit into
hansazilinon-art wants to merge 1 commit into
Conversation
…UMAP UMAP 0.6.0 calls np.issubdtype on the input dtype during fit_transform, which raises TypeError when given a torch.Tensor directly. The fix ensures the input is always a NumPy array by calling .detach().cpu().numpy() for tensors, matching the type of embedding data from auto_model.forward().
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ClusterBackendpasses atorch.Tensordirectly to UMAP. In UMAP 0.6.0,fit_transforminspects the input dtype and callsnp.issubdtype(torch.float32, np.floating), which raises:This breaks CPU speaker diarization for recordings with at least 2,048 embeddings when no
oracle_numis given.Fixes #3908
Root cause
ClusterBackend.forward()infunasr/models/campplus/cluster_backend.pyaccepts embeddings fromauto_model.py, which calls the backend withspk_embedding.cpu()(a tensor). All downstream clustering paths (spectral_cluster,kmeans_cluster,umap_hdbscan_cluster,merge_to_max_spks,merge_by_cos) expect a NumPy array.Change
Normalize the input once at the start of
forward():This is a no-op for callers that already pass NumPy arrays and only converts tensors, so it is safe for all existing call sites.
Testing
Reproduced with the synthetic 2,048 × 192 float32 embeddings from the issue report. With the fix,
ClusterBackend.forward()returns(2048,)with labels[0, 1, 2, 3]instead of raising the TypeError. The NumPy input path is unchanged.