You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix: default asym_w4a8_int8 group_size to 16 like ComfyUI - #2388
ComfyUI asym_w4a8_int8 exports can leave group_size out of .comfy_quant (all 200 entries in the MiniMax-H3 W4A8 file do). ComfyUI falls back to 16 in that case (comfy/ops.py: layer_conf.get("group_size", params_conf.get("group_size", 16))), but read_comfy_quant_config reads it as 0, and the packed INT4 check only accepts 16, so those files always fail with invalid packed INT4 scales/codebook.
This reads group_size with a default of 16 for asym_w4a8_int8, same pattern as the convrot_groupsize default of 256 right above. An explicit group_size in the file is still used as before.
Verification (local only, nothing added to the repo): I built safetensors_io.cpp against ggml-base with a tiny harness calling read_safetensors_file on synthetic files with one asym_w4a8_int8 module (I8 weight [4, 256], codebook [16], s_channel [4], s_rel F8_E4M3 [4, 32]):
before after
no group_size invalid packed INT4 scales/codebook ok
"group_size": 16 ok ok
"group_size": 32 invalid packed INT4 scales/codebook invalid packed INT4 scales/codebook
I don't have the real model / a GPU here, so I didn't run a full generation; the reporter's workaround (adding "group_size": 16 to each entry) is equivalent to what this does at load time.
Summary by CodeRabbit
Bug Fixes
Quantized model metadata that omits a weight group size now uses a default size of 16. Explicitly configured values continue to be respected, helping ensure these models are interpreted consistently.
No actionable comments were generated in the recent review.
Walkthrough
The ComfyUI quantization metadata parser now uses a weight group size of 16 when asym_w4a8_int8 metadata omits group_size. Explicit values remain in effect.
Changes: ComfyUI quantization metadata
Layer / File(s)
Summary
Default W4A8 group size src/model_io/safetensors_io.cpp
When asym_w4a8_int8 metadata omits group_size, the parser now uses 16. An explicit value remains in effect.
leejet/stable-diffusion.cpp#2276: Adds direct loading of ComfyUI asym_w4a8_int8 checkpoints and packed INT4 weight handling, which this parser change supports.
Suggested reviewers:leejet
Merge Risk: Minimal · up to f9768
This change lets ComfyUI W4A8 files that omit group_size load with the default of 16, as ComfyUI does. Files with explicit values behave as before. No merge-blocking risk was found.
Pre-merge checks: 4
Check name
Status
Explanation
Title check
The title names the main change: asym_w4a8_int8 group_size now defaults to 16, matching ComfyUI. The raw summary confirms that the code applies this default when group_size is missing and keeps explic…
Description check
The description fills the Summary section with the cause, the fix, and the scope. It cites the linked issue (Fixes #2387) in the related-issue area and gives detailed verification notes under Addition…
Linked Issues check
[#2387] The issue requires that an asym_w4a8_int8 file with no group_size key load with the ComfyUI default of 16, and that explicit values stay unchanged. At the reviewed head, read_comfy_quant_confi…
Out of Scope Changes check
The PR changes only src/model_io/safetensors_io.cpp, with a reported +4/-0 change. The change adds a default group_size of 16 for asym_w4a8_int8 metadata, with a comment that cites ComfyUI behavior. I…
Full details: Title check
Explanation
The title names the main change: asym_w4a8_int8 group_size now defaults to 16, matching ComfyUI. The raw summary confirms that the code applies this default when group_size is missing and keeps explicit values. The title is short, single-sentence, and specific, and it does not use the 'fix:' prefix as a substitute for content.
Full details: Description check
Explanation
The description fills the Summary section with the cause, the fix, and the scope. It cites the linked issue (Fixes #2387) in the related-issue area and gives detailed verification notes under Additional Information, which covers the template's verification-notes guidance. The Checklist section from the template is missing. Its items cover contribution guidelines and model documentation, so the omission matters for the PR as a whole. The Summary heading is present but not labeled as a separate Related Issue section, so the template structure is only partly followed.
Resolution
Consider adding the Checklist section from the repository template, with both items checked or explained as not applicable. The description is otherwise mostly complete.
Full details: Linked Issues check
Explanation
[#2387] The issue requires that an asym_w4a8_int8 file with no group_size key load with the ComfyUI default of 16, and that explicit values stay unchanged. At the reviewed head, read_comfy_quant_config in src/model_io/safetensors_io.cpp sets config.weight_group_size = json.value("group_size", 16) when format == "asym_w4a8_int8". The packed INT4 check then tests weight_group_size == 16. A missing key now passes this check. An explicit group_size is still read from the file, so a value of 32 still fails. The change matches the requested scope. The PR adds no automated test. The issue does not require one, and the author's synthetic-file harness is local only. The real model was not run. Those checks are not in the repository and do not change the code-level result.
Full details: Out of Scope Changes check
Explanation
The PR changes only src/model_io/safetensors_io.cpp, with a reported +4/-0 change. The change adds a default group_size of 16 for asym_w4a8_int8 metadata, with a comment that cites ComfyUI behavior. It does not change other formats, the convrot_groupsize default, or the explicit group_size path. Every changed line is tied to issue #2387. No unrelated change was found in the available evidence. The whole-PR diff could not be read as a text diff, so this verdict relies on the head file content and the PR summary.
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Repository UI
Review profile: CHILL
Plan: Advanced
Run ID: 4ce1fab5-dcc0-4eb6-b352-933facb20f4f
Commits
Reviewing files that changed from the base of the PR and between 532d2c1 and f9768f9.
Files selected for processing: 1
src/model_io/safetensors_io.cpp
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2387
ComfyUI
asym_w4a8_int8exports can leavegroup_sizeout of.comfy_quant(all 200 entries in the MiniMax-H3 W4A8 file do). ComfyUI falls back to 16 in that case (comfy/ops.py:layer_conf.get("group_size", params_conf.get("group_size", 16))), butread_comfy_quant_configreads it as 0, and the packed INT4 check only accepts 16, so those files always fail withinvalid packed INT4 scales/codebook.This reads
group_sizewith a default of 16 forasym_w4a8_int8, same pattern as theconvrot_groupsizedefault of 256 right above. An explicitgroup_sizein the file is still used as before.Verification (local only, nothing added to the repo): I built
safetensors_io.cppagainst ggml-base with a tiny harness callingread_safetensors_fileon synthetic files with oneasym_w4a8_int8module (I8 weight[4, 256], codebook[16], s_channel[4], s_rel F8_E4M3[4, 32]):I don't have the real model / a GPU here, so I didn't run a full generation; the reporter's workaround (adding
"group_size": 16to each entry) is equivalent to what this does at load time.Summary by CodeRabbit