Repository navigation
Update dependency transformers to v5.20.2 - #24
Open
renovate[bot] wants to merge 1 commit into
Open
renovate[bot] wants to merge 1 commit into
renovate[bot] wants to merge 1 commit into
Conversation
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
renovate
Bot
changed the title
renovate
Bot
force-pushed
the
renovate/transformers-5.x
branch
from
October 16, 2026 15:15
892e39f to
a5e3de5
Compare
renovate
Bot
force-pushed
the
renovate/transformers-5.x
branch
from
October 20, 2026 13:43
a5e3de5 to
c408335
Compare
renovate
Bot
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
==5.19.0→==5.20.2Release Notes
huggingface/transformers (transformers)
v5.20.2: Patch release v5.20.2Compare Source
Patch release: v5.20.2
This patch notably solves an issue where some inputs could loose their original type inside
prepare_inputs_for_generationingenerate, leading to problems with multimodal models. It also fixes some other issues listed on the below commits.It contains the following commits:
v5.20.1Compare Source
Release v5.20.1
Special surprise including the new model Beam from Reflection AI 🥳
Beam is a token-efficient, frontier-scale model focused on coding and agentic use cases. It is built for teams that want to run a powerful model on their own hardware, infrastructure, or cloud, and customize it to their specific needs.
Links: Documentation
Some real patch fixes
We broke loading models on older torch versions 😢 should be fixed now. But here is the full list of fixes
Full Changelog: huggingface/transformers@v5.20.0...v5.20.1
v5.20.0Compare Source
Release v5.20.0
New Model additions
Muse Spark Open
Muse Spark Open is a multimodal mixture-of-experts model from Meta that takes text, images, audio, and video as input and generates text. It has 816B total parameters (42B active per token) and a context window of up to 1,048,576 tokens shared across modalities. It is trained for agentic coding tasks such as multi-file editing, debugging, and tool use. Checkpoints are available in BF16 and NVFP4 formats.
Links: Documentation | Blog Post
Ling 3.0
Ling 3.0 Flash is a hybrid Mixture-of-Experts language model from InclusionAI. Its decoder groups five Kimi Delta Attention (KDA) layers with one Multi-head Latent Attention (MLA) layer. The MLA blocks use interleaved rotary embeddings and a head-wise output gate, and the feed-forward blocks combine routed and shared experts. It is natively supported in Transformers as
BailingHybridForCausalLM, so it loads withouttrust_remote_code=True.Links: Documentation
Kolibri1
Kolibri 1 is Aleph Alpha's mixture-of-experts reasoning model, focused on German and English. Each of its 50 layers routes every token to 6 of 384 experts using sigmoid routing, and adds one shared expert. Four sliding-window attention layers alternate with one full-attention layer, and RoPE is applied only in the sliding-window layers.
Links: Documentation
MiniCPM-V 4.7
MiniCPM-V is a series of efficient multimodal large language models developed by OpenBMB. Like MiniCPM-V 4.6, the MiniCPM-V 4.7 architecture pairs a SigLIP vision encoder that has a window-attention merger with a Qwen3.5 language model backbone, and supports both 4x and 16x visual downsampling modes.
The main addition over 4.6 is canvas M-RoPE: instead of numbering visual tokens along a single 1-D sequence, the model lays every image out on a 2-D canvas and assigns each visual token a
(temporal, height, width)position, so slices of the same image keep their spatial relationship and video frames keep their temporal order.UnlimitedOcr
UnlimitedOcr is a single 3B-parameter OCR model from Baidu with a standard 32,768-token context length. It builds on DeepSeek OCR and replaces all decoder attention layers with Reference Sliding Window Attention (R-SWA). This keeps the KV cache constant during decoding, so the model can transcribe dozens of pages in a single forward pass. It supports "gundam" mode (640x640 tiles with dynamic cropping) and "base" mode (a single 1024x1024 view), and it can output region detections with bounding boxes.
Links: Documentation | Paper
UltraBERT
UltraBERT is a family of bidirectional text encoders trained with a combination of masking ratios, pruning, and distillation for classification and retrieval tasks. The released encoders support sequences of up to 32,768 tokens. They use rotary position embeddings, RMSNorm, SwiGLU feed-forward layers, and alternating local and global attention (every third layer is global). Checkpoints include UltraBERT-150M and UltraBERT-380M for masked language modeling and fine-tuning, and UltraBERT-Embed-150M for text embeddings.
Links: Documentation | Paper
MOSS-Transcribe-Diarize
MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, speaker diarization, and timestamps. It combines a Whisper-style encoder, a 4× frame merge step, a multi-modal projector, and a Qwen3 language model. It produces joint transcription and diarization in a single pass, with segments formatted as
[start][S01]text[end]. It supports long-form audio through Whisper-window chunking in the processor, promptable transcription with hotword biasing, and 50+ languages.Links: Documentation
Breaking changes
🚨 General breaking notice that inference with EP has regressions in speed; we are working on fixing this ASAP! 🚨
Deprecated checks and methods have been removed from
cache_utils.py(affecting models such as minimax_m3_vl and pi0), so any code still calling those deprecated cache methods must switch to the current cache API.Accidental imports from other models were removed from several model files (e.g. embedding_gemma2, ijepa, lightglue), and the modular converter now avoids adding such cross-model imports. If you imported these names through those model modules, import them from their original modules instead.
The minimum
kernelsversion is now 0.17.2, which allowsuse_fallback=KernelizeFallback.ALLso that a failed kernel load falls back to the original implementation instead of raising an error. Upgradekernelsto 0.17.2 or later.A new
SubConfigSpeccentralizes the initialization of sub-configs, so default sub-configs are collected from dataclass fields instead of by instantiating the class, and this touches the configuration utilities and many model configs. Custom or existing nested configs may need to be updated to declare their sub-configs through this mechanism.The LUKE and mLUKE tokenizers now treat entity special tokens such as
<ent>as special tokens (not split, honored byskip_special_tokens, normalized) while keeping the same spacing as regular vocabulary tokens when decoding. Tokenization and decoding output involving these tokens may therefore differ from before, so check any code or saved outputs that depend on the old behavior.Exporters
Transformers exporters now support quantization directly during export across Dynamo, ONNX, OpenVINO, and ExecuTorch. The new unified quantization API integrates PT2E, ONNX Runtime, and NNCF quantizers, enabling static and dynamic quantization as well as weight-only compression (INT8/INT4), with configurable calibration workflows. This makes it easier to export optimized models for efficient inference across different backends.
For more information see the docs: https://huggingface.co/docs/transformers/exporters_quantization
Parallelization
This release improves distributed training and inference. FSDP × TP model and optimizer checkpoints can now be saved and loaded across different parallel configurations, and expert-parallel dispatch, tensor-parallel plans, static-cache
generate(), and the caching allocator's byte count were fixed. Tied LM heads in composite models now get their own TP plan entry,enable_expert_parallelapplies the EP plan alone again, TPU loading uses the `torGeneration
This release fixes static-cache
generate()bugs with host-side inputs and under tensor parallelism. It also speeds up Whisper generation on TPU by cutting device syncs and recompilation, and fixes Whisper batched timestamp decoding for tensor and NumPy inputs. Pipelines get a cleanup of generation-argument handling that removes spurious warnings.Vision
Vision processing was updated in several places. Image decoding now uses torchcodec, Molmo2's image and video kwargs follow the library's standard conventions, and Pixtral Flash Attention no longer detects false image boundaries after the axial RoPE refactor, which had caused CUDA illegal memory access errors and incorrect output. Several CI and Docker changes also shipped: GPU images moved to Python 3.11, a numpy/pyarrow conflict
Kernels
This release improves kernel support across hardware.
flash_attention_2now falls back to the AITER kernel on ROCm whenflash-attnisn't installed, and T5-style models get a contiguous SDPA position-bias mask so they use fused kernels again. The Hub rotary kernel is now limited to CUDA inference because its backward pass gave incorrect gradients. The Atlas Inference Gated DeltaNet kernels for Strix Halo andCache
This release fixes several cache-related bugs: static-cache
generate()now handles host-side nested inputs and tensor parallelism correctly, the offloaded cache prefetch no longer races with the offloading copy, and Phi3 LongRoPE cache rebuilds at the context boundary now receive the full prefix. It also counts the query token in MiMo-V2-Flash's sliding window and corrects the quantized cache (hqq/quanto) documentation examBugfixes and improvements
torch.mm(..., out_dtype=...)to ONNX (#49635) by @IlyasMoutawwakil in [#49635]MtpModelhelpers for frameworks that own the drafting loop (#49544) by @hmellor in [#49544]create_chunked_causal_mask()crashing withblock_sequence_ids(#49457) by @kaixuanliu in [#49457]ep_size != tp_size(#49368) by @qgallouedec in [#49368]Significant community contributions
The following contributors have made significant changes to the library over the last release:
Configuration
📅 Schedule: (UTC)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.
Need help on this PR? Tag
@codesmith-botwith what you need.