Skip to content

mlx backend silently drops weight_decay and seed despite the ignored-with-a-log-line convention #7

Description

@thedataengineer

The mlx backend is the only one that implements the "ignored with a log line" convention (backends/mlx.py:349-358), but its ignored-fields list is incomplete and two fields are dropped silently:

  1. weight_decay is never applied. The optimizer is plain optim.Adam(learning_rate=...) at mlx.py:347; weight_decay has zero occurrences in the file. Since TrainConfig.weight_decay defaults to 0.01 (training.py:48), every mlx run silently discards a value the user can reasonably believe is active. Either switch to AdamW or add it to the ignored list.

  2. seed is honored only on the more_plus path (mlx.py:700). The standard SFT/DPO/GRPO paths never seed, so mlx runs are non-reproducible and the field is dropped with no log line.

  3. The optim check at mlx.py:351 compares against != "adamw_8bit", but that IS the dataclass default (training.py:51), so a user who explicitly requests the default gets no note even though mlx uses plain Adam.

  4. The adapter-kind catch-all at mlx.py:292 ("mlx backend has no attach path for adapter kind {spec.adapter!r}") is fail-closed but not actionable, unlike the soft-prompt error at mlx.py:249-253 which correctly points at backend='torch'. Any custom adapter kind registered via methods.register(...) hits the bare message.

Related, different file: training.py:74 more_plus_tau is accepted and persisted but the per-token gating it configures does not exist yet (comment says "v1.1"). Setting it is silently inert; worth a log line or a docstring note.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions