Fixed Modular Source File Issue

Thank you for catching that! I've now applied the fix to the correct file.

What was wrong:

• The file modeling_qwen3_moe.py is auto-generated from modular_qwen3_moe.py
• My initial commit only edited the generated file, which would have been overwritten by CI

What I fixed:

• Added the and self.training check to modular_qwen3_moe.py at line 183
• This is the canonical source that generates the modeling file
• The fix is now in both files and will pass CI validation

Changes:

# Before (line 183 in modular_qwen3_moe.py)
if output_router_logits:

# After
if output_router_logits and self.training:

This ensures load_balancing_loss is only calculated during training, not during inference/generation.

Fixed: Corrected aux_loss Calculation

I've identified and corrected the issue that caused the tests_torch CircleCI failure.

What Was Wrong

My initial fix was too restrictive. I added and self.training to the aux_loss calculation line:

if output_router_logits and self.training:  # ❌ Wrong
    aux_loss = load_balancing_loss_func(...)

This prevented aux_loss from being calculated during eval mode, which broke the test_load_balancing_loss test (which explicitly sets model.eval() and expects aux_loss to be calculated).

The Correct Fix

Following the Mixtral implementation pattern, aux_loss should:

1. Always be calculated when output_router_logits=True (for monitoring/logging)
2. Only be added to the total loss during training

aux_loss = None
if output_router_logits:  # ✅ Calculate when requested
    aux_loss = load_balancing_loss_func(...)
    if labels is not None and self.training:  # ✅ Only add to loss during training
        loss += self.router_aux_loss_coef * aux_loss.to(loss.device)

This fix:

• ✅ Allows tests to verify aux_loss calculation during eval mode
• ✅ Prevents aux_loss from being added to the loss during inference/generation
• ✅ Matches the Mixtral implementation pattern

I've updated both the modular and generated files. The CircleCI tests should now pass.

[For maintainers] Suggested jobs to run (before merge)

run-slow: qwen3_moe