Gemma 4

[SamuelMarks]

What does this PR do?

Adds Gemma 4

Checklist

• I have read the Contribution Guidelines and used pre-commit hooks to format and squash so that this PR has 1 commit typically.
• I have added all the necessary unit tests for my change (run_model.py, test_outputs.py, and/or model_validation_colab.ipynb).
• (Code Quality) I have reviewed the code and removed all superfluous comments or LLM-generated boilerplate. Only functional, necessary code remains.
• I have signed the Contributor License Agreement (CLA).

[gemini-code-assist]

Code Review

This pull request introduces the Gemma 4 model architecture, featuring a hybrid Mixture of Experts (MoE) and a cycle of local sliding window and global attention layers. The implementation includes specialized RMSNorm, SwiGLU MLPs, and sharding configurations for FSDP and TP. Feedback indicates that the sliding_window_size is not yet utilized in the attention mechanism and that the MoE implementation should be optimized to avoid inefficient Python loops. Additionally, there are suggestions to fix a missing import in the sharding tests and to clean up imports in the modeling file.