Conversation
- Replaced dynamic batch size validation with a fixed list of choices to ensure consistency across trials. - Updated the batch size selection logic to raise a `TrialPruned` exception if the chosen batch size is incompatible with the calculated buffer size, enhancing error handling during hyperparameter tuning.
- Updated the game script to support new gameplay modes: human vs MAPPO, MAPPO vs MAPPO (self-play), and random village demo. - Added a new function for self-play using the same MAPPO policy for both teams, improving testing and training capabilities. - Enhanced command-line arguments to allow selection between opponent and self-play modes, ensuring clear usage instructions. - Updated documentation and logging to reflect the new gameplay features and configurations.
- Introduced new team reward configurations in `bot_rewards.yaml`, including penalties and bonuses related to hunger and food security. - Enhanced the `GameEnv` class to finalize team rewards based on controlled bots, incorporating mean/sum calculations and terminal outcomes. - Added new methods in `BotRewardCalculator` for team-level reward shaping and terminal bonuses, improving the reward system for bot training. - Updated tests in `test_reward_bot.py` to validate the new team reward functionalities and ensure accurate calculations for various scenarios.
- Added `max_grad_norm` parameter to training configuration for improved gradient clipping. - Updated Optuna tuning script to include new choices for `n_steps`, `gamma/gae` presets, and `checkpoint_interval`, enhancing hyperparameter search capabilities. - Introduced `win_townhall_frac` metric in the episode metrics callback to track the fraction of wins via opponent town hall destruction, improving performance analysis. - Modified training function to return `win_townhall_frac` alongside existing metrics, providing more insights into training outcomes.
- Introduced `--objective` argument in the Optuna tuning script to allow selection of metrics for optimization: `ep_rew_mean`, `win_townhall_frac`, and `win_frac`. - Enhanced the episode metrics callback to compute and log the mean episode reward, improving performance tracking. - Updated training function to return the mean episode reward alongside existing metrics, providing deeper insights into training outcomes. - Added tests to validate the mean episode reward calculations, ensuring accuracy in metrics reporting.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.