Enhancing Stability and Convergence in Graph-Based Chess Reinforcement Learning Using Adaptive Optimization Strategies

Graph-based state representations are attractive for board-game reinforcement learning because they encode pieces, squares, and legal interactions as relations rather than as fixed image-like tensors. AlphaGateau follows this direction for chess by applying attention-based message passing to a graph of board states and moves. The same design, however, makes optimization difficult: the targets produced by self-play change over time, and deep residual message-passing layers can amplify unstable gradient behavior. This paper studies a practical optimization package for AlphaGateau that combines linear warmup with cosine learning-rate decay, global-norm gradient clipping, and decoupled weight decay via AdamW. The experiments compare the standard AlphaGateau training configuration with the proposed optimizer setting and include a reduced-scale staged ablation to examine the role of the schedule, clipping, and AdamW under the same multi-seed budget. In a three-seed 30-iteration reproduction, the proposed configuration reduces the final loss mean from 2.491 to 1.669 and lowers the trajectory-average loss from 2.802 to 2.523. It also gives a small improvement in final Elo under the same reduced evaluation protocol.