ROOK-CLF-9M Chess Analysis

Motivation

The ROOK-CLF model employs a custom FEN tokenization strategy to create constant-length positional representations. This design enables the transformer to learn consistent positional patterns and attend to the same board squares reliably, rather than dealing with variable-length sequences that would shift attention positions based on move counts or position complexity.

Standard FEN notation has variable components (e.g., castling rights "KQkq" vs "Kq" vs "-") that create inconsistent sequence lengths. Our approach normalizes all positions to exactly 77 characters + [CLS] token = 78 tokens, ensuring each board square maps to the same token position across all games.

Processing Steps:

1. Expand numbers: 8 → ........ (empty squares become dots)
2. Remove slashes: rnbqkbnr/pppppppp → rnbqkbnrpppppppp
3. Pad components: Fixed-width fields for metadata
4. Add [CLS] token: Classification indicator at end

Example Transformation:

Benefits of Fixed-Length Encoding:

• Positional consistency: Square a1 is always at token position 0, h8 always at position 63
• Attention stability: Self-attention can focus on specific board regions consistently
• Batch processing: All positions have identical tensor shapes for efficient GPU computation
• Pattern learning: Model can learn piece interactions at fixed relative distances

Token Details:

• Vocabulary: 36 tokens (12 pieces + 8 files + 8 ranks + 8 metadata)
• Sequence length: Fixed 78 tokens (64 board + 13 metadata + [CLS])
• Board encoding: Ranks 8→1, files a→h, no positional ambiguity
• Special tokens: [CLS] (classification), [SEP], [PAD], [MASK]