Code Language Models
A research project exploring whether LLMs can act as optimizers for language models by writing and refining Python code as the model itself - replacing learned weights updated by gradient descent with human-readable rules updated by LLM-guided code edits. The system runs a multi-agent optimization loop: a planner agent reviews past results and proposes improvement ideas, parallel improver agents implement each idea on isolated branches, and an integrator agent evaluates and merges the best performers back into the main line. A constraint scanner enforces that the model stays purely rule-based - no neural networks, no corpus statistics, no learned parameters.
Code Language Models explores an intentionally strange constraint: can a language model be represented as executable rules instead of learned parameters?
The loop is multi-agent. A planner studies past failures, improver agents make isolated edits to the rule system, and an integrator evaluates candidates before merging the best performer back into the main line.
The point is not to replace frontier models. It is to test how much useful language behaviour can be made explicit, inspectable, and editable when the model is written as code.