WU Yuxin (吴育昕)

Reinforcement learning infrastructure is indeed a huge challenge. We strive to achieve high efficiency while maintaining good flexibility. In terms of efficiency, we try to fully consider the actual application scenarios of reinforcement learning when developing training and inference systems, so that we can reuse all the heavy computation work and achieve scaling. The deployment logic of the agent swarm is particularly complex, but our system is highly flexible, allowing us to integrate different frameworks and sub-agent setups into the training process.

Managing hallucinations remains a huge challenge for all large models. We have improved this by increasing data quality (more verified knowledge, fewer low-quality claims) and reward mechanisms (e.g., penalizing the model when it hallucinates), but we believe there are still many ways to further improve.

I'm not sure if 1:1 optimality still holds, but in that sense, we do 'waste' some training compute. Otherwise the model would be larger and would 'waste' a lot of inference compute compared to our current model.

Before Sam builds his trillion-dollar data center.