Factoriax: A GPU-Accelerated Factorio-Style Simulator for Reinforcement Learning

Mickey Beurskens, Tristan Tomilin, Thiago D. Simão

We introduce Factoriax, a GPU-accelerated factory-building simulator written in JAX. In Factoriax, an agent must collect resources, build machines using those resources, and then automate the collection and crafting process by arranging machines on the map to build production pipelines. This paper discusses the structure of the Factoriax simulator and an initial benchmark called Easy Rocket in which an agent is tasked with building a resource-intensive machine called the Rocket in a limited number of game ticks to escape the planet. We also publish results from a number of PPO-based training runs on Easy Rocket. Factoriax is built to be fast. A 1-billion-step PPO training run, equivalent to 500,000 episodes, runs on Easy Rocket in about 8 minutes on a single NVIDIA A100. A standard laptop GPU can complete the same run in about 84 minutes. Our trained PPO agent learns to gather resources, craft machines from those resources, and place them on the map through a curriculum reward directly tied to a manually designed set of achievements. After training, the agent does not place machines in a functional spatial configuration, failing to fully complete the benchmark, and leaving the…