LLM-Assisted Workflows for Replicating Agent-Based Models

 

In a previous post we discussed how Large Language Models (LLMs) can be used in replicating models.  Keeping with this theme but also taking advantage of the developments in LLMs we have continued with this line of research which has led to a new paper entitled "Large Language Model-Assisted Workflows for Replicating Agent-Based Model" which we (Boyu Wang and myself ) will be presenting at the 2026 International Conference of the Computational Social Science Society of the Americas.  In this paper we discuss two forms of role-separated LLM assistance to replicate agent-based models: a researcher-mediated workflow and a skill-assisted workflow. 

For the researcher-mediated workflow, which we applied to the Ya-TASERPS model, two LLMs were assigned distinct roles, with one LLM for planning and evaluation, and the other for source code implementation. The planning and evaluation LLM inspected the source materials, divided overall replication and validation tasks into phases, specified acceptance criteria (e.g., if the replications shared the same spatial configuration of the original model), and assessed completed work based on these criteria. While the implementation LLM translated each phase into target source code and addressed evaluation findings. In this workflow, a human researcher oversees their work and adjudicated ambiguities (e.g., discrepancies between the original model and replication). Intermediary outputs from both LLMs such as task list, progress log, findings and design decisions were kept as written records in separate files.

In the skill-assisted workflow which we used to replicate the Sakoda's checkerboard model and the Slumulation model, the LLMs were also tasked with their own designated roles. In addition to source materials such as NetLogo code and paper, LLM skills were also supplied as input, where researcher pre-defined reusable prompts specifying planning, implementation, and verification procedures for the LLMs. Inspired by an emerging practice loop engineering, in contrast to the researcher-mediated workflow, a workflow coordination LLM was introduced to spawn other LLMs and assign tasks. When ambiguities arose, the coordination LLM escalated them to the human researcher for resolution.

If this sounds of interest and you wish to find out more, below you can see the abstract to the paper, along with the researcher-mediated and a skill-assisted workflows and some of our results. More details about the work can be found in the paper itself which is linked at the bottom of the post. Lastly,  for interested readers, the three Mesa implementations and supporting materials (including the agent skills) are available at https://github.com/wang-boyu/csssa2026-abm-replication.

Abstract: Replication is widely recognized as a cornerstone of scientific credibility, yet it remains relatively uncommon in agent-based modeling due to the substantial time, effort, and technical expertise required to reimplement models across platforms. This paper explores whether large language models (LLMs) can reduce these barriers and support the replication of agent-based models. We present two LLM-assisted workflows: a researcher-mediated approach that separates planning, evaluation, and implementation across specialized LLM roles, and a skill-assisted approach that combines reusable prompting strategies with an LLM-based coordination process. Using these workflows, we replicated three models in the Python-based Mesa framework. Validation of the replicated models employed structural, behavioral, visual, and statistical comparisons appropriate to each model. Across all three case studies, the Mesa implementations reproduced the key mechanisms, relationships, and emergent patterns reported in the original work. Where source code and statistical results were available, replicated outputs closely matched those from the original implementations. Our findings demonstrate that LLMs can substantially assist model replication from varying levels of source material. We conclude that LLM-assisted workflows offer a promising avenue for improving reproducibility, transparency, and knowledge transfer within the agent-based modeling community.

Keywords: Agent-Based Modeling, Generative AI, Large Language Models, Replication, Verification and Validation.

Two role-separated LLM-assisted workflows used in this study. 

NetLogo and Mesa distributions for  prosocial score under various input levels.

Representative checkerboard states used as illustrations for comparison with source-described patterns.

Slumulation visualizations from 30 independent runs:  mean city, central, and peripheral slum-population trajectories with 95% bootstrap intervals.

Screenshots of a typical run, shown at t = 0, t = 25, and t = 50.

Full Reference: 

Wang, B. and Crooks, A.T. (2026), Large Language Model-Assisted Workflows for Replicating Agent-Based Models. 2026 International Conference of the Computational Social Science Society of the Americas, Santa Fe, NM. (pdf)