LLM-Assisted Workflows for Replicating Agent-Based Models

 

In a previous post we discussed how Large Language Models (LLMs) can be used in replicating models.  Keeping with this theme but also taking advantage of the developments in LLMs we have continued with this line of research which has led to a new paper entitled "Large Language Model-Assisted Workflows for Replicating Agent-Based Model" which we (Boyu Wang and myself ) will be presenting at the 2026 International Conference of the Computational Social Science Society of the Americas.  In this paper we discuss two forms of role-separated LLM assistance to replicate agent-based models: a researcher-mediated workflow and a skill-assisted workflow. 

For the researcher-mediated workflow, which we applied to the Ya-TASERPS model, two LLMs were assigned distinct roles, with one LLM for planning and evaluation, and the other for source code implementation. The planning and evaluation LLM inspected the source materials, divided overall replication and validation tasks into phases, specified acceptance criteria (e.g., if the replications shared the same spatial configuration of the original model), and assessed completed work based on these criteria. While the implementation LLM translated each phase into target source code and addressed evaluation findings. In this workflow, a human researcher oversees their work and adjudicated ambiguities (e.g., discrepancies between the original model and replication). Intermediary outputs from both LLMs such as task list, progress log, findings and design decisions were kept as written records in separate files.

In the skill-assisted workflow which we used to replicate the Sakoda's checkerboard model and the Slumulation model, the LLMs were also tasked with their own designated roles. In addition to source materials such as NetLogo code and paper, LLM skills were also supplied as input, where researcher pre-defined reusable prompts specifying planning, implementation, and verification procedures for the LLMs. Inspired by an emerging practice loop engineering, in contrast to the researcher-mediated workflow, a workflow coordination LLM was introduced to spawn other LLMs and assign tasks. When ambiguities arose, the coordination LLM escalated them to the human researcher for resolution.

If this sounds of interest and you wish to find out more, below you can see the abstract to the paper, along with the researcher-mediated and a skill-assisted workflows and some of our results. More details about the work can be found in the paper itself which is linked at the bottom of the post. Lastly,  for interested readers, the three Mesa implementations and supporting materials (including the agent skills) are available at https://github.com/wang-boyu/csssa2026-abm-replication.

Abstract: Replication is widely recognized as a cornerstone of scientific credibility, yet it remains relatively uncommon in agent-based modeling due to the substantial time, effort, and technical expertise required to reimplement models across platforms. This paper explores whether large language models (LLMs) can reduce these barriers and support the replication of agent-based models. We present two LLM-assisted workflows: a researcher-mediated approach that separates planning, evaluation, and implementation across specialized LLM roles, and a skill-assisted approach that combines reusable prompting strategies with an LLM-based coordination process. Using these workflows, we replicated three models in the Python-based Mesa framework. Validation of the replicated models employed structural, behavioral, visual, and statistical comparisons appropriate to each model. Across all three case studies, the Mesa implementations reproduced the key mechanisms, relationships, and emergent patterns reported in the original work. Where source code and statistical results were available, replicated outputs closely matched those from the original implementations. Our findings demonstrate that LLMs can substantially assist model replication from varying levels of source material. We conclude that LLM-assisted workflows offer a promising avenue for improving reproducibility, transparency, and knowledge transfer within the agent-based modeling community.

Keywords: Agent-Based Modeling, Generative AI, Large Language Models, Replication, Verification and Validation.

Two role-separated LLM-assisted workflows used in this study. 

NetLogo and Mesa distributions for  prosocial score under various input levels.

Representative checkerboard states used as illustrations for comparison with source-described patterns.

Slumulation visualizations from 30 independent runs:  mean city, central, and peripheral slum-population trajectories with 95% bootstrap intervals.

Screenshots of a typical run, shown at t = 0, t = 25, and t = 50.

Full Reference: 

Wang, B. and Crooks, A.T. (2026), Large Language Model-Assisted Workflows for Replicating Agent-Based Models. 2026 International Conference of the Computational Social Science Society of the Americas, Santa Fe, NM. (pdf)

Visual Representiaon of Complexity

Often I get asked about complex systems and what are some of their properties or examples of such propoteites. When this is the case I give simple examples, such as negative feedbacks in traffic. However,  recently the Computational Social Science Society of the Americas (CSSSA) hosted a webinar by Nigel Gilbert entitled "Policy in a complex world: towards complexity-aware policymaking" and one of the many things discussed was a poster entitled the "Visual Representation of Complexity" which I thought was worth sharing as it give a good overview of the properties of complex systems along with examples. 

While on the complexity theme, I thought I would also highlight the Map of Complexity by Brian Castellani and Lasse Gerrits as it showcases how complexity the evolution of complex systems and how its being used in various disciplines.  

Call for Abstracts: Geosimulation: Human Dynamics, AI-Enabled Agents, and Spatial Decision-Making

We are organizing sessions on "Geosimulation: Human Dynamics, AI-Enabled Agents, and Spatial Decision-Making" at the AAG Annual Meeting in New York City, February 8-12, 2027.

Building on our geosimulation sessions of the past three years, we continue to welcome a wide range of studies exploring simulation theories, data, methodologies, and frameworks. We are interested in case studies that apply geosimulation to real-world challenges, as well as research that develops and evaluates AI-enabled geosimulation models. Potential topic areas include, but are not limited to:

• Geosimulation Models and Applications
• Conceptual Geosimulation Models
• General-Purpose Geosimulation Frameworks
• AI and Geosimulation
• Agent Behavior, Decision-Making, and AI Agents
• Data Generation Frameworks
• Validation and Verification for Geosimulation
• Digital Twins
• Microsimulation
• Multi-Agent Systems
• System Dynamics Models

Interested in presenting? Submit your abstract to AAG by October 15, 2026. Then send your title, abstract (up to 250 words), and AAG abstract code to Boyu Wang (bwang44@buffalo.edu) and Ying Zhou (yzhou59@buffalo.edu) by November 5, 2026.

Organizers: Andrew Crooks, Na (Richard) Jiang, Fuzhen Yin, Boyu Wang, Ying Zhou, Joon-Seok Kim, Taylor Anderson, Yahya Gamal, Wenwu Tang, Sarah Wise, Yao-Yi Chiang, Sean Ahearn, Liliana Perez

Please share with colleagues and students who may be interested!

More details

EPB: Collaborations and Topics Over Decade

When Environment and Planning B: Urban Analytics (EPB) turned 50 we wrote a short commentary about how the topics in the journal have evolved over the years. In the latest issue of the journal we (Na Jiang, Joana Barros, Seraphim Alvanides, Jinlin Wu and myself) revisited this, while at the same time explored how collaborations and topics have been evolving along with how this links to the new Editorial Advisory Board which has recently been selected to reflect the journal’s thematic areas of focus and the breadth of expertise needed to guide it into the future. 

If you wish to find out more, we encourage you to read the commentary (a link is provided at the bottom of this post). However, to give a sense of some of the things we explore, below we show some of the images relating to the commentary ranging from our workflow to some of our results from the analysis of authors, paper titles and abstracts, and how through topic modeling and community detection we can explore how the collaboration networks and topics have changed over time. 

The workflow.

 

Visualization of the EPB author network.

Evolution of the largest 10 communities by decade: (a) The Number of Active Members; (b) The Number of Papers Published; (c) Michael Batty’s Collaboration Network.

Topics evolution overtime.

Full reference: 

Crooks, A.T., Jiang, N., Barros, J., Alvanides, S., & Wu, J. (2026). Environment and Planning B: Collaborations and Topics Over Decades, Environment and Planning B, 53(6), 1189-1199. https://doi.org/10.1177/23998083261474805. (pdf)

A Dual-LLM Supervised Workflow for Replicating Agent-based Models

In the past we have written about large language models (LLMs) and how these can be used in agent-based modeling. However, these previous posts only touched the surface on what is possible. One area we are currently exploring is how LLMs can aid in model replication, which is a major challenge in agent-based modeling. In the sense, there are countless agent-based models but very few are replicated and if  a model is to withstand the test of time, replication is needed. 

To this end, Boyu Wang, JoAnn Lee and myself have an extended abstract entitled: "A Dual-LLM Supervised Workflow for Replicating Agent-based Models: From NetLogo to Mesa" at the 2026 Social Simulation Conference. In this work we demonstrate how LLMs can be used to replicate an exiting model into another modeling package, Specifically, we take the Ya-TASERPS model which was initially implemented in NetLogo and re-implement it in Mesa via a  LLM workflow. 

The workflow uses two GPT-5.4 models with distinct LLM roles under continuous human mediation. One LLM role handles planning and evaluation, from checking source code, decomposing tasks into phases, and reviewing whether implementations are consistent with the source NetLogo model. The other LLM role performs the actual implementations in phases, as outlined by the first LLM role.

Each task phase defines acceptance criteria, proceeds through implementation and verification, and ends with human review before progression. The human researcher mediates both LLM roles, resolves ambiguities, and decides whether the resulting model state is acceptable. The aim is not towards full automation but to reduce prompt drift, limit uncontrolled rewrites, and keep generated code subordinate to explicit validation. 

If you want to read about our replication, why we chose the Ya-TASERPS model along with our findings, please feel free to read the paper and more information about this can be found at: https://github.com/wang-boyu/Ya-TASERPS.

The NetLogo-to-Mesa replication workflow utilizing two distinct LLM roles.

Large-run prosocial outcomes across the six user-controlled inputs in NetLogo and Mesa with the same set of 729 parameter combinations, 10 random seeds, and 1800 days per run. The two implementations preserve the same broad significance and directional patterns.

Full Reference: 

Wang, B., Lee, J. and Crooks, A.T.  (2026), A Dual-LLM Supervised Workflow for Replicating Agent-based Models: From NetLogo to Mesa. Social Simulation Conference 2026, Durham, UK. (pdf)