Among the many areas in which LLMs can accelerate scientific research are applications of LLMs in Agent-based modeling (ABM). So, in this year’s SBP-BRiMS2025 challenge problem, we ask participants to investigate and develop solutions that integrate LLM-driven prompts for ABMs. Participants are asked to consider the toolchain developed by Jimenez-Romero, Yegenogly, and Blum in https://arxiv.org/abs/2503.03800 that facilitates prompt-driven behavior generation applying structured, rule-based prompts and autonomous, knowledge-driven prompts in NetLogo’s classic swarm intelligence models: ant colony foraging and bird flocking.
Using the toolchain provided, https://github.com/crjimene/swarm_gpt/tree/main and NetLogo, https://ccl.northwestern.edu/netlogo/, apply the toolchain to one of the NetLogo social science models of self-organizing processes and emergent behaviors, available at the NetLogo Models Library https://ccl.northwestern.edu/netlogo/models/index.cgi. Then investigate how applications of LLM prompts generate similar model outcomes. Demonstrate results through a graph comparison of the core model metric, such as food collection in the ant colony model, by step for the original Netlogo model and the LLM/NetLogo hybrid model. Discuss how close the two model outcomes are and, if this resulted in any behaviour change, discuss why the behavior changed. In addition to the aforementioned submission requirements, participants must include publicly shared code and documentation of LLM prompts.
Course or thesis projects are welcome submissions with the approval of students’ professors or advisors.
You need to submit 4 things: a 2-page extended abstract in conference format, a poster to be exhibited at the conference, a single PowerPoint teaser slide - that we can use to promote your entry, and the code/prediction output.
Challenge Paper: A 2-page extended abstract describing the procedure, theories and models/ methods used for prediction, performance metrics, and references. Specifically:
Poster: A poster that summarizes the Challenge Paper (above) for presentation at the conference. Posters are not required before the conference. Posters must be 3’ x 4’ brought to the conference, the conference is unable to print posters.
Promotion/teaser Slide: This is a single PowerPoint slide. The purpose of this slide is to excite people to come to your poster. This slide will also be put online. We will use this slide to promote your entry. This slide should contain:
Prediction Output and Code: In addition to the challenge paper, poster, and teaser slide, submit a zip file that contains your prediction code and three output data files. See Evaluation Criteria section below for details.
Each entry for the challenge problem should have the following three output files: (1) an image of the working NetLogo GUI, (2) LLM prompts, and (3) a graph of the original NetLogo results compared with the LLM/NetLogo hybrid.
Challenge Response Submission: 21-Aug-2025
Author Notification: 4-Sep-2025
Final Files Due: 1-Oct-2025
Submissions will be accepted at the conference installation on Easychair, https://easychair.org/conferences?conf=2025sbpbrims. Please email sbp-brims@andrew.cmu.edu with any questions. Authors must register prior to uploading.
All entries will send at least one team member to SBP-BRiMS2025 who will be registered for the conference by the early registration deadline to represent their entry.