search menu icon-carat-right cmu-wordmark

Simulating Realistic Human Activity Using Large Language Model Directives

Technical Report
The authors explore how activities generated from the GHOSTS Framework’s NPC client compare to activities produced by GHOSTS’ default behavior and LLMs.
Publisher

Software Engineering Institute

CMU/SEI Report Number
CMU/SEI-2023-TR-005
DOI (Digital Object Identifier)
10.1184/R1/24150909

Abstract

In this report, we explore how activities generated from the GHOSTS Framework’s non-player character (NPC) client, including software usage, compare to activities produced by GHOSTS’ default behavior and large language models (LLMs). We also explore how the underlying results compare in terms of complexity and sentiment. In our research, we leveraged the advanced natural language processing capabilities of generative artificial intelligence (AI) systems, specifically LLMs (i.e., OpenAI’s GPT-3.5 Turbo and GPT-4) to guide virtual agents (i.e., NPCs) in the GHOSTS Framework, a tool that simulates realistic human activity on a computer. We devised a configuration to fully automate activities by using an LLM, where text outputs become executable agent directives. Our preliminary findings indicate that an LLM can generate directives that result in coherent, realistic agent behavior in the simulated environment. However, the complexity of certain tasks and the translation of directives to actions present unique challenges. This research has potential implications for enhancing the realism of simulations and pushing the boundaries of AI applications within human-like activity modeling. Further studies are recommended to optimize agent understanding and response to LLM directives.

Cite This Technical Report

Updyke, D., Podnar, T., & Huff, S. (2023, October 2). Simulating Realistic Human Activity Using Large Language Model Directives. (Technical Report CMU/SEI-2023-TR-005). Retrieved April 18, 2024, from https://doi.org/10.1184/R1/24150909.

@techreport{updyke_2023,
author={Updyke, Dustin and Podnar, Thomas and Huff, Sean},
title={Simulating Realistic Human Activity Using Large Language Model Directives},
month={Oct},
year={2023},
number={CMU/SEI-2023-TR-005},
howpublished={Carnegie Mellon University, Software Engineering Institute's Digital Library},
url={https://doi.org/10.1184/R1/24150909},
note={Accessed: 2024-Apr-18}
}

Updyke, Dustin, Thomas Podnar, and Sean Huff. "Simulating Realistic Human Activity Using Large Language Model Directives." (CMU/SEI-2023-TR-005). Carnegie Mellon University, Software Engineering Institute's Digital Library. Software Engineering Institute, October 2, 2023. https://doi.org/10.1184/R1/24150909.

D. Updyke, T. Podnar, and S. Huff, "Simulating Realistic Human Activity Using Large Language Model Directives," Carnegie Mellon University, Software Engineering Institute's Digital Library. Software Engineering Institute, Technical Report CMU/SEI-2023-TR-005, 2-Oct-2023 [Online]. Available: https://doi.org/10.1184/R1/24150909. [Accessed: 18-Apr-2024].

Updyke, Dustin, Thomas Podnar, and Sean Huff. "Simulating Realistic Human Activity Using Large Language Model Directives." (Technical Report CMU/SEI-2023-TR-005). Carnegie Mellon University, Software Engineering Institute's Digital Library, Software Engineering Institute, 2 Oct. 2023. https://doi.org/10.1184/R1/24150909. Accessed 18 Apr. 2024.

Updyke, Dustin; Podnar, Thomas; & Huff, Sean. Simulating Realistic Human Activity Using Large Language Model Directives. CMU/SEI-2023-TR-005. Software Engineering Institute. 2023. https://doi.org/10.1184/R1/24150909