I'm just sitting here watching my wife play the Sims-4, and realizing how "perfect" of a learning environment it could be for LLMs, specifically for long-term planning. Is there anyone already looking into this, or am I off the mark for any reason?
If you aren't in the know, the Sims has a dedicated community of people coming up with challenges for the game. One example is having 12 generations of a family all out of the same house. In order to achieve this you need to plan almost the entire game out from the start. Then you need to manage not just the individual Sims, but the household, and overall dynasty timeline to keep things moving smoothly. Things like when to start having the next generation to keep the rooms in the house full, but not overlapping with older generations who refuse to die off, take quite a lot of planning on all levels of detail to your overall goal. I could imagine a testing structure that takes a screenshot every second or so(no need to overoptimize framerate for this game) and then use an LLM to notate key points in the image in relation to the long, mid, and short term goals, and then asking for an update on what actions to take next. Since we are already working in a game environment we get clear and immediate feedback on the success/failure of those actions and how they affect the overall goal.
Doesn't this sound like a great sandbox to probe how LLMs create and execute long-term plans? At least it could act as a way to limit variables while trying to understand how these models work, while at most it could help drive optimizations in planning and information management. What do yall think? Is this worth looking into or am I barking up the wrong tree?
[link] [comments]