I was nerd sniped with a question about automating an actual player character that could either autonomously play the game, or accept instructions in multiplayer to accomplish specific tasks. The original context was about training a model with reinforcement learning, but I think that's probably too large of a first bite. The interface between the bot and the game client (or an autonomous game client) is probably a better first step. My goal is not to just replay perfect input like a speed-runner, but rather to have a bot that can be dropped into various situations, where it will avoid dying and attempt to progress with the tech tree, maybe even examine belt contents and machine states to troubleshoot and determine its own subgoals.
This is a bit hard to get accurate search engine results because "bot" and "automation" have relevant definitions in-game.
Problem 1 - Control the client (without using keyboard/mouse automation, or sending text into the lua console)
Is there a way to make the Lua console available via stdout/stdin, so that the Factorio client's terminal could be used to send commands or pull data out?
Can Lua mods open a socket or webserver, to accept direct input? Or maybe read/write to files (named pipes), which could be used to provide a "client API"
Problem 2 - getting game data out (map, structures, enemies, alerts)
There's probably Lua commands to get all of this stuff out, and a bot mod would probably be the best way to make shorthand to request different relevant info.
Structures - I might use some Lua to get the camera position and zoom, then use the mouse cursor to create a blueprint, and decode the blueprint string directly from the clipboard.
enemies - seems to be the most dynamic and difficult thing to mess with. Maybe there's a debug bounding box I can turn on (or get the coordinates for the boxes without drawing them to screen)
Problem 3 - learning to be useful
A basic start might be a bot that takes items out of chests and places them to replace blueprint ghosts, runs towards pings on the map, and shoots at enemies that come near it.
As the game gets more strategic and less tactical, the bot might mostly just do those same three activities, but it needs to add the step where it selects or creates blueprints to do something, and pick where they should go on the ground. It should almost be like the bot alt-tabbed to a blueprint website and then pasted it in-game.
Reinforcement learning would be an interesting take, but not easy. Items created and power used are probably good reward metrics, but I don't think I would want to train it from scratch to arrange belts, pipes, and inserters: I think separate models or solvers to "figure out what you need to build and where it should go" and to "make a blueprint that turns X into Y using this location respecting ore / oil / water / other machines you shouldn't mess up" is the approach I would take.
Advanced / recursive stuff like the bot itself commanding spidertrons, which are pretty comparable to its own capabilities, would be a long-term goal (which would massively increase the bot's throughput, by slapping down customized blueprints and sending spidertrons all over creation to assemble them)
Any mods or projects already working on problems like these?
Automating the Engineer / Factorio Client
Re: Automating the Engineer / Factorio Client
viewtopic.php?p=247808#p247808
viewtopic.php?p=453851#p453851
That's for aperitif.
As of the main course...
Ten steps forward, nine steps backwards.
I as AI wrangler want to make AI play Factorio productively.
To do so, I need to have an option:
- for AI to perceive the game world
-- which needs some form of widely available API (or other easy-to-wrangle mechanism) which would contain current world state in most actual state possible
- for AI to affect the game world
-- which needs some form of moving player, placing structures, selecting recipes in strucrures, manually crafting things
- for AI to make decisions about affecting game world, using intel perceived game world as an input
-- which is out of scope of this theme
You don't need explicitly direct connection between AI and Factorio client/server.
You have ready-to-perceive snapshots in a form of autosaves.
Parse these. It's not FPS game, after all. Unless you want to make AI dodge incoming trains, of course...
And Linux have autosaves-without-interruptions now
SOME variants on some hardware, at the very least...
Depending on how vanilla-like AI-vs-the-world interactions you might want to have, you might consider using IMITATION of player character interactions.
Don't emulate key presses, emulate whole interactions like said moving (by teleporting), placing (spawn N at X,Y and remove N from inventory), etc-etc.
As of what would gonna serve as the trigger for these interactions...
viewtopic.php?p=453851#p453851
That's for aperitif.
As of the main course...
Ten steps forward, nine steps backwards.
I as AI wrangler want to make AI play Factorio productively.
To do so, I need to have an option:
- for AI to perceive the game world
-- which needs some form of widely available API (or other easy-to-wrangle mechanism) which would contain current world state in most actual state possible
- for AI to affect the game world
-- which needs some form of moving player, placing structures, selecting recipes in strucrures, manually crafting things
- for AI to make decisions about affecting game world, using intel perceived game world as an input
-- which is out of scope of this theme
You don't need explicitly direct connection between AI and Factorio client/server.
You have ready-to-perceive snapshots in a form of autosaves.
Parse these. It's not FPS game, after all. Unless you want to make AI dodge incoming trains, of course...
And Linux have autosaves-without-interruptions now
SOME variants on some hardware, at the very least...Depending on how vanilla-like AI-vs-the-world interactions you might want to have, you might consider using IMITATION of player character interactions.
Don't emulate key presses, emulate whole interactions like said moving (by teleporting), placing (spawn N at X,Y and remove N from inventory), etc-etc.
As of what would gonna serve as the trigger for these interactions...
Re: Automating the Engineer / Factorio Client
Helpful start, thank you. I suspect that mods are not really the way I'd want to go then, because there's no notion of "client-only" mods, and a compatibility goal I'd really like the bot to be able to drop in to multiplayer games without the server adding a specific mod.EustaceCS wrote: ↑Sat Oct 19, 2024 9:06 pm viewtopic.php?p=247808#p247808
viewtopic.php?p=453851#p453851
That's for aperitif.
You're right that unzipping a save file is most of the information all at once, but I don't want to have to save the map to perceive biters, so something more immediate will be necessary (even if reading the zip file is a better way to "get the lay of the land" or plan strategically).
I think reading the screen pixels (image search, detecting sprites, etc) is too error-prone and costly in terms of CPU. I think reading the program's memory is probably appropriate: Factorio gives the player a lot of access to their perspective information, and I think it's equal enough for an AI to be able to perceive anything on map view. I'll probably have to start with a world with biters disabled, then A) modify the world and find the changes in memory, and B) make a little "DFHack" type application that shows summary tables of the world state.
Personally I'd want to explicitly avoid the generated but unseen areas of the map, and respect fog of war. But if we're already reading RAM then there's no reason the bot couldn't have omniscience.
I don't love using keyboard/mouse input, but there's two ways I can think to do this:
A) xdotool / xinput utilities that send input events directly to the application. Unfortunately this still relies on opening and closing menus
B) write to program memory after studying what functions are called by the UI. It should be possible to skip opening menus, and add things to the crafting queue or place an item from inventory or ghost onto the map.
I'd want this to emulate client input and get handled by the server just like any unmodified client's input. So I don't want to teleport or write changes to the map contents, I just want to perform player actions.
This is the most interesting part! As a stub though, I'd just start with simple programs that test that the two systems above are working. "Twitch plays Factorio" would be a good start.
I think I'll call it "FHack", since DFHack is seeming like the correct architecture to follow. "Compilatron Therapist" would be the utility that lets you edit waypoints or a to-do list for the automated client. If these applications have a REST API or web interface, then that would be a great dividing line between the "robot" (automated client) and the "AI" (a person, group, or software to decide what to do).
Re: Automating the Engineer / Factorio Client
What is your goal? Do you want to automate the Factorio client and turn it into some auto-playing entity? Or do you want to automate the player sitting in front of the computer and simulate him navigating through the client, looking, clicking, building, doing stuff?
This defines the tools you need and use, and the horizon of the entity that performs playing.
If you automate the Factorio client, you have access to all the internal game data. Stuff the ordinary player never directly sees. The entity that performs playing has full knowledge over the full game state every moment. It can perform every action available by the internal LUA api, which is vastly more than some human player is able to do, and vastly different.
And if you automate or impersonate a player sitting in front of the computer, your automation should have only the knowledge a human player gets. He looks at the map, he sees what is on his map, he sees the machines nearby, and so on. He can use keyboard and mouse and move himself, interact with objects. He has a much more limited horizon and a much more limited view on the game. He has only access to stuff that is visualized by the game.
I once wrote a bot for an MMO to automate some farming. The character left some outpost to create a map instance, it walked to some space on the map, it activated some skills, it collected some loot. There were bots to do this by reading the memory of the game to get all information, and used reverse engineered procedure calls to initiate interactions (walking, skill activation). This was very efficient, because they directly manipulated the game client. They were all banned, because they did this - of course automating the client this way was against the MMO TOS. It wasn't really playing the game, it was abusing the game.
I did a different approach with my bot. I did screen reading to get information about the current game state. To navigate, I used simulated keyboard input and mouse movements. I did exactly the same as a human player would do and would be able to do. Such bots are seen as inferior and tedious to write, because they don't use the in-memory game state, which is always correct, while screen reading is fuzzy. Instead, you have to write a system that's able to perceive the game through the visuals and graphics, which is not 100% reliable. However, I was successful. The game was very good to communicate the game and game state to the player, so I was able to grab the graphics and do some graphics processing to identify objects on the screen. My bot wasn't detected and wasn't banned - it was like just one of the other human players.
Now in Factorio you can do every automation you like of course, because every server is private and cheating through bots isn't a thing for the global player base. But in my opinion, you should decide on what level your automation should be. By using something like screen reading, you simulate playing like a real human person, and you use the same interfaces a human player has available: monitor, eyes, keyboard and mouse. You also need to build a system to recognize things on the screen - you need to not also automate the game, you need to make the system understand what is presented on the screen. On the other hand, by directly using the internal LUA game api, the "understand" part is already done. You just need to automate what the player should actually do. A major challenge with this is to decide which internal action is still something a real player is able to do, and what information is out of his reach and should not be used, even if it is available.
This defines the tools you need and use, and the horizon of the entity that performs playing.
If you automate the Factorio client, you have access to all the internal game data. Stuff the ordinary player never directly sees. The entity that performs playing has full knowledge over the full game state every moment. It can perform every action available by the internal LUA api, which is vastly more than some human player is able to do, and vastly different.
And if you automate or impersonate a player sitting in front of the computer, your automation should have only the knowledge a human player gets. He looks at the map, he sees what is on his map, he sees the machines nearby, and so on. He can use keyboard and mouse and move himself, interact with objects. He has a much more limited horizon and a much more limited view on the game. He has only access to stuff that is visualized by the game.
I once wrote a bot for an MMO to automate some farming. The character left some outpost to create a map instance, it walked to some space on the map, it activated some skills, it collected some loot. There were bots to do this by reading the memory of the game to get all information, and used reverse engineered procedure calls to initiate interactions (walking, skill activation). This was very efficient, because they directly manipulated the game client. They were all banned, because they did this - of course automating the client this way was against the MMO TOS. It wasn't really playing the game, it was abusing the game.
I did a different approach with my bot. I did screen reading to get information about the current game state. To navigate, I used simulated keyboard input and mouse movements. I did exactly the same as a human player would do and would be able to do. Such bots are seen as inferior and tedious to write, because they don't use the in-memory game state, which is always correct, while screen reading is fuzzy. Instead, you have to write a system that's able to perceive the game through the visuals and graphics, which is not 100% reliable. However, I was successful. The game was very good to communicate the game and game state to the player, so I was able to grab the graphics and do some graphics processing to identify objects on the screen. My bot wasn't detected and wasn't banned - it was like just one of the other human players.
Now in Factorio you can do every automation you like of course, because every server is private and cheating through bots isn't a thing for the global player base. But in my opinion, you should decide on what level your automation should be. By using something like screen reading, you simulate playing like a real human person, and you use the same interfaces a human player has available: monitor, eyes, keyboard and mouse. You also need to build a system to recognize things on the screen - you need to not also automate the game, you need to make the system understand what is presented on the screen. On the other hand, by directly using the internal LUA game api, the "understand" part is already done. You just need to automate what the player should actually do. A major challenge with this is to decide which internal action is still something a real player is able to do, and what information is out of his reach and should not be used, even if it is available.
Re: Automating the Engineer / Factorio Client
Actually... indeed, who needs emulation of player actions if we can send key presses / mouse movements and be fine with that?
In my brain, if keeping whole thing as close to real player's actions as possible is one of desired requirements - then using connected client (or discard whole server thing and run singleplayer) is needed anyway.
Client/singleplayer don't need server specific crutches to get imitation of kayboard/mouse input signal - xdotool or whatever other Linux's variant of Auto Hotkey fits your stack&specs will do.
Since at least SOME interaction with UI seem to be unavoidable (you don't think that AI won't ever need calling a train manually to closest station, do you?), this is a really strong option.
As of world perception, if autosaves are not an option - global map screenshots with "1 tile on player map = 1 pixel on global map" resolution, uncompressed, probably with some mod which would paint each entity into MORE unique color, might be the next best option.
Entities on resource patches are a problem though.
Only after Artifact Fish Bone Gears, menacing with spikes of Spoilage, encrusted with ROCKROCKROCKROCKROCK cabochons, would become a thing in unmodded game
DLC's Quality addition is a step in the right direction though...
(daaaaamn, Amazing Cultivation Simulator patch AND Factorio DLC release in same day gave me a headache due to priorities conflict already - and now I really want to strike the earth as well...)