Currently mega bases are limited by UPS, single processor core performance. I always thought that the limit is somewhere far, but after building a mega base, even with almost no mods, I already got to the point, when FPS/UPS drops below 60 on average gaming PC. One CPU core running at 100% while others are relatively idle.
As I got it, new expansion will allow us to build things on several planets. Will those planets still fit in one processing thread or will be multithreaded? Multithreaded sound much better, since it will remove current bottleneck in big base building.
While searching for similar messages I saw only a reference to a fact, that current UPS is mostly limited not by one CPU core speed, but by memory access speed. Is it really so? Also I have no idea how to measure/evaluate this, since CPU load is easy to display, but what causes that load, and how to display it: calculations or delays of memory access is not obvious for me.
Space Age: will different planets be on different threads?
Moderator: ickputzdirwech
Re: Space Age: will different planets be on different threads?
Devs said it, so it's almost certainly true. Optimizations in the new engine might change the numbers slightly but it still needs to go through a lot of RAM every tick.
Also surfaces are currently in the engine, so that's nothing new. They're not in different threads because things can interact across surfaces, so all the time spent tracking that would make threading it very difficult. Poles can connect cross surface, belts can, linked chests can - it's all the same reasons that you can't thread a single surface that well.
It could be done but the work likely wouldn't be worth it.
Also surfaces are currently in the engine, so that's nothing new. They're not in different threads because things can interact across surfaces, so all the time spent tracking that would make threading it very difficult. Poles can connect cross surface, belts can, linked chests can - it's all the same reasons that you can't thread a single surface that well.
It could be done but the work likely wouldn't be worth it.
Re: Space Age: will different planets be on different threads?
I still think that should be an option for surfaces to be able to run in parallel threads. It may limit some operations and make them work a bit differently then now, but current implementation, as I got it, do not represent surfaces like some distant places with limited connectivity. Since there a a lot of ways to transfer things across, surfaces currently are just like two floors of one building. While for distant planets, apparently nothing instant should work, and typical multithread data exchange will work well.
Re: Space Age: will different planets be on different threads?
Enter mods.
Whether things shouldn't interact from a story or lore perspective matters little if the engine allows that interaction. In addition factorio's determinism requirement means that things must happen in a single global order (or differences in order must never be observable). Or in other words: Different threads must not communicate with each other. This prevents most concurrency strategies (e.g. locking) that'd work if interaction between threads only happens rarely.
Thus, for whatever grouping of game objects to threads we choose, one group must never use data from another group for it's own update and it must never raise events (which allow mods to run). This leaves us with few options to parallelize, mostly by entity type, e.g. all solar panels are handled together (but not even in parallel since it's just number of panels * energy per panel), or by system, e.g. belts and pipes where each network is updated in parallel (FFF 364 & 324).
Re: Space Age: will different planets be on different threads?
If server will allow only item and data package transfer between distant surfaces, that happens at the end of each tick this will keep the system deterministic, but will allow communication and item transfer.
When one send a rocket to another planet or space platform (like in Space Exploration) it can arrive several ticks later, it's not important. Event is quite determinictic. At end of tick when the rocket is launched and disappeared (when non modded Factorio places while pots in space port) all it cargo is placed in cross-thread buffer. Will be moved to another thread buffer either while next tick is being calculated or before next tick is started. Same can be done for "space train" if someone worry about Honkai: Star Rail implementation
This method of transfer will not allow shared chests, since they require locking, but butch packet sending is also a good way to transfer items between distant surfaces. In addition, some way of data packages with signals can be added for communication simplification. But even without this feature it will work, since sending single items as data signals commonly used in Factorio, as I saw it.
Also there is a Clastorio mod, that works for multi server someway.
As I said above, current implementation of cross surface data transfer allow much more flexibility, but require single thread a a result. But if we make "distant surfaces", that are much more limited in communication and data transfer means, this will allow easy multithread. Another question, that "such easy solution" might require a lot of internal work, that is not in plans currently.
When one send a rocket to another planet or space platform (like in Space Exploration) it can arrive several ticks later, it's not important. Event is quite determinictic. At end of tick when the rocket is launched and disappeared (when non modded Factorio places while pots in space port) all it cargo is placed in cross-thread buffer. Will be moved to another thread buffer either while next tick is being calculated or before next tick is started. Same can be done for "space train" if someone worry about Honkai: Star Rail implementation
This method of transfer will not allow shared chests, since they require locking, but butch packet sending is also a good way to transfer items between distant surfaces. In addition, some way of data packages with signals can be added for communication simplification. But even without this feature it will work, since sending single items as data signals commonly used in Factorio, as I saw it.
Also there is a Clastorio mod, that works for multi server someway.
As I said above, current implementation of cross surface data transfer allow much more flexibility, but require single thread a a result. But if we make "distant surfaces", that are much more limited in communication and data transfer means, this will allow easy multithread. Another question, that "such easy solution" might require a lot of internal work, that is not in plans currently.
Re: Space Age: will different planets be on different threads?
No.
There are no plans on having different surfaces to be on different threads. There are multiple shared resources like random generators, number allocators, pollution and production statistics, various system managers that would have to be handled first before we would even consider multithreading entities between surfaces. Due to MP far more more important than performance is game determinism and if we cannot guarantee that in a multithreaded we will fall back to single threaded. All questions "Will X be multithreaded" (insert anything as X) that skip the shared state issues and skip discussion about how we could solve those issues, and just go straight to "the game would run faster" i am going to say is just rude for lack of understanding why it wasnt made multithreaded yet.
Random generators could be made per surface, pollution and production statistics could have thread local accumulators that are added to the shared state after all threads are finished (assuming there are no floating point variables involved because changing order of floating point variable operations may result in floating point errors accumulating differently causing determinism issues) or could be also made per surface, by far the worst problem is with number allocators because some items need a unique item number and if items created on different surfaces would receive the same item number, or worse would receive their item number in a non deterministic way, there would be desyncs. Construction robots build entities which raises events, mods expect events to be raised immediately, mods may touch other surfaces during those events, mods may remove or create other surfaces during those events, there are countless issues to solve, we cannot fix all of them in one go. We just consider Surfaces to be a bag of chunks with entities on them. If anything is multithreaded, it is not made with a spatial division (like different surfaces updated by different threads), but with a system specific divisions (interconnected fluidboxes are part of one fluid system which is updated as one piece but fluidboxes of different systems may be updated by separate threads, interconnected belts are part of one transport line group that is updated as one piece but transport lines that are in different groups may be updated by different threads).
There are no plans on having different surfaces to be on different threads. There are multiple shared resources like random generators, number allocators, pollution and production statistics, various system managers that would have to be handled first before we would even consider multithreading entities between surfaces. Due to MP far more more important than performance is game determinism and if we cannot guarantee that in a multithreaded we will fall back to single threaded. All questions "Will X be multithreaded" (insert anything as X) that skip the shared state issues and skip discussion about how we could solve those issues, and just go straight to "the game would run faster" i am going to say is just rude for lack of understanding why it wasnt made multithreaded yet.
Random generators could be made per surface, pollution and production statistics could have thread local accumulators that are added to the shared state after all threads are finished (assuming there are no floating point variables involved because changing order of floating point variable operations may result in floating point errors accumulating differently causing determinism issues) or could be also made per surface, by far the worst problem is with number allocators because some items need a unique item number and if items created on different surfaces would receive the same item number, or worse would receive their item number in a non deterministic way, there would be desyncs. Construction robots build entities which raises events, mods expect events to be raised immediately, mods may touch other surfaces during those events, mods may remove or create other surfaces during those events, there are countless issues to solve, we cannot fix all of them in one go. We just consider Surfaces to be a bag of chunks with entities on them. If anything is multithreaded, it is not made with a spatial division (like different surfaces updated by different threads), but with a system specific divisions (interconnected fluidboxes are part of one fluid system which is updated as one piece but fluidboxes of different systems may be updated by separate threads, interconnected belts are part of one transport line group that is updated as one piece but transport lines that are in different groups may be updated by different threads).
-
TheKillerChicken
- Long Handed Inserter
- Posts: 80
- Joined: Sat Mar 02, 2019 7:06 am
- Contact:
Re: Space Age: will different planets be on different threads?
The thing with Factorio is that it can use 10 trillion CPU cores @5.7GHz and will probably only get a 1-3% increase. This game specifically depends on the DRAM bandwidth/latencies. Wube is smart at what they do and know their code. If Multicore was beneficial, they would have integrated it a long time ago. Wube also stated that Factorio is SMT compliant. I do see the game and server threads swapping around on my 32 logical cores on both the server and the client. I also will add: Multithreading is beneficial for auto-CAD, datacentres, code compiling, cloud computing for science, and many other advanced tasks whereas games are focused on single threaded processing, or in Factorio's case, DRAM bandwidth. The more DRAM you have, the faster this game will perform. Some computers can populate 16 DRAM modules, maybe even more. My gamer and server has 16-slots for DRAM.