So, I've talked to some different folks about this. At the end of the day, it's a mod. A great mod, but will always be a mod. Factorio is a game, not a science, and while abstraction for some of us sounds like a huge amount of fun, for the vast majority it's more chore than game playing.
Abstraction done right though would be fantastic. Coupled with mods which visually represent factories as singular warehouses, it really adds a lot. This latter concept I think is pretty important, as abstraction isn't just about UPS, but also about removing clutter.
There is dupebox, which in theory is cool, but I could never get it to work properly. Perhaps it's been fixed and I should try again. I'd be curious how it fares under a load test as well.
Search found 19 matches: Mods.factorio
Searched query: +Mods +factorio
Return to “Optimization idea: abstraction”
- Sat Oct 15, 2022 1:14 pm
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
- Mon Oct 03, 2022 9:06 am
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
That has been already discussed here already thoroughly and yes, there has been even a mod, see:mrvn wrote: Sat Oct 01, 2022 10:20 pm You can run one instance of the abstraction for real and place views of that instance wherever there is a copy. For an example how to mod this look at how factorissimo can show you the inside of the factory building.
Using factorissimo as base might also be an option. You build your factory inside the building and then create copies of the building that will have the same input/output without actually running the inside. You just run one instance and mirror the input/output to all copies.
As for the UPS concerns: Removing the inputs and creating the outputs in LUA is costly. So you better make the inside of your factory big. A simple yellow belt column of furnaces might be faster than faking one with LUA. But that just means you need to go bigger and put a blue belt comlumn inside or something.
My implementation should be exact, except for floating-point shenanigans. But no difference in item count, possibly small epsilon in electricity. I wouldn't implement fluids thought, it would change flow behaviour significantly.ssilk wrote: Sun Sep 25, 2022 3:18 am Btw: links to mentioned mods in this thread: search.php?keywords=Mods.factorio&t=82060&sf=msgonly
- Sun Sep 25, 2022 3:18 am
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Cycles: Merge entities, mod for performance idea
True for unlimited precision, but not so true, if you use for example fixed-point floating numbers, which are in fact just integers shifted right. For those the above rule is still correct.Nidan wrote: Sat Sep 24, 2022 2:33 am Even the simplest rules like (a + b) + c = a + (b + c) do not hold for floating point math. Thus, summing the electricity consumption of all machines and subtracting that from the larger network will likely have a different result than accounting for each machine individually.

See for example https://www.factorio.com/blog/post/fff-64
where they used it for other reasons.
Attention: Not saying the solar panels are calculated in fixed-point arithmetic!
The Dupebox mod is in my eyes stiil the most promising try resulting from this thread.
Btw: links to mentioned mods in this thread: search.php?keywords=Mods.factorio&t=82060&sf=msgonly
- Sat Sep 24, 2022 5:20 am
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Cycles: Merge entities, mod for performance idea
I think this is doing things similar to what you describe, the way the cycle is defined differs but i saw similarity in the grouping of machine in a smaller number of entity based on cyclic behavior.
https://mods.factorio.com/mod/factory_modules
Those i feel have less similarities but are based on the same idea of grouping entities to reduce the need for computation :
https://mods.factorio.com/mod/modular-buildings-redux
https://mods.factorio.com/mod/Manifolds
- Fri Jan 29, 2021 2:43 am
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
ok now I'm not sure if you're trolling me because I absolutely mentioned Schall Machine Scaling.blazespinnaker wrote: Fri Jan 29, 2021 1:58 am However, this can be alleviated somewhat if OP removes the constraint of having the duped factories adjacent. Still, you run into this sync thing which really, uber furnaces and assemblers seems like a better approach with pretty much all the same aspects. The benefit of something like schalling's mod over this would be a lot less finnicky silliness that is going to come in play as this is used at scale.
https://mods.factorio.com/mod/SchallMachineScaling
I'd also be curious to see if the Mod author has tried using this at scale as well (say, around 100k spm). I tried, but the mod kept on crashing so I gave up.
Steino, let me know if you get a notification for this quote. Trying this instead of double posting.
I'm using it with Bob's for 1 million SPM..
- Fri Jan 29, 2021 1:58 am
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
Agreed. Maintaining some of the current challenges through abstraction helps maintain a lot of the fun of the game, IMHO.Qon wrote: Thu Jan 28, 2021 8:36 pm The point of the abstraction idea (to me at least) is to not make the game easier. Faster assemblers just makes the game of logistics and game of space management and factory design x times easier when you make assemblers x times faster.
That said, I think once you get to the scale where abstraction is UPS beneficial, I don't think these challenges really exist. You likely have many robots and practically infinite supplies from your mall. At that point, it's less about production, and more about logistical management of resources (miners + trains)
Using modified modules or uber assemblers is probably a reasonable stand in for most folks and is a common reply I've found when I've brought up this topic. It certainly will work just as fine for many purposes.
Abstraction is likely only interesting to a niche few, from what I've seen. For example, if you have a particular interest in building optimal unitary factories for various things.
There is an argument for using abstraction to remove some of the copy/paste element to the game, but that's a different discussion and really a different game.
Now that you mention it, I have to say, you're exactly right. I don't see the practical difference.
However, this can be alleviated somewhat if OP removes the constraint of having the duped factories adjacent. Still, you run into this sync thing which really, uber furnaces and assemblers seems like a better approach with pretty much all the same aspects. The benefit of something like schalling's mod over this would be a lot less finnicky silliness that is going to come in play as this is used at scale.
https://mods.factorio.com/mod/SchallMachineScaling
I'd also be curious to see if the Mod author has tried using this at scale as well (say, around 100k spm). I tried, but the mod kept on crashing so I gave up.
Steino, let me know if you get a notification for this quote. Trying this instead of double posting.
- Wed Jan 27, 2021 7:49 pm
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
Dimava wrote: Thu Jan 21, 2021 12:54 pm I made a mod for this
https://mods.factorio.com/mod/dupebox
please test and review
- Sat Jan 23, 2021 10:44 pm
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
woah, totally missed this. Not enough fanfaressilk wrote: Sat Jan 23, 2021 6:51 amLooks cool. I’ll try it out and replace some structures in my biggest map with this.Dimava wrote: Thu Jan 21, 2021 12:54 pm I made a mod for this
https://mods.factorio.com/mod/dupebox
please test and review

Trying now..
edit: played around with it. Looks like a very fantastic start. My one comment is that the way I did it with my hack on factorissimo, the duped factories are not required to be adjacent. This is important as it doesn't constrain train network construction, which at least for me, was the whole point of the mod.
I'm sure others may have different goals, however, and it does help allow for general abstraction which enables DRY factories. This mod will absolutely help cut down on the "relentless copy/paste" problem I outlined above: viewtopic.php?p=531915#p531915
I didn't test time usage at scale for the mod, but theoretically this mod will help folks with slower computers produce the same as those with faster computers (but not using the mod, obviously).
Which IMHO are both significant achievements.
Though as I mentioned elsewhere, the requirement to have same input/output seems rather artificial. There's a phrase for it in software engineering, "leaky abstraction". But again, others may have a different perspective, and as Spolsky said - all non-trivial abstractions, to some degree, are leaky.
I was a bit saddened to see that you had to resort to using remnants. Was that because there was no alternative? If so, perhaps we can ask Wube to enable some technical approach to using their tile graphics.
- Sat Jan 23, 2021 6:51 am
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
Looks cool. I’ll try it out and replace some structures in my biggest map with this.Dimava wrote: Thu Jan 21, 2021 12:54 pm I made a mod for this
https://mods.factorio.com/mod/dupebox
please test and review
- Thu Jan 21, 2021 12:54 pm
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
- Sat Jan 02, 2021 11:02 pm
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
+1
Factorrismo like addition would allow for much larger scale factories in vanilla. By allowing a player to identify inputs and outputs, a factory could be promotable to abstractable after it had proved itself over some period of time, perhaps time proportional to the complexity of the factory itself.
Perhaps too much volatility in production curves would result in a factory unable to be promoted to abstractable. This would create a very unique challenge in abstracted factory design.
The potential problem at that point would be that if these factories became too easy to put down, you could quickly launch a rocket within very little time.
I think it'd be very reasonable that abstraction shouldn't have any impact on resource cost or time / complexity to place. It would be no different than placing a blueprint.
In fact, you could even punish the player by not allowing a previously abstracted factory to be used even a little bit until it was fully placed. This is a huge penalty and in this way, abstracted factories would actually be negative in time to launch a rocket.
In addition, you could require that the player first terraform any land under the factory such that it has a proper factory floor. Also, walls, to provide a barrier against biters. Other things, such as roboport coverage with mininum # of robots could be required as well.
Any biter getting through a wall would just halt the factory. It would be tempting to take factories out of simulation mode for the player and updating them, but the impact on UPS could be frightening in some cases. Better would be to just let the biters run rampant through a bunch of unpowered entities.
The end benefit would be truly massive. They would allow for much greater scale factories via infinitely more efficient UPS usage, and maybe provide a minor added benefit for newer players in terms of managing complexity.
Allowing trains in factories would be cool
Something may be required here to do faster large scale terraforming tho.
Of course, what would we do with all that scale?
I think adding a bunch of very expensive infinite research would be a terrific addition to go along with this feature. See this thread here for more on that - viewtopic.php?f=5&t=93333&p=528820&hili ... ch#p528820
edit to add: I spent some time on this and I was able to enhance the factorissmo2 mod to do much of the above. https://mods.factorio.com/mod/Factoriss ... c7db4f5997
It's actually a pretty cool mod. Can't wait to use it in more megabase mass scale.
Factorrismo like addition would allow for much larger scale factories in vanilla. By allowing a player to identify inputs and outputs, a factory could be promotable to abstractable after it had proved itself over some period of time, perhaps time proportional to the complexity of the factory itself.
Perhaps too much volatility in production curves would result in a factory unable to be promoted to abstractable. This would create a very unique challenge in abstracted factory design.
The potential problem at that point would be that if these factories became too easy to put down, you could quickly launch a rocket within very little time.
I think it'd be very reasonable that abstraction shouldn't have any impact on resource cost or time / complexity to place. It would be no different than placing a blueprint.
In fact, you could even punish the player by not allowing a previously abstracted factory to be used even a little bit until it was fully placed. This is a huge penalty and in this way, abstracted factories would actually be negative in time to launch a rocket.
In addition, you could require that the player first terraform any land under the factory such that it has a proper factory floor. Also, walls, to provide a barrier against biters. Other things, such as roboport coverage with mininum # of robots could be required as well.
Any biter getting through a wall would just halt the factory. It would be tempting to take factories out of simulation mode for the player and updating them, but the impact on UPS could be frightening in some cases. Better would be to just let the biters run rampant through a bunch of unpowered entities.
The end benefit would be truly massive. They would allow for much greater scale factories via infinitely more efficient UPS usage, and maybe provide a minor added benefit for newer players in terms of managing complexity.
Allowing trains in factories would be cool

Of course, what would we do with all that scale?

edit to add: I spent some time on this and I was able to enhance the factorissmo2 mod to do much of the above. https://mods.factorio.com/mod/Factoriss ... c7db4f5997
It's actually a pretty cool mod. Can't wait to use it in more megabase mass scale.
- Thu Oct 15, 2020 3:20 pm
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
Not exactly sure what 1 - 4 are responses to, but I'll try to answer anyways.
And to supply the 300x3 assembler with items you need a lot of inserters (which are more costly in terms of CPU than assemblers). Or you need some 300x1 inserter. Fed from some 300x1 logistics chest or some kind of super belt. And now the speedup becomes limited or you would have to incorporate it all into a single unit. But now you have completely abandoned actually making your design, you literally have a big box that just produces things super fast and all you need is the space for it. You don't need to think about how to incorporate roboports and radars into the design (which is needed when making exceptionally large structures to make them buildable from map, which is a part of the challenge of building at this scale).
And this only covers a single step, you are limited to a single recipe which limits the speedup. Unless you want to make an assembler that takes iron and copper plates and spits out green chips. Or ores and spits out science packs. But then you have removed the whole building and designing aspect of the game. This method removes the game from the game. I don't like it.
Also you can't call C++ librararies from Lua code executed by Factorio. That would break the security model and the determinism guarantees and everything else.
Also I prefer the game to look good if possible.
Your suggestions don't provide nearly enough benefits compared to all the compromises they introduce and compared to the much, much faster abstractions I'm thinking of implementing. If I can just do it. If I can active=false all entities and just do a handful of API calls to remove a few tiles of ores and just change the research progress bar by a calculated amount once per minute I can achieve about 0.0% CPU utilisation for an entire gigabase, including the mod. And it will all be dynamic with no interaction by the player and "no" restrictions.
Listen, I'm not talking about speeding things up, I'm talking about eliminating computations. Like almost all of them. Making things 100 times faster is not enough, I'm going for something closer to constant time compution for any factory size and for any time step. The ideal is to be able to set game.speed = 100 and have it run at 6k UPS with a 1M SPM factory. That is going to run into many practical issues which means that might not be possible. But I want the closest thing to that and which is still possible. Without altering the factory building process. If I can calculate that a mining outpost will last for 100 hours at the rate I'm mining from it (total/mining rate) and that it produces X science/minute, then why not just use that calculation instead of simulating all the steps? As long as the player playing the game can't really notice any difference then that should be good enough.
I took into acount that you might have meant a faster and larger assembler. But it doesn't change things that much. Larger assemblers prevents you from using your own blueprint layouts, ie it limits your engineering and kind of forces you into using someone elses layout/blueprint. It also forces you to rework your factory to use these kinds of entities and you can't enable or disable the mod on the fly.yagaodirac wrote: Thu Oct 15, 2020 2:14 pm 1, I didn't mean a 100x assembler. If the recipe is 100x assembler into a big one, then the big one is 300x3 in size, and 100x in speed. When it's built on ground, it should look like 100 separated original things in a line. It's not cheating at all. It's cpu friendly only. Nothing logical changes. You literally misunderstood me.
And to supply the 300x3 assembler with items you need a lot of inserters (which are more costly in terms of CPU than assemblers). Or you need some 300x1 inserter. Fed from some 300x1 logistics chest or some kind of super belt. And now the speedup becomes limited or you would have to incorporate it all into a single unit. But now you have completely abandoned actually making your design, you literally have a big box that just produces things super fast and all you need is the space for it. You don't need to think about how to incorporate roboports and radars into the design (which is needed when making exceptionally large structures to make them buildable from map, which is a part of the challenge of building at this scale).
And this only covers a single step, you are limited to a single recipe which limits the speedup. Unless you want to make an assembler that takes iron and copper plates and spits out green chips. Or ores and spits out science packs. But then you have removed the whole building and designing aspect of the game. This method removes the game from the game. I don't like it.
Thanks, it's going to be non-trivial. But I believe I can do it with enough time and effort, with some slight imperfections allowed.
Not sure what I shouldn't talk about here... maybe you can say what you are talking about instead of saying "this" and quoting my entire post!?yagaodirac wrote: Thu Oct 15, 2020 2:14 pm 3, You shouldn't talk about this idea here. You know amd and intel? Ask them for the solution. But in fact, I just started knowing that, lua can call functions written in cpp. If you mod a interface in lua, take the advantage of cpp and graphics card, with some specially designed algo, I think it might have a chance to do what you want.
Also you can't call C++ librararies from Lua code executed by Factorio. That would break the security model and the determinism guarantees and everything else.
There are mods that reduce the graphics. But Factorio is RAM bandwidth limited and CPU limited. The graphics are to some extent handled by the CPU, but reducing animations isn't going to give you 200k SPM. Basically only things on screen this tick are affected by these kinds of changes, and most of it is handled by the GPU anyways. And what is on screen is nothing compared to the 99% of the factory that is outside your screen, which is what I want to abstract away.yagaodirac wrote: Thu Oct 15, 2020 2:14 pm 4, The most practical way. Mod to remake everything, remove all the useless things. Simplify all the images. If you read through the source code of __base__, you would find a lot things that waste cpu time. If you get rid of all the decorations, all the transition sprites, at least you save a lot cpu time which is on the graphics task in vanilla. This is not cheating at all, it only make your game ugly. I know you wouldn't mind this.
Also I prefer the game to look good if possible.
Your suggestions don't provide nearly enough benefits compared to all the compromises they introduce and compared to the much, much faster abstractions I'm thinking of implementing. If I can just do it. If I can active=false all entities and just do a handful of API calls to remove a few tiles of ores and just change the research progress bar by a calculated amount once per minute I can achieve about 0.0% CPU utilisation for an entire gigabase, including the mod. And it will all be dynamic with no interaction by the player and "no" restrictions.
Listen, I'm not talking about speeding things up, I'm talking about eliminating computations. Like almost all of them. Making things 100 times faster is not enough, I'm going for something closer to constant time compution for any factory size and for any time step. The ideal is to be able to set game.speed = 100 and have it run at 6k UPS with a 1M SPM factory. That is going to run into many practical issues which means that might not be possible. But I want the closest thing to that and which is still possible. Without altering the factory building process. If I can calculate that a mining outpost will last for 100 hours at the rate I'm mining from it (total/mining rate) and that it produces X science/minute, then why not just use that calculation instead of simulating all the steps? As long as the player playing the game can't really notice any difference then that should be good enough.
- Tue Oct 13, 2020 11:54 am
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
I've been thinking of making a mod for this (or something similar at least, to make it possible). I'm almost far enough with my specification to start coding it. But I'm too busy at the moment to do it. But it might be my next big mod project.
This post is going to be a bit of a raw brain dump that might not make sense if you haven't read the topic, or maybe even then.
The mod API doesn't give view position, but character position and player selected gives enough that it is almost achievable to know where the players are looking. So for that reason I'm fine with players seeing frozen machines if they use map view and accidentally/deliberately don't mouse over any entities.
Power draw might be tricky to get right, both to estimate, it's probably ok to approximate that with slightly less precision (so within a few %) instead of aiming to get it almost perfect like production stats. People overbuild on solar and nuclear on scales like 25% power excess or sometimes much more.
I think it's ok to only start measuring (abstraction preparation) when input buffers are filled and power is always at 100%. Should be reasonable to maintain for a few minutes.
Measurement really only needs to check that power is consistent every frame. Inputs and outputs are buffered and only need to be checked sometimes. So measurement is almost free, the heavier calculations only happen once or twice for the system at start and end of measurement. Finding and maintaining areas to abstract is another part that could require a fair ammount of work. But this isn't time critical and can be spread out over many tick, over minutes or hours. Doing it spread out over more time just delays the time it takes for a completed factory unit to be abstracted for the first time after being completed/modified and then up and running. For big factories the cell units that have not been modified by the player in the last 10 minutes is probably 90% or more, typically.
I'm not planning on making it completly impossible to "abuse" into behaving differently when abstracted from what it otherwise would do. It aims to do "the same thing as previous minute, repeated over the next hundred minutes". If a factory cell produces one thing at odd hours and other things at even hours then it should be accurate in it's production count, it just wouldn't switch to the other behaviour if there's circuits that route things differently certain times, or factory cells with circuits wouldn't be abstracted maybe.
Buffered items should be counted or assembler types should have crafts counted. Or buffered items should be detected and not be part of the measured and cached output for repetition. So you don't get free outputs because buffers were emptying while measuring. But otherwise it doesn't need to be impossible to abuse with all mod configurations etc. It just shouldn't accidentally give the wrong output when abstracted. If people want to abuse or trick the abstraction then they might as well just install Creative Mod or use infinity chests so it's not like real cheats aren't available if my mod isn't perfect...
The parts that will have items drawn from and outputted to will have to be chests. There items can be created or destroyed faster by Lua than Factorio by itself since items are not inserted individually but filled/drained completely instead with an API call, and big chains of interactions of entities can be abstracted away.
Ores on ground and labs/research also need to be handled. Especially ore deposits since most of material flow happens with mining ores and then smelting so it would be a great boost if even just that was sped up.
But a more limited form that skips cells with logistics chests in it and doesn't pay as much attention to accuracy of the speed, power draw and splits the world into simpler disjointed cells etc might be possible to prototype up a bit quicker and then I'll see if the project is worth improving up to full spec.
This post is going to be a bit of a raw brain dump that might not make sense if you haven't read the topic, or maybe even then.
Goals
My goal is to maintain 60 UPS pretty much regardless of factory size and having production/minute be as close to vanilla as possible but not really paying attention to /tick production stats. It wouldn't be exactly as the devs intended it with each item actually being accounted for precisely. And I do love that about Factorio. But the game is great without it also and it doesn't hurt the game if it's implemented as a mod. If players can continue to play and expand their factories instead of abandoning them because they start dropping UPS then at that point the tradeoff seems to be pretty ok.Features and requirements spec:
I absolutely need active = false to mean that the entities don't use CPU cycles. This makes it "simple" and fast to enable/disable abstraction quickly. And belts and bots etc will stay in position carrying material as if it was always running. With the assumption that the running factory looks "the same" all the time it should be OK to just pause the belts and bots and then unpause when a player comes to inspect. If materials have dried up at inputs inputs then that's just some buffered items that can be accounted for and won't really affect actual production but just when those things get used.- It should just be plug-and-play, install the mod and after a while (if it's added to an already running big base) UPS should increase so that you can play without slowdowns any more.
- No input output marking
- to greatests extent should work with any building style
- no new entities
- This of course excludes Factorissimo style buildings and building with blurry glass roofs etc. The actual entities represent themselves, just active=false'd.
- This also means the player have to manage the factory just the same as if there really was proper simulation of all the items. No boxing of items that reduce belts, all the same space requirements, no difference to blueprints or calculations need to be made etc. Should "feel" like the real thing, except better UPS.
- should be compatible with most mods
- should be able to handle belts, pipes, bots and trains in local networks. Maybe even new logistics methods like Klonans transport drones could be handled, maybe even without specific code to handle those if they are used within local production cells, but that might be strecting it.
- shouldn't be need recomputation for non-affecting edits
- should enable and disable parts as seamlessly as possible, instantly. After initial measurements the goal should be that only what you are currently working on and looking at is working. As soon as you look away it becomes abstracted and the new thing you look at is running instead.
- Less power means slower production, abstraction just adjusts proportionally (depends on power abstraction precision).
Limitations of requirements:
For this to be achievable I'm fine with minor losses of accuracy. A 20.00k SPM factory should still produce 20.00k SPM, losing or gaining less than 1% is not of great concern if I can get everything else good enough. But it should be as close as reasonably possible for number of items consumed and produced. The speed of production depends on power (might be less precise) and isn't as critical to get close to perfect imo, as long as it is within a few %. If things run 3% faster or slower, who cares if it's almost count perfect? The inputs and outputs might be constrained by some other non-abstracted factory anyways so in that case the abstraction could run it 100 times faster and you would just get "spikier" production but not actually any faster production. Sources of inaccuracy would be productivity modules. 2 things in might lead to 2 things out, but 3 things in could give 4 things out. The productivity bars could be read or the output calculated, but then you are talking about possible doing abstracted output higher than what you have measured so it just seems like a lot of work that generates more work for minor accuracy increase but at the risk of accuracy decrease if done with wrong assumptions and more CPU time by the mod. So not worth it I think, just measure for a bit longer for more accurate results.The mod API doesn't give view position, but character position and player selected gives enough that it is almost achievable to know where the players are looking. So for that reason I'm fine with players seeing frozen machines if they use map view and accidentally/deliberately don't mouse over any entities.
Power draw might be tricky to get right, both to estimate, it's probably ok to approximate that with slightly less precision (so within a few %) instead of aiming to get it almost perfect like production stats. People overbuild on solar and nuclear on scales like 25% power excess or sometimes much more.
I think it's ok to only start measuring (abstraction preparation) when input buffers are filled and power is always at 100%. Should be reasonable to maintain for a few minutes.
Measurement really only needs to check that power is consistent every frame. Inputs and outputs are buffered and only need to be checked sometimes. So measurement is almost free, the heavier calculations only happen once or twice for the system at start and end of measurement. Finding and maintaining areas to abstract is another part that could require a fair ammount of work. But this isn't time critical and can be spread out over many tick, over minutes or hours. Doing it spread out over more time just delays the time it takes for a completed factory unit to be abstracted for the first time after being completed/modified and then up and running. For big factories the cell units that have not been modified by the player in the last 10 minutes is probably 90% or more, typically.
I'm not planning on making it completly impossible to "abuse" into behaving differently when abstracted from what it otherwise would do. It aims to do "the same thing as previous minute, repeated over the next hundred minutes". If a factory cell produces one thing at odd hours and other things at even hours then it should be accurate in it's production count, it just wouldn't switch to the other behaviour if there's circuits that route things differently certain times, or factory cells with circuits wouldn't be abstracted maybe.
Buffered items should be counted or assembler types should have crafts counted. Or buffered items should be detected and not be part of the measured and cached output for repetition. So you don't get free outputs because buffers were emptying while measuring. But otherwise it doesn't need to be impossible to abuse with all mod configurations etc. It just shouldn't accidentally give the wrong output when abstracted. If people want to abuse or trick the abstraction then they might as well just install Creative Mod or use infinity chests so it's not like real cheats aren't available if my mod isn't perfect...
The parts that will have items drawn from and outputted to will have to be chests. There items can be created or destroyed faster by Lua than Factorio by itself since items are not inserted individually but filled/drained completely instead with an API call, and big chains of interactions of entities can be abstracted away.
Ores on ground and labs/research also need to be handled. Especially ore deposits since most of material flow happens with mining ores and then smelting so it would be a great boost if even just that was sped up.
Difficulties
Most of the hard things to do will be to maintain data structures over a dynamic factory, find input outputs, not spend too much time on checking if things interfere with earlier abstractions, finding connections, maintaining hierachies and overlapping abstractions, simulating power draw etc, splitting work over several ticks efficiently in Lua...But a more limited form that skips cells with logistics chests in it and doesn't pay as much attention to accuracy of the speed, power draw and splits the world into simpler disjointed cells etc might be possible to prototype up a bit quicker and then I'll see if the project is worth improving up to full spec.
- Sun Mar 15, 2020 4:29 pm
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
Factorio doesn't update assembler progress every tick. Just when product is made or something else happens. At least that what one of the devs told me on reddit when I brought up the topic.
As for the "abstraction".
To even make it feasible I think it would require the user to specify an area and the graphics to be changed to not reflect the internal state, like a warehouse or something.
The big problem though is how to produce stuff. For a simple case it could calculate max rate and output items based on proportion of input items. Though in general it's problematic for many reasons:
- You CANNOT simulate the build to see whether it will work forever and never stop. If belts are turing complete (i think there was a post on reddit where someone made logic gates with belts and splitters) you cannot do it in general in any way.
- The build may have different parts that produce different items based on how many items you supply. For example having an overflow when there's >80% of some fluid going into something else.
- The general performance depends on small stuff, like for example on which side of the belt items are being input.
- Circuit networks in general are impossible to abstract because circuits are turing complete.
The only solution I see is having predefined entities that skip steps, but it sounds like a pain to balance and may impact immersion. Mods are for that.
One could also make something more dynamic based on heuristics like from Max Rate Calculator but it would be broken by design.
edit.
Having recipes which take more and produce more in one iteration would already help with UPS a bit. See https://mods.factorio.com/mod/SchallRecipeScaling But it has to be opt-in as it increases latency
As for the "abstraction".
To even make it feasible I think it would require the user to specify an area and the graphics to be changed to not reflect the internal state, like a warehouse or something.
The big problem though is how to produce stuff. For a simple case it could calculate max rate and output items based on proportion of input items. Though in general it's problematic for many reasons:
- You CANNOT simulate the build to see whether it will work forever and never stop. If belts are turing complete (i think there was a post on reddit where someone made logic gates with belts and splitters) you cannot do it in general in any way.
- The build may have different parts that produce different items based on how many items you supply. For example having an overflow when there's >80% of some fluid going into something else.
- The general performance depends on small stuff, like for example on which side of the belt items are being input.
- Circuit networks in general are impossible to abstract because circuits are turing complete.
The only solution I see is having predefined entities that skip steps, but it sounds like a pain to balance and may impact immersion. Mods are for that.
One could also make something more dynamic based on heuristics like from Max Rate Calculator but it would be broken by design.
edit.
Having recipes which take more and produce more in one iteration would already help with UPS a bit. See https://mods.factorio.com/mod/SchallRecipeScaling But it has to be opt-in as it increases latency
- Fri Mar 13, 2020 5:59 am
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
Sorry, but that's quite offensve. I've modded several stuff, helped improving many mods and libs. And wrote a little mod myself.eradicator wrote: Wed Mar 11, 2020 9:31 am I'm sorry, but as a modder that is all laughable to be polite. No personal offense meant, i realize you're not a modder and just enthusiastic.
(Maybe I'm a bit out, of what can be done with a mod, since I have no time for that since 2 years, but the abilities of mods have improved, not declined.)
Of course I know how difficult this is, but to say "impossible" is too simple, after scribling my ideas about it on half a page.
That's just offensive.
Laughable is, how you think that a non-deterministic behavior should be simulated in a game, where nearly everything is about that; it's the essence of Factorio. Devs are with my opinion. (*)
This is a too theoretical discussion, so I deleted my statements to your arguments (some have already been answered), because nobody knows how it really is, until someone tried. Yes, I know that this is far beyond, of what a mod should do (but does anybody mean, this will be implemented into vanilla the next 2-3 years???) and there are many difficulties, but still possible and no need for offense.

(*) And to repeat the idea (because I think it's misunderstood):
Detect that part of production, that is deterministic. That is only possible for belt driven production and when that is full production or no production (and some more preconditions). Simulate full production and look, if this fragile deterministic state is left somehow to switch simulation off.
And you might see now (**), this is a kind of game in game to keep that fragile determinism running. And not many would love that, because it is quite complex. But that part of game in game is the only sense I can see in this suggestion, I called that concept "factory streets" and it could be read as fifth post in this thread.
... And it would help, if the devs would implement, that you can turn off complete parts of a factory just by turning the power off (inserters doesn't turn off when unpowered). Because belts are not power driven, the simulated part could recover and simulation can restart.
(**) it is difficulty to see, because it is even a game in game, without the whole optimization stuff: A factory street - in my sense of view - is some factory part, that needs to be marked out and feed with full input belts and there is no full compression on output belts. And in that case this part has an inbuild production/speed/efficiency-bonus.
- Tue Mar 10, 2020 1:34 pm
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
Sounds kind of like Whistle Stop Factories.
https://mods.factorio.com/mod/WhistleStopFactories
https://mods.factorio.com/mod/WhistleStopFactories
- Tue Mar 10, 2020 2:08 am
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
I would not expect this to be ever implemented.
But it can be approximated by modding in a huge assembler or furnace and corresponding recipes for batch-processing N times the ingredients for N times the result.
Combine that with loaders (vanilla or Miniloader) for input and output to get most of the blackbox optimization.
But it can be approximated by modding in a huge assembler or furnace and corresponding recipes for batch-processing N times the ingredients for N times the result.
Combine that with loaders (vanilla or Miniloader) for input and output to get most of the blackbox optimization.
- Sat Mar 07, 2020 11:05 am
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
TL;DR
Parts of a factory can be simulated, if the conditions that influence that part have very specific values. E.g. input belts needs to be fully compressed, and many more. In that case it is safe to switch to a simulation.
Long read:
Well, I think this is a way we should think about deeper.
What I’m thinking here is also Factorissimo, but instead of just one building it should have the “real size”.
So what I see here are some possible solutions, which would simulate a realistic production process for very large quantities but switch back to the realistic simulation for low quantities.
How could that be done?
First I think about the concept of factory-streets. Like a gigantic smelting array. This can be simulated very exactly. See for example the max rate calculator https://mods.factorio.com/mod/MaxRateCalculator
But it needs a lot of preconditions to make sure, the simulation is always exact:
- all input belts are always compressed
- enough free space on output belts to put items on
- inserter are fast enough to feed/empty the smelters
- there is no complex belt stuff, like when it is possible that two different items could be on one lane
- same is with mixed input lanes. As soon as another item on a lane is imputed it needs to switch back to simulation.
- there are no circuits involved
- there is always 100% power available (well I think this can be simulated up to some degree)
- pipes are never filled below some level X
- player is not changing anything inside that factory street are
- the theoretical input/output is equal to the current input/output
- and of course some more preconditions.
You see it’s difficult to achieve those conditions so I would like to introduce a “factory-street-area”. Normally this area is hidden with some kind of industrial roof (game doesn’t need to simulate the inside), but when the player enters, the roof becomes transparent. When you want to built inside such an area the factory street switches to the normal mode and everything behaves as it should. You can then built inside the factory street, but you cannot built some constructions. For example you accidentally side-insert a different item into a belt-lane. You cannot do circuits, or you forgot to power a device.
But when you go outside and the above conditions are meet, then it switches to simulation mode after a while. After a while means: the theoretical input/output needs to be more or less equal to the current measured input/output.
And as soon as one condition isn’t meet anymore the factory street switches back to normal mode. That is likely, if the output is blocked or not enough input comes in, or if power fails. But for those cases that input is missing or output is blocked it’s already so, that the inserters and entities go to sleep. So not too much CPU used. The factory street covers only the case, when it is running with full speed.
So what does this bring?
- useless if you only have one factory street, because if you want to change the layout everything is as before. You need to built many streets.
- useless if you are not capable to supply a FS with a constant stream of material. This is an interesting task.
- And this is also a moment, where a factory street could be an advantage for the player: lets say the FS works and it is able to switch to simulation mode. What if the player then is able to increase the speed or productivity etc with modules? The modules work only in the simulated mode. ... not thought to the end but that could be the moment when you really want to build this.
- also useless, if you don’t need so much output...
- ... which is a quite interesting logistic task to build that so, that the factory street is either working in simulated mode or is turned off, because that takes much control from outside. This makes it also much easier to switch off the power, if nothing is produced anymore.
But for the case that the preconditions are meet, and you have many factory streets working in simulation, this should be a real bringer! And it would enable much, much bigger factories than now.
If this is also combined with some boxing mod, like deadlocks https://mods.factorio.com/mod/deadlock- ... es-loaders which enables to produce with boxed items, then I think factor 10-100 bigger factories should be possible.
I think for many players that would be a really good argument.
Parts of a factory can be simulated, if the conditions that influence that part have very specific values. E.g. input belts needs to be fully compressed, and many more. In that case it is safe to switch to a simulation.
Long read:
Well, I think this is a way we should think about deeper.
What I’m thinking here is also Factorissimo, but instead of just one building it should have the “real size”.
So what I see here are some possible solutions, which would simulate a realistic production process for very large quantities but switch back to the realistic simulation for low quantities.
How could that be done?
First I think about the concept of factory-streets. Like a gigantic smelting array. This can be simulated very exactly. See for example the max rate calculator https://mods.factorio.com/mod/MaxRateCalculator
But it needs a lot of preconditions to make sure, the simulation is always exact:
- all input belts are always compressed
- enough free space on output belts to put items on
- inserter are fast enough to feed/empty the smelters
- there is no complex belt stuff, like when it is possible that two different items could be on one lane
- same is with mixed input lanes. As soon as another item on a lane is imputed it needs to switch back to simulation.
- there are no circuits involved
- there is always 100% power available (well I think this can be simulated up to some degree)
- pipes are never filled below some level X
- player is not changing anything inside that factory street are
- the theoretical input/output is equal to the current input/output
- and of course some more preconditions.
You see it’s difficult to achieve those conditions so I would like to introduce a “factory-street-area”. Normally this area is hidden with some kind of industrial roof (game doesn’t need to simulate the inside), but when the player enters, the roof becomes transparent. When you want to built inside such an area the factory street switches to the normal mode and everything behaves as it should. You can then built inside the factory street, but you cannot built some constructions. For example you accidentally side-insert a different item into a belt-lane. You cannot do circuits, or you forgot to power a device.
But when you go outside and the above conditions are meet, then it switches to simulation mode after a while. After a while means: the theoretical input/output needs to be more or less equal to the current measured input/output.
And as soon as one condition isn’t meet anymore the factory street switches back to normal mode. That is likely, if the output is blocked or not enough input comes in, or if power fails. But for those cases that input is missing or output is blocked it’s already so, that the inserters and entities go to sleep. So not too much CPU used. The factory street covers only the case, when it is running with full speed.
So what does this bring?
- useless if you only have one factory street, because if you want to change the layout everything is as before. You need to built many streets.
- useless if you are not capable to supply a FS with a constant stream of material. This is an interesting task.
- And this is also a moment, where a factory street could be an advantage for the player: lets say the FS works and it is able to switch to simulation mode. What if the player then is able to increase the speed or productivity etc with modules? The modules work only in the simulated mode. ... not thought to the end but that could be the moment when you really want to build this.
- also useless, if you don’t need so much output...
- ... which is a quite interesting logistic task to build that so, that the factory street is either working in simulated mode or is turned off, because that takes much control from outside. This makes it also much easier to switch off the power, if nothing is produced anymore.
But for the case that the preconditions are meet, and you have many factory streets working in simulation, this should be a real bringer! And it would enable much, much bigger factories than now.
If this is also combined with some boxing mod, like deadlocks https://mods.factorio.com/mod/deadlock- ... es-loaders which enables to produce with boxed items, then I think factor 10-100 bigger factories should be possible.

I think for many players that would be a really good argument.

- Sat Mar 07, 2020 6:50 am
- Forum: Ideas and Suggestions
- Topic: Optimization idea: abstraction
- Replies: 236
- Views: 81631
Re: Optimization idea: abstraction
The first thing that came to my mind reading OP's post is https://mods.factorio.com/mod/Factorissimo2 too 
