Optimization idea: abstraction

Post your ideas and suggestions how to improve the game.
posila
Factorio Staff
Factorio Staff
Posts: 4883
Joined: Thu Jun 11, 2015 1:35 pm
Contact:

Re: Optimization idea: abstraction

Post by posila »

yagaodirac wrote:
Sun Oct 25, 2020 1:33 am
So, would you remove inserters in 1.1?
No. 1.1 is not a new game.

Qon
Smart Inserter
Smart Inserter
Posts: 1528
Joined: Thu Mar 17, 2016 6:27 am
Contact:

Re: Optimization idea: abstraction

Post by Qon »

posila wrote:
Sat Oct 24, 2020 5:33 pm
yagaodirac is trying to design a new game around the abstraction optimization tech.
Ah ok. I missed that somehow.
posila wrote:
Sat Oct 24, 2020 5:33 pm
But I think it's a bad idea attempting to do it as a mod for Factorio. You'd be attempting to circumvent the game simulation in a game that's built around everything (that is built by player) being simulated and accounted for a all times. Making a new game in UE4/Unity seems easier than trying to bend the game this way through its modding API.
Making a new game would require remaking everything. The mod might take a lot of work but a hacky proof of concept with a more limited feature set and other tradeoffs that can be improved over time is probably feasible at least. Depending on how much all the other update loops (that aren't affected by active false) use processing power when all entities are active=false.
It's not a bad idea for a mod if people want it to be made.

The core idea is rather simple actually, for a "stable" factory. The hard part is getting unstable behaviour right and to make partial factory parts run and splitting the abstraction etc.

User avatar
ptx0
Filter Inserter
Filter Inserter
Posts: 641
Joined: Wed Jan 01, 2020 7:16 pm
Contact:

Re: Optimization idea: abstraction

Post by ptx0 »

no one mentioned Schall Machine Scaling mod yet?

User avatar
AmericanPatriot
Filter Inserter
Filter Inserter
Posts: 613
Joined: Sun Feb 16, 2020 12:47 am
Contact:

Re: Optimization idea: abstraction

Post by AmericanPatriot »

yagaodirac wrote:
Sun Oct 25, 2020 7:30 am
Anyway. I recommend removing inserters and loaders. They are meaningless and helpless, ruin the gameplay and waste both real life time and cpu.
Is this a joke?
Sarcasm and insults are generally neither helpful nor appreciated. :D

yagaodirac
Fast Inserter
Fast Inserter
Posts: 106
Joined: Sun Jun 16, 2019 4:04 pm
Contact:

Re: Optimization idea: abstraction

Post by yagaodirac »

AmericanPatriot wrote:
Sun Oct 25, 2020 5:25 pm
Is this a joke?
In fact not. But it indeed looks like a joke.

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 11788
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Optimization idea: abstraction

Post by ssilk »

I wonder how the game should work without them. Factorio’s magic happens exactly there, when things are linked together.
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

User avatar
KoblerMan
Fast Inserter
Fast Inserter
Posts: 175
Joined: Sat Mar 05, 2016 12:59 am
Contact:

Re: Optimization idea: abstraction

Post by KoblerMan »

The whole abstraction thing is a neat idea on paper, but I really don't see it as being practical, and it seems out of scope for a game like Factorio anyway. That's even assuming the whole idea is compatible with the game engine, which hasn't been built for it. IMO it would create more problems than solutions.

I think Wube has done a fantastic job of optimizing their game as it is, in reality I would argue abstraction on this scale is unnecessary.
ImageImage
System Specs
OS: Windows 10 Professional 64 Bit
CPU: AMD Ryzen 5 3600X (@~3.8 gHz)
GPU: Nvidia RTX 2080
RAM: 32GB DDR4 (2400)
DRIVES: 2x 1TB NVMe SSD, 1x 6TB HDD

Optymistyk
Inserter
Inserter
Posts: 49
Joined: Sun Jun 14, 2020 9:41 pm
Contact:

Re: Optimization idea: abstraction

Post by Optymistyk »

Tbh I was sure this is already in place cuz the game is just so well-optimised and it makes so much sense.

Every production line works in a deterministic manner so it should be possible to express everything as a "lazy" function - only calculated the moment a player checks it
But it needs a lot of preconditions to make sure, the simulation is always exact:
- all input belts are always compressed
- enough free space on output belts to put items on
- inserter are fast enough to feed/empty the smelters
- there is no complex belt stuff, like when it is possible that two different items could be on one lane
- same is with mixed input lanes. As soon as another item on a lane is imputed it needs to switch back to simulation.
- there are no circuits involved
- there is always 100% power available (well I think this can be simulated up to some degree)
- pipes are never filled below some level X
- player is not changing anything inside that factory street are
- the theoretical input/output is equal to the current input/output
- and of course some more preconditions.
None of those are in fact necessary.
- If the belt is not compressed you still know the rate at which items are placed on the belt by the inserters
- If the rate at which items are placed on the belt exceed the belt throughput then it is limited to the belt throughput
- If the inserters are not fast enough then the smelters produce proportionally less items
- It can still be expressed as a mathematical function
- It can still be expressed as a function
- Circuits are literalry just functions
- If there's less than 100% power everything slows down proportionally
- If the liquid flows at a rate lower than the required rate X then the production slows down proportionally
- When the player is nearby the abstraction switches to simulation, after the player makes changes the function can be updated
- Not necessary

It would be incredibly complicated to implement and any bugs could be completely gamebreaking but it would defo improve performance (which I think is fine tbh)

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 11788
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Optimization idea: abstraction

Post by ssilk »

Optymistyk wrote:
Sat Nov 14, 2020 1:19 pm
Every production line works in a deterministic manner so it should be possible to express everything as a "lazy" function - only calculated the moment a player checks it.
Ahem, this is not correct. It works only deterministic under very specific conditions.
None of those are in fact necessary.
That depends on what you want to achieve!

If you want to have a correct simulation it’s wrong. If you want to make an abstraction, that is sometimes correct: maybe.
- If the belt is not compressed you still know the rate at which items are placed on the belt by the inserters
But only after some processing time.
- It can still be expressed as a mathematical function
It’s not deterministic under conditions outside of the function scope.
- Circuits are literalry just functions
- If there's less than 100% power everything slows down proportionally
- If the liquid flows at a rate lower than the required rate X then the production slows down proportionally
Combinators slows down, when they have not enough power. The result is not longer deterministic (in most cases). And the same is it with inserters and fluids, they don’t slow down proportionally and you forget the time it takes for an item to run through the process.
It would be incredibly complicated to implement and any bugs could be completely gamebreaking but it would defo improve performance (which I think is fine tbh)
Well, as said, only under special conditions this behaves deterministic. It’s not super complicated, but I admit it’s complicated and the result might be questionable, because it could be very complicated for a player to achieve the conditions under which that works deterministic.
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

Optymistyk
Inserter
Inserter
Posts: 49
Joined: Sun Jun 14, 2020 9:41 pm
Contact:

Re: Optimization idea: abstraction

Post by Optymistyk »

- If the belt is not compressed you still know the rate at which items are placed on the belt by the inserters
But only after some processing time.
- It can still be expressed as a mathematical function
It’s not deterministic under conditions outside of the function scope.
Well now you're making me do maths

Imagine a somewhat simplified example: There's a straight (infinite) line of yellow belt. A yellow inserter(no upgrades,) is placing items on the right side of the belt. You can express what the inserter is doing as a periodic function. If you want to calculate at what times t an item is placed:
cos(t+φ)=0, t∈[0,+infinity)
Image
Each point where the graph crosses the t axis on the right side is the moment in time when one iron plate is placed on the belt. The function does so at any t∈(1/2π + φ, 3/2π + φ, 5/2π + φ,7/2π + φ...). 1π is the period T, or the time between the placement of each iron on the belt.

Since we know that a yellow inserter works at a rate of 0.83 items/second, π can be expressed as 1/(0.83 items/second * Power/100%) where P is constant Power; π =~ 1.2 seconds with full power. So if π = 1.2s then t∈(0.6s + φ, 1.8s + φ, 3s + φ, 4.2s + φ...). If Power levels are not constant then you have to track the history of power in the system, which Factorio is already doing cuz you can display a nice Power graph by clicking any power pole. Because most things work at a rate proportional to their power level, it might even be enough to just use an average instead.

φ is the Phase of the function, or the initial delay before the first placement. For now let's just say φ=0. Then t(0.6s, 1.8s, 3s, 4.2s...). Let's calculate s - the distance each plate has travelled from it's initial point of placement. s = v*(t0 - t), where v - the speed of the yellow belt belt(1.875 tiles/second), t0 - the time passed since the inserter started placing items. So let's say it's been 10 seconds since the inserter started placing items, then s∈(~17.6,~15.4,~13.1,~11...) tiles.

Okay, but the way it is now we have to keep track of every point of placement for every inserter and it's not too convenient. Let's instead count from the beginning of the belt for each inserter. For that let's reintroduce φ. If the initial point of placement of an inserter is where the belt starts, then φ = 0. However if the inserter is further up the belt then it's as if the items it places had a head start compared to the inserters that are closer to the beginning. Hence it has a negative Phase delay φ = -x/v, where x - distance from the beginning of the belt, v - the speed of the belt(1.875 tiles/second). For an inserter that's 19 tiles up the belt at 10 seconds in, s∈(~27.6,25.4,23.1,22...) tiles from the beginning of the belt.

So if we take it all together we can express how far up the belt each item an inserter places is t0 seconds since the inserter started working:
f(t0) = 1.875[t0-n*1.2s * P/100%] -x/v, where n = the number of the item, n∈(1,2,3,4,5...). If there's several inserters placing the same kind of items we can just sum this: f(t0) = Σ(1.875[t0-n*1.2s*P'/100%] -x'/v), where x' ∈ (x1,x2,x3,x4...),P'∈(P1,P2,P3,P4...). If there's another set of inserters placing another kind of item on the same lane then they can be tracked separately using the same function or combined using Fourier Transform. If the inserters are taking items from a smelter that produces items with a period T then f(t0) = Σ(1.875[t0-n*max(T,1.2s)*P'/100%] -x'/v) This is still somewhat simplified as it does not take into account belt backing up or inserters trying to place an item where there already is one, but I'd be here all day if I were to write and prove a complete function.

So as you can see it can be expressed as a function and it does not require that the belt is compressed or that power is 100% or that there's only 1 type of item on a lane or that the smelter is producing items fast enough and I promise you every other precondition you mentioned also does not apply because the system is 100% understood and deterministic
- Circuits are literalry just functions
- If there's less than 100% power everything slows down proportionally
- If the liquid flows at a rate lower than the required rate X then the production slows down proportionally
Combinators slows down, when they have not enough power. The result is not longer deterministic (in most cases). And the same is it with inserters and fluids, they don’t slow down proportionally and you forget the time it takes for an item to run through the process.
Well, granted circuits sometimes change the way they work at low power. Still, they always work in a deterministic manner, there's no randomness involved here. As far as I'm aware inserters work at a rate proportional to their power level cuz every rotation consumes a fixed ammount of power. I don't know that much about liquids but I'd assume they also work in a deterministic fashion

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 11788
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Optimization idea: abstraction

Post by ssilk »

Hm. Interesting. And yes, everything in Factorio is deterministic. I meant it differently. It’s the base of the game, that it is deterministic. But deterministic means not, that it can be reduced to a simple formula.

Just two examples, why that can’t be expressed as mathematical expression in some special cases.

1. What happens, when an inserter cannot place an item on a belt? It waits a bit, until the belt is free again. I think that’s what you mean with shifting
This shifts the item on the belt so, that it comes out a bit later.

Now what if this shift happens not every time? I mean: there is another assembler before that and he doesn’t get regularly items, because he is the last on the supply-belt.

When you look now on the output belt you see eventually a pattern of items. So it is still mathematical expressable.

But now we add production modules. This randomizes the pattern so strong, that the mathematical formula gets so complicated, that it is easier to run the simulation instead.

2. I turn power on and off, just depending on, if the inserter for the one boiler gets a piece coal. And it gets a piece of coal only, if the smelters don’t need coal.

I know, that is a stupid setup, but what I want to show is, that you now have an extremely fractal design, and even if you are able to express that in a formula a very simple change can change the whole calculations.


That are the reasons why I say it works only, if the belts are fully compressed: the output pattern cannot change anymore. And if there is full power.
And if
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

blazespinnaker
Filter Inserter
Filter Inserter
Posts: 313
Joined: Wed Sep 16, 2020 12:45 pm
Contact:

Re: Optimization idea: abstraction

Post by blazespinnaker »

+1

Factorrismo like addition would allow for much larger scale factories in vanilla. By allowing a player to identify inputs and outputs, a factory could be promotable to abstractable after it had proved itself over some period of time, perhaps time proportional to the complexity of the factory itself.

Perhaps too much volatility in production curves would result in a factory unable to be promoted to abstractable. This would create a very unique challenge in abstracted factory design.

The potential problem at that point would be that if these factories became too easy to put down, you could quickly launch a rocket within very little time.

I think it'd be very reasonable that abstraction shouldn't have any impact on resource cost or time / complexity to place. It would be no different than placing a blueprint.

In fact, you could even punish the player by not allowing a previously abstracted factory to be used even a little bit until it was fully placed. This is a huge penalty and in this way, abstracted factories would actually be negative in time to launch a rocket.

In addition, you could require that the player first terraform any land under the factory such that it has a proper factory floor. Also, walls, to provide a barrier against biters. Other things, such as roboport coverage with mininum # of robots could be required as well.

Any biter getting through a wall would just halt the factory. It would be tempting to take factories out of simulation mode for the player and updating them, but the impact on UPS could be frightening in some cases. Better would be to just let the biters run rampant through a bunch of unpowered entities.

The end benefit would be truly massive. They would allow for much greater scale factories via infinitely more efficient UPS usage, and maybe provide a minor added benefit for newer players in terms of managing complexity.

Allowing trains in factories would be cool ;) Something may be required here to do faster large scale terraforming tho.

Of course, what would we do with all that scale? :) I think adding a bunch of very expensive infinite research would be a terrific addition to go along with this feature. See this thread here for more on that - viewtopic.php?f=5&t=93333&p=528820&hili ... ch#p528820

edit to add: I spent some time on this and I was able to enhance the factorissmo2 mod to do much of the above. https://mods.factorio.com/mod/Factoriss ... c7db4f5997

It's actually a pretty cool mod. Can't wait to use it in more megabase mass scale.
OptimaUPS Mod, pm for info.

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 11788
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: Optimization idea: abstraction

Post by ssilk »

What I find currently much more appealing than this, is the idea, that you create many of equal “elements” and you calculate only one.
Because of the deterministic nature of Factorio all others must behave the same, if they have the same input.

See this thread: viewtopic.php?f=6&t=91587 Nuclear Reactors BluePrint Cloning
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

blazespinnaker
Filter Inserter
Filter Inserter
Posts: 313
Joined: Wed Sep 16, 2020 12:45 pm
Contact:

Re: Optimization idea: abstraction

Post by blazespinnaker »

ssilk wrote:
Mon Jan 04, 2021 9:42 am
What I find currently much more appealing than this, is the idea, that you create many of equal “elements” and you calculate only one.
Because of the deterministic nature of Factorio all others must behave the same, if they have the same input.

See this thread: viewtopic.php?f=6&t=91587 Nuclear Reactors BluePrint Cloning
Yes, this is what I've done, basically.

I'm not entirely clear on what you think the difference is, but perhaps it is this improvement -

I do leave the initial factory running (there is no API that can put entities to sleep that I've seen) and continually let it update a dictionary of its inputs/outputs. In this way the accuracy is more long term and continual.

I can't think of a way to do all of this deterministically, if that is what you were referring to. Hopefully there is a way someone can propose, but I'm not particularly optimistic. Or very concerned, really. When you get to megabase scale these sort of marginal issues are very tiny relative to everything else.

UPS becomes the overwhelming roadblock as you can't expand without torturing your factory designs, and then only in a limited way. I wouldn't use this mod obviously if UPS wasn't an issue, there would be little point. And I agree, deterministic production is a neat aspect of factorio, but afaik impossible to achieve at the sort of scale I envision without making some very creative assumptions.

In terms of deterministically working for multiplayer, I imagine that's fairly easy to do. Factorio is not that rigorously deterministic as there are non deterministic inputs, such as player actions.

But thank you for the nuclear power link. For some reason power didn't occur to me, but now that you shared it, I am so absolutely going to do that and should be fairly straightforward. I detest solar panels and accumulators :)

I'm probably going to make a new mod rather than using forked version of factorissimo. It's fairly limited in factory size and I need something much much larger. It really also has much different goals other than UPS optimization. Finally, I think I will not use pipes, belts, or chests but rather just cargo trains and fluid wagons.

The only downside is I don't really have the art to back this all up with.
OptimaUPS Mod, pm for info.

User avatar
ptx0
Filter Inserter
Filter Inserter
Posts: 641
Joined: Wed Jan 01, 2020 7:16 pm
Contact:

Re: Optimization idea: abstraction

Post by ptx0 »

blazespinnaker wrote:
Mon Jan 04, 2021 12:09 pm

I do leave the initial factory running (there is no API that can put entities to sleep that I've seen) and continually let it update a dictionary of its inputs/outputs. In this way the accuracy is more long term and continual.
well if the assets exist on any surface, and even if you find entities on the factory and put them to sleep when the user leaves the building, inactive entities still contribute to more than 4 update loops in the game engine. The "entity" update loop goes to sleep, but the "ElectricNetwork" and other usages all still must update.

basically, you've done nothing to fix UPS other than build an expensive, Lua-scripted, infinity chest.

blazespinnaker
Filter Inserter
Filter Inserter
Posts: 313
Joined: Wed Sep 16, 2020 12:45 pm
Contact:

Re: Optimization idea: abstraction

Post by blazespinnaker »

ptx0 wrote:
Mon Jan 04, 2021 5:36 pm
blazespinnaker wrote:
Mon Jan 04, 2021 12:09 pm

I do leave the initial factory running (there is no API that can put entities to sleep that I've seen) and continually let it update a dictionary of its inputs/outputs. In this way the accuracy is more long term and continual.
well if the assets exist on any surface, and even if you find entities on the factory and put them to sleep when the user leaves the building, inactive entities still contribute to more than 4 update loops in the game engine. The "entity" update loop goes to sleep, but the "ElectricNetwork" and other usages all still must update.

basically, you've done nothing to fix UPS other than build an expensive, Lua-scripted, infinity chest.
ahh, no, that's not true at all.

However, I'm going to resist getting into a discussion with you as it has been a net negative to these forums in a couple of places now, one of which has resulted in mod intervention. Enjoy your week, sir.
OptimaUPS Mod, pm for info.

User avatar
ptx0
Filter Inserter
Filter Inserter
Posts: 641
Joined: Wed Jan 01, 2020 7:16 pm
Contact:

Re: Optimization idea: abstraction

Post by ptx0 »

blazespinnaker wrote:
Mon Jan 04, 2021 5:46 pm
ptx0 wrote:
Mon Jan 04, 2021 5:36 pm
blazespinnaker wrote:
Mon Jan 04, 2021 12:09 pm

I do leave the initial factory running (there is no API that can put entities to sleep that I've seen) and continually let it update a dictionary of its inputs/outputs. In this way the accuracy is more long term and continual.
well if the assets exist on any surface, and even if you find entities on the factory and put them to sleep when the user leaves the building, inactive entities still contribute to more than 4 update loops in the game engine. The "entity" update loop goes to sleep, but the "ElectricNetwork" and other usages all still must update.

basically, you've done nothing to fix UPS other than build an expensive, Lua-scripted, infinity chest.
ahh, no, that's not true at all.
i'm sorry, which part was untrue?

blazespinnaker
Filter Inserter
Filter Inserter
Posts: 313
Joined: Wed Sep 16, 2020 12:45 pm
Contact:

Re: Optimization idea: abstraction

Post by blazespinnaker »

posila wrote:
Sat Oct 24, 2020 5:33 pm
But I think it's a bad idea attempting to do it as a mod for Factorio. You'd be attempting to circumvent the game simulation in a game that's built around everything (that is built by player) being simulated and accounted for a all times. Making a new game in UE4/Unity seems easier than trying to bend the game this way through its modding API.
Unfortunately, it's either this or trash the megabase and start anew, unless you're aware of a third choice?

I prefer this option as it frees up UPS for more mining operations and much more vast and complex train network, while still allowing for work on optimal factory designs (which aren't redundantly being processed)

The idea on linking factories together for input/output is an interesting one for deterministic simulation, but it seems a bit too unrealistic and awkward (I imagine factories would constantly be falling out of sync), even though it would be more accurate..
OptimaUPS Mod, pm for info.

Eketek
Inserter
Inserter
Posts: 41
Joined: Mon Oct 19, 2015 9:04 pm
Contact:

Re: Optimization idea: abstraction

Post by Eketek »

I have an idea for a mechanism which could feasibly be used to make parts of the factory virtualizeable. Also seems like it would fit into the game reasonably well and has additional uses in general gameplay.

Concept: Synchronized Factory Blocks

A synchronized factory block would be an arbitrary collection of entities which are all either concurrently active or concurrently suspended.

While suspended, every belt/transport line would stop, assembler progress would cease, inserter arms would lock in position, and [depending on game/mod settings] fluids and heat would stop flowing. And, when resumed, everything in the block would re-activate and proceed exactly as if it had continued operating.

Player would designate a factory block and assign the conditions which would activate or de-activate the block. Presumably, these would be manual toggle, a circuit signal, mod behaviour, and [most importantly] the combined state of the block's inputs and outputs.

Auto-toggling a factory block based on the combined state of the block's inputs and outputs (all inputs packed and all outputs open) would enforce the conditions needed for deeper optimizations (and thus would make such optimizations a lot more accessible in casual-ish gameplay).

adam_bise
Fast Inserter
Fast Inserter
Posts: 210
Joined: Fri Jun 08, 2018 10:42 pm
Contact:

Re: Optimization idea: abstraction

Post by adam_bise »

Suppose I build up my smelting arrays to the point where UPS is affected, but then fully compress their inputs to achieve abstraction, raising the UPS back up. Then I do the same for wires, circuits, and so on.. At some point I overlook a shortage in my power generation, and at once nothing is abstracted. I might not be able to recover this situation due to extremely low UPS. Kind of like painting yourself into a corner there.

This problem would not happen if belts required power and their speed was linear to power input I think.

Post Reply

Return to “Ideas and Suggestions”

Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Rseding91