[0.11.19][slpwnd] Production window stats highly inaccurate

[ZorbaTHut]

First, some math:

It takes 0.5s to produce Copper Cable, and you get 2 of them in output. The speed of an Assembling Machine 2 is 0.75. Therefore, the production of three Assembling Machine 2's should be 3*2/0.5*0.75*60=540 items per minute.

It takes 0.5s to produce Electronic Circuit. Two Assembling Machine 2's should produce them at a rate of 2/0.5*0.75*60=180 items per minute.

Finally, smelting an Iron Plate takes 3.5 seconds. An Electric Furnace runs at 2x speed, so a single Electric Furnace should be able to produce Iron Plates at a rate of 2/3.5*60=34.29 (approx) items per minute.

Now let's look at the actual game. Here is a game in which I've been doing that (click the "download" icon at the top) - you'll need Y.Petremann's Creative Mode mod and rk84's Test Mode mod in order to run. As you can see:



this is a very simple "factory" containing all three of those production chains.

So now let's look at the P window. 5s is too inconsistent to get useful numbers out; let's skip ahead to 1m.



As you can see, something has already gone wrong. Each production is 11% too fast. But it gets weirder! Check out the 10m stats next:



Now we're about 1.5% too fast. But we're not done yet . . .



Now we're about 1.1% too *slow*. 10h makes things worse:



2.5% too slow. I haven't sat down to try 50h yet, but I'm expecting 3-4% too slow. Oh, and let's take a look at 5s again!



Yeah, that's right - it's wrong by about 45%. Holy moly. Look at that - it says it's completed 66 copper wire, when 1m says 600 copper wire! That means there are less than 50 seconds in a minute! Something is royally broken here.

Note that most of these numbers were gathered with "/c game.speed = 32". However, I assume this uses the same game speed code that the replay system does, and that system would break catastrophically if there were flaws in it. Changing the game speed to normal doesn't cause any visual discontinuities in any graph, nor does it change the numbers on the low-duration graphs, so I'm guessing the game speed isn't a factor here.

So, aside from the obvious bug report, I've got two questions:

* Are any of the production charts accurate?

* Are the producers constructing at the right speed, or is the entire game slightly mistimed?

[SHiRKiT]

This is really interesting, I never thought that the Production Window could be mistaken. This along with the bug in the Inserter's rotation speed are really worrying to me.

[slpwnd]

Thanks for the nice analysis. The problem was in our smoothing window for the statistics. This is necessary otherwise the graphs would just look rubbish. So the smoothing window actually changes the size of the interval. This is fine for the graphs as they are basically an average from the past smoothing window. However there was no logic to compensate for this in calculation of average output in the overview. I have added that for the 0.11.20. Now the output shows consistent output across different time spans. It is still not equal to theoretical calculations (it is about 2.5% less than that) but imho it is good enough.

[SilverWarior]

How are these averages production values calculated?
I hope you are not calculating average for longer period from multiple average values for shorter periods?
If you are then this is the cause for wrong results.
You should always operate on base data and not intermediate calculations when dealing with averages.

[slpwnd]

How are these averages production values calculated?
I hope you are not calculating average for longer period from multiple average values for shorter periods?
If you are then this is the cause for wrong results.
You should always operate on base data and not intermediate calculations when dealing with averages.

The game is collecting the samples per tick. Then all the overviews for different periods are calculated from these data. As mentioned before the problem with accuracy was related to using a sliding window. Sliding window is necessary for visual graphs, otherwise the graphs output would show very aggressive values and would not really be usable.

[SilverWarior]

Wait are you saying that you store 36000 values (5s production productions) for each product so that you can calculate 50h average and draw that specific graph?

[quinor]

that should be less than ~2mb of data storing numbers is quite easy and efficient.

[SilverWarior]

that should be less than ~2mb of data storing numbers is quite easy and efficient.

About 140KB for each product. Multiply this with the number of vanilla products and you are already on 20MB.
Then add all those new items from other mods and the amount isn't so small as it seemed in the beginning.

Anyway the main reason why I'm asking this is because I know a way to implement this with much lower memory footprint which is faster to process and doesn't suffer same precision problems as calculating average of averages would.

[slpwnd]

Each interval actually uses the same amount of samples at different precision levels. If we forget the smoothing window then there is 300 samples for each interval.

So:
5s ~ 1 sample per tick
1min ~1 sample per 12 ticks
10min ~ 1 sample per 120 ticks
...

[Holy-Fire]

Happy to see this finally resolved. Ironically, it has been exactly 1 year since my last report on the issue:
https://forums.factorio.com/forum/vie ... =10#p22419

I didn't feel like dedicating the time to fight for this to be fixed at the time, and hoped it would be resolved eventually. And it was

By the way, I think the largest errors are for 5s, so looking at larger intervals underestimates the problem.

It is still not equal to theoretical calculations (it is about 2.5% less than that) but imho it is good enough.

I guess we can live with that, but I'll be happy to see it made even more accurate.