CPU performance benchmarks

[azesmbog]

The result is 94 without huge pages. I'm using Factorio 1.0, Kernel 5.8.13, PDS scheduler, performance governor. There is a HUGE amount of TLB misses with standard pages (around 12'000'000/s with standard pages vs. 1'000'000/s with huge pages).

The CPU boosts to 4.625GHz.

As I understand it, 4.6 is a very good overclocking of the AMD processor.
94 versus 85 is also an excellent result!

Immediately the question is - are kernel tweaks possible only for Ryzen processors, or is there a comparable result for Intel processors too? And the second - if this is so good - why is it not in the main kernel, but must be enabled forcibly? However, these are rhetorical questions.
My main question is as follows.
The test map is very small and unpretentious. It works quite well even on 8 GB of RAM, maybe that's why there are few misses with large pages.
Could you please check my "little sandbox" on your system. True, you have only 32 GB of memory, this is too little
My map works with 48 GB of memory, the timings are really low due to the large volume) 19-19-19-38 @ 3333 MHZ, but at the same time it produces 37-38 UPS, it is still playable, but it is impossible to develop further: ((Or an offline script check, well, or at least run, and see the UPS in a few minutes, even while in one place.
A link to saving is in the topic about the sandbox in the topic about megabases, but I can also make a fresh save, the current one.
If not difficult For the truth. Vanil.

[sushi_eater]

Could you please check my "little sandbox" on your system. True, you have only 32 GB of memory, this is too little

Factorio doesn't like your save (I tried with 1.0). (I was able to extract the zip, so I don't think the download is corrupted.)

82.739 Warning Map.cpp:340: Map loading failed: Couldn't read from input file. File could be corrupted.

Immediately the question is - are kernel tweaks possible only for Ryzen processors, or is there a comparable result for Intel processors too? And the second - if this is so good - why is it not in the main kernel, but must be enabled forcibly?

Exactly the same thing works for Intel. Somebody tested huge pages with a 7700K, it improved performance by 8% or so.

Huge pages don't play nice with all applications and can result in decreased performance and large latency spikes (system freezes for hundreds of milliseconds).

[azesmbog]

It's strange.
Firstly, my save was apparently launched, and secondly, I also launched it in 1.0
and yes, there is version 0.18.33. Ok, now I will save the last save and post it to the exchanger. But this is long, maybe an hour)
It is possible that there is still not enough RAM: (((
Can I try to increase the swap file, or in Linux it is completely different?

So, the last save.
It takes 7 minutes to save the game.
The load is 3 minutes.
After one minute, the UPS stabilizes at around 36.6min -37.2max
https://drive.google.com/file/d/1WxIyEd ... sp=sharing

I'll also leave a link to the old save, last year, it's a little smaller. Maybe it will work.
https://drive.google.com/file/d/1XQ84OF ... sp=sharing

[sushi_eater]

It looks like the Linux version doesn't support your maps. Both Factorio 1.0 and 0.17.79 fail with the same error I mentioned.

Running the Windows version of Factorio under Wine could load your old map (123_311219.zip), but huge pages don't get used. I ran a benchmark for 10'000 ticks:

Performed 10000 updates in 404674.581 ms
avg: 40.467 ms, min: 17.698 ms, max: 19629.176 ms

IME, the min value is reasonably consistent across benchmark runs and would reflect something useful. There is too much paging, so the average is useless.

[westjensontexas]

I also remember back when they released the first Athlon X2 dualcore CPUs that each core used to return a different elapsed real time (only the devil knows why they did that instead of a global time that is the same for all cores)... and various applications crashed shortly after starting because of synchronization problems that arised between their threads. It required a microcode/OS driver update to deal with that.

[Pentium100]

I tried running the benchmark (123_311219.zip) on my PC.

Factorio used 33.6GB of RAM and about one core of CPU.

Performed 10000 updates in 707618.482 ms
avg: 70.762 ms, min: 45.903 ms, max: 798.719 ms

My CPU does not have good single-thread performance

[azesmbog]

I tried running the benchmark (123_311219.zip) on my PC.

Factorio used 33.6GB of RAM

Launched the test (123_311219.zip) on version 1.0.0
Memory used 35.7 GB
average: 15 ms, or 65 UPS
Well, in order not to get up twice - test of my gigabase

38.jpg (31.67 KiB) Viewed 8831 times

In principle, correlates with the real game - about 37 ups\fps

[Syhn]

I gave huge pages a shot on my i7-8700k and managed to get the best benchmark to date with Factorio 1.0.0.
109 UPS baseline
125 UPS with huge pages
Sorted list of benchmarks to date

I made some slight changes to the variables/steps listed in this reddit post.

Code: Select all

sudo sysctl -w vm.nr_hugepages=8192 # can likely be much less but I have a lot of spare memory
curl -s https://factoriobox.1au.us/scripts/benchmark.sh | LD_PRELOAD=/usr/lib64/libhugetlbfs.so MALLOC_ARENA_MAX=1 HUGETLB_MORECORE=yes HUGETLB_RESTRICT_EXE=factorio bash

[maxp779]

Will be interesting to see someone do some benchmarks with a Ryzen 5xxx CPU:
https://hexus.net/tech/reviews/cpu/1464 ... n-9-5900x/

They talk about "cleaner caching" and "reduction in memory latency" which might help factorio a bit.

[azesmbog]

so there are already several results - 106 - the best for Windows. and this is a processor with 6 cores \ 12 threads.
For comparison, a 4 core old processor shows 112, at a lower memory frequency.
And further. I don't consider the results in Linux with huge pages, I consider them cheating. It is hardly possible to play with these settings. But this is a private opinion

[Jap2.0]

Maybe I'm misinterpreting your interpretation, but I consider the 5000 series quite promising - the 5600X looks like it's getting roughly comparable performance with the 9900K, so with slightly higher clocks (and maybe a slight bump due to more cores) the 5800X and 5900X could be quite promising.

[maxp779]

so there are already several results - 106 - the best for Windows. and this is a processor with 6 cores \ 12 threads.
For comparison, a 4 core old processor shows 112, at a lower memory frequency.
And further. I don't consider the results in Linux with huge pages, I consider them cheating. It is hardly possible to play with these settings. But this is a private opinion

Ah yeah you're right. Wow Ryzen 5xxx has closed the gap significantly! Someone with 2133mhz DDR4 was getting 95 UPS with the 5600x. And that memory is pretty damn slow. 3600mhz netted someone else 113. Very nice increase from the older gen Ryzens.

[azesmbog]

Wow Ryzen 5xxx has closed the gap significantly! Someone with 2133mhz DDR4 was getting 95 UPS with the 5600x. And that memory is pretty damn slow. 3600mhz netted someone else 113

Let me be wrong in my judgments, but let me only consider the results in Windows. (in Linux, I consider some of the results, although real for the test, but not indicative, and still have a distant relation to the real game
So that's it. Windows.

Two AMD Ryzen 5 5600X results
1.95 UPS with memory 2133
2.106 UPS with 3800 memory

and two results for AMD Ryzen 7 5800X
1.94 UPS with memory 2133
2.113 UPS with 3600 memory

and AMD Ryzen 9 5950X
1.114 UPS with 3600 memory

The last result is generally excellent for AMD processors!
If we assume that the results were obtained at the same processor frequencies, then the effect of fast memory is clearly visible

[jape3]

Just tested the performance difference between 1.0 and 1.15. On my current train / belt / monster heavy death world map the update time went down from 10.2ms to about 8.2ms.. so a pretty nice performance boost

System spec: AMD Ryzen 7 3700X, 32GB of 3600MHz DDR4

[elfstone]

Does anyone have any experience with RAM Speeds beyoned 3600? There are (expensive) sets available with speeds of up to 4400 MhZ, will they bring a similar speed bost as going from 3200 to 3600?

I'm currently planning to build a new PC with a Ryzen 5900, but since it probably won't be available for quite some time, I'm thinking about buying the RAM now, so that I can upgrade my current 8GB sticks.

[maxp779]

Does anyone have any experience with RAM Speeds beyoned 3600? There are (expensive) sets available with speeds of up to 4400 MhZ, will they bring a similar speed bost as going from 3200 to 3600?

I'm currently planning to build a new PC with a Ryzen 5900, but since it probably won't be available for quite some time, I'm thinking about buying the RAM now, so that I can upgrade my current 8GB sticks.

I have no direct experience but I was shopping for fast RAM for Factorio recently and maybe you know this already but idk ill mention it anyways, slower RAM with a low CAS latency seems to be as good as the fast stuff with a higher CAS latency. I found this chart on Reddit:
https://docs.google.com/spreadsheets/d/ ... edit#gid=0

4400mhz CL19 is very close to 3200mhz CL14

Not that the 3200 CL14 is any cheaper or anything, its also very expensive, most cheaper kits at that speed seem to be CL16.

Something to keep in mind.

[orzelek]

https://factoriobox.1au.us/result/39dde ... a322af4206

185 UPS with G.Skill 3200 CL 14 ram.
It seems that most of same CPU ones higher are with higher speed RAM.

[azesmbog]

Not that the 3200 CL14 is any cheaper or anything, its also very expensive, most cheaper kits at that speed seem to be CL16.

This is still a good memory that runs at 3200 with CL16 timings.
My memory is not very good, at 3200 it passes the factoriobox test with timings of 18-17-17-39. But my map doesn't work anymore
Here are two results:
timings 18-17-17-39@3200 - 174 UPS
https://factoriobox.1au.us/result/d02a9 ... 57f78f5226
timings 19-19-19-39@3433 - 177 UPS
https://factoriobox.1au.us/result/c9920 ... d85fa94c45
Alas, this is all that my set of motley memory is capable of
But this is not all that I wanted to say.
Here are the results on my vanilla map

timings 18-18-18-39@3333 - 49 UPS (configuration for test only)
timings 19-19-19-39@3466 - 50 UPS (everyday working configuration)
timings 19-19-19-39@3548 - 51 UPS (configuration for test only)

These are the results obtained using a script, but if you just start the game and stand in one place, then the result is comparable, about ~ 49 UPS

If there are owners of the latest processors from AMD of the 5600-5800-5900 series, then if they do not make it difficult to show their UPS results at startup, and at least when standing in one place for a minute
It is unlikely that my map will ever get to the factoriobox, it is too "non-indicative"
Map link, the latest version of the game, 1.1.21:
https://drive.google.com/file/d/1bZMeGD ... sp=sharing

[ptx0]

using microsoft mimalloc gives me the greatest improvements.

game update time went from 6.8ms to 3.9ms.

[Zlutz]

Fresh 5800X user here.
I did a slight undervolt of the CPU for better speed and power efficiency.
I also limited maximum socket power from 145W to 115W because I only lose like 3% multicore score and 0% single-core AND temperatures never go over 80°C!

16 gigs of 3200MHz DDR4 CL16 RAM with default settings (16-18-18-36-72-1T)

I don't know what is the proper procedure to do them benchmarks with links and stuff, but I saw you guys using this map:

https://factoriobox.1au.us/map/info/4c5 ... e161705504
I downloaded it, put it inside the folder where factorio.exe is and ran:

factorio.exe --benchmark 4c5f65003d84370f16d6950f639be1d6f92984f24c0240de6335d3e161705504.zip

It does a wall of text that ends with this:

Performed 1000 updates in 5050.300 ms
avg: 5.050 ms, min: 4.312 ms, max: 27.025 ms
checksum: 723021452
17.415 Goodbye

That would calculate to an average of 198UPS. Game version is: Factorio 1.1.30 (build 58304, win64, steam)

CPU usage while benchmarking stays at 15% and it uses 67W of power from the CPU socket (only 36W of actual CPU power, measured with Ryzen Master).

When I load this map in the actual game and zoom in with x4 time, it hovers between 185 and 195 UPS

Is this good or bad? I don't even have a reference point...

edit: ok, I got it now... but I don't have my Steam library inside the "Program Files (x86)" so the powershell script doesn't work...