CPU performance benchmarks

Post all other topics which do not belong to any other category.
azesmbog
Fast Inserter
Fast Inserter
Posts: 147
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by azesmbog »

sushi_eater wrote:
Tue Oct 06, 2020 9:46 pm

The result is 94 without huge pages. I'm using Factorio 1.0, Kernel 5.8.13, PDS scheduler, performance governor. There is a HUGE amount of TLB misses with standard pages (around 12'000'000/s with standard pages vs. 1'000'000/s with huge pages).

The CPU boosts to 4.625GHz.
As I understand it, 4.6 is a very good overclocking of the AMD processor.
94 versus 85 is also an excellent result!

Immediately the question is - are kernel tweaks possible only for Ryzen processors, or is there a comparable result for Intel processors too? And the second - if this is so good - why is it not in the main kernel, but must be enabled forcibly? However, these are rhetorical questions.
My main question is as follows.
The test map is very small and unpretentious. It works quite well even on 8 GB of RAM, maybe that's why there are few misses with large pages.
Could you please check my "little sandbox" on your system. True, you have only 32 GB of memory, this is too little :(
My map works with 48 GB of memory, the timings are really low due to the large volume) 19-19-19-38 @ 3333 MHZ, but at the same time it produces 37-38 UPS, it is still playable, but it is impossible to develop further: ((Or an offline script check, well, or at least run, and see the UPS in a few minutes, even while in one place.
A link to saving is in the topic about the sandbox in the topic about megabases, but I can also make a fresh save, the current one.
If not difficult :) For the truth. Vanil.

sushi_eater
Burner Inserter
Burner Inserter
Posts: 10
Joined: Thu Feb 13, 2020 2:22 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by sushi_eater »

azesmbog wrote:
Wed Oct 07, 2020 11:01 am
Could you please check my "little sandbox" on your system. True, you have only 32 GB of memory, this is too little
Factorio doesn't like your save (I tried with 1.0). (I was able to extract the zip, so I don't think the download is corrupted.)

82.739 Warning Map.cpp:340: Map loading failed: Couldn't read from input file. File could be corrupted.
azesmbog wrote:
Wed Oct 07, 2020 11:01 am
Immediately the question is - are kernel tweaks possible only for Ryzen processors, or is there a comparable result for Intel processors too? And the second - if this is so good - why is it not in the main kernel, but must be enabled forcibly?
Exactly the same thing works for Intel. Somebody tested huge pages with a 7700K, it improved performance by 8% or so.

Huge pages don't play nice with all applications and can result in decreased performance and large latency spikes (system freezes for hundreds of milliseconds).

azesmbog
Fast Inserter
Fast Inserter
Posts: 147
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by azesmbog »

It's strange.
Firstly, my save was apparently launched, and secondly, I also launched it in 1.0
and yes, there is version 0.18.33. Ok, now I will save the last save and post it to the exchanger. But this is long, maybe an hour)
It is possible that there is still not enough RAM: (((
Can I try to increase the swap file, or in Linux it is completely different?

So, the last save.
It takes 7 minutes to save the game.
The load is 3 minutes.
After one minute, the UPS stabilizes at around 36.6min -37.2max
https://drive.google.com/file/d/1WxIyEd ... sp=sharing

I'll also leave a link to the old save, last year, it's a little smaller. Maybe it will work.
https://drive.google.com/file/d/1XQ84OF ... sp=sharing

sushi_eater
Burner Inserter
Burner Inserter
Posts: 10
Joined: Thu Feb 13, 2020 2:22 pm
Contact:

Re: CPU performance benchmarks

Post by sushi_eater »

It looks like the Linux version doesn't support your maps. Both Factorio 1.0 and 0.17.79 fail with the same error I mentioned.

Running the Windows version of Factorio under Wine could load your old map (123_311219.zip), but huge pages don't get used. I ran a benchmark for 10'000 ticks:

Performed 10000 updates in 404674.581 ms
avg: 40.467 ms, min: 17.698 ms, max: 19629.176 ms

IME, the min value is reasonably consistent across benchmark runs and would reflect something useful. There is too much paging, so the average is useless.

westjensontexas
Manual Inserter
Manual Inserter
Posts: 1
Joined: Fri Oct 16, 2020 1:56 pm
Contact:

Re: CPU performance benchmarks

Post by westjensontexas »

I also remember back when they released the first Athlon X2 dualcore CPUs that each core used to return a different elapsed real time (only the devil knows why they did that instead of a global time that is the same for all cores)... and various applications crashed shortly after starting because of synchronization problems that arised between their threads. It required a microcode/OS driver update to deal with that.

Pentium100
Burner Inserter
Burner Inserter
Posts: 16
Joined: Fri Aug 21, 2020 5:37 am
Contact:

Re: CPU performance benchmarks

Post by Pentium100 »

I tried running the benchmark (123_311219.zip) on my PC.

Factorio used 33.6GB of RAM and about one core of CPU.

Performed 10000 updates in 707618.482 ms
avg: 70.762 ms, min: 45.903 ms, max: 798.719 ms

My CPU does not have good single-thread performance

azesmbog
Fast Inserter
Fast Inserter
Posts: 147
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: CPU performance benchmarks

Post by azesmbog »

Pentium100 wrote:
Sun Oct 18, 2020 6:30 am
I tried running the benchmark (123_311219.zip) on my PC.

Factorio used 33.6GB of RAM
Launched the test (123_311219.zip) on version 1.0.0
Memory used 35.7 GB
average: 15 ms, or 65 UPS
Well, in order not to get up twice - test of my gigabase
38.jpg
38.jpg (31.67 KiB) Viewed 1932 times
In principle, correlates with the real game - about 37 ups\fps

Syhn
Inserter
Inserter
Posts: 43
Joined: Mon Jan 22, 2018 9:13 pm
Contact:

Re: CPU performance benchmarks

Post by Syhn »

I gave huge pages a shot on my i7-8700k and managed to get the best benchmark to date with Factorio 1.0.0. :D
109 UPS baseline
125 UPS with huge pages
Sorted list of benchmarks to date

I made some slight changes to the variables/steps listed in this reddit post.

Code: Select all

sudo sysctl -w vm.nr_hugepages=8192 # can likely be much less but I have a lot of spare memory
curl -s https://factoriobox.1au.us/scripts/benchmark.sh | LD_PRELOAD=/usr/lib64/libhugetlbfs.so MALLOC_ARENA_MAX=1 HUGETLB_MORECORE=yes HUGETLB_RESTRICT_EXE=factorio bash

maxp779
Inserter
Inserter
Posts: 36
Joined: Thu Oct 08, 2020 4:39 pm
Contact:

Re: CPU performance benchmarks

Post by maxp779 »

Will be interesting to see someone do some benchmarks with a Ryzen 5xxx CPU:
https://hexus.net/tech/reviews/cpu/1464 ... n-9-5900x/

They talk about "cleaner caching" and "reduction in memory latency" which might help factorio a bit.

azesmbog
Fast Inserter
Fast Inserter
Posts: 147
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: CPU performance benchmarks

Post by azesmbog »

so there are already several results - 106 - the best for Windows. and this is a processor with 6 cores \ 12 threads.
For comparison, a 4 core old processor shows 112, at a lower memory frequency.
And further. I don't consider the results in Linux with huge pages, I consider them cheating. It is hardly possible to play with these settings. But this is a private opinion :)

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2333
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: CPU performance benchmarks

Post by Jap2.0 »

azesmbog wrote:
Fri Nov 06, 2020 7:29 pm
Maybe I'm misinterpreting your interpretation, but I consider the 5000 series quite promising - the 5600X looks like it's getting roughly comparable performance with the 9900K, so with slightly higher clocks (and maybe a slight bump due to more cores) the 5800X and 5900X could be quite promising.
There are 10 types of people: those who get this joke and those who don't.

maxp779
Inserter
Inserter
Posts: 36
Joined: Thu Oct 08, 2020 4:39 pm
Contact:

Re: CPU performance benchmarks

Post by maxp779 »

azesmbog wrote:
Fri Nov 06, 2020 7:29 pm
so there are already several results - 106 - the best for Windows. and this is a processor with 6 cores \ 12 threads.
For comparison, a 4 core old processor shows 112, at a lower memory frequency.
And further. I don't consider the results in Linux with huge pages, I consider them cheating. It is hardly possible to play with these settings. But this is a private opinion :)
Ah yeah you're right. Wow Ryzen 5xxx has closed the gap significantly! Someone with 2133mhz DDR4 was getting 95 UPS with the 5600x. And that memory is pretty damn slow. 3600mhz netted someone else 113. Very nice increase from the older gen Ryzens.

azesmbog
Fast Inserter
Fast Inserter
Posts: 147
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: CPU performance benchmarks

Post by azesmbog »

maxp779 wrote:
Sat Nov 07, 2020 12:37 am
Wow Ryzen 5xxx has closed the gap significantly! Someone with 2133mhz DDR4 was getting 95 UPS with the 5600x. And that memory is pretty damn slow. 3600mhz netted someone else 113
Let me be wrong in my judgments, but let me only consider the results in Windows. (in Linux, I consider some of the results, although real for the test, but not indicative, and still have a distant relation to the real game :)
So that's it. Windows.

Two AMD Ryzen 5 5600X results
1.95 UPS with memory 2133
2.106 UPS with 3800 memory

and two results for AMD Ryzen 7 5800X
1.94 UPS with memory 2133
2.113 UPS with 3600 memory

and AMD Ryzen 9 5950X
1.114 UPS with 3600 memory

The last result is generally excellent for AMD processors!
If we assume that the results were obtained at the same processor frequencies, then the effect of fast memory is clearly visible

jape3
Inserter
Inserter
Posts: 24
Joined: Tue Aug 15, 2017 10:31 pm
Contact:

Re: CPU performance benchmarks

Post by jape3 »

Just tested the performance difference between 1.0 and 1.15. On my current train / belt / monster heavy death world map the update time went down from 10.2ms to about 8.2ms.. so a pretty nice performance boost :)

System spec: AMD Ryzen 7 3700X, 32GB of 3600MHz DDR4

elfstone
Burner Inserter
Burner Inserter
Posts: 8
Joined: Fri May 26, 2017 7:18 pm
Contact:

Re: CPU performance benchmarks

Post by elfstone »

Does anyone have any experience with RAM Speeds beyoned 3600? There are (expensive) sets available with speeds of up to 4400 MhZ, will they bring a similar speed bost as going from 3200 to 3600?

I'm currently planning to build a new PC with a Ryzen 5900, but since it probably won't be available for quite some time, I'm thinking about buying the RAM now, so that I can upgrade my current 8GB sticks.

maxp779
Inserter
Inserter
Posts: 36
Joined: Thu Oct 08, 2020 4:39 pm
Contact:

Re: CPU performance benchmarks

Post by maxp779 »

elfstone wrote:
Sun Feb 07, 2021 8:54 am
Does anyone have any experience with RAM Speeds beyoned 3600? There are (expensive) sets available with speeds of up to 4400 MhZ, will they bring a similar speed bost as going from 3200 to 3600?

I'm currently planning to build a new PC with a Ryzen 5900, but since it probably won't be available for quite some time, I'm thinking about buying the RAM now, so that I can upgrade my current 8GB sticks.
I have no direct experience but I was shopping for fast RAM for Factorio recently and maybe you know this already but idk ill mention it anyways, slower RAM with a low CAS latency seems to be as good as the fast stuff with a higher CAS latency. I found this chart on Reddit:
https://docs.google.com/spreadsheets/d/ ... edit#gid=0

4400mhz CL19 is very close to 3200mhz CL14

Not that the 3200 CL14 is any cheaper or anything, its also very expensive, most cheaper kits at that speed seem to be CL16.

Something to keep in mind.

orzelek
Smart Inserter
Smart Inserter
Posts: 3828
Joined: Fri Apr 03, 2015 10:20 am
Contact:

Re: CPU performance benchmarks

Post by orzelek »

https://factoriobox.1au.us/result/39dde ... a322af4206

185 UPS with G.Skill 3200 CL 14 ram.
It seems that most of same CPU ones higher are with higher speed RAM.

azesmbog
Fast Inserter
Fast Inserter
Posts: 147
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: CPU performance benchmarks

Post by azesmbog »

maxp779 wrote:
Thu Feb 11, 2021 3:24 pm

Not that the 3200 CL14 is any cheaper or anything, its also very expensive, most cheaper kits at that speed seem to be CL16.
This is still a good memory that runs at 3200 with CL16 timings.
My memory is not very good, at 3200 it passes the factoriobox test with timings of 18-17-17-39. But my map doesn't work anymore
Here are two results:
timings 18-17-17-39@3200 - 174 UPS
https://factoriobox.1au.us/result/d02a9 ... 57f78f5226
timings 19-19-19-39@3433 - 177 UPS
https://factoriobox.1au.us/result/c9920 ... d85fa94c45
Alas, this is all that my set of motley memory is capable of :)
But this is not all that I wanted to say.
Here are the results on my vanilla map :)

timings 18-18-18-39@3333 - 49 UPS (configuration for test only)
timings 19-19-19-39@3466 - 50 UPS (everyday working configuration)
timings 19-19-19-39@3548 - 51 UPS (configuration for test only)

These are the results obtained using a script, but if you just start the game and stand in one place, then the result is comparable, about ~ 49 UPS

If there are owners of the latest processors from AMD of the 5600-5800-5900 series, then if they do not make it difficult to show their UPS results at startup, and at least when standing in one place for a minute :)
It is unlikely that my map will ever get to the factoriobox, it is too "non-indicative" :)
Map link, the latest version of the game, 1.1.21:
https://drive.google.com/file/d/1bZMeGD ... sp=sharing

User avatar
ptx0
Filter Inserter
Filter Inserter
Posts: 992
Joined: Wed Jan 01, 2020 7:16 pm
Contact:

Re: CPU performance benchmarks

Post by ptx0 »

using microsoft mimalloc gives me the greatest improvements.

game update time went from 6.8ms to 3.9ms.
My Mods - Fish Per Minute base size metric
• • •
Base: Bob's @ 1 Million SPM
• • •
Linear search and overflows are indicative of sloppy coding practices.

Zlutz
Manual Inserter
Manual Inserter
Posts: 4
Joined: Mon Dec 25, 2017 7:58 am
Contact:

Re: CPU performance benchmarks

Post by Zlutz »

Fresh 5800X user here.
I did a slight undervolt of the CPU for better speed and power efficiency.
I also limited maximum socket power from 145W to 115W because I only lose like 3% multicore score and 0% single-core AND temperatures never go over 80°C!

16 gigs of 3200MHz DDR4 CL16 RAM with default settings (16-18-18-36-72-1T)

I don't know what is the proper procedure to do them benchmarks with links and stuff, but I saw you guys using this map:

https://factoriobox.1au.us/map/info/4c5 ... e161705504
I downloaded it, put it inside the folder where factorio.exe is and ran:

factorio.exe --benchmark 4c5f65003d84370f16d6950f639be1d6f92984f24c0240de6335d3e161705504.zip

It does a wall of text that ends with this:

Performed 1000 updates in 5050.300 ms
avg: 5.050 ms, min: 4.312 ms, max: 27.025 ms
checksum: 723021452
17.415 Goodbye

That would calculate to an average of 198UPS. Game version is: Factorio 1.1.30 (build 58304, win64, steam)

CPU usage while benchmarking stays at 15% and it uses 67W of power from the CPU socket (only 36W of actual CPU power, measured with Ryzen Master).

When I load this map in the actual game and zoom in with x4 time, it hovers between 185 and 195 UPS

Is this good or bad? I don't even have a reference point...

edit: ok, I got it now... but I don't have my Steam library inside the "Program Files (x86)" so the powershell script doesn't work...

Post Reply

Return to “General discussion”