CPU performance benchmarks

Post all other topics which do not belong to any other category.
jape3
Inserter
Inserter
Posts: 27
Joined: Tue Aug 15, 2017 10:31 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by jape3 »

Just tried on the ryzen 3700x / 32GB 3600MHz ram, factorio versio 18.34. Got an average of 86.7UPS on 5 runs

Code: Select all

./factorio --benchmark "stevetrovs_10K_mega_belt_base.zip" --benchmark-ticks 1000 --disable-audio --mod-directory /dev/null --benchmark-runs 5

52.028 Loading map /home/janne/factorio_Krastor2_SP/saves/stevetrovs_10K_mega_belt_base.zip: 40021782 bytes.
  52.036 Loading level.dat: 145798854 bytes.
  52.044 Info Scenario.cpp:187: Map version 0.17.31-0
  52.046 Info PrototypeMigrationList.cpp:61: Activating migration base/2019-04-27_Factorio_0.17.35.json
  52.046 Info PrototypeMigrationList.cpp:61: Activating migration base/2019-04-30_Factorio_0.17.35.json
  52.046 Info PrototypeMigrationList.cpp:61: Activating migration base/2019-05-01_Factorio_0.17.35.json
  64.270 Loading script.dat: 339 bytes.
  64.272 Checksum for script /home/janne/factorio_Krastor2_SP/temp/currently-playing/control.lua: 957497751
  64.272 Applying migration: Base Mod: 2020-01-27_Factorio_0.18.02.lua
  64.273 Applying migration: Base Mod: 2020-03-18_Factorio_0.18.14.lua
  64.373 Applying migration: Base Mod: 2020-05-27_Factorio_0.18.28.lua
  64.378 Checksum for script /home/janne/factorio_Krastor2_SP/temp/currently-playing/control.lua: 4108740459

azesmbog
Filter Inserter
Filter Inserter
Posts: 252
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by azesmbog »

orzelek wrote:
Tue Jun 30, 2020 10:19 pm
This benchmark shows how bad Ryzen is for Factorio:
https://factoriobox.1au.us/result/f4fe9 ... 941ac1ea8f
Thats 75 UPS only compared to more then 100 for intel CPU's.
The developers have already quite plausibly explained why the parallelization of tasks is done and as much as possible.
And I believe them.
But I would also like to hear from the developers about such a big difference in the operation of INTEL vs AMD processors.
What is the reason for this and how do they understand it and let them explain on the fingers, as for the smallest)
Why are the best processors from AMD 8 cores 16 threads lose to a processor 10 years ago three generations ago costing less than $ 80 with 4 cores :))

Rseding91
Factorio Staff
Factorio Staff
Posts: 13177
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Anyone Ryzen Benchmarks?

Post by Rseding91 »

azesmbog wrote:
Sat Jul 04, 2020 6:23 pm
orzelek wrote:
Tue Jun 30, 2020 10:19 pm
This benchmark shows how bad Ryzen is for Factorio:
https://factoriobox.1au.us/result/f4fe9 ... 941ac1ea8f
Thats 75 UPS only compared to more then 100 for intel CPU's.
The developers have already quite plausibly explained why the parallelization of tasks is done and as much as possible.
And I believe them.
But I would also like to hear from the developers about such a big difference in the operation of INTEL vs AMD processors.
What is the reason for this and how do they understand it and let them explain on the fingers, as for the smallest)
Why are the best processors from AMD 8 cores 16 threads lose to a processor 10 years ago three generations ago costing less than $ 80 with 4 cores :))
We don't write code for any specific processor: just pure C++. So, any differences in performance being seen is due to the hardware itself. AMD CPUs are slower than Intels when it comes to memory access and games do a ton of that. Factorio optimizes actual calculations done by the CPU a lot and so the limiting factor ends up being how fast memory can be pulled/pushed to system RAM.
If you want to get ahold of me I'm almost always on Discord.

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2339
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: Anyone Ryzen Benchmarks?

Post by Jap2.0 »

azesmbog wrote:
Sat Jul 04, 2020 6:23 pm
orzelek wrote:
Tue Jun 30, 2020 10:19 pm
This benchmark shows how bad Ryzen is for Factorio:
https://factoriobox.1au.us/result/f4fe9 ... 941ac1ea8f
Thats 75 UPS only compared to more then 100 for intel CPU's.
The developers have already quite plausibly explained why the parallelization of tasks is done and as much as possible.
And I believe them.
But I would also like to hear from the developers about such a big difference in the operation of INTEL vs AMD processors.
What is the reason for this and how do they understand it and let them explain on the fingers, as for the smallest)
Why are the best processors from AMD 8 cores 16 threads lose to a processor 10 years ago three generations ago costing less than $ 80 with 4 cores :))
Ryzen has on the order of double the memory latency of Intel chips (due to architectural differences, there's plenty of material online if you look), which is a significant bottleneck in Factorio.
There are 10 types of people: those who get this joke and those who don't.

azesmbog
Filter Inserter
Filter Inserter
Posts: 252
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by azesmbog »

Jap2.0 wrote:
Sat Jul 04, 2020 10:06 pm
Ryzen has on the order of double the memory latency of Intel chips (due to architectural differences, there's plenty of material online if you look)
Rseding91 wrote:
Sat Jul 04, 2020 7:52 pm
Factorio optimizes actual calculations done by the CPU a lot and so the limiting factor ends up being how fast memory can be pulled/pushed to system RAM.
Probably this is what I wanted to hear and understand.
What is the main problem with the speed of access of the processor to the system memory.
Thanks for the clarification.

azesmbog
Filter Inserter
Filter Inserter
Posts: 252
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by azesmbog »

Syhn wrote:
Tue Jun 30, 2020 7:13 pm

Hey, sorry about that. I don't log into the forums often.
I've gone ahead and fix it and all of them show up now. :D

Hey.
I hope you come here again.
thanks for correcting the script, but my results for 6400 could have been deleted, I would have made new ones. Anyway.
Now the question is different.
Here is a screenshot of two results from the same map.
stevetrovs.jpg
stevetrovs.jpg (71.28 KiB) Viewed 7987 times
As you can see, the difference in 7 UPS is quite significant, someone will have more.
I certainly understand that everything is in the same conditions, and the results are therefore valid,
but maybe you need to change the amount of update time in the script,
(instead of 1000, make 5000/10000?) to neutralize the difference in the screenshot.
I hope it's clear where it came from ?? :)

Syhn
Inserter
Inserter
Posts: 43
Joined: Mon Jan 22, 2018 9:13 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by Syhn »

azesmbog wrote:
Sun Aug 09, 2020 10:38 am
As you can see, the difference in 7 UPS is quite significant, someone will have more.
I certainly understand that everything is in the same conditions, and the results are therefore valid,
but maybe you need to change the amount of update time in the script,
(instead of 1000, make 5000/10000?) to neutralize the difference in the screenshot.
I hope it's clear where it came from ?? :)
What versions are those benchmarks from? Base on the first one being 10 runs, it looks like that one is a bit older as I changed that a while ago. ;) A lot can change between Factorio versions so performance shouldn't be expected to be the same. Also the map is fairly outdated at this point and I should update it, though that would make it impossible to compare against other benchmarks until there are a lot of benchmarks with whatever the new map is. I'm soon going to let the use choose which map to benchmark, so maybe the user can choose to benchmark both maps...

azesmbog
Filter Inserter
Filter Inserter
Posts: 252
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by azesmbog »

Syhn wrote:
Sun Aug 09, 2020 6:47 pm
What versions are those benchmarks from? Base on the first one being 10 runs, it looks like that one is a bit older as I changed that a while ago. ;) A lot can change between Factorio versions so performance shouldn't be expected to be the same. Also the map is fairly outdated at this point and I should update it, though that would make it impossible to compare against other benchmarks until there are a lot of benchmarks with whatever the new map is. I'm soon going to let the use choose which map to benchmark, so maybe the user can choose to benchmark both maps...
This is it, that this is the same card, it is tested on the same version, but the first version is the offline version.
Now why is there such a big difference.
You have a map on the website version 0.17.31
Therefore, on any other version other than 0.17.31, the save file is first converted to the current version, and then testing. This is where 7 UPS is lost.
I just took it uploaded first in the game for version 0.18.45 this save, waited for it to convert it, and saved it as 0.18.45, without even leaving the place.
Then I just tested this map offline. Hence the difference in the UPS, you see, a rather significant difference.
And this is only with this map, with others, the difference can be even more significant, up to 20 UPS between the same versions of the game-save, and different game-save.
I expressed myself too incomprehensibly ??

Syhn
Inserter
Inserter
Posts: 43
Joined: Mon Jan 22, 2018 9:13 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by Syhn »

azesmbog wrote:
Sun Aug 09, 2020 7:40 pm
Now why is there such a big difference.
You have a map on the website version 0.17.31
Therefore, on any other version other than 0.17.31, the save file is first converted to the current version, and then testing. This is where 7 UPS is lost.
This should not be the case. Benchmarks do not include the conversion time, only the time running the ticks. I just tested it myself with the original map and pre-converted map:

Code: Select all

factorio --benchmark 0.17.31.zip --benchmark-runs 3
Performed 1000 updates in 13639.490 ms
Performed 1000 updates in 13274.844 ms
Performed 1000 updates in 13551.489 ms

factorio --benchmark 0.18.45.zip --benchmark-runs 3
Performed 1000 updates in 13345.249 ms
Performed 1000 updates in 13328.553 ms
Performed 1000 updates in 13630.641 ms
As you can see they are nearly identical. Here is the map if you want to test yourself: https://drive.google.com/file/d/1NWSnQx ... sp=sharing

I do not know where you are getting the 7 UPS difference that you are.

azesmbog
Filter Inserter
Filter Inserter
Posts: 252
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by azesmbog »

The link is not public.

Syhn
Inserter
Inserter
Posts: 43
Joined: Mon Jan 22, 2018 9:13 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by Syhn »

Whoops... Fixed.

thetalon
Manual Inserter
Manual Inserter
Posts: 1
Joined: Thu Aug 13, 2020 11:19 am
Contact:

Re: Anyone Ryzen Benchmarks?

Post by thetalon »

Keeper wrote:
Fri Mar 03, 2017 12:51 pm
My first thought upon seeing ryzen was that as I currently have an 8350. My mega factory is running at 20 25 ups. The one core is locked at 4.2ghz the rest are chillin. We'll probably see the same in this, when they say how much "better" the cpu is they're on about multi core tasks typically rendering or runnin multiple cpu intensive tasks. My question is how much better are the individual cores? Look at what they compare it to, I'm still thinking to lug it out with this old cpu, it's never done 8 cores work since I found facto rio. a decent motherboard and faster ram is required it seems
I had an FX8350, and I upgraded to a Ryzen 5 2600. My advice to you is to get rid of that thing ASAP. It's a decent CPU for its age, but it doesn't compare to anything that has hyperthreading or multithreading. Your CPU performance will at minimum double, in almost anything that supports multithreading. I say at minimum because the Ryzen CPU has better IPC than the FX series, as well. It's a huge upgrade as far as gaming is concerned
Last edited by thetalon on Thu Aug 13, 2020 11:26 am, edited 1 time in total.

xxdolcexx
Burner Inserter
Burner Inserter
Posts: 14
Joined: Fri Mar 29, 2013 9:27 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by xxdolcexx »

https://factoriobox.1au.us/result/d0b02 ... c01985c182

why do i have so bad results :( 4.5 ghz 3600ddr4 i need new cpu wich>?

azesmbog
Filter Inserter
Filter Inserter
Posts: 252
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by azesmbog »

xxdolcexx wrote:
Wed Sep 02, 2020 3:47 am

why do i have so bad results :(

I would also say that the results are low. I have a much higher result on Skylake 6400, and the processor is much worse

ske
Filter Inserter
Filter Inserter
Posts: 411
Joined: Sat Oct 17, 2015 8:00 am
Contact:

Re: Anyone Ryzen Benchmarks?

Post by ske »

xxdolcexx wrote:
Wed Sep 02, 2020 3:47 am
https://factoriobox.1au.us/result/d0b02 ... c01985c182

why do i have so bad results :( 4.5 ghz 3600ddr4 i need new cpu wich>?
Do you have something else running in the background? (Could be the antivirus doing a full scan or something else.)

Does the benchmark get slower and slower if you run it multiple times? (Check temperature and frequencies. It could be that your cooling is not good enough.)
xxdolcexx wrote:
Wed Sep 02, 2020 3:47 am
i need new cpu wich>?
If you're on a tight budget Ryzen 3950X. If you can afford something decent try Threadripper. If you need more power, go with EPYC.

azesmbog
Filter Inserter
Filter Inserter
Posts: 252
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by azesmbog »

Syhn wrote:
Sun Aug 09, 2020 8:24 pm
As you can see they are nearly identical. Here is the map if you want to test yourself: https://drive.google.com/file/d/1NWSnQx ... sp=sharing

I do not know where you are getting the 7 UPS difference that you are.
With this saving, the results are identical, but I did two resave myself - with them I got the result 6 UPS higher. It does not matter.
I very often add results to the site from everything I can reach. Recently I have added results from several laptops and from one computer with an AMD processor :)
But yesterday I tested my old I5-6400-non-K 2.7 GHz processor. Some of the results are on the site, but I also did a lot of offline tests.
I will share and try to explain the results.
The first stage of tests - with a constant processor frequency of approximately @ 4.7 GHz and fixed primary timings - the memory frequency changed from the minimum to the maximum. This is not entirely correct for real memory. at a lower frequency, the timings should be less, but this is exactly a test.
So the results are:

I5-6400-non-K 2,7@4,7 GHz
18-19-19-38 1390 MHz - 60 UPS
18-19-19-38 1622 MHz - 68 UPS
18-19-19-38 1854 MHz - 76 UPS
18-19-19-38 2086 MHz - 82 UPS
18-19-19-38 2316 MHz - 88 UPS
18-19-19-38 2548 MHz - 93 UPS
18-19-19-38 2780 MHz - 97 UPS
18-19-19-38 3010 MHz - 102 UPS
18-19-19-38 3474 MHz - 110 UPS
18-19-19-38 3704 MHz - 113 UPS
18-19-19-36 3712 MHz - 113 UPS

Note that two memory sticks with a nominal operating frequency of 3000 MHz (16-18-18-38 timing, 1,35 V) worked at the maximum safe voltage of 1.45 V.
Therefore, the second stage of tests was designed for the nominal frequency, but with different timings at the same processor frequency of -4.7 GHz
So the results are:

I5-6400-non-K 2,7@4,7 GHz
18-19-19-38 3010 MHz - 102 UPS
16-18-18-38 3010 MHz - 105 UPS (2T, nominal, default memory)
16-18-18-38 3010 MHz - 106 UPS (1T, nominal, default memory)
15-16-16-34 3010 MHz - 106 UPS
15-16-16-32 3012 MHz - 108 UPS
14-16-16-32 3012 MHz - 108 UPS
14-16-16-32 3016 MHz - 108 UPS

Draw your own conclusions from the tests :)
I can only say that at maximum acceleration
an old five-year-old processor released and several generations ago performs on equal terms in this game with some top-end processors I7-I9 and surpasses any top-end processors from AMD.
I emphasize - with a good memory :)

Scifimyth
Burner Inserter
Burner Inserter
Posts: 6
Joined: Sun Apr 17, 2016 8:30 am
Contact:

Re: Anyone Ryzen Benchmarks?

Post by Scifimyth »

So I just found out about this test and decided to run it to compare to some of the others in this thread. This is on an i7 8700k @ 5ghz with 32gb (4x8gb) of 3200mhz Ram. No single core ever went above 32% under either Task Manager or Resource Monitor.
powershell_2020-10-02_03-33-07.png
powershell_2020-10-02_03-33-07.png (21.02 KiB) Viewed 6991 times

sushi_eater
Burner Inserter
Burner Inserter
Posts: 13
Joined: Thu Feb 13, 2020 2:22 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by sushi_eater »

When using huge pages on Linux, Ryzen 3900x is as fast as a i9-9900K. I'm getting 114UPS:

https://factoriobox.1au.us/result/880e5 ... 4e666983c9

Transparent huge page support needs to be enabled in the kernel:
/sys/kernel/mm/transparent_hugepage/enabled:always [madvise] never
/sys/kernel/mm/transparent_hugepage/defrag:always defer defer+madvise [madvise] never


The following environment variables need to be set when running Factorio (libhugetlbfs needs to be installed):
MALLOC_ARENA_MAX=1
LD_PRELOAD=/usr/lib64/libhugetlbfs.so
HUGETLB_MORECORE=thp
HUGETLB_RESTRICT_EXE=factorio


The MALLOC_ARENA_MAX is critical. If it's not set, only a part of the memory allocations use huge pages.

The CPU is not overclocked and PBO is disabled. The RAM is overclocked and runs at 3600MHz CL16.

azesmbog
Filter Inserter
Filter Inserter
Posts: 252
Joined: Mon Jan 28, 2019 12:05 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by azesmbog »

An interesting result.
Is it possible another test, at the same frequency and with the same memory, but without these kernel tweaks ?? To see the difference, before and after :)
there is a very big difference between 114 and 85 (the usual result is also with good memory :)

sushi_eater
Burner Inserter
Burner Inserter
Posts: 13
Joined: Thu Feb 13, 2020 2:22 pm
Contact:

Re: Anyone Ryzen Benchmarks?

Post by sushi_eater »

azesmbog wrote:
Tue Oct 06, 2020 7:06 pm
An interesting result.
Is it possible another test, at the same frequency and with the same memory, but without these kernel tweaks ?? To see the difference, before and after :)
there is a very big difference between 114 and 85 (the usual result is also with good memory :)
The result is 94 without huge pages. I'm using Factorio 1.0, Kernel 5.8.13, PDS scheduler, performance governor. There is a HUGE amount of TLB misses with standard pages (around 12'000'000/s with standard pages vs. 1'000'000/s with huge pages).

The CPU boosts to 4.625GHz.

Post Reply

Return to “General discussion”