CPU performance benchmarks

[Zeblote]

Not sure if anyone cares but that "reference map" runs at around 52UPS on my i5-6600K @3.5GHz and 32GB of dual-channel RAM at 2400MHz (nothing is overclocked) in Linux with factorio version 0.14.23. During night it drops to ~45UPS. I have GTX 770 video card.

Wait... wat? Are you talking about this map or a different one?

Hi
I Used this save http://www.tuwr.net/~factorio/factory%2 ... 0568hr.zip (without mods) on Factorio 0.15.3

On I5-2500 3.30GHz I got 19.5 UPS
On Ryzen 7 1800X OC 4.1GHZ + DDR4 3200GHz I get 22.5 UPS

This is bad, a mid-range CPU from 7 years ago have almost the same preference as a new, overclocked, high end CPU

On every other game the CPU preforms very well and almost idle

Can the factorio team contact AMD and get a dev kit or something to test optimizing on?
It's very disappointing in the current state

[madpav3l]

Not sure if anyone cares but that "reference map" runs at around 52UPS on my i5-6600K @3.5GHz and 32GB of dual-channel RAM at 2400MHz (nothing is overclocked) in Linux with factorio version 0.14.23. During night it drops to ~45UPS. I have GTX 770 video card.

Wait... wat? Are you talking about this map or a different one?

He is talking about this map viewtopic.php?f=204&t=34068 it's less demanding.

[hoho]

Not sure if anyone cares but that "reference map" runs at around 52UPS on my i5-6600K @3.5GHz and 32GB of dual-channel RAM at 2400MHz (nothing is overclocked) in Linux with factorio version 0.14.23. During night it drops to ~45UPS. I have GTX 770 video card.

Wait... wat? Are you talking about this map or a different one?

Oh, my bad. I used the map from this thread

I Used this save http://www.tuwr.net/~factorio/factory%2 ... 0568hr.zip (without mods) on Factorio 0.15.3

On that map (without mods) I'm getting around 23 UPS/FPS during day in 15.10

[madpav3l]

Hi
I Used this save http://www.tuwr.net/~factorio/factory%2 ... 0568hr.zip (without mods) on Factorio 0.15.3

On I5-2500 3.30GHz I got 19.5 UPS
On Ryzen 7 1800X OC 4.1GHZ + DDR4 3200GHz I get 22.5 UPS

This is bad, a mid-range CPU from 7 years ago have almost the same preference as a new, overclocked, high end CPU

On every other game the CPU preforms very well and almost idle

Can the factorio team contact AMD and get a dev kit or something to test optimizing on?
It's very disappointing in the current state

Out of curiosity I underclocked my 6700K to 4.1 GHz and used the same RAM settings (3200 MHz 16-18-18-36-1T) and I got ~33 UPS http://imgur.com/Wdn1G15

[Rseding91]

Hi
I Used this save http://www.tuwr.net/~factorio/factory%2 ... 0568hr.zip (without mods) on Factorio 0.15.3

On I5-2500 3.30GHz I got 19.5 UPS
On Ryzen 7 1800X OC 4.1GHZ + DDR4 3200GHz I get 22.5 UPS

This is bad, a mid-range CPU from 7 years ago have almost the same preference as a new, overclocked, high end CPU

On every other game the CPU preforms very well and almost idle

Can the factorio team contact AMD and get a dev kit or something to test optimizing on?
It's very disappointing in the current state

We don't optimize for specific CPUs (or any hardware configuration for that matter). We optimize the algorithms we write to be more efficient (just do less work) and the data we use to be more cache-friendly - none of the things we do care about what hardware you have - it will all run faster regardless.

AMD has always slower slower than Intel in single threaded performance for a long time - which is what games (and the majority of applications) care about.

[milo christiansen]

It looks like Ryzen suffers from two issues:

1) It has relatively poor single core performance. It's OK, but not great.
2) At the cache level it acts like a NUMA machine. This is fine for software that is optimized for NUMA machines, but can cause cache misses in other software.

Maybe Factorio would perform a little better if it was restricted to cores on the same cache. See this thread for a little more (Linux) info

EDIT: It seems the Windows 10 scheduler already does a good job of this

[MrHick]

We don't optimize for specific CPUs (or any hardware configuration for that matter). We optimize the algorithms we write to be more efficient (just do less work) and the data we use to be more cache-friendly - none of the things we do care about what hardware you have - it will all run faster regardless.

AMD has always slower slower than Intel in single threaded performance for a long time - which is what games (and the majority of applications) care about.

That is our answer, and there is no fix.
One core trying to pull data from the cash at half the speed of memory.
Memory latency on ryzen i get 57 while on i7 i get 17, 7700k can do 13.

Still runs fine but FACTORIO will never be it's strong side.

[SQLek]

It looks like Ryzen suffers from two issues:

1) It has relatively poor single core performance. It's OK, but not great.
2) At the cache level it acts like a NUMA machine. This is fine for software that is optimized for NUMA machines, but can cause cache misses in other software.

Maybe Factorio would perform a little better if it was restricted to cores on the same cache. See this thread for a little more (Linux) info

EDIT: It seems the Windows 10 scheduler already does a good job of this

Interesting...
Anyone tried process affinity? (stopping Factorio from jumping between cores)
Maybe dedicating whole cores only to Factorio? (to get full potential out of core, works miracle in my machinekit setups)

[lexx]

I guess I should of Google ryzen factorio performance before I got it lol

still I am going from a first gen i7-920 to 1800x 3200 ram, so still going to be good enough to keep up with most hosts (I wount be that guy who can't keep up who is using a laptop to play factorio mutiplayer with a A4 amd cpu lol)

will post results of my i7 920 and ryzen in diferant configurations), guess main issue is not been able to limit factorio to using 8mb L3 cache if that is even a thing you can do (but sure it's exposed as 2x8mb) but i can limit it to 4 cores on one side of the ccx so it's not jumping between the ccx 4+4 core group witch can trigger a latency spike between the L3 caches

I guessing ryzen 2 or 3 they mite go with a single L3 cache so it's a true 8 core with no ccx jumping (unless using thredripper or epic cpu but they are workstation and server cpus so understandable)

be Intresting if the high precision timer is affecting stuff a lot and are pepole changing windows performance profile to high performance as that can have a dramatic effect on how fast the CPU performance States and possibly not having smt enabled, as all them 3 together should provided max performance )

[hoho]

guess main issue is not been able to limit factorio to using 8mb L3 cache if that is even a thing you can do

Factorio working dataset is a LOT bigger than any L3 in any CPUs.

I guessing ryzen 2 or 3 they mite go with a single L3 cache so it's a true 8 core with no ccx jumping

Hopefully. Currently, they basically slapped 2 or 4 distinct CPUs into same socket and called it a day

be Intresting if the high precision timer is affecting stuff a lot

It shouldn't really have any effect on actual performance of things.

[ili]

Hi
I Used this save http://www.tuwr.net/~factorio/factory%2 ... 0568hr.zip (without mods) on Factorio 0.15.3

On I5-2500 3.30GHz I got 19.5 UPS
On Ryzen 7 1800X OC 4.1GHZ + DDR4 3200GHz I get 22.5 UPS

This is bad, a mid-range CPU from 7 years ago have almost the same preference as a new, overclocked, high end CPU

On every other game the CPU preforms very well and almost idle

Can the factorio team contact AMD and get a dev kit or something to test optimizing on?
It's very disappointing in the current state

We don't optimize for specific CPUs (or any hardware configuration for that matter). We optimize the algorithms we write to be more efficient (just do less work) and the data we use to be more cache-friendly - none of the things we do care about what hardware you have - it will all run faster regardless.

AMD has always slower slower than Intel in single threaded performance for a long time - which is what games (and the majority of applications) care about.

This FFF shows that you recently did got into this area of spesific optimization:
https://www.factorio.com/blog/post/fff-204

the measurements for this chart were done on a different CPU (i7-6700K vs i7-4790K previously)
The downside is, that you are entering the realm of architecture-specific micro-optimization. If you aren't careful, it can even be bad for performance.

[MeduSalem]

I guessing ryzen 2 or 3 they mite go with a single L3 cache so it's a true 8 core with no ccx jumping

Hopefully. Currently, they basically slapped 2 or 4 distinct CPUs into same socket and called it a day

I wouldn't place my bets on them ever going back to a huge monolithic manycore design ever again. It's not really efficient in the long run because of increasing size, complexity, latency and yield problems.

If at all they will become even more modular the more cores they add in the future... going for Mesh-interconnects/NUMA systems or multi-die designs with EMIB (Intel)/Infinity Fabric (AMD).

[Loewchen]

This FFF shows that you recently did got into this area of spesific optimization.

What part of what FFF shows that?

[ili]

This FFF shows that you recently did got into this area of spesific optimization.

What part of what FFF shows that?

lol, sorry forgot the link
Edited my post with link and quote.

[lexx]

on this map (did not bother with the mods) on 15.33 http://www.tuwr.net/~factorio/factory%2 ... 0568hr.zip

well i get about 18 UPS on my i7-920 + 3.4Ghz (24GB ram stock speeds)

the ryzen 1800x @ stock gets about 23 UPS, guess if i lock the cores to 4Ghz it be a little higher, ram is at 2933 with XMP loaded for "Corsair 32GB Vengeance LPX 3200 CAS 16-18-18-36" stuff ,as wont post if set higher than that, need to tweak other settings i think

this is like just fast install tests no tweaking at all (just got windows on it) and this is after the 1 minute mark and no mods loaded (seems to stay at them UPS numbers after that point on both systems, after long time it does rize a bit)

this issue is only going to affect factorio (compared to other games as they don't act like a database on steroids) as when bases get so large there is only so much you can do at 16ms per tick before somthing can't keep up (be nice if tick rate could be reduced by half or even to 15 with out slowing normal game speed down but i guess that might be complicated)

[malventano]

It looks like Ryzen suffers from two issues:

1) It has relatively poor single core performance. It's OK, but not great.
2) At the cache level it acts like a NUMA machine. This is fine for software that is optimized for NUMA machines, but can cause cache misses in other software.

Maybe Factorio would perform a little better if it was restricted to cores on the same cache. See this thread for a little more (Linux) info

EDIT: It seems the Windows 10 scheduler already does a good job of this

Agreed on both points, but yes the Windows 10 scheduler will try to keep the threads running on the correct affinity (thanks for linking my article, BTW - I found the one person that reads our site ). I'm going to try a few demanding maps the next time we have CPU testing going. I'm also waiting for the first 0.16 build to come out so I can evaluate both the difference across platforms as well as the improvement across those same platforms.

[lexx]

On the note of poor single threaded performance on ryzen is that a assumption as ryzen cpu is not that far off an Intel cpu now (within 10% assuming the Intel cpu is overclocked) ryzen is not bulldozer (the type A or FX cpus with core assist)

one thing I have noticed as I have been doing some limited testing it does seem to prefer staying on one side of the ccx group (1 side of the 4 cores) i get same performance with it set with 4-8 affinity or with it using all threads, so I guess task scheduler is smart enough ,

i am unsure if windows 10 is aware of the 2x8mb L3 cache but the task scheduler might be passively aware of it due to latency of the L3 cache between the 2 4 cores (as it lists it as 16mb not 2x8mb in task manager)

Setting Windows power management to high performance was affecting UPS by about 2,,, I have not disabled the high precision event timer yet as that was another optimisation AMD recommends and I have not oc the cpu to 4ghz yet (need new case for water cooler unsure if the dinky air cooler will handle it )

[aober93]

Coffee Lake 8700K.. just for the record im getting 42 UPS in the "Factory 1 568hour" map. Albeit my CPU is slightly OC, and my RAM is extremely OC (Teamgroup DDR 4133). This almost doubles stock Ryzen in factorio performance. But also my system with defaults is only like 33 UPS or somesuch. The real deal is the memory. Got my system yesterday and been tinkering to find the best settings. Memory set to 3600 appears the sweet spot i figure.
Specs: 8700K Turbo@4,9 Ghz (from 4,7). RAM DDR4 3600, 14-16-16-32-1T

bench1.jpg (30.97 KiB) Viewed 9196 times

bench2.jpg (28.92 KiB) Viewed 9196 times

Damn near AMD Ryzen Threadripper 1920X performance, by that passmark

Hi
I Used this save http://www.tuwr.net/~factorio/factory%2 ... 0568hr.zip (without mods) on Factorio 0.15.3

On I5-2500 3.30GHz I got 19.5 UPS
On Ryzen 7 1800X OC 4.1GHZ + DDR4 3200GHz I get 22.5 UPS

[lexx]

@aober93
What is your system spec mobo and ram

It's amazing there is so much difference I get about 22 ups on that map at the 60 second mark (think that is where you need to log it at) and the avg after that is floating around that number

I did slightly make a mistake going down the ryzen route,, every thing els is fine though just factorio and maybe KSP as both seem to be cpu and ram speed bound late game, all other stuff I do on it as well as other games are fine as they are generally GPU limited as well as other applications work very smooth (opening chrome with 200 tabs open very quickly)

Guess it's just very unique way factorio hammers the ram controller (well it's effectively a data base in super over drive mode late game) 8700k is about 50% faster at stock and normal ram and about 100% faster at 4.8ghz ish with 3600 14/15cl ram witch is insane (normally the ryzen is only behind by 10-15%)

Well i would not say mistake just slightly annoying that just the one game I play at times could cause problems due to single threaded performance and ram support, every other thing I do on this pc games and applications and Vm work as expected and not disappointed about that (having 8 cores you can do so much at one time is amazing just not at factorio as it runs like a i7-2700k)

[aober93]

Bear in mind the optimizations for 0.16. It is possible a reduction of memory load will lead to ryzen catching up.

What im seeing is mostly the result of OC memory. I got 44UPS now, while i OC'ed my memory to 3800 14-14-14-28-1T @1,5V. Leads to the situation that the 60second mark is in at 30seconds for me due gamespeed.

Other than that its 8700K but i already meantioned that.

bench3.jpg (28.71 KiB) Viewed 9146 times