CPU performance benchmarks

[MrHick]

I would guess it is the game, my single player map runs better on my i7 2.7 HQ laptop.
It runs 60/60 most of the time while on ryzen it runs 50/60 most of the time with crazy dips.
While other games run 2-3 times higher FPS on the ryzen box (4K). Laptop iz 860M so it sucks ass at 4K
Even rimworld runs way better on ryzen and i was expecting the same crappy results.

HMMMMM

[aober93]

My intel is also faster but not by much. The OC Ryzen by Mr Hick is still impressive, cause it wipes the floor with intel in apps *drool* i cant ignore that. Too bad about factorio tho. I very hate the slowdowns in the game. I always think of a power issue when inserters go all slow

[xng]

In the future someone in the dev team will discover C++14 and parallelisation on big datachunks through streams.

[Rseding91]

In the future someone in the dev team will discover C++14 and paralellisation on big datachunks through streams.

What does c+=14 have to do with parallelism? We can already use threading and we do for rendering and chunk generation.

[xng]

In the future someone in the dev team will discover C++14 and parallelisation on big datachunks through streams.

What does c+=14 have to do with parallelism? We can already use threading and we do for rendering and chunk generation.

To a big extent it is C++11 too, but parallelism and allocators is made better with improvements in C++14, with streams you can automatically parallelize computation of big arrays in memory with the help of pure functions. You can't really improve networking and such with it as the latency is mostly waiting times, but if you have for example positions, you can do translation computations for each position equally shared over existing cores, without even having to know if more than one core exists or not, you can literally do pure function calculations in 1/8th of the time on an 8-core, if there are at least 8 of those computations. For real life increase in performance it need to be a lot more than 8 in this scenario because of the setup of the collection stream, but it doesn't take very much to gain a lot of performance for what usually takes most time in applications, iteration and calculations through an array of data. Because it demands pure functions (only thing that is safe in parallelism) you would need to swap buffers, the result of one iteration goes in the second buffer, and then change pointer to the new one, and next time the other way around, and that way you avoid having to do any extra allocations.

Edit: What I am humbly trying to say is that usually the best approach is to put concurrency in all the spots it is possible, instead of dividing the program up in threads with different purposes as that usually leads to having to use mutex and risk locking. You basically usually gain more from dividing each big iteration in threads, calculating data that is one step data (game of life style, one ups in factorio terms I presume) where this exact state leads to the other exact state. I am not very good with sounding humble in text though, so I'm sorry just in case it sounds worse than I mean.

[BenSeidel]

In the future someone in the dev team will discover C++14 and parallelisation on big datachunks through streams.

What does c+=14 have to do with parallelism? We can already use threading and we do for rendering and chunk generation.

To a big extent it is C++11 too, but parallelism and allocators is made better with improvements in C++14, with streams you can automatically parallelize computation of big arrays in memory with the help of pure functions. You can't really improve networking and such with it as the latency is mostly waiting times, but if you have for example positions, you can do translation computations for each position equally shared over existing cores, without even having to know if more than one core exists or not, you can literally do pure function calculations in 1/8th of the time on an 8-core, if there are at least 8 of those computations. For real life increase in performance it need to be a lot more than 8 in this scenario because of the setup of the collection stream, but it doesn't take very much to gain a lot of performance for what usually takes most time in applications, iteration and calculations through an array of data. Because it demands pure functions (only thing that is safe in parallelism) you would need to swap buffers, the result of one iteration goes in the second buffer, and then change pointer to the new one, and next time the other way around, and that way you avoid having to do any extra allocations.

Edit: What I am humbly trying to say is that usually the best approach is to put concurrency in all the spots it is possible, instead of dividing the program up in threads with different purposes as that usually leads to having to use mutex and risk locking. You basically usually gain more from dividing each big iteration in threads, calculating data that is one step data (game of life style, one ups in factorio terms I presume) where this exact state leads to the other exact state. I am not very good with sounding humble in text though, so I'm sorry just in case it sounds worse than I mean.

There are many techniques for introducing parallelism into your code, and the idea of "streams" is probably the worst out of the lot. While the are really easy to explain and make really nice powerpoint slide shows they don't really work in the general case. This is because, as you said, they introduce the requirement of "pure functions" (don't real know what an "impure function" is, let me know if it's anything like a procedure) when the underlying principle: the map function, has no such requirements. Map is able to do everything a stream is able to do with the added benefit of allowing decomposable functions and state filled procedures as the parameter (as it's a bastard-child of the fold function).

While there are these "great" "new" "features" in new programming languages/versions, it's the transformation of your problem into the application of these programming paradigms that causes all the issues. In the case of streams, there aren't many that come naturally. Learning how to use fold or map, both about the same level of complexity as streams for procedural programmers (maybe even less), gives you far more expressive power as the mapping (pun intended) from your problem to the programming structure makes sense.

Don't get me wrong, there are a number of places where it would fit right in. But being able to give one good use-case does not a good model make.

So far we (the programmers and researchers alike) have only two main ways of doing parallelism.
1) Functionally: dividing your application into separate areas, each area dealing with a specific function of your application
2) List comprehension: Anything that takes some collection of data and needs to transform it into another form, where each transformation has some level of independence from the other transformations.

It's sad, but that's all we have. We are sitting on our hands waiting for the next "A case against the goto statement" that will allow us to expand our horizons.

What does c+=14 have to do with parallelism?

It depends on if c is visible from another thread. If so, then the assignment may have a race condition?

[xng]

There are many techniques for introducing parallelism into your code, and the idea of "streams" is probably the worst out of the lot. While the are really easy to explain and make really nice powerpoint slide shows they don't really work in the general case. This is because, as you said, they introduce the requirement of "pure functions" (don't real know what an "impure function" is, let me know if it's anything like a procedure) when the underlying principle: the map function, has no such requirements. Map is able to do everything a stream is able to do with the added benefit of allowing decomposable functions and state filled procedures as the parameter (as it's a bastard-child of the fold function).

While there are these "great" "new" "features" in new programming languages/versions, it's the transformation of your problem into the application of these programming paradigms that causes all the issues. In the case of streams, there aren't many that come naturally. Learning how to use fold or map, both about the same level of complexity as streams for procedural programmers (maybe even less), gives you far more expressive power as the mapping (pun intended) from your problem to the programming structure makes sense.

Don't get me wrong, there are a number of places where it would fit right in. But being able to give one good use-case does not a good model make.

So far we (the programmers and researchers alike) have only two main ways of doing parallelism.
1) Functionally: dividing your application into separate areas, each area dealing with a specific function of your application
2) List comprehension: Anything that takes some collection of data and needs to transform it into another form, where each transformation has some level of independence from the other transformations.

It's sad, but that's all we have. We are sitting on our hands waiting for the next "A case against the goto statement" that will allow us to expand our horizons.

What does c+=14 have to do with parallelism?

It depends on if c is visible from another thread. If so, then the assignment may have a race condition?

Yeah, fair enough, it's never "just to do" anything, the current datastructure and especielly how it is affected by plugin-like modding might demand lots of work to even begin to take advantage of modern concurrency.

...they introduce the requirement of "pure functions" (don't real know what an "impure function" is, let me know if it's anything like a procedure)...

A pure function is a function that takes a parameter (the value/s) and return value/s, and doesn't have side effects. Basically:

Code: Select all

// (Pseudo-Code, in real C++ you have much more meta programming possibilities to make functions like these compile-time, inlined and so on)
// Takes two vectors and adds them together and returns the new vector
// without any possible sideeffects (ie. it doesn't change state)

Vector translateVector(Vector vA, Vector vB) {
  return new Vector(vA.x+vB.x, vA.y+vB.y, vA.y+vB.y);
}

An impure (or normal nonpure function) is a function that changes state, all functions that return void for example must be nonpure or they would not do anything at all:

Code: Select all

void startServer() {
  this->startServer(this->ipadress);
}

// or

int add5(int& x) { x = x + 5; return x; }

[BenSeidel]

The pure function thing was a dig at the terminology of the industry. Putting "pure" in front of "function" is like putting "metallic" in front of iron. It's there by definition. Likewise there is no such thing as an "impure function" it is either a function or it is not. The correct term is "procedure" when state is involved.

[xng]

The pure function thing was a dig at the terminology of the industry. Putting "pure" in front of "function" is like putting "metallic" in front of iron. It's there by definition. Likewise there is no such thing as an "impure function" it is either a function or it is not. The correct term is "procedure" when state is involved.

Yes, calling a pure function pure helps when reasoning about code, and the methodology of functional programming makes it easier to simplify testing and making your "procedures" deterministic because you avoid sideeffects in as much code as possible. It is very real, and it is a valueable thing to know as a programmer no matter what language you are programming in, and it becomes even more important when working in teams.

I am not sure if calling an iron plate metallic helps with reasoning about it, because iron is always metallic while functions are not always side effect free. Iron is also not related to programming at all so it doesn't matter much.

I appreciate your interest in the matter =)

[BenSeidel]

Yes, calling a pure function pure helps when reasoning about code

It's not about your ability to reason about code, it's about your ability to discuss the code with other people. The term function has a well defined meaning but unfortunately programmers have bastardised it beyond all recognition. Physicists don't talk about a proton being an impure neutron because it would become confusing and would prevent any meaningful discussion. It's may feel strange at first, but once you start using the terminology correctly, it becomes far easier to discuss things with your team. The worst part is that our industry is littered with examples of these incorrect uses of terms. Another of my favourite ones is dictionary vs map.

the methodology of functional programming makes it easier to simplify testing and making your "procedures" deterministic because you avoid side effects in as much code as possible. It is very real, and it is a valueable thing to know as a programmer no matter what language you are programming in, and it becomes even more important when working in teams.

Functional programming doesn't really make your testing easier, just limits the types of bugs you come across. Any gain you get from having less bugs is really outweighed by the increase in the difficulty of the code you are writing.
Procedures are pretty much always deterministic (you really have to go out of your way to find a nondeterministic procedure). You know exactly what is going to occur when you execute it, or more to the point, your computer will produce the same outcome each time. If you can unit test it (or automate the testing process) then you must have deterministic code.

Writing functions when in a team gives you no more benefit than writing object oriented code or working in a memory managed environment. The incorrect application of functional programming when in teams will be detrimental to the development process, just like any programming technique. Also, It is not a magic bullet that solves all issues as functional programming sucks.

I am not sure if calling an iron plate metallic helps with reasoning about it, because iron is always metallic while functions are not always side effect free. Iron is also not related to programming at all so it doesn't matter much.

Just like if you have non-metallic iron, you don't have iron, if you have a function with side effects, then you don't have a function. As I said, functions are always side effect free. They cannot rely on anything that is not passed in as a parameter or is not a global constant (like pi). Any "function" that has side effects or relies on some form of non-parameterised state is a procedure.

I appreciate your interest in the matter =)

Software development is a technical field, yet we as professionals in the field don't seem to understand the importance of technical terms in our technical industry. The incorrect application of a term leads to massive confusion between people who should be able to communicate extremely precise meanings. Sure, while it does not matter that you interchange "function" and "procedure" when talking to your team when describing a piece of code, it sure as hell does matter when you are describing what is a valid parameter. It seems ludicrous, but what if you saw documentation saying "the first parameter must not be a void pointer" instead of saying "the first parameter must not be a null pointer"? Once you start looking into the defined meanings of terms you will see them everywhere.

some other fun ones are:
Does anyone know the difference between an interface, a trait or a protocol? Why do we use the term "generic class" instead of the correct term "parameterised class" or "parametric class"? Why do we call polymorphic functions polymorphic? There is nothing polymorphic about them!

[xng]

I'm sorry BenSeidel, I don't follow you, to me it looks like nothing you say comes from reality, it must be some language barrier or something.

[ili]

Hi
I Used this save http://www.tuwr.net/~factorio/factory%2 ... 0568hr.zip (without mods) on Factorio 0.15.3

On I5-2500 3.30GHz I got 19.5 UPS
On Ryzen 7 1800X OC 4.1GHZ + DDR4 3200GHz I get 22.5 UPS

This is bad, a mid-range CPU from 7 years ago have almost the same preference as a new, overclocked, high end CPU

On every other game the CPU preforms very well and almost idle

Can the factorio team contact AMD and get a dev kit or something to test optimizing on?
It's very disappointing in the current state

[d4rkpl4y3r]

Hi
I Used this save http://www.tuwr.net/~factorio/factory%2 ... 0568hr.zip (without mods) on Factorio 0.15.3

On I5-2500 3.30GHz I got 19.5 UPS
On Ryzen 7 1800X OC 4.1GHZ + DDR4 3200GHz I get 22.5 UPS

This is bad, a mid-range CPU from 7 years ago have almost the same preference as a new, overclocked, high end CPU

On every other game the CPU preforms very well and almost idle

Can the factorio team contact AMD and get a dev kit or something to test optimizing on?
It's very disappointing in the current state

I just loaded that save in vanilla 0.15.5 and get 22 - 25 UPS with Ryzen 7 1700X @ 3.8Ghz & DDR4 2933MT/s 14-14-14-34. Are you sure your RAM runs at 3200Mhz?

[ili]

Hi
I Used this save http://www.tuwr.net/~factorio/factory%2 ... 0568hr.zip (without mods) on Factorio 0.15.3

On I5-2500 3.30GHz I got 19.5 UPS
On Ryzen 7 1800X OC 4.1GHZ + DDR4 3200GHz I get 22.5 UPS

This is bad, a mid-range CPU from 7 years ago have almost the same preference as a new, overclocked, high end CPU

On every other game the CPU preforms very well and almost idle

Can the factorio team contact AMD and get a dev kit or something to test optimizing on?
It's very disappointing in the current state

I just loaded that save in vanilla 0.15.5 and get 22 - 25 UPS with Ryzen 7 1700X @ 3.8Ghz & DDR4 2933MT/s 14-14-14-34. Are you sure your RAM runs at 3200Mhz?

Thank you for your input on this
See this screenshot with the game on ~22.5 UPS and PC settings on the right, your DDR timing are better but my memory should be faster overall as the GHz is higher
Can you check again? open the save and wait for 60 seconds for the UPS to stabilize (when its start to get dark) and see what you get then.
Did you set anything spacial on the game or on the system or do you know of something I should check/set?
The only thing I can think of is reinstalling Windows as I saw it's recommended when switching to Ryzen from Intel

Also if you can run PassMark http://www.passmark.com/ftp/petst.exe and share your CPU & Memory score so we can compere
Thank you

2017-04-30_19-49-21.jpg (698.85 KiB) Viewed 8489 times

2017-04-30_20-57-17.jpg (52.34 KiB) Viewed 8483 times

2017-04-30_20-57-27.jpg (46.68 KiB) Viewed 8483 times

[d4rkpl4y3r]

After one minute of running the save it stabilizes at ~22.3UPS. Factorio is very sensitive to ram bandwith and latancy, thighter timings do matter: https://www.reddit.com/r/factorio/comme ... ed_fpsups/

[Zeblote]

That save runs at ~35 ups with my 7700k.

[ili]

After one minute of running the save it stabilizes at ~22.3UPS. Factorio is very sensitive to ram bandwith and latancy, thighter timings do matter: https://www.reddit.com/r/factorio/comme ... ed_fpsups/

Ok
thank you!
so your results are more or less consistent with mine
Your timing are better, my GHz is higher so I get a fraction of a UPS more and a little higher memory score

That save runs at ~35 ups with my 7700k.

Zeblote can you post your PassMark scores to so we have a reference?
thank you

I think it's all comes down to "Memory Latency" (which hopefully can get optimize in software) and "CPU Single Thread Performance" (which will probably not be improved soon as Factorio multithreading is problematic)

[Zeblote]

Zeblote can you post your PassMark scores to so we have a reference?

WIth i7 7700k @ 5Ghz, 16GB memory at 3000Mhz CL15

Seems like the memory latency with your ryzen is absolute garbage, even though it should basically be the same as mine.

[madpav3l]

After one minute of running the save it stabilizes at ~22.3UPS. Factorio is very sensitive to ram bandwith and latancy, thighter timings do matter: https://www.reddit.com/r/factorio/comme ... ed_fpsups/

Ok
thank you!
so your results are more or less consistent with mine
Your timing are better, my GHz is higher so I get a fraction of a UPS more and a little higher memory score

That save runs at ~35 ups with my 7700k.

Zeblote can you post your PassMark scores to so we have a reference?
thank you

I think it's all comes down to "Memory Latency" (which hopefully can get optimize in software) and "CPU Single Thread Performance" (which will probably not be improved soon as Factorio multithreading is problematic)

Part of the problem is that Ryzen is not monolithic 8 core processor, it's 4+4 cores and the communication between them and ram is slow which is not good for Factorio.

FYI on the map (http://www.tuwr.net/~factorio/factory%2 ... 0568hr.zip) I got 37 UPS after 1 min. with i7-6700K 4.6 GHz + DDR4 3600 MHz 15-16-16-32-1T
or on this one viewtopic.php?f=204&t=34068 60+ UPS

PerformanceTest64_2017-05-13_14-20-27.png (11.65 KiB) Viewed 8376 times

PerformanceTest64_2017-05-13_14-20-19.png (11.72 KiB) Viewed 8376 times

[hoho]

Not sure if anyone cares but that "reference map" runs at around 52UPS on my i5-6600K @3.5GHz and 32GB of dual-channel RAM at 2400MHz (nothing is overclocked) in Linux with factorio version 0.14.23. During night it drops to ~45UPS. I have GTX 770 video card.


I don't really think it's possible to make factorio run much faster on the new AMD cpu, especially not with just flicking on some compiler flags. At best what could be done is to somehow force the game to only use cores that are within one "module" so that the game doesn't have to synchronize data between the two CPU modules but that's better handled on operating system level.


By the way, did anyone else get a chuckle remembering how years ago, AMD said they will never do "multi-chip" CPUs and will only do "real" multicore CPUs where all cores can communicate with each other? That was in response to Intel's first dualcore CPUs where they just slapped two independent CPUs to single chip and had them communicate through northbridge causing huge latencies. Seemingly, AMD has done almost exact same thing with their new CPU generation that they were criticizing Intel about years ago