[Rseding91] Need help profiling game

This subforum contains all the issues which we already resolved.
Post Reply
shinooni
Burner Inserter
Burner Inserter
Posts: 6
Joined: Mon Jun 03, 2019 11:33 pm
Contact:

[Rseding91] Need help profiling game

Post by shinooni »

Hi,
Is there something that breaks down what is consuming cycles?
other than the debug menu.

I can see that CRC is taking up 15 .0- 28 .0
factorissimo is taking up a max of 1.02 but tends to hover at 0.7

is there someway to find out what is chewing up CRC.

My server ram usage is around 4.7GB
My Client ram usage is around the same.
Server is running on a i7 7820X OC to 4.7GHZ with 64GB RAM and a 1060 GTX (that was installed purely to host the forest at one point) Dual 512GB M2 Samsung 960PROs with Raid 10 (8 disks - 4 stripe + mirror) - connection is 100/40 - real speed is 93/34 - I can scale the connection up to 250/100 or 500/250 or 1000/400 if that is truly the issue.

Test client is running on i9 7900X oc to 4.7GHZ with 32gb RAM 1080Ti, 512GB - 100/40 connection - real speed 40/10 (dont even get me started on that one.)
At one point the server was in a VM running on ubuntu server LTS latest release. Allocated 4 cores with 30% reserve and highest priority. 16gb allocated to the VM; however for some reason the server only ever utilized 6%.
Moved it out of VM and had it run on host - win server 2016 - gave it realtime priority - factorio only uses 9% at max... Even if factorio is mean't to use only one core it should reach around 12% utilization I think. My math could be off.
It only just recently started to severely slow down. Not sure what was the trigger. Will attach logs from when running in VM. Will also attach logs when running on host. I will also attach the performance profile saved from factorio's debug menu.
I tried removing all fish there seemed to be a huge amount.
I tried removing all enemies. Started removing mods - Creative mod, Helmod
problems still remain. Slow downs abundant.
Slow down = 50UPS cap out at gamespeed 1 but most of the time sits between 32-44 UPS
trying to figure out how to track down the root of the problem so I can:
1. remove what ever is causing the issue.
2. possibly make a solution and modify what ever is causing the problem
Attachments
factorio-perf-01745_20190708-234713_D3D11_i9-7900X@3.30~32GB~GTX1080_Ti~11127MB.csv
(130.69 KiB) Downloaded 125 times
factorio-current.log
(92.14 KiB) Downloaded 121 times
VMfactorio-current.log
(87.15 KiB) Downloaded 126 times

Bilka
Factorio Staff
Factorio Staff
Posts: 3123
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: Need help profiling game

Post by Bilka »

shinooni wrote:
Wed Jul 10, 2019 7:55 am
is there someway to find out what is chewing up CRC.
Use something like Very Sleepy to profile your save. Or upload the save here and someone else might take a look.
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.

shinooni
Burner Inserter
Burner Inserter
Posts: 6
Joined: Mon Jun 03, 2019 11:33 pm
Contact:

Re: Need help profiling game

Post by shinooni »

Thanks bilka,
I'll upload save in the morning.
However I did spin up visual studio on an off chance it reveals anything, I found that flow statistics<id<itemprototype,unsigned short>,unsigned __int64>::precision::save lastvalue
Was using an inordinate amount of time about 70%
Followed by the heuristicCRC method.

Unfortunately I dont know what flow statistics and heuristicCRC does or manages to try and narrow down what is causing this.

Rseding91
Factorio Staff
Factorio Staff
Posts: 13173
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Need help profiling game

Post by Rseding91 »

We'll need a copy of your save file in order to look into it locally.
If you want to get ahold of me I'm almost always on Discord.

shinooni
Burner Inserter
Burner Inserter
Posts: 6
Joined: Mon Jun 03, 2019 11:33 pm
Contact:

Re: Need help profiling game

Post by shinooni »

Hi,
I have attached now the mod-list.json, and the save-file. It also seems to tank the UPS particularly badly when there are more players on the server - (up to 4) - CRC time goes through the roof.
Thank you very much for your time. I really do appreciate it.

If there is some learning material available so I can do this sort of performance diagnosis myself as I know as a developer(or staff adjacent) your eyes will roll when it comes trying to diagnose modded saves and you may even prefer to discourage such activity. So as to not make it a habit it would be great if learning materials for this sort of in-depth analysis were available as the information I have managed to pull together is either out of date or lacking in depth or both.
Again thanks.
Attachments
hg2.zip
(25.77 MiB) Downloaded 127 times
mod-list.json
(5.11 KiB) Downloaded 126 times
Last edited by shinooni on Thu Jul 11, 2019 3:10 am, edited 1 time in total.

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2339
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: Need help profiling game

Post by Jap2.0 »

shinooni wrote:
Thu Jul 11, 2019 1:46 am
Hi,
I have attached now the mod-list.json, and the save-file.
Thank you very much for your time. I really do appreciate it.

If there is some learning material available so I can do this sort of performance diagnosis myself as I know as a developer(or staff adjacent) your eyes will roll when it comes trying to diagnose modded saves and you may even prefer to discourage such activity. So as to not make it a habit it would be great if learning materials for this sort of in-depth analysis were available as the information I have managed to pull together is either out of date or lacking in depth or both.
Again thanks.
The save's not attached. (it could be too big for the forum, or there could be another cause; if so, hosting it elsewhere is fine.)

And on the other count: if there are actual problems, I doubt there's much you can do beyond what you have with a complete lack of source code or anything. And as long as mod's aren't doing stuff that's completely stupid (and half the time they are), they're typically pretty good about it.
There are 10 types of people: those who get this joke and those who don't.

shinooni
Burner Inserter
Burner Inserter
Posts: 6
Joined: Mon Jun 03, 2019 11:33 pm
Contact:

Re: Need help profiling game

Post by shinooni »

My Apologies - I accidentally uploaded the wrong save.
Attached now to both this post and the previous.
Yeh understand that if a mod is doing something completely stupid - I would definitely like to know so at least I can go in a fix the mod and post/suggest the fix to the author.
I'm not scared to get my hands dirty so to speak. I know little bit as far as coding is concerned (python, java, a very small amount of C++ (QT C++ so not really C++)) but I don't know/not familiar with lua or the factorio api as well as should but once again I'll learn if I need to hahaha. :lol:
Attachments
hg2.zip
(25.77 MiB) Downloaded 123 times

Jap2.0
Smart Inserter
Smart Inserter
Posts: 2339
Joined: Tue Jun 20, 2017 12:02 am
Contact:

Re: Need help profiling game

Post by Jap2.0 »

shinooni wrote:
Thu Jul 11, 2019 3:13 am
My Apologies - I accidentally uploaded the wrong save.
Attached now to both this post and the previous.
Yeh understand that if a mod is doing something completely stupid - I would definitely like to know so at least I can go in a fix the mod and post/suggest the fix to the author.
I'm not scared to get my hands dirty so to speak. I know little bit as far as coding is concerned (python, java, a very small amount of C++ (QT C++ so not really C++)) but I don't know/not familiar with lua or the factorio api as well as should but once again I'll learn if I need to hahaha. :lol:
Yeah, in that case you're probably even more out of luck - if it's an issue with the mod (i.e. if disabling the mod fixes it) you could probably find out something about that (mods are lua, but you could look through the code), but if it's something in the base game there's basically nothing you can do beyond profiling - and it's written in C++ ;)

Speaking of that, have you tried without some of the mods to see if any of those are possibly at fault?
There are 10 types of people: those who get this joke and those who don't.

shinooni
Burner Inserter
Burner Inserter
Posts: 6
Joined: Mon Jun 03, 2019 11:33 pm
Contact:

Re: Need help profiling game

Post by shinooni »

Ive already removed helmod, creative mod, removed and then readded (then rolled the save back to before i removed it.) factorissimo which is documented at being UPS challenging. Factorissimo yield partial gains (UPS floor moved to around 38.*) but not as much as I would have hoped not to mention a large chunk of our base is in factories. so avoiding that if possible.
CRC with and without factorissimo seemed to be incredibly high regardless - what should CRC be on average? and what is linked to the CRC functions to optimize?

Rseding91
Factorio Staff
Factorio Staff
Posts: 13173
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: Need help profiling game

Post by Rseding91 »

I took a look at the save and it's now fixed for the next version of 0.17.
If you want to get ahold of me I'm almost always on Discord.

MagmaMcFry
Inserter
Inserter
Posts: 37
Joined: Sun Jul 17, 2016 9:05 pm
Contact:

Re: [Rseding91] Need help profiling game

Post by MagmaMcFry »

Nice!
I don't visit the forum very often. If you want to contact me, message me on the mod portal for Factorissimo-related things, or on Discord.

shinooni
Burner Inserter
Burner Inserter
Posts: 6
Joined: Mon Jun 03, 2019 11:33 pm
Contact:

Re: [Rseding91] Need help profiling game

Post by shinooni »

Thats awesome thanks Rseding91

Post Reply

Return to “Resolved Problems and Bugs”