[0.17.63] Get rocket count under rcon very very unstable data

[IIPoliII]

Hey,

So since a few month i gather a lot of data on all my factorio servers and insert them to my SQL database.

And i founded out factorio is really really really unsatable on rockets, and rcon in general but rockets really kills it.

Sometimes it returns "nil" (okay no rocket sended) BUT sometimes it sends nil on the main force..... wich have 2k rockets. The other graphs i have using the same techno aren't doing this i am trying 5 times to ask to via rcon to be sure on this one and even like that .....

I think in general this command has something lagged. Is there maybe another way to get the launched rockets over rcon too a more stable one?

Thanks c:

[Klonan]

I don't really see how this would be possible, the Lua command will always realiably return the correct value

Can you post some more info about how you are inputting the force names and how you are parsing the printed message?

[IIPoliII]

I don't really see how this would be possible, the Lua command will always realiably return the correct value

Can you post some more info about how you are inputting the force names and how you are parsing the printed message?

Behind it's a bash script in this specific case i try 5 times in total the "getLastestRocketCount" Was a way to fix it by getting the "Old" rocket count in my database here is the code



Note : this request is executed really fastly maybe rcon don't like it? but how does it do for other data for exemple this one


I have checked the packet size of the RCON tool i use and everything is okay in breif the script output in this case is a bit trashy since i added loads of echos but it looks like

[IIPoliII]

Here is the rcon tool i use and the line for the package (network sided) size https://github.com/joinandplaycoop/mcrc ... rcon.c#L60 knowing that i needed to change it for forces due to the size of the list

[Bilka]

Your screenshot shows the rcon command working perfectly, could you further show what the issue is.

[IIPoliII]

Okay i think i founded the issue, i send a lot of reqests to rcon, now i wonder if you send 2 requets at once what does it do i think factorio returns nothing, because the rcon tool i have have a timeout try catch builded in it.

Code: Select all

If you need help for running this script run it with -h
Continuing.....
Done at 12:00
Old DB Tick 460952
Current Tick : 461011
Interval 59
Current Tick : 461011
Old DB Tick 460952

As you can see here it didn't ran the command because i map the forces to an array and it looks like factorio returned me nothing not any forces at this exact moment at this exact moment factorio save is it bounded? Also factorio maybe run a script that send the time but the thing is that sometimes these holes happen when nothing is going on, on this instance...

[IIPoliII]

Your screenshot shows the rcon command working perfectly, could you further show what the issue is.

As you said, in this case it worked perfectly but sometimes the rcon seems like it don't answer or nothing happens.

[IIPoliII]

Okay i think i founded the issue, i send a lot of reqests to rcon, now i wonder if you send 2 requets at once what does it do i think factorio returns nothing, because the rcon tool i have have a timeout try catch builded in it.

Code: Select all

If you need help for running this script run it with -h
Continuing.....
Done at 12:00
Old DB Tick 460952
Current Tick : 461011
Interval 59
Current Tick : 461011
Old DB Tick 460952

As you can see here it didn't ran the command because i map the forces to an array and it looks like factorio returned me nothing not any forces at this exact moment at this exact moment factorio save is it bounded? Also factorio maybe run a script that send the time but the thing is that sometimes these holes happen when nothing is going on, on this instance...

I correct my saying the holes are exactly when 2 commands on rcon are ran at the same time

[steinio]

Why not just throw away faulty responses?
This would also save database space if you just store values if a change has happened.

[IIPoliII]

Why not just throw away faulty responses?
This would also save database space if you just store values if a change has happened.

I am not throwing away faulty responses, i am trying to found a fix for this rcon.

Well that's currently what we do on the database but for stats you can't simply do an update it won't work your stats will be flat not useful....

Well and in this case how do you know when a change happens without calling rcon for it? You can't? Since i need to know if the data of factorio has changed.

Of course if there is another solution to gather it without calling factorio in this specific case i would love to take it

[IIPoliII]

I think factorio has a problem with multiple connection on the same moment on the same port, knowing it opens a new socket each time i run this command i think it does not support the fact that i send some commands at the same time i know that if i run a rcon command on python it takes 0.43 seconds to do the command. Now if something happens with another connection when the command is ran does factorio blocks the command or simply returns nothing?

[Oxyd]

I think factorio has a problem with multiple connection on the same moment on the same port, knowing it opens a new socket each time i run this command i think it does not support the fact that i send some commands at the same time i know that if i run a rcon command on python it takes 0.43 seconds to do the command. Now if something happens with another connection when the command is ran does factorio blocks the command or simply returns nothing?

No, I can't reproduce that. I tested opening two RCON connections at the same time and sending the same command through both of these, and even if the commands were executed in the same game tick, I still received a reply for both of these.

Can you show the log from the server?

[IIPoliII]

I think factorio has a problem with multiple connection on the same moment on the same port, knowing it opens a new socket each time i run this command i think it does not support the fact that i send some commands at the same time i know that if i run a rcon command on python it takes 0.43 seconds to do the command. Now if something happens with another connection when the command is ran does factorio blocks the command or simply returns nothing?

No, I can't reproduce that. I tested opening two RCON connections at the same time and sending the same command through both of these, and even if the commands were executed in the same game tick, I still received a reply for both of these.

Can you show the log from the server?

I tested it as well and weirdly all got transfered here is the log there isn't a lot of things....

Code: Select all

37791.521 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:48760})
37791.584 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:48786})
37791.604 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:48790})
37791.689 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:48806})
37791.744 Info AppManager.cpp:394: Saving game as /home/s1/FactoCord/../saves/japc.zip
37791.790 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:48816})
37791.804 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:50008})
37791.860 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:48838})
37792.124 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:49700})
37793.081 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:50480})
37793.796 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:50852})
37795.077 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:51338})
37795.804 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:51596})
37797.077 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52152})
37797.793 Info AppManagerStates.cpp:1759: Saving finished
37797.813 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52498})
37797.816 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51312})
37797.847 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52518})
37797.851 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51334})
37797.881 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52536})
37797.884 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51350})
37797.921 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51370})
37797.925 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52560})
37797.956 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52570})
37797.977 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51392})
37798.002 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52592})
37798.002 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52594})
37798.026 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51418})
37798.029 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52608})
37798.036 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52610})
37798.037 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51424})
37798.063 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52630})
37798.064 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51444})
37798.069 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52634})
37798.078 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51454})
37798.080 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52646})
37798.101 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52656})
37798.117 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52662})
37798.118 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51476})
37798.133 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52672})
37798.145 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52678})
37798.147 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51494})
37798.181 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52692})
37798.187 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51512})
37798.205 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52708})
37798.238 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51532})
37798.249 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51540})
37798.252 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52730})
37798.285 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52748})
37798.295 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51566})
37798.300 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51570})
37798.318 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52762})
37798.345 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51586})
37798.346 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51588})
37798.348 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52778})
37798.371 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52790})
37798.382 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52794})
37798.389 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52798})
37798.389 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51614})
37798.420 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52812})
37798.433 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51628})
37798.440 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51632})
37798.445 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52826})
37798.475 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52836})
37798.485 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51658})
37798.513 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51672})
37798.513 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52862})
37798.538 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52868})
37798.576 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51692})
37798.583 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52884})
37798.588 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51700})
37798.632 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52904})
37798.648 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51722})
37798.657 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52912})
37798.679 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52924})
37798.686 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52928})
37798.688 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51742})
37798.705 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52938})
37798.721 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51760})
37798.741 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52960})
37798.761 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51786})
37798.774 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51792})
37798.775 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:52982})
37798.799 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51808})
37798.801 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:53002})
37798.833 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:53018})
37798.835 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51836})
37798.863 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:53038})
37798.863 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({127.0.0.1:51852})
37798.912 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:53064})
37798.946 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:53080})
37798.986 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:53094})
37799.017 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:53106})
37799.045 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:53114})
37799.049 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:53116})
37799.057 Info RemoteCommandProcessor.cpp:238: New RCON connection from IP ADDR:({88.99.215.123:53120})

And grafana at this exact moment don't have any data (14:00 is the moment we run a /server-save)



And now again looking in the database :


And if we look at the script here is how we insert it :


Then here is the totalrocketcount



And here is the LastestRocketCount who simply takes the old number in the database



From my side i relaly don't see any issues and what is the "most" weird thing is that if you look all other data is correct we use the same tech to get it except the while that i just added before to retry.

[IIPoliII]

Oh side note grafana is a bit lagged here when i zoomed too much in but at 14:00 it shows 0, i wondered if it was grafana but the data is the database is corrupted and it's really weird it happens exactly at the moment i ask for it

At 14:30 runs the next save i will add a sleep 10s to see if it fixes the issue temporaly

[Oxyd]

I think factorio has a problem with multiple connection on the same moment on the same port, knowing it opens a new socket each time i run this command i think it does not support the fact that i send some commands at the same time i know that if i run a rcon command on python it takes 0.43 seconds to do the command. Now if something happens with another connection when the command is ran does factorio blocks the command or simply returns nothing?

No, I can't reproduce that. I tested opening two RCON connections at the same time and sending the same command through both of these, and even if the commands were executed in the same game tick, I still received a reply for both of these.

Can you show the log from the server?

I tested it as well and weirdly all got transfered here is the log there isn't a lot of things....

Okay, so how do I reproduce it, then? Can you show exactly what data you send over the RCON socket and what data you receive?

I'm not going to debug your scripts, so please limit your description to the RCON interface and the game itself.

[IIPoliII]

Alright i have proof of my words i founded the issue :

So i created a script to always save the server (wich is stupid alright?) if i run this script as shown here it always saves


Here in my console it saves !


Alright now i gonna run the other script to ask for rockets so if i run when it does NOT save it works as shown here :


And now if i run it WHILE it saves :


The rcon command takes longer but outputs nothing

[IIPoliII]

I think factorio has a problem with multiple connection on the same moment on the same port, knowing it opens a new socket each time i run this command i think it does not support the fact that i send some commands at the same time i know that if i run a rcon command on python it takes 0.43 seconds to do the command. Now if something happens with another connection when the command is ran does factorio blocks the command or simply returns nothing?

No, I can't reproduce that. I tested opening two RCON connections at the same time and sending the same command through both of these, and even if the commands were executed in the same game tick, I still received a reply for both of these.

Can you show the log from the server?

I tested it as well and weirdly all got transfered here is the log there isn't a lot of things....

Okay, so how do I reproduce it, then? Can you show exactly what data you send over the RCON socket and what data you receive?

I'm not going to debug your scripts, so please limit your description to the RCON interface and the game itself.

Sure sadly i don't know how you could retry it, but i have an idea one how you could do it.

Try to get a big filesave
You can get one here for exemple (idk if this one is big inaf it needs to slower stop your game) : https://saves.joinandplaycoop.com/Home/ ... -14:05.zip also this is the current save

Now try to save and run a while for exemple while you save you run on rcon the command to get rockets as following

Code: Select all

/silent-command rcon.print(game.forces["'Main Force"].items_launched["satellite"])

(of course modify it with your current force)

So while you are doing this save on one socket open another one asking for the rocket

here are my 2 scripts :

Code: Select all

#!/bin/bash
PATH=/opt/someApp/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

test=0
while [[ $test -ne 10 ]]
do

        timeout 2 mcrcon -c -H 127.0.0.1 -P PORT -p PASSSS "/server-save"

done

Code: Select all

#!/bin/bash
PATH=/opt/someApp/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

test=0
run=0
while [[ $test -ne 10 ]]
do

        mytest=$(timeout 2 mcrcon -c -H 127.0.0.1 -P PORT -p PASSS '/silent-command rcon.print(game.forces["Main Force"].items_launched["satellite"])')


        let "run++"

        echo "At run $run Test gives : $mytest"

done

[I_IBlackI_I]

Is this using a non blocking save?

[IIPoliII]

Is this using a non blocking save?

Nope full blocking

[IIPoliII]

Is this using a non blocking save?

But it should make the socket wait or kill it or die or idk or return not avalible because currently it simply returns nothing :c