Page 1 of 1
Create some way for Inter-Process-Communication
Posted: Mon Apr 11, 2016 8:53 pm
by Wolfspirit
Hi,
I'm looking for a way to get stats for a third-party tool.
Right now I'm writing to a textfile and reading it on change from within the Third-Party tool.
That however creates a massive amount of IO on the Harddrive and is very inefficient.
I'm looking for a way to access sockets from within LUA so I can send that data directly to the third-party tool that is running on the same computer.
Is there any way you could give access to something like that or give any other way to do IPC?
In my personal usecase it is to let the third-party tool call the Razer Chroma API and update the lighting of the Keyboard depending on the state of the Game.
The current alpha state which writes to a file (script_output) can be seen here:
https://insider.razerzone.com/index.php ... pha.12722/
Re: Create some way for Inter-Process-Communication
Posted: Tue Apr 12, 2016 5:49 pm
by Afforess
It probably won't happen, as sockets breaks the lua sandbox and breaks the determinism, which are both bad for Multiplayer.
I recommend using your third-party tool to create an in-memory filesystem where the Factorio-scriptout directory is (RAMFS / TMPFS on Linux), then you will only be writing to an in-memory location.
Re: Create some way for Inter-Process-Communication
Posted: Wed Apr 13, 2016 2:04 am
by Wolfspirit
I can understand that sockets would "break" the sandbox in a way.
Sadly the Chroma API is only available for windows and so my tool (and the actual usecase) is only available for windows and there is no such way of in-memory filesystems for windows (without doing some deep C++ kernel stuff).
Otherwise I agree...something like that would be easier.
It could be implemented to only allow connecting + writing not reading which would put them at the same level then the script_output in my opinion as you won't be able to get and use external data.
It would even be enough to just send UDP packets with a string to an IP + Port without caring about it further keeping it deterministic.
(Maybe it can be used as some way of LUA debug output, too?)
Re: Create some way for Inter-Process-Communication
Posted: Wed Apr 13, 2016 9:30 am
by prg
Simply print to stdout, pipe Factorio output into your own program and look for the magic words from your script in all the log messages? (Might not work on Steam, Factorio seems to restart in the background in that case, no idea where stdout is going then)
Re: Create some way for Inter-Process-Communication
Posted: Wed Apr 13, 2016 8:06 pm
by Wolfspirit
Thanks for the idea.
Yes that could be an alternative but it is what I tried at first (I do have the Steam version) and the Factorio.exe just seems to be some kind of launcher telling Steam to launch the Game with special attributes/different entrypoint.
Not quiet sure what the Steam API does there exactly. Factorio might just check if the starter is steam and if not tell steam to start Factorio instead.
So that solution doesn't work or only works in some (non Steam) cases.
Re: Create some way for Inter-Process-Communication
Posted: Tue Apr 19, 2016 1:43 am
by seronis
Afforess wrote: as sockets breaks the lua sandbox and breaks the determinism
I wish devs would stop sandboxing outside of debug builds. Its more limiting than protective. Breaking determinism with how Factorio implements multiplayer is an issue though. And could also be resolved by having the game use a buffer output where one client does the actual socket connection and writing to the socket, and a read call would have that owning client grab the data and forward it to a buffer that is synced between all clients. Thus deterministic. Not difficult if you have all read() calls already require a callback to be registered. At least no more difficult than how you already handle on_tick() events.
Re: Create some way for Inter-Process-Communication
Posted: Tue Apr 19, 2016 3:43 am
by Afforess
seronis wrote:Afforess wrote: as sockets breaks the lua sandbox and breaks the determinism
I wish devs would stop sandboxing outside of debug builds. Its more limiting than protective.
As a developer on other projects, this is a bad idea. Debug builds should be as close to release builds as possible, or they are not useful in their purpose: debugging release builds.
seronis wrote:Breaking determinism with how Factorio implements multiplayer is an issue though. And could also be resolved by having the game use a buffer output where one client does the actual socket connection and writing to the socket, and a read call would have that owning client grab the data and forward it to a buffer that is synced between all clients. Thus deterministic. Not difficult if you have all read() calls already require a callback to be registered. At least no more difficult than how you already handle on_tick() events.
Yes, well given the choice between maintaining complex read/write state of sockets for multiplayer compatibility, and disabling sockets, I certainly see why they chose the latter.
Re: Create some way for Inter-Process-Communication
Posted: Sat Apr 23, 2016 9:37 pm
by SyncViews
seronis wrote:I wish devs would stop sandboxing outside of debug builds.
Isnt the point of using lua and sandboxing more about user security so that when using mods they dont have to have a massive "at your own risk, we cant guarantee this mod your about to load is not malware etc."? General socket access, filesystem access, etc., makes it a lot easier for a mod to do bad stuff. Otherwise they could have just exported the API symbols and allow a DLL/so plugin.
Re: Create some way for Inter-Process-Communication
Posted: Sat Apr 23, 2016 9:56 pm
by seronis
SyncViews wrote:seronis wrote:I wish devs would stop sandboxing outside of debug builds.
Isnt the point of using lua and sandboxing more about user security so that when using mods they dont have to have a massive "at your own risk, we cant guarantee this mod your about to load is not malware etc."? General socket access, filesystem access, etc., makes it a lot easier for a mod to do bad stuff. Otherwise they could have just exported the API symbols and allow a DLL/so plugin.
Yup thats the reason. I just dont consider it important enough to be a valid excuse for limiting possibilities. I much rather go on modder reputation and the fact that THE CODE ISNT COMPILED AND CAN BE READ. Possibilities > safety from imaginary issues
"Those who would give up liberty for safety deserve neither." -- Ben Franklin
Re: Create some way for Inter-Process-Communication
Posted: Sat Apr 23, 2016 10:15 pm
by Afforess
seronis wrote:
"Those who would give up liberty for safety deserve neither." -- Ben Franklin
Yeah, that's not the quote. The actual quote is:
Code: Select all
Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety.
I hardly see the lua sandbox as giving up
essential liberty.
At any rate, it seems really amusing to quote an American founding father when Wube/Factorio is developed in Eastern Europe, and I rather doubt the quote was meant for anything as trivial as
modding video games.
Re: Create some way for Inter-Process-Communication
Posted: Sat Apr 23, 2016 10:20 pm
by seronis
Yes its paraphrased obviously. But the concept is unchanged. You should never give up options for protection from something that almost never happens.
Modding in factorio involves human readable lua files that are not compiled prior to distribution. Anyone can take the time to read what the mod does before using it. As soon as one person has a legit problem no one will ever trust that modder again. Problems are almost self solving without any restrictions needed.
Re: Create some way for Inter-Process-Communication
Posted: Sun Apr 24, 2016 8:14 am
by Koub
seronis wrote:You should never give up options for protection from something that almost never happens.
I strongly disagree on this one. It's as saying "why lock your car when there is so little chance for it to be stolen" ?
Statistically, if they don't sandbox, there WILL be some time when shit will happen. Consider it as an insurance you only get hoping you won't have to use it.
Re: Create some way for Inter-Process-Communication
Posted: Sun Apr 24, 2016 10:24 am
by prg
seronis wrote:Modding in factorio involves human readable lua files that are not compiled prior to distribution. Anyone can take the time to read what the mod does before using it.
Anyone can. Sure. So you suggest that either everyone becomes knowledgeable in Lua and takes the time to untangle every function and audit every line of code in a mod before using it or wait for an expert to do so and write a review, "yep, this mod won't delete your files and is safe to use." Who is actually going to do this?
seronis wrote:As soon as one person has a legit problem no one will ever trust that modder again. Problems are almost self solving without any restrictions needed.
Right. As soon as a mod erases a couple of hard disks or steals passwords etc. the victims will just complain about it on the forums so people will stop using the mod and the problem fixes itself. Terrific.
Re: Create some way for Inter-Process-Communication
Posted: Mon Apr 25, 2016 1:34 am
by seronis
prg wrote:. So you suggest that either everyone becomes knowledgeable in Lua
The very VERY few (only 2 or 3 that I can remember) mods in minecraft and sims3 that did questionable things got called out for it almost immediately. You dont need everyone to be knowledgable. There will always be 'just enough' curious others trying to learn from someones code that it will be found. Worth noting that those are both 2 games where the code IS COMPILED and source isnt required to be supplied without being decompiled first.
So no. Sandboxing is not helpful. At all. The actual history of what has happened in modding proves this. And the hindrance is not worth the prevention of those few events. They get found out and dealt with.
Re: Create some way for Inter-Process-Communication
Posted: Mon Apr 25, 2016 5:30 am
by Rseding91
The standard game is not getting un-sandboxed.
I was thinking about it and had the idea that we might provide a flag that mods could use or perhaps a startup flag that would be required that enabled all of the unsafe and non-deterministic methods that aren't normally available. It would at the same time disable multiplayer and replays but this is all just ideas - I'm not sure if it would be worth the time at this point.
Re: Create some way for Inter-Process-Communication
Posted: Mon Apr 25, 2016 6:11 am
by Koub
The discussion has derived from inter-process communication to pro/against sandboxing argument.
I consider the original topic answered, so I'll lock this topic.
[Koub] Topic locked