New api in a OOP language?

Ideas that are too old (too many things have changed since) and ones which won't be implemented for certain reasons or if there are obviously better suggestions.

Moderator: ickputzdirwech

yagaodirac
Fast Inserter
Fast Inserter
Posts: 152
Joined: Sun Jun 16, 2019 4:04 pm
Contact:

New api in a OOP language?

Post by yagaodirac »

I know it's a very giant project to make api. But according my experience of reading all the codes in Factorio api, I don't think lua is good enough. Lua is not oop enough. People have to encapsulate the whole thing to make use of the advantage of oop. I'm already thinking redo my mod in an oop way, but I'm curious about, did you plan to make new api in other language? Shall I wait or prepare for the oop in lua?

User avatar
Klonan
Factorio Staff
Factorio Staff
Posts: 5148
Joined: Sun Jan 11, 2015 2:09 pm
Contact:

Re: New api in a OOP language?

Post by Klonan »

You can do OOP just fine in Lua, any table and set of functions can be wrapped with metatables to act as an object with your own behaviors.


We are not going to rewrite the API now, maybe if we start a new game it would use a different scripting language

blazespinnaker
Filter Inserter
Filter Inserter
Posts: 665
Joined: Wed Sep 16, 2020 12:45 pm
Contact:

Re: New api in a OOP language?

Post by blazespinnaker »

Lua is generally the defacto language for most games these days. You can absolutely do OOP, though honestly, speaking as a long time java programmer, OOP is over rated as it so can be easily simulated in more flexible, narrow ways specific to the problem at hand. Often OOP leads to people doing all sorts of unnecessary and bloated things for the simple problems they are solving. They tend to forget the solution to most problems is usually real code rather than design.

Far more important is type safety, especially when you're dealing with other people's code and have no idea what type they meant and really don't have the time to be digging through all their code line by line. I know the roblox folks are adding type safety to lua, hopefully it's getting open sourced.

The other weird thing about lua is not being zero based, which is mathematically painful but makes sense given its intended audience. Ugh. It's hard being a polyglot programmer sometimes.

If it were up to me, gaming folks would be using python3.6+
OptimaUPS Mod, pm for info.

User avatar
Khagan
Fast Inserter
Fast Inserter
Posts: 232
Joined: Mon Mar 25, 2019 9:40 pm
Contact:

Re: New api in a OOP language?

Post by Khagan »

blazespinnaker wrote:
Sun Oct 11, 2020 10:11 am
Lua is generally the defacto language for most games these days.
[...]
If it were up to me, gaming folks would be using python3.6+
Python is powerful and flexible. But that power comes at a price in infrastructure overhead. To embed Lua in an application, including the usual built-in Lua libraries, increases the size of the executable by a fraction of a MB. To embed Python may cost several times that just for the interpreter, plus many MB for the Python standard library ('batteries included').

There is also the question of usage patterns. Lua is designed and optimised for embedded scripting. A callback from the main program to a short scripted function can be executed very efficiently. Python has other priorities, and the context switch from compiled code to script and back is likely to be slower.

For a top-level script (i.e. something you run from the command line or as a stand-alone application) the extra overhead for Python is negligible: the space cost is paid once for the entire system, and the time cost is paid once for the whole program run. But for embedded scripts, the space cost has to be paid separately for each application and the time cost for every scripted call-back.

If you expect your embedded scripts to be doing some heavy computational lifting, then embedding Python may be worth the overhead (Blender is an example that immediately comes to mind). But if those embedded scripts are mostly just short and simple customisation points, then while as a modder or script writer I might still prefer Python to Lua, as an end user I will be grateful for the developers issuing modders with a nutcracker instead of a sledgehammer.

yagaodirac
Fast Inserter
Fast Inserter
Posts: 152
Joined: Sun Jun 16, 2019 4:04 pm
Contact:

Re: New api in a OOP language?

Post by yagaodirac »

I've being following the mod portal for days. I found most of the mods are very short. Normally they added some new items, some are recipe, some are entity. But very few mods do heavy computational things. I started a mod named dynamic water. I coded 40k lua only for an algorithm. That was very painful.
I know it's very expensive to write the api on another language. I know it's not possible. But I still believe that the reason for most mods to be simple and short is that it's really too hard to code big and heavy things in lua.
You know in ue4 and unity, for AActor in ue4 or prefeb in unity, to make the object move, all you need to do is override the Tick function in ue4, or update function in unity. But in lua, I have to encapsulate the framework from on_tick. I have to keep all the "object" in a table. Like:
object = {}
And then whenever I instantiate any object, I need to
table.insert(object, new_object)
In this way, I could do
script.on_script(on_tick, function(every)
for k,v in pairs(object) do
v.tick(1/60)
end
end)
And when I kill any "object", I don't know if it's safe to call something like:
projectile.kill()
if the init function for projectile is like:
projectile = {}

function projectile.kill(self)
self = nil
object[my index] = nil
end

function projectile.init()
result = {}
result.tick = some_tick_function
result.lifespan = somelifespan+game.tick
result.kill = projectile.kill
table.insert(object.projectile, result)
return result
end

in the on_tick
if ........ then object.projectile[100]:kill() end

I mean, because mod devs have to do so many things only to build up the "infrastructure", this limites most of the devs to shorten their mods. The result is that most mods are short, simple. I didn't mean python or js is better. I know you are professional and experienced and chose wisely. But due to the fact that lua is not very capable of big and heavy code without some basic works, could you please make some "recommended frameworks" to help people mod in a heavy way?

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 12888
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: New api in a OOP language?

Post by ssilk »

I don’t know why everyone cries for OOP. functional programming is the thing nowadays.
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

Qon
Smart Inserter
Smart Inserter
Posts: 2091
Joined: Thu Mar 17, 2016 6:27 am
Contact:

Re: New api in a OOP language?

Post by Qon »

yagaodirac wrote:
Mon Oct 12, 2020 1:46 am
You know in ue4 and unity, for AActor in ue4 or prefeb in unity, to make the object move, all you need to do is override the Tick function in ue4, or update function in unity. But in lua, I have to encapsulate the framework from on_tick. I have to keep all the "object" in a table. Like:
object = {}
Encouranging people to make entities than need lua updates every tick is a disaster for performance. All the "heavy" mods you want would be heavy for CPU to run.

User avatar
boskid
Factorio Staff
Factorio Staff
Posts: 2227
Joined: Thu Dec 14, 2017 6:56 pm
Contact:

Re: New api in a OOP language?

Post by boskid »

yagaodirac wrote:
Sun Oct 11, 2020 8:54 am
I don't think lua is good enough.
So what other language would you suggest, but make sure it has to be possible to save-load the whole state (or at least partially so it is possible to recover it when loading) in a reasonable way, without forcing every modder into implementing a serialisation/deserialisation logic for their custom objects.

Qon
Smart Inserter
Smart Inserter
Posts: 2091
Joined: Thu Mar 17, 2016 6:27 am
Contact:

Re: New api in a OOP language?

Post by Qon »

boskid wrote:
Mon Oct 12, 2020 9:11 am
yagaodirac wrote:
Sun Oct 11, 2020 8:54 am
I don't think lua is good enough.
So what other language would you suggest, but make sure it has to be possible to save-load the whole state (or at least partially so it is possible to recover it when loading) in a reasonable way, without forcing every modder into implementing a serialisation/deserialisation logic for their custom objects.
I'd like JS, if one is allowed to dream freely. Fast and also made to be embedded, like Lua. But also way better and more reasonable syntax and feature set. Actually working RegExp, string templating, functional notation, => arrow function notation, not having to do a for loop for every basic feature like getting a list of keys in an object etc.

Not all state can be serialised to JSON, but only global needs serialisation and we should be able to serialise circular references similar to how it is possible with Lua tables. Metatables and functions in Lua aren't serialised anyways in global so we don't really care about actually being able to serialise everything, so not being able to save everything in JS (like promises and functions) isn't a deal breaker since we can probably just avoid using those unserialisable features in the worst case or not store them in global.

So do you know anything that exludes JS as a possibility or a soft reason to not go with JS?

yagaodirac
Fast Inserter
Fast Inserter
Posts: 152
Joined: Sun Jun 16, 2019 4:04 pm
Contact:

Re: New api in a OOP language?

Post by yagaodirac »

Actually, I'm planning some one-mod-scenario. In that special case, performance is not a problem since there aren't gonna be a lot of entities flying at the same time. I literally miss the feeling when I coded in some game engine. But, I'm not gonna talk about this anymore because another set of api in other language is not possible if it's not in the plan in the beginning. I'll pick up lua entirely this time and encapsulate a set of tool for myself. And then, I'll try my plan.
Thank all of you and your replies.

posila
Factorio Staff
Factorio Staff
Posts: 5201
Joined: Thu Jun 11, 2015 1:35 pm
Contact:

Re: New api in a OOP language?

Post by posila »

yagaodirac wrote:
Mon Oct 12, 2020 1:46 am
You know in ue4 and unity, for AActor in ue4 or prefeb in unity, to make the object move, all you need to do is override the Tick function in ue4, or update function in unity. But in lua, I have to encapsulate the framework from on_tick. I have to keep all the "object" in a table.
Hello, that has nothing to do with language, though. Had we used C# as a scripting language, you still would not be able to just override Update function of an entity, because by design the API doesn't provide ability to define custom entity types, because it is not generic game engine.

User avatar
eradicator
Smart Inserter
Smart Inserter
Posts: 5206
Joined: Tue Jul 12, 2016 9:03 am
Contact:

Re: New api in a OOP language?

Post by eradicator »

If there's one thing i can agree with it's that it would've been great if factorio had included a more comprehensive set of "common" functions with the factorio typical great documentation. I've spent much more time on writing a "library" than i have spent on actually modding things, because even basic things like map(), keys(), set(), list() [python] etc have to be manually implemented by each modder. There are now several large library mods with varying documenation quality in addition to the library mods that each of the popular modders maintain just for themselfs. And i like to imagine than an official library would've been able to absorb large parts of that "spread". But it's likely too late for that now. XKCD: "Standards"

The total time that the community as a whole has spent on "reinventing the wheel" must be enormus.
Author of: Belt Planner, Hand Crank Generator, Screenshot Maker, /sudo and more.
Mod support languages: 日本語, Deutsch, English
My code in the post above is dedicated to the public domain under CC0.

yagaodirac
Fast Inserter
Fast Inserter
Posts: 152
Joined: Sun Jun 16, 2019 4:04 pm
Contact:

Re: New api in a OOP language?

Post by yagaodirac »

eradicator wrote:
Mon Oct 12, 2020 2:00 pm
If there's one thing i can agree with it's that it would've been great if factorio had included a more comprehensive set of "common" functions with the factorio typical great documentation. I've spent much more time on writing a "library" than i have spent on actually modding things, because even basic things like map(), keys(), set(), list() [python] etc have to be manually implemented by each modder. There are now several large library mods with varying documenation quality in addition to the library mods that each of the popular modders maintain just for themselfs. And i like to imagine than an official library would've been able to absorb large parts of that "spread". But it's likely too late for that now. XKCD: "Standards"

The total time that the community as a whole has spent on "reinventing the wheel" must be enormus.
That is part of the reason. But to make use of the existing standard library, people might have to create api on c++, and perhaps with a lot custom macros, just like ue4. In that way, all the containers, all the iterators and functors, all the wheels are available. But not very practical.
So, I think what we face now, is just like the time before unreal toolkit came into being, people developed game had to write their own framework. So, why not a standard of general purpose factorio mod framework? At least it saves people's time making wheels again and again. And I prefer the structure of ue4 and unity is because I really benefit from that pattern when make heavily customized behavior. The tick() or update() is so helpful, it's not only a function, but a mechanism, a system.
Some experienced mod dev please lead this work.

User avatar
ssilk
Global Moderator
Global Moderator
Posts: 12888
Joined: Tue Apr 16, 2013 10:35 pm
Contact:

Re: New api in a OOP language?

Post by ssilk »

Qon wrote:
Mon Oct 12, 2020 9:46 am

I'd like JS, if one is allowed to dream freely. Fast and also made to be embedded, like Lua. But also way better and more reasonable syntax and feature set.
...
So do you know anything that exludes JS as a possibility or a soft reason to not go with JS?
I think no, but there are several more reasons:
- if you want to use a typed language you can just switch to typescript, that would make many typical errors impossible to program. (With the disadvantage of small extra compile time before each test)
- js is really fast, I found benchmark which says up to hundred times https://benchmarksgame-team.pages.debia ... t/lua.html
I even doubt that any other pure script language is that fast.
- enables much better functional programming
- professional IDE, professional tools, thousands of libs, really cool test frameworks...
- it has the cool concept of asynchron functions and expectations.
- very similar to lua.
- destructuring would make scripts significantly smaller and easier to write.
- browsers implement the origin policy for loading stuff from outside of the game. I can think of similar mechanics in a game to allow things, that are currently unthinkable.

Disadvantage:
- there is AFAIK no concept of forbidden things like in lua (I don’t know the name). And it’s far more “open”. From security standpoint this is a problem and each mod needs to run in some kind of sandbox... but browsers managed to make js very secure, so why not also in games. I think that’s gives really cool
- JavaScript counts from 0 :) (but with typescript you can define data types that reflects that)
- it’s not made for games...
- for creating configuration and similar things I think lua is a bit better. But why not allow both?
- it needs a complete rework of Factorio how to handle the mods, api,.. etc.
Cool suggestion: Eatable MOUSE-pointers.
Have you used the Advanced Search today?
Need help, question? FAQ - Wiki - Forum help
I still like small signatures...

yagaodirac
Fast Inserter
Fast Inserter
Posts: 152
Joined: Sun Jun 16, 2019 4:04 pm
Contact:

Re: New api in a OOP language?

Post by yagaodirac »

Days ago, someone asked that does ue4 support python. After searching, I found ue4 supports python through a plugin.
So, is it possible to enable factorio to support js or py or anything else through a mod?

blazespinnaker
Filter Inserter
Filter Inserter
Posts: 665
Joined: Wed Sep 16, 2020 12:45 pm
Contact:

Re: New api in a OOP language?

Post by blazespinnaker »

Khagan wrote:
Sun Oct 11, 2020 11:55 pm
blazespinnaker wrote:
Sun Oct 11, 2020 10:11 am
Lua is generally the defacto language for most games these days.
[...]
If it were up to me, gaming folks would be using python3.6+
Python is powerful and flexible. But that power comes at a price in infrastructure overhead. To embed Lua in an application, including the usual built-in Lua libraries, increases the size of the executable by a fraction of a MB. To embed Python may cost several times that just for the interpreter, plus many MB for the Python standard library ('batteries included').

There is also the question of usage patterns. Lua is designed and optimised for embedded scripting. A callback from the main program to a short scripted function can be executed very efficiently. Python has other priorities, and the context switch from compiled code to script and back is likely to be slower.

For a top-level script (i.e. something you run from the command line or as a stand-alone application) the extra overhead for Python is negligible: the space cost is paid once for the entire system, and the time cost is paid once for the whole program run. But for embedded scripts, the space cost has to be paid separately for each application and the time cost for every scripted call-back.

If you expect your embedded scripts to be doing some heavy computational lifting, then embedding Python may be worth the overhead (Blender is an example that immediately comes to mind). But if those embedded scripts are mostly just short and simple customisation points, then while as a modder or script writer I might still prefer Python to Lua, as an end user I will be grateful for the developers issuing modders with a nutcracker instead of a sledgehammer.
Yeah lua is pretty nice, I'll agree. But any callout to scripting will be slow and should never be used in a performance sensitive situation. I can't imagine the memory thing is an issue in factorio, unless they are doing something strange that I'm not aware of.

Generally tho, I don't have a problem with lua except those 1 based arrays. Drives me crazy as I have to change all those algorithms I've stored up in my head over the years. I suspect if he hadn't done that, lua might have put up a better fight against python.

Oh yeah, and maybe a better standard lib.
OptimaUPS Mod, pm for info.

blazespinnaker
Filter Inserter
Filter Inserter
Posts: 665
Joined: Wed Sep 16, 2020 12:45 pm
Contact:

Re: New api in a OOP language?

Post by blazespinnaker »

ssilk wrote:
Tue Oct 13, 2020 3:55 am
Qon wrote:
Mon Oct 12, 2020 9:46 am

I'd like JS, if one is allowed to dream freely. Fast and also made to be embedded, like Lua. But also way better and more reasonable syntax and feature set.
...
So do you know anything that exludes JS as a possibility or a soft reason to not go with JS?
I think no, but there are several more reasons:
- if you want to use a typed language you can just switch to typescript, that would make many typical errors impossible to program. (With the disadvantage of small extra compile time before each test)
- js is really fast, I found benchmark which says up to hundred times https://benchmarksgame-team.pages.debia ... t/lua.html
I even doubt that any other pure script language is that fast.
- enables much better functional programming
- professional IDE, professional tools, thousands of libs, really cool test frameworks...
- it has the cool concept of asynchron functions and expectations.
- very similar to lua.
- destructuring would make scripts significantly smaller and easier to write.
- browsers implement the origin policy for loading stuff from outside of the game. I can think of similar mechanics in a game to allow things, that are currently unthinkable.

Disadvantage:
- there is AFAIK no concept of forbidden things like in lua (I don’t know the name). And it’s far more “open”. From security standpoint this is a problem and each mod needs to run in some kind of sandbox... but browsers managed to make js very secure, so why not also in games. I think that’s gives really cool
- JavaScript counts from 0 :) (but with typescript you can define data types that reflects that)
- it’s not made for games...
- for creating configuration and similar things I think lua is a bit better. But why not allow both?
- it needs a complete rework of Factorio how to handle the mods, api,.. etc.
Lack of threads in js might be an issue. :)
OptimaUPS Mod, pm for info.

Qon
Smart Inserter
Smart Inserter
Posts: 2091
Joined: Thu Mar 17, 2016 6:27 am
Contact:

Re: New api in a OOP language?

Post by Qon »

blazespinnaker wrote:
Tue Oct 13, 2020 7:06 am
Lack of threads in js might be an issue. :)
What mods are using Lua threads for doing things?
Also (modern implementations of) JS is so fast it doesn't need threads to outperform Lua.
It's fast enough that it should be compared to Java and C++ instead of Lua, because it performs similarly to those.
JS vs Java
JS vs C++
Updates to entities needs to be done deterministically so you can't run different mods concurrently or you risk breaking that guarantee.

Bilka
Factorio Staff
Factorio Staff
Posts: 3123
Joined: Sat Aug 13, 2016 9:20 am
Contact:

Re: New api in a OOP language?

Post by Bilka »

You could use something like https://github.com/TypeScriptToLua/TypeScriptToLua to write code in non-Lua, there is at least one mod on the mod portal that was written this way.

(Just to be clear, this is a possibility, not a recommendation.)
I'm an admin over at https://wiki.factorio.com. Feel free to contact me if there's anything wrong (or right) with it.

yagaodirac
Fast Inserter
Fast Inserter
Posts: 152
Joined: Sun Jun 16, 2019 4:04 pm
Contact:

Re: New api in a OOP language?

Post by yagaodirac »

Bilka wrote:
Tue Oct 13, 2020 9:22 am
You could use something like https://github.com/TypeScriptToLua/TypeScriptToLua to write code in non-Lua, there is at least one mod on the mod portal that was written this way.
Which mod?

Post Reply

Return to “Outdated/Not implemented”