Page 1 of 1
Metatables, NOT
Posted: Tue Oct 11, 2016 7:04 pm
by ssilk
I come from this thread:
viewtopic.php?f=25&t=33905
I have (not for everyone interesting) some results: I wanted to create an entity-manager with lua metatales. But before that I made some tests about performance.
First some code:
Implemenation with metatable:
Code: Select all
t={} -- basetable
mt={} -- metatable
mt.mytab={}
mt.__newindex=function(t,k,v)
rawset(mt.mytab, k, {val = v, valid = math.random()})
end
mt.__index=function(t,k)
local tab = rawget(mt.mytab, k)
if tab and tab.valid < 0.05 then
return tab.val
end
return false
end
setmetatable(t,mt)
for i=1, 1000000 do
t[math.random(1, 20000000)] = i
end
count = 0
for i=1, 20000000 do
if t[i] then
count = count + 1
end
end
print(count)
Implemenation without metatable:
Code: Select all
bla = {}
bla.t={} -- basetable
for i=1, 1000000 do
bla.t[math.random(1, 20000000)] = {
val = i,
valid = math.random()
}
end
count = 0
for i=1, 20000000 do
if bla.t[i] and bla.t[i].valid < 0.05 then
count = count + 1
end
end
print(count)
With metatable: 6.161s
Without metatable: 3.557s
Conclusion: Metatables makes Lua slower by 100%.
Consequnce:
No way to use metatables for a game as Factorio is. 
Well, I think most of you did know that, but maybe for others it's new. So I wanted that to share with you
Re: Metatables, NOT
Posted: Tue Oct 11, 2016 8:13 pm
by Adil
Now try that with some api-calls in the cycle.
Metatable is one additional pointer redirection, and, in your example, two additional function calls per step in each cycle.
When you add function calls to the straightforward imperative implementation the gap is starting to decrease:
Also, math.rand() is a rather odd sighting in the benchmark.
With those remarks in mind, if you yarn for performance, indeed you should try express all your code without additional functions, however, it probably won't matter much for anywhere except the on_tick event.
And yes, fancy operator overloading is not a good idea.
Addendum:
ssilk wrote:
Well, I think most of you did know that, but maybe for others it's new. So I wanted that to share with you
Apparently, Choumiko
didn't know that.
Re: Metatables, NOT
Posted: Tue Oct 11, 2016 9:36 pm
by aubergine18
Adil wrote:And yes, fancy operator overloading is not a good idea.
http://lua-api.factorio.com/latest/LuaC ... ustomTable
It's a great idea, if it's done properly in the right context. It always depends on the specific nature of the code, how often it's called, etc.
Re: Metatables, NOT
Posted: Wed Oct 12, 2016 2:21 am
by Afforess
Metatables are just a poor idea because they aren't serialized. That one issue alone makes them not worth the hassle in most mods. Worse performance is just insult to injury.
Re: Metatables, NOT
Posted: Wed Oct 12, 2016 2:50 am
by aubergine18
Afforess wrote:Metatables are just a poor idea because they aren't serialized. That one issue alone makes them not worth the hassle in most mods. Worse performance is just insult to injury.
It is trivial one line of code to re-apply the metatable, there's even an event specifically aimed at enabling that: script.on_load().
Their performance for the most part is not an issue - it's only if you start using them (or anything else for that matter) in an on_tick event handler. But for all other events, particularly if only accessing the metamethods sparingly, there's really nothing bad about them. Even in situations where they are used extensively, you'd still get better performance improvements, in most cases, by refining the code they contain rather than the metamethods themselves.
The performance stats in OP are also somewhat skewed IMO. Local references to tables and functions would improve performance (although local ref to math.random could cause desyncs, at least in older versions of factorio it did). There's also no need to use rawget in that particular example, there's always alternate ways to implement things.
Re: Metatables, NOT
Posted: Wed Oct 12, 2016 4:23 pm
by Rseding91
aubergine18 wrote:... (although local ref to math.random could cause desyncs, at least in older versions of factorio it did).
That won't cause desyncs. I don't think it ever has...
Re: Metatables, NOT
Posted: Fri Oct 14, 2016 5:57 am
by Daid
Your test isn't just testing metatables or not. It's also testing function calls vs inline code. And with so little things to do, inline code is always a lot faster.
Your code with the metatable is also wrong, as it sets the data on the metatable, which means you cannot share the metatable on multiple tables, which is an important idea of the metatables.
If you add back the function call overhead:
Code: Select all
t={} -- basetable
function set(t,k,v)
rawset(t, k, {val = v, valid = math.random()})
end
function check(t,k)
local tab = rawget(t, k)
if tab and tab.valid < 0.05 then
return tab.val
end
return false
end
for i=1, 1000000 do
set(t, math.random(1, 20000000), i)
end
count = 0
for i=1, 20000000 do
if check(t, i) then
count = count + 1
end
end
print(count)
I see 6.92 seconds for the metatable version and 5.99 for the function version.
Also, in 99% of your code, performance is not the issue.
Re: Metatables, NOT
Posted: Mon Oct 24, 2016 9:14 pm
by ssilk
Thanks for answering. I forgot to read the board.
Daid wrote:Your test isn't just testing metatables or not. It's also testing function calls vs inline code. And with so little things to do, inline code is always a lot faster.
Of course. That's the point of that tests: Using of some kind of "clean" concept, vs. ugly, but fast access.
Your code with the metatable is also wrong, as it sets the data on the metatable, which means you cannot share the metatable on multiple tables, which is an important idea of the metatables.
Hm, ah, good to know, that this works also like so. But I thought the point of this is, that I don't need to use function and use the metatable as some kind of "trigger".
Also, in 99% of your code, performance is not the issue.
Running through thousands of entries each tick is.
Re: Metatables, NOT
Posted: Thu Oct 27, 2016 9:14 pm
by Olipro
Doing a quick test of 10M iterations, calling a function that's directly in a table took about 0.55s. This then increased to 0.66 when performed on an empty table with its __index metamethod set to another table. Making __index a function that performed the table load and returned it resulted in the total time going to just over 1s.
So very unscientifically, it's costing my test system about 10ns per function call for the worst case - which if we assume that you're doing maybe 10 function calls on a game tick that's running at the worst case of every 1/60th of a second, the cost of your function calls is running you the tiniest fraction of the available time between game ticks.
If using metatables enables you to write better, more maintainable code, just use them rather than worrying about premature optimisation such as this.