Metatables, NOT

[ssilk]

I come from this thread: viewtopic.php?f=25&t=33905


I have (not for everyone interesting) some results: I wanted to create an entity-manager with lua metatales. But before that I made some tests about performance.

First some code:

Implemenation with metatable:

Code: Select all

t={}  -- basetable
mt={} -- metatable
mt.mytab={}
mt.__newindex=function(t,k,v)
    rawset(mt.mytab, k, {val = v, valid = math.random()})
end
mt.__index=function(t,k)
    local tab = rawget(mt.mytab, k)
    if tab and tab.valid < 0.05 then
        return tab.val
    end
    return false
end
setmetatable(t,mt)

for i=1, 1000000 do
    t[math.random(1, 20000000)] = i
end

count = 0
for i=1, 20000000 do
    if t[i] then
        count = count + 1
    end
end

print(count)

Implemenation without metatable:

Code: Select all

bla = {}
bla.t={}  -- basetable
for i=1, 1000000 do
    bla.t[math.random(1, 20000000)] = {
        val = i,
        valid = math.random()
    }
end


count = 0
for i=1, 20000000 do
    if bla.t[i] and bla.t[i].valid < 0.05 then
        count = count + 1
    end
end

print(count)

With metatable: 6.161s

Without metatable: 3.557s

Conclusion: Metatables makes Lua slower by 100%.
Consequnce: No way to use metatables for a game as Factorio is.

Well, I think most of you did know that, but maybe for others it's new. So I wanted that to share with you

[Adil]

Now try that with some api-calls in the cycle.
Metatable is one additional pointer redirection, and, in your example, two additional function calls per step in each cycle.
When you add function calls to the straightforward imperative implementation the gap is starting to decrease:

code

Code: Select all

bla = {}
bla.t={}  -- basetable
function set(i) 
	return {
        val = i,
        valid = math.random()
    }
end
function get(tt)
	return tt and tt.valid < 0.05
end

for i=1, 1000000 do
    bla.t[math.random(1, 20000000)] = set(i)
end


count = 0
for i=1, 20000000 do
    if get(bla.t[i]) then
        count = count + 1
    end
end

print(count)

Also, math.rand() is a rather odd sighting in the benchmark.

With those remarks in mind, if you yarn for performance, indeed you should try express all your code without additional functions, however, it probably won't matter much for anywhere except the on_tick event.
And yes, fancy operator overloading is not a good idea.

Addendum:

Well, I think most of you did know that, but maybe for others it's new. So I wanted that to share with you

Apparently, Choumiko didn't know that.

[aubergine18]

And yes, fancy operator overloading is not a good idea.

http://lua-api.factorio.com/latest/LuaC ... ustomTable

It's a great idea, if it's done properly in the right context. It always depends on the specific nature of the code, how often it's called, etc.

[Afforess]

Metatables are just a poor idea because they aren't serialized. That one issue alone makes them not worth the hassle in most mods. Worse performance is just insult to injury.

[aubergine18]

Metatables are just a poor idea because they aren't serialized. That one issue alone makes them not worth the hassle in most mods. Worse performance is just insult to injury.

It is trivial one line of code to re-apply the metatable, there's even an event specifically aimed at enabling that: script.on_load().

Their performance for the most part is not an issue - it's only if you start using them (or anything else for that matter) in an on_tick event handler. But for all other events, particularly if only accessing the metamethods sparingly, there's really nothing bad about them. Even in situations where they are used extensively, you'd still get better performance improvements, in most cases, by refining the code they contain rather than the metamethods themselves.

The performance stats in OP are also somewhat skewed IMO. Local references to tables and functions would improve performance (although local ref to math.random could cause desyncs, at least in older versions of factorio it did). There's also no need to use rawget in that particular example, there's always alternate ways to implement things.

[Rseding91]

... (although local ref to math.random could cause desyncs, at least in older versions of factorio it did).

That won't cause desyncs. I don't think it ever has...

[Daid]

Your test isn't just testing metatables or not. It's also testing function calls vs inline code. And with so little things to do, inline code is always a lot faster.

Your code with the metatable is also wrong, as it sets the data on the metatable, which means you cannot share the metatable on multiple tables, which is an important idea of the metatables.

If you add back the function call overhead:

Code: Select all

t={}  -- basetable
function set(t,k,v)
    rawset(t, k, {val = v, valid = math.random()})
end
function check(t,k)
    local tab = rawget(t, k)
    if tab and tab.valid < 0.05 then
        return tab.val
    end
    return false
end

for i=1, 1000000 do
    set(t, math.random(1, 20000000), i)
end

count = 0
for i=1, 20000000 do
    if check(t, i) then
        count = count + 1
    end
end

print(count)

I see 6.92 seconds for the metatable version and 5.99 for the function version.


Also, in 99% of your code, performance is not the issue.

[ssilk]

Thanks for answering. I forgot to read the board.

Your test isn't just testing metatables or not. It's also testing function calls vs inline code. And with so little things to do, inline code is always a lot faster.

Of course. That's the point of that tests: Using of some kind of "clean" concept, vs. ugly, but fast access.

Your code with the metatable is also wrong, as it sets the data on the metatable, which means you cannot share the metatable on multiple tables, which is an important idea of the metatables.

Hm, ah, good to know, that this works also like so. But I thought the point of this is, that I don't need to use function and use the metatable as some kind of "trigger".

Also, in 99% of your code, performance is not the issue.

Running through thousands of entries each tick is.

[Olipro]

Doing a quick test of 10M iterations, calling a function that's directly in a table took about 0.55s. This then increased to 0.66 when performed on an empty table with its __index metamethod set to another table. Making __index a function that performed the table load and returned it resulted in the total time going to just over 1s.

So very unscientifically, it's costing my test system about 10ns per function call for the worst case - which if we assume that you're doing maybe 10 function calls on a game tick that's running at the worst case of every 1/60th of a second, the cost of your function calls is running you the tiniest fraction of the available time between game ticks.

If using metatables enables you to write better, more maintainable code, just use them rather than worrying about premature optimisation such as this.