[1.1.76] Memory leak when iterating UTF-8 character set

Things that we don't consider worth fixing at this moment.
Post Reply
BurninSun
Inserter
Inserter
Posts: 47
Joined: Fri Mar 16, 2018 4:54 am
Contact:

[1.1.76] Memory leak when iterating UTF-8 character set

Post by BurninSun »

When iterating over the UTF-8 character set and sending those characters to be displayed in a label, the game will slow down and eventually crash. I am specifically NOT creating new frames or labels while doing this, I'm just changing `label.caption` repeatedly. I would expect the video memory to be freed up as this runs.

Run the following from the console and wait a few moments:

Code: Select all

game.speed = 10000
local labels = {}
for i=0, 63 do
  labels[i] = game.player.gui.screen.add{type="frame", visible=false}.add{type="label", style="label"}
end
local bytes = {}
local size = 1
local prefix

local function on_tick()
  if size == 1 then
    if not prefix then
      prefix = 0x00
    else
      prefix = prefix + 0x40
    end
    if prefix == 0x80 then
      size = 2
      bytes[1] = 0xC0
    end
  else
    bytes[size-1] = bytes[size-1] + 1
    if size == 2 and bytes[1] == 0xE0 then
      size = 3
      bytes[2] = 0x80
    end
  end
  for i = size-1, 2, -1 do
    if bytes[i] == 0xC0 then
      bytes[i] = 0x80
      bytes[i-1] = bytes[i-1] + 1
      if i == 2 then
        if bytes[1] == 0xF0 then
          size = size + 1
          for j = 2, size-1 do
            bytes[j] = 0x80
          end
        elseif bytes[1] == 0xF8 then
          script.on_event(defines.events.on_tick)
          game.speed = 1
        end
      end
    end
  end

  for i = 0, 63 do
    bytes[size] = i+prefix
    labels[i].caption = string.char(table.unpack(bytes))
  end
end

script.on_event(defines.events.on_tick, on_tick)
The game will progressively slow down until eventually logging the following error:

Code: Select all

 792.989 D3D11_ERROR: ID3D11Device::CreateTexture2D failed in VideoBitmapDX11::createInternalTexture on line 243. Error [0x80070057] - E_INVALIDARG
 792.989 > TextureDesc: Width=9600, Height=19200, MipLevels=1, ArraySize=1, Format=61, Usage=0, BindFlags=8, CPUAccessFlags=0, MiscFlags=0
At this point, the game will continue to run, including sound effects and logging and such, but video will be frozen and the game will not accept any inputs. After a bit more time, the game will crash while logging the same error again.

Rseding91
Factorio Staff
Factorio Staff
Posts: 13371
Joined: Wed Jun 11, 2014 5:23 am
Contact:

Re: [1.1.76] Memory leak when iterating UTF-8 character set

Post by Rseding91 »

Thanks for the report. I tried to reproduce the crash but it seemingly worked fine for me (maybe related to the amount of GPU memory I have?)

Regarding the increased memory usage: as far as I understand that is expected. The font logic caches gliphs the first time they are used and will keep them in memory for the lifetime of the program for future re-use. Memory usage and video memory usage will scale linearly with the unique amount of gliphs requested to be rendered.

Why exactly are you trying to render every single character? (about 1.1 million from what I can find)
If you want to get ahold of me I'm almost always on Discord.

BurninSun
Inserter
Inserter
Posts: 47
Joined: Fri Mar 16, 2018 4:54 am
Contact:

Re: [1.1.76] Memory leak when iterating UTF-8 character set

Post by BurninSun »

I'm using a nVidia GTX 1080 with 8GB of memory.

I tried enabling crash logging but it seems this is unhandled so it doesn't generate a crash log to send, it just pops up an error box "Failed to create a texture".

Why I'm doing it is that I'm trying to create a LuaRendering.draw_text equivalent that supports rich text. To figure out offsets of where to start/stop text vs sprites, I need the rendered width of the text. Since there is no way of getting that on the fly, I'm rendering every character to record its width into a lookup table beforehand and using that to figure out the rendered width of a given text block.

I could parse the .ttf files, but I already had a solution within the Factorio engine itself so figured I'd use that except this crash issue.

Log file is attached.
Attachments
factorio-current.log
(8.02 KiB) Downloaded 79 times

User avatar
atomizer
Long Handed Inserter
Long Handed Inserter
Posts: 86
Joined: Sat Sep 22, 2018 3:18 pm
Contact:

Re: [1.1.76] Memory leak when iterating UTF-8 character set

Post by atomizer »

BurninSun wrote:
Wed Feb 15, 2023 8:13 pm
Since there is no way of getting that on the fly
Could you elaborate? Isn't the code in the first post "on the fly" (minus the fancy loop)?

BurninSun
Inserter
Inserter
Posts: 47
Joined: Fri Mar 16, 2018 4:54 am
Contact:

Re: [1.1.76] Memory leak when iterating UTF-8 character set

Post by BurninSun »

atomizer wrote:
Wed Feb 15, 2023 9:28 pm
Could you elaborate? Isn't the code in the first post "on the fly" (minus the fancy loop)?
rendering.draw_text doesn't give the width of the resulting element. But its possible to get what the width should be by putting that text into a gui label. The problem with a label is that it takes 1 tick to read back the size, and that hack can break on ui scaling/resolution changes.

posila
Factorio Staff
Factorio Staff
Posts: 5203
Joined: Thu Jun 11, 2015 1:35 pm
Contact:

Re: [1.1.76] Memory leak when iterating UTF-8 character set

Post by posila »

Thanks for the report.
I might look into making it not crash and just stop rendering new glyphs, that it can't fit into its cache, but I am not interested in making the game able to display all the glyphs over course of single run.

I am kind of surprised you are able to determine width of text from LUA, as it sound like potential source of desyncs. Width of text depends on font used and font used depends on language settings.

BurninSun
Inserter
Inserter
Posts: 47
Joined: Fri Mar 16, 2018 4:54 am
Contact:

Re: [1.1.76] Memory leak when iterating UTF-8 character set

Post by BurninSun »

posila wrote:
Thu Feb 16, 2023 11:46 am
I am kind of surprised you are able to determine width of text from LUA, as it sound like potential source of desyncs. Width of text depends on font used and font used depends on language settings.
I'm able to get width by putting text into a `.gui.screen` label, call `.force_auto_center()`, wait a tick, get `.location.x` on the frame, then do some math. Since GUI elements are specifically per player, I'd assume that avoids the desync issue.

DaleStan
Filter Inserter
Filter Inserter
Posts: 368
Joined: Mon Jul 09, 2018 2:40 am
Contact:

Re: [1.1.76] Memory leak when iterating UTF-8 character set

Post by DaleStan »

BurninSun wrote:
Thu Feb 16, 2023 7:09 pm
I'm able to get width by putting text into a `.gui.screen` label, call `.force_auto_center()`, wait a tick, get `.location.x` on the frame, then do some math. Since GUI elements are specifically per player, I'd assume that avoids the desync issue.
The desync problem is that now you have access to a number that's different for different players. If that number stays in the GUI code it's probably OK, but if you let it leak out into the game state it becomes not OK. For example, if the width of a progress bar changes based on the text width, that's probably fine by itself. But if you do something to the game state every time the progress bar grows by a pixel, that's not fine.

Xorimuth
Filter Inserter
Filter Inserter
Posts: 639
Joined: Sat Mar 02, 2019 9:39 pm
Contact:

Re: [1.1.76] Memory leak when iterating UTF-8 character set

Post by Xorimuth »

DaleStan wrote:
Sun Feb 19, 2023 4:20 pm
BurninSun wrote:
Thu Feb 16, 2023 7:09 pm
I'm able to get width by putting text into a `.gui.screen` label, call `.force_auto_center()`, wait a tick, get `.location.x` on the frame, then do some math. Since GUI elements are specifically per player, I'd assume that avoids the desync issue.
The desync problem is that now you have access to a number that's different for different players. If that number stays in the GUI code it's probably OK, but if you let it leak out into the game state it becomes not OK. For example, if the width of a progress bar changes based on the text width, that's probably fine by itself. But if you do something to the game state every time the progress bar grows by a pixel, that's not fine.
It is fine, because gui.screen is actually player.gui.screen, i.e. it is per-player. All clients know about all other clients’ GUI state, including positioning. So there’s no problem with desyncs here.
My mods
Content: Lunar Landings | Freight Forwarding | Spidertron Patrols | Spidertron Enhancements | Power Overload
QoL: Factory Search | Remote Configuration | Module Inserter Simplified | Wire Shortcuts X | Ghost Warnings

Post Reply

Return to “Won't fix.”