Page 1 of 1

[0.15.33] Ungraceful handling of UTF8-Byte-order-mark

Posted: Sun Aug 20, 2017 4:14 pm
by eradicator
When encountering a file encoded in UTF-8-with-BOM (Byte Order Mark) factorio currently produces one of several unhelpful errors.I don't know if it's realistic to ask that factorio loads these files normally, so instead i'd be content if it gave a proper error message at least.
(Background context: I frequently deal with files from asian countries, where non Unicode/Latin encodings are common and broken files due to encoding errors far from unheared of, so all my programs are set up to add a BOM by default. I've also seen some other modders be baffled by the errors before.)

Errors currently produced when a file has a BOM:

info.json -> factorio ignores the whole mod completely and starts without it
locale.cfg -> "Missing value at __mod-name__/locale/en/en.cfg:2"
data.lua/control.lua -> "unexpected symbol near char(239)" (seems to always be 239)

Re: [0.15.33] Ungraceful handling of UTF8-Byte-order-mark

Posted: Mon Aug 21, 2017 2:29 pm
by posila
Hi, I appreciate you didn't formulate it as "You should support reading files with BOM", so I thought about it and if we are going to detect BOM to throw better errors, we might aswell just skip it.

So in 0.16 reading json, ini or lua with BOM should not produce any errors.

Re: [0.15.33] Ungraceful handling of UTF8-Byte-order-mark

Posted: Thu Aug 24, 2017 9:31 am
by eradicator
Thanks! This kind of support is the reason i still find it worth to write bug reports in the first place ;).
Looking very much forward to not getting that error message every time i add a new file to the project and forget to change encoding ^_^.
posila wrote:Hi, I appreciate you didn't formulate it as "You should support reading files with BOM".
Well, i've had my share of text encoding "fun", so i respect everyone who dares to jump down into that particular rabbit hole ;)