[2.0.28] Non-latin character limit in combinator description is lower

[dodther]

The description fields in combinators have a 500-character limit. This is more than enough for a small note. However, unfortunately, this only applies to Latin characters. For characters from other alphabets, such as Cyrillic, only 250 characters fit. This is exactly half, and it's no longer enough for a note. I wish this issue could be fixed.

[Matheo]

Most likely reason is a fixed memory allocation for descriptions.

Latin characters are more commonly used, therefore encoding standards such as UTF-8 prioritize them and allow them to only use 1 byte of memory per character. Using characters from different alphabets requires more bytes, increasing memory required for each symbol.

This problem can only be fixed by increasing the description max length. You should be able to do that with a mod, but I don't think this change will be implemented in the base game.

[evanrinehart]

It might be cool and feasible to accommodate the scripts that usually take 2 utf8 bytes per character by doubling the backend allocation then limit it to 500 characters (not bytes) anyway even though ASCII could fit 1000. Nothing would seem to change for existing notes which are 500 or less currently.

[Rseding91]

Thanks for the report however I don't believe we will be changing this at this time. It's desirable for these checks to be fast, and language agnostic, and changing it from a 'byte limit' to a 'character limit' would require changing the check from O(1) to O(N).