[Oxyd] [0.15.28] Two blueprints can't in library at the same time

[bd_]

In 0.15.28, on Windows (64-bit), the following two blueprints can't be added to the blueprint library at the same time:

Code: Select all

0eNqdlltugzAQRfcy36ZihkcStlJVFUncyBIxyDZVUcTea0CR0oQ09vBj8fC5d/BlzAX2TS87o7SD6gLq0GoL1fsFrDrpupmuuaGTUIFy8gwCdH2ezr5q6xLbNco5aWAUoPRR/kCFowib60ytbdcal+xl424INH4IkNopp+RiZT4ZPnV/3nutCp+YENC11s9q9aTsSQkJGPyAb4XHH5WRh+UuTSbvqPSX2nsz5mRaPy4GH+jpAs/u0OJac9u7rp8Ke5DKmFIUL5XHSiX4oiyln0gV/67vUyEaV1hlHItWPecr4A0LTK/jsw0N5bXskEzugpOeLdQ0hIpp5AcURsW4V7tkOn29ZEisYKUBljOO5QTXEos5L/4YYLPg9QtGu8CS2S8oul3ghlcVMqrastYGA7K54/ST1ZZHKQcVEB/CuNa0slv6DXnewaubnwUB39LY+YEyz6nMcn/sxvEXzMLHPQ==

Code: Select all

0eNqdldGKgzAQRf9lniMkaWqrv7KURdthCWiUJF0qkn/fWF922QF1fJEYPd7cucPM0HZPHL11EeoZ7H1wAeqPGYL9ck23PIvTiFCDjdiDANf0ywpfo8cQiugbF8bBx6LFLkISYN0DX1CrdBOALtpocSW+F9One/Yt+vzCFkvAOIT8+eAWFRlZaAFTvqn8l4f1eF/3yiT+wTUbrv/CDQE/HYZLUrgm2Oa4cLXblTNXuN5ml2zhetuVy2E4bQpVzSuXvUN3xY2hSgRNSS5OkjjFjAMtTnMDsKMrFLfl6IOzm0xu94E6M/O0I6qqZLJpGy7cOGkSd2XWiKZV3BqROC2ZzmVanmPv2Vf/GpUCvtGHNQTG6PJk8lWl9ANC8HuF

Here's what it looks like if you try: https://www.youtube.com/watch?v=vs3uEIIrllw

[impetus maximus]

perhaps they have the same CRC info as discussed here.

[bd_]

In fact they do

[Patashu]

That didn't take long for someone to find a collision!

[Loewchen]

As useless as the first BP is, it seems as if it was intentionally created for this exact purpose.

[Rseding91]

As useless as the first BP is, it seems as if it was intentionally created for this exact purpose.

It was https://www.reddit.com/r/factorio/comme ... _same_crc/

[impetus maximus]

ok how bout this for a challenge. show us two blueprints someone would actually use.

[TruePikachu]

ok how bout this for a challenge. show us two blueprints someone would actually use.

By what metric? There is somebody who would use _every_ valid blueprint, even if only for decoration or a compendium of every single possible blueprint.

---

That said, just matching a CRC to detect duplicate blueprints isn't enough, especially since there isn't a huge amount of entropy present in CRC. Aside from possibly moving to a cryptographic hash, my suggestion (which should probably be implemented either way, as a "just in case" percaution) is to, if the CRCs (other hashes are available) match, run an O(n) comparison on the data. In theory, this wouldn't need to be done too frequently, but if there is some facet I'm missing let me know.

[adeade45]

ok how bout this for a challenge. show us two blueprints someone would actually use.

By what metric? There is somebody who would use _every_ valid blueprint, even if only for decoration or a compendium of every single possible blueprint.

---

That said, just matching a CRC to detect duplicate blueprints isn't enough, especially since there isn't a huge amount of entropy present in CRC. Aside from possibly moving to a cryptographic hash, my suggestion (which should probably be implemented either way, as a "just in case" percaution) is to, if the CRCs (other hashes are available) match, run an O(n) comparison on the data. In theory, this wouldn't need to be done too frequently, but if there is some facet I'm missing let me know.

I'm not sure how CRCs work, but would it be possible to simply add a salt to the data then recheck CRC if a collision is detected? If the two data are identical, then the salted data would also produce identical CRCs, but would the salt change the CRC if the original data were different?

[Patashu]

ok how bout this for a challenge. show us two blueprints someone would actually use.

By what metric? There is somebody who would use _every_ valid blueprint, even if only for decoration or a compendium of every single possible blueprint.

---

That said, just matching a CRC to detect duplicate blueprints isn't enough, especially since there isn't a huge amount of entropy present in CRC. Aside from possibly moving to a cryptographic hash, my suggestion (which should probably be implemented either way, as a "just in case" percaution) is to, if the CRCs (other hashes are available) match, run an O(n) comparison on the data. In theory, this wouldn't need to be done too frequently, but if there is some facet I'm missing let me know.

I'm not sure how CRCs work, but would it be possible to simply add a salt to the data then recheck CRC if a collision is detected? If the two data are identical, then the salted data would also produce identical CRCs, but would the salt change the CRC if the original data were different?

You could, but it's the same amount of work as (or more work than), and less secure, than running a stronger cryoptographic hash, which is the original suggestion. (There is some good discussion on WHAT hash to use in viewtopic.php?f=38&t=50663&start=80 )

[Nexarius]

Whats the problem about making a full comparison of the blueprint if the CRC is identical?

[TruePikachu]

You could, but it's the same amount of work as (or more work than), and less secure, than running a stronger cryoptographic hash, which is the original suggestion. (There is some good discussion on WHAT hash to use in viewtopic.php?f=38&t=50663&start=80 )

Strictly speaking, a strong hash will frequently be slower than CRC.

Whats the problem about making a full comparison of the blueprint if the CRC is identical?

The main possible "issue" is that you get a O(n) byte-by-byte comparison, in contrast to the O(1) CRC checking.

Theoretically, however, this shouldn't be much of a problem. The only times that blueprints need to be checked for duplication are when they are being added to the library manually, and the only time the O(n) comparison is needed is if the O(1) comparison matches. This would usually only happen in the case that the blueprints are identical (which results in the maximum runtime from O(n) comparison), but it can happen in the example given here (which would result in less than maximum runtime).

It might be possible to optimize the O(n) comparison quite a bit, though, by first comparing the fields most likely to differ (and length). However, in the case of actual duplicates, it is impossible to get better than O(n) time.

[Patashu]

You could, but it's the same amount of work as (or more work than), and less secure, than running a stronger cryoptographic hash, which is the original suggestion. (There is some good discussion on WHAT hash to use in viewtopic.php?f=38&t=50663&start=80 )

Strictly speaking, a strong hash will frequently be slower than CRC.

Yeah, but it doesn't matter if it's a bit slower, since making a new blueprint happens once per minute or slower, compare that to the UPS savings of making something like belts or inserters run faster (which happens 10000s of times per second)

[Oxyd]

Fixed in 0.15.29.