Issue uploading new release of a mod

[Hiladdar]

Hi,

I made several attempts to upload a new release to one of my mods of several computers. In all cases, after the file is uploaded, it the display is "Status (x) Invalid". I have tested this mod patch on several computers and it works fine. This is the third release of this mod.

Given that there were posts regarding issues with the mod portal made within the last two days, at this time, I am not sure if it is something within the mod, or if there is an issue uploading mods to the mod portal and I need some assistance resolving this issue.

Hiladdar Strila

[Sanqui]

Heya, can you please upload the mod as an attachment here so I can take a look?

Thanks!

[Hiladdar]

Hi

First off thanks for the fast response especially on a weekend. Thinking about the issue over night, since this mod adds wagon trains with both high and standard resolution graphics, the size of the mod as grown from less then 100k to about 52m.

I tried to upload as an attachment and it web site choked, I think the issue is due to the size of the file.

I'll try to upload the structure of what I am doing onto Github. The other thing I will try strip out the high resolution graphics (about 40mb) and just publish the module with the standard graphics for the cargo wagons (about 10mb).

Hiladdar

[Hiladdar]

Apparently there is a cap to the file size uploaded to this forum. I was able to get the content up on Github at https://github.com/Hiladdar/Factorio-Trains.

The readme.txt on github explains how to resemble the zipped mod on a PC since Github has a 25mb limit to the size of an individual file to upload.

One thing I can try on my end today, is to use normal resolution graphics in lieu of high resolution graphics. This should drop about 40mb from the final zip file and can be accomplished with minimal code changes.

One other option I can take a look at is if there is a way add a tint to existing graphic files via the .lua script. If I can do that, then I can bring the size of the mod down to the 1mb range.

Hiladdar

[Hiladdar]

I have an interim solution implemented skipping version 3 of the mod and going directly to version 4.

It appears that the issue was the size of the final zip file. Once I stripped out the high resolution graphic entity files, modified the code to use normal resolution files, I was able to upload the mod and have it listed. Dropping the high resolution files brought the size of the mod down to 12.56m and the mod portal / bandwidth was able to handle that without any issues.

I am not sure if there is a soft or hard limit on the size of the mod, or if the issue is with my upload bandwidth, or time-out trying to upload the mod.

The current version is on: https://mods.factorio.com/mod/Hiladdar_Trains/downloads and version 0.16.4.

If there is a better way for me to do this, I would like to learn it and implement it.

Again, thanks reaching out to help.

Hiladdar

[quyxkh]

If factorio supports the jpeg'd-image-with-a-png'd-transparency-mask trick you could shrink those down by about 85%, on the subdir I tried for instance

Code: Select all

$ for f in *[0-9].png; do convert -quality 20 $f ${f%.png}.jpg; convert -alpha extract $f ${f%.png}.alpha.png; done
$ ls -l
total 23412
-rw-r--r-- 1 quyxkh quyxkh  114548 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-1.alpha.png
-rw-r--r-- 1 quyxkh quyxkh  109913 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-1.jpg
-rw-r--r-- 1 quyxkh quyxkh 1783787 Feb 15 16:16 hsmd-hr-fluid-wagon-mk2-1.png
-rw-r--r-- 1 quyxkh quyxkh  139556 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-2.alpha.png
-rw-r--r-- 1 quyxkh quyxkh  116529 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-2.jpg
-rw-r--r-- 1 quyxkh quyxkh 1947256 Feb 15 16:17 hsmd-hr-fluid-wagon-mk2-2.png
-rw-r--r-- 1 quyxkh quyxkh  138031 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-3.alpha.png
-rw-r--r-- 1 quyxkh quyxkh  117230 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-3.jpg
-rw-r--r-- 1 quyxkh quyxkh 1896695 Feb 15 16:18 hsmd-hr-fluid-wagon-mk2-3.png
-rw-r--r-- 1 quyxkh quyxkh  122265 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-4.alpha.png
-rw-r--r-- 1 quyxkh quyxkh  111962 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-4.jpg
-rw-r--r-- 1 quyxkh quyxkh 1718343 Feb 15 16:18 hsmd-hr-fluid-wagon-mk2-4.png
-rw-r--r-- 1 quyxkh quyxkh  121964 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-5.alpha.png
-rw-r--r-- 1 quyxkh quyxkh  111107 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-5.jpg
-rw-r--r-- 1 quyxkh quyxkh 1711775 Feb 15 16:19 hsmd-hr-fluid-wagon-mk2-5.png
-rw-r--r-- 1 quyxkh quyxkh  141011 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-6.alpha.png
-rw-r--r-- 1 quyxkh quyxkh  117620 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-6.jpg
-rw-r--r-- 1 quyxkh quyxkh 1871424 Feb 15 16:20 hsmd-hr-fluid-wagon-mk2-6.png
-rw-r--r-- 1 quyxkh quyxkh  149437 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-7.alpha.png
-rw-r--r-- 1 quyxkh quyxkh  125439 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-7.jpg
-rw-r--r-- 1 quyxkh quyxkh 2014524 Feb 15 16:21 hsmd-hr-fluid-wagon-mk2-7.png
-rw-r--r-- 1 quyxkh quyxkh  119382 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-8.alpha.png
-rw-r--r-- 1 quyxkh quyxkh  118906 Feb 17 12:50 hsmd-hr-fluid-wagon-mk2-8.jpg
-rw-r--r-- 1 quyxkh quyxkh 1907048 Feb 15 16:21 hsmd-hr-fluid-wagon-mk2-8.png
-rw-r--r-- 1 quyxkh quyxkh  112754 Feb 17 12:50 hsmd-hr-fluid-wagon-mk3-1.alpha.png
-rw-r--r-- 1 quyxkh quyxkh   87626 Feb 17 12:50 hsmd-hr-fluid-wagon-mk3-1.jpg
-rw-r--r-- 1 quyxkh quyxkh 1515467 Feb 15 16:40 hsmd-hr-fluid-wagon-mk3-1.png
-rw-r--r-- 1 quyxkh quyxkh  137340 Feb 17 12:50 hsmd-hr-fluid-wagon-mk3-2.alpha.png
-rw-r--r-- 1 quyxkh quyxkh   90706 Feb 17 12:50 hsmd-hr-fluid-wagon-mk3-2.jpg
-rw-r--r-- 1 quyxkh quyxkh 1652400 Feb 15 16:43 hsmd-hr-fluid-wagon-mk3-2.png
-rw-r--r-- 1 quyxkh quyxkh  135937 Feb 17 12:50 hsmd-hr-fluid-wagon-mk3-3.alpha.png
-rw-r--r-- 1 quyxkh quyxkh   91279 Feb 17 12:50 hsmd-hr-fluid-wagon-mk3-3.jpg
-rw-r--r-- 1 quyxkh quyxkh 1603419 Feb 15 16:50 hsmd-hr-fluid-wagon-mk3-3.png
-rw-r--r-- 1 quyxkh quyxkh  120549 Feb 17 12:50 hsmd-hr-fluid-wagon-mk3-4.alpha.png
-rw-r--r-- 1 quyxkh quyxkh   88342 Feb 17 12:50 hsmd-hr-fluid-wagon-mk3-4.jpg
-rw-r--r-- 1 quyxkh quyxkh 1438070 Feb 15 16:51 hsmd-hr-fluid-wagon-mk3-4.png
$ 

doing some quick arithmetic says that averages about 87% size reduction e.g. for the mk2-1 it's 1-(114548+109913)/1783787 ~ 87.4% reduction.

There's almost no discernible difference between the resulting images.

[Hiladdar]

Thanks for the reply.

I've done some work with JPG files, where I needed to get into the internal structure of JPG files and how they did their compression. They are great for compressing a base image to a fraction of it's size, but bill payer for the compression the loss of transparency / animation.

The lightest file format that supports transparency is GIF, with 256 colors. PNG evolved from the GIF format to support 16M colors.

JPG files that are animated are usually converted into GIF files, than animated, than renamed back to a JPG file. The problem with storing the animation within a JPG file is inside the JPG file and how it divides the image internally. The way it works, is the image is divided into chunks and each chunk is compressed and stored within the JPG file.

I think your recommendation does have merit, if Factorio's graphic engine can handle other file formats other then PNG. Without any testing there is no way to tell. Where your recommendation does warrant testing is trying to use GIF graphic images for the icons or JPG images for entities which do not have any animation.

One thing I am looking at for a possible release, is using the graphics provided within the base folder, and figuring out how to apply a tint to them. Using this technique, would eliminate need to distribute the files within the ./graphics/entity/fluid-wagon folder and changing the color within the .lua definition for the entity.

Hiladdar

[BlueTemplar]

I thought that PNG had smarter algorithms that made the pictures smaller compared to a similar GIF ?

[Hiladdar]

In summery JPG is 24 bit image which uses compression with an image detail reduction upon reproduction, GIF is 8 bit that offers some of the capability that is needed from PNG files but is not lossey as JPG is, while PNG is 24 bit with some compression does not loose quality and transparency, although PNG's compression is not as good as JPGs.

https://www.howtogeek.com/howto/30941/w ... g-and-gif/ provides a non technical explanation of each of the three graphic image formats as well as the advantages and disadvantages of using each one.

Hiladdar

[BlueTemplar]

8-bit PNG is an option, but GIF is smaller

Hmm, how much though ?

[DaveMcW]

GIF is only smaller than 8-bit PNG on icons, due to GIF's more efficient palette definition. On larger 8-bit images PNG's superior compression wins.

Never use GIF, it is inferior to PNG for static images and inferior to MP4 for animations.

[GrumpyJoe]

https://www.howtogeek.com/howto/30941/w ... g-and-gif/ provides a non technical explanation of each of the three graphic image formats as well as the advantages and disadvantages of using each one.

Hiladdar

Interesting read for someone that doesnt need it in his every day life.
I watch alot of Computerphile and this guy is just awesome at explaining stuff. There are some interesting topics, not only for graphics. Not really related to this topic, but information sharing is what forums are for?
https://www.youtube.com/playlist?list=P ... 676yZhcN1V

[BlueTemplar]

GIF is only smaller than 8-bit PNG on icons, due to GIF's more efficient palette definition. On larger 8-bit images PNG's superior compression wins.

Never use GIF, it is inferior to PNG for static images and inferior to MP4 for animations.

Not even for small animations like these ? =>
(Though I guess that browsers now have support for autoplay/autorewind/automute tags ?)

MP4/h.264/i-forgot-what-the-audio-format-is is 1.5 generations old at this point (and closed), we should probably switch to WEBM/AV1 ASAP.

[DaveMcW]

AV1 has no hardware support as of 2019, which makes it useless on most systems (mobile devices).

[BlueTemplar]

I know (- not a single device though? -), that's why the P in ASAP.
Not necessarily useless though - depends on how well it can be decoded in software !

[Sanqui]

Apparently there is a cap to the file size uploaded to this forum. I was able to get the content up on Github at https://github.com/Hiladdar/Factorio-Trains.

I followed your instructions to reconstruct the mod and uploaded it on the mod portal with no problems: https://mods.factorio.com/mod/Hiladdar_ ... Sanquitest

I advise you to try again, it could be a problem with your network connection... if it doesn't work I can try to help you get it working in another way.

[Hiladdar]

Thank you for your response.

You are probably correct in your assessment that it may be an internet connection issue. The internet connection is DSL with effective download of about 4.5m and effective upload of 300k, which is currently the best available where I live. I have already tried to upload off different computers both with and without VPN. Today, I tried to upload version 3 using a different browser (IE and Edge). Neither worked.

For the next subsequent release of this mod, I did come up with an interim work around which consists of skipping release 3, and going with release 4. Release 4 in summery is the same as release 3, but uses normal graphics files in lieu of high resolution graphics for animation.

Long term, I need to come up with way to get content up to the mod portal without developer intervention. At this time I am looking at several different ways to accomplish this, shrinking the size of the mods, as well as better mastery of the .lua programming and graphics. One viable workaround is to post the graphic files on gighub, especially for high res graphics and programmatically have the client use high-resolution graphics if the graphic option is set and high-resolution graphics are downloaded. Another option is to load and tint the graphic files programmatically from within .lua. Other options I am looking at is using an internet connection at a different location or public wifi to upload files, as well as upgrading my internet connection when it becomes available within my area.

I did download the reassembled file and test it. Only change I needed to make to it was drop the reference to "Sanquistest" from the zipped file name and within the .json file for everything to work. That version can be put up, but with version 4 being normal resolution graphics, buy, it may be a bit problematic for the user using having higher resolution graphics in a lower numbered version, so it is probably best to keep everything as is.

Hiladdar

[BlueTemplar]

It's not uncommon to have separate mods for higher resolution graphics (or heavy audio files) :
Alien Biomes for instance

[Hiladdar]

Thanks for the recommendation BlueTemplar!

I've downloaded the Alien Biomes both base and high resolution graphics, and taken a quick look at the code. Those mods contain a lot of code and graphics that I can learn from. I'll take some time to study that code.

Hiladdar