Factorio Forums

www.factorio.com
https://forums.factorio.com/

UPS-optimized green circuits benchmarking

https://forums.factorio.com/viewtopic.php?f=202&t=101354

Page 1 of 1

UPS-optimized green circuits benchmarking

Posted: Mon Jan 24, 2022 1:29 am
by causa-sui
I've done some benchmarks of various electronic circuit processes. I may compile some benchmarks for red and blue circuit builds in another thread at a later date.

Criteria for inclusion

- Produces fully compressed blue belts of electronic circuits
- Productivity mk3 x 4 in all assemblers

Testing process

- Using raiguard's Editor Extensions mod, create a blank sandbox world. Delete all the pre-loaded entities. All other mods must be disabled.
- Enter map editor mode. Use infinity undergrounds to supply the copper and iron as the blueprint requires. Use another infinity underground to consume the output.
- Tile the candidate blueprint to 50 belts of output.
- Power can be provided by an infinity accumulator.
- Let the thing run a bit to buffer up. Save and quit.
- Run benchmarks.

I am running benchmarks with the following shell script:

Code: Select all

#!/usr/bin/env bash
#
set -u

save_file="$1"
benchmark_ticks=50000
benchmark_runs=5
epoch="$(date +%s)" || exit

results_file="benchmark-$save_file-$benchmark_ticks-$benchmark_runs-$epoch-results.txt"

(set -x ;
  ./factorio --benchmark "$save_file" \
  --benchmark-ticks "$benchmark_ticks" \
  --benchmark-runs "$benchmark_runs" \
  --benchmark-sanitize
) | tee "$results_file"

# EOF
Results

Dave Mc W: UPS optimized green circuits


Performed 50000 updates in 21031.081 ms
avg: 0.421 ms, min: 0.263 ms, max: 4.992 ms
checksum: 1027054600
Performed 50000 updates in 21190.961 ms
avg: 0.424 ms, min: 0.265 ms, max: 3.048 ms
checksum: 1027054600
Performed 50000 updates in 21103.821 ms
avg: 0.422 ms, min: 0.271 ms, max: 6.299 ms
checksum: 1027054600
Performed 50000 updates in 20976.527 ms
avg: 0.420 ms, min: 0.264 ms, max: 4.807 ms
checksum: 1027054600
Performed 50000 updates in 21019.949 ms
avg: 0.420 ms, min: 0.256 ms, max: 7.406 ms
checksum: 1027054600
Nilaus: Green Circuits (from "Megabase in a Book")


Performed 50000 updates in 21203.577 ms
avg: 0.424 ms, min: 0.283 ms, max: 4.303 ms
checksum: 877709074
Performed 50000 updates in 20644.047 ms
avg: 0.413 ms, min: 0.263 ms, max: 4.252 ms
checksum: 877709074
Performed 50000 updates in 21497.478 ms
avg: 0.430 ms, min: 0.258 ms, max: 4.356 ms
checksum: 877709074
Performed 50000 updates in 20788.749 ms
avg: 0.416 ms, min: 0.257 ms, max: 2.507 ms
checksum: 877709074
Performed 50000 updates in 21336.080 ms
avg: 0.427 ms, min: 0.263 ms, max: 4.048 ms
checksum: 877709074
Belter: Green circuit v3.3


Performed 50000 updates in 22244.888 ms
avg: 0.445 ms, min: 0.292 ms, max: 3.770 ms
checksum: 3471901602
Performed 50000 updates in 22287.778 ms
avg: 0.446 ms, min: 0.290 ms, max: 2.780 ms
checksum: 3471901602
Performed 50000 updates in 21972.319 ms
avg: 0.439 ms, min: 0.286 ms, max: 2.402 ms
checksum: 3471901602
Performed 50000 updates in 22394.460 ms
avg: 0.448 ms, min: 0.299 ms, max: 2.658 ms
checksum: 3471901602
Performed 50000 updates in 22349.257 ms
avg: 0.447 ms, min: 0.301 ms, max: 4.613 ms
checksum: 3471901602
ranzolger: Green Circuits v3.1


Performed 50000 updates in 23825.908 ms
avg: 0.477 ms, min: 0.327 ms, max: 4.557 ms
checksum: 4011948114
Performed 50000 updates in 23423.982 ms
avg: 0.468 ms, min: 0.325 ms, max: 4.283 ms
checksum: 4011948114
Performed 50000 updates in 23629.409 ms
avg: 0.473 ms, min: 0.325 ms, max: 4.427 ms
checksum: 4011948114
Performed 50000 updates in 23697.277 ms
avg: 0.474 ms, min: 0.331 ms, max: 6.124 ms
checksum: 4011948114
Performed 50000 updates in 23757.411 ms
avg: 0.475 ms, min: 0.332 ms, max: 6.888 ms
checksum: 4011948114
Niz: Electronic Circuits


Performed 50000 updates in 24477.899 ms
avg: 0.490 ms, min: 0.329 ms, max: 3.753 ms
checksum: 1861320690
Performed 50000 updates in 25517.017 ms
avg: 0.510 ms, min: 0.336 ms, max: 3.532 ms
checksum: 1861320690
Performed 50000 updates in 25174.624 ms
avg: 0.503 ms, min: 0.331 ms, max: 4.214 ms
checksum: 1861320690
Performed 50000 updates in 24570.716 ms
avg: 0.491 ms, min: 0.336 ms, max: 4.064 ms
checksum: 1861320690
Performed 50000 updates in 24754.853 ms
avg: 0.495 ms, min: 0.326 ms, max: 4.933 ms
checksum: 1861320690
Nilaus: Green Circuits (from "Factorio Master Class")


Performed 50000 updates in 33288.551 ms
avg: 0.666 ms, min: 0.485 ms, max: 3.243 ms
checksum: 980448276
Performed 50000 updates in 32537.476 ms
avg: 0.651 ms, min: 0.470 ms, max: 3.385 ms
checksum: 980448276
Performed 50000 updates in 31812.525 ms
avg: 0.636 ms, min: 0.454 ms, max: 4.804 ms
checksum: 980448276
Performed 50000 updates in 32645.776 ms
avg: 0.653 ms, min: 0.469 ms, max: 3.446 ms
checksum: 980448276
Performed 50000 updates in 32992.573 ms
avg: 0.660 ms, min: 0.477 ms, max: 3.655 ms
checksum: 980448276

Re: UPS-optimized green circuits benchmarking

Posted: Mon Jan 24, 2022 8:01 am
by Belter
This is really cool! Created a quick chart w/UPS (higher is better).

Suggestions:
- increase the # of belts (100? 500?)
- CSV output would be better for quick charts

Re: UPS-optimized green circuits benchmarking

Posted: Mon Jan 24, 2022 5:19 pm
by causa-sui
Cool. I should really learn graphviz. Then I could generate these automatically.

Re: UPS-optimized green circuits benchmarking

Posted: Mon Jan 24, 2022 9:30 pm
by ranzolger
i tried to have a closer look to understand what makes which build more UPS friendly. but i can't find an obvious reason.

the second most ups friendly version (Nilaus - Megabase in a book) has more (or equal) of everything but is way more UPS friendly than my design. i don't get it. i'm actually curious what may be the reason for this.

Re: UPS-optimized green circuits benchmarking

Posted: Mon Jan 24, 2022 11:16 pm
by causa-sui
That result surprised me too, but I didn't want to re-run it just because it surprised me, since that's a well-known methodological bias in scientific sampling. If someone else would attempt to replicate my results and got something different I'd *love* to know.

Re: UPS-optimized green circuits benchmarking

Posted: Mon Jan 24, 2022 11:43 pm
by quyxkh
ranzolger wrote: ↑Mon Jan 24, 2022 9:30 pm i tried to have a closer look to understand what makes which build more UPS friendly. but i can't find an obvious reason.

the second most ups friendly version (Nilaus - Megabase in a book) has more (or equal) of everything but is way more UPS friendly than my design. i don't get it. i'm actually curious what may be the reason for this.
I took a look, yours is executing more inserter swings per delivered chip. Inserter swings are ex-pen-sive, the game calculates a new arm position every tick for at least 26 ticks. Individually cheap calcs, but they add up.

Re: UPS-optimized green circuits benchmarking

Posted: Tue Jan 25, 2022 7:19 am
by Belter
Would be nice to publish the savegames to replicate the measurements easily. I really want to re-run the benchmarks.

Unfortunately the 3rd one UPS-wise (my BP) is the same as the 2nd just a little worse :D - came up with that w/o seeing the 2nd...

I beleive if I time the inserters it will outperform the 1st one. Will share it here maybe in a week. But then it will be offtopic as not circuit free... measurements will tell, to be fair both the 1st and 2nd should be timed and compared.

Re: UPS-optimized green circuits benchmarking

Posted: Tue Jan 25, 2022 2:37 pm
by causa-sui
Ok, I can post the saves later today.

Re: UPS-optimized green circuits benchmarking

Posted: Tue Jan 25, 2022 7:26 pm
by disentius
I did the benchmarks the same way as with steel.

- made all saves from the same base save (empty.zip)
- 50000 tics, 5 repeats
- removed the roboports from ranzolger's and Niz' save
- ran all saves until they had a stable output for at leas 10 minutes. (100 belts, 270000 i/s)
- Made a second version of dave's save to see if adding stubs and one lane belts would make a difference. (it does, but not much:))

link to all files, including the detailed logs per run:
https://drive.google.com/drive/folders/ ... sp=sharing
the averages row is sorted from best to worst.
2022-01-25 20_24_54-Window.png
2022-01-25 20_24_54-Window.png (115.5 KiB) Viewed 7366 times

Re: UPS-optimized green circuits benchmarking

Posted: Tue Jan 25, 2022 11:22 pm
by causa-sui
Interesting. Removing the roboports helped? Even with no bots in them?

I could probably make a python script that would parse the benchmark outputs into csv, if you would find that easier.

Re: UPS-optimized green circuits benchmarking

Posted: Wed Jan 26, 2022 9:08 am
by disentius
It is csv. I use this powershell script.
Thinking about changing it, some parts are over complicated, i think)
All times are UTC
Page 1 of 1

Powered by phpBB® Forum Software © phpBB Limited