Friday Facts #115 - The power switch

[Dev-iL]

Hey guys, regarding voice acting, why don't you use fiverr for that? You can get professional-level voice actors with any accent you choose, and it won't be expensive at all (I'm talking O(50$))...

[crazybmanp]

This is a really nice solution to making power switches.

[-root]

Compilation times in C++ suck. The fact that I have to wait 3 and half minute whenever Factorio is recompiled makes me impatient and ineffective.

Hey, compiling can be a lot of fun!

Ninja'd.

Hey guys, regarding voice acting, why don't you use fiverr for that? You can get professional-level voice actors with any accent you choose, and it won't be expensive at all (I'm talking O(50$))...

Uh, they could get it for free if they asked. There are numerous people who create content for factorio that have professional setups for recording voice. But if Albert is keen, no need to take his cookie So long as he has good english, he'll be fine!

[LucidMoses]

I have created a few Sting libraries (and other functions that I think should be in std c). Typically for every large project I work on.

I don't know how much string work you do in Factorio but If it's a ton, keep in mind that there is typically a performance issue. Usually the string functions that are part of the base language are written in assembler and are fairly well optimized. This level of performance will be hard to reproduce in std C++ code.

So, if your just using string to show things in the UI then... Yea.... whatever. but if your using them in internal index searches then it may come back to byte you. :p

[Mazzelfassel]

I'm working on a two person project. Its pretty similar to factorio and for personal training in programming and game-dev. The compiling take approx. 40 seconds but it has'nt as much features as Factorio yet. Currently we have a main menu, a UI witch a toolbar and a player inventory, a working item and crafting system and an early map-generator. We are using SFML for the most components. Std and other commonly used functions are included in a global header file which is included in almost every class. So if you dont change this file the compiling only takes approx. 5 seconds.

[fishycat]

the sound box deserves some love <3 <3 <3

At first sight I thought its a huge bullet in a box

[squisher]

I have extensively dealt with this problem of boost + stl compile times. Precompiled headers are the answer.

Have a precompiled.h that includes boost and stl headers.

What compiler are you guys using?

with visual studio, you right-click precompiled.cpp, set precompiled header to create, and set all the other cpp files to use precompiled header "precompiled.h"

gcc supports precompiled headers as well.

gcc -o"precompiled.h.gch" precompiled.h
Anything with extention .gch is treated as precompiled header. For instance, if you have #include "all.h", and you have all.h.gch in the same directory as all.h, then the precompiled header file is used if possible, and the original header is used otherwise.

i use gcc via the eclipse IDE, and my pre-build step in eclipse is this:

if [[ ! -e ../src/precompiled.h.gch ]]; then touch -t 0101010101 ../src/precompiled.h.gch; fi; for i in `find ../src/precompiled.h {and any other dependant files} -newer ../src/precompiled.h.gch | head -1`; do make clean; echo; echo "Recompiling PCH..."; g++ -m32 -D_DEBUG -I{your include dirs} -O0 -Wall -std=c++0x -c -fmessage-length=0 -MMD -MP -MF"src/precompiled.h.gch.d" -MT"src/precompiled.h.gch.d" -o"../src/precompiled.h.gch" "../src/precompiled.h"; echo "Done."; done

which checks to see if precompiled.h is newer than precompiled.h.gch, and if so, recreates precompiled.h.gch

https://gcc.gnu.org/onlinedocs/gcc/Prec ... aders.html

[phorcys02]

C++ slow building solutions:

1. use parallel building , use incredibuild under windows with visual studio. if final link time <30s, total build time will be 30s + 210s/ (machine num in build group). giga network needed.
there is a trial version on xoreax website, try it.

2. use faster machine, overclocked i7 (5ghz) +water cool + faster ssd . os workspace and incredibuild cache must be on the ssd.
3. reduce the symbol number, template and macros generate many unneed symbol that can be optimize. use tools(eg:nm) to dump all symbol list , sort and try to reduce the total symbol number. this can reduce link time.
4. united build
5. custom thirdparty library (like std)

[Isntprepared]

I'd kill to have only a 3 minute compile time. To test a change, I frequently have to wait 1-2 hours for the code to compile o.O .... course the code base easily has 100k+ files and millions of lines of code....

[kovarex]

To answer the C++ stuff.

• Yes, we do use precompiled headers, without it, it takes more than 10 minutes to compile.
• Yes, we use parallel compilation, on my working computer I have 12 logical cores that are used fully.
• I tried the monolith built, and we actually use it to deploy the game, but the monolith build is slower, because it can't be paralleled. The middle point of grouping cpp files in groups of say 10, so there would be 100 modules instead of 1000 could be the optimal middle point, this is what I wanted to try.
• The hardware we use is good enough I believe, I used almost the best thing possible to buy that still isn't a server hardware.
• We have 240k lines curently.
• The module propsal is explained on more places, something here for example: https://meetingcpp.com/index.php/br/ite ... -core.html
• distcc is only usable in linux right? Linux developers in our team could use it, but I use windows currently.

A few ideas to increase compile speed:

1. Turn down optimization levels. Optimizing away templates takes a lot of grunt.
2. Try clang for faster test builds (http://www.phoronix.com/scan.php?page=n ... px=mtgznde)
3. Rent a big computer in the cloud. AWS has 36 core computers with 60G RAM for example.

Can I use clang in visual studio? From what I'v read, the setup isn't simple.

[nobodx]

Any time waiting for a computer is wasted time, doesn't matter if it's 10 seconds or 2 hours.

(I test ATMs and they have to reboot multiple times during the tests, which takes ~7 minutes EVERY time)

[vaderciya]

I'm learning several coding languages so I won't begin to understand the complexity that is compiling, but I was wondering;

What sort of practical uses will the power switch have? No one has ever taught me about the red and green wires and their uses either. So could you guys list some possible things you could build/make with the power switch and the connectors with or without the red/green cables? The only thing I can think of is to power off/on sections of a network when power is low to conserve energy.

I use to be into redstone and logic gates a few years ago when I played minecraft heavily, if that helps at all.

[Kuro-Maii]

I am seriously wondering why people are suggesting to get "better hardware"...
As I said earlier the project I worked on was 1.5GigaByte of source code.
Think about that for a second. every character you type is a single byte.
We had in the entire source tree that we had written about 1610612736 characters ( = 1.5*1024*1024*1024 )
I do not even want to think about the number of files we had or how many lines of code.

I also said we had to wait 20 minutes at max to compile the whole thing.
The hardware we used where HP office workstations with dual core celeron CPUs in them.
no SSDs as they where yet to be thought of when those computers where bought.
And no, we never used anything fancy like a continues build setup, or a "build farm" of multiple computers/servers.

the server we had for testing and release builds where both virtual with only 8 threads a max.
and that server took the same amount of time to build the software as our work stations did.

still, doing a new feature/fixing a bug when compiling it only took a few seconds.

I am just wondering that and mean no offence to any body.

on the topic of clang and visual studio I do not know how they will interact...
I know that you can use mingw on windows in combination with makefiles just like you do on linux with gcc.

also please keep in mind the the .net framework ( even when you do not use it ) visual studio is asking it to restart every time you run a compile command. and .net is stupidly slow on it's own

[kovarex]

I am seriously wondering why people are suggesting to get "better hardware"...
As I said earlier the project I worked on was 1.5GigaByte of source code.
Think about that for a second. every character you type is a single byte.
We had in the entire source tree that we had written about 1610612736 characters ( = 1.5*1024*1024*1024 )
I do not even want to think about the number of files we had or how many lines of code.

I also said we had to wait 20 minutes at max to compile the whole thing.
The hardware we used where HP office workstations with dual core celeron CPUs in them.
no SSDs as they where yet to be thought of when those computers where bought.
And no, we never used anything fancy like a continues build setup, or a "build farm" of multiple computers/servers.

the server we had for testing and release builds where both virtual with only 8 threads a max.
and that server took the same amount of time to build the software as our work stations did.

still, doing a new feature/fixing a bug when compiling it only took a few seconds.

I am just wondering that and mean no offence to any body.

on the topic of clang and visual studio I do not know how they will interact...
I know that you can use mingw on windows in combination with makefiles just like you do on linux with gcc.

also please keep in mind the the .net framework ( even when you do not use it ) visual studio is asking it to restart every time you run a compile command. and .net is stupidly slow on it's own

1.5GB of source code is really a lot. Factorio has 8.8MB of source code only. But are you talking about C++ or or net? Net/Java already doesn't have to struggle with the inlcude system so their build time is obviously dramatically faster (especially for bigger projects). This is why I believe the C++ module thing could help A LOT.

[Kuro-Maii]

*SNIP*

1.5GB of source code is really a lot. Factorio has 8.8MB of source code only. But are you talking about C++ or or net? Net/Java already doesn't have to struggle with the inlcude system so their build time is obviously dramatically faster (especially for bigger projects). This is why I believe the C++ module thing could help A LOT.

yes this is pure C++. No .net or other microsoft/java/other JIT or framework languages.
We had a 'core' that did all the data interaction, and all the parts that reacted to the staff requesting data and wanting to store new/adjusted data for the patients. as well as talking to other hospitals.
the core was only compiled at release time or when a change was done to it ( I.E. new database, refactor of a table/database ) or the occasional bug fix. this made the processes take 20 minutes.

when making new features/fixing bugs/cleaning legacy code only that part got compiled. and everything else got linked into the final binary.

if you use makefiles make will check to see if the *.cpp/*.hpp files are newer then the respective compiled versions ( *.o ) and only compile those that are. the rest gets linked together. and thus saving stupid amounts of time.
I know for a fact that visual studio just disregards this and compiles any and everything every time you want to test a new feature or see if the bug is gone.

so where as with GNU make and mingw the parts of boost and std that you are actually using do not get recompiled every time, as a fact even they are only compiled again if there is an update in the parts of boost/std that you are using,
visual studio will just not care.

make sure you start setting up your version control to ignore *.o and *.a files and make sure that you are compiling outside your source dir. that bit is important otherwise you will end up with a lot of files that you will need to find the source files in between.
do not move the files out with a copy or move action either because otherwise the files are not there when make is checking for the time stamp and will compile everything regardless.

[ratchetfreak]

Can I use clang in visual studio? From what I'v read, the setup isn't simple.

there is a clang-cl developed for chrome that is a drop in replacement for cl (the windows compiler)

http://clang.llvm.org/docs/UsersManual.html#clang-cl

https://www.youtube.com/watch?v=rXi065XC6zY

[production]

I work at a place that uses an in-house C++ library to re-implement or wrap stdlib and Boost functionality. The original reason for creating the library was to speed compilation times. I wouldn't recommend going down this route.

• A custom replacement is harder to learn for new developers, because Boost / stdlib have a ton of documentation, tutorials, StackOverflow posts, etc.
• A custom replacement is a huge distraction for existing developers. You'll spend a lot of time writing it, then you'll have to maintain it.
• Many times, you'll end up using Boost / stdlib anyway, because when it takes a few seconds to #include exsiting functionality, vs. hours / days to add it to your custom library, developers will tend to take the path of least resistance.

Here are some other suggestions:

• Spend a little money on a decent build server. $1000 - $2000 can get you a very nice core i7 with lots of memory. You are using parallel make (make -j8), right?
• Build automation. Use Jenkins to automatically trigger building and running unit tests whenever a commit is pushed. It saves a lot of time -- and worse yet, losing your train of thought by context switching -- if you know it will be built and tested and you can look at the logs later.
• Refactor your application into components isolated at compile time. Have a separate class / file for implementation and interface. The header which defines MyInterface can just contain class MyImplementation; and should not #include MyImplementation.hpp. This one-line class definition is enough to allow MyInterface to contain a member of type unique_ptr<MyImplementation>. This allows you to get away with only having the implementation's member types included in the implementation compilation unit.

[ssilk]

Well, this is not a real comparison, but when I write software for big data operations (where one run takes 1/2-2 hours - and no, there is no way to speed this up much), I first write the tests. Then the code. And in the tests I try to test this: Is my algorithm working with one of each record. And is it running with many of each? If that runs, in the most cases it runs also in reality.

So my credo here is: first the tests that are possible, and if that runs, you can try it with the whole game-engine.Because the test-code should be not part of the game-code, that spares minimum 50% of compile-time with the nice side-effect of having tests.

(I know: sounds easier as it is...)

[kovarex]

Can I use clang in visual studio? From what I'v read, the setup isn't simple.

there is a clang-cl developed for chrome that is a drop in replacement for cl (the windows compiler)

http://clang.llvm.org/docs/UsersManual.html#clang-cl

https://www.youtube.com/watch?v=rXi065XC6zY

Wow, looks interesting, do you have answers to some of these questions?

• Is there performance comparison? Does it compile faster with clang?
• Where can I get the clang that supports it? Is there a guide win for dummies?
• Is there a simple way to integrate it into visual studio? Did anyone try it?

[Lhoto]

Out of curiosity, will the power switches be able to be labeled so we can tell what we're turning off? (IE: having numerous power switches in a central point to control things or doing some sort of obscure power-based logic could be situations where this would help)