Factorio Forums

Feaelin wrote: ↑

Sat Jun 19, 2021 2:46 pm

If you're finding yourself wanting to "manage" which test suites are being run together, that may signal that the implementation code isn't decomposed optimally.

Well, high-performance code in C++ is not a Java, where you can tolerate unnecessary indirections... Sure, you can use static polymorphism to inject mocks of the lower-layer, but it makes code a bit ugly. And those mocks aren't free to setup either.

So, I think devs made a good trade-off here. Code is still testable, you just need to "hide" errors from higher-level tests if you notice errors from the lower-level ones.

refactoring is generally where you change the behaviour of the interface without modifying the interface itself and redesign would be when you change interfaces in a way that must be accounted for with backwards compatibility shims / warnings.. or immediate deprecation, depending on the level of exposure to end-users/integrations.

bormand wrote: ↑

Sat Jun 19, 2021 4:52 am

So, I don't believe into the "extract methods until you can't do it anymore" mantra. Finding a good balance is still an art.

The way I do it is just write the code as long as it is clear, but as soon as a line or block looks complicated and might need a comment, I extract that bit to a method or a function, with a name that summarises the comment I would have made. In other words I refactor for clarity, not just because I can. If that leaves me with lots of three line functions that is great. If it leaves me with a longish function that is a clean and understandable rendition of a single algorithm, that is also great. Usually I get a mixture.

NPE wrote: ↑

Sun Jun 20, 2021 4:43 am

In other words I refactor for clarity, not just because I can.

Yep. That's the point.

I understand why Robert attempts to find a formal, mechanical way to measure clarity. And it would be cool to make an IDE plugin which can take a pile of trash, apply some formal rules to it, and transform it into the clean and readable candy. But I doubt such a metric can be invented...

bormand wrote: ↑

Sun Jun 20, 2021 12:41 pm

NPE wrote: ↑

Sun Jun 20, 2021 4:43 am

In other words I refactor for clarity, not just because I can.

Yep. That's the point.

I understand why Robert attempts to find a formal, mechanical way to measure clarity. And it would be cool to make an IDE plugin which can take a pile of trash, apply some formal rules to it, and transform it into the clean and readable candy. But I doubt such a metric can be invented...

most of the things Robert says, his own example code doesn't even demonstrate: https://qntm.org/clean

ptx0 wrote: ↑

Sun Jun 20, 2021 5:17 pm

most of the things Robert says, his own example code doesn't even demonstrate

I know... I read that book. And the only way to understand example about prime numbers was to inline all those three-liners-full-of-unexpected-side-effects into a single function.

P.S. And the best function there is a freaking isPrime() that pushes a number into the array as a side-effect. What a division between "commands" and "functions", what a referential transparency...

From my own experience and the experience of my colleagues and the whole company I’m working I can say this:
Be careful by just overtaking uncle bobs tips without thinking or without understanding why it is useful.

For example: Clean code is nice, but functional programming is nicer. But that depends highly on what you’re doing, what tools you’re using, what language you use…. For example I currently use mostly typescript and programming here fully functional is such a relief. The first year I hated these “one-liners that tells whole stories”, but now I love it because it reduces all the things you can make errors with.

Uh, the idea of this test dependency is interesting. I will keep that in my mind, but I think, this is already used in other projects, but not under this name and not often, because it is not needed in most projects - I had one project two years ago, where I possibly could use that. But it would have been a waste of time, because it would have been useful only for a month or so. And to be honest what you describe with test dependency sound a bit like an anti-pattern. Because it points to a problem, that two systems are so tightly connected to each other, that they are one big system. If you repeat that, it becomes a big ball of mud.

I think that’s not the case here.

In my last project we also came to a point, where I argued “making this mock really working well will introduce more possible errors”. So we tried to distinct between “real” unit tests, that tried to make unit tests with “good enough mocks” and integration tests, that integrates more tests. Which is - when looking exactly- nothing else as test dependency, but much smaller.

ssilk wrote: ↑

Mon Jun 21, 2021 8:38 am

Because it points to a problem, that two systems are so tightly connected to each other, that they are one big system.

As far as I can see from @kovarex article, they still have a nice separation and layering. They just don't want to go for "true" unit testing due to overhead, probably. In Java or TS indirections are basically free (well, you just can't get rid of them), so you get attachment points for your mocks for free. In C++ "true" unit testing is always a trade-off between performance and readability, detaching upper layers from lower layers to swap them for the mocks isn't free.

kovarex wrote: For example, lets say that we have a test of electric poles connecting properly on the map. But I can hardly test it when I don't know that searching for entities on the map works properly.

Well, you can, at least in theory.

Replace searching function with a mock. Now you can write test cases in the following style: "force the mock to return an inserter here and a pole there, call the algorithm, assert that result is right". The test would be completely decoupled from the map and search stuff.

Or try to split that algorithm into a pure function (that accepts a list of entities that we see around the cursor) and a wrapper (that does the search and calls that pure function). Pure functions are really pleasant to test. And the wrapper can be covered by a single case.

Hi,
Kovarex, you said that you're having difficulty with Mocks (paraphrasing). I understand the frustration as they are about as hard or probably harder than TDD. But, once you get your "AHA" moment they become a really good tool to have in your toolbox (and one that you should be using as often as possible). I implore you to keep trying to figure them out, to figure out when to use them and most importantly, when not to.

tldr; Mocking Good

Mocking Advantages

The biggest advantage of using mocks is it FORCES decoupling of code, because:
1) The mocked object has to be passed into the test function, either explicitly as a parameter or as a member variable if testing a method. So no static references to the "new" function (or other mechanism for getting the function or object instance).
2) The mocked object has to be an interface: You can't mock through extension.

Using mocks also allows your unit tests to double as integration tests! A whole suite of tests written FOR FREE! This happens when good Mock is FULLY REPLACEABLE with the main implementation. Pass in your mock for Unit Testing. Pass in your full implementation for Integration Testing.
Mocks also double as free Specification Based Unit Test, allowing you to check that your EXPECTED behaviour - as documented by your mock object, is your actual behaviour supplied by the main implementation. Because you're implementing interfaces, it's trivial to implement a Comparitor Mock class that takes two instances of the mocked interface (the mock and your real implementation) and simply delegates each function call to the two instances, comparing their results resulting in free Specification Unit Tests!

How to Mock

Something that isn't really talked about in the literature on Mocks is that a good Mock has to have the minimum implementation to support the test it's written for. For example, a mocked "find entity" function call should always return the one entity the test requires, regardless of parameters passed in (this is not the place to test if the parameters are correctly used). In this way, you can run a "simple" test that you can easily step through, debug and see what is going on, as opposed to having complex entity finding systems that you have to contend with.

I 'd really like to hammer home how important this point is. This is the point where most people "stuff it up". They think "I need to mock system X". So they start coding a "mock system X", implementing a whole bunch of methods, storing and updating state, etc.

!!!THIS IS WRONG!!!

It's perfectly fine to have an array of results for a function call, with each subsequent call incrementing a counter and returning the next item out of the list. This is even true when those results ENCODE the side-effects of calling the other functions that your test will be calling. This may sound brittle, but for mocks - brittle is good! You want to capture ANY behavioural changes. You want you Mocks to DOCUMENT the expected behaviour.

eg.
Mock:
List.Get(1) => returns Results.next()
where Results is an iterator (see popular language) containing {1,2,1}

Test:
AssertEq(get(1), 1)
insert(0)
AssertEq(get(1), 2)
delete(0)
AssertEq(get(1), 1)

The test passes as if the mock is fully functional implementation.
Remember: your tests are !!NOT!! testing the mocked code.

Mocking & TDD

The last key idea of writing good Mocks is that they go hand-in-hand with TDD. I think of them as two sides of the same coin:
TDD -> You implement your code incrementally as you implement the test
Mocks -> You implement your mock incrementally as you implement your tests.
TDD & Mocks -> You implement your mocks, tests and code in lock-step increments.

If you understand how to do TDD, then extend your 30-second loop to include the mock!

Finally

If you are finding that your Mocks are causing excessive amounts of clutter, then the most likely cause is that you're trying to re-use your Mocks for multiple Tests. I have found this to be bad practice, the same way that sharing test data in tests is bad (I hope that this is common knowledge). Each test should have it's own copy of it's test data and it's own copy of any mocks that it relies on. I hope from the example above you can see why this is required.

Please be aware that I am NOT talking about mocking a database. While database mocking is an important thing to do, these mocks are less of a mock and more of a standard, well known set of data that whole-system testing can use. I personally don't like that they are called mocks - it causes confusion when trying to discuss this topic.

Thanks,
Ben Seidel.


P.S.
Another really good FFF, Kovarex. Thanks for taking the time to write that all up.

LOL, I haven't played factorio in over a year but I could not resist replying when I saw the topic. And I do plan to return to the game soon.

I have been designing applications for decades and my version of "The only way to go fast, is to go well!" is "Slow down! Because I am in a hurry!". It is counter-intuitive but so true.

In my case I got lucky and early in my career I came across a project where I was the only developer, the product was a prototype/"proof of concept" we'd received and that management in their "wisdom" had decided to use in real life. So, at first they'd come to me about 3 times a month and say "I need you to make this change, in max 3 days, or we'll lose the chance at a $300million transaction."

It meant working continuously for at least 48 hours (no sleep) to get it done. Also, my management was extremely against making more changes than necessary, so anything that I did to improve the base code had to be snuck in without them noticing.

The magic formula I came up with was that if I could make the change a little more generic but only add maybe 10-20% of time and effort I would go for it. And little by little it reduced the number of changes needed for new transactions because my recently enhanced code would already handle more and more of the new ideas they'd come up with. A year later I'd be making just one change for every 30 new types of transactions and after showing to my management that my changes directly earned them about $15mil in net profits I got the biggest raise of my life.

This of course set me on a lifetime path of finding better ways of doing things. At every job and project I'd improve the reusable framework, classes and design patterns I'd use. By now about 90% of any of my new projects code is already written and extremely stable, apps rarely break (although my main job is writing new applications, I also do tier 1,2 and 3 support for well over 50 apps but since they almost never break I don't mind), when a problem happens the apps automatically email me (or preferably the people that support whatever caused the break - like a network or database failure) so that issues are fixed before they are even reported and quite often apps allow users to self-support and fix their own issues, and most importantly... although my bosses think I am working like crazy I really made it so that I can handle the workload with very little effort. In other words, I worked hard to make it so that I could later enjoy a very lazy lifestyle


Moral of the story:

Yes, pausing to make bug fixes and refactoring is good, and certainly better than the typical approaches done by developers.

But you can go one step further and make continuous bug fixing, refactoring and anticipating issues part of the habits and culture. Just put a tiny bit of extra effort each time you fix something with an eye on making things easier later, and before you know it... you'll have fewer things breaking and existing code becoming easier and easier to work on.

Thanks for the posting! I shared it with many of my coworkers at my fintech company, and there was a strong show of gratitude for it. Always great to see the struggle towards progress rather than just the end result.

So I happen to have have watched GiantGrantGames do his 24hours, all starcraft missions twitch stream this weekend (before reading this FFF).
And in the stream they mentioned a saying from the Navy Seals which I feel is quite similar to "The only way to go fast, is to go well!".
That saying is "Slow is smooth and smooth is fast".
Found it interesting that I encountered both of these in such a relatively small amount of time.
Guess life is trying to teach me an important lesson

FrodoOf9Fingers wrote: ↑

Mon Jun 21, 2021 3:55 pm

Thanks for the posting! I shared it with many of my coworkers at my fintech company, and there was a strong show of gratitude for it. Always great to see the struggle towards progress rather than just the end result.

I think a more accurate interpretation is progress isn't always a good thing

argbla wrote: ↑

Tue Jun 22, 2021 5:22 am

I think a more accurate interpretation is progress isn't always a good thing

Progress towards a good goal is always good, no?

Tips for you ...

Hello kovarex,
This is my first time reading Friday Facts. On the one hand, because English is simply difficult, I have no talent for languages. Second, because I didn't know what could be in it . I'm an avid gamer, Factorio is the game of games for me. However, I've never been a good player
So I'm a legacy code specialist. I am extremely pleased that you are making the right decisions here. I am called when a team is desperate about its own code, when a small bug regularly lasts for several days, when features are as complex as a new program and still in the wrong place, generated duplicate code and ignored existing interfaces, etc. Just before the whole program is buried, then I come to the train.
I have already encountered dozens of functions / methods that were over 40,000 lines in size. Or classes over 700,000 lines
All of the scenarios described are simple truths of a developer’s life. I myself also constantly step into the traps, although I have always paid full attention to these issues.
However, through my almost 30 years of experience, I have developed / discovered a few tips and behavior tools here and there that will help me and hopefully you too.
The scenarios you describe are quite normal. Even to myself it happens with guarantee over time that I will also create code like this. Decreasing sharply for 4-5 years. And from it the first tips, exclusively for you . I determined the tips myself and I am sure that others have already mentioned them.
1. As soon as every (!) Developer sees something that he can improve, which is wrong or simply ugly in the old code -> improve immediately!
a. Immediately means in the same week. So that's how I do it.
i. Develop for 4 days and write down everything I see
ii. On Friday the day is reserved to do this refactoring, I say refactoring because I want to encourage people to do it right, i.e. to delete files and code from time to time, so that something better is created.
2. Almost from the start, refactoring 20% ​​of the weekly workload.
a. So
i. Rework classes and interfaces,
ii. make it better and easier to use,
iii. document here, in the head of the code, in such a way that one imagines that a colleague from another area should write an extension here in this code and that you make the instructions and assistance easily accessible to him here in the comment.
b. All of that on Friday. Otherwise, Friday afternoon is always the team game afternoon / evening - if I can decide that.
3. If Friday is not enough to include the points in the next sprint planning.
Now a tip for the style, so that the structure of the code can be read and assessed more professionally and faster:
- K&R Style (Resharper setting)
- No blank lines
- 42 inch monitors (at least two) with 4K resolution
Why:
- My daughter is currently learning to read. Your books have a large font, lots of spacing and only two or three sentences per page before you have to turn the pages.
- This is helpful for beginners. The complete overview is not exactly important to the beginner - he is caught in the details.
- As a journalist - I was a radio journalist about two years ago thirty years ago - you read a lot. And fast. Sometimes I can do 20 times as much as a normal reader.
- In the 1950s, when their customers were high-volume readers, the big newspapers adapted their font and style to accommodate these high and high-volume readers. I know that the New York Times and the NZZ Neue Zürcher Zeitung did that. In the meantime they have switched again because they now have different readers than they did then.
- At the time, the changeover was scientifically supported and researched on it. It is based on the visual visual way we read. We don't actually read letters, but word, sentence and even paragraph images.
- With its structural properties, the program code is excellently suited to processing the figuratively oriented reading of letters and symbols.
- Now in addition to uniform rules that every editor and program language offers, this visual reading and recognition can still be massively promoted:
o Well then: As a programmer, I feel like a professional reader, like an old-school journalist. Code that looks like a first grade read-study book makes me tremble.
o It is important to design the code in such a way that the structured text is optimally used:
 The high structural fidelity of a program code goes far beyond newspaper articles. The code has many structures that are excellent for visual processing.
 These are to be used. This means,
 Methods one page long (well-known rule)
 No blank lines, the structure makes it easy for us to see that something new is beginning. (K&R Style)
 Beginning brace at the end of the line.
 It is important to read and recognize as much as possible at a glance (= picture).
 The effect is that the superordinate structure becomes visually visible -> already when reading the code.
o Regularly consult and work with automatically generated graphics of class hierarchies.
And the last thing to remember
The constant refactoring of the newly emerging code is absolutely decisive.
So nothing beats constantly rolling refactoring refactoring. !!!

If you want, I can also check your code, for free of course.

When worlds collide! Two of my favorite things. Factorio and TDD

I couldn't help myself but jump in here and offer some encouragement to keep exploring the topic. I've been using TDD as one of my primary techniques for over 6 years. It still teaches me things. I also thought I'd share a little thought from a little further down the road and things that come up a lot.

Unit

The Unit in unit testing is almost always scoped to a class, but this causes a lot of frustration for people starting out. It is far more helpful to think of a Unit as the smallest grouping of code that is testable. So in the post where it is noted that it is hard to test class A when it uses two others, the Unit, in this case, is likely all three classes together.

Now, some people are going to have feelings about this because they really want their classes and designs to be independent and loosely coupled. TDD has a funny way of showing how coupled things are. One of the great benefits over time is how TDD begins to reveal some of these subtleties in code design. Either way, if your unit is 20 classes, that's fine. It's where you start.

The Loop

TDD is also described as red, green, refactor. You write one test and see it fail. Failing can be compilation errors, your IDE screaming at you, runtime problems, or an actual failing test. Then you get it to pass as simply as possible. Don't jump ahead to what you think the solution is. Take baby steps towards it. When you have green tests you look at your code AND your tests to see if they can be cleaned up. Refactoring tests is a whole thing. Some tests will stop being useful or can be more expressive.

Yes, this loop sounds tedious and ridiculous, but as you gain skill in it, it winds up taking seconds or minutes. The little loop does a funny thing of giving lots of little rewards along the way instead of huge ones after getting through some gnarly change.

Mocking

Okay, so this one is a hugely contentious topic within the TDD community. What I'll say is that mocking, like TDD, is a unique skill to develop. The same goes for its alternative which is to not mock. A highly developed mocking skill allows you to test your code rapidly while preserving difficult implementations. Poor mocking leaves you testing mocks and having no idea if your code works because the mock has intercepted it.

The bottom line, is to be patient, try multiple ways out.

Someone will, at this point say, "WHAT ABOUT THE DATABASE OR NETWORK LAYER?" I very rarely mock those either. Now before people assume my tests are slow, brittle, or any of that, let me explain. See, a common problem with network and database code is that it isn't isolated. It might be abstracted, but it is rarely isolated. That means that there are expensive integrations everywhere, so mocking takes that pain away. Alternatively, I accept the pain full-on and use it to force very clear isolation so that I only have expensive tests in ONE place. This goes back to that unit thing above.

Existing Code

Code that was written without TDD is very different than code written with TDD. So when you're starting to do TDD in an established codebase it can be pretty depressing. So what I tend to recommend here is that if there are un-tested parts of the code but you want to do TDD you have to live in two worlds. First, you have to add tests to that existing code, and it'll probably be hard to do because of how it was designed. Then, when you are doing new stuff, start to do TDD there. As the older code gets tests behind it, you have room to refactor and move a bit more freely. This, for many groups is a long process, but start one or two tests at a time.

Independence

A lot of people talk about test independence in various literature and often say something like, "The tests need to be independent from the code" or something like that. What test independence is actually about is tests need to be independent of each other. I'm bringing this up because I suspect from the post there might be something brewing here.

The ideal is that you run any test in any order and the results are valid and consistent. Most test runners support this through randomizing test order. Now where things often go wrong is in their implementation. If I can run any test in any order, then each test is able to set up what it needs and clean up after itself. This is where so many problems are. Specifically in the clean-up piece.

What happens if tests aren't independent? Well, it means that when there is a failure you no longer know if it is from the code or the tests themselves. In other words, dependent tests double the debug efforts as now each test that ran before the failing one might have put the system in a condition that caused the failure.

For folks starting out, I typically advocate for two guardrails for the beginning. The first is to ensure the tests are run in a random order, the second is to run them 2-3 times back-to-back. This does a pretty good job of getting confidence around test independence.

Awkwardness

Trying anything new is awkward at first. A lot of people use the awkwardness to justify why TDD doesn't work. I'd recommend pushing past that or doing a small trivial project where you aren't encumbered to give it a fair shake. The other awkwardness comes in when things are just super hard to test. Listen to that awkwardness. It almost always means there is an improved design to what you're working on.

When you look at TDD from a code-design lens, one of those perspectives is that your test code is the first client of your code. In fact, you're declaring the contract between clients through those tests. So when you go to write a test, and its super clumsy, it'll likely be awkward and clumsy to use that same code when you aren't testing.

In other words, that, "This is too slow and hard" feeling is your gut giving you a signal about where some design changes are buried.

Anywho, that's all I'll write for now. If the makers of this fine game ever want to explore any of this more, I'll make time to help.

I love you guys. Thanks for the article. It's sooo relevant and actual for me.
Greetings from Hungary,
a fellow Software Engineer

If I was any more experienced in C++ than the short overview I had during my education, I'd be happy to apply for a company with such a mindset. I wish more companies put that much, or could afford to put that much focus on quality and extendability. It always makes me happy to read about your programming practices, and to see them improving over time to become even better. In the gaming industry, you guys are the odd ducks around, I don't think many studios put so much care in their code (which shows in bugs and stuff breaking when new DLC release etc... Which you guys just manage to fix in day if it even happens at all!).

While I was getting impatient to hear some news about what you're up to, what the new update will bring and such, much like everyone else I imagine, I'm quite satisfied that the time was spent on this. This seems like a wise investment, and hopefully indicates that Factorio will have a long lifespan, well beyond the coming expansion. I'm very excited about that idea

All the best, and thanks for being so diligent where many are not!

Oh, very tasteful.


[Moderated by Koub : Off topic]

BlueTemplar wrote: ↑

Sat Jun 19, 2021 11:00 am

NotRexButCaesar wrote: ↑

Fri Jun 18, 2021 10:32 pm

BlueTemplar wrote: ↑

Fri Jun 18, 2021 10:30 pm

Luxalpa wrote: ↑

Fri Jun 18, 2021 9:57 pm

[...]
Functional Reactive Programming for their UI code (and things like the building code), as these complex state interactions / asynchronous actions are exactly the kind of use case for this paradigm.
[...]

So, what is FRP ?

Here is the wikipedia article: https://en.wikipedia.org/wiki/Functiona ... rogramming.

But they *already* seem to be using asynchronous signals and events, so why does Luxalpa thinks that they aren't using FRP ? ("Too many" classes, "not enough" functions ?)
And is C++ even the right language to do FRP in ?

The game generally, yes, sort of, but not in the UI code. UI code, especially, as Kovarex noted, forms with lots of interactive elements (and I have personal experience with the map preview generator, so I know exactly what he's talking about) tend to become exponentially (or maybe it's just quadratic) more complicated with each new element you add. The reason being that when you add a new interactive element, you need to add logic for what happens when the user changes the value (which might change the state of all the /other/ elements on the form!), and also logic to update it for every other change that might happen.

I don't have experience with FRP, but my understanding of the concept is that it simplifies things by having each interactive element, rather than having to procedurally update the rest of the form, just update the backing data model, and then the state of the entire form is regenerated *by one piece of code* based on that data model. So instead of hundreds of ad-hoc update-Y-and-Z-because-X-changed functions, you just have one update-data-model-because-X-changed functions for each input X (whether that's an interactive element on the form, or something external that might change the data), and a rebuild-the-form-from-the-data function.

It sounds like using lambdas instead of methods for all those helped a bit. I suspect that FRP would be a more fundamental restructuring and probably make things even simpler.

I think React has some built-in features to make that business of rebuilding things efficient. Not sure how easy that would be to adapt to C++.

We have the exact same problem with e.g. the user profile update GUI at the treadmill company where I work now. I hope to use an FRP approach to refactor some of it sometime and then hopefully I'll actually know what I'm talking about.