Friday Facts #186 - Marathon testing

[ealrann]

Nah, don't give release date. It's definitely a bad idea (too stressful for a dev team, source of bad coding and more).

However, a good practice is too make frequent builds (I think to Agile Manifesto). So, why don't you give us some kind of Steam Beta Code in the forum, with a big warning : "These builds will crash or corrupt your save". So we could access some staging builds and satisfy our thirst We definitely want to be yours beta/alpha testers , it's ok if the game crash or if you corrupt our save (we would have been warned).

There were a terrible bug when they released 0.13.1 or something that when you place concrete and remove it, it creates water. Lot of people played their saves and destroyed their factories because of it. Yep, now we can remove water, but it was about a year later (they fixed a bug in a day, but the water remains, you had to dl a mod to fix it). Yep, the warning was there, but still 1) people got angry, and 2) well you don't really want to play with a so broken feature.

I definitely believe they learnt from this day, we never had a massive break-gaming bug since this day.

That's why the beta code I propose should only be accessible in the forum, with a big warning.

[suprnova74]

Working in a job where i have to code myself, i can totally understand how bug-hunting can be a necessary but time-consuming affair. Many times I will think something will take x time and hunting that elusive bug, OS quirk, rare condition, etc makes it take x*10. It's seems to follow the Pareto-80/20 rule. 80% of the functionality is written in 20% of the time. 20% of the remaining items feels like it takes 80% of the time (including bug hunting).

Just remember that the people grumbling about it taking too long are merely excited for the new features. You got them hooked

[aober93]

So what?
It's an Experimental release.

Its a commercial game and with this release being advertised there is nothing experimental about this in terms of legal , customer service and customer satisfaction. The devs testing the game before release is just that.

[biliskner25]

[HammerPiano]

Just bear with us for a very short while please, we are doing literally everything we can to make it happen as soon as possible.

I'm not going to cry about how much time we spend, how much effort we put, how close it can get to taxing on both mental and physical health and whatever. We took a big bit of a release to chew this time on, but when you get it, I believe you will trust me that the wait was worth it.

We can already see the light at the end of the LONG tunnel, and the light is very bright. : )

Thank you

V

<3
Your game is awesome, take your time to make it more awesome !!!

[Engimage]

Just bear with us for a very short while please, we are doing literally everything we can to make it happen as soon as possible.

I'm not going to cry about how much time we spend, how much effort we put, how close it can get to taxing on both mental and physical health and whatever. We took a big bit of a release to chew this time on, but when you get it, I believe you will trust me that the wait was worth it.

We can already see the light at the end of the LONG tunnel, and the light is very bright. : )

Thank you

V

We are with you guys. You are awesome. But we are really tired of a journey on this overhyped train.

[aka13]

Now whz I specifically love factorio is because of your attitude towards the playerbase. Even if it says "early-access" on steam, every second of factorio feels like sitting on mission-critical software with 0 bugs. As much as I would love to have 0.15 "as it is" right now, I really do appreciate that every release, even the experimental ones feel solid and bugless. Take your time, devs, we all love you and appreciate your effort. Also, it is such a relief seeing the devs play their own game, knowing that balancing and gameplay will be thought out and not just a pack of ideas. 0.15 is going to be amazing, as every release.

[brunzenstein]

Now whz I specifically love factorio is because of your attitude towards the playerbase. Even if it says "early-access" on steam, every second of factorio feels like sitting on mission-critical software with 0 bugs. As much as I would love to have 0.15 "as it is" right now, I really do appreciate that every release, even the experimental ones feel solid and bugless. Take your time, devs, we all love you and appreciate your effort. Also, it is such a relief seeing the devs play their own game, knowing that balancing and gameplay will be thought out and not just a pack of ideas. 0.15 is going to be amazing, as every release.

I sign that statement in full

[Albrat]

Working in a job where i have to code myself, i can totally understand how bug-hunting can be a necessary but time-consuming affair. Many times I will think something will take x time and hunting that elusive bug, OS quirk, rare condition, etc makes it take x*10. It's seems to follow the Pareto-80/20 rule. 80% of the functionality is written in 20% of the time. 20% of the remaining items feels like it takes 80% of the time (including bug hunting).

Just remember that the people grumbling about it taking too long are merely excited for the new features. You got them hooked

usually at least 10 to the power times longer than you expect.

I am glad we are getting 0.15 , I am really excited for the new version. But I like to play the game as it is as well... I have a version 0.10.x install , 0.11.15 install, 0.12.35 install, 0.13.21 install and a 0.14.22 with mods install. My current videos are made on the 13.21 install of factorio still. I am planning out my 14.22 mods playlist already, but I am thinking of jumping into a 0.15.x series as soon as it is released. I am really looking forward to this release.

BUT!!! I am willing to wait as long as it takes for the development team to be perfectly happy with the 0.15.x version before they release it. I have been part of a lot of early access and Alpha / beta games. (and operating systems) The development process takes years and can result in no actual product available for release at the end of it as well. (been part of alpha's that got abandonned before they even got to beta versions. NB: abandonned some of my own ideas in Alpha as my skills were not good enough to do the game idea justice.)
I am glad that factorio is developing as fast as it is. We sure get alot of game and gameplay for the money we have paid.

[mattj256]

For the following release (0.16), can you make it a smaller release and more quickly?
It's so hard to wait, especially when no one knows exactly how long it will take!
(Keep up the good work.)

[commandoby]

For the following release (0.16), can you make it a smaller release and more quickly?
It's so hard to wait, especially when no one knows exactly how long it will take!
(Keep up the good work.)

The best way to wait is to get distracted by something else and stop waiting. =)

[BenSeidel]

The best way to wait is to get distracted by something else and stop waiting. =)

Unfortunately there is no other game that has captured my attention like Factorio has. Having tried MANY other games, the well over 3000+ hours that I have sunk into this game and I STILL WANT MORE is testament to the massive chasm that separates it from anything else. As I really don't play a map for anything less than 100 hours, this week by week waiting is just destroying me!

[Vinnie_NL]

To everyone here asking for a release date, I sure feel the same but I think it's for the best for the game and all of us if we just let the devs decide when it's ready. Our job is to enjoy the game and suggest possible game improvements, everything else is for the devs. I think we get the best possible game that way, so that should make waiting easier.

I remember reading a comment here stating that this shouldn't be a new Duke Nukem Forever done when it's done situation. But the development of Factorio can't be compared to that *ahum* game. With Duke Nukem we were teased for years that it would be the best game ever, with only sporadically releasing a new trailer but no detailed information whatsoever about the status of the game, over a period of 16 (!) years. In 2011 we get the game, and it didn't wasn't the classic everyone was hoping for. I think everyone has forgotten about it by now.

I haven't been in that many early access games, but I think it's safe to say that the Factorio development team is one of the most open developers there is. They share a new blog every week with not only interesting game info, but also the challenges and the reasons why they choose this release schedule. Maybe there are other companies having a better release schedule for their situation, but they don't have Factorio, the best industrial automation simulation you can think of

[deasturies]

Regarding automated testing: how it is organised from technical point of view ? What technology stack ? How do you assert that all is good ? Does it involve image capture and comparing to a benchmark ?

[Twinsen]

Regarding automated testing: how it is organised from technical point of view ? What technology stack ? How do you assert that all is good ? Does it involve image capture and comparing to a benchmark ?

We write tests in C++ code. We place some entities using code, set them up as we want and then check that they end up in the correct state.
Normally tests run without graphical mode. Entities are created but nothing is ever rendered. We sometimes use graphical mode to see how the test is running and if we placed the entities correctly.

Here's how one of our simpler tests looks like:

Code: Select all

  TEST(AccumulatorCircuitNetworkReadChargeLevel)
  {
    TestScenarioHolder scenario;
    scenario.createArea(10);

    //set accumulator to different charge levels and test if the circuit network output sends the correct precentage
    Accumulator* accumulator = scenario.addEntity<Accumulator>("accumulator", RealPosition(2, 2));
    accumulator->setAvailableEnergy(accumulator->electricEnergySource->getBufferSize());
    ElectricPole* pole = scenario.addEntity<ElectricPole>("medium-electric-pole", RealPosition(4.5, 2.5));
    WireConnectionDefinition other = WireConnectionDefinition(pole, ElectricPole::circuitConnectorID);
    accumulator->getOrCreateCircuitConnector(AccumulatorControlBehavior::circuitConnectorID)->connectTo(&other, Wire::Type::Green);
    dynamic_cast<AccumulatorControlBehavior*>(accumulator->getOrCreateControlBehavior())->outputSignalId = VirtualSignalPrototypeList::get("signal-A")->getID();

    scenario.update();

    CircuitNetwork* network = pole->getOrCreateCircuitConnector(ElectricPole::circuitConnectorID)->getGreenNetwork();
    CHECK_EQUAL(uint32_t(network->getSignalValues().size()), 1u);
    CHECK_EQUAL(network->getValue(VirtualSignalPrototypeList::get("signal-A")->getID()), 100);
    scenario.update();

    accumulator->setAvailableEnergy(0);
    scenario.update();

    CHECK_EQUAL(uint32_t(network->getSignalValues().size()), 0u);
    scenario.update();

    accumulator->setAvailableEnergy(accumulator->electricEnergySource->getBufferSize() / 2);
    scenario.update();

    CHECK_EQUAL(uint32_t(network->getSignalValues().size()), 1u);
    CHECK_EQUAL(network->getValue(VirtualSignalPrototypeList::get("signal-A")->getID()), 50);
    scenario.update();
}

[biliskner25]

We write tests in C++ code. We place some entities using code, set them up as we want and then check that they end up in the correct state.
Normally tests run without graphical mode. Entities are created but nothing is ever rendered. We sometimes use graphical mode to see how the test is running and if we placed the entities correctly.

Factorioception

[KingBenjai]

The way I've been dealing with these delays is throwing myself into bob/angels petrochem multiple times.

[liij]

Please 0.15 tomorrow ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;..............................................; ;..........................................................................; ;........................; ;..................................; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.; ;.;

[Mercury044]

I want to say thank you to the Dev team for making this great game.

i can wait for 0.15 but don't want to as this game has takes so much of my life and yet i still want more. You have created a masterpiece of a game and I love it.

Don't hurt yourselves but please feed my addiction soonest.

Thanks again!

[drmason13]

A word of encouragement for the devs.

Every post complaining about the progress of this release is a fan who really really likes your game!

In fact I have a theory.
The increase in complaints is caused by "factorio withdrawal". A lot of us are trying really hard NOT to play your game so we are fresh and eager for a binge come 0.15. The problem is your game is too good, so not playing it makes us cranky.

I just started a new game yesterday. I'm not putting any pressure on it. I'll quit it when 0.15 arrives. Boy do I feel better for it

On a side note: trees are a pain at the start of the game aren't they? I have an idea for it in this post here:

viewtopic.php?f=6&t=44254