on extreme programming and testing —>
We can’t just test manually. Since we are changing the program every week, we need to test pretty much everything every week. So if it only took ten percent of a week to test what we write each week, three months in we’d be testing more than 100 percent of the time. We’d have to hire more and more testers, and presumably we have a fixed budget, so we’d have to lay off programmers, and pretty soon all we have left is one programmer and nine testers who can’t keep up with the load.
My current/recent thinking centers around the “increment”, the running tested software which I believe a team should1 produce as soon as possible2 and keep running thereafter.
Look at that right there. I’m asking that the product grow in running tested ready-to-ship code from the very beginning, every day, until we finally stop funding the effort.
I think that when Kent first introduced C3 to the ideas, he did say that we would be “aggressive”. He didn’t mean with each other, he meant as regards going after good things with some fervor.
In subsequent threads, Martijn Meijering said that he felt it connoted “drive” or “striving for excellence”. He went on to remind us that we used to say merciless refactoring, relentless testing, continuous integration, the simplest thing that could possibly work, and to suggest that it was the italicized extreme bits that seem missing from today’s teams.
Excerpts from Ron Jeffries thoughts on Extreme Programming (XP)
My own experience and expectations are:
- Running the tests help me understand how a piece of code works, they should be a go to, providing valuable readable living documentation that shows and proves what the code does.
- They are a contract i.e. that I can easily and confidently make big changes as long as the tests continue to pass, then what was intend is still working, and I can be confident there will be no regressions, no fear!
- They help me write better code, by forcing it to be decoupled and giving me time to play with new code I write.
Flutter says tests fall into three categories:
- Unit
- Widget
- Integration
See the official documentation for more details.
So far we have a few unit and widget tests.
To improve on this I first added widget tests for the SplitView widget that displays content on iPads, as this is a widget we have built from scratch.
Now we need to show and prove the application behaves as expected.
For this I’ve turned to feature tests, which I’ve added as integration tests.
My aim is to follow the Behavioural Driven Development process:
Ta Da
Unit Test
Widget Test
Integration Test
Cucumber style behavioural tests have features that have a number of scenarios and scenarios can have rules. Screenshots can be added for additional clarity.
Next
We move onto Android, starting with the tabbed navigation layout.
BackBurner
Quite a lot still to firm up, injecting test data/adpaters, details on the discovery and formulation of testable specifications, whether to use Gherkin syntax for other tests like widget tests, can we move non-ui specification tests out of the integration tests to improve performance, to mock or not/very much.
Quite a lot still not covered, navigation, adapters to get device details, data access.
Automating the creating of the living documentation (Cucumberjs Report) will be add with the next feature under test.
XP
A starting point
By introducing feature tests we have a good starting point to keep the focus on delivering new functionality.
The details, the discipline will evolve as we add the new features.
We are now moving to test first, outside in. Each new addition will start with a feature test and then move down to widgets and unit tests.
More importantly we are starting to building software we have the confidence to change knowing it still does what was intended and that the checks are automated, quick to run and cover the requirements.
John Ferguson Smart makes the distinction between feature tests that prove the UI and those that prove the applications API’s and logic, I will be exploring this and other questions testing is raising as we progress.
I will post again on testing when I have more to share.
Good luck, be patient, there is a pot of gold at the end, remember what Uncle Bob says:
“The only way to go fast is to go well”.
Hard to test code
Watch Jeremy Fairbanks video on Practical Functional Programming where he looks at code that is hard to test and recommends isolating the code you write in pure functions to simplify testing.
Flutter_Gherkin
The BDD tools for Flutter are maturing so I just had to chose one and work with it, things could change but BDD brings significant benefits and is worth the effort.
My choice was flutter_gherkin because it as it was the closet implementation of the Gherkin syntax used by Cucumber.
I used the new version of flutter_gherkin that is in beta 3.0.0-rc.9, It uses Flutter’s new integration_test framework which Is a breaking change to remove RPC calls by wrapping the app with the tests. It’s much faster and integrates better with external testing tools.
Every time you update or add a feature file then you need to update the generated code files:
flutter pub run build_runner clean
flutter pub run build_runner build
You can run the tests at the command line:
flutter drive --driver=test_driver/integration_test_iphone_driver.dart --target=integration_test/iphone_gherkin_suite_test.dart
On completion you can find a report in json format that can be used with Cucumber reporters.
There are still a few issues/side effects to work on that will be tightened up as the feature tests and package mature e..g. there is an additional warning at the end of the test run, even though the plugin is used:
And some post test exceptions in debug mode when the runner is not stopped.
It appears that ‘context.world.appDriver.waitUntil(’ is not releasing the timer on completion.
Cucumber Html Reporter
Use to generate Cucumber HTML reports with pie charts.
There are some prerequisites to get the reporter running:
Once installed run a command prompt at:
*integration_test\gherkin\reporters\cucumber_html
and run these commands
yarn
node index.js
Note the json output from the feature test needs some formatting for it to work with this reporter, which is something I will integrate when I add the next feature.
If you want to do it manually for now you need to the following:
- Remove additional structure
- Replace all occurrences of
with “
Injecting Providers
To get round this for now I moved the application providers into a file.
We will still need to inject env dependencies like API once we have them.
People
Kent Beck
Quite simply, test-driven development is meant to eliminate fear in application development. While some fear is healthy (often viewed as a conscience that tells programmers to be careful!), the author believes that byproducts of fear include tentative, grumpy, and uncommunicative programmers who are unable to absorb constructive criticism. When programming teams buy into TDD, they immediately see positive results. They eliminate the fear involved in their jobs, and are better equipped to tackle the difficult challenges that face them. TDD eliminates tentative traits, it teaches programmers to communicate, and it encourages team members to seek out criticism However, even the author admits that grumpiness must be worked out individually! In short, the premise behind TDD is that code should be continually tested and refactored. Kent Beck teaches programmers by example, so they can painlessly and dramatically increase the quality of their work.
Dan North
Invented Behavioural Driven Development at Thoughtworks.
Behaviour-driven development is about implementing an application by describing it from the point of view of its stakeholders.
Aslak Hellesøy
Cucumber founder.
Cucumber is a tool that supports BDD, which is a variant of TDD (Test-Driven Development). With BDD, all the tests are customer acceptance tests, written in plain (human) language so that non-technical stakeholders can understand them. Cucumber combines requirements specifications, automated tests and living documentation into a single format called Gherkin, which is just plain English with a little more structure.
The benefits you can get from BDD (if you do it well) is less rework, fewer bugs and more maintainable code. In order to reap those benefits you have to invest some effort in exploring requirements and designing the software to be testable. Exploring requirements typically pays off quicker, while designing testable software tends to pay off in the long run. BDD helps teams discover mistakes quickly, which makes software development more enjoyable and sustainable. – InfoQ
Sound & Vision
Links
- Testing Flutter apps
- Mockito, test mocking package
- Excerpt’s from Ron Jeffries article on Extreme (Programming) Thoughts
- Practical Functional Programming
- Build Flutter with BDD
- Cucumber Html Reporter
- Hellesøy on TDD and BDD
- BDD Books
- Using multiple node versions on a mac
One more thing…
“I’m not a great programmer; I’m just a good programmer with great habits.”
Kent Beck
“When I use a word”, said Humpty Dumpty in a rather scornful tone, “it means just what I want it to mean, neither more nor less”
Lewis Carroll – Through the Looking Glass