22 July 2017

An example failure

There is a problem I see when people write features. We all know, examples are powerful. They can make an explanation come to life in the mind of a reader. But, I too often see features written with the examples, but no explanations. Let us assume you are teaching someone how powers work in mathematics.

You would first explain what a power is:

Given a number
When I calculate a power of this number
Then it is multiplied by itself the power number of times to get the answer

Then you would insert examples:

Given the number <number>
When I calculate it to the power <power>
Then <number> is multiplied by itself the <power> number of times to get <answer>
Examples: 
| number | power | answer |
| 2           | 2         | 4           |
| 3           | 5         | 243       |

More often than not, the explanation behind the examples is missing:

Given the number <number>
When I calculate it to the power <power>
Then I get the answer <answer>
Examples:
| number | power | answer |
| 2           | 2         | 4           |
| 3           | 5         | 243       |

The latter scenario does not explain the relation between the values in the examples, so how is someone supposed to write something that will work for values other than those given? You can't. Yes, in the earlier example you have to know what multiplication is, but that is a concept that would be explained elsewhere (no teacher would try to teach powers before teaching multiplication).

Remember, examples are concrete representations of a concept, but they are not the explanation of a conept in and of themselves.

At least I didn't admit this publicy where anyone could see it

It started on the testers.io slack. Heather Reid was saying how great it is that she is talking to her developers asking "So what do you want to see from testing?" and getting well thought out and useful answers. I wondered what I'd have said had I been asked, which led to this:
and then this:
The original version of the post was going to be me working through just that, but I had an epiphany whilst writing it and I know full well why I can't articulate it. It is because I don't wish to admit to the stupid (and I use the dictionary meaning here, i.e. lacking in intelligence or common sense) reason as to why I don't want anything from a tester.

It is simply, I'm big-headed and think I don't need the help of a tester because I was/am one.

Not only is this incredibly insulting to my friends and colleagues (to whom I apologise unreservedly), it is demonstrably false (even in the short time I've been developing, our testers has saved me from putting out bugs).

How could I possibly think that?

I believe it is because I don't like not being able to do things. Because I don't like to admit I can't do all the things I used to, I stubbornly try and do all the things I used to. And so, because I used to be a tester, I therefore don't need a tester.

So Angie, the barrier?


I'm a fool.

15 February 2017

A Sense of Community

I won't be attending Testbash Brighton this year. It'll be the first Brighton one I've missed since they started in 2013. I encourage anyone who can get to a Testbash to do so, purely for the number of amazing people you get to meet. Below is a list of those people I have met over 5 Testbashes (4 Brighton and 1 Manchester). I hope this shows the value of these conferences and gets across just how much fun they are and why I'm saddened by not being at Brighton this year.

Rosie Sherry
Danny Dainton
Dan Ashby
Mark Tomlinson
Richard Bradshaw
Heather Reid
Rosie Hamilton
Claire Reckless
James Christie
Vernon Richards
Abby Bangser
Martin Hynie
Ioana Serban
Matt Heusser
Matthew Bretten
Tracy Baxter
Kristine Corbus
Michael Bolton
Gem Hill
James Bach
Karin Basel
Simon Knight
Beren Van Daele
Kim Knup
Emma Keaveny
Lisa Crispin
Vera Gehlen-Baum
Kirstin Kerr
Simon Tomes
Jess Ingrassellino
Cassandra Leung
Marcel Gehlen
Patrick Prill
Dominic Kua
Nicola Owen
Gwen Diagram
David Williams
Tammy Lewis
Deborah Lee
Matt Frampton
Clement Pickering
Colin Piper
Mark Winteringham
Del Dewar
Neil Studd
Gary Shepard
Aine McGovern
Neil McCarthy
Ash Winter
Dan Caseley
Nathan Weller
Alan Richardson
Paul Berry
Rob Lambert
Adam Knight
Anna Baik
Daniel Billing
Chris Simms
John Stephenson
Elizabeth Fiennes
Raul Nicholas
Philip De Rop
Pekka Marjamaki
Huib Schoots

This is obviously not a complete list. I do tend to get a bit... far from sober at these events, so if I have met you and your name is not on here, I'm blaming loud venues and nice beers for your name not appearing.

See you in Manchester?

17 September 2016

Risks of Testability

  What potential problems should you be willing to risk in order to make your application more testable? A testable app is one that allows you to question the application in anyway required and to get the answers to those questions quickly.
  A story.
  We were creating an iPad application and we decided to run some tests through the application user interface. Perhaps the main issue with UI test runners is the speed. With browser tests we are usually able to shortcut to the bit of the application we are interested in, generally by using a URL. We couldn't find an equivalent way of doing this in app without introducing code that is test specific (I added configuration flags that allowed skipping certain views, like log in). One of the developers didn't like this.
  We discussed it and agreed to remove it. On the proviso that it would be looked into further and if a better way could be found we would use it, otherwise, we would revert (which was the first mistake on my part).
  Shockingly, it never got revisited. Every time I brought it up it was "We'll look into it as soon as the work on this feature is complete" and I let it slide (which was my second mistake). I couldn't get them to prioritise testability over features, because despite what is said at the start of a project about the 'ilities', when all the PO sees is a list of features required for MVP to be completed, those 'ilities' are never prioritised (which is a rant for another day).
  So we ended up getting stuck with a test suite that, every test, would take at least 6 seconds (log in uses a six digit pin, and the UI Automation framework could only 'press' a button a second) to get to the bit you are interested in. This meant every test we added put at least six seconds onto the run time of the suite. It led to reluctance to add anything as it wouldn't provide fast enough feedback. It led to the tests not being run locally, because of the time it would take.
  Now here is the question. Is the risk of a user being able to get the app into an unusable state because of our testability code greater than the risk of missing issues because it is too painful to test?

09 September 2016

Repo Reviews

It is very easy to get git repositories into horrible states. Summaries that are too long, having non-descriptive messages, or using the same message a number of commits in a row. Why then can we not fix untidy commits as part of code reviews? It happened at work recently and the main objection seemed to come from the fact that we had the branch pushed to remote (which you have to do to raise the merge request) and couldn't change the history because someone may have used it (in this case, the someone was probably Jenkins). I have a number of arguments against that. Firstly, if you have created a merge request, that implies that there is no more work to be done on that branch. If you change the history of the remote branch that invalidates someone else's work, the question should arise why was the merge request raised in the first place. Secondly, if you communicate that the remote history is going to change, anyone who might still be using it for whatever reason can sort it out themselves. Jenkins can be made to checkout a fresh copy of the repo each time. Yes it takes longer, but is a small price to pay for a clean log.

I use http://chris.beams.io/posts/git-commit/ as my guide to creating good commit messages. Obviously, you'll have your own practices, but why should you not be able to refuse merges if people have not followed them? You can use merge request tools to educate others in how to write good commit messages, they will learn to apply it, and we may end up with readable logs.

24 July 2016

A Hole in the Net

  I happen to think unit testing is the bees knees and is absolutely required in the creation of readable, maintainable working code. However, things can go wrong. Here is something that bit me in the arse not too long ago.

  The code itself was trying to create a hash with a key for every hour in a 24hr clock (the project I was using this for is here if you'd like to know the context). This means I needed a key from 0 up to 23. I know a number of ways of doing this in Ruby, but I settled on using the `times` method. My first mistake was I used the wrong number of times in my unit test. My second and frankly worse mistake, I used this method in my unit test but then used the same code in my implementation. So my unit test was happily passing because my production code was written to it, but it wasn't actually doing what I had intended.

  Fortunately, as I hope we are all aware, unit testing is but one of the many nets we use to catch bugs. I found this one with my exploratory testing net.

P.s. Keen-eyed readers will notice that the unit test code and the implementation code in the project are still the same and will be wondering if I've truly learnt anything from this experience. The reason is, the destination may be the same, but the route was different. Having used a different method to confirm the implementation was correct, I refactored the unit test to be more readable for me.

23 July 2016

Setup Assertions

There is a thing in automated tests, be it unit, integration or UI tests, that annoys me every time I see it. That thing is the use of assertions in your Act/Given sections of your tests. When you do this, all I can think is that you do not have enough confidence in your software that it will actually do what it is supposed to do. The response I have always had from people when I question this behaviour is, "Well, I have to make sure it is in the correct state". And my response is always the same, "There is a way of doing that. It is called testing." If your test is relying on a particular function of your system, why isn't that function tested well enough that you can rely on it working? Stop putting assertions in your tests where they do not belong!