Category Archives: Testing

Instructional Games for Software Testers

Abstract:

The educational world has now recognised that we learn best when we are relaxed, enjoying ourselves, and have the freedom to experiment and learn through making mistakes.

 

This workshop engages the participants in playing, developing and understanding how instructional games will help them, as testers, test leads and test managers, to better comprehend the power of communication, and improve their communication skills.

Test Concept Charades

The first game we will play is ‘Test Concept Charades. This is where we have to explain common testing concepts without using some key words. Sounds simple until you try to explain complex technical concepts in plain language.

Throw in a little competition between teams and this quickly becomes quite fun 🙂

Ten Hundred Words

upgoerfiveThe second game, the ‘Ten Hundred Words‘ game, is based on the #upgoerfive concept explained at https://xkcd.com/1133/.

We use this concept to describe complex technical stuff, although software testing isn’t quite rocket science, by only using words on the Ten Hundred most common words list – by the way thousand is not on the list. This helps us to communicate testing and test concepts effectively, whilst not confusing our audience with technical double-speak (or jargon).

After playing each game the delegates will work on developing the game and creating their own ‘specific’ variant suited to their organisational situation. We will then examine when it is appropriate to use the games, and what the desired outcomes are.

Both games combine to help us learn new and enhanced ways to communicate more effectively with others, and especially with those who may not have the same level of technical understanding that we do.

Afterwards & Takeaways

Back in the office attendees are able to develop and use the games with their colleagues to demonstrate the learning points from the workshop. They will also be able to test their own versions developed in the session and be left with a deeper insight into how to communicate with non-technical people.

Downloads:

PowerPoint Show     Powerpoint Show (2.3mb)

Presented at:

1. Test Management Summit, London. Apr 2016.

References:

  1. Up Goer Five
    The original Up Goer Five concept by Randall Munroe.
  2. Saturn 5 Rocket described in Up Goer Five
    Well worth a visit just to marvel at the ingenuity 🙂
  3. The Up Goer Five text editor
    Give it a go yourself?
  4. The Ten Hundred Words List

EuroSTAR 2016 Discount Code

As a speaker at this year’s EuroSTAR conference, the organisers have sent me a discount code (badge), that ‘colleagues / friends / followers’  or just about anyone, can use to get a 10% discount. It might not seem like much, but every little helps.

ES2016SPKR16

PS. If you were wondering, I do not get anything for promoting this. I do so in the hope that it might help someone 🙂

BCS SIGiST Mentoring Program

BCS SIGiSTI have just joined the BCS SIGiST Mentoring Programfor the coming year, as a mentor of new speakers.

The mentoring program, established by the current BCS SIGiST Programme Secretary, Isabel Evans, is designed to mentor new speakers through the whole conference / seminar speaking ‘process‘. This will take the mentee right from the early steps of developing an abstract, through talk design and preparation, and all the way to speaking on the day.

I am sure that many people would like to share their testing experiences, but might be put off by not knowing; where to start, how to go about submitting, getting a talk together, and then standing up on the day in front of your peers. This can sound quite daunting (apologies if I am making it so), and that is where the mentors are here to help. We are all experienced conference speakers so have a wealth and depth of knowledge to draw from in order to; advise, guide and assist testers who are just starting their speaking journey.

I am in illustrious company, the other mentors being; Dorothy Graham, Mieke Gevers and Julian Harty.

BCS SIGiST New Speaker Mentors

BCS SIGiST New Speaker Mentors

The successful candidates will be speaking at the BCS SIGiST on December 7th this year. I am definitely looking forward to the event and I hope that you are, either as new speaker and mentee, or just as a member of the audience along to give your support 🙂

If you would like to know more, then as a candidate, check out the mentoring program here, or as a potential mentor, please get in touch with the BCS SIGiST Program Secretary here.

DNS Servers – A Lesson Learnt

The internet in our house hasn’t been working well for a while now. It works fine when you are connected, and throughput tests show, stable, high speed upstream and down. But leave your device alone for a while and it may struggle to reconnect. Turn a tablet on next to a laptop, and they both lose the connection. Even the wired connection devices were losing the network.

For those with a short attention span I can report that the problem is now solved. If you wish to know more, and find out how, then please read on.

This problem has been steadily developing. It used to be sporadic and last Thursday it became a permanent situation!  There was no connectivity for all devices for over half an hour. BT were not reporting a fault, and our Home Hub 3 still had the Blue, connected, lights on.

Question MarkI am a tester, so I always ask “What has changed? What made this happen?

Thinking back, on Wednesday I received an e-mail from Amazon asking me to upgrade the software on my old Kindle. We both have 2 old Kindles. I charged them up. Downloaded the new software and updated them. All was fine.

A few days before I had bought 2 new Raspberry Pi 3 computers. These come with Ethernet and on board WiFi. So I tested both. All was fine.

I suppose I have a somewhat more extensive home network than many. We have a BT Broadband Infinity 2 connection and a BT Home Hub 3. The Home Hub is connected into a switch, and also connected is a Netgear Wireless Router. We have a separate wireless router because of the enforced positioning of the BT Home Hub, near the phone socket at a front corner of the house, does not provide a good wireless signal to the whole house, so we have positioned a separate Netgear wireless router more centrally. It is also a much better wireless router, but that is by the by.

BC Network

I am sure that this setup is not actually that unusual. The Home Hub 3 has some limitations, like only having one gigabit connection. Whereas the rest of our network is all gigabit. The only setup change that we have made is to disable the WiFi on the Home Hub 3.

Where we tend to depart from the usual is that we do have a lot of devices connected. Although I suspect that this is in no way extraordinary. This is becoming the norm in the modern connected and ‘always on‘ home. So we have desktop computers, laptops, phones & kindles. There is PC stuff, like printers and NAS. And finally some home entertainment connections; media PC, satellite box, smart TV, & remote control. I also have a number of Raspberry Pi computers. 17 all told. These are not all connected, but have been, and some WiFI dongles that I can use with the Raspberry Pis.

I added this up, and including guests who had recently visited, I came to the grand total of 74 different connections through our Home Hub 3 in the recent past.  This is probably a lot.

TelephoneOn Saturday I bit the bullet and called BT to report the fault!  This is always a painful process. It takes hours on the phone. You are treated like a complete idiot. And as with all 3 calls on Saturday the problem went unresolved. I was actually cut off with all 3 calls. It seems to me that there is a 25 minute time limit on a call. If your problem is not solved in that time you are just cut off. Even though, all 3 times, they had my number, I was not rung back. I call that ‘customer disservice’. You may have a different name for it.

The upshot was that after the line testing, all of the Agents pronounced that my connection was fine. But we still couldn’t view webpages

However, one question all 3 agents asked was “How many devices do you have connected?”  Now I was saying 20, or 30. Actually I should have said 100+. I did wonder why this was relevant.

My last resort was to search the internet – maybe this should have been my first?. There I found a few comments about Home Hub 5 not working properly with a large number of connections, and an obscure link about the quality of the BT DNS servers, and an alternative solution using Google.

I began to piece things together. We have a large number of connections. The actual error we were getting, which I had reported to BT, was ‘DNS_Probe_Finished_No_Internet‘.

Now call me old fashioned, but here were all the clues. The article about the BT DNS servers also mentioned switching to use the Google DNS servers . Unfortunately this setting can’t be changed on the BT Home Hub 3. It is hardwired to use the BT DNS servers. So you have to configure each individual device on your network. Every computer, phone and tablet that can be changed – about 40 devices in all! And this isn’t so straight forward on some Android devices!

Of course, begin a good tester, I changed just one, and waited to see what happened …

About half an hour later, an unchanged tablet failed to connect, whilst my changed laptop kept working. There was evidence of success. So we changed all of the devices that were to hand – 10 so far. And to date we have not had a recurrence of the DNS failure. Yey 🙂

I ran 4 changed devices, sitting next to each other on the kitchen table, to check that everything was still working. They all worked just fine. Previously this had not bee possible.

However, as a good tester, I know it has been less than 24 hours, that this may only be a temporary fix, and that further problems may lie headed. In the meantime though, we will luxuriate in a return to speedy and stable internet access across our many connected devices.

And if anyone from BT ever reads this then here is a personal plea,

Fix your DNS servers, fix your Hubs, update your help desk scripts to cover this topic, stop cutting callers off after 25 minutes, and if you do cut them off, have the courtesy to ring them back!

My Digital Life #12,768

The last few weeks have been interesting, digitally, as a software tester.

Part 1: NHS Kiosks

NHSKioskRecently I went to a local hospital for a check-up, a regular planned visit, so nothing to worry about 🙂

When I arrived I headed to the self-registration check-in kiosk. I entered my details, as I have done many times previously. The terminal seemed to have a problem, and looped back 3 times to the same questions – The dialogue checks all of your details, presented in the form of ‘Yes’, ‘No’ options for changes. I pressed ‘No‘ to all of the change options, but after three times around I was presented with the message:

An unexpected error has occurred. Please use one of the Kiosks either side.

I was in a rush, being tight on time for the appointment, so didn’t get my camera phone out and take a picture, which is usual practice when encountering an error like this in the wild. I was also a little in shock. As a tester my mind was racing. So the terminal either side might have a better chance of success? Really? And what was actually wrong? An error message might be helpful, so I don’t make the same mistake again – although I couldn’t work out what I had done wrong. But settled on the thought that making a helpful suggestion might actually be, err . . . . helpful. Maybe ‘Use another Kiosk‘ was actually the most helpful suggestion?

I was successful in using the kiosk to the left (there wasn’t one to the right!)  And made a note not to use that specific kiosk again. Maybe it didn’t like me. Maybe I pressed the screen too hard or something?

Part2: Google Chrome

ChromeI really do like Chrome as a browser and have used it for many years. Yes I know that Google are (allegedly) stealing all of my personal information for their own gain, but Chrome works well, is smooth, looks good, and just, err, works.

Well in the last few days it hasn’t. Worked that well that is. About a week ago, subconsciously I noticed that my laptop battery life had taken a dip. I seemed to be recharging it more frequently. It was low on power when I thought it would be ok. That sort of thing. After about 4 or 5 days I heard the fan kicking in and wondered what was going on.

First stop is Task Manager to see what is actually consuming all of the resource. It turned out to be Chrome, and an .exe called nalc64.exe. Hitting 30% plus cpu, when Chrome was loaded.

So problem identified. Now for the cause? A quick internet search identified nothing of use – there were lots of sites offering me a ‘download‘ tool to fix my problem, ‘Yeah, Right!‘.

I resorted to my backup in these cases, which is to use another browser. IE and Edge seemed fine (btw I am using Win 10.) I quickly imported my latest Chrome favorites and continued as before. The hope being that Google will identify the problem and sort it quickly.
(I also ran a full virus and malware scan, just in case, even though these run in the background as a matter of course.)

I checked every day, and four days after I stopped using Chrome the problem is magically fixed. My personal view is 50/50 that: either Google messed something up and have now fixed it, or Microsoft changed something, which messed up Google, which Google have now fixed.

Part 3: GlowBugs

GlowBugContinuing my adventure with CodeBug, I bought some GlowBugs to connect to my CodeBug. “GlowBugs are full colour LEDs (with over 16 million possible colours!). They are really easy to control and use in your projects and this activity will show you how to use them with a couple of simple blocks.” – CodeBug

Programming the CodeBug is immense fun. Try it sometime. So the GlowBugs must be just as fun?  I wired up the 10 GlowBugs, each connected with 3 croc clip wires.  Then downloaded a simple program to find out how to address the LEDs.

I then started to write my own program, a simple little thing, that would light up the LEDs in sequence, depending upon which CodeBug button I had pressed. Some 4 hours later, when I had ironed out the subtleties of Blockly, we – for Chris had joined in the fun – had a working version of the program.

Along the way I encountered a bit of a problem with web browser, Chrome, running Blockly. At one point all of my blocks just disappeared. My code went AWOL! Fortunately I had saved frequently. I was also getting spurious results from counter settings.

I put this down to the fact that I hadn’t refreshed my browser for a while, and that there was a memory corruption, of some sort that Testers don’t understand. Fortunately, when I reloaded the browser the code had been saved to my account, was still there, and worked just fine.

It was like computing in the 1980s and 1990s. Save a little and often. So, if you think the browser has corrupted, then don’t carry on, but reload from your checkpoint.

ZX81My first computer was a ZX81, way back in . . . 1981. a 1K computer, with a 16K ram pack on the back, secured with blu-tack. Plugged into a portable 14″ TV, and a cassette recorder, for saving and loading programs. I had been coding away for several hours one afternoon. It had gone dark. So someone else came along to turn on the light. Unfortunately I had unplugged the light to connect, ZX81, Tape Recorder, and portable TV. Well the inevitable happened – bang – the TV went off, the ZX81 stopped dead, and my code disappeared into the ether, never to be seen again. Only back then there wasn’t an always on, instant Internet, Cloud Save option available . . . . .  I learn’t that lesson the hard way.

It is that time of year again – Hints and Tips

Yes, it’s Conference Submission Time.

cycleThere is a cycle to conferences. For the larger ones, i.e. EuroSTAR, ATD, etc., then it may even start before the last conference has taken place. The conference chair needs to be appointed. The venue and dates chosen – conference halls have to be booked well in advance.

When last year’s conference finished next years will be introduced. The new chair will set a theme, and post a call for papers.  This is going on now for the EuroSTAR 2016 conference (Call for submissions closes on 5th Feb 2016).

TipHere are some Hints and Tips for submitting to conferences.

I was lucky enough to be on the EuroSTAR 2011 program committee and I wrote a post about tips for submitting here. The current EuroSTAR program team have done the same so don’t forget to check it out if you are considering submitting.

There were 460+ submissions that year, 2011, and only 50 or so successful candidates. It isn’t as simple as saying “Well that is a one in nine chance, so I will put in nine submissions!”  Carefully read through all of the advice, then craft your proposal.  And remember, if you don’t submit, then you definitely won’t be successful.

If you aren’t successful in winning a speaking slot at your chosen conference, then do not despair. There are loads of things that you can do, for free, in the interim, to a) develop yourself, and b) increase your chances of submission success for a future conference.

Here is my How to educate yourself in Software Testing for Free plan. Try it for a year, and get your ‘money back’ if it doesn’t work.

There is loads of free software testing stuff out there on the internet just waiting for you, so take advantage of it.

BlogBlogs:  Lots of software testers blog – you are reading one now – so check them out. I wish it was as simple as telling you which ones to read. It isn’t that easy. Follow the links to interesting blog articles. The bloggers you like will become favourites. But don’t forget that there are also interesting posts from people you may not have heard of.
Tip: Try reading blog posts that you disagree with. It may not change your mind, but you will get another perspective.

PowerPoint ShowPresentations: Amazingly, almost all of the conference presentations can be found online. You can see the slides, read papers, and supporting material. Start by browsing the Resources section of this site 🙂  So you still get to read the presentations even if you didn’t go to the conference.
Tip: Check out Twitter for announcements of posts and uploads.

ebooke-books:  A lot of software testing authors write shorter e-books which are freely available. It doesn’t take long to search for them. Surprisingly, some of them are actually worth reading!
Tip: Try reading them on your daily commute – as long as you don’t drive. Or put away one hour a week to read e-books.

WebinarWebinars: Which are online presentations. These are great. I have done several myself which you can find in the Resources section. Basically you get to hear the presenter as well as see their slides. This adds an extra dimension to the presentation, and can give you further detail not available from just reading the slides.
Tip: Many of the major conferences arrange webinars throughout the year.

YouTube2Live Days: Some conferences will actually broadcast a live day. They select presentations throughout the day that give a positive representation of the conference. These may even be videoed which will give you a real feel for the event.
Tip:  You don’t always have to listen live. They are sometimes available after the event for later viewing. Great for conferences in different time zones.

Armed with all of this information, if nothing else, you will have gained a wider understanding of what is going on around you in the software testing world, and you may have learned a few things on the way.

2015 Some of my Favourite Things

And some of my favourite things in 2015, again (as last year), presented in no particular order other than that in which I thought them up. These are the things that have influenced me this year . . . .

  1. CodeBugCB Detail
    In early October I bought a CodeBug, and you can read about my first week’s experience here. I have since used the CodeBug in a session to make a Fruit Keyboard, following the simple on-line tutorial. You can read about that experience here.
    So why is CodeBug on my 2015 list?
    Well, it is a complete environment. It is very easy to get going and achieve something. Google’s Blockly, which is used for the programming language, is very easy to learn, and use, and it is fun.
    And the CodeBug has provided me with the most immediate and tactile feedback from any computer programming experience, and I have been coding a long time.
    I currently have a CodeBug and Colour Tail attached to a Raspberry Pi and these are providing our Christmas lights this year 🙂 Read more and watch a video of the Christmas lights here.

  2. Raspberry Pi 2

    Raspberry Pi 2

    Raspberry Pi 2
    The first thing I did when I got my Raspberry Pi 2 was write a short program to count up to 100 million. A completely unscientific performance test. Details can be found here.The speed increase from a single to a quad core processor, the extra memory (512mb to 1024mb), and all round smoothness, makes the Raspberry Pi 2 a joy to use.  Yes it is 18 times slower than my desktop, an intel i3770K, but it doesn’t feel much slower in many things. And in all things Pi it is much quicker, like making things and plugging them into the GPIO pins.
    I have a Pi 2 dedicated as a home web server. This is connected to the new 7″ touch screen which has significantly reduced the keyboard footprint on my desktop.
    We also have Jesse, the latest version of the operating system, which is another improvement. And this month, December, the Pi Zero has arrived. This has caused a lot of controversy. A genuine £4 computer. Yes, it is a compromise, but at £4, one that is very easy to live with, and has amazing maker or tinkerer potential. I expect to build a few devices based around this in the new year.


  3. MQTTMQTT
    MQTT is a fantastically simple M2M (machine to machine) connectivity protocol, used as an extremely lightweight publish / subscribe messaging transport. It is ideal for Internet of Things (IOT) devices. I have been using an MQTT service to connect to and remotely control some of my IOT devices.
    I have been building some Home Automation devices; temperature sensors, power switches, intruder sensors, etc,  and MQTT is proving an excellent way of messaging between devices.
    Using a Raspberry Pi I was able to generate a maximum of around 3,000 messages a minute, both sent and received. [The actual ‘in-use‘ message volumes generated by the devices are in the 10s (tens) per hour.]
    The capability is fantastic. To do this on the internet 10 years ago was hard work. 20 years ago it was science-fiction. Now it is becoming part of the IOT tool-kit.


  4. Bugs in Windows 10Win10
    Hands up if you are using Windows 10 and haven’t yet found a bug?  Don’t worry, you soon will.
    Now, don’t get me wrong, I like Windows 10. I absolutely hated Win 8.0 and the ineffectual patch 8.1. I still look back fondly to Win 7.0.  But if you are really asking, what was wrong with 98, and 3.11?  Yes, I know these days that computers have much more power, are doing vast amounts more behind the scenes, and are giving us far greater capabilities. But if Hubble can still run on 1980s technology; New Horizons went to Pluto on 1990s technology; I am just asking?
    NHZChange though, is inevitable, so onwards we go.
    However, my experience with Windows 10 is that it is quite buggy. Many of the bugs appear to be superficial, but my machine is crashing fairly regularly!  Once or twice a month since deployment in late July. And that isn’t including the nightmare that was the download and install process, which seemed to have just one or two deployment bugs.
    Typical problems include; the need to reboot after automatic updates, having to reinstall manufacturer drivers after automatic updates, the Login screen background changes size from standard full screen to small, the mouse changes from left-hand to right-hand orientation on login screen which can be puzzling until you work out what is going on, Netflix stopped serving higher definition downloads because screen scaling was not minimised – Now this might be a Netflix bug, but hey, its a bug, in Win 10.
    What worries me is that as a software tester I know that if you find bugs in areas of a system, (Updates, Login Screen, Display Scaling) then if you look more deeply, you are likely to find even more. I know there are more bugs in there – I just don’t want to find them!
    If you are having negative thoughts about Win 10, and this is swaying you, then think again. It is way better than 8.0. And Win 10 will only get better as the patches and improvements arrive. Just be prepared for things not being quite as stable as Win 7.0.
    And just why is this a favourite? Well, I am a software tester, and finding bugs is fun 🙂


  5. MS Text To SpeechMSTTT
    Earlier in the year I was writing something, probably a conference abstract, which I wanted to be correct. I know that when you review your own written work you never quite see all of the errors, but I wanted to do a better job. Bouncing around in the back of my mind was the knowledge that windows could do speech recognition, having used this a couple of times but never successfully, and also that it could read text, but I had never used that facility.
    I did a quick search, and found links to MS Text to Speech. Within 2 minutes I had added an icon to the quick access tool bar in MS Word. When I select some text in a word document, and click on the Speak icon, the text is read out to me.

    It was amazing. The previously self-reviewed and ok text, when read back to me, contained three glaring errors . Brilliant I thought. I then added the icon into PowerPoint, Excel, and Outlook. And I now use it almost all of the time to listen to what I have just written.
    Yes, it takes a little bit longer, because I still reread everything I write, but the overhead is worth it for the increase in quality of the output. (And yes, I used it to review this blog entry – that is called dogfooding!)
    A note of caution though: there are one or two amusing pronunciations, but the overall quality of the spoken text is amazingly high. Try it the next time you write an e-mail. You won’t be disappointed.


  6. Tweet Cam

    Tweet Cam

    Making an IOT Device
    I co-presented a workshop and discussion session earlier in the year on Testing the Internet of Things . . . I thought it would be useful to actually have an Internet of Things (IOT) device to help people understand what one was, how to build one, and how to go about testing one.

    So I built a Raspberry Pi Tweet Cam. It took 6 hours from start of build to working device. You can read about the Tweet Cam that I built here.
    I know it looks a bit ‘Heath Robinsonish‘, but that is only because it doesn’t have a fancy case. I wanted to show all of the wires to help the delegates understand how the device had been constructed.
    The only problem that we had, a minor one, was that on day 1 the Internet was not available. We were in a basement room and there was no phone signal.
    No Internet = No Tweet Cam !!!
    It worked perfectly on day 2, was fun to use, and the output can be seen here.


  7. Solar Impulse 2

    SI2 Landing in Hawaii

    Solar Impulse 2
    If you haven’t seen or heard about Solar Impulse 2, the solar powered plane that is flying around the world, then do please check it out at solarimpulse.com.I was able to follow much of the non-stop 6 day journey across the Pacific ocean from Japan to Hawaii. I thought the achievement was fantastic, even if it did break the plane, which is currently being repaired for the remainder of the round-the-world journey.
    I was also amazed at the amount of real-time information that was being streamed ‘live‘ from the plane, including HD video.
    I was so influenced it got me thinking about software testing and measures & metrics. As a result I did some work on software testing measures & metrics hierarchies, and ran a workshop at the London Test Management Forum, details of which can be found here.


I hope 2015 also gave you some favourite moments, and that 2016 is equally inspirational.

Interesting error #967

I found an interesting error the other day. Not one I was expecting, and definitely one I hadn’t tested for!

CBCTThis year, in a fit of minimalism our only Christmas decoration (apart from cards) was a CodeBug Colour Tail, tethered to a Raspberry Pi.

The CodeBug played the part of tree and lights.

And I wrote a lights program a day, in python, until I settled upon one that I liked.

on running for longer periods I noticed that the Colour Tail would stop dead. A couple of times the Pi had crashed as well. I obviously had a bug in my python code. I checked it. Rewrote the code. Rebuilt the Pi image. Same problem – random crashes.

It settled into a routine of failing, randomly, any time from a minute after starting, to 12 hours later. But would definitely fail at least once a day, and often more.

That is until 4 days ago. Since when the Colour Tail program will now run all day long, until I close down the Pi.

LED LampSo what has changed? I haven’t altered anything for days. I did though notice, on the first day the program ran error free, that one of our LED ceiling lights had half-blown.

Since then the CodeBug Colour Tail has run perfectly.

I am not saying that the LED light was the cause. It could be that the after 2 weeks of running the Colour Tail has ‘burnt in‘, but it does seem mighty coincidental. Reading about electrical interference from LEDs and noting that there was less than 6’ between bulb and CodeBug / Colour Tail, I concluded that there was a strong possibility I had found the cause.

So I have now added a new item to my ever lengthening ‘Testing Checklist‘. LED Lighting?

Chocolate Testing

Last weekend, as I write, I attended a small and social gathering of testers. On the premise that it was a birthday party, we had planned some ‘fun‘ activities.

We met in an Oxbridge town, and spent the day in the garden room of a pleasant pub, discussing software testing, as you do. Well, more precisely, mostly amusing ourselves with games and activities, with a leaning towards software testing.

PaperPlaneWhilst everyone was gathering, we made paper aeroplanes. This was fun for all but one. Only because when thrown, each plane seemed to deviate towards, and then hit them. And it didn’t seem to matter where the planes were thrown from, or by whom. There must have been a draft? And secretly I think they enjoyed it anyway.

We were told a story. I must admit to being quite shocked by the story 🙂  We all got to play a part, as in the best stories, and it must have been good, because you know when everything else fades out and you only hear the story teller’s voice . . . maybe we were being hypnotised?

Fruit Keyboard

To energise ourselves we made a Fruit Keyboard. We used a CodeBug. Four fruits, from Sainsbury’s. 5 Croc-clip wires. And coded online in Blockly. Yey. It worked. And the squeal when someone touched the orange and the word “Orange” was displayed on the CodeBug 5*5 LED screen was priceless. (I like to collect these ‘Oh Wow‘ magic moments.)

Why should all the fun stuff with computers be for kids? That is my question.

Then the testers amongst us tried to see how many people could be included in the fruit resistance loop. And went on to find a complex timing issue, in a 4 fruit keyboard! But no-one asked why there were only 4 keys on the keyboard? To illustrate its short-comings the next sentence is written on a fruit keyboard. “Apple Orange Pear Banana Banana Banana Pear Orange.

After lunch we played Test Automation Tool Charades – try it sometime. Full credit goes to the chair, who bravely battled through, whilst both sides unashamedly cheated their way forward. To describe the competitive spirit would not do it justice. Let’s just say I ended up on the winning team, without cheating, and I am rubbish at charades. I think this game was actually called ‘Testing the rules‘, and we were unwittingly taking part in an observational study . . . . .

We followed that up with a go at describing test concepts using the Ten Hundred words list #upgoerfive.  If you haven’t heard of this before, it is the concept of describing things only using the 1,000 most popular words in the English language. You will need to check, Up-Goer Five, the permitted words, and the online editor.

These were some of our submissions:

  • Defect Management
    The tracking, control and fixing of problems found in computers.
  • Multiple Condition Coverage
    Way of coming up with seeing many ‘either/or’ cases.

Up Goer Five:

We tried very hard to make this real. The game helps to show things we all know and understand, using different, but easily understood words. I think it helps us see how someone who doesn’t know the idea we are trying to explain will first act. (#UpGoerFive)

Replaces:

We tried very hard to make this meaningful. It is a useful and challenging exercise to try and define things we all know and understand using different and non-technical language. I think it helps us see how someone who doesn’t understand the concept we are trying to explain will initially react.

Draw your own conclusions, but definitely give it a go.

photo 6After Birthday Cake we then undertook Chocolate Testing. The premise was that one person (A) believed that they could tell the difference between ‘good‘ or ‘expensive‘ chocolate and ‘cheap‘ or ‘nasty‘ chocolate. And the other person (B) was convinced that under blind testing, sorry tasting, they would not be able to.

For the test a few bars of chocolate had been procured. The original plan was for 4 – 6 bars. This somehow became 22 bars.

So, 7 people, 22 bars of chocolate, 6 evaluation Criteria, gives nearly 924 tests to perform. And then a master score to be aggregated per person for each chocolate. All tasting to be carried out using a blindfold!  We quickly realised that this wasn’t going to happen in the next 15 minutes, so being good testers we decided to sample. Knowing that the most fun bit was actually blindfolding and spoon feeding the victim(s).

The results. I can officially declare that both A and B can tell the difference between good/expensive chocolate and nasty/cheap chocolate. And that both A and B agree that good chocolate is better than nasty chocolate – no surprise there then? For A this was just confirmation. For B this was genuinely a surprise. B will now be buying nice chocolate!

Please note: This result only holds firm for the limited sample of the 22 chocolate types above.

And on that bombshell the day ended.

GoldFishTo close I will share a note about the room. It was a Garden room. So in effect a brick conservatory. Back into the Pub there was a glass partition, ceiling to floor, with glass doors in it. This was fine when we started as there was no one else in the pub. But come lunchtime, the pub became quite busy. It was like testing in a goldfish bowl. Definitely an experience.

 

 

 

 

A Measures & Metrics Hierarchy?

Abstract:

This session has been inspired by following the journey of Solar Impulse 2 (SI2), an aeroplane powered only by solar power, as it circumnavigates the globe.

In June this year SI2 flew non-stop 5,663km from Japan to Hawaii. It took 6 days, using only solar power. The pilot sleeping in 20 minute naps, flying above and around weather systems, whilst travelling at around 30 knots, between 4,000 and 30,000 feet in altitude.

SI2-s

SI2 Landing in Hawaii

It sounds remarkable now describing the feat which smashed all solar powered flight records.

What was even more amazing was as the plane flew it transmitted real time data back to the control centre in Monaco, and the information was then published, live and real time, on the internet at solarimpulse.com.

This got me thinking. Firstly, wouldn’t it be amazing if our software testing projects could broadcast live, real-time metrics and measures just like Solar Impulse? And this must be easier for us to do because we are not high over the middle of the Pacific Ocean, linked only by a sat phone?

Images used by kind permission of SolarImpulse.com

Secondly, of the 14 key measures and metrics that were being displayed, some were fundamental flight and safety related. Always good in an aeroplane to know your; altitude, speed, and attitude. Some were flight plan related, in terms of direction and timing. And others were more inspirational, such as a view of the overall world circumnavigation. Not needed on a second by second basis, but giving overall context to the journey.

And that was when I had the thought that where we often struggle when producing software testing measures and metrics is that we mix up the safety with the inspirational, the planning with the performance.  Which may be why some people are against measurement, because they are stuck in a loop of safety measurement driven by fear. And others think that measurement and metrics can only be achieved as a complete end-to-end program of works, putting off all but the keenest of measurement disciples.

What software testers may find helpful is a hierarchy of software testing measures and metrics, along the Maslow lines of; Physiological, Safety, Belonging, Esteem, and Self-Actualisation.

Maslow

Maslow’s hierarchy of needs – Wikipedia

Knowing which measures address which needs on our projects may help save us from; life in the dark with no measures, producing the wrong ones at the wrong time, or being driven by measures and metrics rather than using them as tools to help us test software.

That could give us a new appreciation of what each of our favourite measures and metrics, S-Curve, Bug Count, Velocity, DDP, Coverage, Defect Heat Map, Burn Down Charts, <insert your favourite measure here>, etc., is achieving for us on the Maslow Hierarchy. And maybe an understanding of what each measure or metric we use is actually doing for us, where we may be misusing them, or where we may wish to use them in the future

Of course this may not work. But I think it is better to try and understand what our software testing measures and metrics are doing for us, than to either blindly generate screeds of impenetrable charts and numbers, or worse still, not produce any at all!

Downloads:

 PowerPoint Show    PowerPoint show (6.6mb)

Presented at:

1. UK TMF, London, Oct 2015.

UK TMF Session Outputs:

Below are the output flips from the session and the resultant hierarchy.

PNG  Flip 1     PNG  Flip 2    pdf  Hierarchy

References:

https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs
http://badgerscroft.com/home/7-key-measures-for-software-testing/
http://www.solarimpulse.com/