Test Retreat

Welcome to the homepage of Kris Corbus

How To Explain #ExploratoryTesting in 15 Minutes

Every other week I explain basics in software testing, one of them is exploratory testing. It depends from group to group, but sometimes I have only 5-10 minutes on the topic. I love challenges! But I am also aware that I am still learning myself. This is why I asked my peers during Exploratory Testing Peer Conference: how to explain exploratory testing in 5 minutes?

How Did I Explain It So Far

Because I give trainings frequently, I experiment a lot with explanations and observe reactions of students. This is what I have used so far:

+ I used to start the topic with question if they are doing exploratory testing, if yes, then how? Sometimes I needed to interrupt and explain that monkey testing a.k.a. klicki bunti (German expression for mindless clicking) is not exploratory testing. Some students felt bad to find out the difference between their approach and real exploratory testing, so I stopped to ask for student’s experience.

+ my favourite example is sightseeing in unknown city and following unexpected events on the way to planned location. Students seem to benefit from it as well. But one thing I am missing in this case – note taking. Now I mostly use sightseeing as light introduction to the subject. Works well if I traveled to unknown city and day before had time for sightseeing, or if students traveled to unknown city and are planing to have time for sightseeing, or simply several people in the group are open-minded travellers.

+ for in-house trainings once I tried to apply exploratory testing on their system, it didn’t work well. I had not enough information what the system should do and I needed time to gather information. Mission impossible if you have only 10 minutes for the whole thing.

+ once I had group of 14 and almost everyone was somehow involved with football. I used it to encourage the group to look for similarities with exploratory testing. It worked very well, but again almost none of students mentioned or thought about note taking. I retired this analogy for explorative testing, but started to use it to explain test levels.

+ job interview is example which is used by my colleague. It works very well for him, because we can use several techniques, note taking inclusive. This piece almost always cracks up the group, because we use exaggerated situations like:
A: can you please tell us, why did you moved to Australia?
B: I always wanted to live aboard. Besides I hade trouble with mafia in Europe, so I thought why not to develop software somewhere far away!

+ in case I used example without note taking, I explained it separately. I some trainings I showed notes, I took for my group during Anne-Marie‘s exploratory testing workshop with BigTrak robots. Because it takes additional time, I prefer to choose examples with note taking.

I always suggest my students to read “Explore it!“, but I definitely needed more ideas what I can use during training!

Peer suggestions

This is what I got suggested during my session. Most of the ideas are still raw. To visualise authors idea from my comment, I used italic and named the author, who suggested it. The one I started to use in my trainings I listed up as the last one.

Exploratory testing as performance

Jokin: I could explain how to play guitar vs I can show how to play guitar.

I love this approach! I use it with something else, not guitar playing, but in our Berlin office we have a guitar and some of my colleagues really play it.

Lightning talks

Rick had idea to present exploratory testing in lightning talk. I cannot imagine how I could use it in my setting, but may be this could be useful for others.

Flow charts

Alex made her own flow chart, it’s always going back to exploratory testing, showing that you have to learn it anyway.

You can read about it more here.

Selective Attention

Maaret reminded about gorilla videos where you are supposed to count the passes so you don’t see the gorilla, and if you see the gorilla you cannot count the passes. In my topic introduction I forgot to mention that I have used this few times and stopped because too many knew the videos already.

unexpected truth

James on the spot sales pitch: “Systems are weird. Are you looking for trouble? ET can help you to find unexpected truth, about what you really got.”

Games

Ash used go-karts a lot with you people; many were excited about them; planned a lot which car to take; interesting things happen when you’re actually in the car. Lisa used Black Stories.

Improv game

Mel: there are so many improv games to use, can get even physical; “What’s next?” I know people who are practising improv, they say it is their life mindset. I understand the idea, but I do not use it, because I am not really into it.

Get the chocolate

Alex: two groups, goal is to get the chocolate on the other side of the room; 1) write down all steps to get the chocolate, 2) allow them to go directly; then put the chocolate to another place; mean but it will stick. I used similar approach for test automation and we have another game which we play to explain agile approach and team dynamics.

Autonomy / mastery / purpose

Anne-Marie: this concept is often appealing to people.

Escape rooms

Mor: describe exploratory testing using escape rooms; looking for riddles, paths.

Don’t teach it

Martin: you have a room full of tired people, don’t teach exploratory testing; just keep referencing to it! When I heard this I started to laugh, because this is exactly how I am building my next training.

testing mathematics

Maaret suggested puzzle with two ways to solve it – scripted and exploratory. Approach is based on children game where you think of one thing, write it down and hide(fold the paper); people can ask question, you can only answer yes/no; either have them write down all questions in advance, or have them ask, hear the answer and formulate the next question based on the answer.

What I am using now

I never have two identical trainings because there are no two identical groups. Testing mathematics has become my favourite way how I explain exploratory testing. I set it in timebox and limit number of questions. After first round I encourage people to analyse how they did and and how they could do it better, I also give feedback which my observations. Then we have the second round where they can apply the learnings. At the end I explain how they can transfer this approach for software testing.

Besides sightseeing and job interviews I started to use another examples. Both were inspired by Mor’s idea of escape rooms – investigative journalism and crime investigation.

In November I will be presenting “How to explain exploratory testing in 10 minutes” at Agile Testing Days. Come and see me explaining it!

Lies About Work

This week my sister was visiting us for big family celebration, that is the reason why this blog post is extremely short. My sister doesn’t work in IT, her topic is marketing and communications. I like to exchange business staff with her, because she lives in different information bubble and has different views. Last week we talked a lot about communication especially about coding the message. We also talked about team motivation and aspects which indicates or lets us to measure it.
Besides everything else, she suggested a book to read: Nine Lies About Work: A Freethinking Leader’s Guide to the Real World

After checking index, I really got curious about the book. Who else wants to join me to read it?

Update:

I think I did not explain enough why I want to read this book. It is not only about the index. Everyone in testing knows that communication has important role in software development, but how often we really try to understand the other side? I remember once I was listening to my colleague complaining about the project manager she had to work with. I could easily understand her frustration because I have worked with the project manager before and suffered myself. This time as an outsider I could see that the PM is visually stressed, I could see that something is wrong. I surprised myself by suggesting my colleague to have lunch with PM and to find out what is going on. Her reply was: “no way!” I am sure if someone had similar suggestion for me, I would respond in the same way.

For me that conversation became a turning point. I started to look for opportunities to build bridges with other people involved in software development. I had very interesting time, getting to know people, things what they do and work problems what they face. Time to time I met difficult people, who were comfortable in their silo or didn’t want/were afraid to open up and to have a conversation. I chose to leave it like this. I told myself that I respect their choice, but in fact I gave up. Since I have a teenager at home, I keep saying to my husband and to myself – in times when it gets harder, we need to double our love, patience and understanding. “Nine Lies About Work” maybe controversial, but it is mainstream book about corporate world. World – which I always tried to ignore. I learned from my mistakes and now I am ready to have a lunch together.

Feeling Accepted

Have you ever felt alone in crowded place? Have you ever felt not fitting or being not welcomed in a group of people? I know that feeling way too well…

When I was one year old I almost died. Doctor made a mistake. Things happen. After I recovered, I needed to learn to eat, to walk etc again. As you can probably imagine my parents went paranoid and overprotective. For example, I did not attend kindergarten, which was something unheard that time.

Time ago I had disagreement with my brother-in-law. My statement was: do not judge people because you don’t know what you don’t know. His was: everyone is judging everyone and it is very naive to pretend that it is not happening. I know that he is right, but still, I dream of living in the world where people will be accepted and not judged by others.

When I stand before group of people, ready to give a talk, training or workshop, they expect specific behaviour from me. Everybody knows how trainer should look like, talk and walk. Some very quickly notice the difference between me and their image of me. Sometimes people are positively surprised and happy, sometimes very disappointed.

You may ask – what my childhoods trauma has to do with me as professional. The answer is – everything. Only at age of 36 I realised how much my life has been affected by incident at the beginning of my life. It shaped the way how I see the world, it shaped the way how I react on people and situations, it shaped how I build relationships with people and it made me so sensitive and vulnerable. All that I use in my daily work as a tester or as a trainer.

To be vulnerable and to live in society sometimes seams as mission impossible. I am protecting myself by wearing my scars on the inside and cool mask on the outside. I am not ready to share my story from the stage or in even in a classroom, I am so thankful that others are braver then me:

We need to share more stories like this. We need to learn not to be afraid and not to hide the scars. We need to learn to accept others with and without scars.

Two weeks ago one student came to me after the training and said: “I was worried upfront about the course, but then I saw you and immediately knew that I will make it.” And I thought – it was worth it to lift up my mask.

Improving Speaking

Both my mentees has delivered their first meet-up talks and now we are talking how to improve their talk delivery. One question what we discussed this week was: how can I lose my nervousness?   

As a speaker and trainer I have gone through it myself. Following two things I learned six years ago in communication training and still use daily. Only later I realised that it supports “7%-38%-55% Rule” defined by Albert Mehrabian.


Roots

Posture, gesture, eye contact etc according to Albert Mehrabian makes up to 55%. When I stand before people, I imagine that I am a tree. Trees never question themselves. Am I good enough? What others will think of me? What if nobody wants to hear this? One thing what we learned in the training was to root. Imagine that your feet have roots. Let them grow in the ground. This simple routine will calm you down and also will help you to hold the posture. 

Voice

Our voice trainer started the first voice lesson with story about babies, who can cry for hours without hurting themselves. If grown ups would try to do the same, they could not hold it for long period of time. We kind of forgot to use the body for voice as babies do. There are many things what we can do to re-learn it. One thing is to learn to use the whole body for voice as opera singers does or to start somewhere simpler: say something and try to locate where your voice sits. Since I speak four languages, the trainer asked me to do it in every language. With big surprise I realised that for every language my voice sits somewhere else. For my native language it was the deepest, around the heart, for German it was the highest, in mouth-jaw level, the rest two – throat. Since then I practice to get all my languages where my native language lives. I also noticed that when I am nervous, my voice climes up. I experimented with it and managed with help of keeping voice deeper to stay calm. Funny thing when I all that explained over video call to my mentee, I realised how squeaky my voice was. Perfectly set for demonstration? No I was nervous, occupied with thoughts and over analysing. I let is go and the second part of session kept my voice close to my heart. This is another image what I use – I try to speak from my heart or not to speak at all.

Practice

The third thing what I use is very simple – practice. Work on your posture, experiment with your voice, practice your talk. Practice, practice, practice. If your story what you want to share sits, if you know what is your next slide, then you can react on things what happens during a talk – mic is not working, slides are not working, laptop starts to reboot… Life happens. You can deal with it, if you practiced. 
Remember – sometimes it is enough if somebody simply talks from the heart.

I Know Everything

– Hey, listen to me! I am the trainer/big name/white middle-aged man – I know everything!
– emm… no, you don’t. And neither do I.

I have arrived in phase where I know nothing, but I have enough confidence to teach others that little knowledge I have.

Since I was little I was very good middle man. I was reading a lot and used to observe things from different perspectives. Later at university my speciality was to explain something my study mates knew much better than I did, but struggled to understand some aspects of it. How can I do that? Very simple – I listen. Not just words what people say, but words which they do not say as well.

I work as a trainer not because I know everything, but because I like to help people. Training and coaching for me, in first place is about empathy.  I need to be able to connect with trainee and to understand her/his journey, to understand their challenges and problems. Only then I will be able to guide them to their next step. Not to my next step in that situation, but their next step.

My trainer moto is:

docendo discimus,

which means: by teaching, we learn.

I really learn while teaching. Listening to people and empathising with them I learn to know their stories. Based on my trainees questions, I start to explore new topics or dig deeper in domains I knew superficially. In my last training I learned a lot about Ireland – did not expect that, but wow – it was so interesting!

Lisa Crispin is excellent role model for learning attitude. She is three book author, has 20+ years of experience in agile teams, international speaker and, and, and… But I never heard another person then Lisa to say so many times phrases like: “I learned today…”, “that’s interesting!”, “I learned it from …”. Lisa helped me to understand that learning is not something what we do. Open mind to people, situations and new ideas is a mindset.

To know Lisa also showed me that it is easy to ask Lisa for a help. Because you know she will not judge you that you don’t know something. I try to use it in trainings I give. I talk about my mistakes, I say that I do not know everything, but I know material very well and I will do my best to help them to find their answers.

 

 

My Story of Test Automation

I use to think I am not technical and very far away from the thing called Test Automation (TA). Then I started to work at this one company, who were having huge quality problems. I asked how automation looks like and found out that they do not have any (yes, even no unit tests) and that they currently work on defining CI pipeline. This is how my operation and test automation journey started. It turned out that even I thought I am not technical enough and knew a bit about TA, others knew less or were not motivated to introduce it to the company.  Next few years I worked closely together with operation team, supported creation of CI pipeline, defined processes and introduced automation.

One of the first thing that I did – I asked developers from each project to explain how the build process works in his/her project. I found out that many developers did not know how it works. I found out that some developers did not know how to build release and were deploying snapshots. Why did I ask developers about something that usually should be done by operations? Because problems do not appear in operations or in development, they appear where everyone should work together. Another thing – we wanted to create general approach and process to enable employee switchover in case of necessity (vacation or illness).

Parallel to building CI pipeline I started to automate. Just as every other beginner I made many, many mistakes on the way. At the beginning, I was too focused on the tool and too less on the mindset about automation. During TestBash Brighton 2015 several other attendees stayed in my hotel, and I remember that one morning  we all came together at the breakfast table. It was very energised and inspiring morning. That breakfast was an eye-opening moment for me. Richard Bradshaw was sitting at the table as well and told me about his Lego automation workshop, I could see immediately how to implement some of his ideas into project I was currently working on. I was sitting there trembling and panicking – is it too late to change our approach? Some of the projects were already automating everything – a test for each acceptance criteria in a user story. Developers were not involved in creation and maintenance of test automation, but hoped that GUI automation will be their safety net and supported customers wish to automate ALL tests. We started to observe problems with testing and some people got very disappointed. I saw it but did not understand where it comes from and how to fix it. TestBash and ongoing conversations with others helped me to see the cause. I started to learn more about mindset and realised that by automating everything, I have build technical debt. We needed test automation strategy!

If you think that this story has happy ending, I have to disappoint you. One thing I have learned in 20 years of IT – it is not about tools, methods and processes. It is about people and people are stubborn. I saw my mistakes, I learned a lesson and of course it was hard for myself to accept that some of my mistakes were so huge and I misled others. But denial or to hide evidence has never been something what I do. I went back to the teams I was working with, explained what I have discovered and what I want to change and… they refused my suggestions! They thought it is OK how it is running. I tried to point out to problems we were having, but they found another reason and explanation. They just wanted to keep to automate everything…

Soon after that all happened, I felt urge to speak about it and make people aware of pitfalls. I started to give workshops for other in-house teams, our project managers and for customers. I had so much fun by doing that that I got the idea to do workshops and trainings full time. Fast forward – here I am, working as full time trainer and guess what I talk the most in my trainings? Yes, mindset! And very little about tools and concrete examples, because I still need to learn a lot and you could have better ideas than I do. This is my happy ending.

If you want to learn more about automation tests, I suggest you to visit evil testers website.

 

Greediness, Ignorance and Incompetence – Are You In?

Current topic: lawsuit between Accenture and Hertz. In the same time, you know, this is not about those two companies. This is about business world meeting software development world and not understanding each other. …and about profit, interest and presumption. Soon after initial news came out, inside information surfaced:  CIO was making money for own pocket by “saving” companies money on project.

If you work in IT projects, sooner or later you will experience something similar. I have stories myself. Customers PM (no IT background) gave us (working Kanban team) introduction into Kanban (20 step waterfall). Customers Purchasing Manager (no IT background) making decision about agile or not agile and our Sales Manager (no agile understanding) kicking me under the table when I tried to explain differences (the customer was big company and they wanted publicity that it is possible for them to do agile). Last one: customer wanted performance tests without paying money for a server. Greediness, ignorance and incompetence – software development projects trinity. On both sides in management level. Yes, management level. In 20 years I have not met developer who would want to code bad software. But I have seen so many people going silent, because nobody listens and nobody wants to know. Of course there are exceptions! In my experience working on own product was the one.

As a trainer I keep seeing people, who lost their hope to do a good job. Sarcasm on: World is not driven by good job, world keeps spinning because of profit. Sarcasm off. Forget about business and software development, look into climate change. This is about society which knows, but does not react.

I love software development, I really do! That is why I became a trainer. I don’t want to be part of trinity movement. I want to explain customers and what is software requirement and why it is a key element in whole project. I want to train software people to question requirements. I want to coach people not to blindly believe that software development method will guarantee project success. Agile works for me, because for me Agile Manifesto and its 12 principles is common sense. Why should I force you to accept it as your common sense?

Choose the one which fits you the best!

    

    

   

The Champion Of 1 Year Dojo Membership

I have chosen my champion (I watched Shazam! with my son on holidays) for paid 1 year Dojo membership. But first some numbers. My tweets got some attention:

People from MinistryOfTesting supported my initiative: my tweets were retweeted by Mark and published in MOT newsletter. Even if tweet engagements looks good, my blog post with instructions how to apply was viewed only 172 times. Now you may guess how many applications did I got.

I got four (4) applications and two emails with support/thank you for initiative. That means that everyone who participated had 25% chances to get paid membership.

Because I have done this before, it was very important to me to be clear how I will choose the winner: “Information, how will you use it and why you deserve it more then others, will help me to choose.” I wanted to be sure that I am giving away for the right person, the one who will really use it. After reading and rereading submissions, I have the feeling that only one submission showed the commitment I was looking for.

I have chosen my champion and it is Mike. Congratulations, Mike! His submission email has 904 words (others between 113 and 148), those words describe his detailed experienced so far in testing(with many links, I suggest you to read his blog) and what he wants to do with paid Dojo membership. This man has set a target and he works towards it.

To others competitors – well done! I was touched that you copied (font size is different then the rest of the email, it means it has to be copied) my name (I get so many emails with my name wrong, that this is really a highlight for me). Keep going! You did not win this time, but there are many other opportunities! Remember to ask for things you want – it may happen that you will get what you asked for.

Approach To Test Automation

Test automation – frequent topic in testing community, but for businesses and developers it is still something new and unknown. This blog post is very simple, high level my lessons learned in the field.

Strategy

Start by choosing your test automation strategy.  Business sees Test Automation as way to save money. The problem is, if you don’t have strategy and simply write 1000 automated tests, then sooner or later your team will become slower and slower. To pick up speed and save money, test automation has to be flexible and lean. Forget about testing-all-the-things, it is not possible and is wasteful. If your system has 1000 automated tests, your team has to maintain 1000 automated tests. Depending how you created your tests and how oft you will change your software – it can be a LOT.

Martin Fowler created great thinking tool: Test Automation Pyramid

Focus on unit tests, because this is the fastest way how developer can find out that her/his new code broke already existing code.

Testing in Agile world

If you are living in agile world, I would suggest you to read “Agile testing” by Janet Gregory & Lisa Crispin. They have great chapter on test automation with thought provoking mind map. I learned from Janet, test automation approach to write e2e tests for epics, integration tests for stories and unit tests for tasks.

Remember: in agile we believe in zero-bug policy. That means as well that all automated test need to be active and results has to be green. all the time.

e2e test Criteria

Test Automation criteria is part of your strategy because it answers the question: What to automate? Typical approach for e2e is to automate:

  1. only standard features
  2. only features with high risk
  3. only most used configurations

Many teams use that criteria to reduce test automation which has grown too big. My suggestion in trainings is – from very beginning automate only really necessary test cases. If you doubt, then don’t automate on e2e level, talk with your team how you could catch it on integration or unit level.

summary

The biggest challenge in Test Automation is not to write a script, but to decide what to automate and what not to automate. You will easily find people who can automate 1000 tests, but not so easy to find people who can create test automation strategy and efficient test automation design. Just like there are many software developers, but not so many software architects.

If you have strategy in place then you will understand how wrong are statements similar to this: “I often write e2e automated tests for bugs raised by customer because if we made that mistake once, we will likely make it again”. We have to learn from bugs raised by customers, we need to understand what went wrong, but we cannot test everything and it is very unprofessional to automate everything. Testers and developers should learn to think profitably.

If you want to learn more, I suggest you to check what Angie Jones wrote about test automation, or better, her free course on test automation strategy.

Update

Few days after I published this post, Lisa, in cooperation with Joe, published amazing collection of Test Automation Models. My suggestion: explore Rob Meaney’s Risk Mapping and Katrina Clokie’s DevOps Bug Filter. Maybe it is something for your team.

Words Has Meanings: Learning

This is conversation between two friends

Kris: I have to tell you a secret… I have big problem with “learning” if we talk about ET (exploratory testing).

Lisa: Hmm… It’s always been taught to me as “learning about our product”. But I am not an ET expert by any means.

Kris: For me “learning” means “to change behaviour”

Lisa: oh, interesting! for me it’s just gathering knowledge

Kris: Gathering information doesn’t include using it. You use information by changing your behaviour (huge excursion in my experience as mother and trainer). But only thing what we want to change is the software. So no real learning on human side.

Lisa: We might learn about features we are missing?

Kris: That is functional or contractual acceptance testing. No learning.

Lisa: So if I do exploratory testing on some feature, and then I realize, before customers can use this feature, they need another capability that we haven’t even thought of before – I didn’t learn something? Or if I find something is really hard to use – I didn’t learn something?

Kris: How do you know that they need something else? Hard to use – do you mean usability? Testing usability?

Lisa: How do I know? Because I’m exploring as a particular persona or role or job, I have a scenario of what I want to accomplish with the app, and I run into a roadblock. The persona is blocked, Lisa has learned that we didn’t provide a necessary capability.

We might be splitting hairs on words and semantics, but exploratory testing is called a process of learning, and it has seems correct to me.

Kris: For me it is very important to clarify what we mean by words. I always liked language (in Latvian we have more then 30 words how we call a mother), but as older I get as more aware I am about layers of language, coded messages and communication in general. People, who name things, make mistakes (just like everybody else). I am not looking for fight… It just feels wrong to call it “learning” if we only gather information.

Lisa: OK

Kris: The way how I understand whole team approach is that if we have situation as you described it, the tester in the team should be able to identify and categorise the problem. Is it functionality, usability or performance? then to decide what following tests should the team run to gather missing information.

Am I aiming too high?

Lisa: IME it works best if the tester collaborates with the team to do all that. We don’t want to be the safety net so that everyone relies on us to point out issues. We want to help teammates prevent the issues from happening in the first place by building shared understanding of features, using good tech practices for code correctness, fast feedback loops with automation, exploratory testing before committing changes…

More a consultant role than doing all the testing for the team.

You know, you have some interesting ideas here, it would be nice to discuss this on the AgileTestingFellow slack.

Kris: Maybe… but I don’t feel there yet. And I didn’t mean that tester is a safety net. What I meant is that tester has this knowledge about testing techniques and approaches and she/he guides the team. This is what I understand with tester as testing coach for the team.

Lisa: Sure but we have to be continually helping everyone else ramp up those skills. If we make all decisions ourselves, we take autonomy about testing away from the rest of the team. Guides, yes, that’s the ticket

Kris: Yes, helping others. But tester should be able to recognise it her-/himself first. Only then she/he can guide others.

Lisa: I like what you say there about recognizing it in ourselves.

 

Post Scriptum

I really, really LIKE how James Lyndsay phrased it:
“Systems are weird. Are you looking for trouble? Exploratory testing can help you to find unexpected truth, about what you really got.”

Exploratory testing is technique to find out, gather information. As a team we will decide later what we want to do with information with gathered. Maybe we will ignore it, because risk is too low, maybe we will use it and make decision based on it. And maybe we will learn out of it, adapt our behaviour and leave the issue in the past. If it will not reappear, then we really can tell: “we learned something!”

Post Navigation