Test Retreat

Welcome to the homepage of Kris Corbus

Archive for the tag “7 questions”

Exploratory Testing: Peer Conference #ET19

16th February 2019 in the FlyWire office, Valencia, Spain: the time and place for a memorable exploratory testing peer conference!

Important things first

  1. The statement of the peer conference was: “It is 35 years since Cem Kaner described and named Exploratory Testing. What has changed? Where are we standing?”
  2. Many of the attendees tweeted during the event. Look for hashtag #ET19
  3. Results, ideas, following material will be collected and published on https://exploratorytesting.org Give us time to fill it.
  4. Following events will be coming. Stay tuned if you want to participate.

Attendees and rules

I love this tweet from Marianne – it shows almost all of #ET19 attendees (few joined later) and the introduction of K-Cards.

We wanted to give voice to everyone and take voice away if it was misused or a person talked too much and didn’t let others to share their ideas. This is why we used K-Cards, which was new tool for almost everyone. During ETC I was sitting next to Elizabeth as she explained the rules to Alex, who during peer conference took uneasy role of facilitator. For the cards and usage of them we need to thank Paul Holland and his wife Karen. I am very happy that I could learn this method. I already started to create my cards and will use some of Marianne’s illustrations as inspiration. So hier are the cards and their explanation:

·         Green: Please place me on the new thread list – NEW IDEA
·         Yellow: Please place me on the same thread list – EXTENSION TO CURRENT TOPIC
·         Red: I must speak now (or important admin issue: e.g.: I can’t hear) – ANNOUNCEMENT
·         Blue: I feel this discussion is becoming (or has become) a rat hole. – NOT ON TRACK

We did not have any cards in blue, so we used orange as you can see in Marianne’s tweet. But I agree with Paul, that colours need to be really bright or neon. K-Cards helped us to stay on the topic and tracking who spoke, helped us to moderate that everyone’s voice gets heard. Surprisingly I was the first and only one whose voice was taken away, even to that point I have spoken only twice. As in every facilitated conversation the moderator has a lot of influence – to go on with yellow cards or green ones, to give or to take the voice. How I reacted on this action was that I started to save my speaking time for green cards, if I remember right I spoke only on 2 or 3 yellow cards and always at the end, when I saw that others are not covering that point. I would suggest to everyone who is used to speak a lot to follow a similar pattern.

Topics & Drawbacks

When we got the invitation to the peer conference, we got a homework as well. To think about our experience and to choose one of the stories we want to present to the group and to have a discussion about. After we met altogether and decided on ground rules (Twitter – yes or no?, does everyone is OK by taking and publishing pictures? etc), we got 2 minutes to write our experience story (one story per person) on post-it. We placed post-its on the board and everyone got 20 seconds to present  the story to the group. This was the first drawback: not everyone took the homework seriously and prepared something or thought about how the group could contribute, or what kind of help they expected from the group. After all presentations where done, every participant got three voting dots. It turned out that the majority of dots got people who we are used to see on the stage. I keep thinking, was it unconscious choice based on speaker or conscious choice of topic. I wanted to capture all offered topics, but after checking my notes, I realised that I did not…

If you want to know which topics has been presented, Marianne’s sketchnotes are irreplaceable!

  1. Teaching Exploratory Testing to Developers by Anne-Marie

2. Mobbing With Intent by Maaret

3. Exploring Unit Tests by James

4. Scaling ET in Dark Scrum Organisations by Eric

5. How To Explain Exploratory Testing In 5 Minutes by me. I am currently working on separate blog post to describe learnings in detail.

6. Microheuristics by Alex

Every presenter got 10 minutes to tell the experience story, we voted for. Clarifying questions from the group was included in those 10 minutes. Some presentations were really good, but it was not clear how we as the group can contribute to the topic. Timer rung that 10 minutes are over, so we started facilitated open session with the K-Cards.

Another drawback from my point of view – no matter whether the statement was clear, we tended to speak too much on “what is exploratory testing” and less on concrete tools and  “what we can do to improve it”. Because it was the first peer conference for many of us and the tools what we used were new, I guess if we would have had another day, we would have overcome these drawbacks with ease as we became aware of them thought the day .


I was amazed by the K-Cards. I was excited about the topic and statement. I appreciate Maaret for organising peer conference and Jokin & his team for being amazing hosts. A big inspiration was Marianne, who not only took sketch notes on the event, but after I revealed in retrospective that I did not feel safe to raise the orange card (sliding away from the topic), she took initiative and used it herself. Another thanks goes to Marianne for supporting my topic to stay on track. I thought I made clear in my introduction I do not want to talk about giving trainings nor that exploratory testing is important and it is important to learn it, but only how to  explain it in 5 minutes. Unfortunately the group did not get it, so Marianne challenged the group to do exactly what I asked them to do.

After the session James came to me and gave a short presentation how he explains exploratory testing for business people (I wanted to know it since I first talked to him about ET & puzzles). James has been giving trainings on exploratory testing for years. You can find his four exercises for teaching ET on his website; they are free to use. He is also the creator of different puzzles. Later that day, James slided a piece of paper to me. On it was his elevator pitch for exploratory testing:

“Systems are weird. Are you looking for trouble? Exploratory Testing can help you to find unexpected truth, about what you really got.”

Another experience exploratory testing trainer Anne-Marie has collected different sources on the topic. If you have not attended her training with robots, put it on your to do list!


After we left the peer conference most of the group went to coffee place or direct to the hotel. But I needed some quiet time to rethink and categorise my thoughts, so I was extremely happy that Jess shared the same necessity. We walked and talked for three hours. We talked about many things, but the most important piece is this: we would never expect that somebody would be able to play violin after several hours show & tell, so why do we expect that from people who wants to learn exploratory testing? I have been thinking about this since then. As a trainer I know how many people are missing basics, many are using terms in context without understanding. I think we need to stop assuming what others know or understand and to start to teach exploratory testing from very beginning. I am looking forward to the article written by Jess.

Next Steps

Inspired by the conferences (ETC & peer) and conversation with Jess, I decided to create a training which explains basics of exploratory testing. I like to use similarity to music – nobody learns to play an instrument in day or two. I used to play flute and every week we had at least two lessons in solfége, what other kids hated. I loved it, because for me it was like mathematics, another subject what I loved. No wonder that one of my favourite books is The Glass Bead Game by Hermann Hesse. With all that in my mind I am playing with thoughts to create a training as solfége,  7 questions just like 7 tones. Typical human voice covers 2-3 octaves, that would be 2-3 exploratory testing sessions, each 1-2 days long. Oh, I already have so much fun!

My First Exploratory Testing Workshop


Today I did something fun!

Already some time my colleagues &co were asking about an exploratory testing workshop. I did not say yes or no. I wanted to do it, but did not have clear idea how?

Then I went to TestBash Utrecht and had a chance to take a part in Jean-Paul Varwijk‘s exploratory workshop. That was mind blowing!!! There is generally this thing about TestBash workshops – they accelerate my skills and boom my motivation. In this one Jean-Paul literally several times said that I have no idea how to explore. Important was that when he said it, it was not like shaming but like exposing one of my unknown unknowns and that’s up to me what I will do with it. Plus, in case I wanted to do something about it, he gave practical tools for it. LOVE IT!

So today, thanks to Jean-Paul, I gave my very first mini exploratory testing workshop. My task was easier, because I work together with my workshop team, and harder, because I realised, it is part of the workshop to expose unknown unknowns.

It turned out to work out very well. We had AHA moments, fun and bonding.

Outline for 45m exploratory workshop:

  • exploring surrounding
  • exploring software with 7 Questions by Jean-Paul at EuroSTAR Conference. Those slides were not a part of workshop in TB NL, but because of limited time and to support the idea I wanted to transfer in my workshop, found them very fitting.
  • re-exploring software with SFDPO

And we ended with de-briefing strategy: PROOF (stands for Past, Results, Obstacles, Outlook, Feelings)

P.S. if you can, go and take a workshop by Jean-Paul. I recommend it. And do not forget to tell me afterwards how did you liked it.

Post Navigation