My Story of Test Automation

I use to think I am not technical and very far away from the thing called Test Automation (TA). Then I started to work at this one company, who were having huge quality problems. I asked how automation looks like and found out that they do not have any (yes, even no unit tests) and that they currently work on defining CI pipeline. This is how my operation and test automation journey started. It turned out that even I thought I am not technical enough and knew a bit about TA, others knew less or were not motivated to introduce it to the company.  Next few years I worked closely together with operation team, supported creation of CI pipeline, defined processes and introduced automation.

One of the first thing that I did – I asked developers from each project to explain how the build process works in his/her project. I found out that many developers did not know how it works. I found out that some developers did not know how to build release and were deploying snapshots. Why did I ask developers about something that usually should be done by operations? Because problems do not appear in operations or in development, they appear where everyone should work together. Another thing – we wanted to create general approach and process to enable employee switchover in case of necessity (vacation or illness).

Parallel to building CI pipeline I started to automate. Just as every other beginner I made many, many mistakes on the way. At the beginning, I was too focused on the tool and too less on the mindset about automation. During TestBash Brighton 2015 several other attendees stayed in my hotel, and I remember that one morning  we all came together at the breakfast table. It was very energised and inspiring morning. That breakfast was an eye-opening moment for me. Richard Bradshaw was sitting at the table as well and told me about his Lego automation workshop, I could see immediately how to implement some of his ideas into project I was currently working on. I was sitting there trembling and panicking – is it too late to change our approach? Some of the projects were already automating everything – a test for each acceptance criteria in a user story. Developers were not involved in creation and maintenance of test automation, but hoped that GUI automation will be their safety net and supported customers wish to automate ALL tests. We started to observe problems with testing and some people got very disappointed. I saw it but did not understand where it comes from and how to fix it. TestBash and ongoing conversations with others helped me to see the cause. I started to learn more about mindset and realised that by automating everything, I have build technical debt. We needed test automation strategy!

If you think that this story has happy ending, I have to disappoint you. One thing I have learned in 20 years of IT – it is not about tools, methods and processes. It is about people and people are stubborn. I saw my mistakes, I learned a lesson and of course it was hard for myself to accept that some of my mistakes were so huge and I misled others. But denial or to hide evidence has never been something what I do. I went back to the teams I was working with, explained what I have discovered and what I want to change and… they refused my suggestions! They thought it is OK how it is running. I tried to point out to problems we were having, but they found another reason and explanation. They just wanted to keep to automate everything…

Soon after that all happened, I felt urge to speak about it and make people aware of pitfalls. I started to give workshops for other in-house teams, our project managers and for customers. I had so much fun by doing that that I got the idea to do workshops and trainings full time. Fast forward – here I am, working as full time trainer and guess what I talk the most in my trainings? Yes, mindset! And very little about tools and concrete examples, because I still need to learn a lot and you could have better ideas than I do. This is my happy ending.

If you want to learn more about automation tests, I suggest you to visit evil testers website.

 

How To Create A Workshop: Follow-up

screen-shot-2017-01-16-at-01-29-02

End of the last year I wrote about starting to create my first testing workshop. This is follow-up with details how I did it and results after the workshop.

Ash’s MindMap

As I know Ash and loved his workshop in TestBash Brighton, I took his mindmap as a proved recipe. It helped me to tune it and to have an overview. Especially helpful was everything under “Principles”. Small adjustments from my side – I added a node “Emails/Communication” for the “Checklists” section.

Creating Mental Model

With map in the hands, first thing what I did, was to create a mental model of the workshop. I spent more than week to build it. Very classy with pen and paper – my two favorite thinking tools. Some of the ideas what went through my head during that time: why company asked me to do a workshop? What kind of problems do they want to solve? What are possible solutions? What would be my solution? What background has people who will attend the workshop? What kind of skills they have? What kind of skills they need to have? What is the main, core thing to be able to solve their problem? How can I teach it? Can I teach it? How to build it up in small stages/ containers?

Some answers I knew, for the rest I decided for two possible versions – positive and negative. As a next step – I simulated dialogs with me and workshop students, where the students where unwilling to learn, disagreeing with my ideas or frame of work, not cooperating and simply not understanding what I say because of different background and level of information about software testing. May be it sounds silly, but it helps me to prepare and to deal with potential situations. With every simulation I check my model – does it work? Should I improve something?

Test The Workshop

When the mental model is done and tuned, than I started to test it. First I tested it with two of my colleagues. One of them has test management background, never did test automation before and shortly started to learn it. She liked some of my ideas, but her main comment about the subject in general was: it is so difficult and it is not possible to learn in one day; people will block them out because of amount of information.

That gave me a lot of thoughts. I went through my mental model again, cut all advanced level stuff, noted that I need slides for basics and changed the structure.

After I was done, I looked for people outside my company with whom to test the idea. Got very valuable feedback and some tips.

Only then I started to create the slides. When slides were done, I gave a short presentation of it to my colleague, who said that it is not possible to learn TA in one day. My heart singed, when she said, that now I build a good workshop.

Outline

So what was my plan? I started with fusion of “Test Automation with Lego” by Richard and “Mob Testing” by Maaret. The goal was to write script how to build a tower and they had to write it in the team, where one was a driver and everyone should had a posibility to be a driver. After that we “run” the scripts. Next step – to analyse the scripts and think about modularity. First learning block we would end with a little theory about testing and test automation, just to assure that we are on the same page. Before the lunch I wanted to start practical scripting in the teams, which we would continue in the afternoon. I planned to end the day with second theory part with lessons learned and suggestions and to wrap it all with another mob-test-automation-lego to acknowledge what we learned.

Results

How it went? Like in the life – unpredictable :) Lego as an ice-breaker and illustration of test automation worked very well. Light problem was with mob testing – drivers did not want to give away their positions. All my effort went unheard. I stopped pushing and paid attention that everyone is envolved even if as contributer.

The idea to analyze the scripts and try to modulyse them did not work, because scripts did not work. I switched to back up and we analsed one of their test cases.

My light theory part raised a lot of questions and took much more time as I planned. That is why before the lunch we managed only to share the files and set up for practical part.

As we started practical part, team required to change the planned setting and instead of team work they wanted to connect one laptop to the beamer and have a shared understanding how to start scripting. I agreed and than problems started… first two students could not connect their laptops to the beamer, with the third it worked, but again we lost the time. After basics were covered I asked to join their groups. Mob part we skipped, because it just did not work for them. The technical problems kept following. One students keyboard did not work. I suggested to get an external keyboard, but it was ignored. A second student in that team had too old browser version and did not have admin rights to update it. I suggested several times to change the teams but it was ignored as well. On another team driver had a mac book and he missed some files as we diagnosed afterward. I have to be honest – I was not prepared for any of those situations. Who comes to the workshop about test automation with broken keyboard?! But I think the thing which disturbed me the most was – that was OK for them!

At one point it stopped to be an organized workshop in teams. Everyone, except one, tried to script the test case on their laptops, shoutting around, sharing excitment and frustration. I was running from one to another and helping to fix problems. For a moment I saw one blinking word in my mind. Disaster. I had to skip second theory part and lego wrap up, all students wanted to make their scripts to work. We used every minute till the end of the workshop. And you know what? They did!

That moment I switched off that blinking word in my mind. The technical problems were OK for my students, they hit the goals of the workshop and all of them were smiling when they let the workshop. And yes, before we cleaned up, it looked like a workshop. Mission accomplished!

Summary

I am glad that I had a chance to create (in less than a month) and run a workshop. I am happy how it turned out. I realized how spoiled I am with technology at work (new keyboard or mouse is never a question). The students said it was interesting and usefull.

What I will do differently next time? I will put more effort to make sure that there is less as possible technical issues. I heard it before, but did not really understand until experienced myself. Another thing – Andrei has a list of skills what he expects from his students and if they do not cover part of it, he refuses to give a workshop. It sounded a little bit overacted as I talked to him about it, but now I think it would be good to draw a line, to set up basic knowledge level for participants.