Lego Customer Story
Shock and awe…
Allow me to set the stage…at this point we had a relatively young test practice for our e-commerce website and UI automation was steadily improving over time. The team was doing a great job within the boundaries they were given. We were given tools and we used them…not bad overall but not stellar either. Our e-commerce team just finished a difficult upgrade to our ecommerce platform. It was so challenging that we had to put our sugar plum dreams of a responsive web design on the shelf. It was just depressing I tell you…frustrated and somewhat defeated, we were asked to go to London for a project meeting.
We weren't sure what to expect of the project meeting and quickly discovered we were being schooled. A vendor was acquired to take charge and 'help us' move forward with a completely new site design and responsive web design implementation. This is the part where everyone gasps and says 'You just called my baby ugly!' It's true, we all know it whether we care to admit it or not. The reality is we weren't getting it done and the vendor was hired to execute the design and teach us their techniques in the process. It stung at the time but it was the most efficient way to remove our boundaries and catapult us into a brighter, geekier future.
I am going to digress for just a sec and share something important in a situation such as this. I have three words tattooed on my forearm buried in the middle of some very cool abstract art. The three words are 'growth before pride'. That is my center. My center pushes me to learn and grow without fear of inadequacy. No one is perfect, no one knows everything and if we put our egos aside we can take advantage of some very unique opportunities. The power that holds is endless and I highly recommend you open yourself up to it. You never know when you'll be in a type of situation where you'll be 'schooled' but what you choose to do with it can change everything. Enough of my philosophical madness, let's move on to the good stuff.
So the core of the changes we were to take on were; re-architect, redesign and responsive. Pretty much in that order and starting immediately. Redesign and responsive, no problem. We were already headed in that direction and it was a comfortable user experience space. Re-architecting? That's a completely different story and one that ultimately set the stage for our rapid evolution.
The re-architecting primarily consisted of implementing services. So if you're world is UI related, how the heck does a service fit into that world? The short answer, it doesn't. So how do you test it? Rolls up your sleeves and learn to code...welcome to the QA of tomorrow folks. At this point in my career I have all but mastered managing technology and as you know I enjoy breaking software. However, I am not a developer by any stretch of the imagination. With that firmly in hand, I was the first to volunteer to learn how to code and implement this new model. As a test manager I would never push my team in a direction I wasn't willing to journey myself. Time to put up or shut up.
We had been an agile shop since I joined the company so the fact that we were going to be using that methodology was comforting. The new twist was that we were going to use Behavior Driven Development with a Cucumber and Ruby combination as a supporting toolset. We were to write our feature files in Cucumber, using the Gherkin syntax and code in the Ruby programming language. We would implement continuous integration leveraging Jenkins with a code repository in HG Mercurial being controlled by RhodeCode. Easy peasy lemon squeezy.
We covet the gray…
If you have never coded before this can be a bit overwhelming but equally as exciting. So why this combination of tools you ask? Well, in QA we walk a fine line. We aren't developers but we also aren't screen jockeys either. We are somewhere in the gray of the geek profession so if we must code, it makes sense that we foster simplicity and readability. We are business oriented, consumer focused technology warriors. In order to test we must supply simple, highly readable artifacts with more complex underlying code that is easy to understand and maintain.
Now I know that some of you will object that Java rains supreme as a coding standard. Perhaps it does but does that really make sense for quality assurance? Our test automation framework was in Java prior to this endeavor but it was only maintainable by a handful of people. That limited our skillsets by only allowing certain people to be skilled in automation which pigeon holed team members into certain roles. Wouldn't it be a big, bright, beautiful world if all test engineers were created equal? It wouldn't matter if Charlie was out of the office because Sudaka has the same skill set and could pick up and execute the same work. No more specialists, just highly skilled engineers that happily live in the gray.
If you covet the unity of this advanced skillset for your team you must invest in them. For the love of all that is holy, please don't give your 'fringe developers' Windows machines to code on, it's just insulting really. Give them Mac machines and set them up for success. This toolset does not play nice with Windows and honestly, who wants to play with Windows anyway? You want them to code, then foster their skills by giving them the right tools for the job. I was in Windows hell for over a month when trying to get this suite of tools to work and I still mourn the time and effort that was wasted. Now go order a nice shiny MacBook and all will be well.
Behavior Driven Development (BDD); Cucumber, Gherkin and Ruby
Let's talk test structure and Behavior Driven Development. Everything in this automation is about behavior. Four words are all you need really; Given, When, And, Then. Yup, that's Cucumber written in the Gherkin syntax. Here's an example:
Given that I am shopping at my local dollar store When it's raining on a Saturday And I have a purple store discount card Then I purchase twelve bananas
This is where our test journey begins. Nothing geeky about this, just good clean feature fun. The purpose is to start with this readable frame and then turn this into an automated test. Each line is then broken down into steps, methods and other stuff to create the necessary automation. Yes, you need to code but you start with this general feature which keeps you grounded and gives you lifelines back to the intended purpose. It is bit by bit rather than in an overwhelming landslide of functionality. Orderly, simple, dare I say…charming.
The thing that really makes my little heart flutter is reusability. Yes, I'm the crazy lady at the grocery store cheering when I buy one of something and get two free. I love getting a deal and this is quite a deal we are talking about. Let's take piece of the cake if you will; the line 'And I have a purple discount card'. That one line can be used in any test where a discount card is relevant and so can the code behind it if you architect it correctly and so can the validation of results. Sounds sweet and believe me it is. I architected full tests that are completely reused and didn't even write a line of code. I swear to you I just leveraged existing puzzle pieces and architected the relevant test for my own purposes. I swear to you that there is one step is our test suite that has been reused 137 times…that is so beautiful that is brings tears to my eyes.
Where's the code you say? That little step for the purple discount card will link you to your programming steps and subsequently your method in the elegant programming language that is Ruby. Ruby is the backbone for this entire model but Java or others could be used if desired. Cucumber is a Ruby 'gem'; in laymen's terms it's a plug-in, which allows Ruby to tap into that enhanced functionality. Ruby is known for its simplicity and ease of use, which applied for QA purposes, is just magnificent, as we aren't building applications, we are testing them.
Testing with a blindfold on…
If you're a User Interface tester it's very similar to testing with a blindfold on. There is no UI so you are going to have to communicate differently with the system which causes you to weep uncontrollably until you accept your shiny front end is no more. Once you're solidly through the stages of mourning, you dry your eyes and realize it's a gift. In my case all of my giddiness came rushing back when I realized that edits associated with the front end no longer applied. That means I can break things piece by piece and in all kinds of crazy ways sort of like a hacker would. Big fun I tell you! The front end point and click is replaced by your automation…that's right, it's all automation. Well, there's a test harness, which is nice, but if you want repeatability you roll up your sleeves and start coding. It takes some getting used to but Ruby is a forgiving language and adapting to it doesn't require weeks/months in a classroom. Just dive in, mess things up and learn hands on. I had an awesome teacher (all hail Steve) and he kept me grounded and focused. That is worth more than any class so I highly recommend pairing and mentoring.
The big bad test set…
When you first start this evolution, you want to test everything! That's a great goal but don't lose sight of where you are headed. In a couple months we had over 5,000 tests running and if we didn't structure our folders and test hierarchy carefully, it would have been unmanageable. Refactoring, changes in approach and even deleting things at times is a perfectly natural progression. Reserve yourself to the organic nature of your test set and allow it to keep evolving. Easily understood folder structures and naming conventions can go a long way to keep your sanity when test sets bulk up. Just keep it clean and simple so chaos doesn't reign.
Long live continuous integration…
With all of this automation goodness going on, it would be a down right shame to not automate the execution of the full test run. In our case we are using RhodeCode as the code manager and Mercurial as the code repository but these could easily be substituted out if desired. Good ole Jenkins comes into play as the orchestrator of the code builds and execution. Currently, our full automation run will kick off every time our QA environment is built with new code. We control the environment builds as well so we basically have free reign with the automated execution. This allows use to run the test suite at will or in a scheduled fashion. I love flexibility almost as much as I love reusability and trust me; investing time in setting this up pays off big time. I click four times; the environment builds and automation executes (yes, that number includes opening a browser session).
Fast forwarding to current date…
I look back on the past year plus and I am amazed at what we were able to achieve in such a short period of time. We went from a mostly manual test shop with an automation suite that only specialists could manage over to a whole new test framework, decoupled architecture, full automation with continuous integration. Our headless tests number around 9,000 and run in approximately two hours. I will caveat that with the fact that we shouldn't really need to run 9,000 tests and with refactoring we could improve our run time. This is the agile life my friends, when things are done in pieces, they are automated in pieces. Those pieces can eventually fit together more smoothly if you bank some time to refactor. Just as with continuous integration, it's a commitment that pays off big time and should be spiked out as real work. Ideally, I want a couple sprints to refactor and smooth out the runs. What would our lives become if we didn't dream, right?
Test on my friends!