Skip to main content

Test your services and not your UI

As a tester, i have spent more than 6 years trying to automate tests at the UI layer. As years passed, better automation frameworks evolved resulting in writing more manageable and maintainable code, but the only thing that wasn't getting better was the cost of maintaining these tests, especially when the no. of tests were 500+. There are multiple reasons why i feel creating an automated regression suite of tests at the UI layer is not good. Some of them are
  • Limitations on what the tool can do
  • Time taken to execute these tests leading to longer feedback loop
  • Success rate of tests is not 100% due to latency issues
  • Change in UI and UI flow resulting in an increase in cost to maintain UI tests
On the other hand, testing business logic without having to deal with the dumb UI was a concept that a friend of mine at ThoughtWorks, Chirag Doshi introduced to me. He sent me couple of blog links written by Alex Verkhovsky about why is it so hard to do functional test automation at the UI layer.

So, we decided to try the model where we would test the business logic. The project where we wanted to try this concept had a Service Oriented Architecture. So basically, the business logic was driven by services and there was a dumb UI that would consume these services. The plan was to automate realistic end to end business flows using a combination of services. As it turned out, this concept worked really well for us.

Some reasons why this concept worked for us are
  • No limitations of the tool. Absolutely anything could be automated since all we had to do is send a request to the service, and use a part of the response along with further parameters to pass to the next service request and then make assertions where necessary.
  • Test execution time is 100 times faster than that of UI tests. Latency is not an issue here.
  • No inconsistencies in test pass rate. Tests would fail only when defects were introduced.
  • As Testers were consuming these services, they were like customers of the service, which led to driving changes in service design
  • Language and technology is same as what the developers use to unit/integrate test the services
  • Test failures could be fixed by any developer
  • Developers would run the entire regression suite of tests before checking in as the execution time of tests was very fast
  • Most stories would be split into service and UI where the devs would first pick the service part. Testers would start automating tests in parallel while the UI part was being plugged into the service. This resulted in the service tests almost being complete by the time the UI part would be complete.
  • Developers would contribute a lot in the design aspect of the automation framework which made writing tests more manageable and maintainable.

Comments

  1. Kristan VingrysAugust 18, 2010 at 9:23 PM

    I agree with Vijay that business logic should be tested at the lowest layer possible, for example the services and then the higher layer tests focus on ensuring data is passed and received correctly. With Ajax heavy applications or rich clients that have business logic built into the UI layer then the UI needs extensive testing. As testers we should work with the development team to ensure the design is testable and I think reducing the amount of business logic in the UI layer is definitely one way of making the design more testable.

    ReplyDelete

Post a Comment

Popular posts from this blog

How does one decide what to automate?

A question that has been troubling me for years is how much to automate. We all like to achieve maximum coverage and build a robust safety net where possible in order to give us confidence about the system under test. Given there is ample time i would like to automate almost everything, but in the real world, this isn't possible. This is when we introduced the concept of Value and Cost for a test scenario. Every test scenario would be assigned either of these attributes. Lets call the attribute automation classification High Value - Low Cost High Value - High Cost Low Value - Low Cost Low Value - High Cost What this translates into is we are associating business value and cost to automate to every scenario. So, when a test scenario is of high business value and the cost to automate it is low, then this becomes our ideal automation candidate. You would like to start automating all test scenarios that fit into the High Value - Low Cost category first. See picture below If tim...
1 comment
Read more

BDD is over-rated

Over the past few years, I have tried to justify the use of a BDD (Behavioral Driven Development)  framework to express my tests, but not once have been able to say BDD has helped me address a  problem which writing tests the non BDD way would not have addressed. I do understand the value BDD brings to the table, but in most projects that I have implemented BDD on, we have tried to provide a solution (BDD) to a problem that does not exist. Let me try and explain. Lets look at the key benefits of expressing tests the BDD way (There could be more) Collaboration between Business and Development Ubiquitous Domain Language Focus on the behavior of the application Now, more often than not unless your business is co-located with the team, collaboration is not the easiest. The value BDD brings here, is the business validating our understanding by reading our tests expressed in the Given When Then format (BDD) and providing feedback, as BDD expresses the behavior of the syst...
1 comment
Read more