Skip to main content

Test your services and not your UI

As a tester, i have spent more than 6 years trying to automate tests at the UI layer. As years passed, better automation frameworks evolved resulting in writing more manageable and maintainable code, but the only thing that wasn't getting better was the cost of maintaining these tests, especially when the no. of tests were 500+. There are multiple reasons why i feel creating an automated regression suite of tests at the UI layer is not good. Some of them are
  • Limitations on what the tool can do
  • Time taken to execute these tests leading to longer feedback loop
  • Success rate of tests is not 100% due to latency issues
  • Change in UI and UI flow resulting in an increase in cost to maintain UI tests
On the other hand, testing business logic without having to deal with the dumb UI was a concept that a friend of mine at ThoughtWorks, Chirag Doshi introduced to me. He sent me couple of blog links written by Alex Verkhovsky about why is it so hard to do functional test automation at the UI layer.

So, we decided to try the model where we would test the business logic. The project where we wanted to try this concept had a Service Oriented Architecture. So basically, the business logic was driven by services and there was a dumb UI that would consume these services. The plan was to automate realistic end to end business flows using a combination of services. As it turned out, this concept worked really well for us.

Some reasons why this concept worked for us are
  • No limitations of the tool. Absolutely anything could be automated since all we had to do is send a request to the service, and use a part of the response along with further parameters to pass to the next service request and then make assertions where necessary.
  • Test execution time is 100 times faster than that of UI tests. Latency is not an issue here.
  • No inconsistencies in test pass rate. Tests would fail only when defects were introduced.
  • As Testers were consuming these services, they were like customers of the service, which led to driving changes in service design
  • Language and technology is same as what the developers use to unit/integrate test the services
  • Test failures could be fixed by any developer
  • Developers would run the entire regression suite of tests before checking in as the execution time of tests was very fast
  • Most stories would be split into service and UI where the devs would first pick the service part. Testers would start automating tests in parallel while the UI part was being plugged into the service. This resulted in the service tests almost being complete by the time the UI part would be complete.
  • Developers would contribute a lot in the design aspect of the automation framework which made writing tests more manageable and maintainable.

Comments

  1. Kristan VingrysAugust 18, 2010 at 9:23 PM

    I agree with Vijay that business logic should be tested at the lowest layer possible, for example the services and then the higher layer tests focus on ensuring data is passed and received correctly. With Ajax heavy applications or rich clients that have business logic built into the UI layer then the UI needs extensive testing. As testers we should work with the development team to ensure the design is testable and I think reducing the amount of business logic in the UI layer is definitely one way of making the design more testable.

    ReplyDelete

Post a Comment

Popular posts from this blog

Build quality in early and often

One of the most important aspects of building good software is to encourage the concept of build, measure and learn. For companies to be able to innovate and be quick to market they must encourage a good engineering culture that sets up teams for success. In an ideal world, you should deliver to production daily. However, if you deliver software fast, but it is full of bugs, your product has a lower chance of succeeding. As an agile tester, one of your focus points has to be to speed up the feedback loop while maintaining good quality. Over the years I have laid across a few good practices that make teams build the product right and also build the right product. Engage Test Engineers as early as possible in the development cycle Test Engineers are often treated as the last stand against finding problems before release, yet like all software activity; their focus is affected by the information available to them. In order to better understand the risk associated with changes an
Post a Comment
Read more

Can projects do without Business Analysts?

My  last 2 gigs were a bit different to the usual ones from a team composition point of view. The bit that was different was that there weren't any Business Analysts on the team. My initial concerns were who would gather requirements? Who would analyse stories? Who would negotiate scope with the business? Who would be involved in Scope Management? Who would be our proxy customer? The above questions got me concerned, but it wasn't as bad as i thought it to be. The customer was co-located with the team and we had easy access to validate our understanding on business rules, scope, sign-offs etc. Our team composed of an IM, Devs and Testers. The IM was managing scope with the customer and getting the priorities. The team would then sit with the business to understand what the requirements were and we would create Epics and then further break them down into smaller chunks of workable stories. A thing to note here is that all roles would put their BA hat on and identify gaps if
Post a Comment
Read more

BDD is over-rated

Over the past few years, I have tried to justify the use of a BDD (Behavioral Driven Development)  framework to express my tests, but not once have been able to say BDD has helped me address a  problem which writing tests the non BDD way would not have addressed. I do understand the value BDD brings to the table, but in most projects that I have implemented BDD on, we have tried to provide a solution (BDD) to a problem that does not exist. Let me try and explain. Lets look at the key benefits of expressing tests the BDD way (There could be more) Collaboration between Business and Development Ubiquitous Domain Language Focus on the behavior of the application Now, more often than not unless your business is co-located with the team, collaboration is not the easiest. The value BDD brings here, is the business validating our understanding by reading our tests expressed in the Given When Then format (BDD) and providing feedback, as BDD expresses the behavior of the system in a l
1 comment
Read more