And this is exactly what just happened to Arvind (from A journey into testing WebServices in a developer friendly way ). He was testing TeamMentor (TM) WebServices and suddenly there was no content on his test server :)
Couple interesting points:
- That happened because Arvind now has a better test setup which is able to invoke the TM webservice as reader and editor
- His tests have better state (i.e. correct values for real data, like a Library GUID)
- One of the WebServices he is now able to invoke correct is the 'DeleteLibrary'
- You can guess what happened next :)
- Test that were passing before the delete , started to fail
- Arvind will need to take into account the fact that some WebServices methods have 'destructive behaviour' (and only invoke them selectively)
And here is the key point: Most apps that DO something, will have a sequence of events that either corrupt or delete key data required for its normal behaviour. And if the automation tests are working they will trigger that scenario.
Or in another words: if your spider is not blowing up the database, it is not spidering deep enough
Another nice thing of the test suite that Arvind is building is the fact that we will end up with a 'the app is working ok' monitor, which is very valuable to any developer and TM user (good example of Security Tests adding a lot of value to the target application)
