I have seen many a time that performance engineers skip using Assertions in load generation scripts. Many a times they assume that anything returned as an error by the tool itself such as a 404 or the 500 family errors are the only errors that need to be verified. However, there is more to a response than just the status code. For a perfectly valid output response all the resources related to the page need to be present and not just the base page/data for the request.
These are a few points for which assertions should always be present in a performance or load script:
These are a few points for which assertions should always be present in a performance or load script:
- Absence of assertions may result in response times being inaccurate as all the resources may not have been downloaded properly.
- For performance testing tools a 200 OK and similar success tools suggest the affirmative whereas in a performance testing use case such as the ones with login activity even an incorrect login is treated as not an error. This hence does not show the correct picture for the performance engineer. the idea should be to simulate the correct user behavior. Hence adding an assertion or many assertions to check whether a user has logged in correctly does make sense.
- Sometimes especially for web services, we may have 200OK responses without any meaningful data or with mostly blank data. Assertions help catch such issues.
Now people would argue that assertions consume memory. True, they do but the cost of an erroneous load test is much higher.
Assertions can be equivalent to unit tests in software programming. Too many meaningless unit tests add less value and only increase the time of builds. Similarly, assertions used cleverly gives huge security. Use few assertions but make sure you use them. One good example of assertions on a page or response is the most frequent text/link/resource the user is expected to view on the page. Next best assertion type is the page response size. Both the above mentioned assertion types are critical and light weight respectively and are advised to be used.