Working as a consultant

When I first started working as a consultant which was no more than a few months back, I was told that the it is about taking the client on a journey.

So, what are my experiences about being a consultant as compared to working as an engineer in a product company.

1) As a consultant you need to know contractual and time limitations. You should be able to make a decision on whether something can be done or not. While in a product company there is a lot more team involvement in what needs to happen.

2) You need to be more professional in terms of the assignments that come across. This is to do with right from time management to documentation to communication with the stakeholders. In a product company, it again goes down to the team that you work for especially in an agile environment.

3) More responsibility on individuals rather than the team. If things go wrong the blame goes to the consultant rather than the team.

The above are general observations.

Working with people

I consider myself as a individualistic person. Working alone on a challenging assignment is an ideal situation I thrive in. However, working with people is an important aspect of finishing a work on time and with better quality. So here are a few situations I have faced and perhaps how I should have handled them.

1) You are new in a job and are struggling with the process, tools and the setup and end up slowing down the schedule.
- Calm down, its common to be struggling in a new assignment unless you are a genius. For a guy like me who is not the fastest learner, I would suggest putting in extra hours, gaining confidence, being thorough with new things encountered today and reflecting on what went wrong.

2) You are dedicated to get the job done with high quality, however the one with you lacks motivation.
- Do your best. Try and help out whenever needed but only when your tasks are completed. I say this as it has been noticed many a times that people blame others when things go wrong but take credit when things go right. Hence it is important to finish your own work before helping out unless you are sure of the colleague you are working with. It can also be frustrating when dealing with people who act that they are working but in reality are only working for the final paycheck and expect blockages so that work can be procrastinated. Try and avoid arguments, escalate if things don't happen as it should be to achieve high quality objectives.

3) Work place politics, jealousy.
- It is very difficult for humans to robots (LOL) hence politics, jealousy and other such resistance is best avoided by focusing on the goals. This may make you fewer friends but would surely make you work better. However, it may be difficult to be neutral when the problems at work involve you as well. It is best to remind yourself what your duties at work are and stay professional at the place you are paid to do a job.

Load test crashed - Load Runner

Many a times we see that the load test crashes due to various environmental or test tool related factors. A Good part about Load Runner to what I used to work on is the reporting.  Hence most of the times when crashes occur and the results files tend to not have the stop times. Hence there tend to be errors while opening the analysis files.

Fix the error by opening the lrr file in a text editor and add the stoptime parameter as the current unix epoch time. Here is one link to find the same.

Performance Engineer.. vitals

So, what are some of the key attributes one must have to be a better performance engineer? 

I have always had this question. This may perhaps be because I was a lone wolf performance tester/engineer in most of my roles and never had a mentor or a senior engineer for guidance. Any, opportunity I had to pick peoples brains was used upon whenever possible. Those 15 or 30 minute interactions I had with engineers have gone a long way in developing ideas, gaining vital knowledge and building confidence. I even once asked a naive question as to how can one judge a good performance engineer, to which I got an answer that said you never judge one as everyone has his/her own distinct qualities.

But still, how do one become a better performance engineer. After working at a few places and interacting and reading about this I have here a list of few key attributes that I suggest performance engineers or aspirants should have a look at.

For those who want to make a start with performance testing, start off with learning a performance testing tool. JMeter is a good start, however with other tools such as Load Runner I have known so many extra and important feature which are not there in JMeter. Next I suggest learning a bit about server monitoring, server resources to measure, response times and a little bit about statistics. Zabbix is one good open source tool for remote monitoring. It is advisable to keep building knowledge as you progress. Load testing is dangerous and it is advisable to be judicious with the test application you choose or the load you generate. This was a little para about starting with Performance testing. I will perhaps write a separate post on this later.

Now for the intermediates, try and pay attention to detail always, especially with analysis:

·          Every load test you run the one graph you need to pay attention to detail is the Response time graph.

·          Ignoring a spike or ignoring different requests can be disastrous. 

·         Correlation with a corresponding abnormality you see in other graphs, resources or parameters you are monitoring is the next step.

   

     Some advice:

It is also important to fully confirm what the requirements are and what happened the last time the test was run. When you join a new project or are assigned to a new task, just spend some time knowing the requirements thoroughly and the older reports if there are any.

For ongoing projects try and understand what happened, especially for response times and defects.

Performance engineers are expected to provide analysis and recommendations hence analyse data for potential bottlenecks or at least specify that analysis is coming soon in order to avoid any repercussions.

Assertions who? Yes they are vital

I have seen many a time that performance engineers skip using Assertions in load generation scripts. Many a times they assume that anything returned as an error by the tool itself such as a 404 or the 500 family errors are the only errors that need to be verified. However, there is more to a response than just the status code. For a perfectly valid output response all the resources related to the page need to be present and not just the base page/data for the request.

These are a few points for which assertions should always be present in a performance or load script:

1. Absence of assertions may result in response times being inaccurate as all the resources may not have been downloaded properly.
2. For performance testing tools a 200 OK and similar success tools suggest the affirmative whereas in a performance testing use case such as the ones with login activity even an incorrect login is treated as not an error. This hence does not show the correct picture for the performance engineer. the idea should be to simulate the correct user behavior. Hence adding an assertion or many assertions to check whether a user has logged in correctly does make sense.
3. Sometimes especially for web services, we may have 200OK responses without any meaningful data or with mostly blank data. Assertions help catch such issues.

Now people would argue that assertions consume memory. True, they do but the cost of an erroneous load test is much higher. 

Assertions can be equivalent to unit tests in software programming. Too many meaningless unit tests add less value and only increase the time of builds. Similarly, assertions used cleverly gives huge security. Use few assertions but make sure you use them. One good example of assertions on a page or response is the most frequent text/link/resource the user is expected to view on the page. Next best assertion type is the page response size. Both the above mentioned assertion types are critical and light weight respectively and are advised to be used.