Jmeter remote execution, log collection via STAF / STAX

One of the biggest challenges after learning your way around Jmeter is figuring out a way to organize and retrieve results in an automated way. There are different documented ways of doing this via Ant scripts or even neat graphing results like Performance Bling. Though my needs were different and after much looking around, coding and testing my performance environment is made up of 3 major areas.



i) Jmeter
ii) STAF / STAX
iii) Ruby
My focus will be on integrating STAF / STAX why I choose STAX as the glue between the different parts in my performance environment..
So what's Staf?
Staf can be thought of as a pseudo command shell that allows for specific commands of work to be executed on local or remote host machines depending upon permission settings of the caller and receiver of the requesting action. STAF can perform specific functions or tasks that are generally related to process automation such as file manipulations, command line actions, or logging functions. The focus moves away from code and OS specifics to completing pre-defined tasks of work to be called more efficiently.
A STAF action can be broken up into processes or commands. Both contain specific parameters and have pre-defined actions for what they do. In order to really benefit from combining multiple STAF commands into one script we'll need to look at STAX.
STAX is a service plug-in of STAF that allows for STAF commands to be written in XML. In STAX files STAF calls can interact with python using the <script> tags. The resulting action of a STAF call is returned by the STAXResult variable which is really a python list object. Inside this object two values are stored. The first indicates the STAFResult or value of the STAF call's execution status 1 or 0, the second value contains the standard out or error message if its set to store it.
So lets look at our requirements for our Jmeter scenario.
We want to be able to:

• a) start Jmeter and execute a script
• b) collect result logs including error logs
• c) combine files where necessary.

So lets begin by creating a function that will understand this:

jmeter -n -t myLoadTestScript.jmx

and turn it into this

   1: <call function="'runJmeter'">'apu-testmachine',LinuxjmeterPath,'testscript_Linux.jmx','/opt/Jmeter/bin/scripts/GA',testDuration,'/opt/Jmeter/bin/scripts/GA','/opt/Jmeter/bin/scripts/GA' + '/staxRun.log'</call>

The idea here is that we ask STAF to make a STAF Command call using STAX code. We pass our Jmeter executable location, the test script, working directory, duration and other needed values.
This command line call will invoke Jmeter for the length of the test. Once the timer duration value has reached the "testDuration" value then the STAF process executing on the machine will be ended abruptly. This is important to keep in mind because this is the same as killing Jmeter's process during a test run.
Fortunately on Linux this is enough to stop the test by killing the Jmeter process which is a child process of the STAF command executing on that machine. For Windows machines executing this code the Jmeter process will not be killed. Unfortunately you will need to use a separate function to kill Jmeter on windows.

<script>
   
   LinuxjmeterPath = '/opt/Jmeter/bin/jmeter'
   WinjmeterPath = 'C:/Jmeter/jakarta-jmeter-2.3.2/bin/jmeter'
   winKill = "taskkill /f /fi \"imagename ne explorer.exe\" /fi \"memusage gt 40000\""
 
</script>

The basic idea is that taskkill is being performed via a command process through STAF which will kill the Jmeter process. This isn't the prettiest way to kill Jmeter but it's been effective for my testing needs in the past, so I've had no pressing need to change it or improve it.

Additionally one might consider performing some type of list process to verify that Jmeter is not running after the duration time has passed on Linux. This is something nice to have, but something I have not had a need for as stated above.
So the significance of what we've discussed so far is that we are able to have 1 function to run on windows, or linux to call and execute a Jmeter test plan. The advantage is that we don't worry about OS specifics and focus just on the parameters that we are passing. This is very useful if you have several load generator slave machines that need to fire off tests, and each machine may have a slightly different set up. This is also useful for queuing slaves to start tests within the same time frame. Generally the delays between starting the tests are usually insignificant and are impacted more by the processing speed on the machine then in possible bottlenecks of invoking the tests remotely through STAF.

Ok this is neat but why not just manage remote machines through Jmeter master/ slave set up?

There are several reasons for using separate slaves to run separate Jmeter instances. The most critical issue for myself was that I was unable to set up (1) Jmeter master to control multiple Jmeter (slaves) with one main log file for response and error logs. It seemed like I would be creating much work and network traffic to transfer such logs in real time. My solution was to run separate instances of Jmeter per machine and store the logs locally. After the test completes STAX is called to collect the respective logs to one computer, I then use a linux command to cat *.extension > new_combined file.extension I then move this file to another machine for parsing, processing and archiving. For some Jmeter users this wont be as useful though I tend to work with very large result files varying from 25 Mb to 250 Mb that are just csv's; so there's a lot of information that is being trapped and parsed.


So here is another simple example of using a simple command line call that will allow us to something very useful such as combing files. to one file. Before I run this code I'm able to identify the directory and testrun to copy the files from by the naming convention used by the folder name. Every test run creates a timestamp as a part of the directory name. I then scan the directories and identify the last modified directory and am able to construct a relevant path from the last modified folder and then copy the files to be combined.

Here I'll show how to identify the last modified folder or really the last test result written to disk, and how to copy that file over for processing

I've shown how to start Jmeter using STAX, how to collect and copy files when necessary and lastly how to combine different result files that come from Jmeter.








In subsequent posts I will show how to collect additional results such as server side metrics, active heap memory usage for Websphere applications, and parse the raw results to create some nice pretty summary reports using Ruby, Win32ole, FasterCsv, and Excel and even sparklines.