OpenSTA, Load & Stress Testing Tool, Alternatives on Linux

In our effort to fully switchover to Linux as Desktop and Server environment, one of the biggest challenges is to find an alternative to OpenSTA, a popular and very capable open source distributed software testing tool for regression, load and stress testing. In other words OpenSTA is poor man’s alternative to LoadRunner & Silk.

The best alternative so far appears to be running OpenSTA using Wine emulation. Unfortunately there is no documented evidence that anyone ever ran it successfully, which doesn’t mean that it can’t be
I will try it and let you know how it goes.

The other products I want to evaluate are:
JMeter - A mature but somewhat hard to use, based on earlier experience, Java based open source load testing tool.

“Apache JMeter is a 100% pure Java desktop application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions. Apache JMeter may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. You can use it to make a graphical analysis of performance or to test your server/script/object behavior under heavy concurrent load.”

Linux Desktop Testing Project - “Linux Desktop Testing Project (LDTP) is aimed at producing high quality test automation framework and cutting-edge tools that can be used to test Linux Desktop and improve it. It uses the Accessibility libraries to poke through the application’s user interface. The framework also has tools to record test-cases based on user-selection on the application.”

Expect from NIST - “Expect is a tool for automating interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, etc. Expect really makes this stuff trivial. Expect is also useful for testing these same applications. And by adding Tk, you can also wrap interactive applications in X11 GUIs.

Expect can make easy all sorts of tasks that are prohibitively difficult with anything else. You will find that Expect is an absolutely invaluable tool - using it, you will be able to automate tasks that you’ve never even thought of before - and you’ll be able to do this automation quickly and easily. ”

Wikipedia says: Expect is a Unix automation and testing tool, written by Don Libes, for interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, ssh, and others. With Tk, interactive applications can be wrapped in X11 GUIs.

Frankly it doesn’t look too promising for web application testing but I am open to suggestions.

DejaGNU - “DejaGnu is a framework for testing other programs. Its purpose is to provide a single front end for all tests. Think of it as a custom library of Tcl procedures crafted to support writing a test harness. A test harness is the testing infrastructure that is created to support a specific program or tool. Each program can have multiple testsuites, all supported by a single test harness. DejaGnu is written in Expect, which in turn uses Tcl — Tool command language.”

I had developed a Java & xml based regression and load test automation framework. It was pretty successfully used in the last company I worked for. I would evaluate it again in the current context to find out where it stands.

Commercial alternatives are:
ProxySniffer - “Proxy Sniffer measures the response time and the stability of web applications under real load conditions - by simulating hundreds or thousands of web users.”

PureLoad - “PureLoad is a load testing tool that simulates hundreds of users executing requests against server based applications. Use PureLoad to verify that your application will meet the expected performance criteria. PureLoad reports quality and performance problems as well as detailed statistics gathered during a load test. PureLoad do also include extensive support for easy recording and testing of web based applications.”