Load testing using Siege

Siege is an open source load testing and benchmark utility. It’s widely used for performance and load testing of web applications with a user defined number of simulated users.

In this blog, we will learn how can we use siege for website load testing and as benchmark utility.

Install Siege


CentOS

sudo yum install siege

Ubuntu

sudo apt-get install siege

MacOSX

brew uninstall siege
brew install openssl
brew link --force openssl
brew install siege

Manual

Siege requires installation of openssl/libssl for testing https websites. Download the latest version if openssl is not installed. Go to https://www.openssl.org/source/ and download the latest version.

wget https://www.openssl.org/source/openssl-1.1.0e.tar.gz
tar -xvzf openssl-1.1.0e.tar.gz
cd openssl-1.1.0e
./config --prefix=/usr/local/openssl --openssldir=/usr/local/openssl
make
sudo make install

export PATH="/usr/local/bin/openssl/bin:$PATH"

ln -s /usr/local/openssl /usr/local/bin/openssl

Install Siege 4.0.0 with SSL compatibility option. I ran into https://github.com/JoeDog/siege/issues/66 with siege-latest.tar.gz - 4.0.2, so opted for siege 4.0.0.

wget http://download.joedog.org/siege/siege-4.0.0.tar.gz
tar -xvf siege-4.0.0.tar.gz
cd siege-4.0.0
./configure --with-ssl=/usr/local/openssl

make
sudo make install

Validate

Run below command to validate that siege is installed in your system.

siege -V

Siege quick overview


We can run siege in 2 modes

  • Default: This is the default behavior of Siege. Basically, the test hits the URL from all simulated users. For every simulated user, the next hit will come after a default delay of 1 second. The time interval can be changed by modifying the delay property in the configuration file.
  • Benchmark: In this mode, the test engine ignores the delay parameter and runs as fast as the server and the network allow it to. Every simulated user hits the next URL just after the previous one. Use the -b option to enable this mode of execution; optionally, set the benchmark property to true.

The results from Siege are commonly evaluated on following parameters:

  • Throughput: The transaction rate defines this metric, for example, 2000 requests per second
  • Error rate: Signifies whats the error rate and what’s the availability for the server
  • Response time: The results show the average response time

The formats for invoking siege are: siege options and siege options [url] . Following are most common options. For full details, you can run command siege -help

  • -c, –concurrent=NUM CONCURRENT users, default is 10
  • -r, –reps=NUM REPS, number of times to run the test.
  • -t, –time=NUMm TIMED testing where “m” is modifier S, M, or H ex: –time=1H, one hour test.
  • -d, –delay=NUM Time DELAY, random delay before each requst
  • -b, –benchmark BENCHMARK: no delays between requests.
  • -i, –internet INTERNET user simulation, hits URLs randomly.
  • -f, –file=FILE FILE, select a specific URLS FILE.

Siege comes with many default options. You can see the current config options with

siege -C

Load Testing and Benchmarking

Testing with N concurrent users each hitting website N times

Test with 1 concurrent user hitting website once

siege -c1 -r1 http://polyglotdeveloper.com -v

Here is the sample output:

    Transactions:		          11 hits
    Availability:		      100.00 %
    Elapsed time:		        1.27 secs
    Data transferred:	        0.65 MB
    Response time:		        0.08 secs
    Transaction rate:	        8.66 trans/sec
    Throughput:		        0.51 MB/sec
    Concurrency:		        0.68
    Successful transactions:          11
    Failed transactions:	           0
    Longest transaction:	        0.19
    Shortest transaction:	        0.01

Test with 10, 50, 100 concurrent users each hitting website twice

siege -c10 -r2 http://polyglotdeveloper.com -v
siege -c20 -r2 http://polyglotdeveloper.com -v
siege -c30 -r2 http://polyglotdeveloper.com -v

Testing with N concurrent users each hitting website over a period of time

Test with 10 concurrent users over period of 1M

siege -b -c10 -t1M http://polyglotdeveloper.com

Test with N hits every X seconds

If your website peaks at 100 hits every 10 seconds, then we can use these parameters to invoke siege: -c100 -d10. For 100 hits a second you would run it like this: -c100 -d1.

siege -c100 -d1 -t1M http://mocktarget.apigee.net/json
siege -c100 -d10 -t1M http://mocktarget.apigee.net/json
siege -c100 -d20 -t1M http://mocktarget.apigee.net/json

Testing with multiple URL’s

Instead of testing against a single URL, Siege also allows to test against multiple urls. This allows for a more real-world simulation of how a user would use your system.

cat <<EOF > site.txt
http://polyglotdeveloper.com/
http://polyglotdeveloper.com/categories/
http://polyglotdeveloper.com/archive/
EOF

siege -c100 -d1 -i -t1M -f site.txt

Testing with POST URL’s

cat <<EOF > employee.txt
{
 "name" : "alex"
 "age" : "30"
}
EOF

siege -H 'Content-Type: application/json' -c 100 -t 1M -b 'http://mocktarget.apigee.net/json POST < employee.txt'

Automate load increase using Bombardment utility

bombardment utility is part of the Siege package. It calls siege with an initial number of clients. When that run finishes, it immediately calls siege again with that number of clients plus the increment.

bombardment [urlfile] [inital # of clients] [inc value] [# of inc] [delay]

bombardment url.txt 10 5 3 1

Things to remember

We always need to make sure that we have enough capacity at hand on the server side as we increase the concurrency. An HTTP server handles a fixed number of incoming requests. For example, a pre-forking server has a limited number of child processes (MaxClients) and a threaded server has a fixed number of threads (MaxThreads). A server can handle up to its maximum number of handlers. Once its handlers are exhausted, socket connections are queued by the OS. They wait for the next available handler. If the http server is overwhelmed by more users than it can handle, sockets will timeout before they ever get a handler. Siege reports this as timeout, while a browser reports this as “Connection refused.”

Also, There are certain parameters that cannot be controlled from the command-line options such as socket timeout. It can be overridden from the default configuration file located at /etc/siegerc.

Closing thoughts

If you want to measure how your website performs under load, siege is a great tool. Remember that we should schedule concurrent users based on current traffic. An access log analyzer like webalizer or webtrends can help determine the current load.

Most common use cases of siege are running the siege on your website for your expected load requirements and seeing how different metrics such as response time, error rate, availability reacts.

References


https://github.com/JoeDog/siege https://geeksww.com/tutorials/libraries/openssl/installation/installing_openssl_on_ubuntu_linux.php https://www.joedog.org/siege-manual/ http://download.joedog.org/siege/ http://blog.modsaid.com/2014/04/stress-testing-with-siege-and-bombard.html

Version History


Date Description
2017-02-28 Initial Version