Load testing voice systems, and voice applications in particular, is tricky. While several commercial tools exist, there is really only one tool in the Open Source world that is good at efficiently generating SIP load: SIPp. While SIPp does a good job of generating load, it is somewhat clumsy to use, due to a verbose XML format for scenarios, a confusing set of command line parameters, and worst of all, a lack of tools to create media needed to interact with voice applications.
The last problem is especially tricky: Imagine you want to load test an IVR. Testing requires:
- calling a test number
- waiting a certain amount of time
- sending some DTMF
- waiting some more
- sending more DTMF
To test this with SIPp you need a PCAP file that contains the properly timed DTMF interactions. Since there is no tool to create this media, it is usually necessary to call into the system and record the PCAP, isolate the RTP from the captured packets with something like Wireshark, then connect the pcap file into the SIPp scenario. This process is time consuming and error prone, meaning that testing isn't done as often as it should.
SippyCup aims to help solve these problems.
Sippy Cup is a tool to generate SIPp load test profiles and the corresponding media in PCAP format. The goal is to take an input document that describes a load test in a very simple way (call this number, wait this many seconds, send this digit, wait a few more seconds, etc). The ideas are taken from LoadBot, but the goal is for a more performant load generating tool with no dependency on Asterisk.
SippyCup relies on the following to generate scenarios and the associated media PCAP files:
- Ruby 1.9.3 or later (2.1.2 recommended)
- SIPp - Download from http://sourceforge.net/projects/sipp/files/
- "root" user access via sudo: needed to run SIPp so it can bind to raw network sockets
If you do not have Ruby 2.1.2 available (check using
ruby --version), we recommend installing Ruby with RVM
Install via gem (production)
Once Ruby is installed, install SippyCup:
gem install sippy_cup
Now you can start creating scenario files like in the examples below.
Install from repository (development)
bundle command (from the "bundler" package) to install from the source directly. First, clone the repository into a working directory.
bundle via gem:
gem install bundler --no-ri --no-rdoc
Then build the
sippy_cup application with
bundle will then install the gem dependencies and allow you to run
sippy_cup from your working directory.
--- source: 192.0.2.15 destination: 192.0.2.200 max_concurrent: 10 calls_per_second: 5 number_of_calls: 20 steps: - invite - wait_for_answer - ack_answer - sleep 3 - send_digits '3125551234' - sleep 5 - send_digits '#' - wait_for_hangup
destination above may be optionally supplied with a port number, eg.
Next, execute the scenario:
$ sippy_cup -r my_test_scenario.yml I, [2013-09-30T14:48:08.388106 #9883] INFO -- : Preparing to run SIPp command: sudo sipp -i 192.0.2.15 -p 8836 -sf /var/folders/n4/dpzsp6_95tb3c4sp12xj5wdr0000gn/T/scenario20130930-9883-1crejcw -l 10 -m 20 -r 5 -s 1 192.0.2.200 Password: ...snip... I, [2013-09-30T14:48:16.728712 #9883] INFO -- : Test completed successfully.
Example embedding SIPp in another Ruby process
require 'sippy_cup' scenario = SippyCup::Scenario.new 'Sippy Cup', source: '192.168.5.5:10001', destination: '10.10.0.3:19995' do |s| s.invite s.wait_for_answer s.ack_answer s.sleep 3 s.send_digits '3125551234' s.sleep 5 s.send_digits '#' s.wait_for_hangup end # Create the scenario XML and PCAP media. File will be named after the scenario name, in our case: # * sippy_cup.xml # * sippy_cup.pcap scenario.compile!
The above code can be executed as a standalone Ruby script and the resulting scenario file run with SIPp.
Customize Your Scenarios
Available Scenario Steps
sleep <seconds>Wait a specified number of seconds
inviteSend a SIP INVITE to the specified target
register <username> [password]Register the specified user to the target with an optional password
receive_tryingExpect to receive a
100 Tryingresponse from the target
receive_ringingExpect to receive a
180 Ringingresponse from the target
receive_progressExpect to receive a
183 Progressresponse from the target
receive_answerExpect to receive a
200 OK(answering the call) response from the target
wait_for_answerConvenient shortcut for
receive_trying; receive_ringing; receive_progress; receive_answer, with all but the
answermarked as optional
ACKin response to a
send_digits <string>Send a DTMF string. May send one or many digits, including
receive_okExpect to receive a
receive_message [regex]Expect to receive a SIP MESSAGE, optionally matching a regex
receive_byeExpect to receive a
BYEfrom the target
200 OKresponse to a
wait_for_hangupConvenient shortcut for
Alternate Output File Path
Don't want your scenario to end up in the same directory as your script? Need the filename to be different than the scenario name? No problem!
sippy_cup manifest, use
--- filename: /path/to/somewhere
Or, in Ruby:
s = SippyCup::Scenario.new 'SippyCup', source: '192.168.5.5:10001', destination: '10.10.0.3:19995', filename: '/path/to/somewhere' do # scenario definitions here... end s.compile!
This will create the files
somewhere.pcap in the
Customizing the Test Run
Each parameter has an impact on the test, and may either be changed once the XML file is generated or specified in the options hash for
SippyCup::Scenario.new. In addition to the default parameters, some additional parameters can be set:
- Path to a file where call statistics will be stored in a CSV format, defaults to not storing stats
- Frequency (in seconds) of statistics collections. Defaults to 10. Has no effect unless :stats_file is also specified
- SIP user from which traffic should appear. Defaults to "sipp".
- SIP user to send requests to. Defaults to "1" (as in firstname.lastname@example.org).
- Specify the SIP transport. Valid options are `udp` (default) or `tcp`.
- By default, SippyCup will show SIPp's command line output while running a scenario. Set this parameter to `false` to hide full command line output
- A string of SIPp command line options included with the SIPp run.
- By default, SIPp assigns RTP ports dynamically. However, if there is a need for a static RTP port (say, for data collection purposes), it can be done by supplying a port number here.
- Specify the mechanism by which DTMF is signaled. Valid options are `rfc2833` for within the RTP media, or `info` for SIP INFO.
- If you're using sippy_cup to run a SIPp XML file, there may be CSV fields in the scenario ([field0], [field1], etc.). Specify a path to a CSV file containing the required information using this option. (File is semicolon delimeted, information can be found [here](http://sipp.sourceforge.net/doc/reference.html#inffile).)
Additional SIPp Scenario Attributes
With Sippy Cup, you can add additional attributes to each step of the scenario:
# This limits the amount of time the server has to reply to an invite (3 seconds) s.receive_answer timeout: 3000 # You can override the default 'optional' parameters s.receive_ringing optional: false s.receive_answer optional: true # Let's combine multiple attributes... s.receive_answer timeout: 3000, crlf: true
For more information on possible attributes, visit the SIPp Documentation.
Copyright (C) 2013-2014 Mojo Lingo LLC
"Sippy Cup" name suggested by Jamey Owens