QURD - QUeue Resource Daemon

The Queue Resource Daemon is an extensible SQS monitoring service, which can be configured to react to or ignore AutoScaling messages. Qurd can be configured to monitor multiple accounts, any number of queues, and any type of auto scaling event.

When the daemon starts up, it finds the queues it's meant to monitor and sets the visibility_timeout and waittimeseconds for each queue. Qurd uses long polling to monitor the queues, by default.

This daemon makes extensive use of threads, you should really consider running this with Ruby version 2.0 or higher.

Plugin architecture

It is possible to provide your own actions, aside from Chef, Route53, and Dummy. Dummy is provided as a simple example, but, in a nutshell, inherit from Qurd::Action and override the actions you respond to.

Your action class can configure itself, by overriding the class method configure. Instances must override the action methods launch, launch_error, terminate, terminate_error, and test. Action instances have two attributes, message and context. Message is a Qurd::Message instance. Context is a Cabin::Context, used for logging. Callbacks, to interact with the action before and after the instance are executed, can be overridden.

The mixins for AwsClients and Configuration are also available at the class and instance level.

# This contrived example creates a file in s3 when an instance launches
# It can be triggered by adding the class name to the list of actions in the
# configuration file, ie
#       bucket: example-bucket
#       actions:
#         launch:
#           - "Foo"
class Foo < Qurd::Action
  def self.configure(_action)
    qurd_configuration.bucket || qurd_logger!("Missing bucket")
  end

  def run_before
    aws_retryable do
      aws_client(:S3).delete_object(
        bucket: qurd_configuration.bucket,
        key: message.instance_id
      )
    end
  end

  def launch
    aws_retryable do
      aws_client(:S3).put_object(
        bucket: qurd_configuration.bucket,
        key: message.instance_id,
        body: message.instance.private_ip_address
      )
    end
  end

  def run_after
    aws_retryable do
      o = aws_client(:S3).get_object(
        bucket: qurd_configuration.bucket,
        key: message.instance_id
      )
      qurd_logger.debug("Found #{o.body}")
    end
  end

end

AWS IAM Policy

QURD requires, at a minimum, SQS privileges and EC2 privileges, ie

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "qurd_sqs",
      "Effect": "Allow",
      "Action": [
        "sqs:DeleteMessage",
        "sqs:ListQueues",
        "sqs:ReceiveMessage",
        "sqs:SetQueueAttributes"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "qurd_ec2",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}

If you are using the route53 action, you will also need

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1428424119000",
            "Effect": "Allow",
            "Action": [
                "route53:ChangeResourceRecordSets",
                "route53:DeleteHostedZone",
                "route53:GetHostedZone",
                "route53:ListHostedZones",
                "route53:ListResourceRecordSets"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Configuration

To configure the daemon, edit the YAML configuration file. The default path is /etc/qurd/config.yml. An alternate path can be specified on the command line.

Option	Description	Type	Default
`aws_credentials`	AWS credentials	`[Array<Hash>]`
`auto_scaling_queues`	Describe the queues to be monitored	`[Hash<Hash>]`
`actions`	describe actions to take, for any autoscaling event	`[Hash<Array>]`
`daemonize`	Force qurd to log to a file, even if `log_file` is not defined.	`Boolean`	`false`
`dry_run`	Log what qurd would have done	`Boolean`	`false`
`listen_timeout`	Defines the timeout, in seconds, for a thread to process a message	`Float`	`visibility_timeout`
`log_file`	The path to qurd's log file	`String`	`/var/log/qurd/qurd.log`
`log_level`	The log level to catch	`String`	`info`
`pid_file`	The path of qurd's pid file	`String`	`/var/run/qurd/qurd.pid`
`save_failures`	Save messages if any action fails	`Boolean`	`true`
`sqs_set_attributes_timeout`	Defines the timeout, in seconds, for a thread setting SQS attributes	`Float`	`10`
`visibility_timeout`	Set the SQS visibility timeout	`Integer`	`300`
`wait_time`	Set the SQS wait time seconds	`Integer`	`20`

aws_credentials

Qurd supports AssumeRoleCredentials, Credentials, InstanceProfileCredentials, and SharedCredentials. Each credential must be named and must have a type defined. The options key allows the caller to define keys and values, mirroring the options for each of the credential types.

If no aws_credentials are defined in the configuration file, the key default is created and it will attempt to use Aws::InstanceProfileCredentials. Each auto_scaling_queues will have its credentials automatically set to default.

aws_credentials:
  - name: prod
    type: assume_role_credentials
    options:
      role_arn: "arn:aws:iam::1:user/[email protected]"
      role_session_name: foo
  - name: staging
    type: credentials
    options:
      access_key_id: abc123
      secret_access_key: 123abc
  - name: dev
    type: instance_profile_credentials
  - name: test
    type: shared_credentials
    options:
      profile_name: default

auto_scaling_queues

A hash of hashes, which describe the queues to be monitored. The outer key is the name of the group of queues, ie production, staging, etc. The inner keys credentials, region, and queues are required. Credentials should refer to the name of an aws_credential. The region is the region of the queues. The queues key is an array of queue names and regular expressions. Regular expressions are strings, which begin and end with forward slash. Regular expressions can also have modifiers applied to them.

The optional keys wait_time and visibility_timeout override the global options of the same name.

The credentials key will be overridden, and set to default, if no aws_credentials are defined.

auto_scaling_queues:
  dev:
    credentials: dev
    region: us-east-1
    queues: "/scalingnotificationsqueue/i"
  staging:
    credentials: staging
    region: us-west-2
    visibility_timeout: 100
    wait_time: 20
    queues:
      - FooQueue
      - BarQueue
      - "/ScalingNotificationsQueue/"

actions

A hash of arrays, describing actions to take, for any autoscaling event. To test the various options, you could configure the dummy action for each event.

actions:
  launch:
    - "Qurd::Action::Dummy"
  launch_error:
    - "Qurd::Action::Dummy"
  terminate:
    - "Qurd::Action::Dummy"
  terminate_error:
    - "Qurd::Action::Dummy"
  test:
    - "Qurd::Action::Dummy"

Installation

Add this line to your application's Gemfile:

gem 'qurd'

And then execute:

$ bundle

Or install it yourself as:

$ gem install qurd

Usage

qurd [/PATH/TO/CONFIG.yml]

Tests

bundle exec rake

WebMock stubs can be found in test/support/web_mock_stubs.rb, responses are in test/responses.

Contributing

Fork it ( https://github.com/Adaptly/qurd/fork )
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Write some tests!
Push to the branch (git push origin my-new-feature)
Create a new Pull Request