Reyes

Reyes populates IPTables firewall rules based on EC2 security group rules. It is named after the Pt. Reyes Lighthouse, which shines light through the fog, preventing your ships from crashing on the rocks as they make their way to port.

Use Case

Reyes is designed to apply security group rules to IPsec VPN traffic that would otherwise be injected past security group protection. This is useful for enforcing firewalls on VPNs between EC2 instances and security groups in other VPCs, even in other regions.

Stripe uses Reyes to ensure that we apply security group rules to traffic running through our IPsec VPNs between VPCs and between EC2 classic and VPC.

Requirements

Reyes is designed to have as few requirements as possible. It uses Amazon S3 to distribute rule updates in JSON, and it signs these updates with a GPG key to ensure rule integrity and authenticity.

Gem dependencies:

• AWS SDK for Ruby Version 1
• Subprocess
• Chalk::Log

Currently Reyes assumes that your VPC uses a CIDR block in 10.0.0.0/8. It also only understands TCP and UDP traffic.

Architecture

To avoid pummeling the EC2 API, Reyes uses a leader and follower model.

A reyes leader queries the Amazon EC2 API to list the instances and security group rules in all desired regions and VPCs. It then uploads a signed JSON file to an S3 bucket. We recommend running multiple hot spare leaders for redundancy. (All of them will upload the same data, and the last one will win.) The leader should be run periodically, e.g. with a cron job every 5 minutes.

A reyes follower downloads the JSON data from S3 and verifies the GPG signature. (This GPG key should be managed by your normal configuration management process.) It then creates Linux kernel IP sets via ipset that replicate the security group members, and populates IPTables rules with the security group data. To avoid replay attacks, the rule file contains a not_after field defaulting to 1 hour after generation.

Cutover to the new rules is atomic (using iptables-restore). To avoid accidentally locking you out of your whole infrastructure, Reyes has a few safeguards against applying its rules to traffic that did not travel over IPsec.

Caveat emptor: Due to the use of iptables-restore, Reyes assumes that there are no other rules in the IPTables filter table (that includes the INPUT, OUTPUT, and FORWARD chains). All other rules in the filter table will be silently discarded. Reyes followers should not be run on servers that rely on having other rules in the filter table. (This may be addressed in a future release, but for now there is only support for a flag file to temporarily disable Reyes in the event that you need to change the filter table by some other process.)

Getting Started

Questions? Feedback? Please email us! There may be hidden assumptions or gotchas specific to Stripe's infrastructure even though we strove to write it in a generic way.

Reyes can be installed as a stand alone system gem: gem install reyes.

It uses a YAML configuration file to specify the S3 bucket, GPG signing key, and various information about which EC2 regions and VPCs to cover.

You'll want to generate a GPG signing key and place it in a keyring in some directory referred to by the keyring_directory config option. The Reyes leaders will need the secret keyring, while the followers should only have access to the public key. Reyes does not currently support having a passphrase on the key.

On the EC2 classic side, Reyes assumes that any given EC2 instance will be running IPsec directly, so it will ignore any traffic that didn't arrive over IPsec. (It ignores traffic that doesn't match -A INPUT -m policy --pol ipsec.)

On the VPC side, Reyes assumes that there will be dedicated VPN instances acting as IP routers, so on an EC2 instance it won't be possible to filter by policy IPsec. Instead, Reyes will filter all traffic in 10.0.0.0/8, whitelisting the CIDR block of the current VPC so that all VPC-local traffic is allowed through (it should be filtered by normal security group rules).

To configure security groups properly for Reyes in VPC, allow all traffic from foreign VPC (or EC2 classic) CIDR blocks that will be routed by IPsec VPN servers. Make sure that this security group is excluded in config.yaml by excluded_group_names, or else Reyes will dutifully mirror the allow rules and you will have no firewall whatsoever.

It is strongly recommended to use --log-accept and --log-drop during testing. This will add IPTables logging so that you can validate that the firewall is taking the expected action on traffic.

License

Reyes is distribued under the terms of the MIT license, which can be found in this repository in the file called LICENSE.