symmetric-encryption Build Status


Any project that wants to meet PCI compliance has to ensure that the data is encrypted whilst in flight and at rest. Amongst many other other requirements all passwords in configuration files have to be encrypted

This Gem helps achieve compliance by supporting encryption of data in a simple and consistent way

Symmetric Encryption uses OpenSSL to encrypt and decrypt data, and can therefore expose all the encryption algorithms supported by OpenSSL.


Many solutions that encrypt data require the encryption keys to be stored in the applications source code or leave it up to the developer to secure the keys on the application servers. symmetric-encryption takes care of securing the symmetric encryption keys.

The following steps are used to secure the symmetric encryption keys using symmetric-encryption:

  • Symmetric Encryption keys are stored in files that are not part of the application, its source code, or even stored in its source control system. These files can be created, managed and further secured by System Administrators. This prevents developers having or needing to have access to the symmetric encryption keys
  • The Operating System security features limit access to the Symmetric Encryption key files to System Administrators and the userid under which the Rails application runs.
  • The files in which the Symmetric Encryption keys are stored are further encrypted using RSA 2048 bit encryption

In order for anyone to decrypt the data being encrypted in the database, they would need access to ALL of the following: * A copy of the files containing the Symmetric Encryption Keys which are secured by the Operating System * The application source code containing the RSA private key to decrypt the above files * The userid and password for the database to copy the encrypted data itself, or an unsecured copy or export of the database contents

A major feature of symmetric encryption is that it makes the encryption and decryption automatically available when the Rails application is started. This includes all rake tasks and the Rails console. In this way data can be encrypted or decrypted as part of any rake task.

From a security perspective it is important then to properly secure the system so that no hacker can switch to and run as the rails user and thereby gain access to the encryption and decryption capabilities

It is not necessary to encrypt the IV (initialization vector), and it can be placed directly in the configuration file. The encryption key must be kept secure and must never be placed in the configuration file or other Rails source file in production. The IV should be generated using the rails generator described below to ensure it is a truly random key from the key space.


By default symmetric encryption uses the same initialization vector (IV) and encryption key to encrypt data using the SymmetricEncryption.encrypt call. This technique is required in cases where the encrypted data is used as a key to lookup for example a Social Security Number, since for the same input data it must always return the same encrypted result. The drawback is that this technique is not considered secure when encypting large amounts of data.

For non-key fields, such as storing encrypted raw responses, use the :random_iv => true option where possible so that a randomly generated IV is used and included in every encrypted string.

The Symmetric Encryption streaming interface SymmetricEncryption::Writer avoids this problem by using a random IV and key in every file/stream by default. The random IV and key are stored in the header of the output stream so that it is available when reading back the encrypted file/stream. The key is placed in a header on the file in encrypted form using the current global key/cipher.

The ActiveRecord attr_encrypted method supports the :random_iv => true option. Similarly for MongoMapper and Mongoid the :random_iv => true option can be added.

Note that encrypting the same input string with the same key and :random_iv => true option will result in different encrypted output every time it is encrypted.


  • Encryption of passwords in configuration files
  • Encryption of ActiveRecord model attributes by prefixing attributes / column names with encrypted_
  • Encryption of MongoMapper keys by using :encrypted_key
  • Encryption of Mongoid model fields by adding :encrypted option to field definitions
  • Externalization of symmetric encryption keys so that they are not in the source code, or the source code control system
  • Validator for ActiveRecord Models to ensure fields contain encrypted data
  • Stream based encryption and decryption so that large files can be read or written with encryption, along with a random key and IV for every file
  • Stream based encryption and decryption also supports compression and decompression on the fly
  • When :compress => true option is specified Symmetric Encryption will transparently compress the data prior to decryption. When decrypting compressed data Symmetric Encryption will transparently decompress the data after decryption based on the header stored in the encrypted data
  • Uses built-in support in Ruby for OpenSSL and Zlib for high performance and maximum portability without introducing any additional dependencies
  • Drop in replacement for attr_encrypted. Just remove the attr_encrypted gem
  • For maximum security supports fully random keys and initialization vectors extracted from the entire encryption key space


  • Add the encryption header to all encrypted strings. See the always_add_header option in the configuration file.

  • Add random_iv: true for all ActiveRecord attributes, MongoMapper keys, and Mongoid fields which are not used in indexes and will not be used as part of a query.

Binary Data

On decryption an attempt is made to encode the data as UTF-8, if it fails it will be returned as BINARY encoded.


Encryption Example

SymmetricEncryption.encrypt "Sensitive data"

Decryption Example

SymmetricEncryption.decrypt "JqLJOi6dNjWI9kX9lSL1XQ==\n"

ActiveRecord Example

class User < ActiveRecord::Base
  # Requires table users to have a column called encrypted_bank_account_number
  attr_encrypted :bank_account_number

  # Requires users table to have a column called encrypted_social_security_number
  # Note: Encrypting the same value twice will result in the _same_ encrypted value
  #       when :random_iv => false, or is not specified
  attr_encrypted :social_security_number

  # By specifying the type as :integer the value will be returned as an integer and
  # can be set as an integer, even though it is stored in the database as an
  # encrypted string
  # Requires users table to have a column called encrypted_age of type string
  attr_encrypted :age,         type: integer

  # Since string and long_string are not used in the where clause of any SQL
  # queries it is better to ensure that the encrypted value is always different
  # by encrypting every value with a random Initialization Vector.
  # Note: Encrypting the same value twice will result in different encrypted
  #       values when :random_iv is true
  attr_encrypted :string,      random_iv: true

  # Long encrypted strings can also be compressed prior to encryption to save
  # disk space
  attr_encrypted :long_string, random_iv: true, compress: true

  # By specifying the type as :json the value will be serialized to JSON
  # before encryption and deserialized from JSON after decryption.
  # It is sometimes useful to use compression on large fields, so we can enable
  # compression before the string is encrypted
  # Requires users table to have a column called encrypted_values of type string
  attr_encrypted :values,      type: :json, compress: true

  validates :encrypted_bank_account_number, symmetric_encryption: true
  validates :encrypted_social_security_number, symmetric_encryption: true

# Create a new user instance assigning a bank account number
user =
user. = '12345'

# Saves the bank_account_number in the column encrypted_bank_account_number in
# encrypted form!

# Short example using create
User.create(bank_account_number: '12345')

Several types are supported for ActiveRecord models when encrypting or decrypting data. Each type maps to the built-in Ruby types as follows:

  • :string => String
  • :integer => Integer
  • :float => Float
  • :decimal => BigDecimal
  • :datetime => DateTime
  • :time => Time
  • :date => Date
  • :json => Uses JSON serialization, useful for hashes and arrays
  • :yaml => Uses YAML serialization, useful for hashes and arrays

MongoMapper Example

To encrypt a field in a MongoMapper document, use encrypted_key instead of key when specifying a key.

  field :encrypted_age,                    type: String, encrypted: {type: :integer}

# User model MongoMapper
class User
  include MongoMapper::Document

  key           :name,                   String
  encrypted_key :bank_account_number,    String
  encrypted_key :social_security_number, String
  encrypted_key :life_history,           String, encrypted: { random_iv: true, compress: true }

  # Encrypted fields are _always_ stored in Mongo as a String
  # To get the result back as an Integer, Symmetric Encryption will automatically
  # perform the necessary conversions
  encrypted_key :integer_value,          Integer
  encrypted_key :float_value,            Float
  encrypted_key :decimal_value,          BigDecimal
  encrypted_key :datetime_value,         DateTime
  encrypted_key :time_value,             Time
  encrypted_key :date_value,             Date
  encrypted_key :true_value,             Boolean
  encrypted_key :data_json,              Hash, encrypted: {random_iv: true, compress: true}
  # By default Hash is saved as JSON, to save as YAML add the type specifier:
  encrypted_key :data_yaml,              Hash, encrypted: {random_iv: true, compress: true, type: :yaml}

  # Optionally add validation to ensure that encrypted fields are in fact encrypted
  # before the data is saved
  validates :encrypted_bank_account_number,    symmetric_encryption: true
  validates :encrypted_social_security_number, symmetric_encryption: true

# Create a new user document
User.create(bank_account_number: '12345')

# When finding a document, always use the encrypted form of the field name
user = User.where(encrypted_bank_account_number: SymmetricEncryption.encrypt('12345')).first

# Fields can be accessed using their unencrypted names
puts user.bank_account_number

Mongoid Example

To encrypt a field in a Mongoid document, just add "encrypted: true" at the end of the field specifier. The field name must currently begin with "encrypted_"

# User model in Mongoid
class User
  include Mongoid::Document

  field :name,                             type: String
  field :encrypted_bank_account_number,    type: String,  encrypted: true
  field :encrypted_social_security_number, type: String,  encrypted: true
  field :encrypted_life_history,           type: String,  encrypted: {compress: true, random_iv: true}

  # Encrypted fields are _always_ stored in Mongo as a String
  # To get the result back as an Integer, Symmetric Encryption can do the
  # necessary conversions by specifying the internal type as an option
  # to :encrypted
  # #see SymmetricEncryption::COERCION_TYPES for full list of types
  field :encrypted_age,                    type: String, encrypted: {type: :integer}

# Create a new user document
User.create(bank_account_number: '12345')

# When finding a document, always use the encrypted form of the field name
user = User.where(encrypted_bank_account_number: SymmetricEncryption.encrypt('12345')).first

# Fields can be accessed using their unencrypted names
puts user.

Validation Example

class MyModel < ActiveRecord::Base
  validates :encrypted_ssn, symmetric_encryption: true

m =
#  => false
m.encrypted_ssn = SymmetricEncryption.encrypt('123456789')
#  => true

Encrypting Passwords in configuration files

Passwords can be encrypted in any YAML configuration file.

For example config/database.yml

  adapter:  mysql
  host:     db1w
  database: myapp_production
  username: admin
  password: <%= SymmetricEncryption.try_decrypt "JqLJOi6dNjWI9kX9lSL1XQ==\n" %>

Note: Use SymmetricEncryption.try_decrypt method which will return nil if it fails to decrypt the value, which is essential when the encryption keys differ between environments

Note: In order for the above technique to work in other YAML configuration files the YAML file must be processed using ERB prior to passing to YAML. For example

    config_file = Rails.root.join('config', 'redis.yml')
    raise "redis config not found. Create a config file at: config/redis.yml" unless config_file.file?

    cfg = YAML.load([Rails.env]
    raise("Environment #{Rails.env} not defined in redis.yml") unless cfg

Large File Encryption

Example: Read and decrypt a line at a time from a file'encrypted_file') do |file|
  file.each_line do |line|
     puts line

Example: Encrypt and write data to a file'encrypted_file') do |file|
  file.write "Hello World\n"
  file.write "Keep this secret"

Example: Compress, Encrypt and write data to a file'', compress: true) do |file|
  file.write "Hello World\n"
  file.write "Compress this\n"
  file.write "Keep this safe and secure\n"

Standalone test

Before generating keys we can use SymmetricEncryption in a standalone test environment:

# Use test encryption keys
SymmetricEncryption.cipher =
  key:         '1234567890ABCDEF1234567890ABCDEF',
  iv:          '1234567890ABCDEF',
  cipher_name: 'aes-128-cbc'
encrypted = SymmetricEncryption.encrypt('hello world')
puts SymmetricEncryption.decrypt(encrypted)

Rake Tasks

For PCI compliance developers should not be the ones creating or encrypting passwords. The following rake tasks can be used by system administrators to generate and encrypt passwords for databases, or external web calls. It is safe to pass the encrypted password for say MySQL to the developers who can then put it in the config files which are kept in source control.

Generate a random password and display its encrypted form:

rake symmetric_encryption:random_password

Encrypt a known value, such as a password:

rake symmetric_encryption:encrypt

Note: Passwords must be encrypted in the environment in which they will be used. Since each environment should have its own symmetric encryption keys

Note: To use the rake task 'symmetric_encryption:encrypt' the gem 'highline' must first be installed by adding to bundler or installing directly:

gem install 'highline'

Encrypt a file

INFILE="Gemfile.lock" OUTFILE="Gemfile.lock.encrypted" rake symmetric_encryption:encrypt_file

Encrypt and compress a file

INFILE="Gemfile.lock" OUTFILE="Gemfile.lock.encrypted" COMPRESS=1 rake symmetric_encryption:encrypt_file

Decrypt a file encrypted and optionally compressed using symmetric encryption

INFILE="Gemfile.lock.encrypted" OUTFILE="Gemfile.lock2" rake symmetric_encryption:decrypt_file

When decrypting a compressed file it is not necessary to specify whether the file was compressed since the header embedded in the file will indicate whether it was compressed

The file header also contains a random key and iv used to encrypt the files contents. The key and iv is encrypted with the global encryption key being used by the symmetric encryption installation.


  • Ruby 1.9.3 (or above) Or, JRuby 1.7.3 (or above)
  • Optional: To log to MongoDB, Mongo Ruby Driver 1.5.2 or above


Add to an existing Rails project

Add the following line to Gemfile

gem 'symmetric-encryption'

Install the Gem with bundler

bundle install

Rails Configuration

If deploying to Heroku skip to the section "Rails Configuration for a Heroku deployment" below

Creating the configuration file

The configuration file contains the path to the production encryption key files. Generally in development and test the files are not created, so supply the full path to these files in production. Once the config file has been generated it can be modified as needed.

Generate the configuration file:

rails generate symmetric_encryption:config /etc/rails/keys

Note: Ignore the warning about "Symmetric Encryption config not found" since it is being generated

Save to version control

This configuration file should be checked into the source code control system. It does Not include the Symmetric Encryption keys. They will be generated in the next step.

Generating and securing the Symmetric Encryption keys

Once development and testing is complete we need to generate secure encryption key files for production. It is recommended that the step below be run on only one of the production servers. The generated key files must then be copied to all the production web servers.

Note: Do not run this step more than once, otherwise new keys will be generated and any encrypted data will no longer be accessible.

Note: Do not run this step on more than one server in each environment otherwise each server will be encrypting with it's own key and the servers will not be able to decrypt data encrypted on another server. Just copy the generated files to each server

The symmetric encryption key consists of the key itself and an optional initialization vector.

To generate the keys run the following Rake task once only in each environment:

rails generate symmetric_encryption:new_keys production

Replace 'production' as necessary for each environment.

Make sure that the current user has read and write access to the folder listed in the config file option key_filename.

Note: Ignore the warning about the key files "not found or readable" since they are being generated

Once the Symmetric Encryption keys have been generated, secure them further by making the files read-only to the Rails user and not readable by any other user. Change ownership of the keys to the rails user and only give it access to read the key files:

chown rails /etc/rails/keys/*
chmod 0400 /etc/rails/keys/*

Change 'rails' above to the userid under which your Rails processes are run and update the path to the one supplied when generating the config file or look in the config file itself

When running multiple Rails servers in a particular environment copy the same key files to every server in that environment. I.e. All Rails servers in each environment must run the same encryption keys.

Note: The generate step above must only be run once in each environment

Rails Configuration for a Heroku deployment

Deploying to Heroku requires the encrypted key to be stored in an environment variable rather than as a file on disk.

Generate the configuration file:

rails g symmetric_encryption:heroku_config

Note: Ignore the warning about "Symmetric Encryption config not found" since it is being generated.

Note: The encrypted keys for the release and production environments are displayed on screen and must be entered manually as environment variables into Heroku so that the application can find them when it starts.

Save to version control

This configuration file should be checked into the source code control system. It does Not include the Symmetric Encryption keys.

Using in non-Rails environments

SymmetricEncryption can also be used in non-Rails environment.

Install SymmetricEncryption

gem install symmetric-encryption

Manually create a symmetric-encryption.yml configuration file based on the one supplied in examples/symmetric-encryption.yml.

At application startup, run the code below to initialize symmetric-encryption prior to attempting to encrypt or decrypt any data

require 'symmetric-encryption'
SymmetricEncryption.load!('config/symmetric-encryption.yml', 'production')


  • Filename of the configuration file created above
  • Name of the environment to load the configuration for

To manually generate the symmetric encryption keys, run the code below

require 'symmetric-encryption'
SymmetricEncryption.generate_symmetric_key_files('config/symmetric-encryption.yml', 'production')


  • Filename of the configuration file created above
  • Name of the environment to load the configuration for

Supporting Multiple Encryption Keys

According to the PCI Compliance documentation: "Cryptographic keys must be changed on an annual basis."

During the transition period of moving from one encryption key to another symmetric-encryption supports multiple Symmetric Encryption keys. If decryption with the current key fails, any previous keys will also be tried automatically.

By default the latest key is used for encrypting data. Another key can be specified for encryption so that old data can be looked in queries, etc.

Since just the Symmetric Encryption keys are being changed, we can still continue to use the same RSA Private key for gaining access to the Symmetric Encryption Keys

Configuring multiple Symmetric Encryption keys

Create a configuration file in config/symmetric-encryption.yml per the following example:

# Symmetric Encryption for Ruby
# For the development and test environments the test symmetric encryption keys
# can be placed directly in the source code.
# And therefore no RSA private key is required
development: &development_defaults
  key:    1234567890ABCDEF1234567890ABCDEF
  iv:     1234567890ABCDEF
  cipher_name: aes-128-cbc

  <<: *development_defaults

  # Since the key to encrypt and decrypt with must NOT be stored along with the
  # source code, we only hold a RSA key that is used to unlock the file
  # containing the actual symmetric encryption key
  # Sample RSA Key, DO NOT use this RSA key, generate a new one using
  #    openssl genrsa 2048
  private_rsa_key: |
     -----BEGIN RSA PRIVATE KEY-----
     -----END RSA PRIVATE KEY-----

  # List Symmetric Key files in the order of current / latest first
        # Filename containing Symmetric Encryption Key encrypted using the
        # RSA public key derived from the private key above
        key_filename: /etc/rails/.rails.key
        iv_filename:  /etc/rails/.rails.iv

        # Encryption cipher_name
        #   Recommended values:
        #      aes-256-cbc
        #         256 AES CBC Algorithm. Very strong
        #         Ruby 1.8.7 MRI Approximately 100,000 encryptions or decryptions per second
        #         JRuby 1.6.7 with Ruby 1.8.7 Approximately 22,000 encryptions or decryptions per second
        #      aes-128-cbc
        #         128 AES CBC Algorithm. Less strong.
        #         Ruby 1.8.7 MRI Approximately 100,000 encryptions or decryptions per second
        #         JRuby 1.6.7 with Ruby 1.8.7 Approximately 22,000 encryptions or decryptions per second
        cipher_name:  aes-256-cbc

        # OPTIONAL:
        # Any previous Symmetric Encryption Keys
        # Only used when old data still exists that requires old decryption keys
        # to be used
        key_filename: /etc/rails/.rails_old.key
        iv_filename:  /etc/rails/.rails_old.iv
        cipher_name:  aes-256-cbc

New features in V1.1 and V2

  • Ability to randomly generate a new initialization vector (iv) with every encryption and put the iv in the encrypted data as its header, without having to use SymmetricEncryption::Writer

  • With file encryption randomly generate a new key and initialization vector (iv) with every file encryption and put the key and iv in the encrypted data as its header which is encrypted using the global key and iv

  • Support for compression via SymmetricEncryption.encrypt, attr_encrypted and Mongoid fields

  • SymmetricEncryption.encrypt has two additional optional parameters: ``` random_iv [true|false] Whether the encypted value should use a random IV every time the field is encrypted. It is recommended to set this to true where feasible. If the encrypted value could be used as part of a SQL where clause, or as part of any lookup, then it must be false. Setting random_iv to true will result in a different encrypted output for the same input string. Note: Only set to true if the field will never be used as part of the where clause in an SQL query. Note: When random_iv is true it will add a 8 byte header, plus the bytes to store the random IV in every returned encrypted string, prior to the encoding if any. Default: false Highly Recommended where feasible: true

compress [true|false] Whether to compress str before encryption Should only be used for large strings since compression overhead and the overhead of adding the 'magic' header may exceed any benefits of compression Note: Adds a 6 byte header prior to encoding, only if :random_iv is false Default: false ```

Upgrading from earlier versions to SymmetricEncryption V3

In version 3 of SymmetricEncryption, the following changes have been made that may have backward compatibility issues:

  • SymmetricEncryption.decrypt no longer rotates through all the decryption keys when previous ciphers fail to decrypt the encrypted string. In a very small, yet significant number of cases it was possible to decrypt data using the incorrect key. Clearly the data returned was garbage, but it still returned a string of data instead of throwing an exception. See SymmetricEncryption.select_cipher to supply your own custom logic to determine the correct cipher to use when the encrypted string does not have a header and multiple ciphers are defined.

  • Configuration file format prior to V1 is no longer supported

  • New configuration option has been added to support setting encryption keys from environment variables

  • Cipher.parse_magic_header! now returns a Struct instead of an Array

  • New config options :encrypted_key and :encrypted_iv to support setting the encryption key in environment variables


This project uses Semantic Versioning.


Reid Morrison



Copyright 2012, 2013, 2014 Reid Morrison

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.


Although this library has assisted in meeting PCI Compliance and has passed previous PCI audits, it in no way guarantees that PCI Compliance will be achieved by anyone using this library.