Purrrge

Purrrge is a data retention and scrubbing library for Ruby applications. It provides tools for implementing data retention policies and scrubbing sensitive data in compliance with privacy regulations.

Installation

Add this line to your application's Gemfile:

gem 'purrrge'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install purrrge

Usage

Scrubbable Concern

The Scrubbable concern allows you to define fields in your models that should be scrubbed (anonymized or removed) when data retention periods expire. This makes it easy to implement data retention policies across your application.

Basic Usage

class User
  include Purrrge::Scrubbable
  
  # Define fields to be scrubbed and how they should be scrubbed
  scrub_field :email, scrub_value: nil                   # Set to nil
  scrub_field :name, scrub_value: "REDACTED"             # Replace with static text
  scrub_field :phone, scrub_method: :anonymize_phone     # Use custom method
  
  # Define custom scrubbing method
  def anonymize_phone(field)
    self[field] = "xxx-xxx-xxxx"
  end
end

Implementing Scrubbing Logic

You can scrub individual records by calling scrub! on them:

user = User.find(123)
user.scrub!  # Apply scrubbing to this user

For scrubbing based on retention policies, you might implement a background job:

class DataRetentionWorker
  include Sidekiq::Worker
  
  def perform
    Organization.where.not(data_retention_period: nil).find_each do |org|
      cutoff_date = org.data_retention_period.days.ago
      
      # Find users to be scrubbed for this organization
      User.where(organization_id: org.id)
          .where('created_at < ?', cutoff_date)
          .find_each do |user|
        user.scrub!
      end
    end
  end
end

Tracking Scrubbed Records

By default, Purrrge will scrub records based on their creation date and any other criteria you define. However, this can lead to inefficiency as the same records may be processed in each scrubbing run.

To optimize this process, Purrrge supports tracking scrubbed records with a scrubbed_at timestamp.

Adding the scrubbed_at Column

Add a scrubbed_at datetime column to your model:

# Generate the migration
rails generate purrrge:add_scrubbed_at MODEL_NAME

# Run the migration
rails db:migrate

This creates a migration that adds both the column and an index for performance:

add_column :your_table_name, :scrubbed_at, :datetime
add_index :your_table_name, :scrubbed_at

How It Works

When the scrubbed_at column exists:

1. Purrrge automatically sets scrubbed_at = Time.current when scrub! is called on a record
2. Your query scopes can filter out already-scrubbed records with WHERE scrubbed_at IS NULL

Implementation Example

# Model definition
class User < ApplicationRecord
  include Purrrge::Scrubbable

  # Define fields to scrub
  scrub_field :email, scrub_value: nil
  scrub_field :name, scrub_value: "REDACTED"
  scrub_field :phone, scrub_value: "xxx-xxx-xxxx"

  # Optional: Add your own custom callbacks to run after scrubbing
  after_scrub :log_scrubbing_event

  private

  def log_scrubbing_event
    Rails.logger.info "User #{id} was scrubbed at #{scrubbed_at}"
  end
end

# In your scrubbing service
def scrub_user_data(organization, retention_date)
  # Only select records that haven't been scrubbed yet
  users = organization.users.where("created_at < ? AND scrubbed_at IS NULL", retention_date)

  users.find_in_batches do |batch|
    batch.each(&:scrub!)
  end
end

Fallback Behavior

If the scrubbed_at column doesn't exist on a model, Purrrge will still work normally - it just won't track which records have been scrubbed.

Registry for Automatic Model Discovery

Purrrge includes a Registry system that automatically keeps track of all models that include the Scrubbable concern. This makes it easy to implement application-wide data retention policies.

How it Works

When a model includes the Purrrge::Scrubbable concern, it's automatically registered with Purrrge::Registry. You can then discover and process all scrubbable models in your application.

# Get all registered scrubbable models
scrubbable_models = Purrrge::Registry.scrubbable_models

# Discover all scrubbable models in a Rails application
# (This will eager load models and find those that include Purrrge::Scrubbable)
all_models = Purrrge::Registry.discover_models

Using the Registry

The Registry is designed to be application-agnostic and doesn't impose any specific data model. Here are some general ways to use it:

# Get all registered scrubbable models
models = Purrrge::Registry.scrubbable_models

# Process each model according to your application's requirements
models.each do |model|
  # Implement your application-specific logic here
  # This could involve filtering records based on dates, categories,
  # or any other business logic relevant to your application
  
  # Example: Finding records older than a certain date
  if model.respond_to?(:created_at)
    records = model.where('created_at < ?', 1.year.ago)
    records.find_in_batches(batch_size: 1000) do |batch|
      batch.each(&:scrub!)
    end
  end
end

Extending with Custom Behavior

You can define custom class methods on your models to standardize how records are selected for scrubbing:

# Example custom methods for your models
class User < ApplicationRecord
  include Purrrge::Scrubbable
  
  scrub_field :email, scrub_value: nil
  
  # Custom method for finding records to scrub
  def self.scrubbable_records(cutoff_date)
    where('created_at < ?', cutoff_date)
  end
end

# Using the custom method in a worker
Purrrge::Registry.scrubbable_models.each do |model|
  if model.respond_to?(:scrubbable_records)
    records = model.scrubbable_records(1.year.ago)
    records.find_in_batches(batch_size: 1000) do |batch|
      batch.each(&:scrub!)
    end
  end
end

Testing the Registry

For testing purposes, you can clear the registry:

# Clear registry between tests
Purrrge::Registry.clear

Configuration Options

When defining fields to be scrubbed, you have several options:

1. Set to a specific value:

   scrub_field :email, scrub_value: nil
   scrub_field :name, scrub_value: "REDACTED"
   

2. Use a custom method:

   scrub_field :address, scrub_method: :anonymize_address
   
   def anonymize_address(field)
     self[field] = "#{self.country}, #{self.state}" # Keep only country and state
   end
   

Callback Hooks

Purrrge provides callback hooks that allow you to execute code before and after scrubbing occurs:

class User
  include Purrrge::Scrubbable
  
  scrub_field :email, scrub_value: nil
  scrub_field :name, scrub_value: "REDACTED"
  
  # Register callbacks
  before_scrub :log_scrubbing_event
  before_scrub :notify_admin, if: :admin_user?
  after_scrub :update_scrubbed_timestamp
  
  private
  
  def log_scrubbing_event
    Rails.logger.info("Scrubbing user data for User ##{id}")
    # Return false to halt the callback chain and prevent scrubbing
    true
  end
  
  def notify_admin
    AdminMailer.user_data_scrubbed(self).deliver_later
  end
  
  def admin_user?
    role == 'admin'
  end
  
  def update_scrubbed_timestamp
    update_column(:scrubbed_at, Time.current)
  end
end

Callbacks support conditions via :if and :unless options, which can be:

• Symbol naming an instance method
• Proc or lambda
• Array of symbols or procs

If a before_scrub callback returns false, the scrubbing operation will be halted.

Working with Different ORMs

Purrrge is designed to be ORM-agnostic. While it works great with ActiveRecord, you can use it with any Ruby class that:

1. Has attribute accessors for the fields you want to scrub
2. Implements a save method

For non-ActiveRecord classes, just ensure you implement these requirements:

class MyCustomModel
  include Purrrge::Scrubbable
  
  attr_accessor :name, :email
  
  scrub_field :email, scrub_value: nil
  
  def save(options = {})
    # Your custom save implementation
    true
  end
end

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Semantic Versioning

The gem follows Semantic Versioning principles. When creating pull requests, please indicate the type of change:

• PATCH version for backwards-compatible bug fixes
• MINOR version for backwards-compatible new features
• MAJOR version for breaking changes

Our CI system will automatically determine the appropriate version bump based on commit messages and PR descriptions.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/grayscaleapp/purrrge.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add some amazing feature')
   • Use conventional commit messages: feat:, fix:, docs:, etc.
   • Prefix breaking changes with BREAKING CHANGE: or include MAJOR in the description
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request using the provided template