Valkyrie
Valkyrie is a gem for enabling multiple backends for storage of files and metadata in Samvera.
Primary Contacts
Product Owner
Technical Lead
Installation
Add this line to your application's Gemfile:
gem 'valkyrie'
And then execute:
$ bundle
Configuration
Valkyrie is configured in two places: an initializer that registers the persistence options and a YAML configuration file that sets which options are used by default in which environments.
Sample initializer: config/initializers/valkyrie.rb
:
Here is a sample initializer that registers a couple adapters and storage adapters, in each case linking an instance with a short name that can be used to refer to it in your application:
# frozen_string_literal: true
require 'valkyrie'
Rails.application.config.to_prepare do
Valkyrie::MetadataAdapter.register(
Valkyrie::Persistence::Postgres::MetadataAdapter.new,
:postgres
)
Valkyrie::MetadataAdapter.register(
Valkyrie::Persistence::Memory::MetadataAdapter.new,
:memory
)
Valkyrie::StorageAdapter.register(
Valkyrie::Storage::Disk.new(base_path: Rails.root.join("tmp", "files")),
:disk
)
Valkyrie::StorageAdapter.register(
Valkyrie::Storage::Fedora.new(connection: Ldp::Client.new("http://localhost:8988/rest")),
:fedora
)
Valkyrie::StorageAdapter.register(
Valkyrie::Storage::Memory.new,
:memory
)
end
The initializer registers two Valkyrie::MetadataAdapter
instances for storing metadata:
:postgres
which stores metadata in a PostgreSQL database:memory
which stores metadata in an in-memory cache (this cache is not persistent, so it is only appropriate for testing)
Other adapter options include Valkyrie::Persistence::BufferedPersister
for buffering in memory before bulk
updating another persister, Valkyrie::Persistence::CompositePersister
for storing in more than one adapter
at once, Valkyrie::Persistence::Solr
for storing in Solr, and Valkyrie::Persistence::Fedora
for storing in Fedora.
The initializer also registers three Valkyrie::StorageAdapter
instances for storing files:
:disk
which stores files on disk:fedora
which stores files in Fedora:memory
which stores files in an in-memory cache (again, not persistent, so this is only appropriate for testing)
Sample configuration: config/valkyrie.yml
:
A sample configuration file that configures your application to use different adapters:
development:
metadata_adapter: postgres
storage_adapter: disk
test:
metadata_adapter: memory
storage_adapter: memory
production:
metadata_adapter: postgres
storage_adapter: fedora
For each environment, you must set two values:
metadata_adapter
is the store where Valkyrie will put the metadatastorage_adapter
is the store where Valkyrie will put the files
The values are the short names used in your initializer.
Further details can be found on the the Wiki.
Usage
The Public API
Valkyrie's public API is defined by the shared specs that are used to test each of its core classes. This include change sets, resources, persisters, adapters, and queries. When creating your own kinds of these kinds of classes, you should use these shared specs to test your classes for conformance to Valkyrie's API.
When breaking changes are introduced, necessitating a major version change, the shared specs will reflect this. When new features are added and a minor version is released there will be no change to the existing shared specs, but there may be new ones. These new shared specs will fail in your application if you have custom adapters, but your application will still work.
Using the shared specs in your own models is described in more detail.
Define a Custom Work
Define a custom work class:
# frozen_string_literal: true
class MyModel < Valkyrie::Resource
include Valkyrie::Resource::AccessControls
attribute :title, Valkyrie::Types::Set # Sets deduplicate values
attribute :date, Valkyrie::Types::Array # Arrays can contain duplicate values
end
Attributes are unordered by default. Adding ordered: true
to an attribute definition will preserve the
order of multiple values.
attribute :authors, Valkyrie::Types::Array.(ordered: true)
Defining resource attributes is explained in greater detail within the Wiki.
Work Types Generator
To create a custom Valkyrie model in your application, you can use the Rails generator. For example, to
generate a model named FooBar
with an unordered title
field and an ordered member_ids
field:
rails generate valkyrie:resource FooBar title member_ids:array
You can namespace your model class by including a slash in the model name:
rails generate valkyrie:resource Foo/Bar title member_ids:array
Read and Write Data
# initialize a metadata adapter
adapter = Valkyrie::MetadataAdapter.find(:postgres)
# create an object
object1 = MyModel.new title: 'My Cool Object', authors: ['Jones, Alice', 'Smith, Bob']
object1 = adapter.persister.save(resource: object1)
# load an object from the database
object2 = adapter.query_service.find_by(id: object1.id)
# load all objects
objects = adapter.query_service.find_all
# load all MyModel objects
Valkyrie.config..query_service.find_all_of_model(model: MyModel)
The usage of ChangeSets
in writing data are further documented here.
Concurrency Support (Optimistic Locking)
By default, it is assumed that a Valkyrie repository implementation shall use a solution supporting concurrent updates for resources (multiple resources can be updated simultaneously using a Gem such as Sidekiq). In order to handle the possibility of multiple updates applied to the same resource corrupting data, Valkyrie supports optimistic locking. For further details, please reference the overview of optimistic locking for Valkyrie resources.
Installing a Development environment
Without Docker
External Requirements
- PostgreSQL with the uuid-ossp extension.
- Note: Enabling uuid-ossp requires database superuser privileges.
- From
psql
:alter user [username] with superuser;
To run the test suite
- Start Solr and Fedora servers for testing with
rake server:test
- Run
rake db:create
(First time only) - Run
rake db:migrate
With Docker
External Requirements
- Docker version >= 17.09.0 * ### Dependency Setup (Mac OSX)
brew install docker
brew install docker-machine
brew install docker-compose
Starting Docker (Mac OSX)
docker-machine create default
docker-machine start default
eval "$(docker-machine env)"
Starting the development mode dependencies
- Start Solr, Fedora, and PostgreSQL with
rake docker:dev:daemon
(orrake docker:dev:up
in a separate shell to run them in the foreground) - Run
rake db:create db:migrate
to initialize the database - Develop!
- Run
rake docker:dev:down
to stop the server stack- Development servers maintain data between runs. To clean them out, run
rake docker:dev:clean
- Development servers maintain data between runs. To clean them out, run
To run the test suite with all dependencies in one go
rake docker:spec
To run the test suite manually
- Start Solr, Fedora, and PostgreSQL with
rake docker:test:daemon
(orrake docker:test:up
in a separate shell to run them in the foreground) - Run
rake db:create db:migrate
to initialize the database - Run the gem's RSpec test suite with
rspec spec
orrake
- Run
rake docker:test:down
to stop the server stack- The test stack cleans up after itself on exit.
The development and test stacks use fully contained virtual volumes and bind all services to different ports, so they can be running at the same time without issue.
Get Help
If you have any questions regarding Valkyrie you can send a message to the
Samvera community tech list or the #valkyrie
channel in the Samvera community Slack
team.
License
Valkyrie is available under the Apache 2.0 license.
Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/samvera-labs/valkyrie/.