UniverseCompiler

Overview

The goal of this gem is to provide a simple way to manage a consistent highly complex configuration.

The configuration can be split into lot of objects (or entities) and complex relations and constraints can be defined between them using a-la-ActiveRecord relationships like has_many or is_array (see complete list in lib/universe_compiler/entity/field_constraint_management.rb).

These entities are added to a so-called universe. See a universe as a kind of sandbox where entities exist.

A universe could be persisted to any kind of backend by writing an persistence engine, yet only a yaml persistence engine is available by default in the gem.

A universe can be compiled in order to produce a new universe where all constraints and relations defined by these entities have been resolved.

Installation

Add this line to your application's Gemfile:

gem 'universe_compiler'

And then execute:

$ bundle

Or install it yourself as:

$ gem install universe_compiler

Core Concepts

Entities

Overview

Any entity you will create will basically inherit from UniverseCompiler::Entity::Base. This will allow the following kind of code:

class EntityA < UniverseCompiler::Entity::Base
  entity_type :some_entity

  field :some_data, :is_hash
  field :bar, :not_null, should_match: /^Y/
  field :stupid

  has_one EntityA, name: :master

end

class EntityB < UniverseCompiler::Entity::Base

  has_many EntityA, name: :some_entities
  not_empty :some_entities

end

The core concept is that every entity has a hash property named fields. And any field declared using the declaration mechanism as above just adds some content validations mechanisms and direct accessors to the internal fields hash.

For example:

a = EntityA.new fields: { stupid: :foo}

a.stupid # => :foo 
a.stupid == a[:stupid] # => true
a.stupid == a.fields[:stupid] # => true
a.stupid == a['stupid'] # => false, a String is not a Symbol

Therefore the pseudo schema you defined when declaring the class is not limiting... you can always do

a[:non_explicitely_declared_property] = :bar
a[:non_explicitely_declared_property] # => :bar
# but
a.non_explicitely_declared_property # => NoMethodError: undefined method `non_existing_property' for...

Every entity has a valid? method which performs various checks. For example here we have said that an instance of EntityB has many entities of type EntityA. You can really see a has_many relationship an the definition of an Array which content is validated:

b = EntityB.new
b.valid? # => false
b.some_entities << :foo
b.valid? # => false, :foo is not of the expected type
b.clear
b.valid? # => false
b.some_entities << b
b.valid? # => false, b is not of the expected type
b.clear
b.some_entities << a
b.valid? # => true, a is ok

In the same vein:

a.valid? # => false
a.some_entity = a
a.valid? # => false, still false as requiring a non null :bar property
a.bar = 'hey man'
a.valid? # => false, not compliant with regexp specified
a.bar = 'Yo man'
a.valid? # => true

Special directives

By default every entity has a type. It is available using the #type instance method or the ::entity_type class method. The default value for the entity type is coming from the class name but it can be overridden using the entity_type directive:

EntityA.entity_type # => :some_entity
EntityB.entity_type # => "entity_b"
a = EntityA.new # => #<EntityA:47429412219120 composite_key=[:some_entity, nil]>
a.type # => :some_entity
b = EntityB.new # => #<EntityB:47429411925900 composite_key=["entity_b", nil]>
b.type # => "entity_b"

The name of an entity can be automatically generated using the auto_named_entity_type directive optionally providing a seed:

class EntityC < UniverseCompiler::Entity::Base
  auto_named_entity_type
end
class EntityD < UniverseCompiler::Entity::Base
  auto_named_entity_type :my_seed
end

EntityC.new # => #<EntityC:46943375076460 composite_key=["entity_c", "entity_c_1"]>
EntityD.new # => #<EntityD:46943378308720 composite_key=["entity_d", "my_seed_1"]>
EntityD.new # => #<EntityD:46943375641700 composite_key=["entity_d", "my_seed_2"]>

Constraints directives

The generic form to declare a field is the field statement. Any constraint can be declared using the field method. Here is the signature:

def field(field_name, *options)

Then other constraint methods that can be used when describing an entity can be grouped into two. The switches:

  • not_null
  • not_empty
  • is_array
  • is_hash

Then some methods taking parameter:

  • should_match
  • class_name

So for each of these methods can be used either as "real" methods or as field parameter. For example:

class MyEntity < UniverseCompiler::Entity::Base
  field :my_field, :not_null, class_name: AClass 
end

Is strictly equivalent to:

class MyEntity < UniverseCompiler::Entity::Base
  not_null :my_field
  class_name :my_field, AClass 
end

Notice the fact that in the latter form my_field is "declared" more than once.

Relational directives

Basic relations

universe_compiler provides two relational directives

  • has_one
  • has_many

They specify relations to other entities and work both mainly the same way.

In it's simplest form you can define:

class MyEntity < UniverseCompiler::Entity::Base
  has_one :another_entity_type
  not_null :another_entity_type
  has_one AnotherEntityClass
  has_many :bar
end

:information_source: You can notice that you can specify either an entity type or an entity class.

:information_source: You can use not_null and not_empty with has_one directives, but on a separated declaration. With has_many you can use not_empty (you could use not_null but it would always be satisfied as by default a has_many relation returns an empty array).

For has_one, the accessors generated are like for field. With the previous class, for has_many the accessors are pluralized (like in activerecord).

e = MyEntity.new fields: {name: :foo}
# You can then issue
e.another_entity_type # =>nil
e.another_entity_type = ...
e.another_entity_class # =>nil
e.bars # =>[]

You can notice the has_one accessors defined using a class rather than an entity type, has been camelized. :warning: Notice the has_many directive generated pluralized accessors !

This is the default behaviour, but you can override this using the name option (for both has_one and has_many):

class MyEntity < UniverseCompiler::Entity::Base
  has_one :another_entity_type, name: :better_name
  has_many :foo, name: :bars
end

:warning: with the has_many directive if you specify a name, the accessors name is not pluralized (hence there, we specify the name as being bars and not bar).

# You can then issue
e.bettername # =>nil
e.bars # =>[]

:information_source: Of course like any other field, you can still use the internal fields:

e.bettername == e.fields[:bettername]
e.bettername == e.[:bettername]
e.bars == e[:bars]

Advanced relations

Sometimes you may want entities targeted by has_one or has_many relations to be aware of this fact. You can then implement complex relations without duplicating information.

This is called reverse methods.

class EntityA < UniverseCompiler::Entity::Base
  auto_named_entity_type
  entity_type :leaf
end

class EntityB < UniverseCompiler::Entity::Base
  entity_type :root
end

class EntityC < UniverseCompiler::Entity::Base
  entity_type :tree

  has_one :root, with_reverse_method: :tree, unique: true
  has_many :leaf, name: :leaves, with_reverse_method: :trunk, unique: true
end

:warning: When you declare a reverse method using the with_reverse_method option, an extra method is created on the target entity class, not the one containing the has_one/many directive !

It allows the following kind of code:

u = UniverseCompiler::Universe::Base.new
t = EntityC.new fields: {name: :my_tree}
u << t
(1..10).each {|_| l = EntityA.new ; t.leaves << l ; u << l }

t.leaves
#=> [#<EntityA:47410094357440 composite_key=[:leaf, #"2aad17a4-096c-4de5-9be0-ee80ef522b2b"], @universe='Unnamed #Universe'>,
# #<EntityA:47410094356680 composite_key=[:leaf, #"56500c9f-0e64-4a48-9894-0e4e485ab001"], @universe='Unnamed #Universe'>,
# #<EntityA:47410094355880 composite_key=[:leaf, #"2d5eeabb-e237-4741-8c44-0e2e9636b811"], @universe='Unnamed #Universe'>,
# #<EntityA:47410094355100 composite_key=[:leaf, #"f17d7aa1-c505-4594-889a-e3d4f8813246"], @universe='Unnamed #Universe'>,
# #<EntityA:47410094354340 composite_key=[:leaf, #"268d8a0c-cb28-42a0-85ab-2dd6217e6e6f"], @universe='Unnamed #Universe'>,
# #<EntityA:47410094353560 composite_key=[:leaf, #"8d822cde-30d4-4841-bd0a-a1570583c355"], @universe='Unnamed #Universe'>,
# #<EntityA:47410094352740 composite_key=[:leaf, #"dfffe2c2-b283-47ec-b799-3ef79c8584b4"], @universe='Unnamed #Universe'>,
# #<EntityA:47410094351940 composite_key=[:leaf, #"c8c111da-ad70-412e-9f2c-7af11894641c"], @universe='Unnamed #Universe'>,
# #<EntityA:47410094351140 composite_key=[:leaf, #"e1158988-6659-426e-912c-73f827e7429f"], @universe='Unnamed #Universe'>,
# #<EntityA:47410094350360 composite_key=[:leaf, #"64b0e4c6-a718-43b9-ae8f-beeea5145c20"], @universe='Unnamed #Universe'>]
t.leaves.last.trunk
# => #<EntityC:47410094828200 composite_key=[:tree, :my_tree], @universe='Unnamed Universe'>
t.leaves.last.fields
# => {:name=>"c881f93d-7216-49a0-a7d4-b5a2a4a314d4"}
t.leaves.last.respond_to? :trunk
# => true

You can then notice that any leaf has a new trunk method which returns the entity it is referenced from (in this case the tree entity). The fields themselves are not modified !

What happens if multiple entities reference the same entity ?

t2 = EntityC.new fields: {name: :oak}
u << t2
# And let's insert on entity A already added to t
t2.leaves << t.leaves.last
t.leaves.last.trunk
# UniverseCompiler::Error: 'leaf/18693021-c0a3-4e47-be89-291850d7a0ff#trunk' should return only one 'tree' !

An exception is returned. the unique option actually specifies that only one entity should reference it ! If you don't specify this option, an array is returned instead and this check is not performed.

Validations

Every constraint defined on a field or a relation is enforced when an entity is validated (which is as well true when saving it). Continuing on previous example:

t.leaves.last.valid?
# => false
t.leaves.last.valid? raise_error: true
# UniverseCompiler::Error: Invalid entity '[:leaf, "0ecd1283-e98e-43ec-943b-b92e2e8ffa2b"]' for fields trunk !

:information_source: Here above is just an example regarding the reverse methods but any constraint added to an entity is enforced at validation time (not_null, is_hash... all of them).

Compilation

The compilation mechanism is related to universes. When compiling a universe it actually:

  • Creates a new universe containing deep copies of its original entities.
  • Applies entities inheritance defined by the special field extends.
  • Applies overrides defined by the :entity_overide special entity type.

Here is an example

u = UniverseCompiler.new_universe
# Adding entities to universe requires they have a name
a = EntityA.new fields: { name: :a, bar: 'Yo man', stupid: :yeah } # a is valid
b = EntityA.new fields: { name: :b, extends: a } # Notice b is not valid but extends a
u << a << b

v = u.compile # v is a new universe result of the "compilation" of u
u.name # => "Unnamed Universe"
v.name # => "Unnamed Universe - COMPILED #47332840258780"

compiled_b = v.get_entity :some_entity, :b
compiled_b == b # => true, b and compiled_b although different represent the same entity
compiled_b.eql? b # => false, b and compiled_b are in different universe
compiled_b.equal? b # => false, b and compiled_b have different object_id
b.valid? # => false, in the universe u, b is still not valid
compiled_b.valid? # => true, thanks to the fact b extends a
a.fields # => {:name=>:a, :bar=>"Yo man", :stupid=>:yeah, :some_data=>{}}
b.fields # => {:name=>:b, :extends=>#<EntityA:47405166007800 composite_key=[:some_entity, :a], @universe='Unnamed Universe'>, :some_data=>{}}
compiled_b.fields # => {:name=>:b, :bar=>"Yo man", :stupid=>:yeah, :some_data=>{}, :extends=>#<EntityA:47405171122480 composite_key=[:some_entity, :a], @universe='Unnamed Universe - COMPILED #47405171131080'>}

And each entity in the new universe will have the flag compiled set to true.

u.get_entities.map do |entity|
  {name: entity.name, compiled: entity.compiled}
end
# => [{name: :a, compiled: false},{name: :b, compiled: false},{name: :c, compiled: false}]
v.get_entities.map do |entity|
  {name: entity.name, compiled: entity.compiled}
end
# => [{name: :a, compiled: true},{name: :b, compiled: true},{name: :c, compiled: true}]

Inheritance

To be clear, here we talk about entities (instances) inheritance, NOT classes !

Each entity can potentially extend (using the extends field) one entity... which itself could extend as well another entity. Circular references are detected and compilation may fail.

When you extends another entity, it means that when the universe "compiles", it will perform some merge operations. e.g. for the the following inheritance definition:

u1.a --extends--> u1.b --extends--> u1.c

It means that if you have a universe u1 containing these entities a, b, c and you compile it, the resulting universe, let's call it u2, will contain 3 new entities a, b and c which content will be (all content is duplicated):

  • u2.c content is the same as u1.c.
  • u2.b content will be the merge of u1.b into u2.c.
  • u2.a content will be the merge of u1.a into u2.b.

Of course the compilation process keeps the initial relationships. 

u2.a --extends--> u2.b --extends--> u2.c

You can see an example of inheritance in previous paragraph.

Overrides

Overrides are actually a special type of entities. They have a special array called overrides which contains a list of entities you want to inject content into.

When you override entity a with override o, it means that the content (fields) of o will be injected into a (fields). This is why an override can override multiple objects of multiple types, because this is just about content injection. Of course as already said, it occurs during the compilation process and only in the "compiled" universe. The original universe is meant to remain unmodified.

Overrides are only applied in the context of a scenario

u = UniverseCompiler.new_universe
a = EntityA.new fields: { name: :a, bar: 'Yo man', stupid: :yeah } 
b = EntityA.new fields: { name: :b, extends: a } 
o = UniverseCompiler.new_override fields: { name: :my_override, scenario: :test_overrides, a_new_stuff: :hey, overrides: [a, b] }
u << a << b << o

o.type # => :entity_override
v = u.compile scenario: :test_overrides
v.get_entities.map &:fields
# => [{:name=>:a,
#      :bar=>"Yo man",
#      :stupid=>:yeah,
#      :some_data=>{},
#      :a_new_stuff=>:hey},
#     {:name=>:b,
#      :bar=>"Yo man",
#      :stupid=>:yeah,
#      :some_data=>{},
#      :extends=>
#          #<EntityA:47368495795560 composite_key=[:some_entity, :a], @universe='Unnamed Universe - COMPILED #47368495813960'>,
#          :a_new_stuff=>:hey},
#     {:name=>:my_override,
#      :scenario=>:test_overrides,
#      :a_new_stuff=>:hey,
#      :overrides=>
#          [#<EntityA:47368495795560 composite_key=[:some_entity, :a], @universe='Unnamed Universe - COMPILED #47368495813960'>,
#           #<EntityA:47368495725460 composite_key=[:some_entity, :b], @universe='Unnamed Universe - COMPILED #47368495813960'>]}]
#

You can see there that the compiled version of b contains both data coming from the inheritance mechanism as well as those coming from the override...

Generate a graph of entities using Graphviz

Provides a Graphviz graph of relations using the ruby-graphviz gem between a set of entities and allows callback mechanisms for the graph customization.

All you need is to:

include UniverseCompiler::Utils::Graphviz

In any class, and then simply call:

graph_entities_to_file(an_array_of_entities, set_of_options, &customization_block)

Calls to the graph_entities_to_file method returns either the filename of the created file or the graphiz graph object depending on method parameters.

A block can be passed to the method, it will yield the graph object, and optionnally a bi-directional structure of entities and graphwiz nodes, allowing for complete graph display customization (you may actually completely change the graph, but maybe in this case you may want to actually completely build the graph. It is normally intended to change/customize display attributes there...).

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on Gitlab at https://gitlab.com/tools4devops/power_stencil. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the PowerStencil project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.