Purpose
Lutaml::Model is a lightweight library for serializing and deserializing Ruby objects to and from various formats such as JSON, XML, YAML, and TOML. It uses an adapter pattern to support multiple libraries for each format, providing flexibility and extensibility for your data modeling needs.
The name "LutaML" comes from the Latin word for clay, "Lutum", and "ML" for Markup Language. Just as clay can be molded and modeled into beautiful and practical end products, the Lutaml::Model gem is used for data modeling, allowing you to shape and structure your data into useful forms.
|
Note
|
Lutaml::Model is designed to be compatible with the Shale data modeling API. Shale is an amazing Ruby data modeller. Lutaml::Model is meant to address needs that are not currently addressed by Shale. |
Data modeling in a nutshell
Data modeling is the process of creating a data model for the data to be stored in a database or used in an application. It helps in defining the structure, relationships, and constraints of the data, making it easier to manage and use.
Lutaml::Model simplifies data modeling in Ruby by allowing you to define models with attributes and serialize/deserialize them to/from various serialization formats seamlessly.
Features
-
Define models with attributes and types
-
Serialize and deserialize models to/from JSON, XML, YAML, and TOML
-
Support for multiple serialization libraries (e.g.,
toml-rb,tomlib) -
Configurable adapters for different serialization formats
-
Support for collections and default values
-
Custom serialization/deserialization methods
-
XML namespaces and mappings
Installation
Add this line to your application’s Gemfile:
gem 'lutaml-model'
And then execute:
bundle install
Or install it yourself as:
gem install lutaml-model
Defining a data model class
General
There are two ways to define a data model in Lutaml::Model:
-
Inheriting from the
Lutaml::Model::Serializableclass -
Including the
Lutaml::Model::Serializemodule
Definition through inheritance
The simplest way to define a model is to create a class that inherits from
Lutaml::Model::Serializable.
The attribute class method is used to define attributes.
Definition through inclusion
If the model class already has a super class that it inherits from, the model
can be extended using the Lutaml::Model::Serialize module.
Defining attributes
Supported attribute value types
Lutaml::Model supports the following attribute types, they can be referred by a string, a symbol, or their class constant.
Syntax:
attribute :name_of_attribute, {symbol | string | class}
| String | Symbol | Class name | Actual value class |
|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
> s = Studio.new(location: 'London', potter: 'John Doe', kiln: 'Kiln 1')
> # <Studio:0x0000000104ac7240 @location="London", @potter="John Doe", @kiln="Kiln 1">
> s.location
> # "London"
> s.potter
> # "John Doe"
> s.kiln
> # "Kiln 1"
Attribute as a collection
Define attributes as collections (arrays or hashes) to store multiple values
using the collection option.
Syntax:
attribute :name_of_attribute, Type, collection: true
collection option to define a collection attributeAttribute as an enumeration
An attribute can be defined as an enumeration by using the values directive.
The values directive is used to define acceptable values in an attribute. If
any other value is given, a Lutaml::Model::InvalidValueError will be raised.
Syntax:
attribute :name_of_attribute, Type, values: [value1, value2, ...]
values directive to define acceptable values for an attribute> Ceramic.new(type: 'Porcelain').type
> # "Porcelain"
> Ceramic.new(type: 'Earthenware').type
> # "Earthenware"
> Ceramic.new(type: 'Bone China').type
> # Lutaml::Model::InvalidValueError: Invalid value for attribute 'type'
Attribute value default
Specify default values for attributes using the default option.
The default option can be set to a value or a lambda that returns a value.
Syntax:
attribute :name_of_attribute, Type, default: -> { value }
default option to set a default value for an attributeSerialization model mappings
General
Lutaml::Model allows you to translate a data model into serialization models of various serialization formats including XML, JSON, YAML, and TOML.
Depending on the serialization format, different methods are supported for defining serialization and deserialization mappings.
Serialization model mappings are defined under the xml, json, yaml, and
toml blocks.
xml, json, yaml, and toml blocks to define serialization mappingsclass Example < Lutaml::Model::Serializable
xml do
# ...
end
json do
# ...
end
yaml do
# ...
end
toml do
# ...
end
end
XML
Setting root element name
The root method sets the root element tag name of the XML document.
If root is not given, then the snake-cased class name will be used as the
root.
<example> in XML <example>…</example>.
Syntax:
xml do
root 'xml_element_name'
end
exampleclass Example < Lutaml::Model::Serializable
xml do
root 'example'
end
end
> Example.new.to_xml
> #<example></example>
Mapping elements
The map_element method maps an XML element to a data model attribute.
<name> tag in <example><name>John Doe</name></example>.
The value will be set to John Doe.
Syntax:
xml do
map_element 'xml_element_name', to: :name_of_attribute
end
name tag to the name attribute<example><name>John Doe</name></example>
> Example.from_xml(xml)
> #<Example:0x0000000104ac7240 @name="John Doe">
> Example.new(name: "John Doe").to_xml
> #<example><name>John Doe</name></example>
Mapping attributes
The map_attribute method maps an XML attribute to a data model attribute.
Syntax:
xml do
map_attribute 'xml_attribute_name', to: :name_of_attribute
end
map_attribute to map the value attributeThe following class will parse the XML snippet below:
<example value=12><name>John Doe</name></example>
> Example.from_xml(xml)
> #<Example:0x0000000104ac7240 @value=12>
> Example.new(value: 12).to_xml
> #<example value="12"></example>
Mapping content
Content represents the text inside an XML element, inclusive of whitespace.
The map_content method maps an XML element’s content to a data model
attribute.
Syntax:
xml do
map_content to: :name_of_attribute
end
map_content to map content of the description tagThe following class will parse the XML snippet below:
<example>John Doe is my moniker.</example>
> Example.from_xml(xml)
> #<Example:0x0000000104ac7240 @description="John Doe is my moniker.">
> Example.new(description: "John Doe is my moniker.").to_xml
> #<example>John Doe is my moniker.</example>
Example for mapping
The following class will parse the XML snippet below:
class Example < Lutaml::Model::Serializable
attribute :name, Lutaml::Model::Type::String
attribute :description, Lutaml::Model::Type::String
attribute :value, Lutaml::Model::Type::Integer
xml do
root 'example'
map_element 'name', to: :name
map_attribute 'value', to: :value
map_content to: :description
end
end
<example value=12><name>John Doe</name> is my moniker.</example>
> Example.from_xml(xml)
> #<Example:0x0000000104ac7240 @name="John Doe", @description=" is my moniker.", @value=12>
> Example.new(name: "John Doe", description: " is my moniker.", value: 12).to_xml
> #<example value="12"><name>John Doe</name> is my moniker.</example>
Namespaces
Namespace at root
The namespace method in the xml block sets the namespace for the root
element.
Syntax:
xml do
namespace 'http://example.com/namespace'
end
namespace method to set the namespace for the root element<Ceramic xmlns='http://example.com/ceramic'><Type>Porcelain</Type><Glaze>Clear</Glaze></Ceramic>
> Ceramic.from_xml(xml_file)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze="Clear">
> Ceramic.new(type: "Porcelain", glaze: "Clear").to_xml
> #<Ceramic xmlns="http://example.com/ceramic"><Type>Porcelain</Type><Glaze>Clear</Glaze></Ceramic>
Namespace on attribute
If the namespace is defined on an XML attribute, then that will be given priority over the one defined in the class.
Syntax:
xml do
map_element 'xml_element_name', to: :name_of_attribute,
namespace: 'http://example.com/namespace',
prefix: 'prefix'
end
namespace-
The XML namespace used by this element
prefix-
The XML namespace prefix used by this element (optional)
namespace option to set the namespace for an elementIn this example, glz will be used for Glaze if it is added inside the
Ceramic class, and glaze will be used otherwise.
class Glaze < Lutaml::Model::Serializable
attribute :color, Lutaml::Model::Type::String
attribute :temperature, Lutaml::Model::Type::Integer
xml do
root 'Glaze'
namespace 'http://example.com/old_glaze', 'glaze'
map_element 'color', to: :color
map_element 'temperature', to: :temperature
end
end
class Ceramic < Lutaml::Model::Serializable
attribute :type, Lutaml::Model::Type::String
attribute :glaze, Glaze
xml do
root 'Ceramic'
map_element 'Type', to: :type
map_element 'Glaze', to: :glaze, namespace: 'http://example.com/glaze', prefix: "glz"
map_attribute 'xmlns', to: :namespace, namespace: 'http://example.com/ceramic'
end
end
<Ceramic xmlns='http://example.com/ceramic'>
<Type>Porcelain</Type>
<glz:Glaze xmlns='http://example.com/glaze'>
<color>Clear</color>
<temperature>1050</temperature>
</glz:Glaze>
</Ceramic>
> Ceramic.from_xml(xml_file)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze=#<Glaze:0x0000000104ac7240 @color="Clear", @temperature=1050>>
> Ceramic.new(type: "Porcelain", glaze: Glaze.new(color: "Clear", temperature: 1050)).to_xml
> #<Ceramic xmlns="http://example.com/ceramic"><Type>Porcelain</Type><glz:Glaze xmlns="http://example.com/glaze"><color>Clear</color><temperature>1050</temperature></glz:Glaze></Ceramic>
Namespace with inherit option
The inherit option is used at the element level to inherit the namespace from
the root element.
Syntax:
xml do
map_element 'xml_element_name', to: :name_of_attribute, namespace: :inherit
end
inherit option to inherit the namespace from the root elementIn this example, the Type element will inherit the namespace from the root.
class Ceramic < Lutaml::Model::Serializable
attribute :type, Lutaml::Model::Type::String
attribute :glaze, Lutaml::Model::Type::String
attribute :color, Lutaml::Model::Type::String
xml do
root 'Ceramic'
namespace 'http://example.com/ceramic', prefix: 'cera'
map_element 'Type', to: :type, namespace: :inherit
map_element 'Glaze', to: :glaze
map_attribute 'color', to: :color, namespace: 'http://example.com/color', prefix: 'clr'
end
end
<Ceramic
xmlns:cera='http://example.com/ceramic'
xmlns:clr='http://example.com/color'
clr:color="navy-blue">
<cera:Type>Porcelain</cera:Type>
<Glaze>Clear</Glaze>
</Ceramic>
> Ceramic.from_xml(xml_file)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze="Clear", @color="navy-blue">
> Ceramic.new(type: "Porcelain", glaze: "Clear", color: "navy-blue").to_xml
> #<Ceramic xmlns:cera="http://example.com/ceramic"
# xmlns:clr='http://example.com/color'
# clr:color="navy-blue">
# <cera:Type>Porcelain</cera:Type>
# <Glaze>Clear</Glaze>
# </Ceramic>
Mixed content
General
In XML there can be tags that contain content mixed with other tags and where whitespace is significant, such as to represent rich text.
<description><p>My name is <bold>John Doe</bold>, and I'm <i>28</i> years old</p></description>
To map this to Lutaml::Model we can use the mixed option in either way:
-
when defining the model;
-
when referencing the model.
Specifying the mixed option at root
This will always treat the content of the element itself as mixed content.
Syntax:
xml do
root 'xml_element_name', mixed: true
end
mixed to treat root as mixed content> Paragraph.from_xml("<p>My name is <bold>John Doe</bold>, and I'm <i>28</i> years old</p>")
> #<Paragraph:0x0000000104ac7240 @bold="John Doe", @italic="28">
> Paragraph.new(bold: "John Doe", italic: "28").to_xml
> #<p>My name is <bold>John Doe</bold>, and I'm <i>28</i> years old</p>
TODO: How to create mixed content from #new?
Specifying the mixed option when referencing a model
This will only treat the content of the referenced model as mixed content if the
mixed: true is added when referencing it.
Syntax:
xml do
map_element 'xml_element_name', to: :name_of_attribute, mixed: true
end
mixed to treat an inner element as mixed contentclass Paragraph < Lutaml::Model::Serializable
attribute :bold, Lutaml::Model::Type::String
attribute :italic, Lutaml::Model::Type::String
xml do
root 'p'
map_element 'bold', to: :bold
map_element 'i', to: :italic
end
end
class Description < Lutaml::Model::Serializable
attribute :paragraph, Paragraph
xml do
root 'description'
map_element 'p', to: :paragraph, mixed: true
end
end
> Description.from_xml("<description><p>My name is <bold>John Doe</bold>, and I'm <i>28</i> years old</p></description>")
> #<Description:0x0000000104ac7240 @paragraph=#<Paragraph:0x0000000104ac7240 @bold="John Doe", @italic="28">>
> Description.new(paragraph: Paragraph.new(bold: "John Doe", italic: "28")).to_xml
> #<description><p>My name is <bold>John Doe</bold>, and I'm <i>28</i> years old</p></description>
Key value data models
General
Key-value data models like JSON, YAML, and TOML all share a similar structure where data is stored as key-value pairs.
Lutaml::Model works with these formats in a similar way.
Mapping
The map method is used to define key-value mappings.
Syntax:
json | yaml | toml do
map 'key_value_model_attribute_name', to: :name_of_attribute
end
map method to define key-value mappingsclass Example < Lutaml::Model::Serializable
attribute :name, Lutaml::Model::Type::String
attribute :value, Lutaml::Model::Type::Integer
json do
map 'name', to: :name
map 'value', to: :value
end
yaml do
map 'name', to: :name
map 'value', to: :value
end
toml do
map 'name', to: :name
map 'value', to: :value
end
end
{
"name": "John Doe",
"value": 28
}
> Example.from_json(json)
> #<Example:0x0000000104ac7240 @name="John Doe", @value=28>
> Example.new(name: "John Doe", value: 28).to_json
> #{"name"=>"John Doe", "value"=>28}
Nested attribute mappings
The map method can also be used to map nested key-value data models
by referring to a Lutaml::Model class as an attribute class.
class Glaze < Lutaml::Model::Serializable
attribute :color, Lutaml::Model::Type::String
attribute :temperature, Lutaml::Model::Type::Integer
json do
map 'color', to: :color
map 'temperature', to: :temperature
end
end
class Ceramic < Lutaml::Model::Serializable
attribute :type, Lutaml::Model::Type::String
attribute :glaze, Glaze
json do
map 'type', to: :type
map 'glaze', to: :glaze
end
end
{
"type": "Porcelain",
"glaze": {
"color": "Clear",
"temperature": 1050
}
}
> Ceramic.from_json(json)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze=#<Glaze:0x0000000104ac7240 @color="Clear", @temperature=1050>>
> Ceramic.new(type: "Porcelain", glaze: Glaze.new(color: "Clear", temperature: 1050)).to_json
> #{"type"=>"Porcelain", "glaze"=>{"color"=>"Clear", "temperature"=>1050}}
Advanced attribute mapping
Attribute mapping delegation
Delegate attribute mappings to nested objects using the delegate option.
Syntax:
xml | json | yaml | toml do
map 'key_value_model_attribute_name', to: :name_of_attribute, delegate: :model_to_delegate_to
end
delegate option to map attributes to nested objectsThe following class will parse the JSON snippet below:
class Glaze < Lutaml::Model::Serializable
attribute :color, Lutaml::Model::Type::String
attribute :temperature, Lutaml::Model::Type::Integer
json do
map 'color', to: :color
map 'temperature', to: :temperature
end
end
class Ceramic < Lutaml::Model::Serializable
attribute :type, Lutaml::Model::Type::String
attribute :glaze, Glaze
json do
map 'type', to: :type
map 'color', to: :color, delegate: :glaze
end
end
{
"type": "Porcelain",
"color": "Clear"
}
> Ceramic.from_json(json)
> #<Ceramic:0x0000000104ac7240 @type="Porcelain", @glaze=#<Glaze:0x0000000104ac7240 @color="Clear", @temperature=nil>>
> Ceramic.new(type: "Porcelain", glaze: Glaze.new(color: "Clear")).to_json
> #{"type"=>"Porcelain", "color"=>"Clear"}
Attribute serialization with custom methods
Define custom methods for specific attribute mappings using the with: key for
each serialization mapping block for from and to.
Syntax:
xml | json | yaml | toml do
map 'key_value_model_attribute_name', to: :name_of_attribute, with: {
to: :method_name_to_serialize,
from: :method_name_to_deserialize
}
end
with: key to define custom serialization methodsThe following class will parse the JSON snippet below:
class CustomCeramic < Lutaml::Model::Serializable
attribute :name, Lutaml::Model::Type::String
attribute :size, Lutaml::Model::Type::Integer
json do
map 'name', to: :name, with: { to: :name_to_json, from: :name_from_json }
map 'size', to: :size
end
def name_to_json(model, value)
"Masterpiece: #{value}"
end
def name_from_json(model, doc)
doc['name'].sub(/^Masterpiece: /, '')
end
end
{
"name": "Masterpiece: Vase",
"size": 12
}
> CustomCeramic.from_json(json)
> #<CustomCeramic:0x0000000104ac7240 @name="Vase", @size=12>
> CustomCeramic.new(name: "Vase", size: 12).to_json
> #{"name"=>"Masterpiece: Vase", "size"=>12}
Attribute extraction
|
Note
|
This feature is for key-value data model serialization only. |
The child_mappings option is used to extract results from a key-value data
model (JSON, YAML, TOML) into a Lutaml::Model collection.
The values are extracted from the key-value data model using the list of keys provided.
Syntax:
json | yaml | toml do
map 'key_value_model_attribute_name', to: :name_of_attribute,
child_mappings: {
key_attribute_name_1: <b class="conum">(1)</b>
{path_to_value_1}, <b class="conum">(2)</b>
key_attribute_name_2:
{path_to_value_2},
# ...
}
end
-
The
key_attribute_name_1is the attribute name in the model. The value of this attribute will be assigned the key of the hash in the key-value data model. -
The
path_to_value_1is an array of keys that represent the path to the value in the key-value data model. The keys are used to extract the value from the key-value data model and assign it to the attribute in the model.
The path_to_value is in a nested array format with each value a symbol, where
each symbol represents a key to traverse down. The last key in the path is the
value to be extracted.
The following JSON contains 2 keys in schema named engine and gearbox.
{
"components": {
"engine": {
"manufacturer": "Ford",
"model": "V8"
},
"gearbox": {
"manufacturer": "Toyota",
"model": "4-speed"
}
}
}
The path to value for the engine schema is [:components, :engine] and for
the gearbox schema is [:components, :gearbox].
In path_to_value, the :key and :value are reserved instructions used to
assign the key or value of the serialization data respectively as the value to
the attribute.
In the following JSON content, the path_to_value for the object keys named
engine and gearbox will utilize the :key keyword to assign the key of the
object as the value of a designated attribute.
{
"components": {
"engine": { /*...*/ },
"gearbox": { /*...*/ }
}
}
If a specified value path is not found, the corresponding attribute in the model
will be assigned a nil value.
nil when the path_to_value is not foundIn the following JSON content, the path_to_value of [:extras, :sunroof] and
[:extras, :drinks_cooler] at the object "gearbox" would be set to nil.
{
"components": {
"engine": {
"manufacturer": "Ford",
"extras": {
"sunroof": true,
"drinks_cooler": true
}
},
"gearbox": {
"manufacturer": "Toyota"
}
}
}
child_mappings option to extract values from a key-value data modelThe following JSON contains 2 keys in schema named foo and bar.
{
"schemas": {
"foo": { <b class="conum">(1)</b>
"path": { <b class="conum">(2)</b>
"link": "link one",
"name": "one"
}
},
"bar": { <b class="conum">(1)</b>
"path": { <b class="conum">(2)</b>
"link": "link two",
"name": "two"
}
}
}
}
-
The keys
fooandbarare to be mapped to theidattribute. -
The nested
path.linkandpath.namekeys are used as thelinkandnameattributes, respectively.
A model can be defined for this JSON as follows:
class Schema < Lutaml::Model::Serializable
attribute :id, Lutaml::Model::Type::String
attribute :link, Lutaml::Model::Type::String
attribute :name, Lutaml::Model::Type::String
end
class ChildMappingClass < Lutaml::Model::Serializable
attribute :schemas, Schema, collection: true
json do
map "schemas", to: :schemas,
child_mappings: {
id: :key,
link: %i[path link],
name: %i[path name],
}
end
end
The output becomes:
> ChildMappingClass.from_json(json)
> #<ChildMappingClass:0x0000000104ac7240
@schemas=
[#<Schema:0x0000000104ac6e30 @id="foo", @link="link one", @name="one">,
#<Schema:0x0000000104ac58f0 @id="bar", @link="link two", @name="two">]>
> ChildMappingClass.new(schemas: [Schema.new(id: "foo", link: "link one", name: "one"), Schema.new(id: "bar", link: "link two", name: "two")]).to_json
> #{"schemas"=>{"foo"=>{"path"=>{"link"=>"link one", "name"=>"one"}}, {"bar"=>{"path"=>{"link"=>"link two", "name"=>"two"}}}}
In this example:
-
The
keyof each schema (fooandbar) is mapped to theidattribute. -
The nested
path.linkandpath.namekeys are mapped to thelinkandnameattributes, respectively.
Adapters
General
Lutaml::Model uses an adapter pattern to support multiple libraries for each serialization format.
You will need to specify the configuration for the adapter you want to use. The easiest way is to copy and paste the following configuration into your code.
The default configuration is as follows:
require 'lutaml/model'
require 'lutaml/model/xml_adapter/nokogiri_adapter'
require 'lutaml/model/json_adapter/standard_json_adapter'
require 'lutaml/model/toml_adapter/toml_rb_adapter'
require 'lutaml/model/yaml_adapter/standard_yaml_adapter'
Lutaml::Model::Config.configure do |config|
config.xml_adapter = Lutaml::Model::XmlAdapter::NokogiriAdapter
config.yaml_adapter = Lutaml::Model::YamlAdapter::StandardYamlAdapter
config.json_adapter = Lutaml::Model::JsonAdapter::StandardJsonAdapter
config.toml_adapter = Lutaml::Model::TomlAdapter::TomlRbAdapter
end
XML
Lutaml::Model supports the following XML adapters:
-
Nokogiri (default)
-
Oga (optional, plain Ruby suitable for Opal/JS)
-
Ox (optional)
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
require 'lutaml/model/xml_adapter/nokogiri_adapter'
config.xml_adapter = Lutaml::Model::XmlAdapter::NokogiriAdapter
end
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
require 'lutaml/model/xml_adapter/oga_adapter'
config.xml_adapter = Lutaml::Model::XmlAdapter::OgaAdapter
end
YAML
Lutaml::Model supports only one YAML adapter.
-
YAML (default)
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
require 'lutaml/model/yaml_adapter/standard_yaml_adapter'
config.yaml_adapter = Lutaml::Model::YamlAdapter::StandardYamlAdapter
end
JSON
Lutaml::Model supports the following JSON adapters:
-
JSON (default)
-
MultiJson (optional)
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
require 'lutaml/model/json_adapter/standard_json_adapter'
config.json_adapter = Lutaml::Model::JsonAdapter::StandardJsonAdapter
end
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
require 'lutaml/model/json_adapter/multi_json_adapter'
config.json_adapter = Lutaml::Model::JsonAdapter::MultiJsonAdapter
end
TOML
Lutaml::Model supports the following TOML adapters:
-
Toml-rb (default)
-
Tomlib (optional)
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
require 'lutaml/model/toml_adapter/toml_rb_adapter'
config.toml_adapter = Lutaml::Model::TomlAdapter::TomlRbAdapter
end
require 'lutaml/model'
Lutaml::Model::Config.configure do |config|
config.toml_adapter = Lutaml::Model::TomlAdapter::TomlibAdapter
require 'lutaml/model/toml_adapter/tomlib_adapter'
end
License and Copyright
This project is licensed under the BSD 2-clause License. See the LICENSE file for details.
Copyright Ribose.