Alula-Ruby

This is the official Alula ruby API client.

Refer to last two sections if you're making changes in alula-ruby to be used as gem file in AC.

Installation

This gem is public, and can be installed via bundler from Rubygems.

gem 'alula-ruby', '~> 1.1'

If you use Sidekiq and plan on using the Alula-Ruby gem in your Sidekiq workers, you should also bundle the RequestStore-Sidekiq extension. This extension is used to store client authorization info in Thread.current for multithreading support.

gem 'request_store-sidekiq', '~> 0.1'

And then execute:

$ bundle

Authorization

Alula-Ruby requires an OAuth access token to use. You can obtain a token like so:

# Authorize the OAuth client
Alula::Oauth.configure(client_id: "your client id", client_secret: "your client secret", api_url: "the API URL")

# Obtain OAuth tokens with username & password
creds = Alula::Oauth.authenticate(username: "a username", password: "a password")

#> creds.token_type = bearer
#> creds.access_token = some string
#> creds.refresh_token = some string
#> creds.expires_in = integer
#> creds.scope = string

Configuring

There are two types of settings in the Alula::Client configuration: The first one are the ones that will have a fixed value in the application lifecicle so its better to put them in an initializer.

Alula::Client.configure do |c|
  c.api_key = # your API key
  c.debug = # if set to true will log requests to the API
  c.user_agent = # A short name for your script/application
end

The second one are the ones that are stored in the request Rack env, so those need to set on each request (to the application).

Alula::Client.configure do |d|
  c.api_key = # your API key
  c.role = # User/Customer role
  c.customer_id = # The user_id, optional, only for video API requests
end

Usage

Once you have obtained an access token, configure the client to use that token. You should perform this initialization at the start of every request, and at the start of any thread that performs work. The Alula-Ruby client uses the RequestStore gem to stash its configuration data in Thread.current, so keep this limitation in mind.

Alula::Client.config.api_key = "your access token"

If you need to save any records, you will also need to tell the Alula::Client about your users' authorized role. Depending on the role certain fields may be writeable or protected. You can set your role one of 3 ways:

myself = Alula::Self.retrieve

# The user role is inferred from the Alula::Self object
Alula::Client.config.role = myself

# Explicitly set the role via a role symbol
Alula::Client.config.role = :dealer

Any method will do, you just need to perform one of these prior to attempting to save a resource. Role symbols map to uType resources. The Alula Gem supports the following roles & their corrisponding uType:

| Name          | uType | Role           | Description                            |
|---------------|-------|----------------|----------------------------------------|
| System        | 2     | :system        | Highly-privileged system user          |
| Station       | 4     | :station       | Central station user                   |
| Dealer        | 8     | :dealer        | Dealer user; child of station users    |
| Technician    | 16    | :technician    | Technician user; child of dealer users |
| User          | 32    | :user          | Normal user; child of dealer users     |
| Sub-User      | 64    | :sub_user      | Child user; child of normal users      |
| User/Sub-User | 96    | :user_sub_user | Child user; child of normal users      |

See the official Alula API docs for a detailed breakdown of which fields on which resources are changable by which roles.

A Hash object is provided on the Alula::User model mapping uType to Role:

pp Alula::User::UTYPE_ROLE_MAP
{
  2  => :system,
  4  => :station,
  8  => :dealer,
  16 => :technician,
  32 => :user,
  64 => :sub_user,
  96 => :user_sub_user
}

Retrieving Records

Records can be fetched as a single record:

# Fetch a single device
device = Alula::Device.retrieve(device_id)

device.friendly_name

#> 'Test Device'

# Fetch singleton resources, like self, without an ID
me = Alula::Self.retrieve

Collections of records can also be fetched:

# Fetch devices
devices = Alula::Device.list

# Fetch user data
users = Alula::User.list

Paging collections

Collections can be paginated

# Use .offset to request different pages
devices = Alula::Device.offset(2).list

# Use .size to change the default page size
devices = Alula::Device.size(100).list

# Use together
devices = Alula::Device.offset(20).size(25).list

# A raw .page method is offered:
devices = Alula::Device.page(size: 20, number: 2).list

Sorting collections

Collections can be sorted. See the Alula API documentation for details of which fields can be sorted.

devices = Alula::DeviceEventLog.sort(date_entered: :desc).where_like(mac: '%54%').list

Including related models

Many models relate to other models. When loading a model you can specify related models to 'include' in the fetch, and if they exist they will be available on the model.

An Alula::Device has a single Dealer that relates to it, as defined in the Device metadata:

relationship :dealer, type: 'dealers', cardinality: 'To-one'

When you load the Device with and include the Dealer, any available Dealer data will become available as an Alula::Dealer object like so:

device = Alula::Device.includes(:dealer).retrieve(some_id)
puts device.dealer
# A Dealer model will be output here
puts device.dealer.company_name
# A company name here...

TODO: This section is in progress! We are missing many models, so not all relationships are ready for inclusion.

Filter collections

Collections of records can be filtered against using a fluent query API. Check the API documentation to see which fields can be filtered for each model. Errors will be raised when invalid field filter options are selected.

Filters directly map to their Sequelize operator, when composing complex queries it is helpful to know how the Sequelize operators work.

Field Names

All filters take field names as hash keys, and filter values as hash values. You can use the camelCase representation of each field, or a snake_case represenation. Internally all keys are cast to snake_case for validation, and to camelCase for actual querying.

$where filter

The $where filter is the most basic filter. It creates an exact match for any fields passed in:

devices = Alula::Device.where(friendly_name: 'TitusTestDevice', program_id: 33).list
devices = Alula::Device.where(friendly_name: 'TitusTestDevice').where(program_id: 33).list

$and filter

The $and filter is very similar to the $where filter. Unlike $where, each field passed into the $and filter will be wrapped in the explicit $and clause.

devices = Alula::Device.and(friendly_name: 'TitusTestDevice', program_id: 33).list

$like and $notLike filters

The $like and $notLike filters allow for wildcard searches across multiple fields.

# Find all devices where the friendly_name starts with 'Titus'
devices = Alula::Device.like(friendly_name: 'Titus%').list

# Find all devices where the term 'Titus' is not present in the friendly_name
devices = Alula::Device.not_like(friendly_name: '%Titus%').list

$in and $notIn filters

The $in and $notIn filters accept an array of matches to be queried. The values should be simple values, numbers, strings, and ISO8601 string representations of dates.

# Find records with a value containing one of an array of values
devices = Alula::Device.in(program_id: [1, 22, 39]).list

# Exclude records with an array of values
devices = Alula::Device.not_in(program_id: [8, 33, 1]).list

$between and $notBetween filters

The $between and $notBetween filters are similar to an $in query, but they take an array of 2 values and request records between those values. Pass in numbers, ISO8601-formatted date strings, or strings where MariaDB can figure out what it means to be "between" each string.

# Find records created between dates
customers = Alula::User.between(date_entered: [1.year.ago.iso8601, Time.now.iso8601]).list

# Find records outside of a range of values
devices = Alula::Device.not_between(online_status_timestamp: [1.week.ago.to_i, Time.now.to_i]).list

$not and $ne filters

The $not and $ne filters ($ne for Not Equivilant) operators are roughly analagous, though they use different matchers when constructing their SQL queries.

# Where a field is not strictly a value
devices = Alula::Device.not(friendly_name: 'TitusTestDevice').list

# Where a field is not equivilant to a value
devices = Alula::Device.ne(program_id: '2').list

$lt, $lte, $gt, $gte filters

The family of less-than, less-than-or-equal, greater-than, and greater-than-or-equal filters all works the same way. They accept a scalar value, a number or ISO8601 date string, and return records that match their constraints.

devices = Alula::Device.gt(:online_status_timestamp: 1.year.ago.to_i).list
devices = Alula::Device.gte(:online_status_timestamp: 1.year.ago.to_i).list

devices = Alula::Device.lt(:online_status_timestamp: 1.week.ago.to_i).list
devices = Alula::Device.lte(:online_status_timestamp: 1.week.ago.to_i).list

$or filters

The $or filter is unique in how it is called, and how it constructs a query. $or can take any nested set of other filter operators, so you can combine $like and $in filters into a single $or query to retrieve a broad set of records.

The $or filter provides a strict match key->value interface, and a fluent block interface.

# Build an $or query for strict field values
# Will return any device with the friendly_name 'TitusTestDevice' OR a program_id of 33
devices = Alula::Device.or(friendly_name: 'TitusTestDevice', program_id: 33).list

# Build an expressive $or query
# The provided `query_builder` param allows full access to the filter API
# This will search for devices with a program_id of 2, 33, 85, or 12,
# where the friendly_name contains 'Helix', and no device is of program_id 2
#
devices = Alula::Device.or do |query_builder|
  query_builder.in(program_id: [2, 33, 85, 12])
               .like(friendly_name: '%Helix%')
end
devices = devices.not(program_id: 5).page(20).list

$or $like filters

The OR LIKE pattern is commonly used to do a wildcard search for a term across a bunch of fields at the same time. A shorthand query method is provided to make this easier. As this is a LIKE query, you can use wildcards (%) in your term.

devices = Alula::Device.or_like(friendy_name: '%Titus%', email: '%@alula.net').list

Constructing your own filters

You can build your own JSON API-compliant filters with the .filter method if any of the built in operators do not work for you. Be aware that you must use the camelCase field name format, and no validation is performed against your query.

# All devices with a strict friendly_name match and an $in query on the program_id
devices = Alula::Device.filter({
            friendlyName: 'TitusTestDevice',
            $or: {
              $in: {
                programId: [2, 10]
              }
            }
          }).list

Debugging Filters

Before calling .list to execute your query, you can call .as_json, and be given a JSON representation of your built query. Be aware multiple filters against the same field may overwrite one another, so get a JSON dump to ensure that your assembled query is in the form you intend.

Supported API filter methods

The Alula API supports the following filter methods. Access to these filter methods are provided with Ruby methods on collection objects.

API Operator	Ruby Method	Value	Comment
$gt	.gt	Scalar; number, date	Greater than (numeric)
$gte	.gte	^ Ditto	Greater than or equal
$lt	.lt	^ Ditto	Less than
$lte	.lte	^ Ditto	Less than or equal
$not	.not	Scalar; any	Not
$ne	.ne	Scalar; any	Not equal
$between	.between	Array; numbers, dates	Between range
$notBetween	.not_between	^ Ditto	Not between range
$in	.in	Array; any	Array of matches
$notIn	.notIn	Array; any	Array of negative matches
$like	.like	Scalar; string	Match string where % can be wildcard
$notLike	.not_like	Scalar; string	Negative match string where % can be wildcard
$or	.or	Complex	See example
$and	.and	Array of associative arrays	Assoc. array where keys are field names, values like at top-level

Saving Records

And you can save records

device.friendly_name = 'Waaaluigi'
device.save
#> true

Errors saving are reflected with a false return to the save call, and errors on the model:

device.meid = 'Test Test'
device.save
#> false
device.errors.first

Remote Procedure Calls (RPC)

Alula-Ruby partially supports the Alula API's RPC namespace. All RPC methods use the same style method signature, differing only in what params are passed. See the API documentation for a list of params each method supports.

Each remote procedure call supports a single method, named call. This method takes a param list equal to the remote procedures params (underscored, not camelcased).

Responses respond to the method .ok? for inferring if an error took place.

Success responses are custom per RPC method. Some provide response data, some do not. Response data is raw JSON and is available via response.result, and it will be a Hash or Array.

Development

After checking out the repo, run bin/setup to install dependencies. You can also run bundle exec bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, and push git commits and tags.

Using Docker

Alula Ruby runs its unit & integratin tests against a copy of the API running in Docker. The remote API is cleaned up (DB truncated & re-seeded fresh) between every describe or context block.

1. Authenticate with AWS

   Set the registry account number to refer to

   export AWS_ECR_REGISTRY_ID=613707345027 #409473619697
   

   Then execute AWS login where the $env is whatever profile name your Shared Services account is setup under in AWS CLI SSO.

   aws ecr get-login-password --profile $env | docker login \
     --username AWS --password-stdin $AWS_ECR_REGISTRY_ID.dkr.ecr.$AWS_REGION.amazonaws.com
   

2. Set up local Core API cluster/swarm using make:

   make up
   

   or use the alula-docker-compose approach:

   alula-docker-compose -I @test-helper --registry '6z1wlx5zf1.execute-api.us-east-1.amazonaws.com/' -- up -d
   

Note: You do need to be authorized against our private ECR registry. This is manual, talk with an Alula Lead to get pointed in the right direction. TODO: Write out what we need to do for this.

1. Configure your local .env file, you can copy-paste .env.example over to .env

2. Run the complete test suite:

   bundle exec rspec

3. Run a specific test file:

   `bundle exec rspec ./spec/alula/oauth_spec.rb`
   

4. Run Guard to have tests run on file change

   `bundle exec guard`
   

5. If you're using the Ruby Test Explorer extension to run the tests within VS Code, SimpleCov has a dry run error preventing auto detection. You need to turn off minimum coverage to get the dry run command it uses to find tests to work.

   SimpleCov.start do
     add_filter '/spec/'
     minimum_coverage 0
   end
   

6. Update all Docker images to the latest images:

   docker compose -f alula-docker-compose.yml pull

Occasionally under heavy use the dockerized API may lose or drop its databases, resulting in the test suite erroring completly and very quickly. To fix this simply restart the API with docker-compose -f alula-docker-compose.yml down && docker-compose -f alula-docker-compose.yml up -d

Importing GEM in AC Docker

Most of the times, the work we do in alula ruby includes just updating the model/procedures and push the changes, without having to worry about the configuration mentioned above. When we make changes to alula ruby locally we need to test if the gem file works. For that we need to point to the local alula ruby gem we're working on from AC.

This is straight forward if you're not using docker: provide the path of gem file in Gemfile in AC. But if you're using docker then you'll have to load the volume in the docker first and then update the gemfile to that specific volume.

• In docker-compose.yml of your AC update the alulaconnect/volumes as:

- ~/Documents/alula-project/alula-ruby:/app/alula-ruby

In this step, you are basically pointing to the file location of your alula-ruby and loading it in the docker volume as 'app/alula-ruby'. Make sure that you got the path of your alula ruby correct.

• Second step is to import the gem file from the loaded volume. Update your Gemfile in AC as:

gem 'alula-ruby', path: "/app/alula-ruby"

Make sure that you comment out the existing import where we're fetching gem from the remote server.

Once this is done, you're ready to test your local alula-ruby gem on the alula connect side.

Releasing

In your PR:

• Update VERSION.md with a new version number and a change list
• Update lib/alula/version.rb with the new version number
• Commit these changes & include them in your PR.

After merging your PR:

• Check out main and pull to get the latest.

Update the following in AC side, the commits are to be made from AC for the files mentioned:

• Once merge is done and build is successful update the gem 'alula-ruby' in alula connect Gemfile to the latest version you updated. Also run bundle install to update Gemfile.lock file. Commit both files. Failing to do so will break the build process of alula connect. <!--
• Run bundle exec rake release. A tag will be pushed to Github and then the UI will ask you input to push to a nonexistant URL. Just spam the enter key and let it error out. It's the tag on github that we care about. [[ github action takes care of this ]] -->