Class: ElasticGraph::Local::RakeTasks

Inherits:
Rake::TaskLib
  • Object
show all
Defined in:
lib/elastic_graph/local/rake_tasks.rb

Overview

Note:

All tasks (besides the ‘schema_artifacts` tasks) require `docker` and `docker-compose` to be available on your machine.

Defines tasks for local development. These tasks include:

  • Running OpenSearch and/or Elasticsearch locally (‘(elasticsearch|opensearch)::(boot|daemon|halt)`)

  • Managing schema artifacts (‘schema_artifacts:(check|dump)`)

  • Configuring OpenSearch/Elasticsearch locally (‘clusters:configure:(dry_run|perform)`)

  • Indexing fake data (‘index_fake_data:`)

  • Booting an ElasticGraph application locally (‘boot_locally`)

Constant Summary collapse

UI_PORT_OFFSET =

Offset we add to a port number for the UI (e.g. Kibana or OpenSearch Dashboards).

Example: if Elasticsearch/OpenSearch is running on port 9876, the UI for it will run on port 19876.

10_000
VALID_PORT_RANGE =

As per en.wikipedia.org/wiki/Registered_port, valid user port numbers are 1024 to 49151, but with our UI offset we need to truncate the range further.

1024..(49151 - UI_PORT_OFFSET)

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(local_config_yaml:, path_to_schema:) {|RakeTasks| ... } ⇒ RakeTasks

Note:

This method uses keyword args for all required arguments. Optional task settings are instead specified using the block.

Returns a new instance of RakeTasks.

Yields:

  • (RakeTasks) —

    instance for further configuration

Yield Returns:

  • (void) 


342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 342

def initialize(local_config_yaml:, path_to_schema:)
  @local_config_yaml = local_config_yaml.to_s

  self.index_document_sizes = false
  self.schema_element_name_form = :camelCase
  self.schema_element_name_overrides = {}
  self.derived_type_name_formats = {}
  self.type_name_overrides = {}
  self.enum_value_overrides_by_type = {}
  self.schema_definition_extension_modules = []
  self.enforce_json_schema_version = true
  self.env_port_mapping = {}
  self.output = $stdout
  self.daemon_timeout = 120

  database_versions = ::YAML.load_file("#{__dir__}/tested_database_versions.yaml")
  self.elasticsearch_versions = database_versions.fetch("elasticsearch")
  self.opensearch_versions = database_versions.fetch("opensearch")

  @fake_data_batch_generator_by_type = {}

  yield self if block_given?

  # Default the local port from the local_config_yaml file.
  self.env_port_mapping = {"local" => local_datastore_port}.merge(env_port_mapping || {})
  if (invalid_port_mapping = env_port_mapping.reject { |env, port| VALID_PORT_RANGE.cover?(port) }).any?
    raise "`env_port_mapping` has invalid ports: #{invalid_port_mapping.inspect}. Valid ports must be in the #{VALID_PORT_RANGE} range."
  end

  # Load admin and schema def rake tasks...
  Admin::RakeTasks.from_yaml_file(local_config_yaml, output: output)
  SchemaDefinition::RakeTasks.new(
    index_document_sizes: index_document_sizes,
    path_to_schema: path_to_schema,
    schema_artifacts_directory: local_config.fetch("schema_artifacts").fetch("directory"),
    schema_element_name_form: schema_element_name_form,
    schema_element_name_overrides: schema_element_name_overrides,
    derived_type_name_formats: derived_type_name_formats,
    type_name_overrides: type_name_overrides,
    enum_value_overrides_by_type: enum_value_overrides_by_type,
    extension_modules: schema_definition_extension_modules,
    enforce_json_schema_version: enforce_json_schema_version,
    output: output
  )

  # ...then define a bunch of our own.
  define_docker_tasks("Elasticsearch", "Kibana", elasticsearch_versions, /license \[[^\]]+\] mode \[[^\]]+\] - valid/)
  define_docker_tasks("OpenSearch", "OpenSearch Dashboards", opensearch_versions, /o\.o\.n\.Node.+started/)
  define_other_tasks
end

Instance Attribute Details

#daemon_timeout ⇒ Integer

Maximum time (in seconds) to wait for the datastore to boot when booting it as a daemon. Defaults to 120.



302
303
304
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 302

def daemon_timeout
  @daemon_timeout
end

#derived_type_name_formats ⇒ Object

Overrides for the naming formats used by ElasticGraph for derived GraphQL type names. For example, to use ‘Metrics` instead of `AggregatedValues` as the suffix for the generated types supporting getting aggregated metrid values, set to `“%{baseMetrics”}`. See SchemaDefinition::SchemaElements::TypeNamer::DEFAULT_FORMATS for the available formats.

Defaults to an empty hash.

Examples:

Change the ‘AggregatedValues` type suffix to `Metrics`

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.derived_type_name_formats = {AggregatedValues: "Metrics"}
end



109
110
111
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 109

def derived_type_name_formats
  @derived_type_name_formats
end

#elasticsearch_versions ⇒ Array<String>

List of Elasticsearch versions you want to be able to boot. Rake tasks will be defined for each version to support booting and halting Elasticsearch locally. Defaults to the versions of Elasticsearch that are exercised by the ElasticGraph test suite, as defined by ‘lib/elastic_graph/local/tested_database_versions.yaml`:

Examples:

Disable Elasticsearch tasks for a project that uses OpenSearch

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.elasticsearch_versions = []
end

See Also:



249
250
251
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 249

def elasticsearch_versions
  @elasticsearch_versions
end

#enforce_json_schema_version ⇒ Boolean

Note:

Generally speaking, you will want this to be ‘true` for any ElasticGraph application that is in production as the versioning of JSON schemas is what supports safe schema evolution as it allows ElasticGraph to identify which version of the JSON schema the publishing system was operating on when it published an event.

It can be useful to set it to ‘false` before your application is in production, as you do not want to be forced to bump the version after every single schema change while you are building an initial prototype.

Whether or not to enforce the requirement that the JSON schema version is incremented every time dumping the JSON schemas results in a changed artifact. Defaults to ‘true`.

Examples:

Disable enforcement during initial prototyping

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  # TODO: remove this once we're past the prototyping stage
  tasks.enforce_json_schema_version = false
end

See Also:

  • SchemaDefinition::API#json_schema_version


229
230
231
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 229

def enforce_json_schema_version
  @enforce_json_schema_version
end

#enum_value_overrides_by_type ⇒ Hash<Symbol, Hash<Symbol, String>>

Overrides for the names of specific GraphQL enum values for specific enum types. For example, to rename the ‘DayOfWeek.MONDAY` enum to `DayOfWeek.MON`, set to `{MONDAY: “MON”}`.

Defaults to an empty hash.

Examples:

Shorten the names of the ‘DayOfWeek` enum values

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.enum_value_overrides_by_type = {
    DayOfWeek: {
      MONDAY: "MON",
      TUESDAY: "TUE",
      WEDNESDAY: "WED",
      THURSDAY: "THU",
      FRIDAY: "FRI",
      SATURDAY: "SAT",
      SUNDAY: "SUN"
    }
  }
end



155
156
157
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 155

def enum_value_overrides_by_type
  @enum_value_overrides_by_type
end

#env_port_mapping ⇒ Hash<Symbol, Integer>

Note:

When booting Elasticsearch/OpenSearch, Kibana (or its OpenSearch equivalent, “OpenSearch Dashboards”) will also get booted, selecting the port by adding ‘10000` to the configured port.

Hash mapping environments (e.g. ‘:test`, `:dev`, etc) to port numbers for use when booting Elasticsearch or OpenSearch. The hash automatically includes an entry for the `:local` environment, using a port number extracted from `local_config_yaml`.

Examples:

Define what port to use to boot the datastore for the ‘:test` environment

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.env_port_mapping = {test: 9999}
end



288
289
290
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 288

def env_port_mapping
  @env_port_mapping
end

#index_document_sizes ⇒ Boolean

Note:

Enabling this requires the [mapper-size plugin](www.elastic.co/guide/en/elasticsearch/plugins/8.15/mapper-size.html) to be installed on your datastore cluster. You are responsible for ensuring that is installed if you enable this feature. If you enable this and the plugin is not installed, you will get errors!

When enabled, ElasticGraph will configure the index mappings so that the datastore indexes a ‘_size` field in each index document. ElasticGraph itself does not do anything with this field, but it will be available for your use in any direct queries (e.g. via Kibana).

Defaults to ‘false` since it requires a plugin.

Examples:

Enable indexing document sizes

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.index_document_sizes = true
end



49
50
51
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 49

def index_document_sizes
  @index_document_sizes
end

#opensearch_versions ⇒ Array<String>

List of OpenSearch versions you want to be able to boot. Rake tasks will be defined for each version to support booting and halting OpenSearch locally. Defaults to the versions of OpenSearch that are exercised by the ElasticGraph test suite, as defined by ‘lib/elastic_graph/local/tested_database_versions.yaml`:

Examples:

Disable OpenSearch tasks for a project that uses Elasticsearch

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.opensearch_versions = []
end

See Also:



269
270
271
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 269

def opensearch_versions
  @opensearch_versions
end

#output ⇒ IO

IO for printing output (defaults to stdout).



295
296
297
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 295

def output
  @output
end

#schema_definition_extension_modules ⇒ Array<Module>

List of Ruby modules to extend onto the SchemaDefinition::API instance. Designed to support ElasticGraph extensions (such as ‘elasticgraph-apollo`). Defaults to an empty list.

Examples:

Extension that defines a ‘@since` directive and offers a `since` API on fields

module SinceExtension
  # `self.extended` is a standard Ruby hook that gets called when a module is extended onto an object.
  # The argument is the object the module was extended onto (a `SchemaDefinition::API` instance in this case).
  def self.extended(api)
    # Define our `@since` directive
    api.raw_sdl "directive @since(date: Date!) on FIELD_DEFINITION"

    # In order to hook into fields, extend the `SchemaDefinition::Factory` with a module. The factory is used
    # for creation of all schema definition objects.
    api.factory.extend FactoryExtension
  end

  module FactoryExtension
    # Hook into the creation of all `SchemaDefinition::Field` objects so that we can extend each field
    # instance with our `FieldExtension` module.
    def new_field(*args, **options)
      super(*args, **options) do |field|
        field.extend FieldExtension
        yield field if block_given?
      end
    end
  end

  # Offer a `f.since date` API on fields.
  module FieldExtension
    def since(date)
      directive "since", date: date
    end
  end
end

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.schema_definition_extension_modules = [SinceExtension]
end



202
203
204
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 202

def schema_definition_extension_modules
  @schema_definition_extension_modules
end

#schema_element_name_form ⇒ :camelCase, :snake_case

The form of names for schema elements (fields, arguments, directives) generated by ElasticGraph, either ‘:snake_case` or `:camelCase`. For example, if set to `:camelCase`, ElasticGraph will generate a `groupedBy` field, but if set to `:snake_case`, ElasticGraph will generate a `grouped_by` field.

Defaults to ‘:camelCase` since most GraphQL schemas use that casing.

Examples:

Use ‘snake_case` names instead of `camelCase`

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.schema_element_name_form = :snake_case
end



68
69
70
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 68

def schema_element_name_form
  @schema_element_name_form
end

#schema_element_name_overrides ⇒ Hash<Symbol, String>

Overrides for specific names of schema elements (fields, arguments, directives) generated by ElasticGraph. For example, to rename the ‘gt` filter field to `greaterThan`, set to `“greaterThan”`.

Defaults to an empty hash.

Examples:

Spell out comparison operators instead of using shortened forms

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.schema_element_name_overrides = {
    gt: "greaterThan",
    gte: "greaterThanOrEqualTo",
    lt: "lessThan",
    lte: "lessThanOrEqualTo"
  }
end



91
92
93
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 91

def schema_element_name_overrides
  @schema_element_name_overrides
end

#type_name_overrides ⇒ Hash<Symbol, String>

Overrides for the names of specific GraphQL types. For example, to rename the ‘JsonSafeLong` scalar to `BigInt`, set to `“BigInt`.

Defaults to an empty hash.

Examples:

Rename ‘JsonSafeLong` to `BigInt`

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.type_name_overrides = {JsonSafeLong: "BigInt"}
end



127
128
129
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 127

def type_name_overrides
  @type_name_overrides
end

Instance Method Details

#define_fake_data_batch_for(type) {|Array<Hash<String, Object>>, Array<Hash<Symbol, Object>>| ... } ⇒ Object

Register a callback for use when indexing a batch fake data. An ‘index_fake_data:` rake task will be generated for each registered callback.

Examples:

Register a callback to generate fake ‘campaigns` data

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.define_fake_data_batch_for :campaigns do |batch|
    batch.concat(FactoryBot.build_list(:campaigns))
  end
end

Yields:

  • (Array<Hash<String, Object>>, Array<Hash<Symbol, Object>>) —

    list the block should append to when generating data

Yield Returns:

  • (void) 


331
332
333
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 331

def define_fake_data_batch_for(type, &block)
  @fake_data_batch_generator_by_type[type] = block
end