Class: InventoryRefresh::InventoryCollection

Inherits:
Object
  • Object
show all
Defined in:
lib/inventory_refresh/inventory_collection.rb,
lib/inventory_refresh/inventory_collection/graph.rb,
lib/inventory_refresh/inventory_collection/scanner.rb,
lib/inventory_refresh/inventory_collection/reference.rb,
lib/inventory_refresh/inventory_collection/index/proxy.rb,
lib/inventory_refresh/inventory_collection/data_storage.rb,
lib/inventory_refresh/inventory_collection/serialization.rb,
lib/inventory_refresh/inventory_collection/index/type/base.rb,
lib/inventory_refresh/inventory_collection/index/type/data.rb,
lib/inventory_refresh/inventory_collection/references_storage.rb,
lib/inventory_refresh/inventory_collection/index/type/local_db.rb,
lib/inventory_refresh/inventory_collection/index/type/skeletal.rb

Overview

For more usage examples please follow spec examples in

  • spec/models/inventory_refresh/save_inventory/single_inventory_collection_spec.rb

  • spec/models/inventory_refresh/save_inventory/acyclic_graph_of_inventory_collections_spec.rb

  • spec/models/inventory_refresh/save_inventory/graph_of_inventory_collections_spec.rb

  • spec/models/inventory_refresh/save_inventory/graph_of_inventory_collections_targeted_refresh_spec.rb

  • spec/models/inventory_refresh/save_inventory/strategies_and_references_spec.rb

Examples:

storing Vm model data into the DB


@ems = ManageIQ::Providers::BaseManager.first
puts @ems.vms.collect(&:ems_ref) # => []

# Init InventoryCollection
vms_inventory_collection = ::InventoryRefresh::InventoryCollection.new(
  :model_class => ManageIQ::Providers::CloudManager::Vm, :parent => @ems, :association => :vms
)

# Fill InventoryCollection with data
# Starting with no vms, lets add vm1 and vm2
vms_inventory_collection.build(:ems_ref => "vm1", :name => "vm1")
vms_inventory_collection.build(:ems_ref => "vm2", :name => "vm2")

# Save InventoryCollection to the db
InventoryRefresh::SaveInventory.save_inventory(@ems, [vms_inventory_collection])

# The result in the DB is that vm1 and vm2 were created
puts @ems.vms.collect(&:ems_ref) # => ["vm1", "vm2"]

In another refresh, vm1 does not exist anymore and vm3 was added

# Init InventoryCollection
vms_inventory_collection = ::InventoryRefresh::InventoryCollection.new(
  :model_class => ManageIQ::Providers::CloudManager::Vm, :parent => @ems, :association => :vms
)

# Fill InventoryCollection with data
vms_inventory_collection.build(:ems_ref => "vm2", :name => "vm2")
vms_inventory_collection.build(:ems_ref => "vm3", :name => "vm3")

# Save InventoryCollection to the db
InventoryRefresh::SaveInventory.save_inventory(@ems, [vms_inventory_collection])

# The result in the DB is that vm1 was deleted, vm2 was updated and vm3 was created
puts @ems.vms.collect(&:ems_ref) # => ["vm2", "vm3"]

Defined Under Namespace

Modules: Index Classes: DataStorage, Graph, Reference, ReferencesStorage, Scanner, Serialization

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(model_class: nil, manager_ref: nil, association: nil, parent: nil, strategy: nil, custom_save_block: nil, delete_method: nil, dependency_attributes: nil, attributes_blacklist: nil, attributes_whitelist: nil, complete: nil, update_only: nil, check_changed: nil, arel: nil, default_values: {}, create_only: nil, inventory_object_attributes: nil, name: nil, saver_strategy: nil, parent_inventory_collections: nil, manager_uuids: [], all_manager_uuids: nil, targeted_arel: nil, targeted: nil, manager_ref_allowed_nil: nil, secondary_refs: {}, use_ar_object: nil, custom_reconnect_block: nil, batch_extra_attributes: []) ⇒ InventoryCollection

Returns a new instance of InventoryCollection.

Parameters:

  • model_class (Class) (defaults to: nil) —

    A class of an ApplicationRecord model, that we want to persist into the DB or load from the DB.

  • manager_ref (Array) (defaults to: nil) —

    Array of Symbols, that are keys of the InventoryObject’s data, inserted into this InventoryCollection. Using these keys, we need to be able to uniquely identify each of the InventoryObject objects inside.

  • association (Symbol) (defaults to: nil) —

    A Rails association callable on a :parent attribute is used for comparing with the objects in the DB, to decide if the InventoryObjects will be created/deleted/updated or used for obtaining the data from a DB, if a DB strategy is used. It returns objects of the :model_class class or its sub STI.

  • parent (ApplicationRecord) (defaults to: nil) —

    An ApplicationRecord object that has a callable :association method returning the objects of a :model_class.

  • strategy (Symbol) (defaults to: nil) —

    A strategy of the InventoryCollection that will be used for saving/loading of the InventoryObject objects. Allowed strategies are:

    - nil => InventoryObject objects of the InventoryCollection will be saved to the DB, only these objects
         will be referable from the other InventoryCollection objects.
- :local_db_cache_all => Loads InventoryObject objects from the database, it loads all the objects that
                         are a result of a [<:parent>.<:association>, :arel] taking
                         first defined in this order. This strategy will not save any objects in the DB.
- :local_db_find_references => Loads InventoryObject objects from the database, it loads only objects that
                               were referenced by the other InventoryCollections using a filtered result
                               of a [<:parent>.<:association>, :arel] taking first
                               defined in this order. This strategy will not save any objects in the DB.
- :local_db_find_missing_references => InventoryObject objects of the InventoryCollection will be saved to
                                       the DB. Then if we reference an object that is not present, it will
                                       load them from the db using :local_db_find_references strategy.
  • custom_save_block (Proc) (defaults to: nil) —

    A custom lambda/proc for persisting in the DB, for cases where it’s not enough to just save every InventoryObject inside by the defined rules and default saving algorithm.

    Example1 - saving SomeModel in my own ineffective way :-) :

    custom_save = lambda do |_ems, inventory_collection|
  inventory_collection.each |inventory_object| do
    hash = inventory_object.attributes # Loads possible dependencies into saveable hash
    obj = SomeModel.find_by(:attr => hash[:attr]) # Note: doing find_by for many models produces N+1
                                                  # queries, avoid this, this is just a simple example :-)
    obj.update_attributes(hash) if obj
    obj ||= SomeModel.create(hash)
    inventory_object.id = obj.id # If this InventoryObject is referenced elsewhere, we need to store its
                                   primary key back to the InventoryObject
 end

    Example2 - saving parent OrchestrationStack in a more effective way, than the default saving algorithm can achieve. Ancestry gem requires an ActiveRecord object for association and is not defined as a proper ActiveRecord association. That leads in N+1 queries in the default saving algorithm, so we can do better with custom saving for now. The InventoryCollection is defined as a custom dependencies processor, without its own :model_class and InventoryObjects inside:

    InventoryRefresh::InventoryCollection.new({
  :association       => :orchestration_stack_ancestry,
  :custom_save_block => orchestration_stack_ancestry_save_block,
  :dependency_attributes => {
    :orchestration_stacks           => [collections[:orchestration_stacks]],
    :orchestration_stacks_resources => [collections[:orchestration_stacks_resources]]
  }
})

    And the lambda is defined as:

    orchestration_stack_ancestry_save_block = lambda do |_ems, inventory_collection|
  stacks_inventory_collection = inventory_collection.dependency_attributes[:orchestration_stacks].try(:first)

  return if stacks_inventory_collection.blank?

  stacks_parents = stacks_inventory_collection.data.each_with_object({}) do |x, obj|
    parent_id = x.data[:parent].load.try(:id)
    obj[x.id] = parent_id if parent_id
  end

  model_class = stacks_inventory_collection.model_class

  stacks_parents_indexed = model_class
                             .select([:id, :ancestry])
                             .where(:id => stacks_parents.values).find_each.index_by(&:id)

  model_class
    .select([:id, :ancestry])
    .where(:id => stacks_parents.keys).find_each do |stack|
    parent = stacks_parents_indexed[stacks_parents[stack.id]]
    stack.update_attribute(:parent, parent)
  end
end
  • custom_reconnect_block (Proc) (defaults to: nil) —

    A custom lambda for reconnect logic of previously disconnected records

    Example - Reconnect disconnected Vms

    InventoryRefresh::InventoryCollection.new({
  :association            => :orchestration_stack_ancestry,
  :custom_reconnect_block => vms_custom_reconnect_block,
})

    And the lambda is defined as:

    vms_custom_reconnect_block = lambda do |inventory_collection, inventory_objects_index, attributes_index|
  inventory_objects_index.each_slice(1000) do |batch|
    Vm.where(:ems_ref => batch.map(&:second).map(&:manager_uuid)).each do |record|
      index = inventory_collection.object_index_with_keys(inventory_collection.manager_ref_to_cols, record)

      # We need to delete the record from the inventory_objects_index and attributes_index, otherwise it
      # would be sent for create.
      inventory_object = inventory_objects_index.delete(index)
      hash             = attributes_index.delete(index)

      record.assign_attributes(hash.except(:id, :type))
      if !inventory_collection.check_changed? || record.changed?
        record.save!
        inventory_collection.store_updated_records(record)
      end

      inventory_object.id = record.id
    end
  end
  • delete_method (Symbol) (defaults to: nil) —

    A delete method that will be used for deleting of the InventoryObject, if the object is marked for deletion. A default is :destroy, the instance method must be defined on the :model_class.

  • dependency_attributes (Hash) (defaults to: nil) —

    Manually defined dependencies of this InventoryCollection. We can use this by manually place the InventoryCollection into the graph, to make sure the saving is invoked after the dependencies were saved. The dependencies itself are InventoryCollection objects. For a common use-cases we do not need to define dependencies manually, since those are inferred automatically by scanning of the data.

    Example:

    :dependency_attributes => {
  :orchestration_stacks           => [collections[:orchestration_stacks]],
  :orchestration_stacks_resources => [collections[:orchestration_stacks_resources]]
}

    This example is used in Example2 of the <param custom_save_block> and it means that our :custom_save_block will be invoked after the InventoryCollection :orchestration_stacks and :orchestration_stacks_resources are saved.

  • attributes_blacklist (Array) (defaults to: nil) —

    Attributes we do not want to include into saving. We cannot blacklist an attribute that is needed for saving of the object. Note: attributes_blacklist is also used for internal resolving of the cycles in the graph.

    In the Example2 of the <param custom_save_block>, we have a custom saving code, that saves a :parent attribute of the OrchestrationStack. That means we don’t want that attribute saved as a part of InventoryCollection for OrchestrationStack, so we would set :attributes_blacklist => [:parent]. Then the :parent will be ignored while saving.

  • attributes_whitelist (Array) (defaults to: nil) —

    Same usage as the :attributes_blacklist, but defining full set of attributes that should be saved. Attributes that are part of :manager_ref and needed validations are automatically added.

  • complete (Boolean) (defaults to: nil) —

    By default true, :complete is marking we are sending a complete dataset and therefore we can create/update/delete the InventoryObject objects. If :complete is false we will only do create/update without delete.

  • update_only (Boolean) (defaults to: nil) —

    By default false. If true we only update the InventoryObject objects, if false we do create/update/delete.

  • check_changed (Boolean) (defaults to: nil) —

    By default true. If true, before updating the InventoryObject, we call Rails ‘changed?’ method. This can optimize speed of updates heavily, but it can fail to recognize the change for e.g. Ancestry and Relationship based columns. If false, we always update the InventoryObject.

  • arel (ActiveRecord::Associations::CollectionProxy|Arel::SelectManager) (defaults to: nil) —

    Instead of :parent and :association we can provide Arel directly to say what records should be compared to check if InventoryObject will be doing create/update/delete.

    Example: for a targeted refresh, we want to delete/update/create only a list of vms specified with a list of ems_refs:

    :arel => manager.vms.where(:ems_ref => manager_refs)

    Then we want to do the same for the hardwares of only those vms:

    :arel => manager.hardwares.joins(:vm_or_template).where(
  'vms' => {:ems_ref => manager_refs}
)

    And etc. for the other Vm related records.

  • default_values (Hash) (defaults to: {}) —

    A hash of an attributes that will be added to every inventory object created by inventory_collection.build(hash)

    Example: Given

    inventory_collection = InventoryCollection.new({
  :model_class    => ::Vm,
  :arel           => @ems.vms,
  :default_values => {:ems_id => 10}
})

    And building the inventory_object like:

    inventory_object = inventory_collection.build(:ems_ref => "vm_1", :name => "vm1")

    The inventory_object.data will look like:

    {:ems_ref => "vm_1", :name => "vm1", :ems_id => 10}
  • inventory_object_attributes (Array) (defaults to: nil) —

    Array of attribute names that will be exposed as readers/writers on the InventoryObject objects inside.

    Example: Given

    inventory_collection = InventoryCollection.new({
   :model_class                 => ::Vm,
   :arel                        => @ems.vms,
   :inventory_object_attributes => [:name, :label]
 })

    And building the inventory_object like:

    inventory_object = inventory_collection.build(:ems_ref => "vm1", :name => "vm1")

    We can use inventory_object_attributes as setters and getters:

    inventory_object.name = "Name"
inventory_object.label = inventory_object.name

    Which would be equivalent to less nicer way:

    inventory_object[:name] = "Name"
inventory_object[:label] = inventory_object[:name]

    So by using inventory_object_attributes, we will be guarding the allowed attributes and will have an explicit list of allowed attributes, that can be used also for documentation purposes.

  • name (Symbol) (defaults to: nil) —

    A unique name of the InventoryCollection under a Persister. If not provided, the :association attribute is used. If :association is nil as well, the :name will be inferred from the :model_class.

  • saver_strategy (Symbol) (defaults to: nil) —

    A strategy that will be used for InventoryCollection persisting into the DB. Allowed saver strategies are:

    • :default => Using Rails saving methods, this way is not safe to run in multiple workers concurrently, since it will lead to non consistent data.

    • :batch => Using batch SQL queries, this way is not safe to run in multiple workers concurrently, since it will lead to non consistent data.

    • :concurrent_safe => This method is designed for concurrent saving. It uses atomic upsert to avoid data duplication and it uses timestamp based atomic checks to avoid new data being overwritten by the the old data.

    • :concurrent_safe_batch => Same as :concurrent_safe, but the upsert/update queries are executed as batched SQL queries, instead of sending 1 query per record.

  • parent_inventory_collections (Array) (defaults to: nil) —

    Array of symbols having a name pointing to the InventoryRefresh::InventoryCollection objects, that serve as parents to this InventoryCollection. There are several scenarios to consider, when deciding if InventoryCollection has parent collections, see the example.

    Example:

    taking inventory collections :vms and :disks (local disks), if we write that:
inventory_collection = InventoryCollection.new({
             :model_class                 => ::Disk,
             :association                 => :disks,
             :manager_ref                 => [:vm, :location]
             :parent_inventory_collection => [:vms],
           })

Then the decision for having :parent_inventory_collection => [:vms] was probably driven by these
points:
1. We can get list of all disks only by doing SQL query through the parent object (so there will be join
   from vms to disks table).
2. There is no API query for getting all disks from the provider API, we get them inside VM data, or as
   a Vm subquery
3. Part of the manager_ref of the IC is the VM object (foreign key), so the disk's location is unique
   only under 1 Vm. (In current models, this modeled going through Hardware model)
4. In targeted refresh, we always expect that each Vm will be saved with all its disks.

Then having the above points, adding :parent_inventory_collection => [:vms], will bring these
implications:
1. By archiving/deleting Vm, we can no longer see the disk, because those were owned by the Vm. Any
   archival/deletion of the Disk model, must be then done by cascade delete/hooks logic.
2. Having Vm as a parent ensures we always process it first. So e.g. when providing no Vms for saving
   we would have no graph dependency (no data --> no edges --> no dependencies) and Disk could be
   archived/removed before the Vm, while we always want to archive the VM first.
3. For targeted refresh, we always expect that all disks are saved with a VM. So for targeting :disks,
   we are not using #manager_uuids attribute, since the scope is "all disks of all targeted VMs", so we
   always use #manager_uuids of the parent. (that is why :parent_inventory_collections and
   :manager_uuids are mutually exclusive attributes)
4. For automatically building the #targeted_arel query, we need the parent to know what is the root node.
   While this information can be introspected from the data, it creates a scope for create&update&delete,
   which means it has to work with no data provided (causing delete all). So with no data we cannot
   introspect anything.
  • manager_uuids (Array|Proc) (defaults to: []) —

    Array of manager_uuids of the InventoryObjects we want to create/update/delete. Using this attribute, the db_collection_for_comparison will be automatically limited by the manager_uuids, in a case of a simple relation. In a case of a complex relation, we can leverage :manager_uuids in a custom :targeted_arel. We can pass also lambda, for lazy_evaluation.

  • all_manager_uuids (Array) (defaults to: nil) —

    Array of all manager_uuids of the InventoryObjects. With the :targeted true, having this parameter defined will invoke only :delete_method on a complement of this set, making sure the DB has only this set of data after. This :attribute serves for deleting of top level InventoryCollections, i.e. InventoryCollections having parent_inventory_collections nil. The deleting of child collections is already handled by the scope of the parent_inventory_collections and using Rails :dependent => :destroy,

  • targeted_arel (Proc) (defaults to: nil) —

    A callable block that receives this InventoryCollection as a first argument. In there we can leverage a :parent_inventory_collections or :manager_uuids to limit the query based on the manager_uuids available. Example:

    targeted_arel = lambda do |inventory_collection|
  # Getting ems_refs of parent :vms and :miq_templates
  manager_uuids = inventory_collection.parent_inventory_collections.collect(&:manager_uuids).flatten
  inventory_collection.db_collection_for_comparison.hardwares.joins(:vm_or_template).where(
    'vms' => {:ems_ref => manager_uuids}
  )
end

inventory_collection = InventoryCollection.new({
                         :model_class                 => ::Hardware,
                         :association                 => :hardwares,
                         :parent_inventory_collection => [:vms, :miq_templates],
                         :targeted_arel               => targeted_arel,
                       })
  • targeted (Boolean) (defaults to: nil) —

    True if the collection is targeted, in that case it will be leveraging :manager_uuids :parent_inventory_collections and :targeted_arel to save a subgraph of a data.

  • manager_ref_allowed_nil (Array) (defaults to: nil) —

    Array of symbols having manager_ref columns, that are a foreign key an can be nil. Given the table are shared by many providers, it can happen, that the table is used only partially. Then it can happen we want to allow certain foreign keys to be nil, while being sure the referential integrity is not broken. Of course the DB Foreign Key can’t be created in this case, so we should try to avoid this usecase by a proper modeling.

  • use_ar_object (Boolean) (defaults to: nil) —

    True or False. Whether we need to initialize AR object as part of the saving it’s needed if the model have special setters, serialize of columns, etc. This setting is relevant only for the batch saver strategy.

  • batch_extra_attributes (Array) (defaults to: []) —

    Array of symbols marking which extra attributes we want to store into the db. These extra attributes might be a product of :use_ar_object assignment and we need to specify them manually, if we want to use a batch saving strategy and we have models that populate attributes as a side effect.



412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 412

def initialize(model_class: nil, manager_ref: nil, association: nil, parent: nil, strategy: nil,
               custom_save_block: nil, delete_method: nil, dependency_attributes: nil,
               attributes_blacklist: nil, attributes_whitelist: nil, complete: nil, update_only: nil,
               check_changed: nil, arel: nil, default_values: {}, create_only: nil,
               inventory_object_attributes: nil, name: nil, saver_strategy: nil,
               parent_inventory_collections: nil, manager_uuids: [], all_manager_uuids: nil, targeted_arel: nil,
               targeted: nil, manager_ref_allowed_nil: nil, secondary_refs: {}, use_ar_object: nil,
               custom_reconnect_block: nil, batch_extra_attributes: [])
  @model_class            = model_class
  @manager_ref            = manager_ref || [:ems_ref]
  @secondary_refs         = secondary_refs
  @association            = association
  @parent                 = parent || nil
  @arel                   = arel
  @dependency_attributes  = dependency_attributes || {}
  @strategy               = process_strategy(strategy)
  @delete_method          = delete_method || :destroy
  @custom_save_block      = custom_save_block
  @custom_reconnect_block = custom_reconnect_block
  @check_changed          = check_changed.nil? ? true : check_changed
  @internal_attributes    = i( __parent_inventory_collections)
  @complete               = complete.nil? ? true : complete
  @update_only            = update_only.nil? ? false : update_only
  @create_only            = create_only.nil? ? false : create_only
  @default_values         = default_values
  @name                   = name || association || model_class.to_s.demodulize.tableize
  @saver_strategy         = process_saver_strategy(saver_strategy)
  @use_ar_object          = use_ar_object || false
  @batch_extra_attributes = batch_extra_attributes

  @manager_ref_allowed_nil = manager_ref_allowed_nil || []

  # Targeted mode related attributes
  # TODO(lsmola) Should we refactor this to use references too?
  @all_manager_uuids            = all_manager_uuids
  @parent_inventory_collections = parent_inventory_collections
  @targeted_arel                = targeted_arel
  @targeted                     = !!targeted

  @inventory_object_attributes = inventory_object_attributes

  @saved                          ||= false
  @attributes_blacklist             = Set.new
  @attributes_whitelist             = Set.new
  @transitive_dependency_attributes = Set.new
  @dependees                        = Set.new
  @data_storage = ::InventoryRefresh::InventoryCollection::DataStorage.new(self, secondary_refs)
  @references_storage = ::InventoryRefresh::InventoryCollection::ReferencesStorage.new(index_proxy)
  @targeted_scope = ::InventoryRefresh::InventoryCollection::ReferencesStorage.new(index_proxy).merge!(manager_uuids)

  @created_records = []
  @updated_records = []
  @deleted_records = []

  blacklist_attributes!(attributes_blacklist) if attributes_blacklist.present?
  whitelist_attributes!(attributes_whitelist) if attributes_whitelist.present?
end

Instance Attribute Details

#all_manager_uuids ⇒ Array?

If present, InventoryCollection switches into delete_complement mode, where it will delete every record from the DB, that is not present in this list. This is used for the batch processing, where we don’t know which InventoryObject should be deleted, but we know all manager_uuids of all InventoryObject objects that exists in the provider.

Returns:

  • (Array, nil) —

    nil or a list of all :manager_uuids that are present in the Provider’s InventoryCollection.



75
76
77
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 75

def all_manager_uuids
  @all_manager_uuids
end

#arel ⇒ Object (readonly)

Returns the value of attribute arel.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def arel
  @arel
end

#association ⇒ Object (readonly)

Returns the value of attribute association.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def association
  @association
end

#attributes_blacklist ⇒ Object

Returns the value of attribute attributes_blacklist.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def attributes_blacklist
  @attributes_blacklist
end

#attributes_whitelist ⇒ Object

Returns the value of attribute attributes_whitelist.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def attributes_whitelist
  @attributes_whitelist
end

#batch_extra_attributes ⇒ Object (readonly)

Returns the value of attribute batch_extra_attributes.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def batch_extra_attributes
  @batch_extra_attributes
end

#check_changed ⇒ Object (readonly)

Returns the value of attribute check_changed.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def check_changed
  @check_changed
end

#complete ⇒ Object (readonly)

Returns the value of attribute complete.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def complete
  @complete
end

#create_only ⇒ Object (readonly)

Returns the value of attribute create_only.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def create_only
  @create_only
end

#created_records ⇒ Object (readonly)

Returns the value of attribute created_records.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def created_records
  @created_records
end

#custom_reconnect_block ⇒ Object (readonly)

Returns the value of attribute custom_reconnect_block.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def custom_reconnect_block
  @custom_reconnect_block
end

#custom_save_block ⇒ Object (readonly)

Returns the value of attribute custom_save_block.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def custom_save_block
  @custom_save_block
end

#data_collection_finalized ⇒ Boolean

Returns A true value marks that we collected all the data of the InventoryCollection, meaning we also collected all the references.

Returns:

  • (Boolean) —

    A true value marks that we collected all the data of the InventoryCollection, meaning we also collected all the references.



58
59
60
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 58

def data_collection_finalized
  @data_collection_finalized
end

#data_storage ⇒ InventoryRefresh::InventoryCollection::DataStorage

Returns An InventoryCollection encapsulating all data with indexes.

Returns:



62
63
64
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 62

def data_storage
  @data_storage
end

#default_values ⇒ Object (readonly)

Returns the value of attribute default_values.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def default_values
  @default_values
end

#delete_method ⇒ Object (readonly)

Returns the value of attribute delete_method.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def delete_method
  @delete_method
end

#deleted_records ⇒ Object (readonly)

Returns the value of attribute deleted_records.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def deleted_records
  @deleted_records
end

#dependees ⇒ Set

Returns A set of InventoryCollection objects that depends on this InventoryCollection object.

Returns:

  • (Set) —

    A set of InventoryCollection objects that depends on this InventoryCollection object.



78
79
80
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 78

def dependees
  @dependees
end

#dependency_attributes ⇒ Object (readonly)

Returns the value of attribute dependency_attributes.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def dependency_attributes
  @dependency_attributes
end

#internal_attributes ⇒ Object (readonly)

Returns the value of attribute internal_attributes.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def internal_attributes
  @internal_attributes
end

#inventory_object_attributes ⇒ Object (readonly)

Returns the value of attribute inventory_object_attributes.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def inventory_object_attributes
  @inventory_object_attributes
end

#manager_ref ⇒ Object (readonly)

Returns the value of attribute manager_ref.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def manager_ref
  @manager_ref
end

#manager_ref_allowed_nil ⇒ Object (readonly)

Returns the value of attribute manager_ref_allowed_nil.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def manager_ref_allowed_nil
  @manager_ref_allowed_nil
end

#model_class ⇒ Object (readonly)

Returns the value of attribute model_class.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def model_class
  @model_class
end

#name ⇒ Object (readonly)

Returns the value of attribute name.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def name
  @name
end

#parent ⇒ Object (readonly)

Returns the value of attribute parent.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def parent
  @parent
end

#parent_inventory_collections ⇒ Array<Symbol>

Returns @see #parent_inventory_collections documentation of InventoryCollection.new kwargs parameters.

Returns:

  • (Array<Symbol>) —

    @see #parent_inventory_collections documentation of InventoryCollection.new kwargs parameters



82
83
84
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 82

def parent_inventory_collections
  @parent_inventory_collections
end

#references_storage ⇒ Object (readonly)

Returns the value of attribute references_storage.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def references_storage
  @references_storage
end

#saved ⇒ Boolean

Returns true if this collection is already saved into the DB. E.g. InventoryCollections with DB only strategy are marked as saved. This causes InventoryCollection not being a dependency for any other InventoryCollection, since it is already persisted into the DB.

Returns:

  • (Boolean) —

    true if this collection is already saved into the DB. E.g. InventoryCollections with DB only strategy are marked as saved. This causes InventoryCollection not being a dependency for any other InventoryCollection, since it is already persisted into the DB.



67
68
69
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 67

def saved
  @saved
end

#saver_strategy ⇒ Object (readonly)

Returns the value of attribute saver_strategy.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def saver_strategy
  @saver_strategy
end

#strategy ⇒ Object (readonly)

Returns the value of attribute strategy.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def strategy
  @strategy
end

#targeted ⇒ Object (readonly)

Returns the value of attribute targeted.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def targeted
  @targeted
end

#targeted_arel ⇒ Object (readonly)

Returns the value of attribute targeted_arel.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def targeted_arel
  @targeted_arel
end

#targeted_scope ⇒ Object (readonly)

Returns the value of attribute targeted_scope.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def targeted_scope
  @targeted_scope
end

#transitive_dependency_attributes ⇒ Object (readonly)

Returns the value of attribute transitive_dependency_attributes.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def transitive_dependency_attributes
  @transitive_dependency_attributes
end

#update_only ⇒ Object (readonly)

Returns the value of attribute update_only.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def update_only
  @update_only
end

#updated_records ⇒ Object (readonly)

Returns the value of attribute updated_records.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def updated_records
  @updated_records
end

#use_ar_object ⇒ Object (readonly)

Returns the value of attribute use_ar_object.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def use_ar_object
  @use_ar_object
end

Instance Method Details

#association_to_base_class_mapping ⇒ Hash{Symbol => String}

Returns Hash with association name mapped to base class of the association.

Returns:

  • (Hash{Symbol => String}) —

    Hash with association name mapped to base class of the association



942
943
944
945
946
947
948
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 942

def association_to_base_class_mapping
  return {} unless model_class

  @association_to_base_class_mapping ||= model_class.reflect_on_all_associations.each_with_object({}) do |x, obj|
    obj[x.name] = x.klass.base_class.name unless x.polymorphic?
  end
end

#association_to_foreign_key_mapping ⇒ Hash{Symbol => String}

Returns Hash with association name mapped to foreign key column name.

Returns:

  • (Hash{Symbol => String}) —

    Hash with association name mapped to foreign key column name



906
907
908
909
910
911
912
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 906

def association_to_foreign_key_mapping
  return {} unless model_class

  @association_to_foreign_key_mapping ||= belongs_to_associations.each_with_object({}) do |x, obj|
    obj[x.name] = x.foreign_key
  end
end

#association_to_foreign_type_mapping ⇒ Hash{Symbol => String}

Returns Hash with association name mapped to polymorphic foreign key type column name.

Returns:

  • (Hash{Symbol => String}) —

    Hash with association name mapped to polymorphic foreign key type column name



924
925
926
927
928
929
930
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 924

def association_to_foreign_type_mapping
  return {} unless model_class

  @association_to_foreign_type_mapping ||= model_class.reflect_on_all_associations.each_with_object({}) do |x, obj|
    obj[x.name] = x.foreign_type if x.polymorphic?
  end
end

#base_class_name ⇒ String

Returns Base class name of the model_class of this InventoryCollection.

Returns:

  • (String) —

    Base class name of the model_class of this InventoryCollection



972
973
974
975
976
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 972

def base_class_name
  return "" unless model_class

  @base_class_name ||= model_class.base_class.name
end

#base_columns ⇒ Object 



723
724
725
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 723

def base_columns
  @base_columns ||= unique_index_columns + internal_columns
end

#batch_size ⇒ Integer

Returns default batch size for talking to the DB.

Returns:

  • (Integer) —

    default batch size for talking to the DB



996
997
998
999
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 996

def batch_size
  # TODO(lsmola) mode to the settings
  1000
end

#batch_size_pure_sql ⇒ Integer

Returns default batch size for talking to the DB if not using ApplicationRecord objects.

Returns:

  • (Integer) —

    default batch size for talking to the DB if not using ApplicationRecord objects



1002
1003
1004
1005
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1002

def batch_size_pure_sql
  # TODO(lsmola) mode to the settings
  10_000
end

#belongs_to_associations ⇒ Array<ActiveRecord::Reflection::BelongsToReflection">

Returns All belongs_to associations.

Returns:

  • (Array<ActiveRecord::Reflection::BelongsToReflection">) —

    All belongs_to associations



901
902
903
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 901

def belongs_to_associations
  model_class.reflect_on_all_associations.select { |x| x.kind_of?(ActiveRecord::Reflection::BelongsToReflection) }
end

#blacklist_attributes!(attributes) ⇒ Array<Symbol>

Add passed attributes to blacklist. The manager_ref attributes cannot be blacklisted, otherwise we will not be able to identify the inventory_object. We do not automatically remove attributes causing fixed dependencies, so beware that without them, you won’t be able to create the record.

Parameters:

  • attributes (Array<Symbol>) —

    Attributes we want to blacklist

Returns:

  • (Array<Symbol>) —

    All blacklisted attributes



865
866
867
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 865

def blacklist_attributes!(attributes)
  self.attributes_blacklist += attributes - (fixed_attributes + internal_attributes)
end

#build_multi_selection_condition(hashes, keys = manager_ref) ⇒ String

Builds a multiselection conditions like (table1.a = a1 AND table2.b = b1) OR (table1.a = a2 AND table2.b = b2)

Parameters:

  • hashes (Array<Hash>) —

    data we want to use for the query

  • keys (Array<Symbol>) (defaults to: manager_ref) —

    keys of attributes involved

Returns:

  • (String) —

    A condition usable in .where of an ActiveRecord relation



1022
1023
1024
1025
1026
1027
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1022

def build_multi_selection_condition(hashes, keys = manager_ref)
  arel_table = model_class.arel_table
  # We do pure SQL OR, since Arel is nesting every .or into another parentheses, otherwise this would be just
  # inject(:or) instead of to_sql with .join(" OR ")
  hashes.map { |hash| "(#{keys.map { |key| arel_table[key].eq(hash[key]) }.inject(:and).to_sql})" }.join(" OR ")
end

#check_changed? ⇒ Boolean

Returns true means we want to call .changed? on every ActiveRecord object before saving it.

Returns:

  • (Boolean) —

    true means we want to call .changed? on every ActiveRecord object before saving it



533
534
535
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 533

def check_changed?
  check_changed
end

#clone ⇒ InventoryCollection

Returns a shallow copy of InventoryCollection, the copy will share data_storage of the original collection, otherwise we would be copying a lot of records in memory.

Returns:

  • (InventoryCollection) —

    a shallow copy of InventoryCollection, the copy will share data_storage of the original collection, otherwise we would be copying a lot of records in memory.



881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 881

def clone
  cloned = self.class.new(:model_class           => model_class,
                          :manager_ref           => manager_ref,
                          :association           => association,
                          :parent                => parent,
                          :arel                  => arel,
                          :strategy              => strategy,
                          :saver_strategy        => saver_strategy,
                          :custom_save_block     => custom_save_block,
                          # We want cloned IC to be update only, since this is used for cycle resolution
                          :update_only           => true,
                          # Dependency attributes need to be a hard copy, since those will differ for each
                          # InventoryCollection
                          :dependency_attributes => dependency_attributes.clone)

  cloned.data_storage = data_storage
  cloned
end

#complete? ⇒ Boolean

Returns true means the data is not complete, leading to only creating and updating data.

Returns:

  • (Boolean) —

    true means the data is not complete, leading to only creating and updating data



544
545
546
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 544

def complete?
  complete
end

#create_allowed? ⇒ Boolean

Returns true means we will delete/soft-delete data.

Returns:

  • (Boolean) —

    true means we will delete/soft-delete data



559
560
561
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 559

def create_allowed?
  !update_only?
end

#create_only? ⇒ Boolean

Returns true means that only create of new data is allowed.

Returns:

  • (Boolean) —

    true means that only create of new data is allowed



564
565
566
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 564

def create_only?
  create_only
end

#data_collection_finalized? ⇒ Boolean

Returns true if no more data will be added to this InventoryCollection object, that usually happens after the parsing step is finished.

Returns:

  • (Boolean) —

    true if no more data will be added to this InventoryCollection object, that usually happens after the parsing step is finished



729
730
731
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 729

def data_collection_finalized?
  data_collection_finalized
end

#db_collection_for_comparison ⇒ ActiveRecord::Relation

Returns A relation that can fetch all data of this InventoryCollection from the DB.

Returns:

  • (ActiveRecord::Relation) —

    A relation that can fetch all data of this InventoryCollection from the DB



1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1030

def db_collection_for_comparison
  if targeted?
    if targeted_arel.respond_to?(:call)
      targeted_arel.call(self)
    elsif parent_inventory_collections.present?
      targeted_arel_default
    else
      targeted_iterator_for(targeted_scope.primary_references)
    end
  else
    full_collection_for_comparison
  end
end

#db_collection_for_comparison_for(references) ⇒ ActiveRecord::Relation

Builds an ActiveRecord::Relation that can fetch all the references from the DB

Parameters:

Returns:

  • (ActiveRecord::Relation) —

    relation that can fetch all the references from the DB



1097
1098
1099
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1097

def db_collection_for_comparison_for(references)
  full_collection_for_comparison.where(targeted_selection_for(references))
end

#db_collection_for_comparison_for_complement_of(manager_uuids_set) ⇒ ActiveRecord::Relation

Builds an ActiveRecord::Relation that can fetch complement of all the references from the DB

Parameters:

  • manager_uuids_set (Array<String>) —

    passed references

Returns:

  • (ActiveRecord::Relation) —

    relation that can fetch complement of all the references from the DB



1105
1106
1107
1108
1109
1110
1111
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1105

def db_collection_for_comparison_for_complement_of(manager_uuids_set)
  # TODO(lsmola) this should have the build_multi_selection_condition, like in the method above
  # TODO(lsmola) this query will be highly ineffective, we will try approach with updating a timestamp of all
  # records, then we can get list of all records that were not update. That would be equivalent to result of this
  # more effective query and without need of all manager_uuids
  full_collection_for_comparison.where.not(manager_ref.first => manager_uuids_set)
end

#delete_allowed? ⇒ Boolean

Returns true means we will delete/soft-delete data.

Returns:

  • (Boolean) —

    true means we will delete/soft-delete data



554
555
556
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 554

def delete_allowed?
  complete? && !update_only?
end

#dependencies ⇒ Array<InventoryRefresh::InventoryCollection>

Returns all unique non saved dependencies.

Returns:



842
843
844
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 842

def dependencies
  filtered_dependency_attributes.values.map(&:to_a).flatten.uniq.reject(&:saved?)
end

#dependency_attributes_for(inventory_collections) ⇒ Array<InventoryRefresh::InventoryCollection>

Returns what attributes are causing a dependencies to certain InventoryCollection objects.

Parameters:

Returns:



851
852
853
854
855
856
857
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 851

def dependency_attributes_for(inventory_collections)
  attributes = Set.new
  inventory_collections.each do |inventory_collection|
    attributes += filtered_dependency_attributes.select { |_key, value| value.include?(inventory_collection) }.keys
  end
  attributes
end

#filtered_dependency_attributes ⇒ Hash{Symbol => Set}

List attributes causing a dependency and filters them by attributes_blacklist and attributes_whitelist

Returns:

  • (Hash{Symbol => Set}) —

    attributes causing a dependency and filtered by blacklist and whitelist



801
802
803
804
805
806
807
808
809
810
811
812
813
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 801

def filtered_dependency_attributes
  filtered_attributes = dependency_attributes

  if attributes_blacklist.present?
    filtered_attributes = filtered_attributes.reject { |key, _value| attributes_blacklist.include?(key) }
  end

  if attributes_whitelist.present?
    filtered_attributes = filtered_attributes.select { |key, _value| attributes_whitelist.include?(key) }
  end

  filtered_attributes
end

#fixed_attributes ⇒ Array<Symbol>

Attributes that are needed to be able to save the record, i.e. attributes that are part of the unique index and attributes with presence validation or NOT NULL constraint

Returns:

  • (Array<Symbol>) —

    attributes that are needed for saving of the record



819
820
821
822
823
824
825
826
827
828
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 819

def fixed_attributes
  if model_class
    presence_validators = model_class.validators.detect { |x| x.kind_of?(ActiveRecord::Validations::PresenceValidator) }
  end
  # Attributes that has to be always on the entity, so attributes making unique index of the record + attributes
  # that have presence validation
  fixed_attributes = manager_ref
  fixed_attributes += presence_validators.attributes if presence_validators.present?
  fixed_attributes
end

#fixed_dependencies ⇒ Object

Returns fixed dependencies, which are the ones we can’t move, because we wouldn’t be able to save the data



833
834
835
836
837
838
839
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 833

def fixed_dependencies
  fixed_attrs = fixed_attributes

  filtered_dependency_attributes.each_with_object(Set.new) do |(key, value), fixed_deps|
    fixed_deps.merge(value) if fixed_attrs.include?(key)
  end.reject(&:saved?)
end

#fixed_foreign_keys ⇒ Array<Symbol>

Returns List of all column names that are foreign keys and cannot removed, otherwise we couldn’t save the record.

Returns:

  • (Array<Symbol>) —

    List of all column names that are foreign keys and cannot removed, otherwise we couldn’t save the record



959
960
961
962
963
964
965
966
967
968
969
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 959

def fixed_foreign_keys
  # Foreign keys that are part of a manager_ref must be present, otherwise the record would get lost. This is a
  # minimum check we can do to not break a referential integrity.
  return @fixed_foreign_keys_cache unless @fixed_foreign_keys_cache.nil?

  manager_ref_set = (manager_ref - manager_ref_allowed_nil)
  @fixed_foreign_keys_cache = manager_ref_set.map { |x| association_to_foreign_key_mapping[x] }.compact
  @fixed_foreign_keys_cache += foreign_keys & manager_ref
  @fixed_foreign_keys_cache.map!(&:to_sym)
  @fixed_foreign_keys_cache
end

#foreign_key_to_association_mapping ⇒ Hash{String => Hash}

Returns Hash with foreign_key column name mapped to association name.

Returns:

  • (Hash{String => Hash}) —

    Hash with foreign_key column name mapped to association name



915
916
917
918
919
920
921
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 915

def foreign_key_to_association_mapping
  return {} unless model_class

  @foreign_key_to_association_mapping ||= belongs_to_associations.each_with_object({}) do |x, obj|
    obj[x.foreign_key] = x.name
  end
end

#foreign_keys ⇒ Array<Symbol>

Returns List of all column names that are foreign keys.

Returns:

  • (Array<Symbol>) —

    List of all column names that are foreign keys



951
952
953
954
955
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 951

def foreign_keys
  return [] unless model_class

  @foreign_keys_cache ||= belongs_to_associations.map(&:foreign_key).map!(&:to_sym)
end

#foreign_type_to_association_mapping ⇒ Hash{Symbol => String}

Returns Hash with polymorphic foreign key type column name mapped to association name.

Returns:

  • (Hash{Symbol => String}) —

    Hash with polymorphic foreign key type column name mapped to association name



933
934
935
936
937
938
939
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 933

def foreign_type_to_association_mapping
  return {} unless model_class

  @foreign_type_to_association_mapping ||= model_class.reflect_on_all_associations.each_with_object({}) do |x, obj|
    obj[x.foreign_type] = x.name if x.polymorphic?
  end
end

#full_collection_for_comparison ⇒ ActiveRecord::Relation

Returns relation that can fetch all the references from the DB.

Returns:

  • (ActiveRecord::Relation) —

    relation that can fetch all the references from the DB



1114
1115
1116
1117
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1114

def full_collection_for_comparison
  return arel unless arel.nil?
  parent.send(association)
end

#inspect ⇒ String

Returns a concise form of the InventoryCollection for easy logging.

Returns:

  • (String) —

    a concise form of the InventoryCollection for easy logging



991
992
993
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 991

def inspect
  to_s
end

#internal_columns ⇒ Object 



697
698
699
700
701
702
703
704
705
706
707
708
709
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 697

def internal_columns
  return @internal_columns if @internal_columns

  @internal_columns = [] + internal_timestamp_columns
  @internal_columns << :type if supports_sti?
  @internal_columns << :resource_timestamps_max if supports_resource_timestamps_max?
  @internal_columns << :resource_timestamps if supports_resource_timestamps?
  @internal_columns << :resource_timestamp if supports_resource_timestamp?
  @internal_columns << :resource_versions_max if supports_resource_versions_max?
  @internal_columns << :resource_versions if supports_resource_versions?
  @internal_columns << :resource_version if supports_resource_version?
  @internal_columns
end

#internal_timestamp_columns ⇒ Object 



711
712
713
714
715
716
717
718
719
720
721
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 711

def internal_timestamp_columns
  return @internal_timestamp_columns if @internal_timestamp_columns

  @internal_timestamp_columns = []
  @internal_timestamp_columns << :created_on if supports_created_on?
  @internal_timestamp_columns << :created_at if supports_created_at?
  @internal_timestamp_columns << :updated_on if supports_updated_on?
  @internal_timestamp_columns << :updated_at if supports_updated_at?

  @internal_timestamp_columns
end

#inventory_object?(value) ⇒ Boolean

Returns true is value is kind of InventoryRefresh::InventoryObject.

Parameters:

  • value (Object) —

    Object we want to test

Returns:

  • (Boolean) —

    true is value is kind of InventoryRefresh::InventoryObject



735
736
737
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 735

def inventory_object?(value)
  value.kind_of?(::InventoryRefresh::InventoryObject)
end

#inventory_object_lazy?(value) ⇒ Boolean

Returns true is value is kind of InventoryRefresh::InventoryObjectLazy.

Parameters:

  • value (Object) —

    Object we want to test

Returns:

  • (Boolean) —

    true is value is kind of InventoryRefresh::InventoryObjectLazy



741
742
743
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 741

def inventory_object_lazy?(value)
  value.kind_of?(::InventoryRefresh::InventoryObjectLazy)
end

#manager_ref_to_cols ⇒ Array<String>

Convert manager_ref list of attributes to list of DB columns

Returns:

  • (Array<String>) —

    true is processing of this InventoryCollection will be in targeted mode



790
791
792
793
794
795
796
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 790

def manager_ref_to_cols
  # TODO(lsmola) this should contain the polymorphic _type, otherwise the IC with polymorphic unique key will get
  # conflicts
  manager_ref.map do |ref|
    association_to_foreign_key_mapping[ref] || ref
  end
end

#manager_uuids ⇒ Array<String>

Returns a list of stringified uuids of all scoped InventoryObjects, which is used for scoping in targeted mode

Returns:

  • (Array<String>) —

    list of stringified uuids of all scoped InventoryObjects



1010
1011
1012
1013
1014
1015
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1010

def manager_uuids
  # TODO(lsmola) LEGACY: this is still being used by :targetel_arel definitions and it expects array of strings
  raise "This works only for :manager_ref size 1" if manager_ref.size > 1
  key = manager_ref.first
  transform_references_to_hashes(targeted_scope.primary_references).map { |x| x[key] }
end

#new_inventory_object(hash) ⇒ InventoryRefresh::InventoryObject

Creates InventoryRefresh::InventoryObject object from passed hash data

Parameters:

  • hash (Hash) —

    Object data

Returns:



1123
1124
1125
1126
1127
1128
1129
1130
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1123

def new_inventory_object(hash)
  manager_ref.each do |x|
    # TODO(lsmola) with some effort, we can do this, but it's complex
    raise "A lazy_find with a :key can't be a part of the manager_uuid" if inventory_object_lazy?(hash[x]) && hash[x].key
  end

  inventory_object_class.new(self, hash)
end

#noop? ⇒ Boolean

True if processing of this InventoryCollection object would lead to no operations. Then we use this marker to stop processing of the InventoryCollector object very soon, to avoid a lot of unnecessary Db queries, etc.

Returns:

  • (Boolean) —

    true if processing of this InventoryCollection object would lead to no operations.



759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 759

def noop?
  # If this InventoryCollection doesn't do anything. it can easily happen for targeted/batched strategies.
  if targeted?
    if parent_inventory_collections.nil? && targeted_scope.primary_references.blank? &&
       all_manager_uuids.nil? && parent_inventory_collections.blank? && custom_save_block.nil? &&
       skeletal_primary_index.blank?
      # It's a noop Parent targeted InventoryCollection
      true
    elsif !parent_inventory_collections.nil? && parent_inventory_collections.all? { |x| x.targeted_scope.primary_references.blank? } &&
          skeletal_primary_index.blank?
      # It's a noop Child targeted InventoryCollection
      true
    else
      false
    end
  elsif data.blank? && !delete_allowed? && skeletal_primary_index.blank?
    # If we have no data to save and delete is not allowed, we can just skip
    true
  else
    false
  end
end

#object_index_with_keys(keys, record) ⇒ String

Builds string uuid from passed Object and keys

Parameters:

  • keys (Array<Symbol>) —

    Indexes into the Hash data

  • record (ApplicationRecord) —

    ActiveRecord record

Returns:

  • (String) —

    Concatenated values on keys from data



750
751
752
753
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 750

def object_index_with_keys(keys, record)
  # TODO(lsmola) remove, last usage is in k8s reconnect logic
  build_stringified_reference_for_record(record, keys)
end

#parallel_safe? ⇒ Boolean

Returns true if we are using a saver strategy that allows saving in parallel processes.

Returns:

  • (Boolean) —

    true if we are using a saver strategy that allows saving in parallel processes



579
580
581
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 579

def parallel_safe?
  @parallel_safe_cache ||= i(concurrent_safe concurrent_safe_batch).include?(saver_strategy)
end

#process_saver_strategy(saver_strategy) ⇒ Symbol

Processes passed saver strategy

Parameters:

  • saver_strategy (Symbol) —

    Passed saver strategy

Returns:

  • (Symbol) —

    Returns back the passed strategy if supported, or raises exception



495
496
497
498
499
500
501
502
503
504
505
506
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 495

def process_saver_strategy(saver_strategy)
  return :default unless saver_strategy

  saver_strategy = saver_strategy.to_sym
  case saver_strategy
  when :default, :batch, :concurrent_safe, :concurrent_safe_batch
    saver_strategy
  else
    raise "Unknown InventoryCollection saver strategy: :#{saver_strategy}, allowed strategies are "\
          ":default, :batch, :concurrent_safe and :concurrent_safe_batch"
  end
end

#process_strategy(strategy_name) ⇒ Symbol

Processes passed strategy, modifies :data_collection_finalized and :saved attributes for db only strategies

Parameters:

  • strategy_name (Symbol) —

    Passed saver strategy

Returns:

  • (Symbol) —

    Returns back the passed strategy if supported, or raises exception



512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 512

def process_strategy(strategy_name)
  self.data_collection_finalized = false

  return unless strategy_name

  strategy_name = strategy_name.to_sym
  case strategy_name
  when :local_db_cache_all
    self.data_collection_finalized = true
    self.saved = true
  when :local_db_find_references
    self.saved = true
  when :local_db_find_missing_references
  else
    raise "Unknown InventoryCollection strategy: :#{strategy_name}, allowed strategies are :local_db_cache_all, "\
          ":local_db_find_references and :local_db_find_missing_references."
  end
  strategy_name
end

#saveable? ⇒ Boolean

Returns true if all dependencies have all data persisted.

Returns:

  • (Boolean) —

    true if all dependencies have all data persisted



574
575
576
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 574

def saveable?
  dependencies.all?(&:saved?)
end

#saved? ⇒ Boolean

Returns true if the whole InventoryCollection object has all data persisted.

Returns:

  • (Boolean) —

    true if the whole InventoryCollection object has all data persisted



569
570
571
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 569

def saved?
  saved
end

#store_created_records(records) ⇒ Object

Caches what records were created, for later use, e.g. post provision behavior

Parameters:

  • records (Array<ApplicationRecord, Hash>) —

    list of stored records



473
474
475
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 473

def store_created_records(records)
  @created_records.concat(records_identities(records))
end

#store_deleted_records(records) ⇒ Object

Caches what records were deleted/soft-deleted, for later use, e.g. post provision behavior

Parameters:

  • records (Array<ApplicationRecord, Hash>) —

    list of stored records



487
488
489
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 487

def store_deleted_records(records)
  @deleted_records.concat(records_identities(records))
end

#store_updated_records(records) ⇒ Object

Caches what records were updated, for later use, e.g. post provision behavior

Parameters:

  • records (Array<ApplicationRecord, Hash>) —

    list of stored records



480
481
482
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 480

def store_updated_records(records)
  @updated_records.concat(records_identities(records))
end

#supports_created_at? ⇒ Boolean

Returns true if the model_class has created_at column.

Returns:

  • (Boolean) —

    true if the model_class has created_at column



606
607
608
609
610
611
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 606

def supports_created_at?
  if @supports_created_at_cache.nil?
    @supports_created_at_cache = (model_class.column_names.include?("created_at") && ActiveRecord::Base.record_timestamps)
  end
  @supports_created_at_cache
end

#supports_created_on? ⇒ Boolean

Returns true if the model_class has created_on column.

Returns:

  • (Boolean) —

    true if the model_class has created_on column



590
591
592
593
594
595
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 590

def supports_created_on?
  if @supports_created_on_cache.nil?
    @supports_created_on_cache = (model_class.column_names.include?("created_on") && ActiveRecord::Base.record_timestamps)
  end
  @supports_created_on_cache
end

#supports_resource_timestamp? ⇒ Boolean

Returns true if the model_class has resource_timestamp column.

Returns:

  • (Boolean) —

    true if the model_class has resource_timestamp column



632
633
634
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 632

def supports_resource_timestamp?
  @supports_resource_timestamp_cache ||= model_class.column_names.include?("resource_timestamp")
end

#supports_resource_timestamps? ⇒ Boolean

Returns true if the model_class has resource_timestamps column.

Returns:

  • (Boolean) —

    true if the model_class has resource_timestamps column



627
628
629
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 627

def supports_resource_timestamps?
  @supports_resource_timestamps_cache ||= model_class.column_names.include?("resource_timestamps")
end

#supports_resource_timestamps_max? ⇒ Boolean

Returns true if the model_class has resource_timestamps_max column.

Returns:

  • (Boolean) —

    true if the model_class has resource_timestamps_max column



622
623
624
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 622

def supports_resource_timestamps_max?
  @supports_resource_timestamps_max_cache ||= model_class.column_names.include?("resource_timestamps_max")
end

#supports_resource_version? ⇒ Boolean

Returns true if the model_class has resource_version column.

Returns:

  • (Boolean) —

    true if the model_class has resource_version column



647
648
649
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 647

def supports_resource_version?
  @supports_resource_version_cache ||= model_class.column_names.include?("resource_version")
end

#supports_resource_versions? ⇒ Boolean

Returns true if the model_class has resource_versions column.

Returns:

  • (Boolean) —

    true if the model_class has resource_versions column



642
643
644
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 642

def supports_resource_versions?
  @supports_resource_versions_cache ||= model_class.column_names.include?("resource_versions")
end

#supports_resource_versions_max? ⇒ Boolean

Returns true if the model_class has resource_versions_max column.

Returns:

  • (Boolean) —

    true if the model_class has resource_versions_max column



637
638
639
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 637

def supports_resource_versions_max?
  @supports_resource_versions_max_cache ||= model_class.column_names.include?("resource_versions_max")
end

#supports_sti? ⇒ Boolean

Returns true if the model_class supports STI.

Returns:

  • (Boolean) —

    true if the model_class supports STI



584
585
586
587
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 584

def supports_sti?
  @supports_sti_cache = model_class.column_names.include?("type") if @supports_sti_cache.nil?
  @supports_sti_cache
end

#supports_updated_at? ⇒ Boolean

Returns true if the model_class has updated_at column.

Returns:

  • (Boolean) —

    true if the model_class has updated_at column



614
615
616
617
618
619
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 614

def supports_updated_at?
  if @supports_updated_at_cache.nil?
    @supports_updated_at_cache = (model_class.column_names.include?("updated_at") && ActiveRecord::Base.record_timestamps)
  end
  @supports_updated_at_cache
end

#supports_updated_on? ⇒ Boolean

Returns true if the model_class has updated_on column.

Returns:

  • (Boolean) —

    true if the model_class has updated_on column



598
599
600
601
602
603
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 598

def supports_updated_on?
  if @supports_updated_on_cache.nil?
    @supports_updated_on_cache = (model_class.column_names.include?("updated_on") && ActiveRecord::Base.record_timestamps)
  end
  @supports_updated_on_cache
end

#targeted? ⇒ Boolean

Returns true is processing of this InventoryCollection will be in targeted mode.

Returns:

  • (Boolean) —

    true is processing of this InventoryCollection will be in targeted mode



783
784
785
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 783

def targeted?
  targeted
end

#targeted_arel_default ⇒ InventoryRefresh::ApplicationRecordIterator

Builds targeted query limiting the results by the :references defined in parent_inventory_collections

Returns:



1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1047

def targeted_arel_default
  if parent_inventory_collections.collect { |x| x.model_class.base_class }.uniq.count > 1
    raise "Multiple :parent_inventory_collections with different base class are not supported by default. Write "\
          ":targeted_arel manually, or separate [#{self}] into 2 InventoryCollection objects."
  end
  parent_collection = parent_inventory_collections.first
  references        = parent_inventory_collections.map { |x| x.targeted_scope.primary_references }.reduce({}, :merge!)

  parent_collection.targeted_iterator_for(references, full_collection_for_comparison)
end

#targeted_iterator_for(references, query = nil) ⇒ InventoryRefresh::ApplicationRecordIterator

Returns iterator for the passed references and a query

Parameters:

Returns:



1085
1086
1087
1088
1089
1090
1091
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1085

def targeted_iterator_for(references, query = nil)
  InventoryRefresh::ApplicationRecordIterator.new(
    :inventory_collection => self,
    :manager_uuids_set    => references,
    :query                => query
  )
end

#targeted_selection_for(references) ⇒ String

Builds a multiselection conditions like (table1.a = a1 AND table2.b = b1) OR (table1.a = a2 AND table2.b = b2) for passed references

Parameters:

Returns:

  • (String) —

    A condition usable in .where of an ActiveRecord relation



1076
1077
1078
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1076

def targeted_selection_for(references)
  build_multi_selection_condition(transform_references_to_hashes(references))
end

#to_s ⇒ String

Returns a concise form of the inventoryCollection for easy logging.

Returns:

  • (String) —

    a concise form of the inventoryCollection for easy logging



979
980
981
982
983
984
985
986
987
988
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 979

def to_s
  whitelist = ", whitelist: [#{attributes_whitelist.to_a.join(", ")}]" if attributes_whitelist.present?
  blacklist = ", blacklist: [#{attributes_blacklist.to_a.join(", ")}]" if attributes_blacklist.present?

  strategy_name = ", strategy: #{strategy}" if strategy

  name = model_class || association

  "InventoryCollection:<#{name}>#{whitelist}#{blacklist}#{strategy_name}"
end

#transform_references_to_hashes(references) ⇒ Array<Hash>

Gets targeted references and transforms them into list of hashes

Parameters:

  • references (Array, InventoryRefresh::Inventorycollection::TargetedScope) —

    passed references

Returns:

  • (Array<Hash>) —

    References transformed into the array of hashes



1062
1063
1064
1065
1066
1067
1068
1069
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1062

def transform_references_to_hashes(references)
  if references.kind_of?(Array)
    # Sliced InventoryRefresh::Inventorycollection::TargetedScope
    references.map { |x| x.second.full_reference }
  else
    references.values.map(&:full_reference)
  end
end

#unique_index_columns ⇒ Array<Symbol>

Returns all columns that are part of the best fit unique index.

Returns:

  • (Array<Symbol>) —

    all columns that are part of the best fit unique index



652
653
654
655
656
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 652

def unique_index_columns
  return @unique_index_columns if @unique_index_columns

  @unique_index_columns = unique_index_for(unique_index_keys).columns.map(&:to_sym)
end

#unique_index_for(keys) ⇒ ActiveRecord::ConnectionAdapters::IndexDefinition

Finds an index that fits the list of columns (keys) the best

Parameters:

  • keys (Array<Symbol>)

Returns:

  • (ActiveRecord::ConnectionAdapters::IndexDefinition) —

    unique index fitting the keys

Raises:

  • (Exception) —

    if the unique index for the columns was not found



681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 681

def unique_index_for(keys)
  @unique_index_for_keys_cache ||= {}
  @unique_index_for_keys_cache[keys] if @unique_index_for_keys_cache[keys]

  # Find all uniq indexes that that are covering our keys
  uniq_key_candidates = unique_indexes.each_with_object([]) { |i, obj| obj << i if (keys - i.columns.map(&:to_sym)).empty? }

  if @unique_indexes_cache.blank?
    raise "#{self} and its table #{model_class.table_name} must have a unique index defined "\
            "covering columns #{keys} to be able to use saver_strategy :concurrent_safe or :concurrent_safe_batch."
  end

  # Take the uniq key having the least number of columns
  @unique_index_for_keys_cache[keys] = uniq_key_candidates.min_by { |x| x.columns.count }
end

#unique_index_keys ⇒ Object 



658
659
660
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 658

def unique_index_keys
  @unique_index_keys ||= manager_ref_to_cols.map(&:to_sym)
end

#unique_indexes ⇒ Array<ActiveRecord::ConnectionAdapters::IndexDefinition>

Returns array of all unique indexes known to model.

Returns:

  • (Array<ActiveRecord::ConnectionAdapters::IndexDefinition>) —

    array of all unique indexes known to model



663
664
665
666
667
668
669
670
671
672
673
674
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 663

def unique_indexes
  @unique_indexes_cache if @unique_indexes_cache

  @unique_indexes_cache = model_class.connection.indexes(model_class.table_name).select(&:unique)

  if @unique_indexes_cache.blank?
    raise "#{self} and its table #{model_class.table_name} must have a unique index defined, to"\
            " be able to use saver_strategy :concurrent_safe or :concurrent_safe_batch."
  end

  @unique_indexes_cache
end

#update_only? ⇒ Boolean

Returns true means we want to only update data.

Returns:

  • (Boolean) —

    true means we want to only update data



549
550
551
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 549

def update_only?
  update_only
end

#use_ar_object? ⇒ Boolean

Returns true means we want to use ActiveRecord object for writing attributes and we want to perform casting on all columns.

Returns:

  • (Boolean) —

    true means we want to use ActiveRecord object for writing attributes and we want to perform casting on all columns



539
540
541
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 539

def use_ar_object?
  use_ar_object
end

#whitelist_attributes!(attributes) ⇒ Array<Symbol>

Add passed attributes to whitelist. The manager_ref attributes always needs to be in the white list, otherwise we will not be able to identify theinventory_object. We do not automatically add attributes causing fixed dependencies, so beware that without them, you won’t be able to create the record.

Parameters:

  • attributes (Array<Symbol>) —

    Attributes we want to whitelist

Returns:

  • (Array<Symbol>) —

    All whitelisted attributes



875
876
877
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 875

def whitelist_attributes!(attributes)
  self.attributes_whitelist += attributes + (fixed_attributes + internal_attributes)
end