Class: InventoryRefresh::InventoryCollection

Inherits:
Object
  • Object
show all
Defined in:
lib/inventory_refresh/inventory_collection.rb,
lib/inventory_refresh/inventory_collection/graph.rb,
lib/inventory_refresh/inventory_collection/scanner.rb,
lib/inventory_refresh/inventory_collection/reference.rb,
lib/inventory_refresh/inventory_collection/index/proxy.rb,
lib/inventory_refresh/inventory_collection/data_storage.rb,
lib/inventory_refresh/inventory_collection/serialization.rb,
lib/inventory_refresh/inventory_collection/index/type/base.rb,
lib/inventory_refresh/inventory_collection/index/type/data.rb,
lib/inventory_refresh/inventory_collection/references_storage.rb,
lib/inventory_refresh/inventory_collection/index/type/local_db.rb,
lib/inventory_refresh/inventory_collection/index/type/skeletal.rb

Overview

For more usage examples please follow spec examples in

  • spec/models/inventory_refresh/save_inventory/single_inventory_collection_spec.rb

  • spec/models/inventory_refresh/save_inventory/acyclic_graph_of_inventory_collections_spec.rb

  • spec/models/inventory_refresh/save_inventory/graph_of_inventory_collections_spec.rb

  • spec/models/inventory_refresh/save_inventory/graph_of_inventory_collections_targeted_refresh_spec.rb

  • spec/models/inventory_refresh/save_inventory/strategies_and_references_spec.rb

Examples:

storing Vm model data into the DB


@ems = ManageIQ::Providers::BaseManager.first
puts @ems.vms.collect(&:ems_ref) # => []

# Init InventoryCollection
vms_inventory_collection = ::InventoryRefresh::InventoryCollection.new(
  :model_class => ManageIQ::Providers::CloudManager::Vm, :parent => @ems, :association => :vms
)

# Fill InventoryCollection with data
# Starting with no vms, lets add vm1 and vm2
vms_inventory_collection.build(:ems_ref => "vm1", :name => "vm1")
vms_inventory_collection.build(:ems_ref => "vm2", :name => "vm2")

# Save InventoryCollection to the db
InventoryRefresh::SaveInventory.save_inventory(@ems, [vms_inventory_collection])

# The result in the DB is that vm1 and vm2 were created
puts @ems.vms.collect(&:ems_ref) # => ["vm1", "vm2"]

In another refresh, vm1 does not exist anymore and vm3 was added

# Init InventoryCollection
vms_inventory_collection = ::InventoryRefresh::InventoryCollection.new(
  :model_class => ManageIQ::Providers::CloudManager::Vm, :parent => @ems, :association => :vms
)

# Fill InventoryCollection with data
vms_inventory_collection.build(:ems_ref => "vm2", :name => "vm2")
vms_inventory_collection.build(:ems_ref => "vm3", :name => "vm3")

# Save InventoryCollection to the db
InventoryRefresh::SaveInventory.save_inventory(@ems, [vms_inventory_collection])

# The result in the DB is that vm1 was deleted, vm2 was updated and vm3 was created
puts @ems.vms.collect(&:ems_ref) # => ["vm2", "vm3"]

Defined Under Namespace

Modules: Index Classes: DataStorage, Graph, Reference, ReferencesStorage, Scanner, Serialization

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(model_class: nil, manager_ref: nil, association: nil, parent: nil, strategy: nil, custom_save_block: nil, delete_method: nil, dependency_attributes: nil, attributes_blacklist: nil, attributes_whitelist: nil, complete: nil, update_only: nil, check_changed: nil, arel: nil, default_values: {}, create_only: nil, inventory_object_attributes: nil, name: nil, saver_strategy: nil, parent_inventory_collections: nil, manager_uuids: [], all_manager_uuids: nil, targeted_arel: nil, targeted: nil, manager_ref_allowed_nil: nil, secondary_refs: {}, use_ar_object: nil, custom_reconnect_block: nil, batch_extra_attributes: [], assert_graph_integrity: nil) ⇒ InventoryCollection

Returns a new instance of InventoryCollection.

Parameters:

  • model_class (Class) (defaults to: nil) —

    A class of an ApplicationRecord model, that we want to persist into the DB or load from the DB.

  • manager_ref (Array) (defaults to: nil) —

    Array of Symbols, that are keys of the InventoryObject’s data, inserted into this InventoryCollection. Using these keys, we need to be able to uniquely identify each of the InventoryObject objects inside.

  • association (Symbol) (defaults to: nil) —

    A Rails association callable on a :parent attribute is used for comparing with the objects in the DB, to decide if the InventoryObjects will be created/deleted/updated or used for obtaining the data from a DB, if a DB strategy is used. It returns objects of the :model_class class or its sub STI.

  • parent (ApplicationRecord) (defaults to: nil) —

    An ApplicationRecord object that has a callable :association method returning the objects of a :model_class.

  • strategy (Symbol) (defaults to: nil) —

    A strategy of the InventoryCollection that will be used for saving/loading of the InventoryObject objects. Allowed strategies are:

    - nil => InventoryObject objects of the InventoryCollection will be saved to the DB, only these objects
         will be referable from the other InventoryCollection objects.
- :local_db_cache_all => Loads InventoryObject objects from the database, it loads all the objects that
                         are a result of a [<:parent>.<:association>, :arel] taking
                         first defined in this order. This strategy will not save any objects in the DB.
- :local_db_find_references => Loads InventoryObject objects from the database, it loads only objects that
                               were referenced by the other InventoryCollections using a filtered result
                               of a [<:parent>.<:association>, :arel] taking first
                               defined in this order. This strategy will not save any objects in the DB.
- :local_db_find_missing_references => InventoryObject objects of the InventoryCollection will be saved to
                                       the DB. Then if we reference an object that is not present, it will
                                       load them from the db using :local_db_find_references strategy.
  • custom_save_block (Proc) (defaults to: nil) —

    A custom lambda/proc for persisting in the DB, for cases where it’s not enough to just save every InventoryObject inside by the defined rules and default saving algorithm.

    Example1 - saving SomeModel in my own ineffective way :-) :

    custom_save = lambda do |_ems, inventory_collection|
  inventory_collection.each |inventory_object| do
    hash = inventory_object.attributes # Loads possible dependencies into saveable hash
    obj = SomeModel.find_by(:attr => hash[:attr]) # Note: doing find_by for many models produces N+1
                                                  # queries, avoid this, this is just a simple example :-)
    obj.update_attributes(hash) if obj
    obj ||= SomeModel.create(hash)
    inventory_object.id = obj.id # If this InventoryObject is referenced elsewhere, we need to store its
                                   primary key back to the InventoryObject
 end

    Example2 - saving parent OrchestrationStack in a more effective way, than the default saving algorithm can achieve. Ancestry gem requires an ActiveRecord object for association and is not defined as a proper ActiveRecord association. That leads in N+1 queries in the default saving algorithm, so we can do better with custom saving for now. The InventoryCollection is defined as a custom dependencies processor, without its own :model_class and InventoryObjects inside:

    InventoryRefresh::InventoryCollection.new({
  :association       => :orchestration_stack_ancestry,
  :custom_save_block => orchestration_stack_ancestry_save_block,
  :dependency_attributes => {
    :orchestration_stacks           => [collections[:orchestration_stacks]],
    :orchestration_stacks_resources => [collections[:orchestration_stacks_resources]]
  }
})

    And the lambda is defined as:

    orchestration_stack_ancestry_save_block = lambda do |_ems, inventory_collection|
  stacks_inventory_collection = inventory_collection.dependency_attributes[:orchestration_stacks].try(:first)

  return if stacks_inventory_collection.blank?

  stacks_parents = stacks_inventory_collection.data.each_with_object({}) do |x, obj|
    parent_id = x.data[:parent].load.try(:id)
    obj[x.id] = parent_id if parent_id
  end

  model_class = stacks_inventory_collection.model_class

  stacks_parents_indexed = model_class
                             .select([:id, :ancestry])
                             .where(:id => stacks_parents.values).find_each.index_by(&:id)

  model_class
    .select([:id, :ancestry])
    .where(:id => stacks_parents.keys).find_each do |stack|
    parent = stacks_parents_indexed[stacks_parents[stack.id]]
    stack.update_attribute(:parent, parent)
  end
end
  • custom_reconnect_block (Proc) (defaults to: nil) —

    A custom lambda for reconnect logic of previously disconnected records

    Example - Reconnect disconnected Vms

    InventoryRefresh::InventoryCollection.new({
  :association            => :orchestration_stack_ancestry,
  :custom_reconnect_block => vms_custom_reconnect_block,
})

    And the lambda is defined as:

    vms_custom_reconnect_block = lambda do |inventory_collection, inventory_objects_index, attributes_index|
  inventory_objects_index.each_slice(1000) do |batch|
    Vm.where(:ems_ref => batch.map(&:second).map(&:manager_uuid)).each do |record|
      index = inventory_collection.object_index_with_keys(inventory_collection.manager_ref_to_cols, record)

      # We need to delete the record from the inventory_objects_index and attributes_index, otherwise it
      # would be sent for create.
      inventory_object = inventory_objects_index.delete(index)
      hash             = attributes_index.delete(index)

      record.assign_attributes(hash.except(:id, :type))
      if !inventory_collection.check_changed? || record.changed?
        record.save!
        inventory_collection.store_updated_records(record)
      end

      inventory_object.id = record.id
    end
  end
  • delete_method (Symbol) (defaults to: nil) —

    A delete method that will be used for deleting of the InventoryObject, if the object is marked for deletion. A default is :destroy, the instance method must be defined on the :model_class.

  • dependency_attributes (Hash) (defaults to: nil) —

    Manually defined dependencies of this InventoryCollection. We can use this by manually place the InventoryCollection into the graph, to make sure the saving is invoked after the dependencies were saved. The dependencies itself are InventoryCollection objects. For a common use-cases we do not need to define dependencies manually, since those are inferred automatically by scanning of the data.

    Example:

    :dependency_attributes => {
  :orchestration_stacks           => [collections[:orchestration_stacks]],
  :orchestration_stacks_resources => [collections[:orchestration_stacks_resources]]
}

    This example is used in Example2 of the <param custom_save_block> and it means that our :custom_save_block will be invoked after the InventoryCollection :orchestration_stacks and :orchestration_stacks_resources are saved.

  • attributes_blacklist (Array) (defaults to: nil) —

    Attributes we do not want to include into saving. We cannot blacklist an attribute that is needed for saving of the object. Note: attributes_blacklist is also used for internal resolving of the cycles in the graph.

    In the Example2 of the <param custom_save_block>, we have a custom saving code, that saves a :parent attribute of the OrchestrationStack. That means we don’t want that attribute saved as a part of InventoryCollection for OrchestrationStack, so we would set :attributes_blacklist => [:parent]. Then the :parent will be ignored while saving.

  • attributes_whitelist (Array) (defaults to: nil) —

    Same usage as the :attributes_blacklist, but defining full set of attributes that should be saved. Attributes that are part of :manager_ref and needed validations are automatically added.

  • complete (Boolean) (defaults to: nil) —

    By default true, :complete is marking we are sending a complete dataset and therefore we can create/update/delete the InventoryObject objects. If :complete is false we will only do create/update without delete.

  • update_only (Boolean) (defaults to: nil) —

    By default false. If true we only update the InventoryObject objects, if false we do create/update/delete.

  • check_changed (Boolean) (defaults to: nil) —

    By default true. If true, before updating the InventoryObject, we call Rails ‘changed?’ method. This can optimize speed of updates heavily, but it can fail to recognize the change for e.g. Ancestry and Relationship based columns. If false, we always update the InventoryObject.

  • arel (ActiveRecord::Associations::CollectionProxy|Arel::SelectManager) (defaults to: nil) —

    Instead of :parent and :association we can provide Arel directly to say what records should be compared to check if InventoryObject will be doing create/update/delete.

    Example: for a targeted refresh, we want to delete/update/create only a list of vms specified with a list of ems_refs:

    :arel => manager.vms.where(:ems_ref => manager_refs)

    Then we want to do the same for the hardwares of only those vms:

    :arel => manager.hardwares.joins(:vm_or_template).where(
  'vms' => {:ems_ref => manager_refs}
)

    And etc. for the other Vm related records.

  • default_values (Hash) (defaults to: {}) —

    A hash of an attributes that will be added to every inventory object created by inventory_collection.build(hash)

    Example: Given

    inventory_collection = InventoryCollection.new({
  :model_class    => ::Vm,
  :arel           => @ems.vms,
  :default_values => {:ems_id => 10}
})

    And building the inventory_object like:

    inventory_object = inventory_collection.build(:ems_ref => "vm_1", :name => "vm1")

    The inventory_object.data will look like:

    {:ems_ref => "vm_1", :name => "vm1", :ems_id => 10}
  • inventory_object_attributes (Array) (defaults to: nil) —

    Array of attribute names that will be exposed as readers/writers on the InventoryObject objects inside.

    Example: Given

    inventory_collection = InventoryCollection.new({
   :model_class                 => ::Vm,
   :arel                        => @ems.vms,
   :inventory_object_attributes => [:name, :label]
 })

    And building the inventory_object like:

    inventory_object = inventory_collection.build(:ems_ref => "vm1", :name => "vm1")

    We can use inventory_object_attributes as setters and getters:

    inventory_object.name = "Name"
inventory_object.label = inventory_object.name

    Which would be equivalent to less nicer way:

    inventory_object[:name] = "Name"
inventory_object[:label] = inventory_object[:name]

    So by using inventory_object_attributes, we will be guarding the allowed attributes and will have an explicit list of allowed attributes, that can be used also for documentation purposes.

  • name (Symbol) (defaults to: nil) —

    A unique name of the InventoryCollection under a Persister. If not provided, the :association attribute is used. If :association is nil as well, the :name will be inferred from the :model_class.

  • saver_strategy (Symbol) (defaults to: nil) —

    A strategy that will be used for InventoryCollection persisting into the DB. Allowed saver strategies are:

    • :default => Using Rails saving methods, this way is not safe to run in multiple workers concurrently, since it will lead to non consistent data.

    • :batch => Using batch SQL queries, this way is not safe to run in multiple workers concurrently, since it will lead to non consistent data.

    • :concurrent_safe => This method is designed for concurrent saving. It uses atomic upsert to avoid data duplication and it uses timestamp based atomic checks to avoid new data being overwritten by the the old data.

    • :concurrent_safe_batch => Same as :concurrent_safe, but the upsert/update queries are executed as batched SQL queries, instead of sending 1 query per record.

  • parent_inventory_collections (Array) (defaults to: nil) —

    Array of symbols having a name pointing to the InventoryRefresh::InventoryCollection objects, that serve as parents to this InventoryCollection. There are several scenarios to consider, when deciding if InventoryCollection has parent collections, see the example.

    Example:

    taking inventory collections :vms and :disks (local disks), if we write that:
inventory_collection = InventoryCollection.new({
             :model_class                 => ::Disk,
             :association                 => :disks,
             :manager_ref                 => [:vm, :location]
             :parent_inventory_collection => [:vms],
           })

Then the decision for having :parent_inventory_collection => [:vms] was probably driven by these
points:
1. We can get list of all disks only by doing SQL query through the parent object (so there will be join
   from vms to disks table).
2. There is no API query for getting all disks from the provider API, we get them inside VM data, or as
   a Vm subquery
3. Part of the manager_ref of the IC is the VM object (foreign key), so the disk's location is unique
   only under 1 Vm. (In current models, this modeled going through Hardware model)
4. In targeted refresh, we always expect that each Vm will be saved with all its disks.

Then having the above points, adding :parent_inventory_collection => [:vms], will bring these
implications:
1. By archiving/deleting Vm, we can no longer see the disk, because those were owned by the Vm. Any
   archival/deletion of the Disk model, must be then done by cascade delete/hooks logic.
2. Having Vm as a parent ensures we always process it first. So e.g. when providing no Vms for saving
   we would have no graph dependency (no data --> no edges --> no dependencies) and Disk could be
   archived/removed before the Vm, while we always want to archive the VM first.
3. For targeted refresh, we always expect that all disks are saved with a VM. So for targeting :disks,
   we are not using #manager_uuids attribute, since the scope is "all disks of all targeted VMs", so we
   always use #manager_uuids of the parent. (that is why :parent_inventory_collections and
   :manager_uuids are mutually exclusive attributes)
4. For automatically building the #targeted_arel query, we need the parent to know what is the root node.
   While this information can be introspected from the data, it creates a scope for create&update&delete,
   which means it has to work with no data provided (causing delete all). So with no data we cannot
   introspect anything.
  • manager_uuids (Array|Proc) (defaults to: []) —

    Array of manager_uuids of the InventoryObjects we want to create/update/delete. Using this attribute, the db_collection_for_comparison will be automatically limited by the manager_uuids, in a case of a simple relation. In a case of a complex relation, we can leverage :manager_uuids in a custom :targeted_arel. We can pass also lambda, for lazy_evaluation.

  • all_manager_uuids (Array) (defaults to: nil) —

    Array of all manager_uuids of the InventoryObjects. With the :targeted true, having this parameter defined will invoke only :delete_method on a complement of this set, making sure the DB has only this set of data after. This :attribute serves for deleting of top level InventoryCollections, i.e. InventoryCollections having parent_inventory_collections nil. The deleting of child collections is already handled by the scope of the parent_inventory_collections and using Rails :dependent => :destroy,

  • targeted_arel (Proc) (defaults to: nil) —

    A callable block that receives this InventoryCollection as a first argument. In there we can leverage a :parent_inventory_collections or :manager_uuids to limit the query based on the manager_uuids available. Example:

    targeted_arel = lambda do |inventory_collection|
  # Getting ems_refs of parent :vms and :miq_templates
  manager_uuids = inventory_collection.parent_inventory_collections.collect(&:manager_uuids).flatten
  inventory_collection.db_collection_for_comparison.hardwares.joins(:vm_or_template).where(
    'vms' => {:ems_ref => manager_uuids}
  )
end

inventory_collection = InventoryCollection.new({
                         :model_class                 => ::Hardware,
                         :association                 => :hardwares,
                         :parent_inventory_collection => [:vms, :miq_templates],
                         :targeted_arel               => targeted_arel,
                       })
  • targeted (Boolean) (defaults to: nil) —

    True if the collection is targeted, in that case it will be leveraging :manager_uuids :parent_inventory_collections and :targeted_arel to save a subgraph of a data.

  • manager_ref_allowed_nil (Array) (defaults to: nil) —

    Array of symbols having manager_ref columns, that are a foreign key an can be nil. Given the table are shared by many providers, it can happen, that the table is used only partially. Then it can happen we want to allow certain foreign keys to be nil, while being sure the referential integrity is not broken. Of course the DB Foreign Key can’t be created in this case, so we should try to avoid this usecase by a proper modeling.

  • use_ar_object (Boolean) (defaults to: nil) —

    True or False. Whether we need to initialize AR object as part of the saving it’s needed if the model have special setters, serialize of columns, etc. This setting is relevant only for the batch saver strategy.

  • batch_extra_attributes (Array) (defaults to: []) —

    Array of symbols marking which extra attributes we want to store into the db. These extra attributes might be a product of :use_ar_object assignment and we need to specify them manually, if we want to use a batch saving strategy and we have models that populate attributes as a side effect.

  • assert_graph_integrity (Boolean) (defaults to: nil) —

    Default false. If true then extra checks for graph integrity will be performed and an exception will be raised if an error is found.



415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 415

def initialize(model_class: nil, manager_ref: nil, association: nil, parent: nil, strategy: nil,
               custom_save_block: nil, delete_method: nil, dependency_attributes: nil,
               attributes_blacklist: nil, attributes_whitelist: nil, complete: nil, update_only: nil,
               check_changed: nil, arel: nil, default_values: {}, create_only: nil,
               inventory_object_attributes: nil, name: nil, saver_strategy: nil,
               parent_inventory_collections: nil, manager_uuids: [], all_manager_uuids: nil, targeted_arel: nil,
               targeted: nil, manager_ref_allowed_nil: nil, secondary_refs: {}, use_ar_object: nil,
               custom_reconnect_block: nil, batch_extra_attributes: [], assert_graph_integrity: nil)
  @model_class            = model_class
  @manager_ref            = manager_ref || [:ems_ref]
  @secondary_refs         = secondary_refs
  @association            = association
  @parent                 = parent || nil
  @arel                   = arel
  @dependency_attributes  = dependency_attributes || {}
  @strategy               = process_strategy(strategy)
  @delete_method          = delete_method || :destroy
  @custom_save_block      = custom_save_block
  @custom_reconnect_block = custom_reconnect_block
  @check_changed          = check_changed.nil? ? true : check_changed
  @internal_attributes    = i( __parent_inventory_collections)
  @complete               = complete.nil? ? true : complete
  @update_only            = update_only.nil? ? false : update_only
  @create_only            = create_only.nil? ? false : create_only
  @default_values         = default_values
  @name                   = name || association || model_class.to_s.demodulize.tableize
  @saver_strategy         = process_saver_strategy(saver_strategy)
  @use_ar_object          = use_ar_object || false
  @batch_extra_attributes = batch_extra_attributes
  @assert_graph_integrity = assert_graph_integrity.nil? ? false : assert_graph_integrity

  @manager_ref_allowed_nil = manager_ref_allowed_nil || []

  # Targeted mode related attributes
  # TODO(lsmola) Should we refactor this to use references too?
  @all_manager_uuids            = all_manager_uuids
  @parent_inventory_collections = parent_inventory_collections
  @targeted_arel                = targeted_arel
  @targeted                     = !!targeted

  @inventory_object_attributes = inventory_object_attributes

  @saved                          ||= false
  @attributes_blacklist             = Set.new
  @attributes_whitelist             = Set.new
  @transitive_dependency_attributes = Set.new
  @dependees                        = Set.new
  @data_storage = ::InventoryRefresh::InventoryCollection::DataStorage.new(self, secondary_refs)
  @references_storage = ::InventoryRefresh::InventoryCollection::ReferencesStorage.new(index_proxy)
  @targeted_scope = ::InventoryRefresh::InventoryCollection::ReferencesStorage.new(index_proxy).merge!(manager_uuids)

  @created_records = []
  @updated_records = []
  @deleted_records = []

  blacklist_attributes!(attributes_blacklist) if attributes_blacklist.present?
  whitelist_attributes!(attributes_whitelist) if attributes_whitelist.present?
end

Instance Attribute Details

#all_manager_uuids ⇒ Array?

If present, InventoryCollection switches into delete_complement mode, where it will delete every record from the DB, that is not present in this list. This is used for the batch processing, where we don’t know which InventoryObject should be deleted, but we know all manager_uuids of all InventoryObject objects that exists in the provider.

Returns:

  • (Array, nil) —

    nil or a list of all :manager_uuids that are present in the Provider’s InventoryCollection.



75
76
77
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 75

def all_manager_uuids
  @all_manager_uuids
end

#arel ⇒ Object (readonly)

Returns the value of attribute arel.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def arel
  @arel
end

#assert_graph_integrity ⇒ Object (readonly)

Returns the value of attribute assert_graph_integrity.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def assert_graph_integrity
  @assert_graph_integrity
end

#association ⇒ Object (readonly)

Returns the value of attribute association.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def association
  @association
end

#attributes_blacklist ⇒ Object

Returns the value of attribute attributes_blacklist.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def attributes_blacklist
  @attributes_blacklist
end

#attributes_whitelist ⇒ Object

Returns the value of attribute attributes_whitelist.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def attributes_whitelist
  @attributes_whitelist
end

#batch_extra_attributes ⇒ Object (readonly)

Returns the value of attribute batch_extra_attributes.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def batch_extra_attributes
  @batch_extra_attributes
end

#check_changed ⇒ Object (readonly)

Returns the value of attribute check_changed.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def check_changed
  @check_changed
end

#complete ⇒ Object (readonly)

Returns the value of attribute complete.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def complete
  @complete
end

#create_only ⇒ Object (readonly)

Returns the value of attribute create_only.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def create_only
  @create_only
end

#created_records ⇒ Object (readonly)

Returns the value of attribute created_records.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def created_records
  @created_records
end

#custom_reconnect_block ⇒ Object (readonly)

Returns the value of attribute custom_reconnect_block.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def custom_reconnect_block
  @custom_reconnect_block
end

#custom_save_block ⇒ Object (readonly)

Returns the value of attribute custom_save_block.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def custom_save_block
  @custom_save_block
end

#data_collection_finalized ⇒ Boolean

Returns A true value marks that we collected all the data of the InventoryCollection, meaning we also collected all the references.

Returns:

  • (Boolean) —

    A true value marks that we collected all the data of the InventoryCollection, meaning we also collected all the references.



58
59
60
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 58

def data_collection_finalized
  @data_collection_finalized
end

#data_storage ⇒ InventoryRefresh::InventoryCollection::DataStorage

Returns An InventoryCollection encapsulating all data with indexes.

Returns:



62
63
64
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 62

def data_storage
  @data_storage
end

#default_values ⇒ Object (readonly)

Returns the value of attribute default_values.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def default_values
  @default_values
end

#delete_method ⇒ Object (readonly)

Returns the value of attribute delete_method.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def delete_method
  @delete_method
end

#deleted_records ⇒ Object (readonly)

Returns the value of attribute deleted_records.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def deleted_records
  @deleted_records
end

#dependees ⇒ Set

Returns A set of InventoryCollection objects that depends on this InventoryCollection object.

Returns:

  • (Set) —

    A set of InventoryCollection objects that depends on this InventoryCollection object.



78
79
80
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 78

def dependees
  @dependees
end

#dependency_attributes ⇒ Object (readonly)

Returns the value of attribute dependency_attributes.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def dependency_attributes
  @dependency_attributes
end

#internal_attributes ⇒ Object (readonly)

Returns the value of attribute internal_attributes.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def internal_attributes
  @internal_attributes
end

#inventory_object_attributes ⇒ Object (readonly)

Returns the value of attribute inventory_object_attributes.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def inventory_object_attributes
  @inventory_object_attributes
end

#manager_ref ⇒ Object (readonly)

Returns the value of attribute manager_ref.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def manager_ref
  @manager_ref
end

#manager_ref_allowed_nil ⇒ Object (readonly)

Returns the value of attribute manager_ref_allowed_nil.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def manager_ref_allowed_nil
  @manager_ref_allowed_nil
end

#model_class ⇒ Object (readonly)

Returns the value of attribute model_class.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def model_class
  @model_class
end

#name ⇒ Object (readonly)

Returns the value of attribute name.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def name
  @name
end

#parent ⇒ Object (readonly)

Returns the value of attribute parent.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def parent
  @parent
end

#parent_inventory_collections ⇒ Array<Symbol>

Returns @see #parent_inventory_collections documentation of InventoryCollection.new kwargs parameters.

Returns:

  • (Array<Symbol>) —

    @see #parent_inventory_collections documentation of InventoryCollection.new kwargs parameters



82
83
84
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 82

def parent_inventory_collections
  @parent_inventory_collections
end

#references_storage ⇒ Object (readonly)

Returns the value of attribute references_storage.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def references_storage
  @references_storage
end

#saved ⇒ Boolean

Returns true if this collection is already saved into the DB. E.g. InventoryCollections with DB only strategy are marked as saved. This causes InventoryCollection not being a dependency for any other InventoryCollection, since it is already persisted into the DB.

Returns:

  • (Boolean) —

    true if this collection is already saved into the DB. E.g. InventoryCollections with DB only strategy are marked as saved. This causes InventoryCollection not being a dependency for any other InventoryCollection, since it is already persisted into the DB.



67
68
69
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 67

def saved
  @saved
end

#saver_strategy ⇒ Object (readonly)

Returns the value of attribute saver_strategy.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def saver_strategy
  @saver_strategy
end

#strategy ⇒ Object (readonly)

Returns the value of attribute strategy.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def strategy
  @strategy
end

#targeted ⇒ Object (readonly)

Returns the value of attribute targeted.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def targeted
  @targeted
end

#targeted_arel ⇒ Object (readonly)

Returns the value of attribute targeted_arel.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def targeted_arel
  @targeted_arel
end

#targeted_scope ⇒ Object (readonly)

Returns the value of attribute targeted_scope.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def targeted_scope
  @targeted_scope
end

#transitive_dependency_attributes ⇒ Object (readonly)

Returns the value of attribute transitive_dependency_attributes.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def transitive_dependency_attributes
  @transitive_dependency_attributes
end

#update_only ⇒ Object (readonly)

Returns the value of attribute update_only.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def update_only
  @update_only
end

#updated_records ⇒ Object (readonly)

Returns the value of attribute updated_records.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def updated_records
  @updated_records
end

#use_ar_object ⇒ Object (readonly)

Returns the value of attribute use_ar_object.



84
85
86
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 84

def use_ar_object
  @use_ar_object
end

Instance Method Details

#association_to_base_class_mapping ⇒ Hash{Symbol => String}

Returns Hash with association name mapped to base class of the association.

Returns:

  • (Hash{Symbol => String}) —

    Hash with association name mapped to base class of the association



946
947
948
949
950
951
952
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 946

def association_to_base_class_mapping
  return {} unless model_class

  @association_to_base_class_mapping ||= model_class.reflect_on_all_associations.each_with_object({}) do |x, obj|
    obj[x.name] = x.klass.base_class.name unless x.polymorphic?
  end
end

#association_to_foreign_key_mapping ⇒ Hash{Symbol => String}

Returns Hash with association name mapped to foreign key column name.

Returns:

  • (Hash{Symbol => String}) —

    Hash with association name mapped to foreign key column name



910
911
912
913
914
915
916
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 910

def association_to_foreign_key_mapping
  return {} unless model_class

  @association_to_foreign_key_mapping ||= belongs_to_associations.each_with_object({}) do |x, obj|
    obj[x.name] = x.foreign_key
  end
end

#association_to_foreign_type_mapping ⇒ Hash{Symbol => String}

Returns Hash with association name mapped to polymorphic foreign key type column name.

Returns:

  • (Hash{Symbol => String}) —

    Hash with association name mapped to polymorphic foreign key type column name



928
929
930
931
932
933
934
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 928

def association_to_foreign_type_mapping
  return {} unless model_class

  @association_to_foreign_type_mapping ||= model_class.reflect_on_all_associations.each_with_object({}) do |x, obj|
    obj[x.name] = x.foreign_type if x.polymorphic?
  end
end

#base_class_name ⇒ String

Returns Base class name of the model_class of this InventoryCollection.

Returns:

  • (String) —

    Base class name of the model_class of this InventoryCollection



976
977
978
979
980
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 976

def base_class_name
  return "" unless model_class

  @base_class_name ||= model_class.base_class.name
end

#base_columns ⇒ Object 



727
728
729
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 727

def base_columns
  @base_columns ||= unique_index_columns + internal_columns
end

#batch_size ⇒ Integer

Returns default batch size for talking to the DB.

Returns:

  • (Integer) —

    default batch size for talking to the DB



1000
1001
1002
1003
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1000

def batch_size
  # TODO(lsmola) mode to the settings
  1000
end

#batch_size_pure_sql ⇒ Integer

Returns default batch size for talking to the DB if not using ApplicationRecord objects.

Returns:

  • (Integer) —

    default batch size for talking to the DB if not using ApplicationRecord objects



1006
1007
1008
1009
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1006

def batch_size_pure_sql
  # TODO(lsmola) mode to the settings
  10_000
end

#belongs_to_associations ⇒ Array<ActiveRecord::Reflection::BelongsToReflection">

Returns All belongs_to associations.

Returns:

  • (Array<ActiveRecord::Reflection::BelongsToReflection">) —

    All belongs_to associations



905
906
907
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 905

def belongs_to_associations
  model_class.reflect_on_all_associations.select { |x| x.kind_of?(ActiveRecord::Reflection::BelongsToReflection) }
end

#blacklist_attributes!(attributes) ⇒ Array<Symbol>

Add passed attributes to blacklist. The manager_ref attributes cannot be blacklisted, otherwise we will not be able to identify the inventory_object. We do not automatically remove attributes causing fixed dependencies, so beware that without them, you won’t be able to create the record.

Parameters:

  • attributes (Array<Symbol>) —

    Attributes we want to blacklist

Returns:

  • (Array<Symbol>) —

    All blacklisted attributes



869
870
871
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 869

def blacklist_attributes!(attributes)
  self.attributes_blacklist += attributes - (fixed_attributes + internal_attributes)
end

#build_multi_selection_condition(hashes, keys = manager_ref) ⇒ String

Builds a multiselection conditions like (table1.a = a1 AND table2.b = b1) OR (table1.a = a2 AND table2.b = b2)

Parameters:

  • hashes (Array<Hash>) —

    data we want to use for the query

  • keys (Array<Symbol>) (defaults to: manager_ref) —

    keys of attributes involved

Returns:

  • (String) —

    A condition usable in .where of an ActiveRecord relation



1026
1027
1028
1029
1030
1031
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1026

def build_multi_selection_condition(hashes, keys = manager_ref)
  arel_table = model_class.arel_table
  # We do pure SQL OR, since Arel is nesting every .or into another parentheses, otherwise this would be just
  # inject(:or) instead of to_sql with .join(" OR ")
  hashes.map { |hash| "(#{keys.map { |key| arel_table[key].eq(hash[key]) }.inject(:and).to_sql})" }.join(" OR ")
end

#check_changed? ⇒ Boolean

Returns true means we want to call .changed? on every ActiveRecord object before saving it.

Returns:

  • (Boolean) —

    true means we want to call .changed? on every ActiveRecord object before saving it



537
538
539
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 537

def check_changed?
  check_changed
end

#clone ⇒ InventoryCollection

Returns a shallow copy of InventoryCollection, the copy will share data_storage of the original collection, otherwise we would be copying a lot of records in memory.

Returns:

  • (InventoryCollection) —

    a shallow copy of InventoryCollection, the copy will share data_storage of the original collection, otherwise we would be copying a lot of records in memory.



885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 885

def clone
  cloned = self.class.new(:model_class           => model_class,
                          :manager_ref           => manager_ref,
                          :association           => association,
                          :parent                => parent,
                          :arel                  => arel,
                          :strategy              => strategy,
                          :saver_strategy        => saver_strategy,
                          :custom_save_block     => custom_save_block,
                          # We want cloned IC to be update only, since this is used for cycle resolution
                          :update_only           => true,
                          # Dependency attributes need to be a hard copy, since those will differ for each
                          # InventoryCollection
                          :dependency_attributes => dependency_attributes.clone)

  cloned.data_storage = data_storage
  cloned
end

#complete? ⇒ Boolean

Returns true means the data is not complete, leading to only creating and updating data.

Returns:

  • (Boolean) —

    true means the data is not complete, leading to only creating and updating data



548
549
550
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 548

def complete?
  complete
end

#create_allowed? ⇒ Boolean

Returns true means we will delete/soft-delete data.

Returns:

  • (Boolean) —

    true means we will delete/soft-delete data



563
564
565
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 563

def create_allowed?
  !update_only?
end

#create_only? ⇒ Boolean

Returns true means that only create of new data is allowed.

Returns:

  • (Boolean) —

    true means that only create of new data is allowed



568
569
570
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 568

def create_only?
  create_only
end

#data_collection_finalized? ⇒ Boolean

Returns true if no more data will be added to this InventoryCollection object, that usually happens after the parsing step is finished.

Returns:

  • (Boolean) —

    true if no more data will be added to this InventoryCollection object, that usually happens after the parsing step is finished



733
734
735
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 733

def data_collection_finalized?
  data_collection_finalized
end

#db_collection_for_comparison ⇒ ActiveRecord::Relation

Returns A relation that can fetch all data of this InventoryCollection from the DB.

Returns:

  • (ActiveRecord::Relation) —

    A relation that can fetch all data of this InventoryCollection from the DB



1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1034

def db_collection_for_comparison
  if targeted?
    if targeted_arel.respond_to?(:call)
      targeted_arel.call(self)
    elsif parent_inventory_collections.present?
      targeted_arel_default
    else
      targeted_iterator_for(targeted_scope.primary_references)
    end
  else
    full_collection_for_comparison
  end
end

#db_collection_for_comparison_for(references) ⇒ ActiveRecord::Relation

Builds an ActiveRecord::Relation that can fetch all the references from the DB

Parameters:

Returns:

  • (ActiveRecord::Relation) —

    relation that can fetch all the references from the DB



1101
1102
1103
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1101

def db_collection_for_comparison_for(references)
  full_collection_for_comparison.where(targeted_selection_for(references))
end

#db_collection_for_comparison_for_complement_of(manager_uuids_set) ⇒ ActiveRecord::Relation

Builds an ActiveRecord::Relation that can fetch complement of all the references from the DB

Parameters:

  • manager_uuids_set (Array<String>) —

    passed references

Returns:

  • (ActiveRecord::Relation) —

    relation that can fetch complement of all the references from the DB



1109
1110
1111
1112
1113
1114
1115
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1109

def db_collection_for_comparison_for_complement_of(manager_uuids_set)
  # TODO(lsmola) this should have the build_multi_selection_condition, like in the method above
  # TODO(lsmola) this query will be highly ineffective, we will try approach with updating a timestamp of all
  # records, then we can get list of all records that were not update. That would be equivalent to result of this
  # more effective query and without need of all manager_uuids
  full_collection_for_comparison.where.not(manager_ref.first => manager_uuids_set)
end

#delete_allowed? ⇒ Boolean

Returns true means we will delete/soft-delete data.

Returns:

  • (Boolean) —

    true means we will delete/soft-delete data



558
559
560
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 558

def delete_allowed?
  complete? && !update_only?
end

#dependencies ⇒ Array<InventoryRefresh::InventoryCollection>

Returns all unique non saved dependencies.

Returns:



846
847
848
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 846

def dependencies
  filtered_dependency_attributes.values.map(&:to_a).flatten.uniq.reject(&:saved?)
end

#dependency_attributes_for(inventory_collections) ⇒ Array<InventoryRefresh::InventoryCollection>

Returns what attributes are causing a dependencies to certain InventoryCollection objects.

Parameters:

Returns:



855
856
857
858
859
860
861
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 855

def dependency_attributes_for(inventory_collections)
  attributes = Set.new
  inventory_collections.each do |inventory_collection|
    attributes += filtered_dependency_attributes.select { |_key, value| value.include?(inventory_collection) }.keys
  end
  attributes
end

#filtered_dependency_attributes ⇒ Hash{Symbol => Set}

List attributes causing a dependency and filters them by attributes_blacklist and attributes_whitelist

Returns:

  • (Hash{Symbol => Set}) —

    attributes causing a dependency and filtered by blacklist and whitelist



805
806
807
808
809
810
811
812
813
814
815
816
817
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 805

def filtered_dependency_attributes
  filtered_attributes = dependency_attributes

  if attributes_blacklist.present?
    filtered_attributes = filtered_attributes.reject { |key, _value| attributes_blacklist.include?(key) }
  end

  if attributes_whitelist.present?
    filtered_attributes = filtered_attributes.select { |key, _value| attributes_whitelist.include?(key) }
  end

  filtered_attributes
end

#fixed_attributes ⇒ Array<Symbol>

Attributes that are needed to be able to save the record, i.e. attributes that are part of the unique index and attributes with presence validation or NOT NULL constraint

Returns:

  • (Array<Symbol>) —

    attributes that are needed for saving of the record



823
824
825
826
827
828
829
830
831
832
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 823

def fixed_attributes
  if model_class
    presence_validators = model_class.validators.detect { |x| x.kind_of?(ActiveRecord::Validations::PresenceValidator) }
  end
  # Attributes that has to be always on the entity, so attributes making unique index of the record + attributes
  # that have presence validation
  fixed_attributes = manager_ref
  fixed_attributes += presence_validators.attributes if presence_validators.present?
  fixed_attributes
end

#fixed_dependencies ⇒ Object

Returns fixed dependencies, which are the ones we can’t move, because we wouldn’t be able to save the data



837
838
839
840
841
842
843
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 837

def fixed_dependencies
  fixed_attrs = fixed_attributes

  filtered_dependency_attributes.each_with_object(Set.new) do |(key, value), fixed_deps|
    fixed_deps.merge(value) if fixed_attrs.include?(key)
  end.reject(&:saved?)
end

#fixed_foreign_keys ⇒ Array<Symbol>

Returns List of all column names that are foreign keys and cannot removed, otherwise we couldn’t save the record.

Returns:

  • (Array<Symbol>) —

    List of all column names that are foreign keys and cannot removed, otherwise we couldn’t save the record



963
964
965
966
967
968
969
970
971
972
973
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 963

def fixed_foreign_keys
  # Foreign keys that are part of a manager_ref must be present, otherwise the record would get lost. This is a
  # minimum check we can do to not break a referential integrity.
  return @fixed_foreign_keys_cache unless @fixed_foreign_keys_cache.nil?

  manager_ref_set = (manager_ref - manager_ref_allowed_nil)
  @fixed_foreign_keys_cache = manager_ref_set.map { |x| association_to_foreign_key_mapping[x] }.compact
  @fixed_foreign_keys_cache += foreign_keys & manager_ref
  @fixed_foreign_keys_cache.map!(&:to_sym)
  @fixed_foreign_keys_cache
end

#foreign_key_to_association_mapping ⇒ Hash{String => Hash}

Returns Hash with foreign_key column name mapped to association name.

Returns:

  • (Hash{String => Hash}) —

    Hash with foreign_key column name mapped to association name



919
920
921
922
923
924
925
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 919

def foreign_key_to_association_mapping
  return {} unless model_class

  @foreign_key_to_association_mapping ||= belongs_to_associations.each_with_object({}) do |x, obj|
    obj[x.foreign_key] = x.name
  end
end

#foreign_keys ⇒ Array<Symbol>

Returns List of all column names that are foreign keys.

Returns:

  • (Array<Symbol>) —

    List of all column names that are foreign keys



955
956
957
958
959
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 955

def foreign_keys
  return [] unless model_class

  @foreign_keys_cache ||= belongs_to_associations.map(&:foreign_key).map!(&:to_sym)
end

#foreign_type_to_association_mapping ⇒ Hash{Symbol => String}

Returns Hash with polymorphic foreign key type column name mapped to association name.

Returns:

  • (Hash{Symbol => String}) —

    Hash with polymorphic foreign key type column name mapped to association name



937
938
939
940
941
942
943
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 937

def foreign_type_to_association_mapping
  return {} unless model_class

  @foreign_type_to_association_mapping ||= model_class.reflect_on_all_associations.each_with_object({}) do |x, obj|
    obj[x.foreign_type] = x.name if x.polymorphic?
  end
end

#full_collection_for_comparison ⇒ ActiveRecord::Relation

Returns relation that can fetch all the references from the DB.

Returns:

  • (ActiveRecord::Relation) —

    relation that can fetch all the references from the DB



1118
1119
1120
1121
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1118

def full_collection_for_comparison
  return arel unless arel.nil?
  parent.send(association)
end

#inspect ⇒ String

Returns a concise form of the InventoryCollection for easy logging.

Returns:

  • (String) —

    a concise form of the InventoryCollection for easy logging



995
996
997
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 995

def inspect
  to_s
end

#internal_columns ⇒ Object 



701
702
703
704
705
706
707
708
709
710
711
712
713
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 701

def internal_columns
  return @internal_columns if @internal_columns

  @internal_columns = [] + internal_timestamp_columns
  @internal_columns << :type if supports_sti?
  @internal_columns << :resource_timestamps_max if supports_resource_timestamps_max?
  @internal_columns << :resource_timestamps if supports_resource_timestamps?
  @internal_columns << :resource_timestamp if supports_resource_timestamp?
  @internal_columns << :resource_versions_max if supports_resource_versions_max?
  @internal_columns << :resource_versions if supports_resource_versions?
  @internal_columns << :resource_version if supports_resource_version?
  @internal_columns
end

#internal_timestamp_columns ⇒ Object 



715
716
717
718
719
720
721
722
723
724
725
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 715

def internal_timestamp_columns
  return @internal_timestamp_columns if @internal_timestamp_columns

  @internal_timestamp_columns = []
  @internal_timestamp_columns << :created_on if supports_created_on?
  @internal_timestamp_columns << :created_at if supports_created_at?
  @internal_timestamp_columns << :updated_on if supports_updated_on?
  @internal_timestamp_columns << :updated_at if supports_updated_at?

  @internal_timestamp_columns
end

#inventory_object?(value) ⇒ Boolean

Returns true is value is kind of InventoryRefresh::InventoryObject.

Parameters:

  • value (Object) —

    Object we want to test

Returns:

  • (Boolean) —

    true is value is kind of InventoryRefresh::InventoryObject



739
740
741
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 739

def inventory_object?(value)
  value.kind_of?(::InventoryRefresh::InventoryObject)
end

#inventory_object_lazy?(value) ⇒ Boolean

Returns true is value is kind of InventoryRefresh::InventoryObjectLazy.

Parameters:

  • value (Object) —

    Object we want to test

Returns:

  • (Boolean) —

    true is value is kind of InventoryRefresh::InventoryObjectLazy



745
746
747
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 745

def inventory_object_lazy?(value)
  value.kind_of?(::InventoryRefresh::InventoryObjectLazy)
end

#manager_ref_to_cols ⇒ Array<String>

Convert manager_ref list of attributes to list of DB columns

Returns:

  • (Array<String>) —

    true is processing of this InventoryCollection will be in targeted mode



794
795
796
797
798
799
800
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 794

def manager_ref_to_cols
  # TODO(lsmola) this should contain the polymorphic _type, otherwise the IC with polymorphic unique key will get
  # conflicts
  manager_ref.map do |ref|
    association_to_foreign_key_mapping[ref] || ref
  end
end

#manager_uuids ⇒ Array<String>

Returns a list of stringified uuids of all scoped InventoryObjects, which is used for scoping in targeted mode

Returns:

  • (Array<String>) —

    list of stringified uuids of all scoped InventoryObjects



1014
1015
1016
1017
1018
1019
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1014

def manager_uuids
  # TODO(lsmola) LEGACY: this is still being used by :targetel_arel definitions and it expects array of strings
  raise "This works only for :manager_ref size 1" if manager_ref.size > 1
  key = manager_ref.first
  transform_references_to_hashes(targeted_scope.primary_references).map { |x| x[key] }
end

#new_inventory_object(hash) ⇒ InventoryRefresh::InventoryObject

Creates InventoryRefresh::InventoryObject object from passed hash data

Parameters:

  • hash (Hash) —

    Object data

Returns:



1127
1128
1129
1130
1131
1132
1133
1134
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1127

def new_inventory_object(hash)
  manager_ref.each do |x|
    # TODO(lsmola) with some effort, we can do this, but it's complex
    raise "A lazy_find with a :key can't be a part of the manager_uuid" if inventory_object_lazy?(hash[x]) && hash[x].key
  end

  inventory_object_class.new(self, hash)
end

#noop? ⇒ Boolean

True if processing of this InventoryCollection object would lead to no operations. Then we use this marker to stop processing of the InventoryCollector object very soon, to avoid a lot of unnecessary Db queries, etc.

Returns:

  • (Boolean) —

    true if processing of this InventoryCollection object would lead to no operations.



763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 763

def noop?
  # If this InventoryCollection doesn't do anything. it can easily happen for targeted/batched strategies.
  if targeted?
    if parent_inventory_collections.nil? && targeted_scope.primary_references.blank? &&
       all_manager_uuids.nil? && parent_inventory_collections.blank? && custom_save_block.nil? &&
       skeletal_primary_index.blank?
      # It's a noop Parent targeted InventoryCollection
      true
    elsif !parent_inventory_collections.nil? && parent_inventory_collections.all? { |x| x.targeted_scope.primary_references.blank? } &&
          skeletal_primary_index.blank?
      # It's a noop Child targeted InventoryCollection
      true
    else
      false
    end
  elsif data.blank? && !delete_allowed? && skeletal_primary_index.blank?
    # If we have no data to save and delete is not allowed, we can just skip
    true
  else
    false
  end
end

#object_index_with_keys(keys, record) ⇒ String

Builds string uuid from passed Object and keys

Parameters:

  • keys (Array<Symbol>) —

    Indexes into the Hash data

  • record (ApplicationRecord) —

    ActiveRecord record

Returns:

  • (String) —

    Concatenated values on keys from data



754
755
756
757
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 754

def object_index_with_keys(keys, record)
  # TODO(lsmola) remove, last usage is in k8s reconnect logic
  build_stringified_reference_for_record(record, keys)
end

#parallel_safe? ⇒ Boolean

Returns true if we are using a saver strategy that allows saving in parallel processes.

Returns:

  • (Boolean) —

    true if we are using a saver strategy that allows saving in parallel processes



583
584
585
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 583

def parallel_safe?
  @parallel_safe_cache ||= i(concurrent_safe concurrent_safe_batch).include?(saver_strategy)
end

#process_saver_strategy(saver_strategy) ⇒ Symbol

Processes passed saver strategy

Parameters:

  • saver_strategy (Symbol) —

    Passed saver strategy

Returns:

  • (Symbol) —

    Returns back the passed strategy if supported, or raises exception



499
500
501
502
503
504
505
506
507
508
509
510
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 499

def process_saver_strategy(saver_strategy)
  return :default unless saver_strategy

  saver_strategy = saver_strategy.to_sym
  case saver_strategy
  when :default, :batch, :concurrent_safe, :concurrent_safe_batch
    saver_strategy
  else
    raise "Unknown InventoryCollection saver strategy: :#{saver_strategy}, allowed strategies are "\
          ":default, :batch, :concurrent_safe and :concurrent_safe_batch"
  end
end

#process_strategy(strategy_name) ⇒ Symbol

Processes passed strategy, modifies :data_collection_finalized and :saved attributes for db only strategies

Parameters:

  • strategy_name (Symbol) —

    Passed saver strategy

Returns:

  • (Symbol) —

    Returns back the passed strategy if supported, or raises exception



516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 516

def process_strategy(strategy_name)
  self.data_collection_finalized = false

  return unless strategy_name

  strategy_name = strategy_name.to_sym
  case strategy_name
  when :local_db_cache_all
    self.data_collection_finalized = true
    self.saved = true
  when :local_db_find_references
    self.saved = true
  when :local_db_find_missing_references
  else
    raise "Unknown InventoryCollection strategy: :#{strategy_name}, allowed strategies are :local_db_cache_all, "\
          ":local_db_find_references and :local_db_find_missing_references."
  end
  strategy_name
end

#saveable? ⇒ Boolean

Returns true if all dependencies have all data persisted.

Returns:

  • (Boolean) —

    true if all dependencies have all data persisted



578
579
580
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 578

def saveable?
  dependencies.all?(&:saved?)
end

#saved? ⇒ Boolean

Returns true if the whole InventoryCollection object has all data persisted.

Returns:

  • (Boolean) —

    true if the whole InventoryCollection object has all data persisted



573
574
575
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 573

def saved?
  saved
end

#store_created_records(records) ⇒ Object

Caches what records were created, for later use, e.g. post provision behavior

Parameters:

  • records (Array<ApplicationRecord, Hash>) —

    list of stored records



477
478
479
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 477

def store_created_records(records)
  @created_records.concat(records_identities(records))
end

#store_deleted_records(records) ⇒ Object

Caches what records were deleted/soft-deleted, for later use, e.g. post provision behavior

Parameters:

  • records (Array<ApplicationRecord, Hash>) —

    list of stored records



491
492
493
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 491

def store_deleted_records(records)
  @deleted_records.concat(records_identities(records))
end

#store_updated_records(records) ⇒ Object

Caches what records were updated, for later use, e.g. post provision behavior

Parameters:

  • records (Array<ApplicationRecord, Hash>) —

    list of stored records



484
485
486
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 484

def store_updated_records(records)
  @updated_records.concat(records_identities(records))
end

#supports_created_at? ⇒ Boolean

Returns true if the model_class has created_at column.

Returns:

  • (Boolean) —

    true if the model_class has created_at column



610
611
612
613
614
615
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 610

def supports_created_at?
  if @supports_created_at_cache.nil?
    @supports_created_at_cache = (model_class.column_names.include?("created_at") && ActiveRecord::Base.record_timestamps)
  end
  @supports_created_at_cache
end

#supports_created_on? ⇒ Boolean

Returns true if the model_class has created_on column.

Returns:

  • (Boolean) —

    true if the model_class has created_on column



594
595
596
597
598
599
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 594

def supports_created_on?
  if @supports_created_on_cache.nil?
    @supports_created_on_cache = (model_class.column_names.include?("created_on") && ActiveRecord::Base.record_timestamps)
  end
  @supports_created_on_cache
end

#supports_resource_timestamp? ⇒ Boolean

Returns true if the model_class has resource_timestamp column.

Returns:

  • (Boolean) —

    true if the model_class has resource_timestamp column



636
637
638
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 636

def supports_resource_timestamp?
  @supports_resource_timestamp_cache ||= model_class.column_names.include?("resource_timestamp")
end

#supports_resource_timestamps? ⇒ Boolean

Returns true if the model_class has resource_timestamps column.

Returns:

  • (Boolean) —

    true if the model_class has resource_timestamps column



631
632
633
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 631

def supports_resource_timestamps?
  @supports_resource_timestamps_cache ||= model_class.column_names.include?("resource_timestamps")
end

#supports_resource_timestamps_max? ⇒ Boolean

Returns true if the model_class has resource_timestamps_max column.

Returns:

  • (Boolean) —

    true if the model_class has resource_timestamps_max column



626
627
628
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 626

def supports_resource_timestamps_max?
  @supports_resource_timestamps_max_cache ||= model_class.column_names.include?("resource_timestamps_max")
end

#supports_resource_version? ⇒ Boolean

Returns true if the model_class has resource_version column.

Returns:

  • (Boolean) —

    true if the model_class has resource_version column



651
652
653
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 651

def supports_resource_version?
  @supports_resource_version_cache ||= model_class.column_names.include?("resource_version")
end

#supports_resource_versions? ⇒ Boolean

Returns true if the model_class has resource_versions column.

Returns:

  • (Boolean) —

    true if the model_class has resource_versions column



646
647
648
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 646

def supports_resource_versions?
  @supports_resource_versions_cache ||= model_class.column_names.include?("resource_versions")
end

#supports_resource_versions_max? ⇒ Boolean

Returns true if the model_class has resource_versions_max column.

Returns:

  • (Boolean) —

    true if the model_class has resource_versions_max column



641
642
643
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 641

def supports_resource_versions_max?
  @supports_resource_versions_max_cache ||= model_class.column_names.include?("resource_versions_max")
end

#supports_sti? ⇒ Boolean

Returns true if the model_class supports STI.

Returns:

  • (Boolean) —

    true if the model_class supports STI



588
589
590
591
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 588

def supports_sti?
  @supports_sti_cache = model_class.column_names.include?("type") if @supports_sti_cache.nil?
  @supports_sti_cache
end

#supports_updated_at? ⇒ Boolean

Returns true if the model_class has updated_at column.

Returns:

  • (Boolean) —

    true if the model_class has updated_at column



618
619
620
621
622
623
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 618

def supports_updated_at?
  if @supports_updated_at_cache.nil?
    @supports_updated_at_cache = (model_class.column_names.include?("updated_at") && ActiveRecord::Base.record_timestamps)
  end
  @supports_updated_at_cache
end

#supports_updated_on? ⇒ Boolean

Returns true if the model_class has updated_on column.

Returns:

  • (Boolean) —

    true if the model_class has updated_on column



602
603
604
605
606
607
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 602

def supports_updated_on?
  if @supports_updated_on_cache.nil?
    @supports_updated_on_cache = (model_class.column_names.include?("updated_on") && ActiveRecord::Base.record_timestamps)
  end
  @supports_updated_on_cache
end

#targeted? ⇒ Boolean

Returns true is processing of this InventoryCollection will be in targeted mode.

Returns:

  • (Boolean) —

    true is processing of this InventoryCollection will be in targeted mode



787
788
789
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 787

def targeted?
  targeted
end

#targeted_arel_default ⇒ InventoryRefresh::ApplicationRecordIterator

Builds targeted query limiting the results by the :references defined in parent_inventory_collections

Returns:



1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1051

def targeted_arel_default
  if parent_inventory_collections.collect { |x| x.model_class.base_class }.uniq.count > 1
    raise "Multiple :parent_inventory_collections with different base class are not supported by default. Write "\
          ":targeted_arel manually, or separate [#{self}] into 2 InventoryCollection objects."
  end
  parent_collection = parent_inventory_collections.first
  references        = parent_inventory_collections.map { |x| x.targeted_scope.primary_references }.reduce({}, :merge!)

  parent_collection.targeted_iterator_for(references, full_collection_for_comparison)
end

#targeted_iterator_for(references, query = nil) ⇒ InventoryRefresh::ApplicationRecordIterator

Returns iterator for the passed references and a query

Parameters:

Returns:



1089
1090
1091
1092
1093
1094
1095
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1089

def targeted_iterator_for(references, query = nil)
  InventoryRefresh::ApplicationRecordIterator.new(
    :inventory_collection => self,
    :manager_uuids_set    => references,
    :query                => query
  )
end

#targeted_selection_for(references) ⇒ String

Builds a multiselection conditions like (table1.a = a1 AND table2.b = b1) OR (table1.a = a2 AND table2.b = b2) for passed references

Parameters:

Returns:

  • (String) —

    A condition usable in .where of an ActiveRecord relation



1080
1081
1082
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1080

def targeted_selection_for(references)
  build_multi_selection_condition(transform_references_to_hashes(references))
end

#to_s ⇒ String

Returns a concise form of the inventoryCollection for easy logging.

Returns:

  • (String) —

    a concise form of the inventoryCollection for easy logging



983
984
985
986
987
988
989
990
991
992
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 983

def to_s
  whitelist = ", whitelist: [#{attributes_whitelist.to_a.join(", ")}]" if attributes_whitelist.present?
  blacklist = ", blacklist: [#{attributes_blacklist.to_a.join(", ")}]" if attributes_blacklist.present?

  strategy_name = ", strategy: #{strategy}" if strategy

  name = model_class || association

  "InventoryCollection:<#{name}>#{whitelist}#{blacklist}#{strategy_name}"
end

#transform_references_to_hashes(references) ⇒ Array<Hash>

Gets targeted references and transforms them into list of hashes

Parameters:

  • references (Array, InventoryRefresh::Inventorycollection::TargetedScope) —

    passed references

Returns:

  • (Array<Hash>) —

    References transformed into the array of hashes



1066
1067
1068
1069
1070
1071
1072
1073
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 1066

def transform_references_to_hashes(references)
  if references.kind_of?(Array)
    # Sliced InventoryRefresh::Inventorycollection::TargetedScope
    references.map { |x| x.second.full_reference }
  else
    references.values.map(&:full_reference)
  end
end

#unique_index_columns ⇒ Array<Symbol>

Returns all columns that are part of the best fit unique index.

Returns:

  • (Array<Symbol>) —

    all columns that are part of the best fit unique index



656
657
658
659
660
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 656

def unique_index_columns
  return @unique_index_columns if @unique_index_columns

  @unique_index_columns = unique_index_for(unique_index_keys).columns.map(&:to_sym)
end

#unique_index_for(keys) ⇒ ActiveRecord::ConnectionAdapters::IndexDefinition

Finds an index that fits the list of columns (keys) the best

Parameters:

  • keys (Array<Symbol>)

Returns:

  • (ActiveRecord::ConnectionAdapters::IndexDefinition) —

    unique index fitting the keys

Raises:

  • (Exception) —

    if the unique index for the columns was not found



685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 685

def unique_index_for(keys)
  @unique_index_for_keys_cache ||= {}
  @unique_index_for_keys_cache[keys] if @unique_index_for_keys_cache[keys]

  # Find all uniq indexes that that are covering our keys
  uniq_key_candidates = unique_indexes.each_with_object([]) { |i, obj| obj << i if (keys - i.columns.map(&:to_sym)).empty? }

  if @unique_indexes_cache.blank?
    raise "#{self} and its table #{model_class.table_name} must have a unique index defined "\
            "covering columns #{keys} to be able to use saver_strategy :concurrent_safe or :concurrent_safe_batch."
  end

  # Take the uniq key having the least number of columns
  @unique_index_for_keys_cache[keys] = uniq_key_candidates.min_by { |x| x.columns.count }
end

#unique_index_keys ⇒ Object 



662
663
664
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 662

def unique_index_keys
  @unique_index_keys ||= manager_ref_to_cols.map(&:to_sym)
end

#unique_indexes ⇒ Array<ActiveRecord::ConnectionAdapters::IndexDefinition>

Returns array of all unique indexes known to model.

Returns:

  • (Array<ActiveRecord::ConnectionAdapters::IndexDefinition>) —

    array of all unique indexes known to model



667
668
669
670
671
672
673
674
675
676
677
678
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 667

def unique_indexes
  @unique_indexes_cache if @unique_indexes_cache

  @unique_indexes_cache = model_class.connection.indexes(model_class.table_name).select(&:unique)

  if @unique_indexes_cache.blank?
    raise "#{self} and its table #{model_class.table_name} must have a unique index defined, to"\
            " be able to use saver_strategy :concurrent_safe or :concurrent_safe_batch."
  end

  @unique_indexes_cache
end

#update_only? ⇒ Boolean

Returns true means we want to only update data.

Returns:

  • (Boolean) —

    true means we want to only update data



553
554
555
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 553

def update_only?
  update_only
end

#use_ar_object? ⇒ Boolean

Returns true means we want to use ActiveRecord object for writing attributes and we want to perform casting on all columns.

Returns:

  • (Boolean) —

    true means we want to use ActiveRecord object for writing attributes and we want to perform casting on all columns



543
544
545
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 543

def use_ar_object?
  use_ar_object
end

#whitelist_attributes!(attributes) ⇒ Array<Symbol>

Add passed attributes to whitelist. The manager_ref attributes always needs to be in the white list, otherwise we will not be able to identify theinventory_object. We do not automatically add attributes causing fixed dependencies, so beware that without them, you won’t be able to create the record.

Parameters:

  • attributes (Array<Symbol>) —

    Attributes we want to whitelist

Returns:

  • (Array<Symbol>) —

    All whitelisted attributes



879
880
881
 
# File 'lib/inventory_refresh/inventory_collection.rb', line 879

def whitelist_attributes!(attributes)
  self.attributes_whitelist += attributes + (fixed_attributes + internal_attributes)
end