Class: Webhookdb::BackfillJob

Inherits:
Object
  • Object
show all
Defined in:
lib/webhookdb/backfill_job.rb

Overview

Represents the boundaries around a single execution of backfilling an integration. Each instance points to a single run of a single integration. There may be child jobs pointing to dependent integrations, or a parent job for a dependency backfill.

When creating jobs, you can create a single job (a ‘shallow’ backfill) with create, or use create_recursive to create jobs for all dependencies (a ‘job group’).

Each job tracks when the backfill starts and ends. Iterating the full job graph can determine if a group is fully finished, or still in-progress.

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#_fixture_cascade ⇒ Object

Returns the value of attribute _fixture_cascade.



24
25
26
 
# File 'lib/webhookdb/backfill_job.rb', line 24

def _fixture_cascade
  @_fixture_cascade
end

Class Method Details

.create_recursive(service_integration:, incremental:, created_by: nil, parent_job: nil, criteria: nil) ⇒ Webhookdb::BackfillJob

Returns:



27
28
29
30
31
32
33
 
# File 'lib/webhookdb/backfill_job.rb', line 27

def self.create_recursive(service_integration:, incremental:, created_by: nil, parent_job: nil, criteria: nil)
  self.db.transaction do
    root = self.create(service_integration:, parent_job:, incremental:, created_by:, criteria: criteria || {})
    root.setup_recursive
    root
  end
end

Instance Method Details

#before_create ⇒ Object

:section: Sequel Hooks



78
79
80
 
# File 'lib/webhookdb/backfill_job.rb', line 78

def before_create
  self[:opaque_id] ||= Webhookdb::Id.new_opaque_id("bfj")
end

#enqueue ⇒ Object 



66
67
68
 
# File 'lib/webhookdb/backfill_job.rb', line 66

def enqueue
  self.publish_deferred("run", self.id)
end

#enqueue_children ⇒ Object 



70
71
72
 
# File 'lib/webhookdb/backfill_job.rb', line 70

def enqueue_children
  self.child_jobs.each(&:enqueue)
end

#finished? ⇒ Boolean

Returns:

  • (Boolean) 


47
 
# File 'lib/webhookdb/backfill_job.rb', line 47

def finished? = !!self.finished_at

#fully_finished? ⇒ Boolean

Returns:

  • (Boolean) 


58
 
# File 'lib/webhookdb/backfill_job.rb', line 58

def fully_finished? = !!self.fully_finished_at

#fully_finished_at ⇒ Object 



49
50
51
52
53
54
55
56
 
# File 'lib/webhookdb/backfill_job.rb', line 49

def fully_finished_at
  parent_finished = self.finished_at
  return nil if parent_finished.nil?
  children_finished = self.child_jobs.map(&:fully_finished_at)
  return nil if children_finished.any?(&:nil?)
  children_finished << parent_finished
  return children_finished.max
end

#incremental? ⇒ Boolean

Returns:

  • (Boolean) 


44
 
# File 'lib/webhookdb/backfill_job.rb', line 44

def incremental? = self.incremental

#setup_recursive ⇒ Object

You should use ::create_recursive instead. This is mostly here for use in tests/fixtures.

Raises:



37
38
39
40
41
42
 
# File 'lib/webhookdb/backfill_job.rb', line 37

def setup_recursive
  raise Webhookdb::InvalidPrecondition, "already has children" if self.child_jobs.present?
  self.service_integration.dependents.map do |dep|
    self.class.create_recursive(service_integration: dep, parent_job: self, incremental:, criteria:)
  end
end

#started? ⇒ Boolean

Returns:

  • (Boolean) 


46
 
# File 'lib/webhookdb/backfill_job.rb', line 46

def started? = !!self.started_at

#status ⇒ Object 



60
61
62
63
64
 
# File 'lib/webhookdb/backfill_job.rb', line 60

def status
  return "enqueued" unless self.started?
  return "finished" if self.fully_finished?
  return "inprogress"
end