Module: RubyReactor::Worker

Included in:
Adapters::ActiveJob::Worker, Adapters::Sidekiq::Worker
Defined in:
lib/ruby_reactor/worker.rb

Overview

Framework-agnostic resume/snooze/escalate logic shared by every queueing backend's worker class. Each backend (Adapters::Sidekiq::Worker, Adapters::ActiveJob::Worker, ...) includes this and supplies its own self.class.perform_in (native on Sidekiq::Worker, via Adapters::ActiveJob::Compat on ActiveJob::Base) — nothing here references a specific backend.

Constant Summary collapse

TERMINAL_STATUSES = 
%w[completed failed cancelled skipped].freeze

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.record_retries_exhausted(args, exception) ⇒ Object

Last line of observability when a job burns its whole retry budget on an infrastructure failure and the backend then discards it (Sidekiq runs with dead: false): without this, the context would stay "running" forever with zero surface anywhere, and every reader would wait out its full timeout. Called from the backends' retries-exhausted hooks with the job's own args. Best-effort — never raises back into the backend.



19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
 
# File 'lib/ruby_reactor/worker.rb', line 19

def self.record_retries_exhausted(args, exception)
  context_id, reactor_class_name = args
  return unless context_id

  reactor_class_name ||= RubyReactor.reactor_storage_name(nil)
  storage = RubyReactor.configuration.storage_adapter
  data = storage.retrieve_context(context_id, reactor_class_name)
  return if data.nil? || TERMINAL_STATUSES.include?((data["status"] || data[:status]).to_s)

  data["status"] = "failed"
  data["failure_reason"] = {
    "message" => "job retries exhausted: #{exception.class.name}: #{exception.message}",
    "exception_class" => exception.class.name
  }
  storage.store_context(context_id, JSON.generate(data), reactor_class_name)
  storage.publish(RubyReactor.async_reactor_channel(context_id), "failed")
rescue StandardError => e
  RubyReactor.configuration.logger.error(
    "RubyReactor: could not record retries-exhausted failure for #{context_id}: #{e.class.name}: #{e.message}"
  )
end

Instance Method Details

#perform(context_id, reactor_class_name = nil, snooze_count = 0) ⇒ Object

Identity-only payload: storage is the source of truth. Rehydrate the live context from storage by id, then resume. A nil read means the context was swept, expired, or already terminal-and-collected — nothing to resume.



44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
 
# File 'lib/ruby_reactor/worker.rb', line 44

def perform(context_id, reactor_class_name = nil, snooze_count = 0)
  # Normalize so a nil/omitted name resolves to the same storage key the
  # enqueue path wrote (always via reactor_storage_name). Without this a
  # nil here builds "reactor::context:<id>" and misses the stored
  # "reactor:AnonymousReactor:context:<id>", silently no-op'ing.
  reactor_class_name ||= RubyReactor.reactor_storage_name(nil)
  data = RubyReactor.configuration.storage_adapter.retrieve_context(context_id, reactor_class_name)
  return if data.nil?

  begin
    context = ContextSerializer.deserialize_hash(data)
  rescue RubyReactor::Error::DeserializationError,
         RubyReactor::Error::SchemaVersionError => e
    # Permanent failures — re-reading the same stored blob will keep
    # failing. Mark the context as failed (best-effort) and return so
    # the job does not burn its retry budget.
    handle_deserialization_failure(context_id, reactor_class_name, e)
    return
  end

  resolve_reactor_class!(context, reactor_class_name)
  unless context.reactor_class
    # Still unresolved (class not loaded, or an anonymous reactor with no
    # storage name to look up) — Executor.new below would blow up on a nil
    # class and burn the job's retry budget forever. Fail the context now.
    error = RubyReactor::Error::DeserializationError.new(
      "reactor class '#{reactor_class_name}' could not be resolved"
    )
    handle_deserialization_failure(context_id, reactor_class_name, error)
    return
  end

  # Mark that we're executing inline to prevent nested async calls
  context.inline_async_execution = true

  begin
    # Resume execution from the failed step
    executor = Executor.new(context.reactor_class, {}, context)
    executor.resume_execution
    # No explicit save here: resume_execution's ensure block already persists
    # the final root state (`save_context unless skip_context_persist?`), and
    # in the worker the executor's context IS the root, so an extra checkpoint!
    # would just re-write the identical blob to the identical key. The
    # skip_context_persist? guard (stale-batch redelivery of an already-terminal
    # context) is likewise honored there.

    # Return the executor (which now has the result stored in it)
    executor
  rescue RubyReactor::Lock::AcquisitionError,
         RubyReactor::Semaphore::AcquisitionError,
         RubyReactor::RateLimit::ExceededError,
         RubyReactor::OrderedLock::WaitError,
         RubyReactor::Error::AsyncResultPending => e
    # Snooze on expected concurrency, rate, or ordering contention.
    # OrderedLock::WaitError carries a poison-pill-derived retry hint,
    # consumed by compute_snooze_delay below. We avoid the framework's native
    # retry path so this doesn't burn the job's retry budget or appear
    # as an error in dashboards. After the configured cap is reached we
    # escalate by marking the reactor as failed.
    handle_snooze(context_id, reactor_class_name, context, snooze_count, e)
  rescue RubyReactor::RateLimitRegistry::UnknownLimitError => e
    # Permanent configuration error — snoozing or retrying the same job
    # will keep failing. Mark the context failed immediately.
    escalate_snooze(context, snooze_count, e)
  end
end