Class: HotCell::Sweeper

Inherits:
Object
  • Object
show all
Defined in:
lib/hot_cell/sweeper.rb

Overview

Unlinks what the supervisor renamed aside, in a process of its own.

How long a recursive delete takes is chosen by the input that filled the tree, so it runs in neither the supervisor, whose loop enforces every deadline, nor a worker's request. A worker sweeps its own slot after it has answered, but a worker killed at its deadline never reaches that ensure — and a slot whose every request is killed stacked one tree per kill until the scratch was full. This is the sweep that needs no request to run: the supervisor forks it on a timer, holds it to a deadline, and never runs two at once.

Instance Method Summary collapse

Constructor Details

#initialize(workspace:, configuration:, log:) ⇒ Sweeper

Returns a new instance of Sweeper.



13
14
15
16
17
 
# File 'lib/hot_cell/sweeper.rb', line 13

def initialize(workspace:, configuration:, log:)
  @workspace = workspace
  @configuration = configuration
  @log = log
end

Instance Method Details

#run ⇒ Object

exit! for the reason Worker#run does: nothing inherited from the supervisor may run its teardown here.

The cell's memory limit goes on first, and only that one. FileUtils.remove_entry lists a directory before it unlinks anything in it, so a tree one directory wide enough allocates in proportion to its width; RLIMIT_DATA makes that this process's NoMemoryError and a sweeper.crashed line. It is a bound on this process and not on the cell: the cgroup counts every worker and the tmpfs too, and can run out first. A sweep that dies this way makes no progress on that directory, and that is accepted here.

Not file_size: this process writes nothing but log lines, and a log that is a regular file is past any worker's limit already, so the first line would have killed the sweeper with SIGXFSZ.



29
30
31
32
33
34
35
36
37
38
39
 
# File 'lib/hot_cell/sweeper.rb', line 29

def run
  configuration.limits.merge(file_size: nil, open_files: nil).apply
  started = Clock.now
  swept = slots.sum { |slot| sweep slot }

  log.write "scratch.swept", pid: Process.pid, swept: swept, duration_ms: Clock.ms_since(started)
  exit! 0
rescue Exception => error
  log.write "sweeper.crashed", pid: Process.pid, error: error.class.name, message: Failure.sanitize(error.message)
  exit! 1
end