Module: RunnerExecution

Included in:
GitFastClone::Runner
Defined in:
lib/runner_execution.rb

Overview

Execution primitives that force explicit error handling and never call the shell. Cargo-culted from internal BuildExecution code on top of public version: github.com/square/build_execution

Defined Under Namespace

Classes: RunnerExecutionRuntimeError

Class Method Summary collapse

Class Method Details

.check_status(cmd, status, output: nil, quiet: false, print_on_failure: false) ⇒ Object

Raises:



171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
 
# File 'lib/runner_execution.rb', line 171

def check_status(cmd, status, output: nil, quiet: false, print_on_failure: false)
  return if status.exited? && status.exitstatus == 0

  logger.info(output) if print_on_failure
  # If we exited nonzero or abnormally, print debugging info and explode.
  if status.exited?
    logger.debug("Process Exited normally. Exit status:#{status.exitstatus}") unless quiet
  else
    # This should only get executed if we're stopped or signaled
    logger.debug("Process exited abnormally:\nProcessStatus: #{status.inspect}\n" \
      "Raw POSIX Status: #{status.to_i}\n") unless quiet
  end

  raise RunnerExecutionRuntimeError.new(status, cmd, output)
end

.debug_print_cmd_list(cmd_list) ⇒ Object 



109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
 
# File 'lib/runner_execution.rb', line 109

def debug_print_cmd_list(cmd_list)
  # Take a list of command argument lists like you'd sent to open3.pipeline or
  # fail_on_error_pipe and print out a string that would do the same thing when
  # entered at the shell.
  #
  # This is a converter from our internal representation of commands to a subset
  # of bash that can be executed directly.
  #
  # Note this has problems if you specify env or opts
  # TODO: make this remove those command parts
  "\"" +
    cmd_list.map do |cmd|
      cmd.map do |arg|
        arg.gsub("\"", "\\\"") # Escape all double quotes in command arguments
      end.join("\" \"") # Fully quote all command parts, beginning and end.
    end.join("\" | \"") + "\"" # Pipe commands to one another.
end

.exit_on_status(output, cmd_list, status_list, quiet: false, print_on_failure: false) ⇒ Object

If any of the statuses are bad, exits with the return code of the first one.

Otherwise returns first argument (output)



160
161
162
163
164
165
166
167
168
 
# File 'lib/runner_execution.rb', line 160

def exit_on_status(output, cmd_list, status_list, quiet: false, print_on_failure: false)
  status_list.each_index do |index|
    status = status_list[index]
    cmd = cmd_list[index]
    check_status(cmd, status, output: output, quiet: quiet, print_on_failure: print_on_failure)
  end

  output
end

.fail_on_error(*cmd, stdin_data: nil, binmode: false, quiet: false, print_on_failure: false, **opts) ⇒ Object

Runs a command that fails on error. Uses popen2e wrapper. Handles bad statuses with potential for retries.



25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
 
# File 'lib/runner_execution.rb', line 25

def fail_on_error(*cmd, stdin_data: nil, binmode: false, quiet: false, print_on_failure: false, **opts)
  print_command('Running Shell Safe Command:', [cmd]) unless quiet
  shell_safe_cmd = shell_safe(cmd)
  retry_times = opts[:retry] || 0
  opts.delete(:retry)

  while retry_times >= 0
    output, status = popen2e_wrapper(*shell_safe_cmd, stdin_data: stdin_data, binmode: binmode,
                                     quiet: quiet, **opts)

    break unless status.exitstatus != 0

    logger.debug("Command failed with exit status #{status.exitstatus}, retrying #{retry_times} more time(s).") if retry_times > 0
    retry_times -= 1
  end

  # Get out with the status, good or bad.
  # When quiet, we don't need to print the output, as it is already streamed from popen2e_wrapper
  needs_print_on_failure = quiet && print_on_failure
  exit_on_status(output, [shell_safe_cmd], [status], quiet: quiet, print_on_failure: needs_print_on_failure)
end

.logger ⇒ Object 



191
192
193
 
# File 'lib/runner_execution.rb', line 191

def logger
  DEFAULT_LOGGER
end

.popen2e_wrapper(*shell_safe_cmd, stdin_data: nil, binmode: false, quiet: false, **opts) ⇒ Object

Wrapper around open3.popen2e

We emulate open3.capture2e with the following changes in behavior: 1) The command is printed to stdout before execution. 2) Attempts to use the shell implicitly are blocked. 3) Nonzero return codes result in the process exiting. 4) Combined stdout/stderr goes to callers stdout

(continuously streamed) and is returned as a string

If you’re looking for more process/stream control read the spawn documentation, and pass options directly here

Raises:

  • (ArgumentError) 


59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
 
# File 'lib/runner_execution.rb', line 59

def popen2e_wrapper(*shell_safe_cmd, stdin_data: nil, binmode: false,
                     quiet: false, **opts)

  env = opts.delete(:env) { {} }
  raise ArgumentError, "The :env option must be a hash, not #{env.inspect}" if !env.is_a?(Hash)

  # Most of this is copied from Open3.capture2e in ruby/lib/open3.rb
  _output, _status = Open3.popen2e(env, *shell_safe_cmd, opts) do |i, oe, t|
    if binmode
      i.binmode
      oe.binmode
    end

    outerr_reader = Thread.new do
      if quiet
        oe.read
      else
        # Instead of oe.read, we redirect. Output from command goes to stdout
        # and also is returned for processing if necessary.
        tee(oe, STDOUT)
      end
    end

    if stdin_data
      begin
        i.write stdin_data
      rescue Errno::EPIPE
      end
    end

    i.close
    [outerr_reader.value, t.value]
  end
end

.print_command(message, cmd) ⇒ Object

Prints a formatted string with command



129
130
131
 
# File 'lib/runner_execution.rb', line 129

def print_command(message, cmd)
  logger.debug("#{message} #{debug_print_cmd_list(cmd)}\n")
end

.shell_safe(cmd) ⇒ Object

Look at a cmd list intended for spawn. determine if spawn will call the shell implicitly, fail in that case.



97
98
99
100
101
102
103
104
105
106
 
# File 'lib/runner_execution.rb', line 97

def shell_safe(cmd)
  # Take the first string and change it to a list of [executable,argv0]
  # This syntax for calling popen2e (and eventually spawn) avoids
  # the shell in all cases
  shell_safe_cmd = Array.new(cmd)
  if shell_safe_cmd[0].class == String
    shell_safe_cmd[0] = [shell_safe_cmd[0], shell_safe_cmd[0]]
  end
  shell_safe_cmd
end

.tee(in_stream, out_stream) ⇒ Object

Takes in an input stream and an output stream Redirects data from one to the other until the input stream closes. Returns all data that passed through on return.



137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
 
# File 'lib/runner_execution.rb', line 137

def tee(in_stream, out_stream)
  alldata = ''
  loop do
    begin
      data = in_stream.read_nonblock(4096)
      alldata += data
      out_stream.write(data)
      out_stream.flush
    rescue IO::WaitReadable
      IO.select([in_stream])
      retry
    rescue IOError
      break
    end
  end
  alldata
end