Class: Subprocess::Process

Inherits:
Object
  • Object
show all
Defined in:
lib/subprocess.rb

Overview

A child process. The preferred way of spawning a subprocess is through the functions on Subprocess (especially check_call and check_output).

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(cmd, opts = {}) {|process| ... } ⇒ Process

Create a new process.

Parameters:

  • cmd (Array<String>) —

    The command to run and its arguments (in the style of an ‘argv` array). Unlike Python’s subprocess module, ‘cmd` cannnot be a String.

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :stdin (IO, Fixnum, String, Subprocess::PIPE, nil) —

    The ‘IO`, file descriptor number, or file name to use for the process’s standard input. If the magic value Subprocess::PIPE is passed, a new pipe will be opened.

  • :stdout (IO, Fixnum, String, Subprocess::PIPE, nil) —

    The ‘IO`, file descriptor number, or file name to use for the process’s standard output. If the magic value Subprocess::PIPE is passed, a pipe will be opened and attached to the process.

  • :stderr (IO, Fixnum, String, Subprocess::PIPE, Subprocess::STDOUT, nil) —

    The ‘IO`, file descriptor number, or file name to use for the process’s standard error. If the special value Subprocess::PIPE is passed, a pipe will be opened and attached to the process. If the special value STDOUT is passed, the process’s ‘stderr` will be redirected to its `stdout` (much like bash’s ‘2>&1`).

  • :cwd (String) —

    The directory to change to before executing the child process.

  • :env (Hash<String, String>) —

    The environment to use in the child process.

  • :retain_fds (Array<Fixnum>) —

    An array of file descriptor numbers that should not be closed before executing the child process. Note that, unlike Python (which has :close_fds defaulting to false), all file descriptors not specified here will be closed.

  • :exec_opts (Hash) —

    A hash that will be merged into the options hash of the call to Kernel#exec.

  • :preexec_fn (Proc) —

    A function that will be called in the child process immediately before executing ‘cmd`.

Yields:

  • (process) —

    Yields the just-spawned Subprocess::Process to the optional block. This occurs after all of Subprocess::Process‘s error handling has been completed, and is a great place to call #communicate, especially when used in conjunction with check_call.

Yield Parameters:

  • process (Process) —

    The process that was just spawned.

Raises:

  • (ArgumentError) 


226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
 
# File 'lib/subprocess.rb', line 226

def initialize(cmd, opts={}, &blk)
  raise ArgumentError, "cmd must be an Array" unless Array === cmd
  raise ArgumentError, "cmd cannot be empty" if cmd.empty?

  @command = cmd

  # Figure out what file descriptors we should pass on to the child (and
  # make externally visible ourselves)
  @child_stdin, @stdin = parse_fd(opts[:stdin], 'r')
  @child_stdout, @stdout = parse_fd(opts[:stdout], 'w')
  unless opts[:stderr] == STDOUT
    @child_stderr, @stderr = parse_fd(opts[:stderr], 'w')
  end

  retained_fds = Set.new(opts[:retain_fds] || [])

  # A control pipe for ferrying errors back from the child
  control_r, control_w = IO.pipe

  @pid = fork do
    begin
      FileUtils.cd(opts[:cwd]) if opts[:cwd]

      ::STDIN.reopen(@child_stdin) if @child_stdin
      ::STDOUT.reopen(@child_stdout) if @child_stdout
      if opts[:stderr] == STDOUT
        ::STDERR.reopen(::STDOUT)
      else
        ::STDERR.reopen(@child_stderr) if @child_stderr
      end

      # Set up a new environment if we're requested to do so.
      if opts[:env]
        ENV.clear
        ENV.update(opts[:env])
      end

      # Call the user back, maybe?
      opts[:preexec_fn].call if opts[:preexec_fn]

      options = {close_others: true}.merge(opts.fetch(:exec_opts, {}))
      if opts[:retain_fds]
        retained_fds.each { |fd| options[fd] = fd }
      end

      # Ruby's Kernel#exec will call an exec(3) variant if called with two
      # or more arguments, but when called with just a single argument will
      # spawn a subshell with that argument as the command. Since we always
      # want to call exec(3), we use the third exec form, which passes a
      # [cmdname, argv0] array as its first argument and never invokes a
      # subshell.
      exec([cmd[0], cmd[0]], *cmd[1..-1], options)

    rescue Exception => e
      # Dump all errors up to the parent through the control socket
      Marshal.dump(e, control_w)
      control_w.flush
    end

    # Something has gone terribly, terribly wrong if we're hitting this :(
    exit!(1)
  end

  # Meanwhile, in the parent process...

  # First, let's close some things we shouldn't have access to
  @child_stdin.close if our_fd?(opts[:stdin])
  @child_stdout.close if our_fd?(opts[:stdout])
  @child_stderr.close if our_fd?(opts[:stderr])
  control_w.close

  # Any errors during the spawn process? We'll get past this point when the
  # child execs and the OS closes control_w
  begin
    e = Marshal.load(control_r)
    e = "Unknown Failure" unless e.is_a?(Exception) || e.is_a?(String)
    # Because we're throwing an exception and not returning a
    # Process, we need to make sure the child gets reaped
    wait
    raise e
  rescue EOFError # Nothing to read? Great!
  ensure
    control_r.close
  end

  # Everything is okay. Good job, team!
  blk.call(self) if blk
end

Instance Attribute Details

#command ⇒ Array<String> (readonly)

Returns The command this process was invoked with.

Returns:

  • (Array<String>) —

    The command this process was invoked with.



184
185
186
 
# File 'lib/subprocess.rb', line 184

def command
  @command
end

#pid ⇒ Fixnum (readonly)

Returns The process ID of the spawned process.

Returns:

  • (Fixnum) —

    The process ID of the spawned process.



184
 
# File 'lib/subprocess.rb', line 184

attr_reader :command, :pid, :status

#status ⇒ Object (readonly)

Returns the value of attribute status.



184
 
# File 'lib/subprocess.rb', line 184

attr_reader :command, :pid, :status

#stderr ⇒ Object (readonly)

Returns the value of attribute stderr.



175
 
# File 'lib/subprocess.rb', line 175

attr_reader :stdin, :stdout, :stderr

#stdin ⇒ IO (readonly)

Returns The ‘IO` that is connected to this process’s ‘stdin`.

Returns:

  • (IO) —

    The ‘IO` that is connected to this process’s ‘stdin`.



175
176
177
 
# File 'lib/subprocess.rb', line 175

def stdin
  @stdin
end

#stdout ⇒ IO (readonly)

Returns The ‘IO` that is connected to this process’s ‘stdout`.

Returns:

  • (IO) —

    The ‘IO` that is connected to this process’s ‘stdout`.



175
 
# File 'lib/subprocess.rb', line 175

attr_reader :stdin, :stdout, :stderr

Instance Method Details

#communicate(input = nil) ⇒ Array<String>

Write the (optional) input to the process’s ‘stdin`. Also, read (and buffer in memory) the contents of `stdout` and `stderr`. Do this all using `IO::select`, so we don’t deadlock due to full pipe buffers.

This is only really useful if you set some of ‘:stdin`, `:stdout`, and `:stderr` to Subprocess::PIPE.

Parameters:

  • input (String) (defaults to: nil) —

    A string to feed to the child’s standard input.

Returns:

  • (Array<String>) —

    An array of two elements: the data read from the child’s standard output and standard error, respectively.

Raises:

  • (ArgumentError) 


361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
 
# File 'lib/subprocess.rb', line 361

def communicate(input=nil)
  raise ArgumentError if !input.nil? && @stdin.nil?

  stdout, stderr = "", ""
  stdout_encoding = @stdout.external_encoding if @stdout
  stderr_encoding = @stderr.external_encoding if @stderr

  input = input.dup unless input.nil?

  @stdin.close if (input.nil? || input.empty?) && !@stdin.nil?

  self_read, self_write = IO.pipe
  self.class.catching_sigchld(pid, self_write) do
    wait_r = [@stdout, @stderr, self_read].compact
    wait_w = [input && @stdin].compact
    loop do
      ready_r, ready_w = select(wait_r, wait_w)

      # If the child exits, we still have to be sure to read any data left
      # in the pipes. So we poll the child, drain all the pipes, and *then*
      # check @status.
      #
      # It's very important that we do not call poll between draining the
      # pipes and checking @status. If we did, we open a race condition
      # where the child writes to stdout and exits in that brief window,
      # causing us to lose that data.
      poll

      if ready_r.include?(@stdout)
        if drain_fd(@stdout, stdout)
          wait_r.delete(@stdout)
        end
      end

      if ready_r.include?(@stderr)
        if drain_fd(@stderr, stderr)
          wait_r.delete(@stderr)
        end
      end

      if ready_r.include?(self_read)
        if drain_fd(self_read)
          raise "Unexpected internal error -- someone closed our self-pipe!"
        end
      end

      if ready_w.include?(@stdin)
        begin
          written = @stdin.write_nonblock(input)
        rescue EOFError # Maybe I shouldn't catch this...
        rescue Errno::EINTR
        end
        input[0...written] = ''
        if input.empty?
          @stdin.close
          wait_w.delete(@stdin)
        end
      end

      break if @status

      # If there's nothing left to wait for, we're done!
      break if wait_r.length == 0 && wait_w.length == 0
    end
  end

  wait

  stdout.force_encoding(stdout_encoding) if stdout_encoding
  stderr.force_encoding(stderr_encoding) if stderr_encoding

  [stdout, stderr]
end

#drain_fd(fd, buf = nil) ⇒ true, false

Do nonblocking reads from ‘fd`, appending all data read into `buf`.

Parameters:

  • fd (IO) —

    The file to read from.

  • buf (String) (defaults to: nil) —

    A buffer to append the read data to.

Returns:

  • (true, false) —

    Whether ‘fd` was closed due to an exceptional condition (`EOFError` or `EPIPE`).



338
339
340
341
342
343
344
345
346
347
348
349
 
# File 'lib/subprocess.rb', line 338

def drain_fd(fd, buf=nil)
  loop do
    tmp = fd.read_nonblock(4096)
    buf << tmp unless buf.nil?
  end
rescue EOFError, Errno::EPIPE
  fd.close
  true
rescue Errno::EINTR
rescue Errno::EWOULDBLOCK, Errno::EAGAIN
  false
end

#poll ⇒ ::Process::Status?

Poll the child, setting (and returning) its status. If the child has not terminated, return nil and exit immediately

Returns:

  • (::Process::Status, nil) —

    The exit status of the process



319
320
321
 
# File 'lib/subprocess.rb', line 319

def poll
  @status ||= (::Process.waitpid2(@pid, ::Process::WNOHANG) || []).last
end

#send_signal(signal) ⇒ Object

Does exactly what it says on the box.

Parameters:

  • signal (String, Symbol, Fixnum) —

    The signal to send to the child process. Accepts all the same arguments as Ruby’s built-in Process::kill, for instance a string like “INT” or “SIGINT”, or a signal number like 2.



441
442
443
 
# File 'lib/subprocess.rb', line 441

def send_signal(signal)
  ::Process.kill(signal, pid)
end

#terminate ⇒ Object

Sends ‘SIGTERM` to the process.



446
447
448
 
# File 'lib/subprocess.rb', line 446

def terminate
  send_signal("TERM")
end

#wait ⇒ ::Process::Status

Wait for the child to return, setting and returning the status of the child.

Returns:

  • (::Process::Status) —

    The exit status of the process



327
328
329
 
# File 'lib/subprocess.rb', line 327

def wait
  @status ||= ::Process.waitpid2(@pid).last
end