Class: Subprocess::Process

Inherits:
Object
  • Object
show all
Defined in:
lib/subprocess.rb

Overview

A child process. The preferred way of spawning a subprocess is through the functions on Subprocess (especially check_call and check_output).

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(cmd, opts = {}) {|process| ... } ⇒ Process

Create a new process.

Parameters:

  • cmd (Array<String>) —

    The command to run and its arguments (in the style of an ‘argv` array). Unlike Python’s subprocess module, ‘cmd` cannot be a String.

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :stdin (IO, Fixnum, String, Subprocess::PIPE, nil) —

    The ‘IO`, file descriptor number, or file name to use for the process’s standard input. If the magic value Subprocess::PIPE is passed, a new pipe will be opened.

  • :stdout (IO, Fixnum, String, Subprocess::PIPE, nil) —

    The ‘IO`, file descriptor number, or file name to use for the process’s standard output. If the magic value Subprocess::PIPE is passed, a pipe will be opened and attached to the process.

  • :stderr (IO, Fixnum, String, Subprocess::PIPE, Subprocess::STDOUT, nil) —

    The ‘IO`, file descriptor number, or file name to use for the process’s standard error. If the special value Subprocess::PIPE is passed, a pipe will be opened and attached to the process. If the special value STDOUT is passed, the process’s ‘stderr` will be redirected to its `stdout` (much like bash’s ‘2>&1`).

  • :cwd (String) —

    The directory to change to before executing the child process.

  • :env (Hash<String, String>) —

    The environment to use in the child process.

  • :retain_fds (Array<Fixnum>) —

    An array of file descriptor numbers that should not be closed before executing the child process. Note that, unlike Python (which has :close_fds defaulting to false), all file descriptors not specified here will be closed.

  • :exec_opts (Hash) —

    A hash that will be merged into the options hash of the call to Kernel#exec.

  • :preexec_fn (Proc) —

    A function that will be called in the child process immediately before executing ‘cmd`.

Yields:

  • (process) —

    Yields the just-spawned Subprocess::Process to the optional block. This occurs after all of Subprocess::Process‘s error handling has been completed, and is a great place to call #communicate, especially when used in conjunction with check_call.

Yield Parameters:

  • process (Process) —

    The process that was just spawned.

Raises:

  • (ArgumentError) 


241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
 
# File 'lib/subprocess.rb', line 241

def initialize(cmd, opts={}, &blk)
  raise ArgumentError, "cmd must be an Array of strings" unless Array === cmd
  raise ArgumentError, "cmd cannot be empty" if cmd.empty?

  @command = cmd

  # Figure out what file descriptors we should pass on to the child (and
  # make externally visible ourselves)
  @child_stdin, @stdin = parse_fd(opts[:stdin], 'r')
  @child_stdout, @stdout = parse_fd(opts[:stdout], 'w')
  unless opts[:stderr] == STDOUT
    @child_stderr, @stderr = parse_fd(opts[:stderr], 'w')
  end

  retained_fds = Set.new(opts[:retain_fds] || [])

  # A control pipe for ferrying errors back from the child
  control_r, control_w = IO.pipe

  @pid = fork do
    begin
      ::STDIN.reopen(@child_stdin) if @child_stdin
      ::STDOUT.reopen(@child_stdout) if @child_stdout
      if opts[:stderr] == STDOUT
        ::STDERR.reopen(::STDOUT)
      else
        ::STDERR.reopen(@child_stderr) if @child_stderr
      end

      # Set up a new environment if we're requested to do so.
      if opts[:env]
        ENV.clear
        begin
          ENV.update(opts[:env])
        rescue TypeError => e
          raise ArgumentError, "`env` option must be a hash where all keys and values are strings (#{e})"
        end
      end

      # Call the user back, maybe?
      if opts[:preexec_fn]
        if opts[:cwd]
          Dir.chdir(opts[:cwd], &opts[:preexec_fn])
        else
          opts[:preexec_fn].call
        end
      end

      options = {close_others: true}.merge(opts.fetch(:exec_opts, {}))
      if opts[:retain_fds]
        retained_fds.each { |fd| options[fd] = fd }
      end
      if opts[:cwd]
        # We use the chdir option to `exec` since wrapping the
        # `exec` in a Dir.chdir block caused these sporadic errors on macOS:
        # Too many open files - getcwd (Errno::EMFILE)
        options[:chdir] = opts[:cwd]
      end

      begin
        # Ruby's Kernel#exec will call an exec(3) variant if called with two
        # or more arguments, but when called with just a single argument will
        # spawn a subshell with that argument as the command. Since we always
        # want to call exec(3), we use the third exec form, which passes a
        # [cmdname, argv0] array as its first argument and never invokes a
        # subshell.
        exec([cmd[0], cmd[0]], *cmd[1..-1], options)
      rescue TypeError => e
        raise ArgumentError, "cmd must be an Array of strings (#{e})"
      end

    rescue Exception => e
      # Dump all errors up to the parent through the control socket
      Marshal.dump(e, control_w)
      control_w.flush
    end

    # Something has gone terribly, terribly wrong if we're hitting this :(
    exit!(1)
  end

  # Meanwhile, in the parent process...

  # First, let's close some things we shouldn't have access to
  @child_stdin.close if our_fd?(opts[:stdin])
  @child_stdout.close if our_fd?(opts[:stdout])
  @child_stderr.close if our_fd?(opts[:stderr])
  control_w.close

  # Any errors during the spawn process? We'll get past this point when the
  # child execs and the OS closes control_w
  begin
    e = Marshal.load(control_r)
    e = "Unknown Failure" unless e.is_a?(Exception) || e.is_a?(String)
    # Because we're throwing an exception and not returning a
    # Process, we need to make sure the child gets reaped
    wait
    raise e
  rescue EOFError # Nothing to read? Great!
  ensure
    control_r.close
  end

  # Everything is okay. Good job, team!
  blk.call(self) if blk
end

Instance Attribute Details

#command ⇒ Array<String> (readonly)

Returns The command this process was invoked with.

Returns:

  • (Array<String>) —

    The command this process was invoked with.



199
200
201
 
# File 'lib/subprocess.rb', line 199

def command
  @command
end

#pid ⇒ Fixnum (readonly)

Returns The process ID of the spawned process.

Returns:

  • (Fixnum) —

    The process ID of the spawned process.



199
 
# File 'lib/subprocess.rb', line 199

attr_reader :command, :pid, :status

#status ⇒ Object (readonly)

Returns the value of attribute status.



199
 
# File 'lib/subprocess.rb', line 199

attr_reader :command, :pid, :status

#stderr ⇒ Object (readonly)

Returns the value of attribute stderr.



190
 
# File 'lib/subprocess.rb', line 190

attr_reader :stdin, :stdout, :stderr

#stdin ⇒ IO (readonly)

Returns The ‘IO` that is connected to this process’s ‘stdin`.

Returns:

  • (IO) —

    The ‘IO` that is connected to this process’s ‘stdin`.



190
191
192
 
# File 'lib/subprocess.rb', line 190

def stdin
  @stdin
end

#stdout ⇒ IO (readonly)

Returns The ‘IO` that is connected to this process’s ‘stdout`.

Returns:

  • (IO) —

    The ‘IO` that is connected to this process’s ‘stdout`.



190
 
# File 'lib/subprocess.rb', line 190

attr_reader :stdin, :stdout, :stderr

Instance Method Details

#communicate(input = nil, timeout_s = nil) {|stdout, stderr| ... } ⇒ Array(String, String)?

Write the (optional) input to the process’s ‘stdin` and read the contents of `stdout` and `stderr`. If a block is provided, stdout and stderr are yielded as they are read. Otherwise they are buffered in memory and returned when the process exits. Do this all using `IO::select`, so we don’t deadlock due to full pipe buffers.

This is only really useful if you set some of ‘:stdin`, `:stdout`, and `:stderr` to Subprocess::PIPE.

Parameters:

  • input (String) (defaults to: nil) —

    A string to feed to the child’s standard input.

  • timeout_s (Numeric) (defaults to: nil) —

    Raise CommunicateTimeout if communicate does not finish after timeout_s seconds.

Yield Parameters:

  • stdout (String) —

    Data read from stdout since the last yield

  • stderr (String) —

    Data read from stderr since the last yield

Returns:

  • (Array(String, String), nil) —

    An array of two elements: the data read from the child’s standard output and standard error, respectively. Returns nil if a block is provided.

Raises:

  • (ArgumentError) 


401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
 
# File 'lib/subprocess.rb', line 401

def communicate(input=nil, timeout_s=nil)
  raise ArgumentError if !input.nil? && @stdin.nil?

  stdout, stderr = "", ""

  input = input.dup unless input.nil?

  @stdin.close if (input.nil? || input.empty?) && !@stdin.nil?

  timeout_at = Time.now + timeout_s if timeout_s

  self.class.catching_sigchld(pid) do |global_read, self_read|
    wait_r = [@stdout, @stderr, self_read, global_read].compact
    wait_w = [input && @stdin].compact
    done = false
    while !done
      # If the process has exited, we want to drain any remaining output before returning
      if poll
        ready_r = wait_r - [self_read, global_read]
        ready_w = []
        done = true
      else
        ready_r, ready_w = select_until(wait_r, wait_w, [], timeout_at)
        raise CommunicateTimeout.new(@command, stdout, stderr) if ready_r.nil?
      end

      if ready_r.include?(@stdout)
        if drain_fd(@stdout, stdout)
          wait_r.delete(@stdout)
        end
      end

      if ready_r.include?(@stderr)
        if drain_fd(@stderr, stderr)
          wait_r.delete(@stderr)
        end
      end

      if ready_r.include?(global_read)
        if drain_fd(global_read)
          raise "Unexpected internal error -- someone closed the global self-pipe!"
        end
        self.class.wakeup_sigchld
      end

      if ready_r.include?(self_read)
        if drain_fd(self_read)
          raise "Unexpected internal error -- someone closed our self-pipe!"
        end
      end

      if ready_w.include?(@stdin)
        written = 0
        begin
          written = @stdin.write_nonblock(input)
        rescue EOFError # Maybe I shouldn't catch this...
        rescue Errno::EINTR
        rescue IO::WaitWritable
          # On OS X, a pipe can raise EAGAIN even after select indicates
          # that it is writable. Once the process consumes from the pipe,
          # the next write should succeed and we should make forward progress.
          # Until then, treat this as not writing any bytes and continue looping.
          # For details see: https://github.com/stripe/subprocess/pull/22
          nil
        rescue Errno::EPIPE
          # The other side of the pipe closed before we could
          # write all of our input. This can happen if the
          # process exits prematurely.
          @stdin.close
          wait_w.delete(@stdin)
        end
        input[0...written] = ''
        if input.empty?
          @stdin.close
          wait_w.delete(@stdin)
        end
      end

      if block_given? && !(stderr.empty? && stdout.empty?)
        yield stdout, stderr
        stdout, stderr = "", ""
      end
    end
  end

  wait

  if block_given?
    nil
  else
    [stdout, stderr]
  end
end

#drain_fd(fd, buf = nil) ⇒ true, false

Do nonblocking reads from ‘fd`, appending all data read into `buf`.

Parameters:

  • fd (IO) —

    The file to read from.

  • buf (String) (defaults to: nil) —

    A buffer to append the read data to.

Returns:

  • (true, false) —

    Whether ‘fd` was closed due to an exceptional condition (`EOFError` or `EPIPE`).



371
372
373
374
375
376
377
378
379
380
381
382
 
# File 'lib/subprocess.rb', line 371

def drain_fd(fd, buf=nil)
  loop do
    tmp = fd.read_nonblock(4096).force_encoding(fd.external_encoding)
    buf << tmp unless buf.nil?
  end
rescue EOFError, Errno::EPIPE
  fd.close
  true
rescue Errno::EINTR
rescue Errno::EWOULDBLOCK, Errno::EAGAIN
  false
end

#poll ⇒ ::Process::Status?

Poll the child, setting (and returning) its status. If the child has not terminated, return nil and exit immediately

Returns:

  • (::Process::Status, nil) —

    The exit status of the process



352
353
354
 
# File 'lib/subprocess.rb', line 352

def poll
  @status ||= (::Process.waitpid2(@pid, ::Process::WNOHANG) || []).last
end

#send_signal(signal) ⇒ Object

Does exactly what it says on the box.

Parameters:

  • signal (String, Symbol, Fixnum) —

    The signal to send to the child process. Accepts all the same arguments as Ruby’s built-in Process::kill, for instance a string like “INT” or “SIGINT”, or a signal number like 2.



501
502
503
 
# File 'lib/subprocess.rb', line 501

def send_signal(signal)
  ::Process.kill(signal, pid)
end

#terminate ⇒ Object

Sends ‘SIGTERM` to the process.



506
507
508
 
# File 'lib/subprocess.rb', line 506

def terminate
  send_signal("TERM")
end

#wait ⇒ ::Process::Status

Wait for the child to return, setting and returning the status of the child.

Returns:

  • (::Process::Status) —

    The exit status of the process



360
361
362
 
# File 'lib/subprocess.rb', line 360

def wait
  @status ||= ::Process.waitpid2(@pid).last
end