Class: Subprocess::Process

Inherits:
Object
  • Object
show all
Defined in:
lib/subprocess.rb

Overview

A child process. The preferred way of spawning a subprocess is through the functions on Subprocess (especially check_call and check_output).

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(cmd, opts = {}) {|process| ... } ⇒ Process

Create a new process.

Parameters:

  • cmd (Array<String>) —

    The command to run and its arguments (in the style of an ‘argv` array).

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :stdin (IO, Fixnum, String, Subprocess::PIPE, nil) —

    The ‘IO`, file descriptor number, or file name to use for the process’s standard input. If the magic value Subprocess::PIPE is passed, a new pipe will be opened.

  • :stdout (IO, Fixnum, String, Subprocess::PIPE, nil) —

    The ‘IO`, file descriptor number, or file name to use for the process’s standard output. If the magic value Subprocess::PIPE is passed, a pipe will be opened and attached to the process.

  • :stderr (IO, Fixnum, String, Subprocess::PIPE, Subprocess::STDOUT, nil) —

    The ‘IO`, file descriptor number, or file name to use for the process’s standard error. If the special value Subprocess::PIPE is passed, a pipe will be opened and attached to the process. If the special value STDOUT is passed, the process’s ‘stderr` will be redirected to its `stdout` (much like bash’s ‘2>&1`).

  • :cwd (String) —

    The directory to change to before executing the child process.

  • :env (Hash<String, String>) —

    The environment to use in the child process.

  • :retain_fds (Array<Fixnum>) —

    An array of file descriptor numbers that should not be closed before executing the child process. Note that, unlike Python (which has :close_fds defaulting to false), all file descriptors not specified here will be closed.

  • :preexec_fn (Proc) —

    A function that will be called in the child process immediately before executing ‘cmd`. Note: we don’t actually close file descriptors, but instead set them to auto-close on ‘exec` (using `FD_CLOEXEC`), so your application will probably continue to behave as expected.

Yields:

  • (process) —

    Yields the just-spawned Subprocess::Process to the optional block. This occurs after all of Subprocess::Process‘s error handling has been completed, and is a great place to call #communicate, especially when used in conjunction with check_call.

Yield Parameters:

  • process (Process) —

    The process that was just spawned.



238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
 
# File 'lib/subprocess.rb', line 238

def initialize(cmd, opts={}, &blk)
  @command = cmd

  # Figure out what file descriptors we should pass on to the child (and
  # make externally visible ourselves)
  @child_stdin, @stdin = parse_fd(opts[:stdin], 'r')
  @child_stdout, @stdout = parse_fd(opts[:stdout], 'w')
  unless opts[:stderr] == STDOUT
    @child_stderr, @stderr = parse_fd(opts[:stderr], 'w')
  end

  retained_fds = Set.new(opts[:retain_fds] || [])

  # A control pipe for ferrying errors back from the child
  control_r, control_w = IO.pipe

  @pid = fork do
    begin
      require 'fcntl'

      FileUtils.cd(opts[:cwd]) if opts[:cwd]

      # The only way to mark an fd as CLOEXEC in ruby is to create an IO
      # object wrapping it. In 1.8, however, there's no way to create that
      # IO without it believing it owns the underlying fd, s.t. it will
      # close the fd if the IO is GC'd before the exec. Since we don't want
      # that, we stash a list of these IO objects to prevent them from
      # getting GC'd, since we are about to exec, which will clean
      # everything up anyways.
      fds = []

      # We have a whole ton of file descriptors that we don't want leaking
      # into the child. Set them all to close when we exec away.
      #
      # Ruby 1.9+ note: exec has a :close_others argument (and 2.0 closes
      # FDs by default). When we stop supporting Ruby 1.8, all of this can
      # go away.
      if File.directory?("/dev/fd")
        # On many modern UNIX-y systems, we can perform an optimization by
        # looking through /dev/fd, which is a sparse listing of all the
        # descriptors we have open. This allows us to avoid an expensive
        # linear scan.
        Dir.foreach("/dev/fd") do |file|
          fd = file.to_i
          if file.start_with?('.') || fd < 3 || retained_fds.include?(fd)
            next
          end
          begin
            fds << mark_fd_cloexec(fd)
          rescue Errno::EBADF
            # The fd might have been closed by now; that's peaceful.
          end
        end
      else
        # This is the big hammer. There's not really a good way of doing
        # this comprehensively across all platforms without just trying them
        # all. We only go up to the soft limit here. If you've been messing
        # with the soft limit, we might miss a few. Also, on OSX (perhaps
        # BSDs in general?), where the soft limit means something completely
        # different.
        special = [@child_stdin, @child_stdout, @child_stderr].compact
        special = Hash[special.map { |f| [f.fileno, f] }]
        3.upto(::Process.getrlimit(::Process::RLIMIT_NOFILE).first) do |fd|
          next if retained_fds.include?(fd)
          begin
            # I don't know why we need to do this, but OSX started freaking
            # out when trying to dup2 below if FD_CLOEXEC had been set on a
            # fresh IO instance referring to the same underlying file
            # descriptor as what we were trying to dup2 from.
            if special[fd]
              special[fd].fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC)
            else
              fds << mark_fd_cloexec(fd)
            end
          rescue Errno::EBADF # Ignore FDs that don't exist
          end
        end
      end

      # dup2 the correct descriptors into place. Note that this clears the
      # FD_CLOEXEC flag on the new file descriptors (but not the old ones).
      ::STDIN.reopen(@child_stdin) if @child_stdin
      ::STDOUT.reopen(@child_stdout) if @child_stdout
      if opts[:stderr] == STDOUT
        ::STDERR.reopen(::STDOUT)
      else
        ::STDERR.reopen(@child_stderr) if @child_stderr
      end

      # Set up a new environment if we're requested to do so.
      if opts[:env]
        ENV.clear
        ENV.update(opts[:env])
      end

      # Call the user back, maybe?
      opts[:preexec_fn].call if opts[:preexec_fn]

      # Ruby 1.8's exec is really stupid--there's no way to specify that
      # you want to exec a single thing *without* performing shell
      # expansion. So this is the next best thing.
      args = cmd
      if cmd.length == 1
        args = ["'" + cmd[0].gsub("'", "\\'") + "'"]
      end
      if opts[:retain_fds]
        redirects = {}
        retained_fds.each { |fd| redirects[fd] = fd }
        args << redirects
      end
      exec(*args)

    rescue Exception => e
      # Dump all errors up to the parent through the control socket
      Marshal.dump(e, control_w)
      control_w.flush
    end

    # Something has gone terribly, terribly wrong if we're hitting this :(
    exit!(1)
  end

  # Meanwhile, in the parent process...

  # First, let's close some things we shouldn't have access to
  [@child_stdin, @child_stdout, @child_stderr, control_w].each do |fd|
    fd.close unless fd.nil?
  end

  # Any errors during the spawn process? We'll get past this point when the
  # child execs and the OS closes control_w because of the FD_CLOEXEC
  begin
    e = Marshal.load(control_r)
    e = "Unknown Failure" unless e.is_a?(Exception) || e.is_a?(String)
    raise e
  rescue EOFError # Nothing to read? Great!
  ensure
    control_r.close
  end

  # Everything is okay. Good job, team!
  blk.call(self) if blk
end

Instance Attribute Details

#command ⇒ Array<String> (readonly)

Returns The command this process was invoked with.

Returns:

  • (Array<String>) —

    The command this process was invoked with.



196
197
198
 
# File 'lib/subprocess.rb', line 196

def command
  @command
end

#pid ⇒ Fixnum (readonly)

Returns The process ID of the spawned process.

Returns:

  • (Fixnum) —

    The process ID of the spawned process.



196
 
# File 'lib/subprocess.rb', line 196

attr_reader :command, :pid, :status

#status ⇒ Object (readonly)

Returns the value of attribute status.



196
 
# File 'lib/subprocess.rb', line 196

attr_reader :command, :pid, :status

#stderr ⇒ Object (readonly)

Returns the value of attribute stderr.



187
 
# File 'lib/subprocess.rb', line 187

attr_reader :stdin, :stdout, :stderr

#stdin ⇒ IO (readonly)

Returns The ‘IO` that is connected to this process’s ‘stdin`.

Returns:

  • (IO) —

    The ‘IO` that is connected to this process’s ‘stdin`.



187
188
189
 
# File 'lib/subprocess.rb', line 187

def stdin
  @stdin
end

#stdout ⇒ IO (readonly)

Returns The ‘IO` that is connected to this process’s ‘stdout`.

Returns:

  • (IO) —

    The ‘IO` that is connected to this process’s ‘stdout`.



187
 
# File 'lib/subprocess.rb', line 187

attr_reader :stdin, :stdout, :stderr

Instance Method Details

#communicate(input = nil) ⇒ Array<String>

Write the (optional) input to the process’s ‘stdin`. Also, read (and buffer in memory) the contents of `stdout` and `stderr`. Do this all using `IO::select`, so we don’t deadlock due to full pipe buffers.

This is only really useful if you set some of ‘:stdin`, `:stdout`, and `:stderr` to Subprocess::PIPE.

Parameters:

  • input (String) (defaults to: nil) —

    A string to feed to the child’s standard input.

Returns:

  • (Array<String>) —

    An array of two elements: the data read from the child’s standard output and standard error, respectively.

Raises:

  • (ArgumentError) 


428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
 
# File 'lib/subprocess.rb', line 428

def communicate(input=nil)
  raise ArgumentError if !input.nil? && @stdin.nil?

  stdout, stderr = "", ""
  input = input.dup unless input.nil?

  @stdin.close if (input.nil? || input.empty?) && !@stdin.nil?

  self_read, self_write = IO.pipe
  self.class.catching_sigchld(pid, self_write) do
    wait_r = [@stdout, @stderr, self_read].compact
    wait_w = [input && @stdin].compact
    loop do
      ready_r, ready_w = select(wait_r, wait_w)

      # If the child exits, we still have to be sure to read any data left
      # in the pipes. So we poll the child, drain all the pipes, and *then*
      # check @status.
      #
      # It's very important that we do not call poll between draining the
      # pipes and checking @status. If we did, we open a race condition
      # where the child writes to stdout and exits in that brief window,
      # causing us to lose that data.
      poll

      if ready_r.include?(@stdout)
        if drain_fd(@stdout, stdout)
          wait_r.delete(@stdout)
        end
      end

      if ready_r.include?(@stderr)
        if drain_fd(@stderr, stderr)
          wait_r.delete(@stderr)
        end
      end

      if ready_r.include?(self_read)
        if drain_fd(self_read)
          raise "Unexpected internal error -- someone closed our self-pipe!"
        end
      end

      if ready_w.include?(@stdin)
        begin
          written = @stdin.write_nonblock(input)
        rescue EOFError # Maybe I shouldn't catch this...
        rescue Errno::EINTR
        end
        input[0...written] = ''
        if input.empty?
          @stdin.close
          wait_w.delete(@stdin)
        end
      end

      break if @status

      # If there's nothing left to wait for, we're done!
      break if wait_r.length == 0 && wait_w.length == 0
    end
  end

  wait

  [stdout, stderr]
end

#drain_fd(fd, buf = nil) ⇒ true, false

Do nonblocking reads from ‘fd`, appending all data read into `buf`.

Parameters:

  • fd (IO) —

    The file to read from.

  • buf (String) (defaults to: nil) —

    A buffer to append the read data to.

Returns:

  • (true, false) —

    Whether ‘fd` was closed due to an exceptional condition (`EOFError` or `EPIPE`).



405
406
407
408
409
410
411
412
413
414
415
416
 
# File 'lib/subprocess.rb', line 405

def drain_fd(fd, buf=nil)
  loop do
    tmp = fd.read_nonblock(4096)
    buf << tmp unless buf.nil?
  end
rescue EOFError, Errno::EPIPE
  fd.close
  true
rescue Errno::EINTR
rescue Errno::EWOULDBLOCK, Errno::EAGAIN
  false
end

#poll ⇒ ::Process::Status?

Poll the child, setting (and returning) its status. If the child has not terminated, return nil and exit immediately

Returns:

  • (::Process::Status, nil) —

    The exit status of the process



386
387
388
 
# File 'lib/subprocess.rb', line 386

def poll
  @status ||= (::Process.waitpid2(@pid, ::Process::WNOHANG) || []).last
end

#send_signal(signal) ⇒ Object

Does exactly what it says on the box.

Parameters:

  • signal (String, Symbol, Fixnum) —

    The signal to send to the child process. Accepts all the same arguments as Ruby’s built-in Process::kill, for instance a string like “INT” or “SIGINT”, or a signal number like 2.



502
503
504
 
# File 'lib/subprocess.rb', line 502

def send_signal(signal)
  ::Process.kill(signal, pid)
end

#terminate ⇒ Object

Sends ‘SIGTERM` to the process.



507
508
509
 
# File 'lib/subprocess.rb', line 507

def terminate
  send_signal("TERM")
end

#wait ⇒ ::Process::Status

Wait for the child to return, setting and returning the status of the child.

Returns:

  • (::Process::Status) —

    The exit status of the process



394
395
396
 
# File 'lib/subprocess.rb', line 394

def wait
  @status ||= ::Process.waitpid2(@pid).last
end