Module: Cmds

Defined in:
lib/cmds/debug.rb,
lib/cmds/cmd.rb,
lib/cmds/pipe.rb,
lib/cmds/util.rb,
lib/cmds/spawn.rb,
lib/cmds/sugar.rb,
lib/cmds/result.rb,
lib/cmds/version.rb,
lib/cmds/io_handler.rb,
lib/cmds/erb_context.rb,
lib/cmds/shell_eruby.rb,
lib/cmds/util/params.rb,
lib/cmds/util/defaults.rb,
lib/cmds/util/tokenize_option.rb,
lib/cmds/util/tokenize_options.rb

Overview

debug logging stuff

Defined Under Namespace

Modules: Debug Classes: Cmd, ERBContext, IOHandler, Params, Pipe, Result, ShellEruby

Constant Summary collapse

VERSION = 
"0.1.5"
DEFAULTS =

hash of common default values used in method options.

don't use them directly -- use defaults.

the values themselves are frozen so we don't have to worry about cloning them before providing them for use.

the constant Hash itself is not frozen -- you can mutate this to change the default options for ALL Cmds method calls... just be aware of what you're doing. not recommended outside of quick hacks and small scripts since other pieces and parts you don't even know about may depend on said behavior.

{
  # positional arguments for a command
  args: [],
  
  # keyword arguments for a command
  kwds: {},
  
  # how to format a command string for execution
  format: :squish,
  
  # what to do with array option values
  array_mode: :join,
  
  # what to join array option values with when using `array_mode = :join`
  array_join_string: ',',
  
  # what to do with false array values
  false_mode: :omit,
}.map {|k, v| [k, v.freeze]}.to_h

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.assert(template, *args, **kwds, &io_block) ⇒ Object

create a new Cmd and



86
87
88
 
# File 'lib/cmds/sugar.rb', line 86

def self.assert template, *args, **kwds, &io_block
  Cmd.new(template).capture(*args, **kwds, &io_block).assert
end

.capture(template, *args, **kwds, &input_block) ⇒ Result

create a new Cmd from template with parameters and call capture on it.



59
60
61
 
# File 'lib/cmds/sugar.rb', line 59

def self.capture template, *args, **kwds, &input_block
  Cmd.new(template).capture *args, **kwds, &input_block
end

.chomp(template, *args, **kwds, &input_block) ⇒ String

captures a new Cmd, captures and chomps stdout (sugar for Cmds.out(template, *args, **kwds, &input_block).chomp).

See Also:



153
154
155
 
# File 'lib/cmds/sugar.rb', line 153

def self.chomp template, *args, **kwds, &input_block
  out(template, *args, **kwds, &input_block).chomp
end

.chomp!(template, *args, **kwds, &input_block) ⇒ String

captures and chomps stdout, raising an error if the command fails. (sugar for Cmds.out!(template, *args, **kwds, &input_block).chomp).

Raises:

  • (SystemCallError) —

    if the command fails (non-zero exit status).

See Also:



174
175
176
 
# File 'lib/cmds/sugar.rb', line 174

def self.chomp! template, *args, **kwds, &input_block
  out!(template, *args, **kwds, &input_block).chomp
end

.debug(msg, values = {}) ⇒ Object

log a debug message along with an optional hash of values.



96
97
98
99
100
 
# File 'lib/cmds/debug.rb', line 96

def self.debug msg, values = {}
  # don't even bother unless debug logging is turned on
  return unless Debug.on?
  Debug.logger.debug Debug.format(msg, values)
end

.defaults(opts, keys = '*', extras = {}) ⇒ Hash<Symbol, Object>

merge an method call options hash with common defaults for the module.

this makes it easy to use the same defaults in many different methods without repeating the declarations everywhere.



52
53
54
55
56
57
58
59
60
61
62
63
64
 
# File 'lib/cmds/util/defaults.rb', line 52

def self.defaults opts, keys = '*', extras = {}
  if keys == '*'
    DEFAULTS
  else      
    keys.
      map {|key|
        [key, DEFAULTS.fetch(key)]
      }.
      to_h
  end.
    merge!(extras).
    merge!(opts)
end

.err(template, *args, **kwds, &input_block) ⇒ String

captures and returns stderr (sugar for Cmds.capture(template, *args, **kwds, &input_block).err).

See Also:



192
193
194
 
# File 'lib/cmds/sugar.rb', line 192

def self.err template, *args, **kwds, &input_block
  capture(template, *args, **kwds, &input_block).err
end

.error?(template, *args, **kwds, &io_block) ⇒ Boolean 



80
81
82
 
# File 'lib/cmds/sugar.rb', line 80

def self.error? template, *args, **kwds, &io_block
  Cmd.new(template).error? *args, **kwds, &io_block
end

.esc(str) ⇒ Object

shortcut for Shellwords.escape

also makes it easier to change or customize or whatever



15
16
17
 
# File 'lib/cmds/util.rb', line 15

def self.esc str
  Shellwords.escape str
end

.expand_sub(sub) ⇒ Object

expand one of the substitutions



20
21
22
23
24
25
26
27
28
29
30
 
# File 'lib/cmds/util.rb', line 20

def self.expand_sub sub
  case sub
  when nil
    # nil is just an empty string, NOT an empty string bash token
    ''
  when Hash
    tokenize_options sub
  else
    esc sub.to_s
  end
end

.format(string, with = :squish) ⇒ Object

formats a command string



33
34
35
36
37
38
39
40
41
42
43
44
 
# File 'lib/cmds/util.rb', line 33

def self.format string, with = :squish
  case with
  when :squish
    NRSER.squish string
    
  when :pretty
    pretty_format string
  
  else
    with.call string
  end
end

.ok?(template, *args, **kwds, &io_block) ⇒ Result

create a new Cmd from template with parameters and call Cmds::Cmd#ok? on it.



75
76
77
 
# File 'lib/cmds/sugar.rb', line 75

def self.ok? template, *args, **kwds, &io_block
  Cmd.new(template).ok? *args, **kwds, &io_block
end

.out(template, *args, **kwds, &input_block) ⇒ String

creates a new Cmd, captures and returns stdout (sugar for Cmds.capture(template, *args, **kwds, &input_block).out).

See Also:



114
115
116
 
# File 'lib/cmds/sugar.rb', line 114

def self.out template, *args, **kwds, &input_block
  Cmd.new(template).out *args, **kwds, &input_block
end

.out!(template, *args, **kwds, &input_block) ⇒ String

creates a new Cmd, captures and returns stdout. raises an error if the command fails.

Raises:

  • (SystemCallError) —

    if the command fails (non-zero exit status).

See Also:



135
136
137
 
# File 'lib/cmds/sugar.rb', line 135

def self.out! template, *args, **kwds, &input_block
  Cmd.new(template).out! *args, **kwds, &input_block
end

.prepare(template, *args, **kwds) ⇒ String

create a new Cmd instance with the template and parameters and calls Cmds::Cmd#prepare.



40
41
42
 
# File 'lib/cmds/sugar.rb', line 40

def self.prepare template, *args, **kwds
  Cmd.new(template).prepare *args, **kwds
end

.pretty_format(string) ⇒ Object 



46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
 
# File 'lib/cmds/util.rb', line 46

def self.pretty_format string
  string = string.gsub(/\n(\s*\n)+\n/, "\n\n")
  
  string.lines.map {|line|
    line = line.rstrip
    
    if line.end_with? '\\'
      line
    elsif line == ''
      '\\'
    elsif line =~ /\s$/
      line + '\\'
    else
      line + ' \\'
    end
  }.join("\n")
end

.replace_shortcuts(template) ⇒ Object 



64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
 
# File 'lib/cmds/util.rb', line 64

def self.replace_shortcuts template
  template
    .gsub(
      # %s => <%= arg %>
      /(?<=\A|\=|[[:space:]])\%s(?=\Z|[[:space:]])/,
      '<%= arg %>'
    )
    .gsub(
      # %%s => %s (escaping)
      /(?<=\A|[[:space:]])(\%+)\%s(?=\Z|[[:space:]])/,
      '\1s'
    )
    .gsub(
      # %{key} => <%= key %>, %{key?} => <%= key? %>
      /(?<=\A|\=|[[:space:]])\%\{([a-zA-Z_]+\??)\}(?=\Z|[[:space:]])/,
      '<%= \1 %>'
    )
    .gsub(
      # %%{key} => %{key}, %%{key?} => %{key?} (escaping)
      /(?<=\A|[[:space:]])(\%+)\%\{([a-zA-Z_]+\??)\}(?=\Z|[[:space:]])/,
      '\1{\2}\3'
    )
    .gsub(
      # %<key>s => <%= key %>, %<key?>s => <%= key? %>
      /(?<=\A|\=|[[:space:]])\%\<([a-zA-Z_]+\??)\>s(?=\Z|[[:space:]])/,
      '<%= \1 %>'
    )
    .gsub(
      # %%<key>s => %<key>s, %%<key?>s => %<key?>s (escaping)
      /(?<=\A|[[:space:]])(\%+)\%\<([a-zA-Z_]+\??)\>s(?=\Z|[[:space:]])/,
      '\1<\2>s'
    )
end

.spawn(cmd, **opts, &io_block) ⇒ Object

internal core function to spawn and stream inputs and/or outputs using threads.

originally inspired by

https://nickcharlton.net/posts/ruby-subprocesses-with-stdout-stderr-streams.html

with major modifications from looking at Ruby's open3 module.

Raises:

  • (ArgumentError) —

    if &io_block has arity greater than 1.



48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
 
# File 'lib/cmds/spawn.rb', line 48

def self.spawn cmd, **opts, &io_block
  Cmds.debug "entering Cmds#spawn",
    cmd: cmd,
    opts: opts,
    io_block: io_block
  
  env = opts[:env] || {}
  input = opts[:input]
  chdir = opts[:chdir]

  # create the handler that will be yielded to the input block
  handler = Cmds::IOHandler.new

  # handle input
  # 
  # if a block was provided it overrides the `input` argument.
  # 
  if io_block
    case io_block.arity
    when 0
      # when the input block takes no arguments it returns the input
      input = io_block.call
    when 1
      # when the input block takes one argument, give it the handler and
      # ignore the return value
      io_block.call handler

      # if input was assigned to the handler in the block, use it as input
      input = handler.in unless handler.in.nil?
    else
      # bad block provided
      raise ArgumentError.new "        provided input block must have arity 0 or 1\n      BLOCK\n    end # case io_block.arity\n  end # if io_block\n\n  # hash of options that will be passed to `spawn`\n  spawn_opts = {}\n  \n  # add chdir if provided\n  spawn_opts[:chdir] = chdir if chdir\n\n  Cmds.debug \"looking at input...\",\n    input: input\n\n  # (possibly) create the input pipe... this will be nil if the provided\n  # input is io-like. in this case it will be used directly in the \n  # `spawn` options.\n  in_pipe = case input\n  when nil, String\n    Cmds.debug \"input is a String or nil, creating pipe...\"\n\n    in_pipe = Cmds::Pipe.new \"INPUT\", :in\n    spawn_opts[:in] = in_pipe.r\n\n    # don't buffer input\n    in_pipe.w.sync = true\n    in_pipe\n\n  else\n    Cmds.debug \"input should be io-like, setting spawn opt.\",\n      input: input\n    if input == $stdin\n      Cmds.debug \"input is $stdin.\"\n    end\n    spawn_opts[:in] = input\n    nil\n\n  end # case input\n\n  # (possibly) create the output pipes.\n  # \n  # `stream` can be told to send it's output to either:\n  # \n  # 1.  a Proc that will invoked with each line.\n  # 2.  an io-like object that can be provided as `spawn`'s `:out` or \n  #     `:err` options.\n  # \n  # in case (1) a `Cmds::Pipe` wrapping read and write piped `IO` instances\n  # will be created and assigned to the relevant of `out_pipe` or `err_pipe`.\n  # \n  # in case (2) the io-like object will be sent directly to `spawn` and\n  # the relevant `out_pipe` or `err_pipe` will be `nil`.\n  #\n  out_pipe, err_pipe = [\n    [\"ERROR\", :err],\n    [\"OUTPUT\", :out],\n  ].map do |name, sym|\n    Cmds.debug \"looking at \#{ name }...\"\n    # see if hanlder.out or hanlder.err is a Proc\n    if handler.send(sym).is_a? Proc\n      Cmds.debug \"\#{ name } is a Proc, creating pipe...\"\n      pipe = Cmds::Pipe.new name, sym\n      # the corresponding :out or :err option for spawn needs to be\n      # the pipe's write handle\n      spawn_opts[sym] = pipe.w\n      # return the pipe\n      pipe\n\n    else\n      Cmds.debug \"\#{ name } should be io-like, setting spawn opt.\",\n        output: handler.send(sym)\n      spawn_opts[sym] = handler.send(sym)\n      # the pipe is nil!\n      nil\n    end\n  end # map outputs\n\n  Cmds.debug \"spawning...\",\n    env: env,\n    cmd: cmd,\n    opts: spawn_opts\n\n  pid = Process.spawn env.map {|k, v| [k.to_s, v]}.to_h,\n                      cmd,\n                      spawn_opts\n\n  Cmds.debug \"spawned.\",\n    pid: pid\n\n  wait_thread = Process.detach pid\n  wait_thread[:name] = \"WAIT\"\n\n  Cmds.debug \"wait thread created.\",\n    thread: wait_thread\n\n  # close child ios if created\n  # the spawned process will read from in_pipe.r so we don't need it\n  in_pipe.r.close if in_pipe\n  # and we don't need to write to the output pipes, that will also happen\n  # in the spawned process\n  [out_pipe, err_pipe].each {|pipe| pipe.w.close if pipe}\n\n  # create threads to handle any pipes that were created\n\n  in_thread = if in_pipe\n    Thread.new do\n      Thread.current[:name] = in_pipe.name\n      Cmds.debug \"thread started, writing input...\"\n\n      in_pipe.w.write input unless input.nil?\n\n      Cmds.debug \"write done, closing in_pipe.w...\"\n      in_pipe.w.close\n\n      Cmds.debug \"thread done.\"\n    end # Thread\n  end\n\n  out_thread, err_thread = [out_pipe, err_pipe].map do |pipe|\n    if pipe\n      Thread.new do\n        Thread.current[:name] = pipe.name\n        Cmds.debug \"thread started\"\n\n        loop do\n          Cmds.debug \"blocking on gets...\"\n          line = pipe.r.gets\n          if line.nil?\n            Cmds.debug \"received nil, output done.\"\n          else\n            Cmds.debug <<-BLOCK.squish\n              received \#{ line.bytesize } bytes, passing to handler.\n            BLOCK\n          end\n          handler.thread_send_line pipe.sym, line\n          break if line.nil?\n        end\n\n        Cmds.debug \"reading done, closing pipe.r (unless already closed)...\"\n        pipe.r.close unless pipe.r.closed?\n\n        Cmds.debug \"thread done.\"\n      end # thread\n    end # if pipe\n  end # map threads\n\n  Cmds.debug \"handing off main thread control to the handler...\"\n  begin\n    handler.start\n\n    Cmds.debug \"handler done.\"\n\n  ensure\n    # wait for the threads to complete\n    Cmds.debug \"joining threads...\"\n\n    [in_thread, out_thread, err_thread, wait_thread].each do |thread|\n      if thread\n        Cmds.debug \"joining \#{ thread[:name] } thread...\"\n        thread.join\n      end\n    end\n\n    Cmds.debug \"all threads done.\"\n  end\n\n  status = wait_thread.value.exitstatus\n  Cmds.debug \"exit status: \#{ status.inspect }\"\n\n  Cmds.debug \"checking @assert and exit status...\"\n  if @assert && status != 0\n    # we don't necessarily have the err output, so we can't include it\n    # in the error message\n    msg = <<-BLOCK.squish\n      streamed command `\#{ cmd }` exited with status \#{ status }\n    BLOCK\n\n    raise SystemCallError.new msg, status\n  end\n\n  Cmds.debug \"streaming completed.\"\n\n  return status\nend\n".squish

.stream(template, *subs, &input_block) ⇒ Object 



91
92
93
 
# File 'lib/cmds/sugar.rb', line 91

def self.stream template, *subs, &input_block
  Cmds::Cmd.new(template).stream *subs, &input_block
end

.stream!(template, *subs, &input_block) ⇒ Object 



96
97
98
 
# File 'lib/cmds/sugar.rb', line 96

def self.stream! template, *subs, &input_block
  Cmds::Cmd.new(template, assert: true).stream *subs, &input_block
end

.tokenize_option(name, value, opts = {}) ⇒ Array<String>

turn an option name and value into an array of shell-escaped string token suitable for use in a command.



38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/cmds/util/tokenize_option.rb', line 38

def self.tokenize_option name, value, opts = {}
  opts = defaults opts, [:array_mode, :array_join_string, :false_mode]
  
  unless name.is_a?(String) && name.length > 0
    raise ArgumentError.new "      `name` must be a String of length greater than zero,\n      found \#{ name.inspect }\n    END\n  end\n  \n  prefix, separator = if name.length == 1\n    # -b <value> style\n    ['-', ' ']\n  else\n    # --blah=<value> style\n    ['--', '=']\n  end\n    \n  case value\n  when nil\n    []\n    \n  when Array\n    # the PITA one\n    case opts[:array_mode]\n    when :repeat\n      # `-b 1 -b 2 -b 3` / `--blah=1 --blah=2 --blah=3` style\n      value.flatten.map {|v|\n        prefix + esc(name) + separator + esc(v)\n      }\n      \n    when :join\n      # `-b 1,2,3` / `--blah=1,2,3` style\n      [ prefix + \n        esc(name) + \n        separator + \n        esc(value.join opts[:array_join_string]) ]\n      \n    when :json\n      [prefix + esc(name) + separator + esc(JSON.dump value)]\n      \n    else\n      # SOL\n      raise ArgumentError.new <<-END.squish\n        bad array_mode option: \#{ opts[:array_mode] }, \n        should be :repeat, :join or :json\n      END\n      \n    end\n    \n  when true\n    # `-b` or `--blah`\n    [prefix + esc(name)]\n    \n  when false\n    case opts[:false_mode]\n    when :omit\n      # don't emit any token for a false boolean\n      []\n    when :no\n      # `--no-blah` style\n      # \n      # but there's not really a great way to handle short names...\n      # we use `--no-b`\n      # \n      [\"--no-\#{ esc(name) }\"]\n      \n    else\n      raise ArgumentError.new <<-END.squish\n        bad :false_mode option: \#{ opts[:false_mode] }, \n        should be :omit or :no\n      END\n    end\n    \n  else\n    # we let .esc handle it\n    [prefix + esc(name) + separator + esc(value)]\n    \n  end\nend\n".squish

.tokenize_options(hash, opts = {}) ⇒ Object

escape option hash.

this is only useful for the two common option styles:

  • single character keys become -<char> <value>

    {x: 1}    => "-x 1"

  • longer keys become --<key>=<value> options

    {blah: 2} => "--blah=2"

if you have something else, you're going to have to just put it in the cmd itself, like:

Cmds "blah -assholeOptionOn:%{s}", "ok"

or whatever similar shit said command requires.

however, if the value is an Array, it will repeat the option for each value:

{x:     [1, 2, 3]} => "-x 1 -x 2 -x 3"
{blah:  [1, 2, 3]} => "--blah=1 --blah=2 --blah=3"

i can't think of any right now, but i swear i've seen commands that take opts that way.



32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
 
# File 'lib/cmds/util/tokenize_options.rb', line 32

def self.tokenize_options hash, opts = {}
  opts = defaults opts, [:array_mode, :array_join_string, :false_mode]
  
  hash.map {|key, value|
    # keys need to be strings
    key = key.to_s unless key.is_a? String

    [key, value]

  }.sort {|(key_a, value_a), (key_b, value_b)|
    # sort by the (now string) keys
    key_a <=> key_b

  }.map {|key, value|
    tokenize_option key, value
    
  }.flatten.join ' '
end

Instance Method Details

#defaults(opts, keys = '*', extras = {}) ⇒ Object

proxy through to class method defaults.



68
69
70
 
# File 'lib/cmds/util/defaults.rb', line 68

def defaults opts, keys = '*', extras = {}
  self.class.defaults opts, keys, extras
end