Module: Vanagon::Utilities

Extended by:
Utilities
Included in:
Component, Component::Source::Http, Driver, Platform, Project, Utilities
Defined in:
lib/vanagon/utilities.rb,
lib/vanagon/utilities/shell_utilities.rb

Defined Under Namespace

Modules: ShellUtilities

Instance Method Summary collapse

Instance Method Details

#erb_file(erbfile, outfile = nil, remove_orig = false, opts = { :binding => binding }) ⇒ Object

Helper method that takes a template and writes the evaluated contents to a file on disk

Parameters:

  • erbfile (String)
  • outfile (String) (defaults to: nil)
  • remove_orig (true, false) (defaults to: false)
  • opts (Hash) (defaults to: { :binding => binding }) 


279
280
281
282
283
284
285
286
 
# File 'lib/vanagon/utilities.rb', line 279

def erb_file(erbfile, outfile = nil, remove_orig = false, opts = { :binding => binding })
  outfile ||= File.join(Dir.mktmpdir, File.basename(erbfile).sub(File.extname(erbfile), ""))
  output = erb_string(erbfile, opts[:binding])
  File.open(outfile, 'w') { |f| f.write output }
  warn "Generated: #{outfile}"
  FileUtils.rm_rf erbfile if remove_orig
  outfile
end

#erb_string(erbfile, b = binding) ⇒ String

Helper method that takes a template file and runs it through ERB

Parameters:

  • erbfile (String) —

    template to be evaluated

  • b (Binding) (defaults to: binding) —

    binding to evaluate the template under

Returns:

  • (String) —

    the evaluated template



265
266
267
268
269
270
271
 
# File 'lib/vanagon/utilities.rb', line 265

def erb_string(erbfile, b = binding)
  template = File.read(erbfile)
  message  = ERB.new(template, nil, "-")
  message.result(b)
    .gsub(/[\n]+{3,}/, "\n\n")
    .squeeze("\s")
end

#ex(command) ⇒ String

Similar to rake’s sh, the passed command will be executed and an exception will be raised on command failure. However, in contrast to rake’s sh, this method returns the output of the command instead of a boolean.

Parameters:

  • command (String) —

    The command to be executed

Returns:

  • (String) —

    The standard output of the executed command

Raises:



98
99
100
101
102
103
104
 
# File 'lib/vanagon/utilities.rb', line 98

def ex(command)
  ret = %x(#{command})
  unless $CHILD_STATUS.success?
    raise Vanagon::Error, "'#{command}' did not succeed"
  end
  ret
end

#find_program_on_path(command, required = true) ⇒ String, false Also known as: which

Similar to the command-line utility which, the method will search the PATH for the passed command and return the full path to the command if it exists.

Parameters:

  • command (String) —

    Command to search for on PATH

  • required (true, false) (defaults to: true) —

    Whether or not to raise an exception if the command cannot be found

Returns:

  • (String, false) —

    Returns either the full path to the command or false if the command cannot be found

Raises:

  • (RuntimeError) —

    If the command is required and cannot be found



114
115
116
117
118
119
120
121
122
123
124
125
 
# File 'lib/vanagon/utilities.rb', line 114

def find_program_on_path(command, required = true)
  ENV['PATH'].split(File::PATH_SEPARATOR).each do |path_elem|
    location = File.join(path_elem, command)
    return location if FileTest.executable?(location)
  end

  if required
    fail "Could not find '#{command}'. Please install (or ensure it is on $PATH), and try again."
  else
    return false
  end
end

#get_md5sum(file) ⇒ String

Deprecated.

Please use #get_sum instead, this will be removed in a future vanagon release.

Utility to get the md5 sum of a file

Parameters:

  • file (String) —

    file to md5sum

Returns:

  • (String) —

    md5sum of the given file



22
23
24
 
# File 'lib/vanagon/utilities.rb', line 22

def get_md5sum(file)
  get_sum(file, 'md5')
end

#get_sum(file, type = 'md5') ⇒ String

Generic file summing utility

Parameters:

  • file (String) —

    file to sum

  • type (String) (defaults to: 'md5') —

    type of sum to provide, defaults to md5

Returns:

  • (String) —

    sum of the given file

Raises:

  • (RuntimeError) —

    raises an exception if the given sum type is not supported



32
33
34
35
36
37
38
39
 
# File 'lib/vanagon/utilities.rb', line 32

def get_sum(file, type = 'md5')
 Digest.const_get(type.upcase).file(file).hexdigest.to_s

    # If Digest::const_get fails, it'll raise a LoadError when it tries to
    # pull in the subclass `type`. We catch that error, and fail instead.
rescue LoadError
  fail "Don't know how to produce a sum of type: '#{type}' for '#{file}'"
end

#http_request(url, type, payload = {}.to_json, header = nil) ⇒ Hash

Simple wrapper around Net::HTTP. Will make a request of the given type to the given url and return the body as parsed by JSON.

action is not supported, or if there is a problem with the http request, or if the response is not JSON

Parameters:

  • url (String) —

    The url to make the request against (needs to be parsable by URI

  • type (String) —

    One of the supported request types (currently ‘get’, ‘post’, ‘delete’)

  • payload (String) (defaults to: {}.to_json) —

    The request body data payload used for POST and PUT

  • header (Hash) (defaults to: nil) —

    Send additional information in the HTTP request header

Returns:

  • (Hash) —

    The response body is parsed by JSON and returned

Raises:



52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
 
# File 'lib/vanagon/utilities.rb', line 52

def http_request(url, type, payload = {}.to_json, header = nil) # rubocop:disable Metrics/AbcSize
  uri = URI.parse(url)
  http = Net::HTTP.new(uri.host, uri.port)
  http.use_ssl = true if uri.scheme == 'https'

  case type.downcase
  when "get"
    request = Net::HTTP::Get.new(uri.request_uri)
  when "post"
    request = Net::HTTP::Post.new(uri.request_uri)
    request.body = payload
  when "put"
    request = Net::HTTP::Put.new(uri.request_uri)
    request.body = payload
  when "delete"
    request = Net::HTTP::Delete.new(uri.request_uri)
  else
    fail "ACTION: #{type} not supported by #http_request method. Maybe you should add it?"
  end

  # Add any headers to the request
  if header && header.is_a?(Hash)
    header.each do |key, val|
      request[key] = val
    end
  end

  response = http.request(request)

  JSON.parse(response.body)
rescue Errno::ETIMEDOUT, Timeout::Error, Errno::EINVAL, Errno::ECONNRESET,
  EOFError, Net::HTTPBadResponse, Net::HTTPHeaderSyntaxError,
  Net::ProtocolError => e
  raise Vanagon::Error.wrap(e, "Problem reaching #{url}. Is #{uri.host} down?")
rescue JSON::ParserError => e
  raise Vanagon::Error.wrap(e, "#{uri.host} handed us a response that doesn't look like JSON.")
end

#local_command(command, return_command_output: false) ⇒ true, String

Runs the command on the local host

Parameters:

  • command (String) —

    command to run locally

  • return_command_output (Boolean) (defaults to: false) —

    whether or not command output should be returned

Returns:

  • (true, String) —

    Returns true if the command was successful or the output of the command if return_command_output is true

Raises:

  • (RuntimeError) —

    If the command fails an exception is raised



237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
 
# File 'lib/vanagon/utilities.rb', line 237

def local_command(command, return_command_output: false)
  clean_environment do
    warn "Executing '#{command}' locally"
    if return_command_output
      ret = %x(#{command}).chomp
      if $CHILD_STATUS.success?
        return ret
      else
        raise "Local command (#{command}) failed."
      end
    else
      Kernel.system(command)
      $CHILD_STATUS.success? or raise "Local command (#{command}) failed."
    end
  end
end

#remote_ssh_command(target, command, port = 22, return_command_output: false) ⇒ true, String

Runs the command on the given host via ssh call

Parameters:

  • target (String) —

    ssh host to run command on (user@machine)

  • command (String) —

    command to run on the target

  • port (Integer) (defaults to: 22) —

    port number for ssh (default 22)

  • return_command_output (Boolean) (defaults to: false) —

    whether or not command output should be returned

Returns:

  • (true, String) —

    Returns true if the command was successful or the output of the command if return_command_output is true

Raises:

  • (RuntimeError) —

    If there is no target given or the command fails an exception is raised



215
216
217
218
219
220
221
222
223
224
225
226
227
228
 
# File 'lib/vanagon/utilities.rb', line 215

def remote_ssh_command(target, command, port = 22, return_command_output: false)
  warn "Executing '#{command}' on '#{target}'"
  if return_command_output
    ret = %x(#{ssh_command(port)} -T #{target} '#{command.gsub("'", "'\\\\''")}').chomp
    if $CHILD_STATUS.success?
      return ret
    else
      raise "Remote ssh command (#{command}) failed on '#{target}'."
    end
  else
    Kernel.system("#{ssh_command(port)} -T #{target} '#{command.gsub("'", "'\\\\''")}'")
    $CHILD_STATUS.success? or raise "Remote ssh command (#{command}) failed on '#{target}'."
  end
end

#retry_with_timeout(tries = 5, timeout = 1, &blk) ⇒ true

Method to retry a ruby block and fail if the command does not succeed within the number of tries and timeout.

Parameters:

  • tries (Integer) (defaults to: 5) —

    number of times to try calling the block

  • timeout (Integer) (defaults to: 1) —

    number of seconds to run the block before timing out

Returns:

  • (true) —

    If the block succeeds, true is returned

Raises:

  • (Vanagon::Error) —

    if the block fails after the retries are exhausted, an error is raised



135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
 
# File 'lib/vanagon/utilities.rb', line 135

def retry_with_timeout(tries = 5, timeout = 1, &blk) # rubocop:disable Metrics/AbcSize
  error = nil
  tries.to_i.times do
    Timeout::timeout(timeout.to_i) do
      begin
        yield
        return true
      rescue StandardError => e
        warn 'An error was encountered evaluating block. Retrying..'
        error = e
      end
    end
  end

  message = "Block failed maximum number of #{tries} tries"
  unless error.nil?
    message += "\n with error #{error.message}" + "\n#{error.backtrace.join("\n")}"
  end
  message += "\nExiting..."
  raise error, message unless error.nil?
  raise Vanagon::Error, "Block failed maximum number of #{tries} tries"
end

#rsync_from(source, target, dest, port = 22, extra_flags = []) ⇒ String

Retrieves the desired file/directory from the destination using rsync

Parameters:

  • source (String) —

    path on target to retrieve from

  • target (String) —

    ssh host to retrieve from (user@machine)

  • dest (String) —

    path on local host to place the source

  • port (Integer) (defaults to: 22) —

    port number for ssh (default 22)

  • extra_flags (Array) (defaults to: []) —

    any additional flags to pass to rsync

Returns:

  • (String) —

    output of rsync command



197
198
199
200
201
202
203
204
 
# File 'lib/vanagon/utilities.rb', line 197

def rsync_from(source, target, dest, port = 22, extra_flags = [])
  rsync = find_program_on_path('rsync')
  flags = "-rHlv -O --no-perms --no-owner --no-group"
  unless extra_flags.empty?
    flags << " " << extra_flags.join(" ")
  end
  ex("#{rsync} -e '#{ssh_command(port)}' #{flags} #{target}:#{source} #{dest}")
end

#rsync_to(source, target, dest, port = 22, extra_flags = ["--ignore-existing"]) ⇒ String

Sends the desired file/directory to the destination using rsync

Parameters:

  • source (String) —

    file or directory to send

  • target (String) —

    ssh host to send to (user@machine)

  • dest (String) —

    path on target to place the source

  • extra_flags (Array) (defaults to: ["--ignore-existing"]) —

    any additional flags to pass to rsync

  • port (Integer) (defaults to: 22) —

    Port number for ssh (default 22)

Returns:

  • (String) —

    output of rsync command



166
167
168
169
170
171
172
173
 
# File 'lib/vanagon/utilities.rb', line 166

def rsync_to(source, target, dest, port = 22, extra_flags = ["--ignore-existing"])
  rsync = find_program_on_path('rsync')
  flags = "-rHlv --no-perms --no-owner --no-group"
  unless extra_flags.empty?
    flags << " " << extra_flags.join(" ")
  end
  ex("#{rsync} -e '#{ssh_command(port)}' #{flags} #{source} #{target}:#{dest}")
end

#ssh_command(port = 22) ⇒ String

Hacky wrapper to add on the correct flags for ssh to be used in ssh and rsync methods

Parameters:

  • port (Integer) (defaults to: 22) —

    Port number for ssh (default 22)

Returns:

  • (String) —

    start of ssh command, including flags for ssh keys



179
180
181
182
183
184
185
186
187
 
# File 'lib/vanagon/utilities.rb', line 179

def ssh_command(port = 22)
  ssh = find_program_on_path('ssh')
  args = ENV['VANAGON_SSH_KEY'] ? " -i #{ENV['VANAGON_SSH_KEY']}" : ""
  args << " -p #{port} "
  args << " -o UserKnownHostsFile=/dev/null"
  args << " -o StrictHostKeyChecking=no"
  args << " -o ForwardAgent=yes" if ENV['VANAGON_SSH_AGENT']
  return ssh + args
end