Class: Beaker::Options::Parser

Inherits:
Object
  • Object
show all
Defined in:
lib/beaker/options/parser.rb

Overview

An Object that parses, merges and normalizes all supported Beaker options and arguments

Constant Summary collapse

GITREPO = 
'git://github.com/puppetlabs'
LONG_OPTS =

These options can have the form of arg1,arg2 or [arg] or just arg, should default to []

[:helper, :load_path, :tests, :pre_suite, :post_suite, :install, :modules]
RB_FILE_OPTS =

These options expand out into an array of .rb files

[:tests, :pre_suite, :post_suite]
PARSE_ERROR = 
Psych::SyntaxError

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Parser

Constructor for Parser



162
163
164
165
 
# File 'lib/beaker/options/parser.rb', line 162

def initialize
   @command_line_parser = Beaker::Options::CommandLineParser.new
   @presets = Beaker::Options::Presets.new
end

Instance Attribute Details

#options ⇒ Object

The OptionsHash of all parsed options



17
18
19
 
# File 'lib/beaker/options/parser.rb', line 17

def options
  @options
end

Instance Method Details

#check_yaml_file(f, msg = "") ⇒ Object

Determine is a given file exists and is a valid YAML file

Parameters:

  • f (String) —

    The YAML file path to examine

  • msg (String) (defaults to: "") —

    An options message to report in case of error

Raises:

  • (ArgumentError) —

    Raise if file does not exist or is not valid YAML



221
222
223
224
225
226
227
228
229
230
 
# File 'lib/beaker/options/parser.rb', line 221

def check_yaml_file(f, msg = "")
  if not File.file?(f)
    parser_error "#{f} does not exist (#{msg})"
  end
  begin
    YAML.load_file(f)
  rescue PARSE_ERROR => e
    parser_error "#{f} is not a valid YAML file (#{msg})\n\t#{e}"
  end
end

#file_list(paths) ⇒ Array

Generates a list of files based upon a given path or list of paths.

Looks recursively for .rb files in paths.

Parameters:

  • paths (Array) —

    Array of file paths to search for .rb files

Returns:

  • (Array) —

    An Array of fully qualified paths to .rb files

Raises:

  • (ArgumentError) —

    Raises if no .rb files are found in searched directory or if no .rb files are found overall



68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/beaker/options/parser.rb', line 68

def file_list(paths)
  files = []
  if not paths.empty?
    paths.each do |root|
      if File.file?(root)
        files << root
      elsif File.directory?(root) #expand and explore
        discover_files = Dir.glob(
          File.join(root, "**/*.rb")
        ).select { |f| File.file?(f) }
        if discover_files.empty?
          parser_error "empty directory used as an option (#{root})!"
        end
        files += discover_files.sort_by {|file| [file.count("/"), file]}
      else #not a file, not a directory, not nothin'
        parser_error "#{root} used as a file option but is not a file or directory!"
      end
    end
  end
  if files.empty?
    parser_error "no .rb files found in #{paths.to_s}"
  end
  files
end

#normalize_and_validate_tags ⇒ Object

Takes tag arguments given at this point, which are strings, converts them into the proper format (an array of strings) for Beaker use, and lastly, validates them.

Returns:

  • nil



352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
 
# File 'lib/beaker/options/parser.rb', line 352

def normalize_and_validate_tags()
  @options[:tag_includes] ||= ''
  @options[:tag_excludes] ||= ''
  @options[:tag_includes] = @options[:tag_includes].split(',') if @options[:tag_includes].respond_to?(:split)
  @options[:tag_excludes] = @options[:tag_excludes].split(',') if @options[:tag_excludes].respond_to?(:split)
  @options[:tag_includes].map!(&:downcase)
  @options[:tag_excludes].map!(&:downcase)

  #check that a tag is not both included & excluded sets
  @options[:tag_includes].each do |included_tag|
    @options[:tag_excludes].each do |excluded_tag|
      parser_error "tag '#{included_tag}' cannot be in both the included and excluded tag sets" \
        if included_tag == excluded_tag
    end
  end
end

#normalize_args ⇒ Object

Validate all merged options values for correctness

Currently checks:

- each host has a valid platform
- if a keyfile is provided then use it
- paths provided to --test, --pre-suite, --post-suite provided lists of .rb files for testing
- --fail-mode is one of 'fast', 'stop' or nil
- if using blimpy hypervisor an EC2 YAML file exists
- if using the aix, solaris, or vcloud hypervisors a .fog file exists
- that one and only one master is defined per set of hosts
- that solaris/windows/aix hosts are agent only for PE tests OR
- sets the default host based upon machine definitions
- if an ssh user has been defined make it the host user

Raises:

  • (ArgumentError) —

    Raise if argument/options values are invalid



247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
 
# File 'lib/beaker/options/parser.rb', line 247

def normalize_args

  @options['HOSTS'].each_key do |name|
    if not @options['HOSTS'][name]['platform']
      parser_error "Host #{name} does not have a platform specified"
    else
      @options['HOSTS'][name]['platform'] = Platform.new(@options['HOSTS'][name]['platform'])
    end
  end

  #use the keyfile if present
  if @options.has_key?(:keyfile)
    @options[:ssh][:keys] = [@options[:keyfile]]
  end

  #split out arguments - these arguments can have the form of arg1,arg2 or [arg] or just arg
  #will end up being normalized into an array
  LONG_OPTS.each do |opt|
    if @options.has_key?(opt)
      @options[opt] = split_arg(@options[opt])
      if RB_FILE_OPTS.include?(opt) && (not @options[opt] == [])
        @options[opt] = file_list(@options[opt])
      end
      if opt == :install
        @options[:install] = parse_git_repos(@options[:install])
      end
    else
      @options[opt] = []
    end
  end

  #check for valid fail mode
  if @options[:fail_mode] !~ /stop|fast|slow/
    parser_error "--fail-mode must be one of fast or slow, not '#{@options[:fail_mode]}'"
  end

  #check for valid preserve_hosts option
  if @options[:preserve_hosts] !~ /always|onfail|onpass|never/
    parser_error "--preserve_hosts must be one of always, onfail, onpass or never, not '#{@options[:preserve_hosts]}'"
  end

  #check for config files necessary for different hypervisors
  hypervisors = []
  @options[:HOSTS].each_key do |name|
    hypervisors << @options[:HOSTS][name][:hypervisor].to_s
  end
  hypervisors.uniq!
  hypervisors.each do |visor|
    if ['blimpy'].include?(visor)
      check_yaml_file(@options[:ec2_yaml], "required by #{visor}")
    end
    if ['aix', 'solaris', 'vcloud'].include?(visor)
      check_yaml_file(@options[:dot_fog], "required by #{visor}")
    end
  end

  #check that roles of hosts make sense
  # - must be one and only one master
  roles = []
  @options[:HOSTS].each_key do |name|
    roles << @options[:HOSTS][name][:roles]
  end
  master = 0
  roles.each do |role_array|
    if role_array.include?('master')
      master += 1
    end
    if role_array.include?('frictionless') and !(role_array & ['master', 'database', 'dashboard', 'console']).empty?
      parser_error "Only agent nodes may have the role 'frictionless', fix #{@options[:hosts_file]}"
    end
  end
  if master > 1
    parser_error "Only one host/node may have the role 'master', fix #{@options[:hosts_file]}"
  end

  #check that windows/el-4 boxes are only agents (solaris can be a master in foss cases)
  @options[:HOSTS].each_key do |name|
    host = @options[:HOSTS][name]
    if host[:platform] =~ /windows|el-4/
      test_host_roles(name, host)
    end

    #check to see if a custom user account has been provided, if so use it
    if host[:ssh] && host[:ssh][:user]
      host[:user] = host[:ssh][:user]
    end

    # merge host tags for this host with the global/preset host tags
    host[:host_tags] = @options[:host_tags].merge(host[:host_tags] || {})
  end

  normalize_and_validate_tags()
  resolve_symlinks()

  #set the default role
  set_default_host!(@options[:HOSTS])

end

#parse_args(args = ARGV) ⇒ Object

Parses ARGV or provided arguments array, file options, hosts options and combines with environment variables and preset defaults to generate a Hash representing the Beaker options for a given test run

Order of priority is as follows:

1.  environment variables are given top priority
2.  ARGV or provided arguments array
3.  the 'CONFIG' section of the hosts file
4.  options file values
5.  default or preset values are given the lowest priority

Parameters:

  • args (Array) (defaults to: ARGV) —

    ARGV or a provided arguments array

Raises:

  • (ArgumentError) —

    Raises error on bad input



179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
 
# File 'lib/beaker/options/parser.rb', line 179

def parse_args(args = ARGV)
  @options = @presets.presets
  cmd_line_options = @command_line_parser.parse(args)
  cmd_line_options[:command_line] = ([$0] + args).join(' ')
  file_options = Beaker::Options::OptionsFileParser.parse_options_file(cmd_line_options[:options_file])

  # merge together command line and file_options
  #   overwrite file options with command line options
  cmd_line_and_file_options = file_options.merge(cmd_line_options)

  # merge command line and file options with defaults
  #   overwrite defaults with command line and file options
  @options = @options.merge(cmd_line_and_file_options)

  if not @options[:help] and not @options[:beaker_version_print]
    #read the hosts file that contains the node configuration and hypervisor info
    hosts_options = Beaker::Options::HostsFileParser.parse_hosts_file(@options[:hosts_file])

    # merge in host file vars
    #   overwrite options (default, file options, command line) with host file options
    @options = @options.merge(hosts_options)

    # re-merge the command line options
    #   overwrite options (default, file options, hosts file ) with command line arguments
    @options = @options.merge(cmd_line_options)

    # merge in env vars
    #   overwrite options (default, file options, command line, hosts file) with env
    env_vars = @presets.env_vars

    @options = @options.merge(env_vars)

    normalize_args
  end

  @options
end

#parse_git_repos(git_opts) ⇒ Array

Converts array of paths into array of fully qualified git repo URLS with expanded keywords

Supports the following keywords

PUPPET
FACTER
HIERA
HIERA-PUPPET

Examples:

opts = ["PUPPET/3.1"]
parse_git_repos(opts) == ["#{GITREPO}/puppet.git#3.1"]

Parameters:

  • git_opts (Array) —

    An array of paths

Returns:

  • (Array) —

    An array of fully qualified git repo URLs with expanded keywords



105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
 
# File 'lib/beaker/options/parser.rb', line 105

def parse_git_repos(git_opts)
  git_opts.map! { |opt|
    case opt
      when /^PUPPET\//
        opt = "#{GITREPO}/puppet.git##{opt.split('/', 2)[1]}"
      when /^FACTER\//
        opt = "#{GITREPO}/facter.git##{opt.split('/', 2)[1]}"
      when /^HIERA\//
        opt = "#{GITREPO}/hiera.git##{opt.split('/', 2)[1]}"
      when /^HIERA-PUPPET\//
        opt = "#{GITREPO}/hiera-puppet.git##{opt.split('/', 2)[1]}"
    end
    opt
  }
  git_opts
end

#parser_error(msg = "") ⇒ Object

Raises an ArgumentError with associated message

Parameters:

  • msg (String) (defaults to: "") —

    The error message to be reported

Raises:

  • (ArgumentError) —

    Takes the supplied message and raises it as an ArgumentError



22
23
24
 
# File 'lib/beaker/options/parser.rb', line 22

def parser_error msg = ""
  raise ArgumentError, msg.to_s
end

#repo ⇒ String

Returns the git repository used for git installations

Returns:

  • (String) —

    The git repository



28
29
30
 
# File 'lib/beaker/options/parser.rb', line 28

def repo
  GITREPO
end

#resolve_symlinks ⇒ Object

Note:

doing it here allows us to not need duplicate logic, which we would need if we were doing it in the parser (–hosts & –config)

resolves all file symlinks that require it.

Returns:

  • nil



376
377
378
 
# File 'lib/beaker/options/parser.rb', line 376

def resolve_symlinks()
  @options[:hosts_file] = File.realpath(@options[:hosts_file]) if @options[:hosts_file]
end

#set_default_host!(hosts) ⇒ Object

Add the ‘default’ role to the host determined to be the default. If a host already has the role default then do nothing. If more than a single host has the role ‘default’, raise error. Default host determined to be 1) the only host in a single host configuration, 2) the host with the role ‘master’ defined.

Parameters:

  • hosts (Hash) —

    A hash of hosts, each identified by a String name. Each named host will have an Array of roles



127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
 
# File 'lib/beaker/options/parser.rb', line 127

def set_default_host!(hosts)
  default = []
  master = []
  default_host_name = nil

  #look through the hosts and find any hosts with role 'default' and any hosts with role 'master'
  hosts.each_key do |name|
    host = hosts[name]
    if host[:roles].include?('default')
      default << name
    elsif host[:roles].include?('master')
      master << name
    end
  end

  if not default.empty?
    #we already have a default set, do nothing
    if default.length > 1
      parser_error "Only one host may have the role 'default', default roles assigned to #{default}"
    end
  else
    #no default set, let's make one
    if not master.empty? and master.length == 1
      default_host_name = master[0]
    elsif hosts.length == 1
      default_host_name = hosts.keys[0]
    end
    if default_host_name
      hosts[default_host_name][:roles] << 'default'
    end
  end
end

#split_arg(arg) ⇒ Array

Normalizes argument into an Array. Argument can either be converted into an array of a single value, or can become an array of multiple values by splitting arg over ‘,’. If argument is already an array that array is returned untouched.

Examples:

split_arg([1, 2, 3]) == [1, 2, 3]
split_arg(1) == [1]
split_arg("1,2") == ["1", "2"]
split_arg(nil) == []

Parameters:

  • arg (Array, String) —

    Either an array or a string to be split into an array

Returns:

  • (Array) —

    An array of the form arg, [arg], or arg.split(‘,’)



48
49
50
51
52
53
54
55
56
57
58
 
# File 'lib/beaker/options/parser.rb', line 48

def split_arg arg
  arry = []
  if arg.is_a?(Array)
    arry += arg
  elsif arg =~ /,/
    arry += arg.split(',')
  else
    arry << arg
  end
  arry
end

#usage ⇒ String

Returns a description of Beaker’s supported arguments

Returns:

  • (String) —

    The usage String



34
35
36
 
# File 'lib/beaker/options/parser.rb', line 34

def usage
 @command_line_parser.usage
end