Class: TestLab

Inherits:
Object
  • Object
show all
Extended by:
DualMethods
Includes:
DualMethods, Support::Parallel, Utility::Misc
Defined in:
lib/testlab.rb,
lib/testlab/node.rb,
lib/testlab/user.rb,
lib/testlab/source.rb,
lib/testlab/labfile.rb,
lib/testlab/network.rb,
lib/testlab/support.rb,
lib/testlab/utility.rb,
lib/testlab/version.rb,
lib/testlab/node/lxc.rb,
lib/testlab/node/ssh.rb,
lib/testlab/provider.rb,
lib/testlab/container.rb,
lib/testlab/interface.rb,
lib/testlab/dependency.rb,
lib/testlab/node/doctor.rb,
lib/testlab/node/status.rb,
lib/testlab/provisioner.rb,
lib/testlab/container/io.rb,
lib/testlab/network/bind.rb,
lib/testlab/node/actions.rb,
lib/testlab/utility/cidr.rb,
lib/testlab/utility/misc.rb,
lib/testlab/container/lxc.rb,
lib/testlab/container/ssh.rb,
lib/testlab/providers/aws.rb,
lib/testlab/container/user.rb,
lib/testlab/network/status.rb,
lib/testlab/node/provision.rb,
lib/testlab/user/lifecycle.rb,
lib/testlab/utility/logger.rb,
lib/testlab/container/clone.rb,
lib/testlab/network/actions.rb,
lib/testlab/providers/local.rb,
lib/testlab/container/status.rb,
lib/testlab/provisioners/apt.rb,
lib/testlab/support/parallel.rb,
lib/testlab/container/actions.rb,
lib/testlab/container/support.rb,
lib/testlab/network/provision.rb,
lib/testlab/providers/vagrant.rb,
lib/testlab/provisioners/bind.rb,
lib/testlab/provisioners/chef.rb,
lib/testlab/support/execution.rb,
lib/testlab/support/lifecycle.rb,
lib/testlab/node/class_methods.rb,
lib/testlab/provisioners/route.rb,
lib/testlab/provisioners/shell.rb,
lib/testlab/container/interface.rb,
lib/testlab/container/provision.rb,
lib/testlab/node/method_missing.rb,
lib/testlab/provisioners/raring.rb,
lib/testlab/provisioners/resolv.rb,
lib/testlab/providers/bare_metal.rb,
lib/testlab/providers/open_stack.rb,
lib/testlab/network/class_methods.rb,
lib/testlab/provisioners/nfs_mount.rb,
lib/testlab/container/class_methods.rb,
lib/testlab/container/configuration.rb,
lib/testlab/provisioners/hosts_file.rb,
lib/testlab/provisioners/apt_cacher_ng.rb,
lib/testlab/provisioners/chef/omni_bus.rb,
lib/testlab/provisioners/chef/omni_truck.rb,
lib/testlab/provisioners/chef/ruby_gem_client.rb,
lib/testlab/provisioners/chef/ruby_gem_server.rb

Overview

TestLab - A framework for building lightweight virtual infrastructure using LXC

The core concept with the TestLab is the Labfile. This file dictates the topology of your virtual infrastructure. With simple commands you can build and demolish this infrastructure on the fly for all sorts of purposes from automating infrastructure testing to testing new software to experimenting in general where you want to spin up alot of servers but do not want the overhead of virtualization. At it’s core TestLab uses Linux Containers (LXC) to accomplish this.

Examples:

Sample Labfile:

node 'vagrant' do

  provider      TestLab::Provider::Vagrant
  provisioners  [
    TestLab::Provisioner::Raring,
    TestLab::Provisioner::Bind
  ]

  config      ({
    :vagrant => {
      :id       => "chef-rubygem-#{TestLab.hostname}".downcase,
      :cpus     => ZTK::Parallel::MAX_FORKS.div(2),                    # use half of the available processors
      :memory   => ZTK::Parallel::MAX_MEMORY.div(3).div(1024 * 1024),  # use a third of available RAM
      :box      => 'raring64',
      :box_url  => 'https://dl.dropboxusercontent.com/u/22904185/boxes/raring64.box',
      :file     => File.dirname(__FILE__)
    },
    :bind => {
      :domain => "default.zone"
    }
  })

  network 'labnet' do
    provisioners  [TestLab::Provisioner::Route]
    address       '10.10.0.1/16'
    bridge        :br0
  end

  container "chef-server" do
    distro        "ubuntu"
    release       "precise"

    provisioners   [
      TestLab::Provisioner::Resolv,
      TestLab::Provisioner::AptCacherNG,
      TestLab::Provisioner::Apt,
      TestLab::Provisioner::Chef::RubyGemServer
    ]

    user 'deployer' do
      password         'deployer'
      identity         File.join(ENV['HOME'], '.ssh', 'id_rsa')
      public_identity  File.join(ENV['HOME'], '.ssh', 'id_rsa.pub')
      uid              2600
      gid              2600
    end

    interface do
      network_id  'labnet'
      name        :eth0
      address     '10.10.0.254/16'
      mac         '00:00:5e:63:b5:9f'
    end
  end

  container "chef-client" do
    distro        "ubuntu"
    release       "precise"

    provisioners  [
      TestLab::Provisioner::Resolv,
      TestLab::Provisioner::AptCacherNG,
      TestLab::Provisioner::Apt,
      TestLab::Provisioner::Chef::RubyGemClient
    ]

    user 'deployer' do
      password         'deployer'
      identity         File.join(ENV['HOME'], '.ssh', 'id_rsa')
      public_identity  File.join(ENV['HOME'], '.ssh', 'id_rsa.pub')
      uid              2600
      gid              2600
    end

    interface do
      network_id  'labnet'
      name        :eth0
      address     '10.10.0.20/16'
      mac         '00:00:5e:b7:e5:15'
    end
  end

end

TestLab can be instantiated easily:

log_file = File.join(Dir.pwd, "testlab.log")
logger = ZTK::Logger.new(log_file)
ui = ZTK::UI.new(:logger => logger)
testlab = TestLab.new(:ui => ui)

We can control things via code easily as well:

testlab.create   # creates the lab
testlab.up       # ensures the lab is up and running
testlab.build    # build the lab, creating all networks and containers
testlab.demolish # demolish the lab, destroy all networks and containers

Author:

  • Zachary Patten <zachary AT jovelabs DOT com>

Defined Under Namespace

Modules: DualMethods, Support, Utility Classes: Container, ContainerError, Dependency, DependencyError, Interface, InterfaceError, Labfile, LabfileError, Network, NetworkError, Node, NodeError, Provider, ProviderError, Provisioner, ProvisionerError, Source, SourceError, SupportError, TestLabError, User, UserError, UtilityError

Constant Summary collapse

VERSION =

TestLab Gem Version

"1.22.4"

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from DualMethods

build_command_line, gem_dir, hostname, ui, ui=

Methods included from Utility::Misc

#do_provisioner_callbacks, #format_message, #format_object_action, #please_wait, #sudo, #sudo_prompt

Methods included from Support::Parallel

#before_fork, #do_parallel_actions, #reset_screen

Constructor Details

#initialize(options = {}) ⇒ TestLab 



147
148
149
150
151
152
153
154
155
156
157
158
 
# File 'lib/testlab.rb', line 147

def initialize(options={})
  self.ui        = (options[:ui] || ZTK::UI.new)
  self.class.ui  = self.ui

  _labfile_path  = (options[:labfile_path] || ENV['LABFILE'] || 'Labfile')
  @labfile_path  = ZTK::Locator.find(_labfile_path)

  @repo_dir      = (options[:repo_dir] || File.dirname(@labfile_path))

  @config_dir    = (options[:config_dir] || File.join(@repo_dir, ".testlab-#{TestLab.hostname}"))
  File.exists?(@config_dir) or FileUtils.mkdir_p(@config_dir)
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(method_name, *method_args) ⇒ Object

Provider Method Handler

Proxies missing provider method calls to all nodes.

See Also:



466
467
468
469
470
471
472
473
474
 
# File 'lib/testlab.rb', line 466

def method_missing(method_name, *method_args)
  self.ui.logger.debug { "TESTLAB METHOD MISSING: #{method_name.inspect}(#{method_args.inspect})" }

  if TestLab::Provider::PROXY_METHODS.include?(method_name)
    node_method_proxy(method_name, *method_args)
  else
    super(method_name, *method_args)
  end
end

Instance Attribute Details

#config_dir ⇒ Object

Returns the value of attribute config_dir.



143
144
145
 
# File 'lib/testlab.rb', line 143

def config_dir
  @config_dir
end

#labfile_path ⇒ Object

Returns the value of attribute labfile_path.



145
146
147
 
# File 'lib/testlab.rb', line 145

def labfile_path
  @labfile_path
end

#repo_dir ⇒ Object

Returns the value of attribute repo_dir.



144
145
146
 
# File 'lib/testlab.rb', line 144

def repo_dir
  @repo_dir
end

Instance Method Details

#alive? ⇒ Boolean

Test Lab Alive?

Are all of our nodes up and running?



252
253
254
 
# File 'lib/testlab.rb', line 252

def alive?
  nodes.map(&:state).all?{ |state| state == :running }
end

#boot ⇒ Boolean

Boot TestLab

Change to the defined repository directory and load the Labfile.



165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
 
# File 'lib/testlab.rb', line 165

def boot
  TestLab::Utility.log_header(self).each { |line| self.ui.logger.info { line } }

  # Raise how many files we can have open to the hard limit.
  nofile_cur, nofile_max = Process.getrlimit(Process::RLIMIT_NOFILE)
  if nofile_cur != nofile_max

    # OSX likes to indicate we can set the infinity value here.
    #
    # Doing so causes the following exception to throw:
    #   Errno::EINVAL: Invalid argument - setrlimit
    #
    # In the event infinity is returned as the max value, use 4096 as the max
    # value.
    if (nofile_max == Process::RLIM_INFINITY)
      nofile_max = 4096
    end

    # Attempt to increment the number of open files we can have.  Now we just
    # rescue this, because OSX doesn't seem to like us doing this in general.
    if (nofile_max > nofile_cur)
      begin
        self.ui.logger.info { "Attempting to change maximum open file descriptors from #{nofile_cur.inspect} to #{nofile_max.inspect}" }
        Process.setrlimit(Process::RLIMIT_NOFILE, nofile_max)
      rescue Exception => e
        self.ui.logger.warn { "Failed to change maximum open file descriptors from #{nofile_cur.inspect} to #{nofile_max.inspect}!" }
      end
    end
  end

  @labfile         = TestLab::Labfile.load(labfile_path)
  @labfile.testlab = self

  Dir.chdir(@repo_dir)
end

#bounce ⇒ Boolean

Test Lab Bounce

Attempts to bounce our lab topology. This calls various methods on all of our nodes, networks and containers.



367
368
369
370
371
372
 
# File 'lib/testlab.rb', line 367

def bounce
  self.down
  self.up

  true
end

#build(force = false) ⇒ Boolean

Test Lab Build

Attempts to build our lab topology. This calls various methods on all of our nodes, networks and containers.



343
344
345
346
347
 
# File 'lib/testlab.rb', line 343

def build(force=false)
  method_proxy(:build, force)

  true
end

#config ⇒ Hash

Test Lab Configuration

The hash defined in our Labfile DSL object which represents any high-level lab configuration options.



243
244
245
 
# File 'lib/testlab.rb', line 243

def config
  @labfile.config
end

#containers ⇒ Array<TestLab::Container>

Test Lab Containers

Returns an array of our defined containers.



215
216
217
 
# File 'lib/testlab.rb', line 215

def containers
  TestLab::Container.all
end

#create ⇒ Boolean

Test Lab Create

Attempts to create our lab topology. This calls the create method on all of our nodes.



271
272
273
274
275
 
# File 'lib/testlab.rb', line 271

def create
  method_proxy(:create)

  true
end

#dead? ⇒ Boolean

Test Lab Dead?

Are any of our nodes not up and running?



261
262
263
 
# File 'lib/testlab.rb', line 261

def dead?
  !alive?
end

#demolish ⇒ Boolean

Test Lab Demolish

Attempts to demolish our lab topology. This calls various methods on all of our nodes, networks and containers.



355
356
357
358
359
 
# File 'lib/testlab.rb', line 355

def demolish
  reverse_method_proxy(:demolish)

  true
end

#deprovision ⇒ Boolean

Test Lab Deprovision

Attempts to tearddown our lab topology. This calls the deprovision method on all of our nodes.



331
332
333
334
335
 
# File 'lib/testlab.rb', line 331

def deprovision
  reverse_method_proxy(:deprovision)

  true
end

#destroy ⇒ Boolean

Test Lab Destroy

Attempts to destroy our lab topology. This calls the destroy method on all of our nodes.



283
284
285
286
287
 
# File 'lib/testlab.rb', line 283

def destroy
  reverse_method_proxy(:destroy)

  true
end

#doctor ⇒ Boolean

Test Lab Doctor

Attempts to analyze the lab for issues.



392
393
394
395
396
397
398
399
400
401
402
403
404
405
 
# File 'lib/testlab.rb', line 392

def doctor
  results = Array.new

  if ((rlimit_nofile = Process.getrlimit(Process::RLIMIT_NOFILE)[0]) < 1024)
    @ui.stderr.puts(format_message("The maximum number of file handles is set to #{rlimit_nofile}!  Please raise it to 1024 or higher!".red.bold))
    results << false
  end

  results << nodes.all? do |node|
    node.doctor
  end

  results.flatten.compact.all?
end

#down ⇒ Boolean

Test Lab Down

Attempts to down our lab topology. This calls the down method on all of our nodes.



307
308
309
310
311
 
# File 'lib/testlab.rb', line 307

def down
  reverse_method_proxy(:down)

  true
end

#labfile ⇒ TestLab::Labfile

Test Lab Labfile

Returns our top-level Labfile instance.



233
234
235
 
# File 'lib/testlab.rb', line 233

def labfile
  @labfile
end

#method_proxy(method_name, *method_args) ⇒ Boolean

Method Proxy

Iterates all of the lab objects sending the supplied method name and arguments to each object.



427
428
429
430
431
432
433
434
435
436
437
438
439
 
# File 'lib/testlab.rb', line 427

def method_proxy(method_name, *method_args)
  nodes.each do |node|
    node.send(method_name, *method_args)

    node.networks.each do |network|
      network.send(method_name, *method_args)
    end

    do_parallel_actions(TestLab::Container, node.containers, method_name) do |object, action, klass|
      object.send(method_name, *method_args)
    end
  end
end

#networks ⇒ Array<TestLab::Network>

Test Lab Networks

Returns an array of our defined networks.



224
225
226
 
# File 'lib/testlab.rb', line 224

def networks
  TestLab::Network.all
end

#node_method_proxy(method_name, *method_args) ⇒ Boolean

Node Method Proxy

Iterates all of the lab nodes, sending the supplied method name and arguments to each node.



413
414
415
416
417
418
419
 
# File 'lib/testlab.rb', line 413

def node_method_proxy(method_name, *method_args)
  nodes.each do |node|
    node.send(method_name.to_sym, *method_args)
  end

  true
end

#nodes ⇒ Array<TestLab::Node>

Test Lab Nodes

Returns an array of our defined nodes.



206
207
208
 
# File 'lib/testlab.rb', line 206

def nodes
  TestLab::Node.all
end

#provision ⇒ Boolean

Test Lab Provision

Attempts to provision our lab topology. This calls the provision method on all of our nodes.



319
320
321
322
323
 
# File 'lib/testlab.rb', line 319

def provision
  method_proxy(:provision)

  true
end

#recycle(force = false) ⇒ Boolean

Test Lab Recycle

Attempts to recycle our lab topology. This calls various methods on all of our nodes, networks and containers.



380
381
382
383
384
385
 
# File 'lib/testlab.rb', line 380

def recycle(force=false)
  self.demolish
  self.build(force)

  true
end

#reverse_method_proxy(method_name, *method_args) ⇒ Boolean

Reverse Method Proxy

Iterates all of the lab objects sending the supplied method name and arguments to each object.



447
448
449
450
451
452
453
454
455
456
457
458
459
 
# File 'lib/testlab.rb', line 447

def reverse_method_proxy(method_name, *method_args)
  nodes.reverse.each do |node|
    do_parallel_actions(TestLab::Container, node.containers.reverse, method_name, true) do |object, action, klass|
      object.send(method_name, *method_args)
    end

    node.networks.reverse.each do |network|
      network.send(method_name, *method_args)
    end

    node.send(method_name, *method_args)
  end
end

#up ⇒ Boolean

Test Lab Up

Attempts to up our lab topology. This calls the up method on all of our nodes.



295
296
297
298
299
 
# File 'lib/testlab.rb', line 295

def up
  method_proxy(:up)

  true
end