Class: Chef::Provisioning::FogDriver::Driver

Inherits:
Provisioning::Driver
  • Object
show all
Includes:
Mixin::ShellOut
Defined in:
lib/chef/provisioning/fog_driver/driver.rb

Overview

Provisions cloud machines with the Fog driver.

## Fog Driver URLs

All Chef Provisioning drivers use URLs to uniquely identify a driver’s “bucket” of machines. Fog URLs are of the form fog:<provider>:<identifier:> - see individual providers for sample URLs.

Identifier is generally something uniquely identifying the account. If multiple users can access the account, the identifier should be the same for all of them (do not use the username in these cases, use an account ID or auth server URL).

In particular, the identifier should be specific enough that if you create a server with a driver with this URL, the server should be retrievable from the same URL *no matter what else changes*. For example, an AWS account ID is not enough for this–if you varied the region, you would no longer see your server in the list. Thus, AWS uses both the account ID and the region.

## Supporting a new Fog provider

The Fog driver does not immediately support all Fog providers out of the box. Some minor work needs to be done to plug them into Chef.

To add a new supported Fog provider, pick an appropriate identifier, go to from_provider and compute_options_for, and add the new provider in the case statements so that URLs for your Fog provider can be generated. If your cloud provider has environment variables or standard config files (like ~/.aws/credentials or ~/.aws/config), you can read those and merge that information in the compute_options_for function.

## Reference format

All machines have a reference hash to find them. These are the keys used by the Fog provisioner:

  • driver_url: fog:<driver>:<unique_account_info>

  • server_id: the ID of the server so it can be found again

  • created_at: timestamp server was created

  • started_at: timestamp server was last started

  • is_windows, ssh_username, sudo: copied from machine_options

## Machine options

Machine options (for allocation and readying the machine) include:

  • bootstrap_options: hash of options to pass to compute.servers.create

  • is_windows: true if windows. TODO detect this from ami?

  • create_timeout: the time to wait for the instance to boot to ssh (defaults to 180)

  • start_timeout: the time to wait for the instance to start (defaults to 180)

  • ssh_timeout: the time to wait for ssh to be available if the instance is detected as up (defaults to 20)

  • ssh_username: username to use for ssh

  • sudo: true to prefix all commands with “sudo”

  • transport_address_location: ssh into machine via ‘:public_ip`, `:private_ip`, or `:ip_addresses`

  • use_private_ip_for_ssh: (DEPRECATED and is replaced with ‘transport_address_location`) hint to use private floating_ip when available

  • convergence_options: hash of options for the convergence strategy

    • chef_client_timeout: the time to wait for chef-client to finish

    • chef_server - the chef server to point convergence at

Example bootstrap_options for ec2:

:bootstrap_options => {
  :image_id =>'ami-311f2b45',
  :flavor_id =>'t1.micro',
  :key_name => 'key-pair-name'
}

Constant Summary collapse

DEFAULT_OPTIONS = 
{
  create_timeout: 180,
  start_timeout: 180,
  ssh_timeout: 20
}.freeze
RETRYABLE_ERRORS = 
[Fog::Compute::AWS::Error].freeze
RETRYABLE_OPTIONS = 
{ tries: 12, sleep: 5, on: RETRYABLE_ERRORS }.freeze
@@ip_pool_lock = 
Mutex.new
@@registered_provider_classes = 
{}

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(driver_url, config) ⇒ Driver

Create a new Fog driver.

## Parameters driver_url - URL of driver. “fog:<provider>:<provider_id>” config - configuration. :driver_options, :keys, :key_paths and :log_level are used.

driver_options is a hash with these possible options:
- compute_options: the hash of options to Fog::Compute.new.
- aws_config_file: aws config file (defaults: ~/.aws/credentials, ~/.aws/config)
- aws_csv_file: aws csv credentials file downloaded from EC2 interface
- aws_profile: profile name to use for credentials
- aws_credentials: AWSCredentials object. (will be created for you by default)
- log_level: :debug, :info, :warn, :error



169
170
171
172
173
174
175
176
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 169

def initialize(driver_url, config)
  super(driver_url, config)
  if config[:log_level] == :debug
    Fog::Logger[:debug] = ::STDERR
    Excon.defaults[:debug_request] = true
    Excon.defaults[:debug_response] = true
  end
end

Class Method Details

.__new__ ⇒ Object 



111
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 111

alias __new__ new

.canonicalize_url(driver_url, config) ⇒ Object 



140
141
142
143
144
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 140

def self.canonicalize_url(driver_url, config)
  _, provider, id = driver_url.split(":", 3)
  config, id = provider_class_for(provider).compute_options_for(provider, id, config)
  ["fog:#{provider}:#{id}", config]
end

.from_provider(provider, config) ⇒ Object

Passed in a config which is not merged with driver_url (because we don’t know what it is yet) but which has the same keys



148
149
150
151
152
153
154
155
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 148

def self.from_provider(provider, config)
  # Figure out the options and merge them into the config
  config, id = provider_class_for(provider).compute_options_for(provider, nil, config)

  driver_url = "fog:#{provider}:#{id}"

  Provisioning.driver_for_url(driver_url, config)
end

.from_url(driver_url, config) ⇒ Object

Passed in a driver_url, and a config in the format of Driver.config.



136
137
138
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 136

def self.from_url(driver_url, config)
  Driver.new(driver_url, config)
end

.inherited(klass) ⇒ Object 



113
114
115
116
117
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 113

def inherited(klass)
  class << klass
    alias_method :new, :__new__
  end
end

.new(driver_url, config) ⇒ Object 



130
131
132
133
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 130

def self.new(driver_url, config)
  provider = driver_url.split(":")[1]
  provider_class_for(provider).new(driver_url, config)
end

.provider_class_for(provider) ⇒ Object 



125
126
127
128
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 125

def self.provider_class_for(provider)
  require "chef/provisioning/fog_driver/providers/#{provider.downcase}"
  @@registered_provider_classes[provider]
end

.register_provider_class(name, driver) ⇒ Object 



121
122
123
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 121

def self.register_provider_class(name, driver)
  @@registered_provider_classes[name] = driver
end

Instance Method Details

#allocate_machine(action_handler, machine_spec, machine_options) ⇒ Object

Acquire a machine, generally by provisioning it. Returns a Machine object pointing at the machine, allowing useful actions like setup, converge, execute, file and directory.



189
190
191
192
193
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 189

def allocate_machine(action_handler, machine_spec, machine_options)
  # If the server does not exist, create it
  create_servers(action_handler, { machine_spec => machine_options }, Chef::ChefFS::Parallelizer.new(0))
  machine_spec
end

#allocate_machines(action_handler, specs_and_options, parallelizer) ⇒ Object 



195
196
197
198
199
200
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 195

def allocate_machines(action_handler, specs_and_options, parallelizer)
  create_servers(action_handler, specs_and_options, parallelizer) do |machine_spec, _server|
    yield machine_spec
  end
  specs_and_options.keys
end

#compute ⇒ Object 



264
265
266
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 264

def compute
  @compute ||= Fog::Compute.new(compute_options)
end

#compute_options ⇒ Object 



178
179
180
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 178

def compute_options
  JSON.parse(driver_options[:compute_options].to_h.to_json, symbolize_names: true) || {}
end

#connect_to_machine(machine_spec, machine_options) ⇒ Object

Connect to machine without acquiring it



235
236
237
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 235

def connect_to_machine(machine_spec, machine_options)
  machine_for(machine_spec, machine_options)
end

#create_volume(_action_handler, _volume_spec, _volume_options) ⇒ Object 



282
283
284
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 282

def create_volume(_action_handler, _volume_spec, _volume_options)
  raise "##create_volume not implemented in in #{self.class}"
end

#destroy_machine(action_handler, machine_spec, machine_options) ⇒ Object 



239
240
241
242
243
244
245
246
247
248
249
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 239

def destroy_machine(action_handler, machine_spec, machine_options)
  server = server_for(machine_spec)
  if server
    action_handler.perform_action "destroy machine #{machine_spec.name} (#{machine_spec.reference['server_id']} at #{driver_url})" do
      server.destroy
      machine_spec.reference = nil
    end
  end
  strategy = ConvergenceStrategy::NoConverge.new(machine_options[:convergence_options], config)
  strategy.cleanup_convergence(action_handler, machine_spec)
end

#destroy_volume(_action_handler, _volume_spec, _volume_options) ⇒ Object 



286
287
288
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 286

def destroy_volume(_action_handler, _volume_spec, _volume_options)
  raise "##destroy_volume not implemented in in #{self.class}"
end

#image_for(image_spec) ⇒ Object 



260
261
262
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 260

def image_for(image_spec)
  compute.images.get(image_spec.reference["image_id"])
end

#provider ⇒ Object 



182
183
184
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 182

def provider
  compute_options[:provider]
end

#ready_machine(action_handler, machine_spec, machine_options) ⇒ Object 



202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 202

def ready_machine(action_handler, machine_spec, machine_options)
  server = server_for(machine_spec)
  if server.nil?
    raise "Machine #{machine_spec.name} does not have a server associated with it, or server does not exist."
  end

  # Start the server if needed, and wait for it to start
  start_server(action_handler, machine_spec, server)
  wait_until_ready(action_handler, machine_spec, machine_options, server)

  converge_floating_ips(action_handler, machine_spec, machine_options, server)

  begin
    wait_for_transport(action_handler, machine_spec, machine_options, server)
  rescue Fog::Errors::TimeoutError
    # Only ever reboot once, and only if it's been less than 10 minutes since we stopped waiting
    if machine_spec.reference["started_at"] || remaining_wait_time(machine_spec, machine_options) < -(10 * 60)
      raise
    else
      # Sometimes (on EC2) the machine comes up but gets stuck or has
      # some other problem.  If this is the case, we restart the server
      # to unstick it.  Reboot covers a multitude of sins.
      Chef::Log.warn "Machine #{machine_spec.name} (#{server.id} on #{driver_url}) was started but SSH did not come up.  Rebooting machine in an attempt to unstick it ..."
      restart_server(action_handler, machine_spec, server)
      wait_until_ready(action_handler, machine_spec, machine_options, server)
      wait_for_transport(action_handler, machine_spec, machine_options, server)
    end
  end

  machine_for(machine_spec, machine_options, server)
end

#stop_machine(action_handler, machine_spec, _machine_options) ⇒ Object 



251
252
253
254
255
256
257
258
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 251

def stop_machine(action_handler, machine_spec, _machine_options)
  server = server_for(machine_spec)
  if server
    action_handler.perform_action "stop machine #{machine_spec.name} (#{server.id} at #{driver_url})" do
      server.stop
    end
  end
end

#transport_for(machine_spec, machine_options, server, action_handler = nil) ⇒ Object

Not meant to be part of public interface



269
270
271
272
273
274
275
276
277
278
279
280
 
# File 'lib/chef/provisioning/fog_driver/driver.rb', line 269

def transport_for(machine_spec, machine_options, server, action_handler = nil)
  Chef::Log.debug("Creating transport for #{server}")
  Chef::Log.debug("Machine Spec: #{machine_spec}")
  if machine_spec.reference["is_windows"]
    action_handler.report_progress "Waiting for admin password on #{machine_spec.name} to be ready (may take up to 15 minutes)..." if action_handler
    transport = create_winrm_transport(machine_spec, machine_options, server)
    action_handler.report_progress "Admin password available ..." if action_handler
    transport
  else
    create_ssh_transport(machine_spec, machine_options, server)
  end
end