Class: Vanagon::Engine::AlwaysBeScheduling

Inherits:
Base
  • Object
show all
Defined in:
lib/vanagon/engine/always_be_scheduling.rb

Overview

This engine allows build resources to be managed by the [“Always be Scheduling” (ABS) scheduler](github.com/puppetlabs/always-be-scheduling)

ABS expects to ask ‘build_host_info` for the needed resources for a build, and to have that return a platform name. ABS will then acquire the desired build host resources and will later run a vanagon build, passing those resource hostnames in specifically.

‘build_host_info` will normally use the `hardware` engine when a hardware platform is queried. The `always_be_scheduling` engine’s behavior will be invoked instead when:

‘build_host_info … –engine always_be_scheduling` is specified on the command-line.

Configuration


Project platform configurations can specify the platform name to be returned via the ‘abs_resource_name` attribute. If this is not set but `vmpooler_template` is set, then the `vmpooler_template` value will be used. Otherwise, the platform name will be returned unchanged.

Example 1


“‘ platform ’ubuntu-10.04-amd64’ do |plat|

plat.vmpooler_template 'ubuntu-1004-amd64'

end “‘

“‘ $ build_host_info puppet-agent ubuntu-10.04-amd64 “name”:“ubuntu-10.04-amd64”,“engine”:“pooler”

$ build_host_info puppet-agent ubuntu-10.04-amd64 –engine always_be_scheduling “name”:“ubuntu-10.04-amd64”,“engine”:“always_be_scheduling” “‘

Example 2


“‘ platform ’aix-5.3-ppc’ do |plat|

plat.build_host ['aix53-builder-1.example.com']
plat.abs_resource_name 'aix-53-ppc'

end “‘

“‘ $ build_host_info puppet-agent aix-5.3-ppc “name”:“aix53-builder-1.example.com”,“engine”:“hardware”

$ build_host_info puppet-agent aix-5.3-ppc –engine always_be_scheduling “name”:“aix-53-ppc”,“engine”:“always_be_scheduling” “‘

Example 3


“‘ platform ’aix-5.3-ppc’ do |plat|

plat.build_host ['aix53-builder-1.example.com']
plat.vmpooler_template
plat.abs_resource_name 'aix-53-ppc'

end “‘

“‘ $ build_host_info puppet-agent aix-5.3-ppc “name”:“aix53-builder-1.example.com”,“engine”:“hardware”

$ build_host_info puppet-agent aix-5.3-ppc –engine always_be_scheduling “name”:“aix-53-ppc”,“engine”:“always_be_scheduling” “‘

Instance Attribute Summary collapse

Attributes inherited from Base

#remote_workdir, #target

Instance Method Summary collapse

Methods inherited from Base

#dispatch, #get_remote_workdir, #parse_target_defaults, #retrieve_built_artifact, #setup, #ship_workdir, #startup, #validate_platform

Constructor Details

#initialize(platform, target, **opts) ⇒ AlwaysBeScheduling

Returns a new instance of AlwaysBeScheduling.



90
91
92
93
94
95
96
97
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 90

def initialize(platform, target, **opts)
  super

  @available_abs_endpoint = "https://abs-prod.k8s.infracore.puppet.net/api/v2"
  @token_vmpooler = ENV['VMPOOLER_TOKEN']
  @token = load_token
  Vanagon::Driver.logger.debug "AlwaysBeScheduling engine invoked."
end

Instance Attribute Details

#tokenObject (readonly)

Returns the value of attribute token.



87
88
89
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 87

def token
  @token
end

#token_vmpoolerObject (readonly)

Returns the value of attribute token_vmpooler.



88
89
90
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 88

def token_vmpooler
  @token_vmpooler
end

Instance Method Details

#build_host_nameObject

return the platform name as the “host” name order of preference: abs_resource_name, vmpooler_template or name



106
107
108
109
110
111
112
113
114
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 106

def build_host_name
  if @platform.abs_resource_name
    @platform.abs_resource_name
  elsif @platform.vmpooler_template
    @platform.vmpooler_template
  else
    @platform.name
  end
end

#check_queue(pooler, request_object) ⇒ Object

main loop where the status of the request is checked, to see if the request has been allocated



268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 268

def check_queue(pooler, request_object)
  # 360 retries takes about an hour
  retries = 360
  response_body = nil
  begin
    (1..retries).each do |i|
      response = Vanagon::Utilities.http_request_generic(
        "#{pooler}/request",
        'POST',
        request_object.to_json,
        { 'X-AUTH-TOKEN' => @token }
      )
      response_body = validate_queue_status_response(response.code, response.body)
      break if response_body

      sleep_seconds = 10 if i >= 10
      sleep_seconds = i if i < 10
      VanagonLogger.info "Waiting #{sleep_seconds} seconds to fill ABS request (x#{i})"

      sleep(sleep_seconds)
    end
  rescue SystemExit, Interrupt
    VanagonLogger.error "\nVanagon interrupted during mains ABS polling. " \
                        "Remember to delete the requested job_id #{@saved_job_id}"
    raise
  end

  if response_body
    translated(response_body, @saved_job_id)
  else
    VanagonLogger.error "ABS timed out after #{retries} retries."
    { 'retry-failure': retries }
  end
end

#load_tokenString?

Retrieve the ABS token from an environment variable (“ABS_TOKEN”) or from a number of potential configuration files (~/.vanagon-token or ~/.vmfloaty.yml).

Returns:

  • (String, nil)

    token for use with the vmpooler



120
121
122
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 120

def load_token
  ENV['ABS_TOKEN'] || token_from_file
end

#nameObject

Get the engine name



100
101
102
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 100

def name
  'always_be_scheduling'
end

#select_targetObject

Used to obtain a vm to build upon using Puppet’s internal ABS (github.com/puppetlabs/always-be-scheduling) which is a level of abstraction for other engines using similar APIs

Raises:



230
231
232
233
234
235
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 230

def select_target
  @pooler = select_target_from(@available_abs_endpoint)
  if @pooler.empty?
    raise Vanagon::Error, "No available ABS machine from #{@available_abs_endpoint}"
  end
end

#select_target_from(pooler) ⇒ Object

Attempt to provision a host from a specific pooler.



238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 238

def select_target_from(pooler) # rubocop:disable Metrics/AbcSize
  request_object = build_request_object

  VanagonLogger.info "Requesting VMs with job_id: #{@saved_job_id}.  Will poll for up to an hour."
  #the initial request is always replied with "come back again"
  response = Vanagon::Utilities.http_request_generic(
    "#{pooler}/request",
    'POST',
    request_object.to_json,
    { 'X-AUTH-TOKEN' => @token }
  )

  unless response.code == "202"
    VanagonLogger.info "failed to request ABS with code #{response.code}"
    if valid_json?(response.body)
      response_json = JSON.parse(response.body)
      VanagonLogger.info "reason: #{response_json['reason']}"
    end
    return ''
  end
  response_body = check_queue(pooler, request_object)

  return '' unless response_body["ok"]
  @target = response_body[build_host_name]['hostname']
  Vanagon::Driver.logger.info "Reserving #{@target} (#{build_host_name}) [#{@token ? 'token used' : 'no token used'}]"
  return pooler
end

#teardownObject

This method is used to tell the ABS to delete the job_id requested otherwise the resources will eventually get allocated asynchronously and will keep running until the end of their lifetime.



319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 319

def teardown
  request_object = {
      'job_id' => @saved_job_id,
  }

  response = Vanagon::Utilities.http_request_generic(
    "#{@available_abs_endpoint}/return",
    "POST",
    request_object.to_json,
    { 'X-AUTH-TOKEN' => @token }
  )
  if response && response.body == 'OK'
    Vanagon::Driver.logger.info "#{@saved_job_id} has been scheduled for removal"
    VanagonLogger.info "#{@saved_job_id} has been scheduled for removal"
  else
    Vanagon::Driver.logger.info "#{@saved_job_id} could not be scheduled for removal: #{response.body}"
    VanagonLogger.info "#{@saved_job_id} could not be scheduled for removal"
  end
rescue Vanagon::Error => e
  Vanagon::Driver.logger.info "#{@saved_job_id} could not be scheduled for removal (#{e.message})"
  VanagonLogger.info "#{@saved_job_id} could not be scheduled for removal (#{e.message})"
end

#validate_queue_status_response(status_code, body) ⇒ Object



303
304
305
306
307
308
309
310
311
312
313
314
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 303

def validate_queue_status_response(status_code, body)
  case status_code
  when "200"
    return JSON.parse(body) unless body.empty? || !valid_json?(body)
  when "202", "503"
    return nil
  when "401"
    raise Vanagon::Error, "HTTP #{status_code}: The token provided could not authenticate.\n#{body}"
  else
    raise Vanagon::Error, "HTTP #{status_code}: request to ABS failed!\n#{body}"
  end
end