Class: Vanagon::Engine::AlwaysBeScheduling

Inherits:
Base
  • Object
show all
Defined in:
lib/vanagon/engine/always_be_scheduling.rb

Overview

This engine allows build resources to be managed by the [“Always be Scheduling” (ABS) scheduler](github.com/puppetlabs/always-be-scheduling)

ABS expects to ask ‘build_host_info` for the needed resources for a build, and to have that return a platform name. ABS will then acquire the desired build host resources and will later run a vanagon build, passing those resource hostnames in specifically.

‘build_host_info` will normally use the `hardware` engine when a hardware platform is queried. The `always_be_scheduling` engine’s behavior will be invoked instead when:

‘build_host_info … –engine always_be_scheduling` is specified on the command-line.

Configuration

Project platform configurations can specify the platform name to be returned via the ‘abs_resource_name` attribute. If this is not set but `vmpooler_template` is set, then the `vmpooler_template` value will be used. Otherwise, the platform name will be returned unchanged.

Example 1

“‘ platform ’ubuntu-10.04-amd64’ do |plat|

plat.vmpooler_template 'ubuntu-1004-amd64'

end “‘

“‘ $ build_host_info puppet-agent ubuntu-10.04-amd64 “name”:“ubuntu-10.04-amd64”,“engine”:“pooler”

$ build_host_info puppet-agent ubuntu-10.04-amd64 –engine always_be_scheduling “name”:“ubuntu-10.04-amd64”,“engine”:“always_be_scheduling” “‘

Example 2

“‘ platform ’aix-5.3-ppc’ do |plat|

plat.build_host ['aix53-builder-1.example.com']
plat.abs_resource_name 'aix-53-ppc'

end “‘

“‘ $ build_host_info puppet-agent aix-5.3-ppc “name”:“aix53-builder-1.example.com”,“engine”:“hardware”

$ build_host_info puppet-agent aix-5.3-ppc –engine always_be_scheduling “name”:“aix-53-ppc”,“engine”:“always_be_scheduling” “‘

Example 3

“‘ platform ’aix-5.3-ppc’ do |plat|

plat.build_host ['aix53-builder-1.example.com']
plat.vmpooler_template
plat.abs_resource_name 'aix-53-ppc'

end “‘

“‘ $ build_host_info puppet-agent aix-5.3-ppc “name”:“aix53-builder-1.example.com”,“engine”:“hardware”

$ build_host_info puppet-agent aix-5.3-ppc –engine always_be_scheduling “name”:“aix-53-ppc”,“engine”:“always_be_scheduling” “‘

Instance Attribute Summary collapse

Attributes inherited from Base

#remote_workdir, #target

Instance Method Summary collapse

Methods inherited from Base

#dispatch, #get_remote_workdir, #retrieve_built_artifact, #setup, #ship_workdir, #startup, #validate_platform

Constructor Details

#initialize(platform, target, **opts) ⇒ AlwaysBeScheduling

Returns a new instance of AlwaysBeScheduling.



90
91
92
93
94
95
96
97
 
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 90

def initialize(platform, target, **opts)
  super

  @available_abs_endpoint = "https://abs-prod.k8s.infracore.puppet.net/api/v2"
  @token_vmpooler = ENV['VMPOOLER_TOKEN']
  @token = load_token
  Vanagon::Driver.logger.debug "AlwaysBeScheduling engine invoked."
end

Instance Attribute Details

#token ⇒ Object (readonly)

Returns the value of attribute token.



87
88
89
 
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 87

def token
  @token
end

#token_vmpooler ⇒ Object (readonly)

Returns the value of attribute token_vmpooler.



88
89
90
 
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 88

def token_vmpooler
  @token_vmpooler
end

Instance Method Details

#build_host_name ⇒ Object

return the platform name as the “host” name order of preference: abs_resource_name, vmpooler_template or name



106
107
108
109
110
111
112
113
114
 
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 106

def build_host_name
  if @platform.abs_resource_name
    @platform.abs_resource_name
  elsif @platform.vmpooler_template
    @platform.vmpooler_template
  else
    @platform.name
  end
end

#check_queue(pooler, request_object) ⇒ Object

main loop where the status of the request is checked, to see if the request has been allocated



266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
 
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 266

def check_queue(pooler, request_object)
  retries = 360 # ~ one hour
  response_body = nil
  begin
    (1..retries).each do |i|
      response = Vanagon::Utilities.http_request_generic(
        "#{pooler}/request",
        'POST',
        request_object.to_json,
        { 'X-AUTH-TOKEN' => @token }
      )
      response_body = validate_queue_status_response(response.code, response.body)
      break if response_body

      sleep_seconds = 10 if i >= 10
      sleep_seconds = i if i < 10
      VanagonLogger.info "Waiting #{sleep_seconds} seconds to check if ABS request has been filled. (x#{i})"

      sleep(sleep_seconds)
    end
  rescue SystemExit, Interrupt
    VanagonLogger.error "\nVanagon interrupted during mains ABS polling. Make sure you delete the requested job_id #{@saved_job_id}"
    raise
  end
  response_body = translated(response_body, @saved_job_id)
  response_body
end

#load_token ⇒ String?

Retrieve the ABS token from an environment variable (“ABS_TOKEN”) or from a number of potential configuration files (~/.vanagon-token or ~/.vmfloaty.yml).

Returns:

  • (String, nil) —

    token for use with the vmpooler



120
121
122
 
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 120

def load_token
  ENV['ABS_TOKEN'] || token_from_file
end

#name ⇒ Object

Get the engine name



100
101
102
 
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 100

def name
  'always_be_scheduling'
end

#select_target ⇒ Object

This method is used to obtain a vm to build upon using Puppet’s internal ABS (github.com/puppetlabs/always-be-scheduling) which is a level of abstraction for other engines using similar APIs

Raises:



230
231
232
233
 
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 230

def select_target
  @pooler = select_target_from(@available_abs_endpoint)
  raise Vanagon::Error, "Something went wrong getting a target vm to build on" if @pooler.empty?
end

#select_target_from(pooler) ⇒ Object

Attempt to provision a host from a specific pooler.



236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
 
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 236

def select_target_from(pooler) # rubocop:disable Metrics/AbcSize
  request_object = build_request_object

  VanagonLogger.info "Requesting VMs with job_id: #{@saved_job_id}.  Will poll for up to an hour."
  #the initial request is always replied with "come back again"
  response = Vanagon::Utilities.http_request_generic(
    "#{pooler}/request",
    'POST',
    request_object.to_json,
    { 'X-AUTH-TOKEN' => @token }
  )

  unless response.code == "202"
    VanagonLogger.info "failed to request ABS with code #{response.code}"
    if valid_json?(response.body)
      response_json = JSON.parse(response.body)
      VanagonLogger.info "reason: #{response_json['reason']}"
    end
    return ''
  end
  response_body = check_queue(pooler, request_object)

  return '' unless response_body["ok"]
  @target = response_body[build_host_name]['hostname']
  Vanagon::Driver.logger.info "Reserving #{@target} (#{build_host_name}) [#{@token ? 'token used' : 'no token used'}]"
  return pooler
end

#teardown ⇒ Object

This method is used to tell the ABS to delete the job_id requested otherwise the resources will eventually get allocated asynchronously and will keep running until the end of their lifetime.



312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
 
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 312

def teardown # rubocop:disable Metrics/AbcSize
  request_object = {
      'job_id' => @saved_job_id,
  }

  response = Vanagon::Utilities.http_request_generic(
    "#{@available_abs_endpoint}/return",
    "POST",
    request_object.to_json,
    { 'X-AUTH-TOKEN' => @token }
  )
  if response && response.body == 'OK'
    Vanagon::Driver.logger.info "#{@saved_job_id} has been scheduled for removal"
    VanagonLogger.info "#{@saved_job_id} has been scheduled for removal"
  else
    Vanagon::Driver.logger.info "#{@saved_job_id} could not be scheduled for removal: #{response.body}"
    VanagonLogger.info "#{@saved_job_id} could not be scheduled for removal"
  end
rescue Vanagon::Error => e
  Vanagon::Driver.logger.info "#{@saved_job_id} could not be scheduled for removal (#{e.message})"
  VanagonLogger.info "#{@saved_job_id} could not be scheduled for removal (#{e.message})"
end

#validate_queue_status_response(status_code, body) ⇒ Object 



294
295
296
297
298
299
300
301
302
303
304
305
306
307
 
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 294

def validate_queue_status_response(status_code, body)
  case status_code
  when "200"
    return JSON.parse(body) unless body.empty? || !valid_json?(body)
  when "202"
    return nil
  when "401"
    raise Vanagon::Error, "HTTP #{status_code}: The token provided could not authenticate.\n#{body}"
  when "503"
    return nil
  else
    raise Vanagon::Error, "HTTP #{status_code}: request to ABS failed!\n#{body}"
  end
end