Class: Vanagon::Engine::AlwaysBeScheduling
- Defined in:
- lib/vanagon/engine/always_be_scheduling.rb
Overview
This engine allows build resources to be managed by the [“Always be Scheduling” (ABS) scheduler](github.com/puppetlabs/always-be-scheduling)
ABS expects to ask ‘build_host_info` for the needed resources for a build, and to have that return a platform name. ABS will then acquire the desired build host resources and will later run a vanagon build, passing those resource hostnames in specifically.
‘build_host_info` will normally use the `hardware` engine when a hardware platform is queried. The `always_be_scheduling` engine’s behavior will be invoked instead when:
‘build_host_info … –engine always_be_scheduling` is specified on the command-line.
Configuration
Project platform configurations can specify the platform name to be returned via the ‘abs_resource_name` attribute. If this is not set but `vmpooler_template` is set, then the `vmpooler_template` value will be used. Otherwise, the platform name will be returned unchanged.
Example 1
“‘ platform ’ubuntu-10.04-amd64’ do |plat|
plat.vmpooler_template 'ubuntu-1004-amd64'
end “‘
“‘ $ build_host_info puppet-agent ubuntu-10.04-amd64 “name”:“ubuntu-10.04-amd64”,“engine”:“pooler”
$ build_host_info puppet-agent ubuntu-10.04-amd64 –engine always_be_scheduling “name”:“ubuntu-10.04-amd64”,“engine”:“always_be_scheduling” “‘
Example 2
“‘ platform ’aix-5.3-ppc’ do |plat|
plat.build_host ['aix53-builder-1.example.com']
plat.abs_resource_name 'aix-53-ppc'
end “‘
“‘ $ build_host_info puppet-agent aix-5.3-ppc “name”:“aix53-builder-1.example.com”,“engine”:“hardware”
$ build_host_info puppet-agent aix-5.3-ppc –engine always_be_scheduling “name”:“aix-53-ppc”,“engine”:“always_be_scheduling” “‘
Example 3
“‘ platform ’aix-5.3-ppc’ do |plat|
plat.build_host ['aix53-builder-1.example.com']
plat.vmpooler_template
plat.abs_resource_name 'aix-53-ppc'
end “‘
“‘ $ build_host_info puppet-agent aix-5.3-ppc “name”:“aix53-builder-1.example.com”,“engine”:“hardware”
$ build_host_info puppet-agent aix-5.3-ppc –engine always_be_scheduling “name”:“aix-53-ppc”,“engine”:“always_be_scheduling” “‘
Instance Attribute Summary collapse
-
#token ⇒ Object
readonly
Returns the value of attribute token.
-
#token_vmpooler ⇒ Object
readonly
Returns the value of attribute token_vmpooler.
Attributes inherited from Base
Instance Method Summary collapse
-
#build_host_name ⇒ Object
return the platform name as the “host” name order of preference: abs_resource_name, vmpooler_template or name.
-
#check_queue(pooler, request_object) ⇒ Object
main loop where the status of the request is checked, to see if the request has been allocated.
-
#initialize(platform, target, **opts) ⇒ AlwaysBeScheduling
constructor
A new instance of AlwaysBeScheduling.
-
#load_token ⇒ String?
Retrieve the ABS token from an environment variable (“ABS_TOKEN”) or from a number of potential configuration files (~/.vanagon-token or ~/.vmfloaty.yml).
-
#name ⇒ Object
Get the engine name.
-
#select_target ⇒ Object
Used to obtain a vm to build upon using Puppet’s internal ABS (github.com/puppetlabs/always-be-scheduling) which is a level of abstraction for other engines using similar APIs.
-
#select_target_from(pooler) ⇒ Object
Attempt to provision a host from a specific pooler.
-
#teardown ⇒ Object
This method is used to tell the ABS to delete the job_id requested otherwise the resources will eventually get allocated asynchronously and will keep running until the end of their lifetime.
- #validate_queue_status_response(status_code, body) ⇒ Object
Methods inherited from Base
#dispatch, #get_remote_workdir, #parse_target_defaults, #retrieve_built_artifact, #setup, #ship_workdir, #startup, #validate_platform
Constructor Details
#initialize(platform, target, **opts) ⇒ AlwaysBeScheduling
Returns a new instance of AlwaysBeScheduling.
90 91 92 93 94 95 96 97 |
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 90 def initialize(platform, target, **opts) super @available_abs_endpoint = "https://abs-prod.k8s.infracore.puppet.net/api/v2" @token_vmpooler = ENV['VMPOOLER_TOKEN'] @token = load_token Vanagon::Driver.logger.debug "AlwaysBeScheduling engine invoked." end |
Instance Attribute Details
#token ⇒ Object (readonly)
Returns the value of attribute token.
87 88 89 |
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 87 def token @token end |
#token_vmpooler ⇒ Object (readonly)
Returns the value of attribute token_vmpooler.
88 89 90 |
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 88 def token_vmpooler @token_vmpooler end |
Instance Method Details
#build_host_name ⇒ Object
return the platform name as the “host” name order of preference: abs_resource_name, vmpooler_template or name
106 107 108 109 110 111 112 113 114 |
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 106 def build_host_name if @platform.abs_resource_name @platform.abs_resource_name elsif @platform.vmpooler_template @platform.vmpooler_template else @platform.name end end |
#check_queue(pooler, request_object) ⇒ Object
main loop where the status of the request is checked, to see if the request has been allocated
268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 |
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 268 def check_queue(pooler, request_object) # 360 retries takes about an hour retries = 360 response_body = nil begin (1..retries).each do |i| response = Vanagon::Utilities.http_request_generic( "#{pooler}/request", 'POST', request_object.to_json, { 'X-AUTH-TOKEN' => @token } ) response_body = validate_queue_status_response(response.code, response.body) break if response_body sleep_seconds = 10 if i >= 10 sleep_seconds = i if i < 10 VanagonLogger.info "Waiting #{sleep_seconds} seconds to fill ABS request (x#{i})" sleep(sleep_seconds) end rescue SystemExit, Interrupt VanagonLogger.error "\nVanagon interrupted during mains ABS polling. " \ "Remember to delete the requested job_id #{@saved_job_id}" raise end if response_body translated(response_body, @saved_job_id) else VanagonLogger.error "ABS timed out after #{retries} retries." { 'retry-failure': retries } end end |
#load_token ⇒ String?
Retrieve the ABS token from an environment variable (“ABS_TOKEN”) or from a number of potential configuration files (~/.vanagon-token or ~/.vmfloaty.yml).
120 121 122 |
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 120 def load_token ENV['ABS_TOKEN'] || token_from_file end |
#name ⇒ Object
Get the engine name
100 101 102 |
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 100 def name 'always_be_scheduling' end |
#select_target ⇒ Object
Used to obtain a vm to build upon using Puppet’s internal ABS (github.com/puppetlabs/always-be-scheduling) which is a level of abstraction for other engines using similar APIs
230 231 232 233 234 235 |
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 230 def select_target @pooler = select_target_from(@available_abs_endpoint) if @pooler.empty? raise Vanagon::Error, "No available ABS machine from #{@available_abs_endpoint}" end end |
#select_target_from(pooler) ⇒ Object
Attempt to provision a host from a specific pooler.
238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 |
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 238 def select_target_from(pooler) # rubocop:disable Metrics/AbcSize request_object = build_request_object VanagonLogger.info "Requesting VMs with job_id: #{@saved_job_id}. Will poll for up to an hour." #the initial request is always replied with "come back again" response = Vanagon::Utilities.http_request_generic( "#{pooler}/request", 'POST', request_object.to_json, { 'X-AUTH-TOKEN' => @token } ) unless response.code == "202" VanagonLogger.info "failed to request ABS with code #{response.code}" if valid_json?(response.body) response_json = JSON.parse(response.body) VanagonLogger.info "reason: #{response_json['reason']}" end return '' end response_body = check_queue(pooler, request_object) return '' unless response_body["ok"] @target = response_body[build_host_name]['hostname'] Vanagon::Driver.logger.info "Reserving #{@target} (#{build_host_name}) [#{@token ? 'token used' : 'no token used'}]" return pooler end |
#teardown ⇒ Object
This method is used to tell the ABS to delete the job_id requested otherwise the resources will eventually get allocated asynchronously and will keep running until the end of their lifetime.
319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 |
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 319 def teardown request_object = { 'job_id' => @saved_job_id, } response = Vanagon::Utilities.http_request_generic( "#{@available_abs_endpoint}/return", "POST", request_object.to_json, { 'X-AUTH-TOKEN' => @token } ) if response && response.body == 'OK' Vanagon::Driver.logger.info "#{@saved_job_id} has been scheduled for removal" VanagonLogger.info "#{@saved_job_id} has been scheduled for removal" else Vanagon::Driver.logger.info "#{@saved_job_id} could not be scheduled for removal: #{response.body}" VanagonLogger.info "#{@saved_job_id} could not be scheduled for removal" end rescue Vanagon::Error => e Vanagon::Driver.logger.info "#{@saved_job_id} could not be scheduled for removal (#{e.})" VanagonLogger.info "#{@saved_job_id} could not be scheduled for removal (#{e.})" end |
#validate_queue_status_response(status_code, body) ⇒ Object
303 304 305 306 307 308 309 310 311 312 313 314 |
# File 'lib/vanagon/engine/always_be_scheduling.rb', line 303 def validate_queue_status_response(status_code, body) case status_code when "200" return JSON.parse(body) unless body.empty? || !valid_json?(body) when "202", "503" return nil when "401" raise Vanagon::Error, "HTTP #{status_code}: The token provided could not authenticate.\n#{body}" else raise Vanagon::Error, "HTTP #{status_code}: request to ABS failed!\n#{body}" end end |