Class: Ufo::Ship

Inherits:
Object
  • Object
show all
Includes:
AwsServices, Defaults, Util
Defined in:
lib/ufo/ship.rb

Instance Method Summary collapse

Methods included from Util

#execute, #pretty_time

Methods included from AwsServices

#cloudwatchlogs, #ecr, #ecs, #elb

Methods included from Defaults

#default_cluster, #default_desired_count, #default_maximum_percent, #default_minimum_healthy_percent, #new_service_settings, #settings

Constructor Details

#initialize(service, task_definition, options = {}) ⇒ Ship 



29
30
31
32
33
34
35
36
37
38
 
# File 'lib/ufo/ship.rb', line 29

def initialize(service, task_definition, options={})
  @service = service
  @task_definition = task_definition
  @options = options
  @project_root = options[:project_root] || '.'
  @target_group_prompt = @options[:target_group_prompt].nil? ? true : @options[:target_group_prompt]
  @cluster = @options[:cluster] || default_cluster
  @wait_for_deployment = @options[:wait].nil? ? true : @options[:wait]
  @stop_old_tasks = @options[:stop_old_tasks].nil? ? false : @options[:stop_old_tasks]
end

Instance Method Details

#add_load_balancer!(container, options, target_group) ⇒ Object

Only support Application Load Balancer Think there is an AWS bug that complains about not having the LB name but you cannot pass both a LB Name and a Target Group.



289
290
291
292
293
294
295
296
297
298
299
300
 
# File 'lib/ufo/ship.rb', line 289

def add_load_balancer!(container, options, target_group)
  options.merge!(
    role: "ecsServiceRole", # assumption that we're using the ecsServiceRole
    load_balancers: [
      {
        container_name: container[:name],
        container_port: container[:port],
        target_group_arn: target_group,
      }
    ]
  )
end

#cluster_arn ⇒ Object 



382
383
384
 
# File 'lib/ufo/ship.rb', line 382

def cluster_arn
  @cluster_arn ||= ecs_clusters.first.cluster_arn
end

#container_info(task_definition) ⇒ Object

assume only 1 container_definition assume only 1 port mapping in that container_defintion



341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
 
# File 'lib/ufo/ship.rb', line 341

def container_info(task_definition)
  task_definition_path = "ufo/output/#{task_definition}.json"
  task_definition_full_path = "#{@project_root}/#{task_definition_path}"
  unless File.exist?(task_definition_full_path)
    puts "ERROR: Unable to find the task definition at #{task_definition_path}.".colorize(:red)
    puts "Are you sure you have defined it in ufo/template_definitions.rb?".colorize(:red)
    exit
  end
  task_definition = JSON.load(IO.read(task_definition_full_path))
  container_def = task_definition["containerDefinitions"].first
  mappings = container_def["portMappings"]
  if mappings
    map = mappings.first
    port = map["containerPort"]
  end
  {
    name: container_def["name"],
    port: port
  }
end

#create_service ⇒ Object

$ aws ecs create-service –generate-cli-skeleton {

"cluster": "",
"serviceName": "",
"taskDefinition": "",
"desiredCount": 0,
"loadBalancers": [
    {
        "targetGroupArn": "",
        "containerName": "",
        "containerPort": 0
    }
],
"role": "",
"clientToken": "",
"deploymentConfiguration": {
    "maximumPercent": 0,
    "minimumHealthyPercent": 0
}

}

If the service needs to be created it will get created with some default settings. When does a normal deploy where an update happens only the only thing that ufo will update is the task_definition. The other settings should normally be updated with the ECS console. ‘ufo scale` will allow you to updated the desired_count from the CLI though.



229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
 
# File 'lib/ufo/ship.rb', line 229

def create_service
  puts "This service #{@service} does not yet exist in the #{@cluster} cluster.  This deploy will create it."
  container = container_info(@task_definition)
  target_group = target_group_prompt(container)

  message = "#{@service} service created on #{@cluster} cluster"
  if @options[:noop]
    message = "NOOP #{message}"
  else
    options = {
      cluster: @cluster,
      service_name: @service,
      desired_count: default_desired_count,
      deployment_configuration: {
        maximum_percent: default_maximum_percent,
        minimum_healthy_percent: default_minimum_healthy_percent
      },
      task_definition: @task_definition
    }
    unless target_group.nil? || target_group.empty?
      add_load_balancer!(container, options, target_group)
    end
    response = ecs.create_service(options)
    service = response.service # must set service here since this might never be called if @wait_for_deployment is false
  end
  puts message unless @options[:mute]
  service
end

#deploy ⇒ Object

If it looks like a regexp is passed in then it’ll only update the services This is because regpex cannot be used to determined a list of service_names.

Example:

No way to map: hi-.*-prod -> hi-web-prod hi-worker-prod hi-clock-prod



52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
 
# File 'lib/ufo/ship.rb', line 52

def deploy
  message = "Shipping #{@service}..."
  unless @options[:mute]
    if @options[:noop]
      puts "NOOP: #{message}"
      return
    else
      puts message.green
    end
  end

  ensure_cluster_exist
  process_single_service

  puts "Software shipped!" unless @options[:mute]
end

#deployment_complete(deployed_service) ⇒ Object

aws ecs describe-services –services hi-web-prod –cluster prod-hi Passing in the service because we need to capture the deployed task_definition that was actually deployed. We use it to pull the describe_services until all the paramters we expect upon a completed deployment are updated.



188
189
190
191
192
193
194
195
196
197
198
199
200
201
 
# File 'lib/ufo/ship.rb', line 188

def deployment_complete(deployed_service)
  deployed_task_definition = deployed_service.task_definition # want the stale task_definition out of the wa
  service = find_updated_service(deployed_service) # polling
  deployment = service.deployments.first
  # Edge case when another deploy superseds this deploy in this case break out of this loop
  deployed_task_version = task_version(deployed_task_definition)
  current_task_version = task_version(service.task_definition)
  if current_task_version > deployed_task_version
    raise ShipmentOverridden.new("deployed_task_version was #{deployed_task_version} but task_version is now #{current_task_version}")
  end

  (deployment.task_definition == deployed_task_definition &&
   deployment.desired_count == deployment.running_count)
end

#ecs_clusters ⇒ Object 



399
400
401
 
# File 'lib/ufo/ship.rb', line 399

def ecs_clusters
  ecs.describe_clusters(clusters: [@cluster]).clusters
end

#ensure_cluster_exist ⇒ Object 



386
387
388
389
390
391
392
393
394
395
396
397
 
# File 'lib/ufo/ship.rb', line 386

def ensure_cluster_exist
  cluster_exist = ecs_clusters.first
  unless cluster_exist
    message = "#{@cluster} cluster created."
    if @options[:noop]
      message = "NOOP #{message}"
    else
      ecs.create_cluster(cluster_name: @cluster)
    end
    puts message unless @options[:mute]
  end
end

#find_all_ecs_services ⇒ Object

find all services on a cluster yields ECS::Service object



368
369
370
371
372
373
374
375
376
 
# File 'lib/ufo/ship.rb', line 368

def find_all_ecs_services
  ecs_services = []
  service_arns.each do |service_arn|
    ecs_service = ECS::Service.new(cluster_arn, service_arn)
    yield(ecs_service) if block_given?
    ecs_services << ecs_service
  end
  ecs_services
end

#find_ecs_service ⇒ Object 



362
363
364
 
# File 'lib/ufo/ship.rb', line 362

def find_ecs_service
  find_all_ecs_services.find { |ecs_service| ecs_service.service_name == @service }
end

#find_updated_service(service) ⇒ Object

used for polling must pass in a service and cannot use @service for the case of multi_services mode



179
180
181
 
# File 'lib/ufo/ship.rb', line 179

def find_updated_service(service)
  ecs.describe_services(services: [service.service_name], cluster: @cluster).services.first
end

#old_task?(deployed_task_definition_arn, task_definition_arn) ⇒ Boolean 



91
92
93
94
95
96
97
98
99
100
101
 
# File 'lib/ufo/ship.rb', line 91

def old_task?(deployed_task_definition_arn, task_definition_arn)
  puts "deployed_task_definition_arn: #{deployed_task_definition_arn.inspect}"
  puts "task_definition_arn: #{task_definition_arn.inspect}"
  deployed_version = deployed_task_definition_arn.split(':').last.to_i
  version = task_definition_arn.split(':').last.to_i
  puts "deployed_version #{deployed_version.inspect}"
  puts "version #{version.inspect}"
  is_old = version < deployed_version
  puts "is_old #{is_old.inspect}"
  is_old
end

#process_single_service ⇒ Object

A single service name shouold had been passed and the service automatically gets created if it does not exist.



71
72
73
74
75
76
77
78
79
80
81
82
83
 
# File 'lib/ufo/ship.rb', line 71

def process_single_service
  ecs_service = find_ecs_service
  deployed_service = if ecs_service
                       # update all existing service
                       update_service(ecs_service)
                     else
                       # create service on the first cluster
                       create_service
                     end

  wait_for_deployment(deployed_service) if @wait_for_deployment && !@options[:noop]
  stop_old_task(deployed_service) if @stop_old_tasks
end

#service_arns ⇒ Object 



378
379
380
 
# File 'lib/ufo/ship.rb', line 378

def service_arns
  ecs.list_services(cluster: @cluster).service_arns
end

#service_tasks(cluster, service) ⇒ Object 



85
86
87
88
89
 
# File 'lib/ufo/ship.rb', line 85

def service_tasks(cluster, service)
  all_task_arns = ecs.list_tasks(cluster: cluster, service_name: service).task_arns
  return [] if all_task_arns.empty?
  ecs.describe_tasks(cluster: cluster, tasks: all_task_arns).tasks
end

#stop_old_task(deployed_service) ⇒ Object

aws ecs list-tasks –cluster prod-hi –service-name gr-web-prod aws ecs describe-tasks –tasks arn:aws:ecs:us-east-1:467446852200:task/09038fd2-f989-4903-a8c6-1bc41761f93f –cluster prod-hi



111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
 
# File 'lib/ufo/ship.rb', line 111

def stop_old_task(deployed_service)
  deployed_task_definition_arn = deployed_service.task_definition
  puts "deployed_task_definition_arn #{deployed_task_definition_arn.inspect}"

  # cannot use @serivce because of multiple mode
  all_tasks = service_tasks(@cluster, deployed_service.service_name)
  old_tasks = all_tasks.select do |task|
    old_task?(deployed_task_definition_arn, task.task_definition_arn)
  end

  reason = "Ufo #{Ufo::VERSION} has deployed new code and waited until the newer code is running."
  puts reason
  # Stopping old tasks after we have confirmed that the new task definition has the same
  # number of desired_count and running_count speeds up clean up and ensure that we
  # dont have any stale code being served.  It seems to take a long time for the
  # ELB to drain the register container otherwise. This might cut off some requests but
  # providing this as an option that can be turned of beause I've seen deploys go way too
  # slow.
  puts "@options[:stop_old_tasks] #{@options[:stop_old_tasks].inspect}"
  puts "old_tasks.size #{old_tasks.size}"
  old_tasks.each do |task|
    puts "stopping task.task_definition_arn #{task.task_definition_arn.inspect}"
    ecs.stop_task(cluster: @cluster, task: task.task_arn, reason: reason)
  end if @options[:stop_old_tasks]
end

#stop_old_tasks(services) ⇒ Object 



103
104
105
106
107
 
# File 'lib/ufo/ship.rb', line 103

def stop_old_tasks(services)
  services.each do |service|
    stop_old_task(service)
  end
end

#target_group_prompt(container) ⇒ Object

Returns the target_group. Will only allow an target_group and the service to use a load balancer if the container name is “web”.



305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
 
# File 'lib/ufo/ship.rb', line 305

def target_group_prompt(container)
  return if @options[:noop]
  # If a target_group is provided at the CLI return it right away.
  return @options[:target_group] if @options[:target_group]
  # Allows skipping the target group prompt.
  return unless @target_group_prompt

  # If the container name is web then it is assume that this is a web service that
  # needs a target group/elb.
  return unless container[:name] == 'web'

  puts "Would you like this service to be associated with an Application Load Balancer?"
  puts "If yes, please provide the Application Load Balancer Target Group ARN."
  puts "If no, simply press enter."
  print "Target Group ARN: "

  arn = $stdin.gets.strip
  until arn == '' or validate_target_group(arn)
    puts "You have provided an invalid Application Load Balancer Target Group ARN: #{arn}."
    puts "It should be in the form: arn:aws:elasticloadbalancing:us-east-1:123456789:targetgroup/target-name/2378947392743"
    puts "Please try again or skip adding a Target Group by just pressing enter."
    print "Target Group ARN: "
    arn = $stdin.gets.strip
  end
  arn
end

#task_name(task_definition) ⇒ Object 



403
404
405
406
407
408
 
# File 'lib/ufo/ship.rb', line 403

def task_name(task_definition)
  # "arn:aws:ecs:us-east-1:123456789:task-definition/hi-web-prod:72"
  #   ->
  # "task-definition/hi-web-prod:72"
  task_definition.split('/').last
end

#task_version(task_definition) ⇒ Object 



410
411
412
413
 
# File 'lib/ufo/ship.rb', line 410

def task_version(task_definition)
  # "task-definition/hi-web-prod:72" -> 72
  task_name(task_definition).split(':').last.to_i
end

#update_service(ecs_service) ⇒ Object

$ aws ecs update-service –generate-cli-skeleton {

"cluster": "",
"service": "",
"taskDefinition": "",
"desiredCount": 0,
"deploymentConfiguration": {
    "maximumPercent": 0,
    "minimumHealthyPercent": 0
}

} Only thing we want to change is the task-definition



270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
 
# File 'lib/ufo/ship.rb', line 270

def update_service(ecs_service)
  message = "#{ecs_service.service_name} service updated on #{ecs_service.cluster_name} cluster with task #{@task_definition}"
  if @options[:noop]
    message = "NOOP #{message}"
  else
    response = ecs.update_service(
      cluster: ecs_service.cluster_arn, # can use the cluster name also since it is unique
      service: ecs_service.service_arn, # can use the service name also since it is unique
      task_definition: @task_definition
    )
    service = response.service # must set service here since this might never be called if @wait_for_deployment is false
  end
  puts message unless @options[:mute]
  service
end

#validate_target_group(arn) ⇒ Object 



332
333
334
335
336
337
 
# File 'lib/ufo/ship.rb', line 332

def validate_target_group(arn)
  elb.describe_target_groups(target_group_arns: [arn])
  true
rescue Aws::ElasticLoadBalancingV2::Errors::ValidationError
  false
end

#wait_for_all_deployments(deployed_services) ⇒ Object 



158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
 
# File 'lib/ufo/ship.rb', line 158

def wait_for_all_deployments(deployed_services)
  start_time = Time.now
  threads = deployed_services.map do |deployed_service|
    Thread.new do
      # http://stackoverflow.com/questions/1383390/how-can-i-return-a-value-from-a-thread-in-ruby
      Thread.current[:output] = wait_for_deployment(deployed_service, quiet=true)
    end
  end
  threads.each { |t| t.join }
  total_took = Time.now - start_time
  puts ""
  puts "Shipments for all #{deployed_service.size} services took a total of #{pretty_time(total_took).green}."
  puts "Each deployment took:"
  threads.each do |t|
    service_name, took = t[:output]
    puts "  #{service_name}: #{pretty_time(took)}"
  end
end

#wait_for_deployment(deployed_service, quiet = false) ⇒ Object

service is the returned object from aws-sdk not the @service which is just a String. Returns [service_name, time_took]



139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
 
# File 'lib/ufo/ship.rb', line 139

def wait_for_deployment(deployed_service, quiet=false)
  start_time = Time.now
  deployed_task_name = task_name(deployed_service.task_definition)
  puts "Waiting for deployment of task definition #{deployed_task_name.green} to complete" unless quiet
  begin
    until deployment_complete(deployed_service)
      print '.'
      sleep 5
    end
  rescue ShipmentOverridden => e
    puts "This deployed was overridden by another deploy"
    puts e.message
  end
  puts '' unless quiet
  took = Time.now - start_time
  puts "Time waiting for ECS deployment: #{pretty_time(took).green}." unless quiet
  [deployed_service.service_name, took]
end