Class: Ufo::CLI::Ps::Errors

Inherits:
Ufo::CLI::Ps show all
Extended by:
Memoist
Defined in:
lib/ufo/cli/ps/errors.rb

Instance Attribute Summary

Attributes inherited from Base

#task_definition

Instance Method Summary collapse

Methods inherited from Ufo::CLI::Ps

#convert_to_task_objects, #run, #scalable_target, #show_errors, #show_tasks, #summary, #tasks_counts

Methods included from Ufo::Concerns::Autoscaling

#autoscaling, #autoscaling_enabled?

Methods included from AwsServices

#acm, #applicationautoscaling, #aws_options, #cfn, #cloudwatchlogs, #ec2, #ecr, #ecs, #elb, #s3, #ssm_client, #waf_client

Methods included from AwsServices::Concerns

#find_stack, #find_stack_resources, #stack_resources, #status, #task_definition_arns

Methods included from Utils::Sure

#sure?

Methods included from Utils::Pretty

#pretty_home, #pretty_path, #pretty_time

Methods included from Utils::Logging

#logger

Methods included from Ufo::Concerns

#build, #deploy, #info, #ps

Methods included from Ufo::Concerns::Names

#names

Constructor Details

#initialize(options = {}) ⇒ Errors

Returns a new instance of Errors.



5
6
7
8
 
# File 'lib/ufo/cli/ps/errors.rb', line 5

def initialize(options={})
  super
  @tasks = options[:tasks]
end

Instance Method Details

#catchall ⇒ Object

Example:

(service app1-web-dev-EcsService-8FMliG8m6M2p) was unable to stop or start tasks during a deployment because of the service deployment configuration. Update the minimumHealthyPercent or maximumPercent value and try again.



131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
 
# File 'lib/ufo/cli/ps/errors.rb', line 131

def catchall
  words = %w[fail unable error]
  recent_messages = recent_events.map(&:message)
  message = recent_messages.find  do |message|
    words.detect { |word| message.include?(word) }
  end

  return unless message
  logger.error "ERROR: #{message}".color(:red)

  logger.error <<~EOL
    You might have to #{cancel_actions[:cfn]} the stack with:

        ufo #{cancel_actions[:ufo]}

    And try again after fixing the issue.
  EOL
end

#deployment_configuration ⇒ Object

To reproduce

.ufo/config.rb

Ufo.configure do |config|
  config.ecs.maximum_percent = 150 # need at least 200 to go from 1 to 2 containers
  config.ecs.minimum_healthy_percent = 100
end

Event message error:

ERROR: (service app1-web-dev-EcsService-8FMliG8m6M2p) was unable to stop or start tasks during a deployment because of the service deployment configuration. Update the minimumHealthyPercent or maximumPercent value and try again.



77
78
79
80
81
82
83
84
85
86
87
88
 
# File 'lib/ufo/cli/ps/errors.rb', line 77

def deployment_configuration
  message = recent_message
  return unless message.include?("unable") && message.include?("deployment configuration")

  logger.error "ERROR: Deployment Configuration".color(:red)
  logger.error <<~EOL
    You might have a Deployment Configuration that prevents the deployment from completing.

    See: https://ufoships.com/docs/debug/deployment-configuration/

  EOL
end

#scale ⇒ Object

If running count < desired account for a long time And see was unable to place a task Probably not enough capacity



25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
 
# File 'lib/ufo/cli/ps/errors.rb', line 25

def scale
  return if service.running_count >= service.desired_count

  error_event = recent_events.find do |e|
    e.message =~ /was unable to place a task/
  end
  return unless error_event

  logger.info "There is an issue scaling the #{@stack_name.color(:green)} service to #{service.desired_count}.  Here's the error:"
  logger.info error_event.message.color(:red)
  if service.launch_type == "EC2"
    logger.info <<~EOL
      If AutoScaling is set up for the container instances,
      it can take a little time to add additional instances.
      You'll see this message until the capacity is added.
    EOL
  end
end

#show ⇒ Object 



10
11
12
13
14
15
16
17
18
19
20
 
# File 'lib/ufo/cli/ps/errors.rb', line 10

def show
  message = recent_message
  return unless message
  return if message =~ /has reached a steady state/

  scale
  target_group
  deployment_configuration
  wrong_vpc
  catchall
end

#target_group ⇒ Object

The error currently happens to be the 5th element.

Example:

(service XXX) (instance i-XXX) (port 32875) is unhealthy in (target-group arn:aws:elasticloadbalancing:us-east-1:111111111111:targetgroup/devel-Targe-1111111111111/1111111111111111) due to (reason Health checks failed with these codes: [400])">]



48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
 
# File 'lib/ufo/cli/ps/errors.rb', line 48

def target_group
  error_event = recent_events.find do |e|
    e.message =~ /is unhealthy in/ &&
    e.message =~ /targetgroup/
  end
  return unless error_event

  logger.error "There are targets in the target group reporting unhealthy.  This can cause containers to cycle. Here's the error:"
  logger.error error_event.message.color(:red)
  logger.error <<~EOL
    Check out the ECS console and EC2 Load Balancer console for more info.
    Sometimes they may not helpful :(
    Docs that may help: https://ufoships.com/docs/debug/unhealthy-targets/
  EOL
end

#wrong_vpc ⇒ Object

To reproduce #1

1. Deploy to with settings where ECS cluster is in custom VPC successfully
2. Deploy again. Accidentally with default VPC settings <= Reproduction

This will produce a CloudFormation stack failure

> All subnets must belong to the same VPC: ‘vpc-11111111’ (Service: AmazonElasticLoadBalancing; Status Code: 400; Error Code: InvalidConfigurationRequest; Request ID: b8c683ca-4c6d-4bf9-bf9b-3eb468fa9ea9; Proxy: null)

So it’s not actually an ECS failure and is caught early on. Notiing it for posterity.

To reproduce #2

Deploy to default VPC. Even though ECS cluster is running on a custom VPC <= Reproduction

This reproduces:

> ERROR: (service demo-web-dev-EcsService-RkMBAhHBfx9A) failed to register targets in (target-group arn:aws:elasticloadbalancing:us-west-2:111111111111:targetgroup/demo-Targe-1HEN2QPS5LO9B/0c69c3eb5aa23bc9) with (error The following targets are not in the target group VPC ‘vpc-11111111’: ‘i-11111111111111111’)

The first deploy suceeeds because CloudFormation doesn’t check on the ECS service as much here. ECS does report the error though.



112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
 
# File 'lib/ufo/cli/ps/errors.rb', line 112

def wrong_vpc
  error_event = recent_events.find do |e|
    e.message =~ /targets are not in the target group VPC/ ||
    e.message =~ /All subnets must belong to the same VPC/
  end
  return unless error_event

  logger.info "ERROR: VPC Configuration error".color(:red)
  logger.info error_event.message.color(:red)
  logger.info <<~EOL
    It seems like the ECS Service was deployed to an ECS Cluster running on
    a different VPC than what's the ECS Service is configured with.

    See: https://ufoships.com/docs/debug/vpc-subnets/
  EOL
end