Class: TFWrapper::RakeTasks

Inherits:
Object
  • Object
show all
Includes:
Rake::DSL
Defined in:
lib/tfwrapper/raketasks.rb

Overview

Generates Rake tasks for working with Terraform at Manheim.

Before using this, the CONSUL_HOST environment variable must be set.

NOTE: Be sure to document all tasks in README.md

Class Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(tf_dir, opts = {}) ⇒ RakeTasks

Generate Rake tasks for working with Terraform at Manheim.

Parameters:

  • tf_dir (String) —

    Terraform config directory, relative to Rakefile. Set to ‘.’ if the Rakefile is in the same directory as the .tf configuration files.

  • options (Hash) —

    to use when adding tasks

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :backend_config (Hash) —

    hash of Terraform remote state backend configuration options, to override or supplement those in the terraform configuration. See the [Remote State](www.terraform.io/docs/state/remote.html) documentation for further information.

  • :namespace_prefix (String) —

    if specified and not nil, this will put all tasks in a “#namespace_prefix_tf:” namespace instead of “tf:”. This allows using manheim_helpers for multiple terraform configurations in the same Rakefile.

  • :tf_vars_from_env (Hash) —

    hash of Terraform variables to the (required) environment variables to populate their values from

  • :tf_extra_vars (Hash) —

    hash of Terraform variables to their values; overrides any same-named keys in tf_vars_from_env

  • :consul_url (String) —

    URL to access Consul at, for the :consul_env_vars_prefix option.

  • :consul_env_vars_prefix (String) —

    if specified and not nil, write the environment variables used from tf_vars_from_env and their values to JSON at this path in Consul. This should have the same naming constraints as consul_prefix.



59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
 
# File 'lib/tfwrapper/raketasks.rb', line 59

def initialize(tf_dir, opts = {})
  # find the directory that contains the Rakefile
  rakedir = File.realpath(Rake.application.rakefile)
  rakedir = File.dirname(rakedir) if File.file?(rakedir)
  @tf_dir = File.realpath(File.join(rakedir, tf_dir))
  @ns_prefix = opts.fetch(:namespace_prefix, nil)
  @consul_env_vars_prefix = opts.fetch(:consul_env_vars_prefix, nil)
  @tf_vars_from_env = opts.fetch(:tf_vars_from_env, {})
  @tf_extra_vars = opts.fetch(:tf_extra_vars, {})
  @backend_config = opts.fetch(:backend_config, {})
  @consul_url = opts.fetch(:consul_url, nil)
  # rubocop:disable Style/GuardClause
  if @consul_url.nil? && !@consul_env_vars_prefix.nil?
    raise StandardError, 'Cannot set env vars in Consul when consul_url ' \
      'option is nil.'
  end
  # rubocop:enable Style/GuardClause
end

Class Attribute Details

.instance ⇒ Object

set when installed



20
21
22
 
# File 'lib/tfwrapper/raketasks.rb', line 20

def instance
  @instance
end

Class Method Details

.install_tasks(tf_dir, opts = {}) ⇒ Object

Install the Rake tasks for working with Terraform at Manheim.

Parameters:

  • tf_dir (String) —

    Terraform config directory, relative to Rakefile. Set to ‘.’ if the Rakefile is in the same directory as the .tf configuration files.

  • options (Hash) —

    to use when adding tasks

  • opts (Hash) (defaults to: {}) —

    a customizable set of options



25
26
27
 
# File 'lib/tfwrapper/raketasks.rb', line 25

def install_tasks(tf_dir, opts = {})
  new(tf_dir, opts).install
end

Instance Method Details

#check_tf_version ⇒ Object

Check that the terraform version is compatible



284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
 
# File 'lib/tfwrapper/raketasks.rb', line 284

def check_tf_version
  # run: terraform -version
  all_out_err, exit_status = TFWrapper::Helpers.run_cmd_stream_output(
    'terraform version', @tf_dir
  )
  unless exit_status.zero?
    raise StandardError, "ERROR: 'terraform -version' exited " \
      "#{exit_status}: #{all_out_err}"
  end
  all_out_err = all_out_err.strip
  # Find the terraform version string
  m = /Terraform v(\d+\.\d+\.\d+).*/.match(all_out_err)
  unless m
    raise StandardError, 'ERROR: could not determine terraform version ' \
      "from 'terraform -version' output: #{all_out_err}"
  end
  # the version will be a string like:
  # Terraform v0.9.2
  # or:
  # Terraform v0.9.3-dev (<GIT SHA><+CHANGES>)
  tf_ver = Gem::Version.new(m[1])
  unless tf_ver >= min_tf_version
    raise StandardError, "ERROR: tfwrapper #{TFWrapper::VERSION} is only " \
      "compatible with Terraform >= #{min_tf_version} but your terraform " \
      "binary reports itself as #{m[1]} (#{all_out_err})"
  end
  puts "Running with: #{all_out_err}"
end

#cmd_with_targets(cmd_array, target, extras) ⇒ Object

Create a terraform command line with optional targets specified; targets are inserted between cmd_array and suffix_array.

This is intended to simplify parsing Rake task arguments and inserting them into the command as targets; to get a Rake task to take a variable number of arguments, we define a first argument (:target) which is either a String or nil. Any additional arguments specified end up in args.extras, which is either nil or an Array of additional String arguments.

Parameters:

  • cmd_array (Array) —

    array of the beginning parts of the terraform command; usually something like:

    ‘terraform’, ‘ACTION’, ‘-var’file’, ‘VAR_FILE_PATH’
  • target (String) —

    the first target parameter given to the Rake task, or nil.

  • extras (Array) —

    array of additional target parameters given to the Rake task, or nil.



348
349
350
351
352
353
354
355
 
# File 'lib/tfwrapper/raketasks.rb', line 348

def cmd_with_targets(cmd_array, target, extras)
  final_arr = cmd_array
  final_arr.concat(['-target', target]) unless target.nil?
  # rubocop:disable Style/SafeNavigation
  extras.each { |e| final_arr.concat(['-target', e]) } unless extras.nil?
  # rubocop:enable Style/SafeNavigation
  final_arr.join(' ')
end

#install ⇒ Object

install all Rake tasks - calls other install_* methods rubocop:disable Metrics/CyclomaticComplexity



88
89
90
91
92
93
94
95
 
# File 'lib/tfwrapper/raketasks.rb', line 88

def install
  install_init
  install_plan
  install_apply
  install_refresh
  install_destroy
  install_write_tf_vars
end

#install_apply ⇒ Object

add the ‘tf:apply’ Rake task



140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
 
# File 'lib/tfwrapper/raketasks.rb', line 140

def install_apply
  namespace nsprefix do
    desc 'Apply a terraform plan that will provision your resources; ' \
      'specify optional CSV targets'
    task :apply, [:target] => [
      :"#{nsprefix}:init",
      :"#{nsprefix}:write_tf_vars",
      :"#{nsprefix}:plan"
    ] do |_t, args|
      cmd = cmd_with_targets(
        ['terraform', 'apply', "-var-file #{var_file_path}"],
        args[:target],
        args.extras
      )
      terraform_runner(cmd)

      update_consul_stack_env_vars unless @consul_env_vars_prefix.nil?
    end
  end
end

#install_destroy ⇒ Object

add the ‘tf:destroy’ Rake task



180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
 
# File 'lib/tfwrapper/raketasks.rb', line 180

def install_destroy
  namespace nsprefix do
    desc 'Destroy any live resources that are tracked by your state ' \
      'files; specify optional CSV targets'
    task :destroy, [:target] => [
      :"#{nsprefix}:init",
      :"#{nsprefix}:write_tf_vars"
    ] do |_t, args|
      cmd = cmd_with_targets(
        ['terraform', 'destroy', '-force', "-var-file #{var_file_path}"],
        args[:target],
        args.extras
      )

      terraform_runner(cmd)
    end
  end
end

#install_init ⇒ Object

add the ‘tf:init’ Rake task. This checks environment variables, runs “terraform -version“, and then runs “terraform init“ with the backend_config options, if any.



100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/tfwrapper/raketasks.rb', line 100

def install_init
  namespace nsprefix do
    desc 'Run terraform init with appropriate arguments'
    task :init do
      TFWrapper::Helpers.check_env_vars(@tf_vars_from_env.values)
      check_tf_version
      cmd = [
        'terraform',
        'init',
        '-input=false'
      ].join(' ')
      @backend_config.each do |k, v|
        cmd = cmd + ' ' + "-backend-config='#{k}=#{v}'"
      end
      terraform_runner(cmd)
    end
  end
end

#install_plan ⇒ Object

add the ‘tf:plan’ Rake task



120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
 
# File 'lib/tfwrapper/raketasks.rb', line 120

def install_plan
  namespace nsprefix do
    desc 'Output the set plan to be executed by apply; specify ' \
      'optional CSV targets'
    task :plan, [:target] => [
      :"#{nsprefix}:init",
      :"#{nsprefix}:write_tf_vars"
    ] do |_t, args|
      cmd = cmd_with_targets(
        ['terraform', 'plan', "-var-file #{var_file_path}"],
        args[:target],
        args.extras
      )

      terraform_runner(cmd)
    end
  end
end

#install_refresh ⇒ Object

add the ‘tf:refresh’ Rake task



162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
 
# File 'lib/tfwrapper/raketasks.rb', line 162

def install_refresh
  namespace nsprefix do
    task refresh: [
      :"#{nsprefix}:init",
      :"#{nsprefix}:write_tf_vars"
    ] do
      cmd = [
        'terraform',
        'refresh',
        "-var-file #{var_file_path}"
      ].join(' ')

      terraform_runner(cmd)
    end
  end
end

#install_write_tf_vars ⇒ Object

add the ‘tf:write_tf_vars’ Rake task



208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
 
# File 'lib/tfwrapper/raketasks.rb', line 208

def install_write_tf_vars
  namespace nsprefix do
    desc "Write #{var_file_path}"
    task :write_tf_vars do
      tf_vars = terraform_vars
      puts 'Terraform vars:'
      tf_vars.sort.map do |k, v|
        if k == 'aws_access_key' || k == 'aws_secret_key'
          puts "#{k} => (redacted)"
        else
          puts "#{k} => #{v}"
        end
      end
      File.open(var_file_path, 'w') do |f|
        f.write(tf_vars.to_json)
      end
      STDERR.puts "Terraform vars written to: #{var_file_path}"
    end
  end
end

#min_tf_version ⇒ Object 



30
31
32
 
# File 'lib/tfwrapper/raketasks.rb', line 30

def min_tf_version
  Gem::Version.new('0.9.0')
end

#nsprefix ⇒ Object 



78
79
80
81
82
83
84
 
# File 'lib/tfwrapper/raketasks.rb', line 78

def nsprefix
  if @ns_prefix.nil?
    'tf'.to_sym
  else
    "#{@ns_prefix}_tf".to_sym
  end
end

#terraform_runner(cmd) ⇒ Object

Run a Terraform command, providing some useful output and handling AWS API rate limiting gracefully. Raises StandardError on failure. The command is run in @tf_dir.

rubocop:disable Metrics/PerceivedComplexity

Parameters:

  • cmd (String) —

    Terraform command to run



242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
 
# File 'lib/tfwrapper/raketasks.rb', line 242

def terraform_runner(cmd)
  require 'retries'
  STDERR.puts "terraform_runner command: '#{cmd}' (in #{@tf_dir})"
  out_err = nil
  status = nil
  # exponential backoff as long as we're getting 403s
  handler = proc do |exception, attempt_number, total_delay|
    STDERR.puts "terraform_runner failed with #{exception}; retry " \
      "attempt #{attempt_number}; #{total_delay} seconds have passed."
  end
  status = -1
  with_retries(
    max_tries: 5,
    handler: handler,
    base_sleep_seconds: 1.0,
    max_sleep_seconds: 10.0
  ) do
    # this streams STDOUT and STDERR as a combined stream,
    # and also captures them as a combined string
    out_err, status = TFWrapper::Helpers.run_cmd_stream_output(cmd, @tf_dir)
    if status != 0 && out_err.include?('hrottling')
      raise StandardError, 'Terraform hit AWS API rate limiting'
    end
    if status != 0 && out_err.include?('status code: 403')
      raise StandardError, 'Terraform command got 403 error - access ' \
        'denied or credentials not propagated'
    end
    if status != 0 && out_err.include?('status code: 401')
      raise StandardError, 'Terraform command got 401 error - access ' \
        'denied or credentials not propagated'
    end
  end
  # end exponential backoff
  unless status.zero?
    raise StandardError, "Errors have occurred executing: '#{cmd}' " \
      "(exited #{status})"
  end
  STDERR.puts "terraform_runner command '#{cmd}' finished and exited 0"
end

#terraform_vars ⇒ Object 



229
230
231
232
233
234
 
# File 'lib/tfwrapper/raketasks.rb', line 229

def terraform_vars
  res = {}
  @tf_vars_from_env.each { |tfname, envname| res[tfname] = ENV[envname] }
  @tf_extra_vars.each { |name, val| res[name] = val }
  res
end

#update_consul_stack_env_vars ⇒ Object

update stack status in Consul



314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
 
# File 'lib/tfwrapper/raketasks.rb', line 314

def update_consul_stack_env_vars
  require 'diplomat'
  require 'json'
  data = {}
  @tf_vars_from_env.values.each { |k| data[k] = ENV[k] }

  Diplomat.configure do |config|
    config.url = @consul_url
  end

  puts "Writing stack information to #{@consul_url} at: "\
    "#{@consul_env_vars_prefix}"
  puts JSON.pretty_generate(data)
  raw = JSON.generate(data)
  Diplomat::Kv.put(@consul_env_vars_prefix, raw)
end

#var_file_path ⇒ Object 



199
200
201
202
203
204
205
 
# File 'lib/tfwrapper/raketasks.rb', line 199

def var_file_path
  if @ns_prefix.nil?
    File.absolute_path('build.tfvars.json')
  else
    File.absolute_path("#{@ns_prefix}_build.tfvars.json")
  end
end