Class: AppEngine::Exec

Inherits:
Object
  • Object
show all
Defined in:
lib/appengine/exec.rb

Overview

App Engine remote execution

This class provides a client for App Engine remote execution, allowing App Engine applications to perform on-demand tasks in the App Engine environment. This may be used for safe running of ops and maintenance tasks, such as database migrations, that access production cloud resources.

About App Engine execution

App Engine execution spins up an image of a deployed App Engine app, and runs a command in that image. For example, if your app runs on Ruby on Rails, then your app provides a bin/rails tool, and you may invoke it using App Engine execution---for example to run a command such as bundle exec bin/rails db:migrate in the image.

When App Engine execution runs your command, it provides access to key elements of the App Engine environment, including:

  • The same runtime that runs your application in App Engine itself.
  • Any Cloud SQL connections requested by your application.
  • Any environment variables set by your application.

The command runs on virtual machines provided by Google Cloud Container Builder, and has access to the credentials of the Cloud Container Builder service account.

Prerequisites

To use App Engine remote execution, you will need:

You may also need to grant the Cloud Container Builder service account any permissions needed by your command. Often, Project Editor permissions will be sufficient for most tasks. You can find the service account configuration in the IAM tab in the Cloud Console under the name [your-project-number]@cloudbuild.gserviceaccount.com.

You may use the AppEngine::Exec class to run commands directly. However, in most cases, it will be easier to run commands via the provided rake tasks. See Tasks for more info.

Configuring

This class uses three parameters to specify which application image to use to run your command: service, config_path, and version.

In most cases, you can use the defaults. The Exec class will look in your current directory for a file called ./app.yaml which describes your App Engine service. It gets the service name from this file (or uses the "default" name if none is specified), then looks up the most recently created deployment version for that service. That deployment version then provides the application image that runs your command.

If your app has multiple services, you may specify which config file (other than ./app.yaml) describes the desired service, by providing the config_path parameter. Alternately, you may specify a service name directly by providing the service parameter. If you provide both parameters, service takes precedence.

Usually, App Engine execution uses the image for the most recently created version of the service. (Note: the most recently created version is used, regardless of whether that version is currently receiving traffic.) If you want to use the image for a different version, you may specify a version by providing the version parameter.

You may also provide a timeout, which is the length of time that App Engine execution will allow your command to run before it is considered to have stalled and is terminated. The timeout should be a string of the form 2h15m10s. The default is 10m.

Resource usage and billing

App Engine remote execution uses virtual machine resources provided by Google Cloud Container Builder. Generally, a certain number of usage minutes per day is covered under a free tier, but additional compute usage beyond that time is billed to your Google Cloud account. For more details, see https://cloud.google.com/container-builder/pricing

If your command makes API calls or utilizes other cloud resources, you may also be billed for that usage. However, remote execution does not use actual App Engine instances, and you will not be billed for additional App Engine instance usage.

Defined Under Namespace

Classes: BadConfigFileFormat, ConfigFileNotFound, NoSuchVersion, ServiceNameConflict, UsageError

Class Attribute Summary collapse

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(command, service: nil, config_path: nil, version: nil, timeout: nil) {|_self| ... } ⇒ Exec

Create an execution for the given command.

Parameters:

  • command (Array<String>) —

    The command in array form.

  • service (String, nil) (defaults to: nil) —

    Name of the service. If omitted, obtains the service name from the config file.

  • config_path (String, nil) (defaults to: nil) —

    App Engine config file to get the service name from if the service name is not provided directly. Defaults to the value of AppEngine::Exec.default_config_path.

  • version (String, nil) (defaults to: nil) —

    Version string. Defaults to the most recently created version of the given service (which may not be the one currently receiving traffic).

  • timeout (String, nil) (defaults to: nil) —

    Timeout string. Defaults to the value of AppEngine::Exec.default_timeout.

Yields:

  • (_self)

Yield Parameters:



250
251
252
253
254
255
256
257
258
259
260
 
# File 'lib/appengine/exec.rb', line 250

def initialize command,
               service: nil, config_path: nil, version: nil, timeout: nil
  @command = command
  @service = service
  @config_path = config_path
  @version = version
  @timeout = timeout
  @wrapper_image = nil

  yield self if block_given?
end

Class Attribute Details

.default_config_path ⇒ String

Returns Path to default config file.

Returns:

  • (String) —

    Path to default config file.



193
194
195
 
# File 'lib/appengine/exec.rb', line 193

def default_config_path
  @default_config_path
end

.default_service ⇒ String

Returns Default service name if the config doesn't specify.

Returns:

  • (String) —

    Default service name if the config doesn't specify.



190
191
192
 
# File 'lib/appengine/exec.rb', line 190

def default_service
  @default_service
end

.default_timeout ⇒ String

Returns Default command timeout.

Returns:

  • (String) —

    Default command timeout.



187
188
189
 
# File 'lib/appengine/exec.rb', line 187

def default_timeout
  @default_timeout
end

.default_wrapper_image ⇒ String

Returns Docker image that implements the app engine wrapper.

Returns:

  • (String) —

    Docker image that implements the app engine wrapper.



196
197
198
 
# File 'lib/appengine/exec.rb', line 196

def default_wrapper_image
  @default_wrapper_image
end

Instance Attribute Details

#command ⇒ String+

Returns Command to run.

Returns:

  • (String, Array<String>) —

    Command to run.



276
277
278
 
# File 'lib/appengine/exec.rb', line 276

def command
  @command
end

#config_path ⇒ String?

Returns Path to the config file, or nil to use the default.

Returns:

  • (String, nil) —

    Path to the config file, or nil to use the default.



267
268
269
 
# File 'lib/appengine/exec.rb', line 267

def config_path
  @config_path
end

#service ⇒ String?

Returns The service name, or nil to read from the config.

Returns:

  • (String, nil) —

    The service name, or nil to read from the config.



264
265
266
 
# File 'lib/appengine/exec.rb', line 264

def service
  @service
end

#timeout ⇒ String?

Returns Command timeout, or nil to use the default.

Returns:

  • (String, nil) —

    Command timeout, or nil to use the default.



273
274
275
 
# File 'lib/appengine/exec.rb', line 273

def timeout
  @timeout
end

#version ⇒ String?

Returns Service version, or nil to use the most recent.

Returns:

  • (String, nil) —

    Service version, or nil to use the most recent.



270
271
272
 
# File 'lib/appengine/exec.rb', line 270

def version
  @version
end

#wrapper_image ⇒ String

Returns Custom wrapper image to use, or nil to use the default.

Returns:

  • (String) —

    Custom wrapper image to use, or nil to use the default.



279
280
281
 
# File 'lib/appengine/exec.rb', line 279

def wrapper_image
  @wrapper_image
end

Class Method Details

.new_rake_task(name, args: [], env_args: [], service: nil, config_path: nil, version: nil, timeout: nil) ⇒ Object

Create an execution for a rake task.

Parameters:

  • name (String) —

    Name of the task

  • args (Array<String>) (defaults to: []) —

    Args to pass to the task

  • env_args (Array<String>) (defaults to: []) —

    Environment variable settings, each of the form NAME=value.

  • service (String, nil) (defaults to: nil) —

    Name of the service. If omitted, obtains the service name from the config file.

  • config_path (String, nil) (defaults to: nil) —

    App Engine config file to get the service name from if the service name is not provided directly. Defaults to the value of AppEngine::Exec.default_config_path.

  • version (String, nil) (defaults to: nil) —

    Version string. Defaults to the most recently created version of the given service (which may not be the one currently receiving traffic).

  • timeout (String, nil) (defaults to: nil) —

    Timeout string. Defaults to the value of AppEngine::Exec.default_timeout.



216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
 
# File 'lib/appengine/exec.rb', line 216

def new_rake_task name, args: [], env_args: [],
                  service: nil, config_path: nil, version: nil,
                  timeout: nil
  escaped_args = args.map{ |arg|
    arg.gsub(/[,\[\]]/){ |m| "\\#{m}" }
  }
  if escaped_args.empty?
    name_with_args = name
  else
    name_with_args = "#{name}[#{escaped_args.join ','}]"
  end
  new ["bundle", "exec", "rake", name_with_args] + env_args,
      service: service, config_path: config_path, version: version,
      timeout: timeout
end

Instance Method Details

#start ⇒ Object

Executes the command synchronously. Streams the logs back to standard out and does not return until the command has completed or timed out.



286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
 
# File 'lib/appengine/exec.rb', line 286

def start
  resolve_parameters

  version_info = version_info @service, @version
  env_variables = version_info["envVariables"] || {}
  beta_settings = version_info["betaSettings"] || {}
  cloud_sql_instances = beta_settings["cloud_sql_instances"] || []
  image = version_info["deployment"]["container"]["image"]

  config = build_config command, image, env_variables, cloud_sql_instances
  file = ::Tempfile.new ["cloudbuild_", ".json"]
  begin
    ::JSON.dump config, file
    file.flush
    Util::Gcloud.execute [
        "container", "builds", "submit",
        "--no-source",
        "--config=#{file.path}",
        "--timeout=#{@timeout}"]
  ensure
    file.close!
  end
end