Class: Drydock::Project

Inherits:
Object
  • Object
show all
Defined in:
lib/drydock/project.rb

Constant Summary collapse

DEFAULT_OPTIONS =
{
  auto_remove: true,
  author: nil,
  cache: nil,
  event_handler: false,
  ignorefile: '.dockerignore'
}

Instance Method Summary collapse

Constructor Details

#initialize(build_opts = {}) ⇒ Project

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Create a new project. Do not use directly.

Parameters:

  • build_opts (Hash) (defaults to: {})

    Build-time options

Options Hash (build_opts):

  • :auto_remove (Boolean)

    Whether intermediate images created during the build of this project should be automatically removed.

  • :author (String)

    The default author field when an author is not provided explicitly with #author.

  • :cache (ObjectCaches::Base)

    An object cache manager.

  • :event_handler (#call)

    A handler that responds to a #call message with four arguments: [event, is_new, serial_no, event_type] most useful to override logging or

  • :chain (PhaseChain)

    A phase chain manager.

  • :ignorefile (String)

    The name of the ignore-file to load.



27
28
29
30
31
32
33
34
35
36
37
38
# File 'lib/drydock/project.rb', line 27

def initialize(build_opts = {})
  @chain   = build_opts.key?(:chain) && build_opts.delete(:chain).derive
  @plugins = {}

  @run_path = []
  @serial  = 0

  @build_opts = DEFAULT_OPTIONS.clone
  build_opts.each_pair { |key, value| set(key, value) }

  @stream_monitor = build_opts[:event_handler] ? StreamMonitor.new(build_opts[:event_handler]) : nil
end

Instance Method Details

#author(name: nil, email: nil) ⇒ Object

Set the author for commits. This is not an instruction, per se, and only takes into effect after instructions that cause a commit.

This instruction affects all instructions after it, but nothing before it.

At least one of name or email must be given. If one is provided, the other is optional.

If no author instruction is provided, the author field is left blank by default.

Parameters:

  • name (String) (defaults to: nil)

    The name of the author or maintainer of the image.

  • email (String) (defaults to: nil)

    The email of the author or maintainer.

Raises:

  • (InvalidInstructionArgumentError)

    when neither name nor email is provided



53
54
55
56
57
58
59
60
# File 'lib/drydock/project.rb', line 53

def author(name: nil, email: nil)
  if (name.nil? || name.empty?) && (email.nil? || name.empty?)
    raise InvalidInstructionArgumentError, 'at least one of `name:` or `email:` must be provided'
  end

  value = email ? "#{name} <#{email}>" : name.to_s
  set :author, value
end

#build_idObject

Retrieve the current build ID for this project. If no image has been built, returns the string '0'.



64
65
66
# File 'lib/drydock/project.rb', line 64

def build_id
  chain ? chain.serial : '0'
end

#cd(path = '/') { ... } ⇒ Object

Change directories for operations that require a directory.

Parameters:

  • path (String) (defaults to: '/')

    The path to change directories to.

Yields:

  • block containing instructions to run inside the new directory



72
73
74
75
76
77
# File 'lib/drydock/project.rb', line 72

def cd(path = '/', &blk)
  @run_path << path
  blk.call if blk
ensure
  @run_path.pop
end

#cmd(command) ⇒ Object

Set the command that is automatically executed by default when the image is run through the docker run command.

#cmd corresponds to the CMD Dockerfile instruction. This instruction does not run the command, but rather provides the default command to be run when the image is run without specifying a command.

As with the CMD Dockerfile instruction, the #cmd instruction has three forms:

  • ['executable', 'param1', 'param2', '...'], which would run the executable directly when the image is run;
  • ['param1', 'param2', '...'], which would pass the parameters to the executable provided in the #entrypoint instruction; or
  • 'executable param1 param2', which would run the executable inside a subshell.

The first two forms are preferred over the last one. See also #entrypoint to see how the instruction interacts with this one.

Parameters:

  • command (String, Array<String>)

    The command set to run. When a String is provided, the command is run inside a shell (/bin/sh). When an Array is given, the command is run as-is given.



102
103
104
105
106
107
108
109
110
111
112
# File 'lib/drydock/project.rb', line 102

def cmd(command)
  requires_from!(:cmd)
  log_step('cmd', command)

  unless command.is_a?(Array)
    command = ['/bin/sh', '-c', command.to_s]
  end

  chain.run("# CMD #{command.inspect}", command: command)
  self
end

#copy(source_path, target_path, chmod: false, no_cache: false, recursive: true) ⇒ Object

Copies files from source_path on the the build machine, into target_path in the container. This instruction automatically commits the result.

The copy instruction always respects the ignorefile.

When no_cache is true (also see parameter explanation below), then any instruction after #copy will also be rebuilt every time.

Parameters:

  • source_path (String)

    The source path on the build machine (where drydock is running) from which to copy files.

  • target_path (String)

    The target path inside the image to which to copy the files. This path must already exist before copying begins.

  • chmod (Integer, Boolean) (defaults to: false)

    When false (the default), the original file mode from its source file is kept when copying into the container. Otherwise, the mode provided (in integer octal form) will be used to override all file and directory modes.

  • no_cache (Boolean) (defaults to: false)

    When false (the default), the hash digest of the source path—taking into account all its files, directories, and contents—is used as the cache key. When true, the image is rebuilt every time.

  • recursive (Boolean) (defaults to: true)

    When true, then source_path is expected to be a directory, at which point all its contents would be recursively searched. When false, then source_path is expected to be a file.

Raises:



144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
# File 'lib/drydock/project.rb', line 144

def copy(source_path, target_path, chmod: false, no_cache: false, recursive: true)
  requires_from!(:copy)
  log_step('copy', source_path, target_path, chmod: (chmod ? sprintf('%o', chmod) : false))

  if source_path.start_with?('/')
    Drydock.logger.warn("#{source_path.inspect} is an absolute path; we recommend relative paths")
  end

  raise InvalidInstructionError, "#{source_path} does not exist" unless File.exist?(source_path)

  source_files = if File.directory?(source_path)
    FileManager.find(source_path, ignorefile, prepend_path: true, recursive: recursive)
  else
    [source_path]
  end
  source_files.sort!

  raise InvalidInstructionError, "#{source_path} is empty or does not match a path" if source_files.empty?

  buffer = StringIO.new
  log_info("Processing #{source_files.size} files in tree")
  TarWriter.new(buffer) do |tar|
    source_files.each do |source_file|
      File.open(source_file, 'r') do |input|
        stat = input.stat
        mode = chmod || stat.mode
        tar.add_entry(source_file, mode: stat.mode, mtime: stat.mtime) do |tar_file|
          tar_file.write(input.read)
        end
      end
    end
  end

  buffer.rewind
  digest = Digest::MD5.hexdigest(buffer.read)

  log_info("Tree digest is md5:#{digest}")
  chain.run("# COPY #{source_path} #{target_path} DIGEST #{digest}", no_cache: no_cache) do |container|
    target_stat = container.archive_head(target_path)

    # TODO(rpasay): cannot autocreate the target, because `container` here is already dead
    unless target_stat
      raise InvalidInstructionError, "Target path #{target_path.inspect} does not exist"
    end

    unless target_stat.directory?
      Drydock.logger.debug(target_stat)
      raise InvalidInstructionError, "Target path #{target_path.inspect} exists, but is not a directory in the container"
    end

    container.archive_put(target_path) do |output|
      buffer.rewind
      output.write(buffer.read)
    end
  end

  self
end

#derive(opts = {}, &blk) ⇒ Object

Derive a new project based on the current state of the current project. This instruction returns the new project that can be referred to elsewhere, and most useful when combined with other inter-project instructions, such as #import.

For example:

  from 'some-base-image'

  APP_ROOT = '/app'
  mkdir APP_ROOT

  # 1:
  ruby_build = derive {
    copy 'Gemfile', APP_ROOT
    run 'bundle install --path vendor'
  }

  # 2:
  js_build = derive {
    copy 'package.json', APP_ROOT
    run 'npm install'
  }

  # 3:
  derive {
    import APP_ROOT, from: ruby_build
    import APP_ROOT, from: js_build
    tag 'jdoe/app', 'latest', force: true
  }

In the example above, an image is created with a new directory /app. From there, the build branches out into three directions:

  1. Create a new project referred to as ruby_build. The result of this project is an image with /app, a Gemfile in it, and a vendor directory containing vendored gems.
  2. Create a new project referred to as js_build. The result of this project is an image with /app, a package.json in it, and a node_modules directory containing vendored node.js modules. This project does not contain any of the contents of ruby_build.
  3. Create an anonymous project containing only the empty /app directory. Onto that, we'll import the contents of /app from ruby_build into this anonymous project. We'll do the same with the contents of /app from js_build. Finally, the resulting image is given the tag jdoe/app:latest.

Because each derived project lives on its own and only depends on the root project (whose end state is essentially the #mkdir instruction), when Gemfile changes but package.json does not, only the first derived project will be rebuilt (and following that, the third as well).



509
510
511
512
513
514
515
516
# File 'lib/drydock/project.rb', line 509

def derive(opts = {}, &blk)
  clean_opts  = build_opts.delete_if { |k, v| v.nil? }
  derive_opts = clean_opts.merge(opts).merge(chain: chain)

  Project.new(derive_opts).tap do |project|
    project.instance_eval(&blk) if blk
  end
end

#destroy!(force: false) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Destroy the images and containers created, and attempt to return the docker state as it was before the project.



207
208
209
210
# File 'lib/drydock/project.rb', line 207

def destroy!(force: false)
  chain.destroy!(force: force) if chain
  finalize!(force: force)
end

#done!Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Meta instruction to signal to the builder that the build is done.



215
216
217
# File 'lib/drydock/project.rb', line 215

def done!
  throw :done
end

#download_once(source_url, target_path, chmod: 0644) ⇒ Object

Download (and cache) a file from source_url, and copy it into the target_path in the container with a specific chmod (defaults to 0644).

The cache currently cannot be disabled.



223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
# File 'lib/drydock/project.rb', line 223

def download_once(source_url, target_path, chmod: 0644)
  requires_from!(:download_once)

  unless cache.key?(source_url)
    cache.set(source_url) do |obj|
      chunked = Proc.new do |chunk, remaining_bytes, total_bytes|
        obj.write(chunk)
      end
      Excon.get(source_url, response_block: chunked)
    end
  end

  log_step('download_once', source_url, target_path, chmod: sprintf('%o', chmod))

  # TODO(rpasay): invalidate cache when the downloaded file changes,
  # and then force rebuild
  chain.run("# DOWNLOAD #{source_url} #{target_path}") do |container|
    container.archive_put do |output|
      TarWriter.new(output) do |tar|
        cache.get(source_url) do |input|
          tar.add_file(target_path, chmod) do |tar_file|
            tar_file.write(input.read)
          end
        end
      end
    end
  end

  self
end

#drydock(version = '>= 0') ⇒ Object

This instruction is optional, but if specified, must appear at the beginning of the file.

This instruction is used to restrict the version of drydock required to run the Drydockfile. When not specified, any version of drydock is allowed to run the file.

The version specifier understands any bundler-compatible (and therefore gem-compatible) version specification; it even understands the twiddle-waka (~>) operator.

Examples:

drydock '~> 0.5'

Parameters:

  • version (String) (defaults to: '>= 0')

    The version specification to use.

Raises:



268
269
270
271
272
273
274
275
276
277
278
279
280
# File 'lib/drydock/project.rb', line 268

def drydock(version = '>= 0')
  raise InvalidInstructionError, '`drydock` must be called before `from`' if chain
  log_step('drydock', version)
  
  requirement = Gem::Requirement.create(version)
  current     = Gem::Version.create(Drydock.version)

  unless requirement.satisfied_by?(current)
    raise InsufficientVersionError, "build requires #{version.inspect}, but you're on #{Drydock.version.inspect}"
  end

  self
end

#entrypoint(command) ⇒ Object

Sets the entrypoint command for an image.

#entrypoint corresponds to the ENTRYPOINT Dockerfile instruction. This instruction does not run the command, but rather provides the default command to be run when the image is run without specifying a command.

As with the #cmd instruction, #entrypoint has three forms, of which the first two forms are preferred over the last one.

Parameters:

  • command (String, Array<String>)

    The command set to run. When a String is provided, the command is run inside a shell (/bin/sh). When an Array is given, the command is run as-is given.



292
293
294
295
296
297
298
299
300
301
302
# File 'lib/drydock/project.rb', line 292

def entrypoint(command)
  requires_from!(:entrypoint)
  log_step('entrypoint', command)

  unless command.is_a?(Array)
    command = ['/bin/sh', '-c', command.to_s]
  end

  chain.run("# ENTRYPOINT #{command.inspect}", entrypoint: command)
  self
end

#env(name, value) ⇒ Object

Set an environment variable, which will be persisted in future images (unless it is specifically overwritten) and derived projects.

Subsequent commands can refer to the environment variable by preceeding the variable with a $ sign, e.g.:

  env 'APP_ROOT', '/app'
  mkdir '$APP_ROOT'
  run ['some-command', '--install-into=$APP_ROOT']

Multiple calls to this instruction will build on top of one another. That is, after the following two instructions:

  env 'APP_ROOT',   '/app'
  env 'BUILD_ROOT', '/build'

the resulting image will have both APP_ROOT and BUILD_ROOT set. Later instructions overwrites previous instructions of the same name:

  # 1
  env 'APP_ROOT', '/app'
  # 2
  env 'APP_ROOT', '/home/jdoe/app'
  # 3

At #1, APP_ROOT is not set (assuming no other instruction comes before it). At #2, APP_ROOT is set to '/app'. At #3, APP_ROOT is set to /home/jdoe/app, and its previous value is no longer available.

Note that the environment variable is not evaluated in ruby; in fact, the $ sign should be passed as-is to the instruction. As with shell programming, the variable name should not be preceeded by the $ sign when declared, but must be when referenced.

Parameters:

  • name (String)

    The name of the environment variable. By convention, the name should be uppercased and underscored. The name should not be preceeded by a $ sign in this context.

  • value (String)

    The value of the variable. No extra quoting should be necessary here.



349
350
351
352
353
354
# File 'lib/drydock/project.rb', line 349

def env(name, value)
  requires_from!(:env)
  log_step('env', name, value)
  chain.run("# SET ENV #{name}", env: ["#{name}=#{value}"])
  self
end

#envs(pairs) ⇒ Object

Set multiple environment variables at once. The values will be persisted in future images and derived projects, unless specifically overwritten.

The following instruction:

  envs APP_ROOT: '/app', BUILD_ROOT: '/tmp/build'

is equivalent to the more verbose:

  env 'APP_ROOT', '/app'
  env 'BUILD_ROOT', '/tmp/build'

When the same key appears more than once in the same #envs instruction, the same rules for ruby hashes are used, which most likely (but not guaranteed between ruby version) means the last value set is used.

See also notes for #env.

Parameters:

  • pairs (Hash, #map)

    A hash-like enumerable, where #map yields exactly two elements. See #env for any restrictions of the name (key) and value.



380
381
382
383
384
385
386
387
# File 'lib/drydock/project.rb', line 380

def envs(pairs)
  requires_from!(:envs)
  log_step('envs', pairs)

  values = pairs.map { |name, value| "#{name}=#{value}" }
  chain.run("# SET ENVS #{pairs.inspect}", env: values)
  self
end

#expose(*ports, tcp: [], udp: []) ⇒ Object

Expose one or more ports. The values will be persisted in future images

When ports is specified, the format must be: ##/type where ## is the port number and type is either tcp or udp. For example, "80/tcp", "53/udp".

Otherwise, when the tcp or udp options are specified, only the port numbers are required.

Examples:

Different ways of exposing port 53 UDP and ports 80 and 443 TCP:

expose '53/udp', '80/tcp', '443/tcp'
expose udp: 53, tcp: [80, 443]

Parameters:

  • ports (Array<String>)

    An array of strings of port specifications. Each port specification must look like #/type, where # is the port number, and type is either udp or tcp.

  • tcp (Integer, Array<Integer>) (defaults to: [])

    A TCP port number to open, or an array of TCP port numbers to open.

  • udp (Integer, Array<Integer>) (defaults to: [])

    A UDP port number to open, or an array of UDP port numbers to open.



407
408
409
410
411
412
413
414
415
416
# File 'lib/drydock/project.rb', line 407

def expose(*ports, tcp: [], udp: [])
  requires_from!(:expose)

  Array(tcp).flatten.each { |p| ports << "#{p}/tcp" }
  Array(udp).flatten.each { |p| ports << "#{p}/udp" }

  log_step('expose', *ports)

  chain.run("# SET PORTS #{ports.inspect}", expose: ports)
end

#finalize!(force: false) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Finalize everything. This will be automatically invoked at the end of the build, and should not be called manually.



442
443
444
445
446
447
448
449
450
451
452
453
# File 'lib/drydock/project.rb', line 442

def finalize!(force: false)
  if chain
    chain.finalize!(force: force)
  end

  if stream_monitor
    stream_monitor.kill
    stream_monitor.join
  end

  self
end

#from(repo, tag = 'latest') ⇒ Object

Build on top of the from image. This must be the first instruction of the project, although non-instructions may appear before this.

If the drydock instruction is provided, from should come after it.

Parameters:

  • repo (#to_s)

    The name of the repository, which may be any valid docker repository name, and may optionally include the registry address, e.g., johndoe/thing or quay.io/jane/app. The name must not contain the tag name.

  • tag (#to_s) (defaults to: 'latest')

    The tag to use.

Raises:



427
428
429
430
431
432
433
434
435
436
# File 'lib/drydock/project.rb', line 427

def from(repo, tag = 'latest')
  raise InvalidInstructionError, '`from` must only be called once per project' if chain

  repo = repo.to_s
  tag  = tag.to_s

  log_step('from', repo, tag)
  @chain = PhaseChain.from_repo(repo, tag)
  self
end

#import(path, from: nil, force: false, spool: false) ⇒ Object

TODO:

Add a #load method as an alternative to #import Doing so would allow importing a full container, including things from /etc, some of which may be mounted from the host.

TODO:

Do not always append /. to the #archive_get calls We must check the type of path inside the container first.

TODO:

Break this large method into smaller ones.

Import a path from a different project. The from option should be project, usually the result of a derive instruction.



537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
# File 'lib/drydock/project.rb', line 537

def import(path, from: nil, force: false, spool: false)
  mkdir(path)

  requires_from!(:import)
  raise InvalidInstructionError, 'cannot `import` from `/`' if path == '/' && !force
  raise InvalidInstructionError, '`import` requires a `from:` option' if from.nil?
  log_step('import', path, from: from.last_image.id)

  total_size = 0

  if spool
    spool_file = Tempfile.new('drydock')
    log_info("Spooling to #{spool_file.path}")

    from.send(:chain).run("# EXPORT #{path}", no_commit: true) do |source_container|
      source_container.archive_get(path + "/.") do |chunk|
        spool_file.write(chunk.to_s).tap { |b| total_size += b }
      end
    end

    spool_file.rewind
    chain.run("# IMPORT #{path}", no_cache: true) do |target_container|
      target_container.archive_put(path) do |output|
        output.write(spool_file.read)
      end
    end

    spool_file.close
  else
    chain.run("# IMPORT #{path}", no_cache: true) do |target_container|
      target_container.archive_put(path) do |output|
        from.send(:chain).run("# EXPORT #{path}", no_commit: true) do |source_container|
          source_container.archive_get(path + "/.") do |chunk|
            output.write(chunk.to_s).tap { |b| total_size += b }
          end
        end
      end
    end
  end

  log_info("Imported #{Formatters.number(total_size)} bytes")
end

#last_imageObject

Retrieve the last image object built in this project.

If no image has been built, returns nil.



583
584
585
# File 'lib/drydock/project.rb', line 583

def last_image
  chain ? chain.last_image : nil
end

#loggerLogger

Access to the logger object.

Returns:

  • (Logger)

    A logger object on which one could call #info, #error, and the likes.



522
523
524
# File 'lib/drydock/project.rb', line 522

def logger
  Drydock.logger
end

#mkdir(path, chmod: nil) ⇒ Object

Create a new directory specified by path in the image.

Parameters:

  • path (String)

    The path to create inside the image.

  • chmod (String) (defaults to: nil)

    The mode to which the new directory will be chmodded. If not specified, the default umask is used to determine the mode.



592
593
594
595
596
597
598
# File 'lib/drydock/project.rb', line 592

def mkdir(path, chmod: nil)
  if chmod
    run "mkdir -p #{path} && chmod #{chmod} #{path}"
  else
    run "mkdir -p #{path}"
  end
end

#on_build(instruction = nil, &blk) ⇒ Object



601
602
603
604
605
606
# File 'lib/drydock/project.rb', line 601

def on_build(instruction = nil, &blk)
  requires_from!(:on_build)
  log_step('on_build', instruction)
  chain.run("# ON_BUILD #{instruction}", on_build: instruction)
  self
end

#run(cmd, opts = {}, &blk) ⇒ Object

This instruction is used to run the command cmd against the current project. The opts may be one of:

  • no_commit, when true, the container will not be committed to a new image. Most of the time, you want this to be false (default).
  • no_cache, when true, the container will be rebuilt every time. Most of the time, you want this to be false (default). When no_commit is true, this option is automatically set to true.
  • env, which can be used to specify a set of environment variables. For normal usage, you should use the env or envs instructions.
  • expose, which can be used to specify a set of ports to expose. For normal usage, you should use the expose instruction instead.
  • on_build, which can be used to specify low-level on-build options. For normal usage, you should use the on_build instruction instead.

Additional opts are also recognized:

  • author, a string, preferably in the format of "Name [email protected]". If provided, this overrides
  • comment, an arbitrary string used as a comment for the resulting image

If run results in a container being created and &blk is provided, the container will be yielded to the block.



631
632
633
634
635
636
637
638
639
640
641
642
643
# File 'lib/drydock/project.rb', line 631

def run(cmd, opts = {}, &blk)
  requires_from!(:run)

  cmd = build_cmd(cmd)

  run_opts = opts.dup
  run_opts[:author]  = opts[:author]  || build_opts[:author]
  run_opts[:comment] = opts[:comment] || build_opts[:comment]

  log_step('run', cmd, run_opts)
  chain.run(cmd, run_opts, &blk)
  self
end

#set(key, value = nil, &blk) ⇒ Object

Set project options.

Raises:

  • (ArgumentError)


646
647
648
649
650
651
652
653
# File 'lib/drydock/project.rb', line 646

def set(key, value = nil, &blk)
  key = key.to_sym
  raise ArgumentError, "unknown option #{key.inspect}" unless build_opts.key?(key)
  raise ArgumentError, "one of value or block is required" if value.nil? && blk.nil?
  raise ArgumentError, "only one of value or block may be provided" if value && blk

  build_opts[key] = value || blk
end

#tag(repo, tag = 'latest', force: false) ⇒ Object

Tag the current state of the project with a repo and tag.

When force is false (default), this instruction will raise an error if the tag already exists. When true, the tag will be overwritten without any warnings.



660
661
662
663
664
665
666
# File 'lib/drydock/project.rb', line 660

def tag(repo, tag = 'latest', force: false)
  requires_from!(:tag)
  log_step('tag', repo, tag, force: force)

  chain.tag(repo, tag, force: force)
  self
end

#with(plugin, &blk) ⇒ Object

Use a plugin to issue other commands. The block form can be used to issue multiple commands:

  with Plugins::APK do |apk|
    apk.update
  end

In cases of single commands, the above is the same as:

  with(Plugins::APK).update


682
683
684
685
686
# File 'lib/drydock/project.rb', line 682

def with(plugin, &blk)
  (@plugins[plugin] ||= plugin.new(self)).tap do |instance|
    yield instance if block_given?
  end
end