Class: Vagrant::BoxCollection

Inherits:
Object
  • Object
show all
Defined in:
lib/vagrant/box_collection.rb,
lib/vagrant/box_collection/remote.rb

Overview

Represents a collection a boxes found on disk. This provides methods for accessing/finding individual boxes, adding new boxes, or deleting boxes.

Defined Under Namespace

Modules: Remote

Constant Summary collapse

TEMP_PREFIX =
"vagrant-box-add-temp-".freeze
VAGRANT_SLASH =
"-VAGRANTSLASH-".freeze
VAGRANT_COLON =
"-VAGRANTCOLON-".freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(directory, options = nil) ⇒ BoxCollection

Initializes the collection.

Parameters:

  • directory (Pathname)

    The directory that contains the collection of boxes.



54
55
56
57
58
59
60
61
62
# File 'lib/vagrant/box_collection.rb', line 54

def initialize(directory, options=nil)
  options ||= {}

  @directory = directory
  @hook      = options[:hook]
  @lock      = Monitor.new
  @temp_root = options[:temp_dir_root]
  @logger    = Log4r::Logger.new("vagrant::box_collection")
end

Instance Attribute Details

#directoryPathname (readonly)

The directory where the boxes in this collection are stored.

A box collection matches a very specific folder structure that Vagrant expects in order to easily manage and modify boxes. The folder structure is the following:

COLLECTION_ROOT/BOX_NAME/PROVIDER/[ARCHITECTURE]/.json

Where:

  • COLLECTION_ROOT - This is the root of the box collection, and is the directory given to the initializer.
  • BOX_NAME - The name of the box. This is a logical name given by the user of Vagrant.
  • PROVIDER - The provider that the box was built for (VirtualBox, VMware, etc.).
  • ARCHITECTURE - Optional. The architecture that the box was built for (amd64, arm64, 386, etc.).
  • metadata.json - A simple JSON file that at the bare minimum contains a "provider" key that matches the provider for the box. This metadata JSON, however, can contain anything.

Returns:

  • (Pathname)


48
49
50
# File 'lib/vagrant/box_collection.rb', line 48

def directory
  @directory
end

Instance Method Details

#add(path, name, version, **opts) ⇒ Object

This adds a new box to the system.

There are some exceptional cases:

  • BoxAlreadyExists - The box you're attempting to add already exists.
  • BoxProviderDoesntMatch - If the given box provider doesn't match the actual box provider in the untarred box.
  • BoxUnpackageFailure - An invalid tar file.

Preconditions:

  • File given in path must exist.

Parameters:

  • path (Pathname)

    Path to the box file on disk.

  • name (String)

    Logical name for the box.

  • version (String)

    The version of this box.

  • providers (Array<String>)

    The providers that this box can be a part of. This will be verified with the metadata.json and is meant as a basic check. If this isn't given, then whatever provider the box represents will be added.

  • force (Boolean)

    If true, any existing box with the same name and provider will be replaced.



84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
# File 'lib/vagrant/box_collection.rb', line 84

def add(path, name, version, **opts)
  architecture = opts[:architecture]
  providers = opts[:providers]
  providers = Array(providers) if providers
  provider = nil

  # A helper to check if a box exists. We store this in a variable
  # since we call it multiple times.
  check_box_exists = lambda do |box_formats, box_architecture|
    box = find(name, box_formats, version, box_architecture)
    next if !box

    if !opts[:force]
      @logger.error(
        "Box already exists, can't add: #{name} v#{version} #{box_formats.join(", ")}")
      raise Errors::BoxAlreadyExists,
        name: name,
        provider: box_formats.join(", "),
        version: version
    end

    # We're forcing, so just delete the old box
    @logger.info(
      "Box already exists, but forcing so removing: " +
      "#{name} v#{version} #{box_formats.join(", ")}")
    box.destroy!
  end

  with_collection_lock do
    log_provider = providers ? providers.join(", ") : "any provider"
    @logger.debug("Adding box: #{name} (#{log_provider} - #{architecture.inspect}) from #{path}")

    # Verify the box doesn't exist early if we're given a provider. This
    # can potentially speed things up considerably since we don't need
    # to unpack any files.
    check_box_exists.call(providers, architecture) if providers

    # Create a temporary directory since we're not sure at this point if
    # the box we're unpackaging already exists (if no provider was given)
    with_temp_dir do |temp_dir|
      # Extract the box into a temporary directory.
      @logger.debug("Unpacking box into temporary directory: #{temp_dir}")
      result = Util::Subprocess.execute(
        "bsdtar", "--no-same-owner", "--no-same-permissions", "-v", "-x", "-m", "-S", "-s", "|\\\\\|/|", "-C", temp_dir.to_s, "-f", path.to_s)
      if result.exit_code != 0
        raise Errors::BoxUnpackageFailure,
          output: result.stderr.to_s
      end

      # If we get a V1 box, we want to update it in place
      if v1_box?(temp_dir)
        @logger.debug("Added box is a V1 box. Upgrading in place.")
        temp_dir = v1_upgrade(temp_dir)
      end

      # We re-wrap ourselves in the safety net in case we upgraded.
      # If we didn't upgrade, then this is still safe because the
      # helper will only delete the directory if it exists
      with_temp_dir(temp_dir) do |final_temp_dir|
        # Get an instance of the box we just added before it is finalized
        # in the system so we can inspect and use its metadata.
        box = Box.new(name, nil, version, final_temp_dir)

        # Get the provider, since we'll need that to at the least add it
        # to the system or check that it matches what is given to us.
        box_provider = box.["provider"]

        if providers
          found = providers.find { |p| p.to_sym == box_provider.to_sym }
          if !found
            @logger.error("Added box provider doesnt match expected: #{log_provider}")
            raise Errors::BoxProviderDoesntMatch,
              expected: log_provider, actual: box_provider
          end
        else
          # Verify the box doesn't already exist
          check_box_exists.call([box_provider], architecture)
        end

        # We weren't given a provider, so store this one.
        provider = box_provider.to_sym

        # Create the directory for this box, not including the provider
        root_box_dir = @directory.join(dir_name(name))
        box_dir = root_box_dir.join(version)
        box_dir.mkpath
        @logger.debug("Box directory: #{box_dir}")

        # This is the final directory we'll move it to
        if architecture
          arch = architecture
          arch = Util::Platform.architecture if architecture == :auto
          box_dir = box_dir.join(arch)
        end
        provider_dir = box_dir.join(provider.to_s)
        @logger.debug("Provider directory: #{provider_dir}")

        if provider_dir.exist?
          @logger.debug("Removing existing provider directory...")
          provider_dir.rmtree
        end

        # Move to final destination
        provider_dir.mkpath

        # Recursively move individual files from the temporary directory
        # to the final location. We do this instead of moving the entire
        # directory to avoid issues on Windows. [GH-1424]
        copy_pairs = [[final_temp_dir, provider_dir]]
        while !copy_pairs.empty?
          from, to = copy_pairs.shift
          from.children(true).each do |f|
            dest = to.join(f.basename)

            # We don't copy entire directories, so create the
            # directory and then add to our list to copy.
            if f.directory?
              dest.mkpath
              copy_pairs << [f, dest]
              next
            end

            # Copy the single file
            @logger.debug("Moving: #{f} => #{dest}")
            FileUtils.mv(f, dest)
          end
        end

        if opts[:metadata_url]
          root_box_dir.join("metadata_url").open("w") do |f|
            f.write(opts[:metadata_url])
          end
        end
      end
    end
  end

  # Return the box
  find(name, provider, version, architecture)
end

#allArray

This returns an array of all the boxes on the system, given by their name and their provider.

Returns:

  • (Array)

    Array of [name, version, provider, architecture] of the boxes installed on this system.



230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
# File 'lib/vagrant/box_collection.rb', line 230

def all
  results = []

  with_collection_lock do
    @logger.debug("Finding all boxes in: #{@directory}")
    @directory.children(true).each do |child|
      # Ignore non-directories, since files are not interesting to
      # us in our folder structure.
      next if !child.directory?

      box_name = undir_name(child.basename.to_s)

      # Otherwise, traverse the subdirectories and see what versions
      # we have.
      child.children(true).each do |versiondir|
        next if !versiondir.directory?
        next if versiondir.basename.to_s.start_with?(".")

        version = versiondir.basename.to_s
        # Ensure version of box is correct before continuing
        if !Gem::Version.correct?(version)
          ui = Vagrant::UI::Prefixed.new(Vagrant::UI::Colored.new, "vagrant")
          ui.warn(I18n.t("vagrant.box_version_malformed",
                         version: version, box_name: box_name))
          @logger.debug("Invalid version #{version} for box #{box_name}")
          next
        end

        versiondir.children(true).each do |architecture_or_provider|
          # If the entry is not a directory, it is invalid and should be ignored
          if !architecture_or_provider.directory?
            @logger.debug("Invalid box #{box_name} (v#{version}) - invalid item: #{architecture_or_provider}")
            next
          end

          # Now the directory can be assumed to be the architecture
          architecture_name = architecture_or_provider.basename.to_s.to_sym

          # Cycle through directories to find providers
          architecture_or_provider.children(true).each do |provider|
            if !provider.directory?
              @logger.debug("Invalid box #{box_name} (v#{version}, #{architecture_name}) - invalid item: #{provider}")
              next
            end

            # If the entry contains a metadata file, add it
            if provider.join("metadata.json").file?
              provider_name = provider.basename.to_s.to_sym
              @logger.debug("Box: #{box_name} (#{provider_name} (#{architecture_name}), #{version})")
              results << [box_name, version, provider_name, architecture_name]
            end
          end

          # If the base entry contains a metadata file, then it was
          # added prior to architecture support and is a provider directory.
          # If it contains a metadata file, include it with the results only
          # if an entry hasn't already been included for the local system's
          # architecture
          if architecture_or_provider.join("metadata.json").file?
            provider_name = architecture_or_provider.basename.to_s.to_sym
            if results.include?([box_name, version, provider_name, Util::Platform.architecture.to_sym])
              next
            end
            @logger.debug("Box: #{box_name} (#{provider_name}, #{version})")
            results << [box_name, version, provider_name, nil]
          end
        end
      end
    end
  end
  # Sort the list to group like providers and properly ordered versions
  results.sort_by! do |box_result|
    [box_result[0], box_result[2], Gem::Version.new(box_result[1]), box_result[3]]
  end
  results
end

#clean(name) ⇒ Object

Cleans the directory for a box by removing the folders that are empty.



456
457
458
459
460
# File 'lib/vagrant/box_collection.rb', line 456

def clean(name)
  return false if exists?(name)
  path = File.join(directory, dir_name(name))
  FileUtils.rm_rf(path)
end

#find(name, providers, version, box_architecture = :auto) ⇒ Box

Find a box in the collection with the given name and provider.

Parameters:

  • name (String)

    Name of the box (logical name).

  • providers (Array)

    Providers that the box implements.

  • version (String)

    Version constraints to adhere to. Example: "~> 1.0" or "= 1.0, ~> 1.1"

Returns:

  • (Box)

    The box found, or nil if not found.



314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
# File 'lib/vagrant/box_collection.rb', line 314

def find(name, providers, version, box_architecture=:auto)
  providers = Array(providers)
  architecture = box_architecture
  architecture = Util::Platform.architecture if architecture == :auto

  # Build up the requirements we have
  requirements = version.to_s.split(",").map do |v|
    begin
      Gem::Requirement.new(v.strip)
    rescue Gem::Requirement::BadRequirementError
      raise Errors::BoxVersionInvalid,
            version: v.strip
    end
  end

  with_collection_lock do
    box_directory = @directory.join(dir_name(name))
    if !box_directory.directory?
      @logger.info("Box not found: #{name} (#{providers.join(", ")})")
      return nil
    end

    # Keep a mapping of Gem::Version mangled versions => directories.
    # ie. 0.1.0.pre.alpha.2 => 0.1.0-alpha.2
    # This is so we can sort version numbers properly here, but still
    # refer to the real directory names in path checks below and pass an
    # unmangled version string to Box.new
    version_dir_map = {}

    versions = box_directory.children(true).map do |versiondir|
      next if !versiondir.directory?
      next if versiondir.basename.to_s.start_with?(".")

      version = Gem::Version.new(versiondir.basename.to_s)
      version_dir_map[version.to_s] = versiondir.basename.to_s
      version
    end.compact

    # Traverse through versions with the latest version first
    versions.sort.reverse.each do |v|
      if !requirements.all? { |r| r.satisfied_by?(v) }
        # Unsatisfied version requirements
        next
      end

      versiondir = box_directory.join(version_dir_map[v.to_s])

      providers.each do |provider|
        providerdir = versiondir.join(architecture.to_s).join(provider.to_s)

        # If the architecture was automatically set to the host
        # architecture, then a match on the architecture subdirectory
        # or the provider directory (which is a box install prior to
        # architecture support) is valid
        if box_architecture == :auto
          if !providerdir.directory?
            providerdir = versiondir.join(provider.to_s)
          end

          if providerdir.join("metadata.json").file?
            @logger.info("Box found: #{name} (#{provider})")

             = nil
             = box_directory.join("metadata_url")
             = .read if .file?

            if  && @hook
              hook_env     = @hook.call(
                :authenticate_box_url, box_urls: [])
               = hook_env[:box_urls].first
            end

            return Box.new(
              name, provider, version_dir_map[v.to_s], providerdir,
              architecture: architecture, metadata_url: , hook: @hook
            )
          end
        end

        # If there is no metadata file found, skip
        next if !providerdir.join("metadata.json").file?

        @logger.info("Box found: #{name} (#{provider})")

         = nil
         = box_directory.join("metadata_url")
         = .read if .file?

        if  && @hook
          hook_env     = @hook.call(
            :authenticate_box_url, box_urls: [])
           = hook_env[:box_urls].first
        end

        return Box.new(
          name, provider, version_dir_map[v.to_s], providerdir,
          architecture: architecture, metadata_url: , hook: @hook
        )
      end
    end
  end

  nil
end

#upgrade_v1_1_v1_5Object

This upgrades a v1.1 - v1.4 box directory structure up to a v1.5 directory structure. This will raise exceptions if it fails in any way.



422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
# File 'lib/vagrant/box_collection.rb', line 422

def upgrade_v1_1_v1_5
  with_collection_lock do
    temp_dir = Pathname.new(Dir.mktmpdir(TEMP_PREFIX, @temp_root))

    @directory.children(true).each do |boxdir|
      # Ignore all non-directories because they can't be boxes
      next if !boxdir.directory?

      box_name = boxdir.basename.to_s

      # If it is a v1 box, then we need to upgrade it first
      if v1_box?(boxdir)
        upgrade_dir = v1_upgrade(boxdir)
        FileUtils.mv(upgrade_dir, boxdir.join("virtualbox"))
      end

      # Create the directory for this box
      new_box_dir = temp_dir.join(dir_name(box_name), "0")
      new_box_dir.mkpath

      # Go through each provider and move it
      boxdir.children(true).each do |providerdir|
        FileUtils.cp_r(providerdir, new_box_dir.join(providerdir.basename))
      end
    end

    # Move the folder into place
    @directory.rmtree
    FileUtils.mv(temp_dir.to_s, @directory.to_s)
  end
end