Module: CircleCI::Parallel

Extended by:
SingleForwardable
Defined in:
lib/circleci/parallel.rb,
lib/circleci/parallel/hook.rb,
lib/circleci/parallel/node.rb,
lib/circleci/parallel/build.rb,
lib/circleci/parallel/version.rb,
lib/circleci/parallel/task/base.rb,
lib/circleci/parallel/task/slave.rb,
lib/circleci/parallel/environment.rb,
lib/circleci/parallel/task/master.rb,
lib/circleci/parallel/configuration.rb,
lib/circleci/parallel/task/mock_slave.rb,
lib/circleci/parallel/task/mock_master.rb

Overview

Provides simple APIs for joining CircleCI's parallel builds and sharing files between the builds.

Examples:

merged_data = {}

CircleCI::Parallel.configure do |config|
  config.before_join do
    data = do_something
    json = JSON.generate(data)
    File.write('data.json', json)
  end

  config.after_download do
    Dir.glob('*/data.json') do |path|
      json = File.read(path)
      data = JSON.parse(json)
      node_name = File.dirname(path)
      merged_data[node_name] = data
    end
  end
end

CircleCI::Parallel.join

p merged_data

Defined Under Namespace

Modules: Task, Version Classes: Build, Configuration, Environment, Hook, Node

Constant Summary collapse

WORK_DIR =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future. 

'/tmp/circleci-parallel'.freeze
BASE_DATA_DIR =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future. 

File.join(WORK_DIR, 'data')
JOIN_MARKER_FILE =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future. 

File.join(WORK_DIR, 'JOINING')
DOWNLOAD_MARKER_FILE =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future. 

File.join(WORK_DIR, 'DOWNLOADED')

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.configure {|config| ... }

This method returns an undefined value.

Provides a block for configuring RSpec::ComposableJSONMatchers.

Examples:

CircleCI::Parallel.configure do |config|
  config.silent = true
end

Yield Parameters:

See Also:



74
 
# File 'lib/circleci/parallel.rb', line 74

def_delegator :environment, :configure

.current_build ⇒ Build

Returns the current CircleCI build.

Returns:

  • (Build) —

    the current build

See Also:



85
 
# File 'lib/circleci/parallel.rb', line 85

def_delegator :environment, :current_build

.current_node ⇒ Build

Returns the current CircleCI node.

Returns:

  • (Build) —

    the current node

See Also:



96
 
# File 'lib/circleci/parallel.rb', line 96

def_delegator :environment, :current_node

.download_data_dir ⇒ String

Returns the download data directory where all node data will be downloaded. Note that only master node downloads data from other slave node. When the downloads are complete, the directory structure on the master node will be the following:

.
├── node0
│   └── node_specific_data_you_saved_on_node0.txt
├── node1
│   └── node_specific_data_you_saved_on_node1.txt
└── node2
    └── node_specific_data_you_saved_on_node2.txt

Examples:

Dir.chdir(CircleCI::Parallel.download_data_dir) do
  merged_data = Dir['*/data.json'].each_with_object({}) do |path, merged_data|
    data = JSON.parse(File.read(path))
    node_name = File.dirname(path)
    merged_data[node_name] = data
  end
end

Returns:

  • (String) —

    the download data directory

See Also:



159
160
161
162
163
 
# File 'lib/circleci/parallel.rb', line 159

def download_data_dir
  BASE_DATA_DIR.tap do |path|
    FileUtils.makedirs(path) unless Dir.exist?(path)
  end
end

.join ⇒ Object

Join all nodes in the same build and gather all node data into the master node. Invoking this method blocks until the join and data downloads are complete.

Raises:

  • (RuntimeError) —

    when CIRCLECI environment variable is not set

See Also:



110
 
# File 'lib/circleci/parallel.rb', line 110

def_delegator :environment, :join

.local_data_dir ⇒ String

Returns the local data directory where node specific data should be saved in.

Examples:

path = File.join(CircleCI::Parallel.local_data_dir, 'data.json')
File.write(path, JSON.generate(some_data))

Returns:

  • (String) —

    the local data directory

See Also:



128
129
130
131
132
 
# File 'lib/circleci/parallel.rb', line 128

def local_data_dir
  current_node.data_dir.tap do |path|
    FileUtils.makedirs(path) unless Dir.exist?(path)
  end
end

.puts ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



115
 
# File 'lib/circleci/parallel.rb', line 115

def_delegator :environment, :puts

.reset! ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



166
167
168
169
 
# File 'lib/circleci/parallel.rb', line 166

def reset!
  environment.clean
  @environment = nil
end

Instance Method Details

#configuration ⇒ Object 




# File 'lib/circleci/parallel.rb', line 47