Class: Bones::Engine

Inherits:
Common show all
Defined in:
lib/bones/engine.rb

Overview

This class holds the main functionality: the Bones source- to-source compilation engine based on algorithmic skeletons. This class processes command line arguments, makes calls to the Bones preprocessor and the CAST gem, analyzes the source code, performs source transformations, instantiates the skeletons, and finally writes output code to file.

Constant Summary collapse

BONES_DIR_SKELETONS =

Locate the skeletons directory.

File.join(BONES_DIR,'skeletons')
SKELETON_FILE =

Set the name of the transformations file as found in the skeleton library.

'skeletons.txt'
TIMER_FILES =

A list of timer files to be found in the skeleton library.

['timer_1_start','timer_1_stop','timer_2_start','timer_2_stop']
COMMON_FILES =

A list of files to be found in the common directory of the skeleton library (excluding timer files).

['prologue','epilogue','mem_prologue','mem_copy_H2D','mem_copy_D2H','mem_epilogue','mem_global']
COMMON_GLOBALS =

The name of the file containing the globals as found in the skeleton library

'globals'
COMMON_HEADER =

The name of the file containing the header file for the original C code as found in the skeleton library

'header'
COMMON_GLOBALS_KERNEL =

The name of the file containing the globals for the kernel files as found in the skeleton library

'globals_kernel'
COMMON_SCHEDULER =

The name of the file containing the scheduler code

'scheduler'
GLOBAL_TIMERS =

Global timers

'timer_globals'
SKELETON_HOST =

The extension of a host file in the skeleton library. See also SKELETON_DEVICE.

'.host'
SKELETON_DEVICE =

The extension of a device file in the skeleton library. See also SKELETON_HOST.

'.kernel'
OUTPUT_HOST =

The suffix added to the generated output file for the host file. See also OUTPUT_DEVICE.

'_host'
OUTPUT_DEVICE =

The suffix added to the generated output file for the device file. See also OUTPUT_HOST.

'_device'
OUTPUT_VERIFICATION =

The suffix added to the generated verification file. See also OUTPUT_DEVICE and OUTPUT_HOST.

'_verification'

Instance Method Summary collapse

Methods inherited from Common

#flatten_hash, #from, #replace_defines, #search_and_replace, #search_and_replace!, #sum, #sum_and_from, #to

Constructor Details

#initialize ⇒ Engine

Initializes the engine and processes the command line arguments. This method uses the ‘trollop’ gem to parse the arguments and to create a nicely formatted help menu. This method additionally initializes a result-hash and reads the contents of the source file from disk.

Command-line usage:

bones --application <input> --target <target> [OPTIONS]

Options:

--application, -a <s>:   Input application file
     --target, -t <s>:   Target processor (choose from: 'GPU-CUDA','GPU-OPENCL-AMD','CPU-OPENCL-INTEL','CPU-OPENCL-AMD','CPU-OPENMP','CPU-C')
   --measurements, -m:   Enable/disable timers
        --version, -v:   Print version and exit
           --help, -h:   Show this message



60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'lib/bones/engine.rb', line 60

def initialize
  @result = {:original_code            => [],
             :header_code              => [],
             :host_declarations        => [],
             :host_code_lists          => [],
             :algorithm_declarations   => [],
             :algorithm_code_lists     => [],
             :verify_code              => [],
             :host_device_mem_globals  => []}
  @state = 0
  
  # Provides a list of possible targets (e.g. GPU-CUDA, 'CPU-OPENCL-INTEL').
  targets = []
  Dir[File.join(BONES_DIR_SKELETONS,'*')].each do |entry|
    if (File.directory?(entry)) && !(entry =~ /verification/)
      targets.push(File.basename(entry))
    end
  end
  targets = targets.sort
  
  # Parse the command line options using the 'trollop' gem.
  pp_targets = targets.inspect.gsub(/("|\[)|\]/,'')
  @options = Trollop::options do
    version 'Bones '+File.read(BONES_DIR+'/VERSION').strip+' (c) 2012 Cedric Nugteren, Eindhoven University of Technology'
    banner  NL+'Bones is a parallelizing source-to-source compiler based on algorithmic skeletons. ' +
            'For more information, see the README.rdoc file or visit the Bones website at http://parse.ele.tue.nl/bones/.' + NL + NL +
            'Usage:' + NL +
            '    bones --application <input> --target <target> [OPTIONS]' + NL +
            'using the following flags:'
    opt :application,     'Input application file',                               :short => 'a', :type => String
    opt :target,          'Target processor (choose from: '+pp_targets+')',       :short => 't', :type => String
    opt :measurements,    'Enable/disable timers',                                :short => 'm', :default => false
    opt :verify,          'Verify correctness of the generated code',             :short => 'c', :default => false
    opt :only_alg_number, 'Only generate code for the x-th species (99 -> all)',  :short => 'o', :type => Integer, :default => 99
    opt :merge_factor,    'Thread merge factor, default is 1 (==disabled)',       :short => 'f', :type => Integer, :default => 1
    opt :register_caching,'Enable register caching: 1:enabled (default), 0:disabled',      :short => 'r', :type => Integer, :default => 1
    opt :zero_copy       ,'Enable OpenCL zero-copy: 1:enabled (default), 0:disabled',      :short => 'z', :type => Integer, :default => 1
    opt :skeletons       ,'Enable non-default skeletons: 1:enabled (default), 0:disabled', :short => 's', :type => Integer, :default => 1
  end
  Trollop::die 'no input file supplied (use: --application)'              if !@options[:application_given]
  Trollop::die 'no target supplied (use: --target)'                       if !@options[:target_given]
  Trollop::die 'input file "'+@options[:application]+'"does not exist '   if !File.exists?(@options[:application])
  Trollop::die 'target not supported, supported targets are: '+pp_targets if !targets.include?(@options[:target].upcase)
  @options[:name] = @options[:application].split('/').last.split('.').first
  @options[:target] = @options[:target].upcase
  
  # Extension for the host files corresponding to the target.
  @extension = File.extname(Dir[File.join(BONES_DIR_SKELETONS,@options[:target],'common','*')][0])
  
  # Extension for the device files corresponding to the target.
  @algorithm_extension = File.extname(Dir[File.join(BONES_DIR_SKELETONS,@options[:target],'kernel','*.kernel.*')][0])
  
  # Set a prefix for functions called from the original file but defined in a host file
  @prefix = (@options[:target] == 'GPU-CUDA') ? '' : ''
  
  # Setting to include the scheduler (CUDA only)
  @scheduler = (@options[:target] == 'GPU-CUDA') ? true : false
  
  # Skip analyse passes for certain targets
  @skiptarget = false #(@options[:target] == 'PAR4ALL') ? true : false
  
  # Set the location for the skeleton library
  @dir = {}
  @dir[:library] = File.join(BONES_DIR_SKELETONS,@options[:target])
  @dir[:skeleton_library] = File.join(@dir[:library],'kernel')
  @dir[:common_library] = File.join(@dir[:library],'common')
  @dir[:verify_library] = File.join(BONES_DIR_SKELETONS,'verification')
  
  # Obtain the source code from file
  @source = File.open(@options[:application],'r'){|f| f.read}
  @basename = File.basename(@options[:application],'.c')
end

Instance Method Details

#process ⇒ Object

Method to process a file and to output target code. This method calls all relevant private methods.

Tasks:

  • Run the preprocessor to obtain algorithm information.

  • Use the ‘CAST’ gem to parse the source into an AST.

  • Call the code generator to perform the real work and produce output.



140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
 
# File 'lib/bones/engine.rb', line 140

def process
  
  # Run the preprocessor
  preprocessor = Bones::Preprocessor.new(@source,File.dirname(@options[:application]),@basename,@scheduler)
  preprocessor.process
  @result[:header_code] = preprocessor.header_code
  @result[:device_header] = preprocessor.device_header
  @result[:header_code] += '#include <sys/time.h>'+NL if @options[:measurements]
  
  # Parse the source code into AST
  parser = C::Parser.new
  parser.type_names << 'FILE'
  parser.type_names << 'size_t'
  ast = parser.parse(preprocessor.target_code)
  ast.preprocess

  # Add the scheduler's global code
  if @scheduler
    @result[:host_code_lists].push(File.read(File.join(@dir[:common_library],COMMON_SCHEDULER+@extension)))
  end
  
  # Set the algorithm's skeleton and generate the global code
  one_time = true
  preprocessor.algorithms.each_with_index do |algorithm,algorithm_number|
    algorithm.species.set_skeleton(File.join(@dir[:library],SKELETON_FILE))
    if @options[:skeletons] == 0
      algorithm.species.skeleton_name = 'default'
      algorithm.species.settings.gsub!('10','00').gsub!('20','00').gsub!('30','00')
    end
    if algorithm.species.skeleton_name && one_time
      @result[:host_code_lists].push(File.read(File.join(@dir[:common_library],COMMON_GLOBALS+@extension)))
      @result[:algorithm_code_lists].push(File.read(File.join(@dir[:common_library],COMMON_GLOBALS_KERNEL+@extension)))
      one_time = false
    end
  end
  
  # Perform code generation (per-species code)
  @result[:original_code] = ast
  arrays = []
  preprocessor.algorithms.each_with_index do |algorithm,algorithm_number|
    if @options[:only_alg_number] == 99 || algorithm_number == [@options[:only_alg_number],preprocessor.algorithms.length-1].min
      puts MESSAGE+'Starting code generation for algorithm "'+algorithm.name+'"'
      if algorithm.species.skeleton_name
        algorithm.merge_factor = @options[:merge_factor] if (@options[:target] == 'GPU-CUDA')
        algorithm.register_caching_enabled = @options[:register_caching]
        algorithm.set_function(ast)
        algorithm.populate_variables(ast,preprocessor.defines) if !@skiptarget
        algorithm.populate_lists()
        algorithm.populate_hash() if !@skiptarget
        generate(algorithm)
        puts MESSAGE+'Code generated using the "'+algorithm.species.skeleton_name+'" skeleton'
        arrays.concat(algorithm.arrays)
      else
        puts WARNING+'Skeleton "'+algorithm.species.name+'" not available'
      end
    end
  end
  
  # Only if the scheduler is included
  if @scheduler
  
    # Perform code generation (sync statements)
    @result[:host_declarations].push('void bones_synchronize(int bones_task_id);')
    
    # Perform code generation (memory allocs)
    allocs = []
    preprocessor.copies.each do |copy|
      if !allocs.include?(copy.name)
        generate_memory('alloc',copy,arrays,0)
        allocs << copy.name
      end
    end
    
    # Perform code generation (memory copies)
    preprocessor.copies.each_with_index do |copy,index|
      #puts MESSAGE+'Generating copy code for array "'+copy.name+'"'
      generate_memory('copy',copy,arrays,index)
    end
    
    # Perform code generation (memory frees)
    frees = []
    preprocessor.copies.each do |copy|
      if !frees.include?(copy.name)
        generate_memory('free',copy,arrays,0)
        frees << copy.name
      end
    end
  
  end
  
end

#write_output ⇒ Object

This method writes the output code to files. It creates a new directory formatted as ‘name_target’ and produces three files.

Output files:

  • main - a file containing the original code with function calls substituting the original algorithms.

  • target - a file containing the host code for the target.

  • kernel - a file containing the kernel code for the target.



240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
 
# File 'lib/bones/engine.rb', line 240

def write_output
  
  # Create a new directory for the output
  directory = @options[:application].split('.').first+'_'+@options[:target]
  Dir.mkdir(directory,0744) unless File.directory?(directory)
  
  parser = C::Parser.new
  parser.type_names << 'FILE'
  parser.type_names << 'size_t'
  
  # Populate the main file
  File.open(File.join(directory,@options[:application].split(File::SEPARATOR).last),'w') do |main|
    main.puts '#include <string.h>' if @options[:verify]
    main.puts @result[:header_code]
    main.puts File.read(File.join(@dir[:common_library],COMMON_HEADER+@extension))
    main.puts @result[:host_declarations]
    main.puts
    begin
      main.puts parser.parse(@result[:original_code]).to_s
    rescue
      puts WARNING+'Recovering from CAST parse error'
      main.puts parser.parse(@result[:original_code].clone).to_s
    end
  end
  
  # Populate the verification file
  if @options[:verify]
    File.open(File.join(directory,@options[:name]+OUTPUT_VERIFICATION+@extension),'w') do |verification|
      verification.puts @result[:header_code]
      verification.puts File.read(File.join(@dir[:verify_library],'header.c'))
      verification.puts
      verification.puts @result[:verify_code]
    end
  end
  
  # Populate the target file (host)
  File.open(File.join(directory,@options[:name]+OUTPUT_HOST+@extension),'w') do |target|
    target.puts '#include <cuda_runtime.h>'+NL if @options[:target] == 'GPU-CUDA'
    target.puts "#define ZEROCOPY 0"+NL if @options[:zero_copy] == 0 && @options[:target] == 'CPU-OPENCL-INTEL'
    target.puts "#define ZEROCOPY 1"+NL if @options[:zero_copy] == 1 && @options[:target] == 'CPU-OPENCL-INTEL'
    target.puts @result[:header_code]
    target.puts
    target.puts @result[:host_device_mem_globals]
    target.puts
    target.puts @result[:algorithm_declarations]
    target.puts @result[:host_code_lists]
    target.puts
    target.puts File.read(File.join(@dir[:common_library],GLOBAL_TIMERS+@extension))
  end
  
  # Populate the algorithm file (device)
  File.open(File.join(directory,@options[:name]+OUTPUT_DEVICE+@algorithm_extension),'w') do |algorithm|
    algorithm.puts @result[:device_header]
    algorithm.puts @result[:algorithm_code_lists]
  end
  
end