Class: CodeRunner

Inherits:
Object show all
Extended by:
const_get(SYSconst_get(SYS.variable_to_class_name)
Includes:
const_get(SYSconst_get(SYS.variable_to_class_name), Log
Defined in:
lib/coderunner/instance_methods.rb,
lib/coderunner.rb,
lib/coderunner.rb,
lib/coderunner/run.rb,
lib/cubecalccrmod/empty.rb,
lib/cubecalccrmod/sleep.rb,
lib/cubecalccrmod/cubecalc.rb,
lib/coderunner/class_methods.rb,
lib/coderunner/fortran_namelist.rb,
lib/coderunner/graphs_and_films.rb,
lib/coderunner/instance_methods.rb,
lib/cubecalccrmod/with_namelist.rb,
lib/coderunner/merged_code_runner.rb,
lib/coderunner/interactive_methods.rb,
lib/coderunner/system_modules/loki.rb,
lib/coderunner/system_modules/moab.rb,
lib/coderunner/system_modules/dirac.rb,
lib/coderunner/system_modules/slurm.rb,
lib/coderunner/heuristic_run_methods.rb,
lib/coderunner/system_modules/archer.rb,
lib/coderunner/system_modules/edison.rb,
lib/coderunner/system_modules/hector.rb,
lib/coderunner/system_modules/helios.rb,
lib/coderunner/system_modules/hopper.rb,
lib/coderunner/system_modules/iridis.rb,
lib/coderunner/system_modules/juropa.rb,
lib/coderunner/system_modules/macosx.rb,
lib/coderunner/system_modules/saturne.rb,
lib/coderunner/system_modules/franklin.rb,
lib/coderunner/system_modules/stampede.rb,
lib/coderunner/system_modules/new_hydra.rb,
lib/coderunner/system_modules/blue_joule.rb,
lib/coderunner/system_modules/load_leveler.rb,
lib/coderunner/system_modules/generic_linux.rb,
lib/coderunner/system_modules/genericlinux_testsystem.rb

Overview

CodeRunner Overview

CodeRunner is a class designed to make the running an analysis of large simulations and easy task. An instance of this class is instantiated for a given root folder. The runner, as it is known, knows about every simulation in this folder, each of which has a unique id and a unique subfolder.

The heart of the runner is the variable run_list. This is a hash of IDs and runs. A run is an instance of a class which inherits from CodeRunner::Run, and which is customised to be able to handle the input variables, and the output data, from the given simulation code. This is achieved by a module which contains a child class of CodeRunner::Run, which is provided independently of CodeRunner.

CodeRunner has methods to sort these runs, filter them according to quite complex conditions, print out the status of these runs, submit new runs, plot graphs using data from these runs, cancel running jobs, delete unwanted runs, and so on.

CodeRunner Interfaces

CodeRunner has two different interfaces:

  1. Instance methods

  2. Class methods

Instance Methods

The instance methods provide a classic Ruby scripting interface. A runner is instantiated from the CodeRunner class, and passed a root folder, and possibly some default options. Instance methods can then be called individually. This is what should be used for complex and non-standard tasks.

Class methods

The class methods are what are used by the command line interface. They define a standard set of tasks, each of which can be customised by a set of options known as command options, or copts for short.

There is a one-to-one correspondence between the long form of the commandline commands, and the class methods that handle those commands, and between the command line flags, and the options that are passed to the class methods. So for example:

$ coderunner submit -p ‘23.4, resolution: 256’ -n 32x4 -W 600

Becomes

CodeRunner.submit(p: ‘23.4, resolution: 256’, n: “32x4”, W: 600) # remembering that braces are not needed around a hash if it is the final parameter.

These methods are what should be used to automate a large number of standard tasks, which would be a pain to run from the command line.

Direct Known Subclasses

Merged, RemoteCodeRunner

Defined Under Namespace

Modules: Archer, BlueJoule, Dirac, Edison, Franklin, GenericLinux, GenericlinuxTestsystem, Hector, Helios, Hopper, InteractiveMethods, Iridis, Juropa, LoadLeveler, Loki, Macosx, Moab, NewHydra, Saturne, Slurm, Stampede Classes: CRError, CRFatal, CRMild, Cubecalc, Merged, Run

Constant Summary collapse

COMMAND_FOLDER = 
Dir.pwd
SCRIPT_FOLDER =

i.e. where this script is

File.dirname(File.expand_path(__FILE__)) + '/coderunner'
GLOBAL_OPTIONS =

global options are set by the environment but some can be changed.

{}
SYS = 
(GLOBAL_OPTIONS[:system] or ENV['CODE_RUNNER_SYSTEM'] or ENV['SYSTEM'] or "generic_linux")
SYSTEM_MODULE = 
const_get(SYS.variable_to_class_name)
GraphKit =

Backwards compatibility

GraphKit
CODE_RUNNER_VERSION = 
Version.new(Gem.loaded_specs['coderunner'].version.to_s) rescue "test"
GLOBAL_BINDING = 
binding
COMMAND_LINE_FLAGS_WITH_HELP =

Here are all the methods that map the command line invocation into the correct class method call

[
	["--recalc-all", "-A", GetoptLong::NO_ARGUMENT, %[Causes each directory to be reprocessed, rather than reading the cache of data. Its exact effect depends on the code module being used. By convention it implies that ALL data analysis will be redone.]],
	["--reprocess-all", "-a", GetoptLong::NO_ARGUMENT, %[Causes each directory to be reprocessed, rather than reading the cache of data. Its exact effect depends on the code module being used. By convention it implies that VERY LITTLE data analysis will be redone.]],
	["--code", "-C", GetoptLong::REQUIRED_ARGUMENT, %[The code that is being used for simulations in this folder. This string must correspond to a code module supplied to CodeRunner. It usually only needs to be specified once as it will be stored as a default in the folder.]],
	["--comment", "-c", GetoptLong::REQUIRED_ARGUMENT, %[A comment about the submitted run.]],
	["--debug", "-d", GetoptLong::NO_ARGUMENT, %[Submit the simulation to the debug queue. This usually only has meaning on HPC systems. It does not mean debug CodeRunner!]],
	["--defaults-file", "-D", GetoptLong::REQUIRED_ARGUMENT, %[Specify a defaults file to be used when submitting runs. The name should correspond to file named something like '<name>_defaults.rb' in the correct folders within the code module. Every time a different defaults file is specified, a local copy of that defaults file is stored in the root folder. This local copy can be edited and all runs will use the local copy to get defaults. CodeRunner will never overwrite the local copy.]],
	["--film-options", "-F", GetoptLong::OPTIONAL_ARGUMENT, %[Specify a hash of options when making films. The most important one is fa (frame array). For example  -F '{fa: [0, 200]}'. For all possible options see the CodeRunner method make_film_from_lists.]],
	["--conditions", "-f", GetoptLong::REQUIRED_ARGUMENT, %[A string specifying conditions used to filter runs. This filter is used in a variety of circumstances, for example when printing out the status, plotting graphs etc. Example: '@height == 10 and @width = 2.2 and @status==:Complete'.]],
	["--run-graph", "-g", GetoptLong::REQUIRED_ARGUMENT, %[Specify a run_graphkit to plot. A run_graphkit is one that is plotted for an individual run. The run graphkits available depend on the code module. The syntax is graphkit shorthand:\n       -g '<graph_name>[ ; <graph_options> [ ; <conditions> [ ; <sort> ] ] ]'\n where conditions (i.e. filter) and sort will override the -f and -O flags respectively. The -g flag can be specified multiple times, which will plot multiple graphs on the same page.]],
	["--graph", "-G", GetoptLong::REQUIRED_ARGUMENT, %[Specify a graphkit to plot. A graphkit combines data for every filtered run. The syntax is graphkit shorthand:\n        -G '<axis1>[ : <axis2> [ : <axis3 [ : <axis4> ] ] ] [ ; <graph_options> [ ; <conditions> [ ; <sort> ] ] ]'\n        where conditions (i.e. filter) and sort will override the -f and -O flags respectively. <axis1> etc are strings which can be evaluated by the runs. The -G flag can be specified multiple times, which will plot multiple graphs on the same page. For example\n                   -G 'width : 2*height ; {} ; depth == 2 ; width'\n          will plot twice the height against the width for every run where the depth is equal to 2, and will order the data points by width.]],
	["--heuristic-analysis", "-H", GetoptLong::NO_ARGUMENT, %[Should be specified whenever CodeRunner is being used to analyse simulations which did not originally submit (and which will therefore not have the usual CodeRunner meta data stored with them).] ],
	["--use-phantom", "-h", GetoptLong::OPTIONAL_ARGUMENT, %[Specify whether to use real or phantom runs]],
	["--just", "-j",  GetoptLong::REQUIRED_ARGUMENT, %[Specify individual run ids. For example -j 45,63,128 is shorthand for -f 'id==45 or id==63 or id==128']],
	["--job_chain", "-J",  GetoptLong::NO_ARGUMENT, %[Chain multiple simulations into one batch/submission job. Most useful for HPC systems.]],
	["--skip-similar-jobs-off", "-k", GetoptLong::NO_ARGUMENT, %[Normally CodeRunner will not submit a run whose input parameters identical to a previous run (to avoid wasting computer time). Specifying the flag will override that behaviour and force submission of an identical run.]],
	["--loop", "-l", GetoptLong::NO_ARGUMENT, %[Used with the status command. Keep continually printing out live status information.]],
	["--multiple-processes", "-M", GetoptLong::REQUIRED_ARGUMENT],
	["--modlet", "-m", GetoptLong::REQUIRED_ARGUMENT, %[Specify the modlet to be used in the current folder. Only needs to be specified once as it will be stored as a default.]],
	["--no-run", "-N", GetoptLong::NO_ARGUMENT, %[On some machines getting a list of currently running jobs takes a long time. Specifying this flag tells CodeRunner that you definitely know that no runs in the folder are still queueing or running. Do not specify it if there are still running jobs as it will cause their statuses to be updated incorrectly.]],
	["--nprocs", "-n", GetoptLong::REQUIRED_ARGUMENT, %[A string specifying the processor layout for the simulation. For example -n 46x4 means use 46 nodes with four processors per node. In the case of a personal computer something like -n 2 is more likely. The default is 1]],
	["--sort", "-O", GetoptLong::REQUIRED_ARGUMENT, %[Specify the sort order for the runs. Used for a variety of commands, for example status. It is a string of semicolon separated sort keys: for example -O height;width will sort the runs by height and then width.]],
	["--project", "-P", GetoptLong::REQUIRED_ARGUMENT, %[Specify the project to be used for billing purposes. Only necessary on some systems.]],
	["--parameters", "-p",  GetoptLong::REQUIRED_ARGUMENT, %[A hash of parameters for the simulation. For example -p '{height: 20, width: 2.3}'. These parameters will override the defaults in the local defaults file.]],
	["--queue", "-Q", GetoptLong::REQUIRED_ARGUMENT, %[The name of the queue to submit to on HPC systems. Not yet implemented for all systems. Please submit a feature request if it is not working on your system.]],
	["--no-auto-create-runner", "-q", GetoptLong::NO_ARGUMENT, %[Used for interactive mode when you don't want CodeRunner to analyse the current directory.]],
	["--terminal-size", "-t", GetoptLong::REQUIRED_ARGUMENT, %[Specify the terminal size for situations where CodeRunner cannot work it out:  -t '[rows, cols]' (square brackets are part of the syntax)]],
	["--test-submission", "-T", GetoptLong::NO_ARGUMENT, %[Don't actually submit the run, but exit after printing out the run parameters and generating any input files necessary.]],
	["--use-large-cache-but-recheck-incomplete", "-u", GetoptLong::NO_ARGUMENT, %[Use the large cache for speed, but check any runs whose status is not :Complete or :Failed.]],
	["--use-large-cache", "-U", GetoptLong::NO_ARGUMENT, %[Use the large cache for speed. No run data will be updated.]],
	["--version", "-v", GetoptLong::REQUIRED_ARGUMENT, %[Specify the version of the simulation code being used. Only has an effect for certain code modules.]],
	["--wall-mins", "-W", GetoptLong::REQUIRED_ARGUMENT, %[Specify the wall clock limit in minutes.]],
	["--write-options", "-w", GetoptLong::REQUIRED_ARGUMENT, %[Use when plotting graphs. A hash of custom options which are applied to the graphkit just before plotting it; for example: -w '{xlabel: 'X Axis Quantity, log_axis: 'y'}']],
	["--executable", "-X", GetoptLong::REQUIRED_ARGUMENT, %[Specify the location of the executable of the simulation code. It only needs to be specified once in any folder, unless it needs to be changed.]],
	["--other-folder", "-Y", GetoptLong::REQUIRED_ARGUMENT, %[Run CodeRunner in a different folder. On a local machine 'coderunner st -Y some/other/folder' is identical to 'cd some/other/folder; coderunner st -Y'. However, this flag can also be used for remote folders using RemoteCodeRunner (as long as CodeRunner is installed on the remote machine). e.g. -Y [email protected]:path/to/folder.]],
	["--supplementary-options", "-y", GetoptLong::REQUIRED_ARGUMENT],
	["--server", "-Z", GetoptLong::REQUIRED_ARGUMENT, %[Technical use only]],	
	["--log", "-z", GetoptLong::NO_ARGUMENT, %[Switch logging on (currently not working very well (05/2010)).]]	 # :nodoc:
]
CLF = 
COMMAND_LINE_FLAGS = COMMAND_LINE_FLAGS_WITH_HELP.map{|arr| arr.slice(0..2)}
CODE_COMMAND_OPTIONS =

NEEDS FIXING!!!!

[]
LONG_COMMAND_LINE_OPTIONS =

end

[
["--replace-existing", "", GetoptLong::NO_ARGUMENT, %[Use with resubmit: causes each resubmitted run to replace the run being resubmitted.]],
["--smart-resubmit-name", "", GetoptLong::NO_ARGUMENT, %[Use with resubmit: causes each resubmitted run to only contain its original id and changed parameters in its run name.]],
] + CODE_COMMAND_OPTIONS
LONG_COMMAND_LINE_FLAGS = 
LONG_COMMAND_LINE_OPTIONS.map{|arr| [arr[0], arr[2]]}
COMMANDS_WITH_HELP = 
[
	["available_modlets", "av", 0, 'List the available modlets for the code module.', [], [:C]],
	["available_defaults_files", "avd", 0, 'List the defaults files for the code module.', [], [:C]],
	["cancel", "can", 1, 'Cancel the specified job.', ['id'], [:U]],
	["code_command", "cc", 1, 'Call a class method of the run class. Effectively this will call run_class.class_eval(command). See documentation for whichever code module is in use.', ['command'], []],
	["continue_in_new_folder", "cnf", 1, 'Make a new folder in the parent directory and copy all coderunner configuration files to that folder. If options j or f are specified, copy all matching runs to that new folder.', ['folder'], [:j, :f, :U, :N]],
	["code_runner_execute", "crex",  1, 'Run (within the CodeRunner class) the fragment of Ruby code given.', ['Ruby fragment'], []],	
	["delete", "del", 0, 'Permanently erase all filtered runs.', [], [:j, :F, :U, :N]],
	["differences_between", "diff", 0, 'Print a table of all the differences between the input parameters of the filtered ids.', [], [:j, :F, :U, :N]],
	["directory", "dir", 1, 'Print out the directory for the given run.', ['id'], []],
	['dumb_film', "dfm", 0, 'Create a film of the specified graphkits using gnuplot "dumb" ASCII terminal.', [], [:F, :G, :g, :U, :N, :j, :f]],
	["execute", "ex",  1, 'Run (at the top level) the fragment of Ruby code given.', ['Ruby fragment'], []],	
	['film', "fm", 0, 'Create a film of the specified graphkits.', [], [:F, :G, :g, :U, :N, :j, :f]],
	["generate_modlet_from_input_file", "gm", 1, 'Deprecated', [], []],
	["generate_cubecalc", "gencc", 0, 'Generate the file cubecalc.cc, the source code for the coderunner test program.', [], []],
	["generate_documentation", "rdoc", 1, 'Create automatic documentation using the rdoc tool (deprecated, use the command line tool ri for getting help, or see rubygems.org/gems/coderunner).', [], []],
	["interactive_mode", "im", 0, 'Launch an interactive terminal. Any command line flags specified set the defaults for the session.', [], [:U, :N, :j, :q]],	
	["load_file", "ld",  1, 'Load a Ruby script file using the CodeRunner framework.', ['script file'], []],	
	['manual', 'man', 0, 'Print out command line manual', [], []],
	['netcdf_plot', 'ncplot', 3, 'Plot a comma separated list of variables, at a comma separated list of indices (nil for all) from the specified netcdf file against each other using gnuplot.', ['netcdf_file', 'vars', 'indices'], [:w]],
	["plot_graph", "plot", 0, 'Plot the specified graphkits using Gnuplot', [], [:G, :g, :w, :O, :U, :N, :j, :f]],
	["parameter_scan", "ps", 1, 'Read a parameter scan from file. For full details of how to write a parameter scan, see online documentation (coderunner.sourceforge.net).', ['scan file'], [:n, :W, :k, :v, :p, :T, :d]],
	['print_queue_status', 'qstat', 0, 'Show the current status of the queue', [], [:U, :u]],
	["readout", "ro", 0, 'Print a simple text readout of all data from the runs.', [], []],
	["reference", "ri", 1, "Print out documentation for the given class or method. #{rihelp}", ['ruby_class_or_method'], []],
	["resubmit", "resub", 0, 'Resubmit the filtered runs to be simulated. All parameters will be the same bar those altered by the p option.', [], [:p, :n, :W, :k, :v, :T, :d, :J, :f, :j]],
	["run_command", "rc", 1, 'Cause all filtered runs to evaluate the given string.', ['command string'], [:U, :f, :j, :N]],
	["runner_eval", "ev", 1, 'Cause the runner (the CodeRunner instance) to evaluate the given string.', ['command string'], [:U, :N, :j, :f]],
	["scan", "scan", 1, 'Submit a simple scan. For full details of how to write a simple scan, see online documentation (coderunner.sourceforge.net).', ['scan string'],  [:p, :n, :W, :k, :v, :T, :d]],
	["show_values_of", "shvl", 1, 'Evaluate the expression for each run and print a unique sorted list of them.', ['expression'],  [:U, :N, :j, :f]],
    ['start_launcher', 'launch', 2, 'Start a simple job launcher for non batch systems.', ['refresh_interval', 'max_queue_size'], []],
	["status", "st", 0, 'Print out a summary of the status of the filtered runs.', [], [:U, :N, :j, :f, :O]],
	["status_with_comments", "sc", 0, 'Print a list of ids with their status and any comments.', [], [:U, :N, :j, :f, :O]],
	["status_loop", "sl", 0, 'Loop, updating the filtered runs, then printing out a summary of the status of the filtered runs.  ', [], [:U, :N, :j, :f, :O]],
	["submit", "sub", 0, 'Submit a run to be simulated.', [], [:p, :n, :W, :k, :v, :T, :d, :J]],
	["submit_command", "subcom", 2, 'Submit an arbitrary shell command to the batch queue.', ['job name', 'command'], [:n, :W, :v, :T, :d]],
	["write_graph", "wg", 1, 'Write a graph to disk.', ['filename'], [:G, :g, :w, :O, :U, :N, :j, :f]]
]
COMMANDS =

This lists all the commands available on the command line. The first two items in each array indicate the long and short form of the command, and the third indicates the number of arguments the command takes. They are all implemented as Code Runner class methods (the method is named after the long form). The short form of the command is available as a global method in Code Runner interactive mode.

COMMANDS_WITH_HELP.map{|arr| arr.slice(0..2)}
CLF_TO_SHORT_COPTS =

A lookup hash which gives the appropriate short command option (copt) key for a given long command flag

COMMAND_LINE_FLAGS.inject({}){ |hash, (long, short, req)|
	letter = short[1,1]
	hash[long] = letter.to_sym 
	hash
}
CLF_BOOLS =

specifying flag sets a bool to be true

[:H, :U, :u, :A, :a, :T, :N, :q, :z, :d, :J, :replace_existing]
CLF_INVERSE_BOOLS =

CLF_BOOLS = [:s, :r, :D, :H, :U, :u, :L, :l, :A, :a, :T, :N,:V, :q, :z, :d] #

[:k]
LONG_TO_SHORT =

a look up hash that converts the long form of the command options to the short form (NB command options e.g. use_large_cache have a different form from command line flags e.g. –use-large-cache)

COMMAND_LINE_FLAGS.inject({}){ |hash, (long, short, req)|
	letter = short[1,1]
	hash[long[2, long.size].gsub(/\-/, '_').to_sym] = letter.to_sym 
	hash
}
CLF_TO_LONG =

A look up table that converts long only command line options (in LONG_COMMAND_LINE_OPTIONS) to the equivalent CodeRunner command option

LONG_COMMAND_LINE_OPTIONS.inject({}) do |hash, (long, short, req, help)|
	option = long[2, long.size].gsub(/\-/, '_').to_sym
	hash[long] = option
	hash
end
DEFAULT_COMMAND_OPTIONS =

Default command options; they are usually determined by the command line flags, but can be set independently

{}
CODE_OPTIONS = 
{}
FOLDER_DEFAULTS =

The defaults that are saved in the root folder

[:code, :modlet, :executable, :defaults_file, :project]
SUBMIT_OPTIONS =

Parameters important to the submission of a run, which can be set by command line flags. The runner values provide the default values for the submit function, but can be overidden in that function. All the runner does with them is set them as properties of the run to be submitted. It is the run itself for which the options are relevant.

[:nprocs, :wall_mins, :sys, :project, :queue, :comment, :executable]
DEFAULT_RUNNER_OPTIONS =

A hash containing the defaults for most runner options. They are overridden by any options provided during initialisation. They are mostly set at the command line (in practice, the command line flags are read into the command options, which set these defaults in the function CodeRunner.process_command_options which calls CodeRunner.set_runner_defaults). However, if Code Runner is being scripted, these defaults must be set manually or else the options they specify must be provided when initialising a runner.

([:conditions, :sort, :debug, :script_folder, :recalc_all, :multiple_processes, :heuristic_analysis, :test_submission, :reprocess_all, :use_large_cache, :use_large_cache_but_recheck_incomplete, :use_phantom, :no_run, :server, :version, :parameters] + SUBMIT_OPTIONS + FOLDER_DEFAULTS).inject({}){|hash, option| hash[option] = nil; hash}
CLASS_OPTIONS =

Options that apply across the CodeRunner class

[:multiple_processes].inject({}){|hash, option| 
class_accessor option
set(option, nil)
hash[option] = nil; 
hash
}
NECESSARY_RUN_CLASS_PROPERTIES =

These are properties of the run class that must be defined. For more details see CodeRunner::Run.

{
	:code => [String],
	:variables => [Array], 
	:naming_pars => [Array], 
	:results => [Array], 
	:run_info => [Array],  
	:code_long => [String],
	:excluded_sub_folders => [Array],
	:modlet_required => [TrueClass, FalseClass],
	:uses_mpi => [TrueClass, FalseClass]
}
NECESSARY_RUN_CODE_METHODS =

These are methods that the run class must implement. They should be defined in a code module.

[
	:process_directory_code_specific, 
	:print_out_line, 
	:parameter_string, 
	:generate_input_file, 
	:parameter_transition, 
	:executable_location,
	:executable_name
]
NECESSARY_RUN_SYSTEM_METHODS =

These are methods that the run class must implement. They should be defined in a system module.

[
	:queue_status, 
	:run_command, 
	:execute,
	:error_file, 
	:output_file, 
	:cancel_job
]
PERMITTED_STATI =

These are the only permitted values for the run instance variable @status.

[:Unknown, :Complete, :Incomplete, :NotStarted, :Failed, :Queueing, :Running, :Held]
SETUP_RUN_CLASSES = 
[]
@@sys =

end

SYS
@@wait =

Do you wait for the previous run to have completed when using simple scan?

true

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(root_folder, options = {}) ⇒ CodeRunner

Instantiate a new runner. The root folder contains a set of simulations, each of which has a unique ID. There is a one-to-one correspondence between a runner and a root folder: no two runners should ever be given the same root folder in the same script (there are safeguards to prevent this causing much trouble, but it should still be avoided on philosophical grounds), and no runner should be given a folder which has more than one set of simulations within it (as these simulations will contain duplicate IDs).

Options is a hash whose keys may be any of the keys of the constant DEFAULT_RUNNER_OPTIONS. I.e. to see what options can be passed:

p CodeRunner::DEFAULT_RUNNER_OPTIONS.keys

Raises:



166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
 
# File 'lib/coderunner/instance_methods.rb', line 166

def initialize(root_folder, options={})
	logf :initialize
	raise CRFatal.new("System not defined") unless SYS
	root_folder.sub!(/~/, ENV['HOME'])
	@root_folder = root_folder 
	read_defaults
	options.each do |key,value|
		key = LONG_TO_SHORT.key(key) if LONG_TO_SHORT.key(key)
		set(key, value) if value
	end
# 		ep options
	
	log 'modlet in initialize', @modlet
	
	@version= options[:version]
# 		ep 'ex', @executable
	#ep 'modlet is ', @modlet
	get_run_class

	@cache = {}
	
	@n_checks = 0
	@print_out_size = 0
	
	@run_list = {}; @ids = []
	set_max_id(0)

	@phantom_run_list = {}; @phantom_ids = []
	@phantom_id = -1

	@combined_run_list = {}; @combined_ids = []

	@current_request = nil
	@requests = []

	@pids= []
	@maxes = {}; @mins = {}
	@cmaxes = {}; @cmins = {}
end

Instance Attribute Details

#cache ⇒ Object

Returns the value of attribute cache.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def cache
  @cache
end

#cmaxes ⇒ Object (readonly)

Returns the value of attribute cmaxes.



157
158
159
 
# File 'lib/coderunner/instance_methods.rb', line 157

def cmaxes
  @cmaxes
end

#cmins ⇒ Object (readonly)

Returns the value of attribute cmins.



157
158
159
 
# File 'lib/coderunner/instance_methods.rb', line 157

def cmins
  @cmins
end

#code ⇒ Object

Returns the value of attribute code.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def code
  @code
end

#combined_ids ⇒ Object

Returns the value of attribute combined_ids.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def combined_ids
  @combined_ids
end

#combined_run_list ⇒ Object

Returns the value of attribute combined_run_list.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def combined_run_list
  @combined_run_list
end

#current_request ⇒ Object

Returns the value of attribute current_request.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def current_request
  @current_request
end

#current_status ⇒ Object

Returns the value of attribute current_status.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def current_status
  @current_status
end

#defaults_file ⇒ Object

Returns the value of attribute defaults_file.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def defaults_file
  @defaults_file
end

#executable ⇒ Object

Returns the value of attribute executable.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def executable
  @executable
end

#ids ⇒ Object

Returns the value of attribute ids.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def ids
  @ids
end

#max_id ⇒ Object (readonly)

Returns the value of attribute max_id.



157
158
159
 
# File 'lib/coderunner/instance_methods.rb', line 157

def max_id
  @max_id
end

#maxes ⇒ Object (readonly)

Returns the value of attribute maxes.



157
158
159
 
# File 'lib/coderunner/instance_methods.rb', line 157

def maxes
  @maxes
end

#mins ⇒ Object (readonly)

Returns the value of attribute mins.



157
158
159
 
# File 'lib/coderunner/instance_methods.rb', line 157

def mins
  @mins
end

#modlet ⇒ Object

Returns the value of attribute modlet.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def modlet
  @modlet
end

#phantom_ids ⇒ Object

Returns the value of attribute phantom_ids.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def phantom_ids
  @phantom_ids
end

#phantom_run_list ⇒ Object

Returns the value of attribute phantom_run_list.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def phantom_run_list
  @phantom_run_list
end

#print_out_size ⇒ Object

Returns the value of attribute print_out_size.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def print_out_size
  @print_out_size
end

#requests ⇒ Object

Returns the value of attribute requests.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def requests
  @requests
end

#root_folder ⇒ Object

Returns the value of attribute root_folder.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def root_folder
  @root_folder
end

#run_class ⇒ Object

Returns the value of attribute run_class.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def run_class
  @run_class
end

#run_list ⇒ Object

Returns the value of attribute run_list.



156
157
158
 
# File 'lib/coderunner/instance_methods.rb', line 156

def run_list
  @run_list
end

#start_id ⇒ Object (readonly)

Returns the value of attribute start_id.



157
158
159
 
# File 'lib/coderunner/instance_methods.rb', line 157

def start_id
  @start_id
end

Class Method Details

.available_defaults_files(copts = {}) ⇒ Object

List the available defaults files for the given code (copts or -C on the command line).



41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
 
# File 'lib/coderunner/class_methods.rb', line 41

def self.available_defaults_files(copts={})
	process_command_options(copts)
	entries = []
	#begin
		##entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/my_defaults_files")
		#entries += 
	#rescue
	#end
	#begin
	#run_class = setup_run_class(copts[:C], modlet: copts[:m])
	rc = run_class(copts)
	entries = [rc.rcp.user_defaults_location, rc.rcp.code_module_folder	+ "/defaults_files"].map{|folder|  Dir.entries(folder).grep(/_defaults\.rb$/) rescue []}.sum
		#entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/defaults_files")
		#entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/defaults_files")
	#rescue
	#end
	puts "\nAvailable defaults files for #{copts[:C]}:"
	entries.each do |defaults_file|
		#puts "\t" + File.basename(defaults_file, '.rb').sub(/_defaults/, '') unless ['.', '..', '.svn', '.directory'].include? defaults_file
		puts "\t" + File.basename(defaults_file, '.rb').sub(/_defaults/, '')
	end
end

.available_modlets(copts = {}) ⇒ Object

List the available modlets for the given code (copts or -C on the command line).



22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
 
# File 'lib/coderunner/class_methods.rb', line 22

def self.available_modlets(copts={})
	process_command_options(copts)
	puts "\nAvailable modlets for #{copts[:C]}:"
	entries = []
	begin
		entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/my_modlets")
	rescue
	end
	begin
		entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/default_modlets")
	rescue
	end
	entries.each do |modlet|
		puts "\t" + File.basename(modlet, '.rb') unless ['.', '..', '.svn', '.directory'].include? modlet or modlet=~ /defaults/
	end
end

.cancel(id, copts = {}) ⇒ Object

Cancel the job with the given id. The user is asked interactively for confirmation and whether they would like to delete the folder for that job as well.



66
67
68
69
 
# File 'lib/coderunner/class_methods.rb', line 66

def self.cancel(id, copts={})
	runner = fetch_runner(copts)
	runner.cancel_job(id.to_i)
end

.code_command(string, copts = {}) ⇒ Object 



465
466
467
468
469
470
 
# File 'lib/coderunner/class_methods.rb', line 465

def self.code_command(string, copts = {})
	run_class(copts).class_eval(string)
	                            
# 		 runner = fetch_runner(copts)
# 		 runner.run_class.class_eval(string)
end

.code_runner_execute(ruby_fragment, copts = {}) ⇒ Object 



386
387
388
389
 
# File 'lib/coderunner/class_methods.rb', line 386

def self.code_runner_execute(ruby_fragment, copts={})
	#eval(ruby_fragment, GLOBAL_BINDING)
	eval(ruby_fragment)
end

.continue_in_new_folder(folder, copts = {}) ⇒ Object 



71
72
73
74
75
76
77
78
79
 
# File 'lib/coderunner/class_methods.rb', line 71

def self.continue_in_new_folder(folder, copts={})
	runner=fetch_runner(copts)
	options = {}
	if copts[:f] or copts[:j]
		options[:copy_ids] = runner.filtered_ids
	end

	runner.continue_in_new_folder(folder, options)
end

.delete(copts = {}) ⇒ Object 



81
82
83
84
 
# File 'lib/coderunner/class_methods.rb', line 81

def self.delete(copts={})
	runner = fetch_runner(copts)
	runner.destroy
end

.differences_between(copts = {}) ⇒ Object 



85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
 
# File 'lib/coderunner/class_methods.rb', line 85

def self.differences_between(copts = {})
	runner = fetch_runner(copts)
	runs = runner.filtered_ids.map{|id| runner.run_list[id]}
	vars = runner.run_class.rcp.variables.dup + runner.run_class.rcp.run_info.dup
	vars.delete_if{|var| runs.map{|r| r.send(var)}.uniq.size == 1}
	vars.delete :id
	vars.delete :run_name
	vars.delete :output_file
	vars.delete :error_file
	vars.delete :executable
	vars.delete :comment
	vars.delete :naming_pars
	vars.delete :parameter_hash
	vars.unshift :id
	#vars.push 'File.basename(executable)'
	table = vars.map{|var| [var] + runs.map{|r| str = r.instance_eval(var.to_s).to_s; str.size>10?str[0..9]:str} }
	#vars[-1] = 'exec'
	col_widths = table.map{|row| row.map{|v| v.to_s.size}}.inject{|o,n| o.zip(n).map{|a| a.max}}		
	eputs
	table.each{|row| i=0; eputs row.map{|v| str = sprintf(" %#{col_widths[i]}s ", v.to_s); i+=1; str}.join('|'); eputs '-' * (col_widths.sum + col_widths.size*3 - 1) }
	#p table, col_widths
end

.directory(id, copts = {}) ⇒ Object 



201
202
203
204
 
# File 'lib/coderunner/class_methods.rb', line 201

def self.directory(id, copts={})
	runner = fetch_runner(copts)
	puts runner.run_list[id.to_i].directory
end

.dumb_film(copts = {}) ⇒ Object 



108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
 
# File 'lib/coderunner/class_methods.rb', line 108

def self.dumb_film(copts = {})
# 		process_copts(copts)
	#old_term = GraphKit::GNUPLOT_DEFAULT_TERM
	size = Terminal.terminal_size
	size[0] -= 2
	term = "dumb #{size.reverse.join(' ')}"
	string = "\n" * size[0]

	runner = fetch_runner(copts)
	string_to_eval = copts[:w]
	frame_array = copts[:F][:frame_array] || copts[:F][:fa]
	index_name = copts[:F][:index_name] || copts[:F][:in]
	#options = (options and options =~ /\S/) ? eval(options): {}
	puts string
	for index in frame_array[0]..frame_array[1]
		string.true_lines.times{print "\033[A"}
		kit = runner.graphkit_from_lists(copts[:G], copts[:g], index_name => index)
     kit.gp.term =  term
		kit.gnuplot(eval: string_to_eval)
		sleep(copts[:F][:fr] ? 1.0/copts[:F][:fr] :  0.1)
	end 
end

.execute(ruby_fragment, copts = {}) ⇒ Object 



390
391
392
393
 
# File 'lib/coderunner/class_methods.rb', line 390

def self.execute(ruby_fragment, copts={})
	eval(ruby_fragment, GLOBAL_BINDING)
	#eval(ruby_fragment)
end

.fetch_runner(copts = {}) ⇒ Object

Retrieve the runner with the folder (and possibly server) given in copts. If no runner has been loaded for that folder, load one.



731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
 
# File 'lib/coderunner/class_methods.rb', line 731

def self.fetch_runner(copts={})
# 		ep copts
   # If copts(:Y) is an array of locations, return a merged runner of those locations
	if copts[:Y].kind_of? Array
		runners = copts[:Y].map do |location|
			new_copts = copts.dup.absorb(Y: location)
			fetch_runner(new_copts)
		end
		return Merged.new(*runners)
	end
	process_command_options(copts)
# 		ep copts
	@runners ||= {}
	runner = nil
	if copts[:Y] and copts[:Y] =~ /:/ 
		copts_r = copts.dup
		host, folder = copts[:Y].split(':')
		copts_r[:Y] = nil
		copts[:Y] = nil
		unless @runners[[host, folder]]
			copts[:cache] ||= :auto
			runner = @runners[[host, folder]] = RemoteCodeRunner.new(host, folder, copts)
			#(eputs 'Updating remote...'; runner.update) unless (copts[:g] and (copts[:g].kind_of? String or copts[:g].size > 0)) or copts[:no_update] or copts[:cache] 
		else 
			runner = @runners[[host, folder]]
		end
		runner.process_copts(copts)
	else
	  
		copts[:Y] ||= Dir.pwd
	  	Dir.chdir((copts[:Y] or Dir.pwd)) do
			unless @runners[copts[:Y]]
				runner = @runners[copts[:Y]] = CodeRunner.new(Dir.pwd, code: copts[:C], modlet: copts[:m], version: copts[:v], executable: copts[:X], defaults_file: copts[:D])
				runner.update unless copts[:no_update]
			else
				runner = @runners[copts[:Y]]
			end
				#p 'reading defaults', runner.recalc_all, DEFAULT_RUNNER_OPTIONS
			runner.read_defaults
				#p 'read defaults', runner.recalc_all
			
		end #Dir.chdir
	end
# 		ep copts
	return runner
# 		@r.read_defaults
end

.film(copts = {}) ⇒ Object 



205
206
207
208
209
 
# File 'lib/coderunner/class_methods.rb', line 205

def self.film(copts={})
	runner = fetch_runner(copts)
	copts[:F][:graphkit_modify] = copts[:w]
	runner.make_film_from_lists(copts[:G], copts[:g], copts[:F])
end

.film_graphkit_frame_array(graphkit_frame_array, options) ⇒ Object 



650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
 
# File 'lib/coderunner/graphs_and_films.rb', line 650

def self.film_graphkit_frame_array(graphkit_frame_array, options)
	possible_options = [:frame_array, :fa, :skip_frames, :sf, :normalize, :n, :normalize_pieces, :np, :increment, :i, :skip_encoding, :se, :frame_rate, :fr, :size]
# 		fa = (options[:frame_array] or options[:fa] or list[0][0].run_list[list[0][0].filtered_ids[0]].frame_array(options)	)

	fd = frame_digits = options[:fd]||Math.log10(graphkit_frame_array.map{|f, g| f}.max).ceil
	extension = (options[:extension] or options[:ext] or '.png')
	extension = '.' + extension unless extension =~ /^\./
	unless options[:skip_frames] or options[:sf]
	FileUtils.rm_r('film_frames') if FileTest.exist?('film_frames')

	FileUtils.makedirs('film_frames')
# 		puts @@multiple_processes; gets
	no_forks = (@@multiple_processes or 1)
	ep @@multiple_processes, no_forks
# 		end_graphkit = graphkit_multiple_runners(list, frame_index: fa[1])
# 		begin_graphkit = graphkit_multiple_runners(list, frame_index: fa[0])

# 		end_area = end_graphkit.plot_area_size
# 		begin_area = begin_graphkit.plot_area_size
# 		p end_area, begin_area, options
# 		plot_size = {}
# 		axes = [:x, :y, :z]
# 		options[:normalize] ||= options[:nm]  
# 		options[:normalize_pieces] ||= options[:nmp]
# 		for i in 0...end_area.size
# 			next unless options[:normalize] and options[:normalize].include? axes[i]
# 			min = [end_area[i][0], begin_area[i][0]].min
# 			max = [end_area[i][1], begin_area[i][1]].max
# 			key =  axes[i]
# 			plot_size[key + :range] = [min, max]
# 		end
# 		ep plot_size
# 		exit
# 		frames = []
# 		actual_frames = {}
# 		i = fa[0]
# 		j = 0
# 		while i <= fa[1]
# 			frames.push i
# 			actual_frames[i] = j
# 			i += (options[:ic] or options[:increment] or 1)
# 			j += 1
# 		end
	i = 0
	actual_frames = graphkit_frame_array.map{|f, g| f}.inject({}){|hash, f| hash[f] = i; i+=1; hash}
	graphkit_frame_array.pieces(no_forks).each_with_index do |graphkit_frame_array_piece, myrank|
		fork do
# 				if options[:normalize_pieces]
# 				end_area = graphkit_multiple_runners(list, frame_index: piece.max).plot_area_size
# 				begin_area = graphkit_multiple_runners(list, frame_index: piece.min).plot_area_size
# 				axes = [:x, :y, :z]
# 				for i in 0...end_area.size
# 					next unless options[:normalize_pieces].include? axes[i]
# 					min = [end_area[i][0], begin_area[i][0]].min
# 					max = [end_area[i][1], begin_area[i][1]].max
# 					key =  axes[i]
# 
# 					plot_size[key + :range] = [min, max]
# 				end
# 				end
# 				eputs 'making graphs...'; sleep 1
# 				graph_array = graphkit_multiple_runners_with_frame_array(piece, list, myrank==0)
# # 				ep graph_array
# 				eputs
			graphkit_frame_array_piece.each_with_index do |(frame_index,g), pindex|
				if myrank == 0
					eputs "\033[2A" # Terminal jargon - go back one line
					eputs sprintf("Plotting graphs: %2.2f", pindex.to_f/graphkit_frame_array_piece.size.to_f * 100.0) + "% Complete"
				end
# 					g = graph_kit_multiple_runners(list, plot_size + {frame_index: frame_index})
				
# 					g.modify(plot_size)
# 					g.modify(options)
# 					p g; exit
# 				        g.title +=  sprintf(", frame %0#{fd}d", frame_index) unless options[:frame_title] == false
				folder = ("film_frames/"); 
				file_name = sprintf("frame_%0#{fd}d", actual_frames[frame_index])
# 					g.gnuplot; gets; g.close
# 					ep folder + file_name + '.png'; gets
				g.gnuplot_write(folder + file_name + extension, size: options[:size])
			end
		end
	end
	eputs "Waiting on subprocesses..."
	Process.waitall
	end
	unless options[:skip_encoding]
	eputs "making film"
	frame_rate = (options[:frame_rate] or options[:fr] || 15)
	film_name = (options[:film_name] or options [:fn] or graphkit_frame_array[0][1].file_name + '_film').gsub(/\s/, '_')
	puts `ffmpeg -y #{options[:bitrate] ? "-b #{options[:bitrate]}" : ""} -r #{frame_rate} -threads #{(@multiple_processes or 1)} -i film_frames/frame_%0#{fd}d#{extension}  #{film_name}.mp4`
	end
end

.generate_cubecalc(copts = {}) ⇒ Object 



259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
 
# File 'lib/coderunner/class_methods.rb', line 259

def self.generate_cubecalc(copts={})
	#return
	File.open('cubecalc.cc', 'w') do |file|
	file.puts <<EOF
#include <stdio.h>
#include <iostream>
#include <cstdlib>
#include <fstream>
#include <cstring>
#include <ctime>
using namespace std;

int main(int argc, char* argv[]){
string line;
cout << "Starting..." << endl;

int calculate_sides = atoi(argv[1]); //Should the program calculate the area of the sides of the cube? 
cout << calculate_sides << endl;

char* input_file_name = argv[2]; //Get the input file name from the command line
cout << input_file_name << endl;

if (argc > 3){ //It has been told to sleep for a time
	bool cont = true;
	time_t start_t;
	time(&start_t);
	while (cont){
		time_t new_t;
		time(&new_t);
		cont = (new_t < (start_t + atoi(argv[3]) * 1.0));
	}
}

ifstream edges_file(input_file_name); //Read the edges from the input file
float* edges = new float[3];
int j = 0;
while (edges_file >> edges[j++]){
	cout << edges[j-1] << endl;
}


FILE* output = fopen("results.txt", "w"); //Write the volume to the output file
fprintf(output, "Volume was %f", edges[0] * edges[1] * edges[2]);
fclose(output);

if (calculate_sides == 1){ //If it has been told to calculate the sides
	cout << "calculating sides" << endl;
	FILE* sides = fopen("sides.txt", "w");
	for(int i=0; i<3; i++){
		cout << "Side " << i << ": " << edges[(i%3)] * edges[((i+1)%3)] << endl;
		fprintf(sides, "The area of side %d is %f\\n", i, edges[i%3] * edges[(i+1)%3]);
	}
	fclose(sides);
}
}



EOF

	end
end

.generate_documentation(username = nil, copts = {}) ⇒ Object 



211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
 
# File 'lib/coderunner/class_methods.rb', line 211

def self.generate_documentation(username = nil, copts = {})
	ep 'username', username||=ENV['USER']
	
	####### Here we use the command line documentation to generate a fake ruby file that rdoc will understand.
	File.open("class_methods_rdoc.rb", 'w') do |file|
		file.puts <<EOF
 class CodeRunner


#{COMMANDS_WITH_HELP.inject("") do |str, (long, short, nargs, comhelp, argnames, options)| 
(puts "Please run this command in the coderunner trunk directory"; exit) unless Dir.pwd =~ /coderunner\/trunk$/
	str << <<EOF2
 # #{comhelp.gsub(/\n/, "\n  # ")}
 #
 # Possible options:
 #
#{options.inject("") do |str, opt|
	longop, shortop, req, ophelp = COMMAND_LINE_FLAGS_WITH_HELP.find{|arr| arr[1] == "-" + opt.to_s}
	str << "  # :#{opt} --- #{ophelp.gsub(/\n/, "\n  # ")}\n  #\n"
end}

def self.#{long}(#{(argnames+[""]).join(",")}command_options={}) 
end

EOF2
end
}
 end
EOF
	end
# 		exit

	system "rm -rf doc/"
	system "rm -rf ri/"
	raise 'Please set RDOC_COMMAND' unless ENV['RDOC_COMMAND']
	system "#{ENV['RDOC_COMMAND']} --format=html -t 'CodeRunner Documentation' -m INDEX.rb  INDEX.rb code_runner_version.rb gnuplot.rb graphkit_gnuplot.rb graphkit.rb gsl_tools.rb  run_backwards_compatibility.rb feedback.rb run.rb fortran_namelist.rb graphs_and_films.rb  class_methods_rdoc.rb instance_methods.rb"
	system "#{ENV['RDOC_COMMAND']} -r --op ri INDEX.rb code_runner_version.rb gnuplot.rb graphkit_gnuplot.rb graphkit.rb gsl_tools.rb  run_backwards_compatibility.rb feedback.rb run.rb fortran_namelist.rb graphs_and_films.rb  class_methods_rdoc.rb instance_methods.rb"
	
	exit if username == ""
	
	string = "rsync -av --delete doc/  #{username},[email protected]:htdocs/api_documentation/"

	puts string
	exec string

end

.get_run_class_name(code, modlet = nil) ⇒ Object

Return the name of the run class according to the standard CodeRunner naming scheme. If the code name is ‘a_code_name’, with no modlet, the run class name will be ACodeName. If on the other hand there is a modlet called ‘modlet_name’, the class name will be ACodeName::ModletName.



347
348
349
 
# File 'lib/coderunner/instance_methods.rb', line 347

def self.get_run_class_name(code, modlet=nil) 
	return modlet ? "#{code.capitalize}::#{modlet.capitalize.sub(/\.rb$/, '').variable_to_class_name}" : "#{code.capitalize}"
end

.gets ⇒ Object 



42
43
44
 
# File 'lib/coderunner.rb', line 42

def self.gets
	$stdin.gets
end

.graphkit_multiple_runners(list, options = {}) ⇒ Object

list is an array of [[runner, [graphs, run_graphs]], … ]



424
425
426
427
428
429
430
431
432
 
# File 'lib/coderunner/graphs_and_films.rb', line 424

def self.graphkit_multiple_runners(list, options={})
	return list.inject(nil) do |kit, (runner, graph_lists)|
		graphs, run_graphs = graph_lists
		graphs.map!{|graph| runner.graphkit_shorthand(graph)}
		run_graphs.map!{|graph| runner.run_graphkit_shorthand(graph)}
		newkit = runner.graphkit_from_lists(graphs, run_graphs, options)
		kit ? kit + newkit : newkit
	end
end

.graphkit_multiple_runners_with_frame_array(frame_array, list, extra_options, print_message = false) ⇒ Object 



434
435
436
437
438
439
440
441
442
443
444
445
 
# File 'lib/coderunner/graphs_and_films.rb', line 434

def self.graphkit_multiple_runners_with_frame_array(frame_array, list, extra_options, print_message = false)
	i = 0
# 		
	return list.inject(nil) do |kit_array, (runner, graph_lists)|
		graphs, run_graphs = graph_lists
		graphs.map!{|graph| runner.graphkit_shorthand(graph)}
		run_graphs.map!{|graph| runner.run_graphkit_shorthand(graph)}
		newkit_array = runner.graphkit_from_lists_with_frame_array(frame_array, graphs, run_graphs, extra_options)
# 			eputs newkit_array.pretty_inspect
		kit_array ? (i=-1; kit.map{|(frame_index, kit)| i+= 1;[frame_index, new_kit[i][1]]}) : newkit_array
	end
end

.interactive_mode(copts = {}) ⇒ Object

module InteractiveMethods



53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/coderunner/interactive_methods.rb', line 53

def CodeRunner.interactive_mode(copts={})
	process_command_options(copts)
  			unless false and FileTest.exist? (ENV['HOME'] + '/code_runner_interactive_options.rb')
			File.open(ENV['HOME'] + '/.code_runner_interactive_options.rb', 'w') do |file|
				file.puts <<EOF
$has_put_startup_message_for_code_runner = true #please leave!
$code_runner_interactive_mode = true #please leave!
require 'yaml'

def reset
  Dispatcher.reset_application!
end
  
IRB.conf[:AUTO_INDENT] = true
IRB.conf[:USE_READLINE] = true
IRB.conf[:LOAD_MODULES] = []  unless IRB.conf.key?(:LOAD_MODULES)
unless IRB.conf[:LOAD_MODULES].include?('irb/completion')
  IRB.conf[:LOAD_MODULES] << 'irb/completion'
end      

			
require 'irb/completion'
require 'irb/ext/save-history'
require 'socket'
unless ENV['CODE_RUNNER_SYSTEM'] == 'macosx'
prompt_start = "\001#{Terminal::LIGHT_GREEN}\002CodeRunner\001#{Terminal::RESET}\002 v#{CODE_RUNNER_VERSION} - \001#{Terminal::CYAN}\002#\{ENV['USER']\}@#\{Socket::gethostname\}:#\{File.basename(Dir.pwd)}\001#{Terminal::RESET}\002 timemarker\n"
prompt_start = "\001#{Terminal::LIGHT_GREEN}\002CodeRunner\001#{Terminal::RESET}\002 v#{CODE_RUNNER_VERSION} - \001#{Terminal::CYAN}\002#\{Socket::gethostname\}:#\{File.basename(Dir.pwd)}\001#{Terminal::RESET}\002 timemarker %02n,%i \n"
else
	prompt_start = "CodeRunner #\{File.basename(Dir.pwd)}"
end
 IRB.conf[:PROMPT][:CODE_RUNNER] = {:PROMPT_I=>"#\{prompt_start}>> %02n,%i >> ", :PROMPT_N=>"#\{prompt_start}>> %02n:%i > ", :PROMPT_S=>"#\{prompt_start}>> %02n:%i (%l)> ", :PROMPT_C=>"#\{prompt_start}>> %02n:%i >>  ", :RETURN=>""}
 IRB.conf[:PROMPT][:CODE_RUNNER] = {:PROMPT_I=>"#\{prompt_start}>> ", :PROMPT_N=>"#\{prompt_start}> ", :PROMPT_S=>"#\{prompt_start}(%l)> ", :PROMPT_C=>"#\{prompt_start}>>  ", :RETURN=>""}
#   IRB.conf[:PROMPT][:CODE_RUNNER] = {:PROMPT_I=>"#\{prompt_start} %02n,%i>> ", :PROMPT_N=>"#\{prompt_start} %02n:%i> ", :PROMPT_S=>"#\{prompt_start} %02n:%i (%l)> ", :PROMPT_C=>"#\{prompt_start} %02n:%i>>  ", :RETURN=>""}


IRB.conf[:PROMPT_MODE] = :CODE_RUNNER
IRB.conf[:SAVE_HISTORY] = 400
IRB.conf[:HISTORY_FILE] = "\#\{Dir.pwd}/.code-runner-irb-save-history"
IRB.conf[:INSPECT_MODE] = false


EOF
			end # File.open
		end # unless
		File.open(".int.tmp.rb", 'w')do |file|
			file.puts "#{copts.inspect}.each do |key, val|
				CodeRunner::DEFAULT_COMMAND_OPTIONS[key] = val
			end"
			file.puts CodeRunner::InteractiveMethods::INTERACTIVE_METHODS
		end
# 			asdfa
		exec %[#{RbConfig::CONFIG['bindir']}/irb#{RbConfig::CONFIG['ruby_install_name'].sub(/ruby/, '')} -f -I '#{Dir.pwd}' -I '#{SCRIPT_FOLDER}'   -I '#{ENV['HOME']}' -r '.code_runner_interactive_options' -r 'coderunner' -r .int.tmp ]
end

.launcher_directory ⇒ Object 



321
322
323
 
# File 'lib/coderunner/class_methods.rb', line 321

def self.launcher_directory
	ENV['HOME'] + "/.coderunner/to_launch/#{ENV['CODE_RUNNER_LAUNCHER']}"
end

.load_file(files, copts = {}) ⇒ Object

eval(ruby_fragment)



394
395
396
397
398
399
400
401
402
403
404
405
406
 
# File 'lib/coderunner/class_methods.rb', line 394

def self.load_file(files, copts={})
	process_command_options(copts)
# 		begin
		files.split(/\s*,\s*/).each do |file|
# 				p file
				raise ArgumentError.new("#{file} is not a file.") unless File.file? file
			load file
		end
# 		rescue
# 			eval(files)
# 		end
		
end

.make_film_multiple_runners(list, options) ⇒ Object

__END__



544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
 
# File 'lib/coderunner/graphs_and_films.rb', line 544

def self.make_film_multiple_runners(list, options)
	possible_options = [:frame_array, :fa, :skip_frames, :sf, :normalize, :n, :normalize_pieces, :np, :increment, :i, :skip_encoding, :se]
	fa = (options[:frame_array] or options[:fa] or list[0][0].run_list[list[0][0].filtered_ids[0]].frame_array(options)	)
	iname = options[:in]||options[:index_name]||:frame_index

	fd = frame_digits = Math.log10(fa[1]).ceil
	unless options[:skip_frames] or options[:sf]
# 		`rm -rf film_frames`
# 		extension = (options[:extension] or options[:ext] or '.png')
# 		extension = '.' + extension unless extension =~ /^\./

# 		FileUtils.makedirs('film_frames')
# 		puts @@multiple_processes; gets
	no_forks = (@@multiple_processes or 1)
	ep @@multiple_processes, no_forks
	end_graphkit = graphkit_multiple_runners(list, iname => fa[1])
	begin_graphkit = graphkit_multiple_runners(list, iname => fa[0])

	end_area = end_graphkit.plot_area_size
	begin_area = begin_graphkit.plot_area_size
# 		p end_area, begin_area, options
	plot_size = {}
	axes = [:x, :y, :z]
	options[:normalize] ||= options[:nm]  
	options[:normalize_pieces] ||= options[:nmp]
	for i in 0...end_area.size
		next unless options[:normalize] and options[:normalize].include? axes[i]
		min = [end_area[i][0], begin_area[i][0]].min
		max = [end_area[i][1], begin_area[i][1]].max
		key =  axes[i]
		plot_size[key + :range] = [min, max]
	end
	ep plot_size
# 		exit
	frames = []
	actual_frames = {}
	i = fa[0]
	j = 0
	while i <= fa[1]
		frames.push i
		actual_frames[i] = j
		i += (options[:ic] or options[:increment] or 1)
		j += 1
	end
# 		graphkit_frame_array = []
	
	myrank = -1
# 		graphkit_frame_array = (frames.pieces(no_forks).parallel_map(n_procs: no_forks, with_rank: true) do |piece, myrank|
	graphkit_frame_array = (frames.pieces(no_forks).map do |piece|
	                        myrank +=1
# 		                        ep 'myrank is', myrank
# 				unless myrank==0
# # 					$stdout = $stderr = StringIO.new
# 				end
# 			fork do
			if options[:normalize_pieces]
			end_area = graphkit_multiple_runners(list, iname => piece.max).plot_area_size
			begin_area = graphkit_multiple_runners(list, iname => piece.min).plot_area_size
			axes = [:x, :y, :z]
			for i in 0...end_area.size
				next unless options[:normalize_pieces].include? axes[i]
				min = [end_area[i][0], begin_area[i][0]].min
				max = [end_area[i][1], begin_area[i][1]].max
				key =  axes[i]

				plot_size[key + :range] = [min, max]
			end
			end
			eputs 'making graphs...'; sleep 1 if myrank==0
			graph_array = graphkit_multiple_runners_with_frame_array(piece, list, {:in => iname}, myrank==0)
# 				ep graph_array
			eputs
			graph_array.each_with_index do |(frame_index,g), pindex|
# 					if myrank == 0
# 						eputs "\033[2A" # Terminal jargon - go back one line
# 						eputs sprintf("Plotting graphs: %2.2f", pindex.to_f/piece.size.to_f * 100.0) + "% Complete"
# 					end
# 					g = graph_kit_multiple_runners(list, plot_size + {frame_index: frame_index})
				
				g.modify(plot_size)
				g.modify(options)
				g.instance_eval options[:graphkit_modify] if options[:graphkit_modify]
# 					p g; exit
			        g.title +=  sprintf(", frame %0#{fd}d", frame_index) unless options[:frame_title] == false
# 					folder = ("film_frames/"); 
# 					file_name = sprintf("frame_%0#{fd}d", actual_frames[frame_index])
# 					g.gnuplot; gets; g.close
# 					ep folder + file_name + '.png'; gets
# 					g.gnuplot_write(folder + file_name + extension)
			end
		 graph_array
# 			end
	end).sum
# 		eputs "Waiting on subprocesses..."
# 		Process.waitall
	end
	
	film_graphkit_frame_array(graphkit_frame_array, options)
# 		unless options[:skip_encoding]
# 		eputs "making film"
# 		frame_rate = (options[:frame_rate] or options[:fr] || 15)
# 		film_name = (options[:film_name] or options [:fn] or end_graphkit.file_name + '_film').gsub(/\s/, '_')
# 		puts `ffmpeg -y #{options[:bitrate] ? "-b #{options[:bitrate]}" : ""} -r #{frame_rate} -threads #{(@multiple_processes or 1)} -i film_frames/frame_%0#{fd}d#{extension} -sameq #{film_name}.mp4`
# 		end
end

.make_film_multiple_runners_old(list, options) ⇒ Object 



447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
 
# File 'lib/coderunner/graphs_and_films.rb', line 447

def self.make_film_multiple_runners_old(list, options)
	possible_options = [:frame_array, :fa, :skip_frames, :sf, :normalize, :n, :normalize_pieces, :np, :increment, :i, :skip_encoding, :se, :index_name, :in]
	fa = (options[:frame_array] or options[:fa] or list[0][0].run_list[list[0][0].filtered_ids[0]].frame_array(options)	)
	iname = options[:in]||options[:index_name]||:frame_index

	fd = frame_digits = Math.log10(fa[1]).ceil
	unless options[:skip_frames] or options[:sf]
	`rm -rf film_frames`
	extension = (options[:extension] or options[:ext] or '.png')
	extension = '.' + extension unless extension =~ /^\./

	FileUtils.makedirs('film_frames')
# 		puts @@multiple_processes; gets
	no_forks = (@@multiple_processes or 1)
	ep @@multiple_processes, no_forks
	end_graphkit = graphkit_multiple_runners(list, iname => fa[1])
	begin_graphkit = graphkit_multiple_runners(list, iname => fa[0])

	end_area = end_graphkit.plot_area_size
	begin_area = begin_graphkit.plot_area_size
# 		p end_area, begin_area, options
	plot_size = {}
	axes = [:x, :y, :z]
	options[:normalize] ||= options[:nm]  
	options[:normalize_pieces] ||= options[:nmp]
	for i in 0...end_area.size
		next unless options[:normalize] and options[:normalize].include? axes[i]
		min = [end_area[i][0], begin_area[i][0]].min
		max = [end_area[i][1], begin_area[i][1]].max
		key =  axes[i]
		plot_size[key + :range] = [min, max]
	end
	ep plot_size
# 		exit
	frames = []
	actual_frames = {}
	i = fa[0]
	j = 0
	while i <= fa[1]
		frames.push i
		actual_frames[i] = j
		i += (options[:ic] or options[:increment] or 1)
		j += 1
	end
	frames.pieces(no_forks).each_with_index do |piece, myrank|
		fork do
			if options[:normalize_pieces]
			end_area = graphkit_multiple_runners(list, iname => piece.max).plot_area_size
			begin_area = graphkit_multiple_runners(list, iname => piece.min).plot_area_size
			axes = [:x, :y, :z]
			for i in 0...end_area.size
				next unless options[:normalize_pieces].include? axes[i]
				min = [end_area[i][0], begin_area[i][0]].min
				max = [end_area[i][1], begin_area[i][1]].max
				key =  axes[i]

				plot_size[key + :range] = [min, max]
			end
			end
			eputs 'making graphs...'; sleep 1
			graph_array = graphkit_multiple_runners_with_frame_array(piece, list, {:in => iname}, myrank==0)
# 				ep graph_array
			eputs
			graph_array.each_with_index do |(frame_index,g), pindex|
				if myrank == 0
					eputs "\033[2A" # Terminal jargon - go back one line
					eputs sprintf("Plotting graphs: %2.2f", pindex.to_f/piece.size.to_f * 100.0) + "% Complete"
				end
# 					g = graph_kit_multiple_runners(list, plot_size + {frame_index: frame_index})
				
				g.modify(plot_size)
				g.modify(options)
# 					p g; exit
			        g.title +=  sprintf(", frame %0#{fd}d", frame_index) unless options[:frame_title] == false
				folder = ("film_frames/"); 
				file_name = sprintf("frame_%0#{fd}d", actual_frames[frame_index])
# 					g.gnuplot; gets; g.close
# 					ep folder + file_name + '.png'; gets
				g.gnuplot_write(folder + file_name + extension)
			end
		end
	end
	eputs "Waiting on subprocesses..."
	Process.waitall
	end
	unless options[:skip_encoding]
	eputs "making film"
	frame_rate = (options[:frame_rate] or options[:fr] || 15)
	film_name = (options[:film_name] or options [:fn] or end_graphkit.file_name + '_film').gsub(/\s/, '_')
	puts `ffmpeg -y #{options[:bitrate] ? "-b #{options[:bitrate]}" : ""} -r #{frame_rate} -threads #{(@multiple_processes or 1)} -i film_frames/frame_%0#{fd}d#{extension} -qscale 0 #{film_name}.mp4`
	end
end

.manual(copts = {}) ⇒ Object 



790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
 
# File 'lib/coderunner/class_methods.rb', line 790

def self.manual(copts={})
	help = <<EOF

	
-------------CodeRunner Manual---------------

Written by Edmund Highcock (2009)

NAME

coderunner


SYNOPSIS
	
coderunner <command> [arguments] [options]


DESCRIPTION
	
CodeRunner is a framework for the running and analysis of large simulations. It is a Ruby package and can be used to write Ruby scripts. However it also has a powerful command line interface. The aim is to be able to submit simulations, analyse data and plot graphs all using simple commands. This manual is a quick reference. For a more tutorial style introduction to CodeRunner go to      
     http://coderunner.sourceforge.net
     
This help page documents the commandline interface. For API documentation see  
     http://coderunner.sourceforge.net/api_documentation

As is standard, <> indicates a parameter to be supplied, and [] indicates an option, unless otherwise stated.

EXAMPLES

 $ coderunner sub -p '{height: 34.2, width: 221}' -n 24x4 -W 300
 
 $ coderunner can 34 -U
 
 $ coderunner plot -G 'height:width;{};depth==2.4 and status == :Completed;height'
 
 $ coderunner st -ul
 
 $ coderunner rc 'p status' -U

COMMANDS

 Either the long or the short form of the command may be used, except in interactive mode, where only short form can be used.

  Long(Short)  <Arguments>  (Meaningful Options)  
  ---------------------------------------------

#{(COMMANDS_WITH_HELP.sort_by{|arr| arr[0]}.map do |arr| 
  sprintf(" %s %s(%s) \n\t%s", "#{arr[0]}(#{arr[1]})",    arr[4].map{|arg| "<#{arg}>"}.join(' ').sub(/(.)$/, '\1 '), arr[5].map{|op| op.to_s}.join(','), arr[3], )
  end).join("\n\n")}

OPTIONS

#{((COMMAND_LINE_FLAGS_WITH_HELP + LONG_COMMAND_LINE_OPTIONS).map do |arr|
 sprintf("%-15s %-2s\n\t%s", arr[0], arr[1], arr[3])
end).join("\n\n")
}

EOF
 #help.gsub(/(.{63,73} |.{73})/){"#$1\n\t"}.paginate
 help.paginate
end

.netcdf_plot(netcdf_file, vars, indices, copts = {}) ⇒ Object

end



137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
 
# File 'lib/coderunner/class_methods.rb', line 137

def self.netcdf_plot(netcdf_file, vars, indices, copts={})
	process_command_options(copts)
	begin     
		require "numru/netcdf"
	rescue LoadError
			eputs "Error: No Ruby NetCDF library (was it installed correctly?): data analysis for netcdf files not possible."
			return
	end
	start_indices = indices.split(',').map{|idx|  idx = idx.split(':')[0] if idx =~ /:/ ; eval(idx) || 0}
	end_indices = indices.split(',').map{|idx| idx = idx.split(':')[1] if idx =~ /:/ ; eval(idx) || -1}
		
	ep 'start_indices', start_indices, 'end_indices', end_indices
	file = NumRu::NetCDF.open(netcdf_file)
	to_plot = vars.split(',').map do |var|
	  ep 'var', var
		[file.var(var).get('start'=> start_indices, 'end'=> end_indices).to_a.flatten]
	end
	ep 'to_plot', to_plot
	kit = GraphKit.quick_create(*to_plot)
	ep 'copts', copts
	kit.instance_eval(copts[:w]) if copts[:w]
	kit.gnuplot
	STDIN.gets
	kit.close
end

.old_get_run_class_name(code, modlet) ⇒ Object

:nodoc:



341
342
343
 
# File 'lib/coderunner/instance_methods.rb', line 341

def self.old_get_run_class_name(code, modlet) # :nodoc:
	return modlet ? "#{code.capitalize}#{modlet.capitalize.sub(/\.rb$/, '').sub(/_(\w)/){"#$1".capitalize}}Run" : "#{code.capitalize}Run"
end

.parameter_scan(parameter_scan_array_file, copts = {}) ⇒ Object 



408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
 
# File 'lib/coderunner/class_methods.rb', line 408

def self.parameter_scan(parameter_scan_array_file, copts={})
	parameter_scan_array = eval(File.read(parameter_scan_array_file))
# 	  	process_copts(copts)
	runner = fetch_runner(copts)
	skip = true unless copts[:k] == false
	folder = Dir.pwd
	Log.logf("self.parameter_scan")
# 		@@runners = {}
	@@mutex = Mutex.new
# 		@runner = new(folder, code: copts[:C], modlet: copts[:m], version: copts[:v], executable: copts[:X])
	@@psppipe = PPipe.new(parameter_scan_array.size + 2,  true, controller_refresh: 0.5, redirect: false)
	parameter_scan_array.each do |parameter_scan|
			@@psppipe.fork do
				runner.parameter_scan(parameter_scan, copts[:p][0], skip: skip, nprocs: copts[:n])
			end
	end
	@@psppipe.finish
	@@psppipe = nil
end

.plot_graph(copts = {}) ⇒ Object 



427
428
429
430
431
432
433
434
435
436
437
 
# File 'lib/coderunner/class_methods.rb', line 427

def self.plot_graph(copts = {})
# 		process_copts(copts)
	runner = fetch_runner(copts)
	string_to_eval = copts[:w]
	#options = (options and options =~ /\S/) ? eval(options): {}
	eputs 'Starting Graph'
	kit = runner.graphkit_from_lists(copts[:G], copts[:g])
	kit.gnuplot(eval: string_to_eval)
	gets
	kit.close
end

.print_queue_status(copts = {}) ⇒ Object 



165
166
167
168
169
170
171
172
173
174
 
# File 'lib/coderunner/class_methods.rb', line 165

def self.print_queue_status(copts={})
	begin 
		eputs queue_status
	rescue => err
		eputs "General queue status doesn't work on this system; showing queue status for this folder"
# 			p err
		runner = fetch_runner(copts)
		eputs runner.queue_status
	end
end

.process_command_line_option(opt, arg, copts) ⇒ Object

Converts a command line flag opt with value arg to a command option which is stored in copts



274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
 
# File 'lib/coderunner.rb', line 274

def self.process_command_line_option(opt, arg, copts)
	case opt   
	when "--change-directory"
		copts[:c] = arg.to_i
	when "--delete"
		copts[:x] = arg.to_i
	when "--graph"
		copts[:G].push arg
	when "--run-graph"
		copts[:g].push arg
# 		when "--cancel"
# 			copts[:K] = arg.to_i
	when "--multiple-processes"
		copts[:M] = arg.to_i
	when "--film"
		copts[:F] = (arg or true)
	when "--recheck"
		copts[:R] = arg.to_i
	when "--use-large-cache-but-recheck-incomplete"
		copts[:U] = true
		copts[:u]=true
	when "--wall-mins"
		copts[:W] = arg.to_i
	when "--use-phantom"
		copts[:h] = (arg and arg =~ /\S/) ? arg.to_sym : :phantom 
	when "--terminal-size"
		array = eval arg
		ENV['ROWS'], ENV['COLS'] = array[0].to_s, array[1].to_s 
	when "--interactive-mode"
		@@interactive_mode = true
	when "--parameters"
		copts[:p].push arg
	else
		if CLF_BOOLS.include? CLF_TO_SHORT_COPTS[opt]
			copts[CLF_TO_SHORT_COPTS[opt]] = true
		elsif CLF_INVERSE_BOOLS.include? CLF_TO_SHORT_COPTS[opt]
			copts[CLF_TO_SHORT_COPTS[opt]] = false
		elsif CLF_TO_SHORT_COPTS[opt] # Applies to most options
			copts[CLF_TO_SHORT_COPTS[opt]] = arg
		elsif CLF_BOOLS.include? CLF_TO_LONG[opt]
			copts[CLF_TO_LONG[opt]] = true
		elsif CLF_INVERSE_BOOLS.include? CLF_TO_LONG[opt]
			copts[CLF_TO_LONG[opt]] = false
		elsif CODE_COMMAND_OPTIONS.map{|o| o[0]}.include? opt
			begin
				#copts[:code_copts] ||= {}
				copts[
					#CLF_TO_LONG[opt].to_s.sub('_options','').to_sym
					CLF_TO_LONG[opt]
				] = eval(arg)
			rescue SyntaxError => err
				eputs "\nOption #{opt} must be a hash\n\n"
				raise err
			end

		elsif CLF_TO_LONG[opt]
			copts[CLF_TO_LONG[opt]] = arg
		else 
			raise "Unknown command line argument: #{opt}"
		end	
	end
	copts
end

.process_command_options(copts) ⇒ Object 



669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
 
# File 'lib/coderunner/class_methods.rb', line 669

def self.process_command_options(copts)
	if copts[:true]
		copts[:true].to_s.split(//).each do |letter|
			copts[letter.to_sym] = true
		end
	end
	if copts[:false]
		copts[:false].to_s.split(//).each do |letter|
			copts[letter.to_sym] = false
		end
	end

	read_default_command_options(copts)
	copts.each do |key, value|
		copts[LONG_TO_SHORT[key]] = value if LONG_TO_SHORT[key]
	end

	
	if copts[:j] # j can be a number '65' or list of numbers '65,43,382' 
		copts[:f]= "#{eval("[#{copts[:j]}]").inspect}.include? id"
	end
	if copts[:X] and FileTest.exist? copts[:X]
		copts[:X] = File.expand_path(copts[:X])
	end
	if copts[:z]
		Log.log_file = Dir.pwd + '/.cr_logfile.txt'
		Log.clean_up
	else 
		Log.log_file = nil
	end
	copts[:F] = (copts[:F].class == Hash ? copts[:F] : (copts[:F].class == String and copts[:F] =~ /\A\{.*\}\Z/) ? eval(copts[:F]) : {})
	copts[:G]= [copts[:G]] if copts[:G].kind_of? String
	copts[:g]= [copts[:g]] if copts[:g].kind_of? String
# 		if copts[:p] and copts[:p].class == String # should be a hash or an inspected hash
# 			copts[:p] = eval(copts[:p])
# 		end
	copts[:p] = [copts[:p]].compact unless copts[:p].class == Array
	#for i in 0...copts[:p].size

	copts[:Y] ||= DEFAULT_COMMAND_OPTIONS[:Y] if DEFAULT_COMMAND_OPTIONS[:Y]
	if copts[:Y] and copts[:Y] =~ /:/ 
		set_class_defaults(copts)
		copts[:running_remotely] = true
	else
		copts[:Y] = copts[:Y].gsub(/~/, ENV['HOME']) if copts[:Y]
		Dir.chdir((copts[:Y] or Dir.pwd)) do
			set_runner_defaults(copts)
# 				ep DEFAULT_RUNNER_OPTIONS
		end
	end
# 		ep Log.log_file
	#copts[:code_copts].each{|k,v| CODE_OPTIONS[k] = v} if copts[:code_copts]
	copts.keys.map{|k| k.to_s}.grep(/_options$/).map{|k| k.to_sym}.each do |k|
 		CODE_OPTIONS[k.to_s.sub('_options','').to_sym] = copts[k]
	end	
		
end

.read_default_command_options(copts) ⇒ Object 



664
665
666
667
668
 
# File 'lib/coderunner/class_methods.rb', line 664

def self.read_default_command_options(copts)
	DEFAULT_COMMAND_OPTIONS.each do |key, value|
		copts[key] ||= value
	end
end

.readout(copts = {}) ⇒ Object 



438
439
440
441
442
 
# File 'lib/coderunner/class_methods.rb', line 438

def self.readout(copts={})
# 		process_copts(copts)
	runner = fetch_runner(copts)
	puts runner.readout
end

.recheck(id, copts = {}) ⇒ Object 



443
444
445
446
447
448
 
# File 'lib/coderunner/class_methods.rb', line 443

def self.recheck(id, copts={})
# 		process_copts(copts)
	runner = fetch_runner(copts)
	runner.run_list[copts[:R]].recheck
	runner.respond_to_requests
end

.reference(class_or_method, copts = {}) ⇒ Object 



180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
 
# File 'lib/coderunner/class_methods.rb', line 180

def self.reference(class_or_method, copts={})
	code_folders = Dir.recursive_entries(SCRIPT_FOLDER + '/code_modules').grep(/\/ri$/).map{|fold| ['-d', fold]}.flatten
# 		ep code_folders
	
# 		require 'rdoc/ri/driver'
	
# "
# 		op = @ri_count ? [] : (@ri_count = true; ['--no-use-cache'])
# 		trap(1){puts 'No help available'}
# 		at_exit{raise ""}
# 		p op
	begin
		eputs "Looking up #{class_or_method}"
		RDoc::RI::Driver.run ['-d', SCRIPT_FOLDER + '/ri', class_or_method.to_s] + code_folders
	rescue => err
		eputs "Unknown class or method or no help available: #{err}"
	end
# 		trap(1){}
end

.repair_marshal_run_class_not_found_error(err) ⇒ Object 



352
353
354
355
356
357
358
359
 
# File 'lib/coderunner/instance_methods.rb', line 352

def self.repair_marshal_run_class_not_found_error(err)
    #ep 'error', err, err.message
		code, modlet = err.message.scan(/CodeRunner\:\:([A-Z][a-z0-9_]+)(?:::([A-Z]\w+))?/)[0]
		#ep 'merror', err, code, modlet; gets
		code.gsub!(/([a-z0-9])([A-Z])/, '\1_\2')
		(modlet.gsub!(/([a-z0-9])([A-Z])/, '\1_\2'); modlet.downcase!) if modlet	
		setup_run_class(code.downcase, modlet: modlet)
end

.resubmit(copts = {}) ⇒ Object

puts “Got here” exit(0)



554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
 
# File 'lib/coderunner/class_methods.rb', line 554

def self.resubmit(copts = {})
# 		process_copts(copts)
	runner = fetch_runner(copts)
# 		raise "something is already submitting" if FileTest.exist? "submitting" 
	runs = []
	raise "Parameters must be an array of inspected hashes" unless copts[:p].kind_of? Array
	Dir.chdir(copts[:Y]) do 
		runs = runner.filtered_ids.map do |id|
			eputs id
			run = runner.run_list[id].dup
			if copts[:smart_resubmit_name]
				eputs "Smart name"
				run.set(:naming_pars,  [:resubmit_id])
				run.resubmit_id = run.id
			end
			run.update_submission_parameters(copts[:p][0], false)
			run.run_name = nil unless copts[:rerun]
			run
		end
	end
	#throw(:here)

	runner.submit(runs, nprocs: copts[:n], version: copts[:v], skip: copts[:k], job_chain: copts[:J], no_update_before_submit: copts[:no_update_before_submit], replace_existing: copts[:replace_existing], smart_resubmit_name: copts[:smart_resubmit_name], rerun: copts[:rerun])
end

.run_class(copts = {}) ⇒ Object 



449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
 
# File 'lib/coderunner/class_methods.rb', line 449

def self.run_class(copts={})
	process_command_options(copts)
	copts[:no_update] = true
	unless copts[:C]
		if FileTest.exist? file=Dir.pwd + '/.code_runner_script_defaults.rb'
			copts[:C] = eval(File.read(file))[:code]
			copts[:m] = eval(File.read(file))[:modlet]
		elsif self.runner
			copts[:C] = self.runner.code
			copts[:m] = self.runner.modlet
		end
	end
	#ep ['code', 'modlet is', copts[:C], copts[:m]]
	            
	return setup_run_class(copts[:C], modlet: copts[:m])
end

.run_command(string, copts = {}) ⇒ Object

runner = fetch_runner(copts) runner.run_class.class_eval(string)



471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
 
# File 'lib/coderunner/class_methods.rb', line 471

def self.run_command(string, copts={})
# 	  	process_copts(copts)
	runner = fetch_runner(copts)
	
	eputs "Calling run_commmand..."
# 		puts "Warning: Use large cache is on (-U or -u) -- no results will be saved" if runner.use_large_cache
	ppipe = PPipe.new(runner.filtered_ids.size + 1, false) if copts[:M]
	no_save = (runner.class == RemoteCodeRunner or copts[:y] =~ /no-save/)
#		runner.generate_combined_ids
#		ep runner.filtered_ids
	runner.filtered_ids.each do |id|
		run = runner.combined_run_list[id]
		
		if no_save or run.is_phantom
			if copts[:M]
				fork{run.instance_eval(string)}
			else
				run.instance_eval(string)
			end
		else
			if copts[:M]
				pn = ppipe.fork do 
					Dir.chdir(run.directory) do
						run.instance_eval(string);
						run.save 
						run.write_results
					end
					ppipe.i_send(id, Marshal.dump(run), tp: 0)
				end
			else
				Dir.chdir(run.directory){run.instance_eval(string); run.save; run.write_results}
			end
			
		end
	end
	unless no_save
		(runner.filtered_ids.each{|id| runner.run_list[id] = Marshal.load(ppipe.w_recv(id).contents)};ppipe.finish) if copts[:M]
		runner.save_large_cache
	end
	
# 		Process.waitall
	runner.respond_to_requests
end

.run_script ⇒ Object 



371
372
373
374
375
376
377
378
379
 
# File 'lib/coderunner.rb', line 371

def self.run_script
	if DEFAULT_COMMAND_OPTIONS[:Z]
		DEFAULT_COMMAND_OPTIONS.absorb(eval(DEFAULT_COMMAND_OPTIONS[:Z])) 
		puts "Begin Output"
	end
	command = COMMANDS.find{|com| com.slice(0..1).include? ARGV[0]}
	raise "\n-------------------\nCommand #{ARGV[0].inspect} not found: try 'coderunner man' for help\n-------------------\n" unless command
	send(command[0].to_sym, *ARGV.values_at(*(1...(1+command[2])).to_a))
end

.runner ⇒ Object 



784
785
786
787
 
# File 'lib/coderunner/class_methods.rb', line 784

def self.runner
	@runners ||={}
	@runners.values[0]
end

.runner_eval(string, copts = {}) ⇒ Object 



514
515
516
517
518
519
520
521
522
523
524
525
526
 
# File 'lib/coderunner/class_methods.rb', line 514

def self.runner_eval(string, copts = {})
# 		 process_copts(copts)
	 runner = fetch_runner(copts)
	 
	 return_val = runner.instance_eval(string)

	 if copts[:Z]
		 Kernel.puts(server_dump(return_val))
	 else
		 return return_val
	 end

end

.scan(scan_string, copts = {}) ⇒ Object 



527
528
529
530
531
 
# File 'lib/coderunner/class_methods.rb', line 527

def self.scan(scan_string, copts={})
# 	  	process_copts(copts)
	runner = fetch_runner(copts)
  	runner.simple_scan(scan_string, nprocs: copts[:n], version: copts[:v], skip: copts[:k], parameters: copts[:p][0])
end

.server_dump(obj) ⇒ Object 



312
313
314
 
# File 'lib/coderunner/instance_methods.rb', line 312

def self.server_dump(obj)
	"code_runner_server_dump_start_E#{Marshal.dump(obj)}code_runner_server_dump_end_E"
end

.set_class_defaults(copts = {}) ⇒ Object 



13
14
15
16
17
18
 
# File 'lib/coderunner/class_methods.rb', line 13

def self.set_class_defaults(copts={})
	(CLASS_OPTIONS.keys - []).each do |var|
		CLASS_OPTIONS[var] =  copts[LONG_TO_SHORT[var]]
		set(var, CLASS_OPTIONS[var])
	end
end

.set_default_command_options_from_command_line ⇒ Object 



342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
 
# File 'lib/coderunner.rb', line 342

def self.set_default_command_options_from_command_line

	#some defaults
# 		DEFAULT_COMMAND_OPTIONS[:p] ||= {}
	DEFAULT_COMMAND_OPTIONS[:v] ||= ""
	#DEFAULT_COMMAND_OPTIONS[:n] ||= "1"
	DEFAULT_COMMAND_OPTIONS[:G] = []
	DEFAULT_COMMAND_OPTIONS[:g] = []
	DEFAULT_COMMAND_OPTIONS[:k] = true
	DEFAULT_COMMAND_OPTIONS[:h] ||= :real
	DEFAULT_COMMAND_OPTIONS[:p] = []

# 		ep COMMAND_LINE_FLAGS
	opts = GetoptLong.new(*(COMMAND_LINE_FLAGS + LONG_COMMAND_LINE_FLAGS))
	opts.each do |opt, arg|
	      process_command_line_option(opt, arg, DEFAULT_COMMAND_OPTIONS)
	end
	#raise "\n\nCannot use large cache ('-U' or '-u' ) if submitting runs" if DEFAULT_COMMAND_OPTIONS[:U] and (DEFAULT_COMMAND_OPTIONS[:s] or DEFAULT_COMMAND_OPTIONS[:P])


	if DEFAULT_COMMAND_OPTIONS[:z]
		Log.log_file = Dir.pwd + '/.cr_logfile.txt'
		Log.clean_up
	else 
		Log.log_file = nil
	#	puts Log.log_file
	end
end

.set_runner_defaults(copts = {}) ⇒ Object

In the next section are the implementations of all the standard Code Runner commands and some helper functions.



5
6
7
8
9
10
11
 
# File 'lib/coderunner/class_methods.rb', line 5

def self.set_runner_defaults(copts = {})
  	(DEFAULT_RUNNER_OPTIONS.keys - [:sys, :script_folder]).each do |var|
		DEFAULT_RUNNER_OPTIONS[var] =  copts[LONG_TO_SHORT[var]]
	end
	set_class_defaults(copts)
# 		ep DEFAULT_RUNNER_OPTIONS
end

.setup_run_class(code, options = {}) ⇒ Object

Raises:



368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
 
# File 'lib/coderunner/instance_methods.rb', line 368

def self.setup_run_class(code, options={})
# 		logf(:setup_code)
# 		log(:code, code)
		
	modlet = options[:modlet]
	version = options[:version]
# 		log('modlet in setup_run_class', modlet)
	eputs "Loading modules for #{code}, #{modlet.inspect}..."
	
# 		modlet = modlet.sub(/\.rb$/, '') if modlet
	raise CRFatal.new("Code must contain only lowercase letters, digits, and underscore, and must begin with a letter or underscore; it is '#{code}'") unless code =~ /\A[a-z_][a-z_\d]*\Z/ 
	raise CRFatal.new("Input_file_module must contain only lowercase letters, digits, and underscore, and must begin with a letter or underscore; it is '#{modlet}'") if modlet and not modlet =~ /\A[a-z_][a-z_\d]*\Z/  

	run_class_name = get_run_class_name(code, modlet)

# 		p run_class_name

	return recursive_const_get(run_class_name) if SETUP_RUN_CLASSES.include?(run_class_name) #constants.include? (run_class_name).to_sym unless options[:force] #and const_get(run_class_name).rcp.code?
	#return const_get(run_class_name) if constants.include? (run_class_name).to_sym unless options[:force] #and const_get(run_class_name).rcp.code?
	SETUP_RUN_CLASSES.push run_class_name #.downcase
	FileUtils.makedirs(ENV['HOME'] + "/.coderunner/#{code}crmod/")
	FileUtils.makedirs(ENV['HOME'] + "/.coderunner/#{code}crmod/defaults_files")

	#Create the run_class, a special dynamically created class which knows how to process runs of the given code on the current system.
	#run.rb contains the basic methods of the class
# 		puts run_class_name; gets
# 		run_class = add_a_child_class(run_class_name, "")
	
# 		run_class.class_eval(%[
# 			@@code=#{code.inspect}; @@version=#{version.inspect}
# 			@@modlet=#{modlet.inspect} #modlet should be nil if not @@modlet_required
# 		        SYS = #{SYS.inspect}
# 		        include Log
# 			Log.logi(:code_just_after_runfile, @@code) 
# 			]
# 		)


	
	code_module_name = SCRIPT_FOLDER+ "/code_modules/#{code}/#{code}.rb"
	code_module_name = "#{code}crmod"
	require code_module_name
# 		ep get_run_class_name(code, nil)
	run_class = const_get(get_run_class_name(code, nil))
	run_class.instance_variable_set(:@code, code)
	run_class.instance_variable_set(:@sys, SYS)

	raise "#{run_class} must inherit from CodeRunner::Run: its ancestors are: #{run_class.ancestors}" unless run_class.ancestors.include? Run
	
	if options[:runner]
		run_class.runner = options[:runner]
		run_class.instance_variable_set(:@runner, options[:runner])
	end
	#Add methods appropriate to the current system

	system_module_name = SCRIPT_FOLDER+ "/system_modules/#{SYS}.rb"
	require system_module_name
	run_class.send(:include, const_get(SYS.variable_to_class_name))
# 		run_class.class_eval(File.read(system_module_name), system_module_name)


	
	#Some codes require an modlet; the flag modlet_required is specified in the code module
	if run_class.rcp.modlet_required
		raise CRFatal.new("Modlet necessary and none given") unless modlet
	end
	#If a modlet is specified (modlets can also be optional)
	if modlet
# 			Log.logi(:modlet, modlet)
		#if (	modlet_location =  "#{SCRIPT_FOLDER}/code_modules/#{code}/default_modlets";
		        #file_name = "#{modlet_location}/#{modlet}.rb"; 
			#FileTest.exist? file_name
		#)
		#elsif (		modlet_location =  "#{SCRIPT_FOLDER}/code_modules/#{code}/my_modlets";
		        #file_name = "#{modlet_location}/#{modlet}.rb"; 
			#FileTest.exist? file_name
		#)
		#else
			#raise CRFatal.new("Could not find modlet file: #{modlet}.rb")
		#end
		#require file_name
# 			ep run_class.constants
# 			ep run_class_names
		
	  modlet_file = "#{code}crmod/#{modlet}.rb"
		#ep ['requiring modlet file']
		require modlet_file
		run_class = recursive_const_get(run_class_name)
		run_class.instance_variable_set(:@modlet, modlet)
	end

	run_class.check_and_update
# 		log("random element of variables", run_class.variables.random)


# 		logi("run_class.variables[0] in setup_code", run_class.variables[0])
# 			ep 'finished'

	return run_class
end

.show_values_of(expression, copts = {}) ⇒ Object 



609
610
611
612
 
# File 'lib/coderunner/class_methods.rb', line 609

def self.show_values_of(expression, copts={})
	runner = fetch_runner(copts)
	p runner.filtered_ids.map{|id| runner.run_list[id].instance_eval(expression)}.uniq.sort
end

.start_launcher(refresh, max_queue, copts = {}) ⇒ Object 



324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
 
# File 'lib/coderunner/class_methods.rb', line 324

def self.start_launcher(refresh, max_queue, copts={})
#eputs "Starting launcher!"
raise "Raise refresh is #{refresh}: it must be >= 0.1" if refresh.to_f < 0.1
raise "Raise max_queue is #{max_queue}: it must be >= 5" if max_queue.to_i < 5
  #raise "Launcher already running" if %x[ps -e -o cmd].split("\n").grep(/coderunner\s+launch/).size > 0
  require 'thread'
  tl = launcher_directory #SCRIPT_FOLDER + '/to_launch'
#exit unless Feedback.get_boolean( "Launch directory #{tl} already exists: it is suggested that you change the prefix by changing the environment variable CODE_RUNNER_LAUNCHER. Do you wish to continue (don't select yes unless you know what you are doing)?") if FileTest.exist? tl
raise "Launch directory #{tl} already exists: it is suggested that you change the prefix by changing the environment variable CODE_RUNNER_LAUNCHER. Do you wish to continue (don't select yes unless you know what you are doing)?" if FileTest.exist? tl
#     FileUtils.rm_r tl if FileTest.exist? tl
  eputs "Starting launcher\n"
at_exit{FileUtils.rm_r tl}
  FileUtils.makedirs tl
Thread.new{loop{`cp #{tl}/queue_status.txt #{tl}/queue_status2.txt; ps > #{tl}/queue_status.txt`; sleep 1}}

  mutex = Mutex.new
  processes= []

Thread.new do
	loop do
      Dir.entries(tl).each do |file|
        next unless file =~ (/(^.*)\.stop/)
			pid = $1
			mutex.synchronize{Process.kill(pid); processes.delete pid}
		end
		sleep refresh.to_i
	end
end

  #Dir.chdir(tl) do
    ppid = $$
    loop do
		sleep refresh.to_i while processes.size >= max_queue.to_i
#         processes = []
      Dir.entries(tl).grep(/(^.*)\.start/).each do |file|
        file =~ (/(^.*)\.start/)
        id = $1
			command = ""
        command = File.read tl + '/' + file while command == ""
        pid = fork do
          processes.each do |wpid|
					# Make sure all previously submitted jobs have finished.
            sleep refresh.to_i while %x[ps -e -o pid,ppid].split("\n").grep(Regexp.new("^\\s*#{wpid}\\s+#{ppid}")).size > 0
          end              
				#p ["command", command]
          exec(command)
        end
			`cp #{tl}/queue_status.txt #{tl}/queue_status2.txt; ps > #{tl}/queue_status.txt`
        mutex.synchronize{processes.push pid}

        File.open(tl + '/' + id + '.pid', 'w'){|file| file.puts pid}
        FileUtils.rm(tl + '/' + file)
        
        Thread.new{Process.wait pid; mutex.synchronize{processes.delete pid}}
      end
#         processes.each{|pid| Process.wait pid}
      sleep refresh.to_i
    end
  #end
end

.status(copts = {}) ⇒ Object 



617
618
619
620
621
 
# File 'lib/coderunner/class_methods.rb', line 617

def self.status(copts={})
# 		process_copts(copts)
	runner = fetch_runner(copts)
	runner.print_out(0, with_comments: copts[:with_comments]) unless copts[:interactive_start] or copts[:Z] or copts[:no_print_out]
end

.status_loop(copts = {}) ⇒ Object 



622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
 
# File 'lib/coderunner/class_methods.rb', line 622

def self.status_loop(copts={})
# 		process_copts(copts)
	runner = fetch_runner(copts)
	runner.print_out(0, with_comments: copts[:with_comments]) unless copts[:interactive_start] or copts[:Z] or copts[:no_print_out]
	break_out = false
	loop do 
		old_trap = trap(2){eputs " Terminating loop, please wait..."; break_out = true}
		runner.use_large_cache = true
		runner.update(false)
		(trap(2, old_trap); break) if break_out
		runner.recheck_filtered_runs(false)
		runner.print_out(nil, with_comments: copts[:with_comments])
		trap(2, old_trap)
		break if break_out
		break if not runner.run_list.values.find do |r| 
			not [:Complete, :Failed].include? r.status
		end
		#ep "sleep"
	 	sleep 3
		#ep "end sleep"
	end
end

.status_with_comments(copts = {}) ⇒ Object 



613
614
615
616
 
# File 'lib/coderunner/class_methods.rb', line 613

def self.status_with_comments(copts={})
	copts[:with_comments] = true
	status(copts)
end

.submit(copts = {}) ⇒ Object 



532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
 
# File 'lib/coderunner/class_methods.rb', line 532

def self.submit(copts = {})
# 		process_copts(copts)
	runner = fetch_runner(copts)
# 		raise "something is already submitting" if FileTest.exist? "submitting" 
	runs = []
	raise "Parameters must be an array of inspected hashes" unless copts[:p].kind_of? Array
	Dir.chdir(copts[:Y]) do 

		copts[:p].push nil if copts[:p] == []
# 				ep copts[:p]; exit
		copts[:p].each do |pars|
			run = runner.run_class.new(runner)
# 			p pars
			run.update_submission_parameters(pars)
			runs.push run
		end
# 			exit
	end
	runner.submit(runs, nprocs: copts[:n], version: copts[:v], skip: copts[:k], job_chain: copts[:J], no_update_before_submit: copts[:no_update_before_submit])
	#puts "Got here"
	#exit(0)
end

.submit_command(jid, comm, copts = {}) ⇒ Object

This method allows the straightforward submission of a single command using the batch queue on any system.



580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
 
# File 'lib/coderunner/class_methods.rb', line 580

def self.submit_command(jid, comm, copts={})
	process_command_options(copts)
	submitter = Object.new
	submitter.instance_variable_set(:@command, comm)
	submitter.instance_variable_set(:@jid, jid)
	submitter.instance_variable_set(:@nprocs, copts[:n])
	submitter.instance_variable_set(:@wall_mins, copts[:W])
	submitter.instance_variable_set(:@project, copts[:P])
	class << submitter
		include CodeRunner::SYSTEM_MODULE
		def executable_name
			'custom'
		end
		def job_identifier 
			@jid
		end
		def run_command
			@command
		end
	end
	submitter.execute
end

.update_runners ⇒ Object

@r.read_defaults



778
779
780
781
 
# File 'lib/coderunner/class_methods.rb', line 778

def self.update_runners
	@runners ||= {}
	@runners.each{|runner| runner.update}
end

.write_graph(name, copts = {}) ⇒ Object 



644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
 
# File 'lib/coderunner/class_methods.rb', line 644

def self.write_graph(name, copts={})
# 		process_copts(copts)
	runner = fetch_runner(copts)
	eputs 'Starting Graph'
	kit = runner.graphkit_from_lists(copts[:G], copts[:g])
	#options = copts[:w]
	#options = (options and options =~ /\S/) ? eval(options): {}
	name = nil unless name =~ /\S/
	max = 0
	name.sub!(/^\~/, ENV['HOME']) if name
	if name and name =~ /%d\./	
		regex = Regexp.new(Regexp.escape(File.basename(name)).sub(/%d/, '(?<number>\d+)'))
		Dir.entries(File.dirname(name)).join("\n").scan(regex) do
			max = [max, $~[:number].to_i].max
		end
		name = name.sub(/%d/, (max + 1).to_s)
	end
	raise "kit doesn't have a file_name and no filename specified; can't write graph" unless name or (kit.file_name.class == String and kit.file_name =~ /\S/)
	Dir.chdir(COMMAND_FOLDER){kit.gnuplot_write((name or kit.file_name), {eval: copts[:w]})}
end

Instance Method Details

#add_phantom_run(run) ⇒ Object

def rename_variable(old, new) puts “Please confirm complete renaming of #old to #new” gets @run_list.each do |directory| Dir.chdir directory do begin @readme = File.read(“CODE_RUNNER_INPUTS”) rescue Errno::ENOENT => err # puts err, err.class @readme = File.read(“README”) end @readme.sub!(Regexp.new(“^s+#old:”), “^s+#new:”) File.open(“CODE_RUNNER_INPUTS_TEST”, ‘w’){|file| file.puts @readme} end end old_recalc, @recalc_all = @recalc_all, true update @recalc_all = old_recalc end



1603
1604
1605
1606
1607
1608
1609
 
# File 'lib/coderunner/instance_methods.rb', line 1603

def add_phantom_run(run)
	@phantom_run_list[@phantom_id] = run
	@phantom_ids.push @phantom_id
	#run.real_id = run.id
	run.id = @phantom_id
	@phantom_id += -1
end

#alter_ids(num, options = {}) ⇒ Object

Permanently change the id of every run in the folder by adding num to them. Num can be negative unless it makes any of the ids negative. Use if you want to combine these runs with the runs in another folder, either by creating an instance of CodeRunner::Merge, or by directly copying and pasting the run folders.



1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
 
# File 'lib/coderunner/instance_methods.rb', line 1662

def alter_ids(num, options={})
	Dir.chdir(@root_folder) do
		return unless options[:no_confirm] or Feedback.get_boolean("This will permanently alter all the ids in the folder #@root_folder. Scripts that use those ids may be affected. Do you wish to continue?")
		raise ArgumentError.new("Cannot have negative ids") if @run_list.keys.min + num < 0
		runs = @run_list.values
		fids = filtered_ids
		@run_list = {}
		runs.each do |run|
			old_id = run.id
			if fids.include? old_id
				run.id += num
				old_dir = run.relative_directory
				new_dir = old_dir.sub(Regexp.new("id_#{old_id}(\\D|$)")){"id_#{run.id}#$1"}
# 				ep old_dir, new_dir
				FileUtils.mv(old_dir, new_dir)
				run.relative_directory = new_dir
				run.directory = File.expand_path(new_dir)
			end
			@run_list[run.id] = run
		end
		@ids = @run_list.keys
		set_max_id(@ids.max || 0)
		save_all_and_overwrite_info
	end
end

#axiskit(string) ⇒ Object

def gnuplot(axes, options) a = graphkit(axes, options).gnuplot end



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'lib/coderunner/graphs_and_films.rb', line 315

def axiskit(string)
	generate_combined_ids
	data = filtered_ids.inject([]) do |arr, id|
		begin
			arr.push @combined_run_list[id].instance_eval(string)
		rescue => err
			eputs "Evaluation of #{string} failed for run #{id}"
			raise err
		end
	end
# 		p data
	return GraphKit::AxisKit.new(title: string, data: data)
end

#cancel_job(id, options = {}) ⇒ Object

Cancel the job with the given ID. Options are: :no_confirm —> true or false, cancel without asking for confirmation if true :delete —> if (no_confirm and delete), delete cancelled run



1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
 
# File 'lib/coderunner/instance_methods.rb', line 1562

def cancel_job(id, options={})
	@run=@run_list[id]
	raise "Run with id #{id} does not exist" unless @run
	unless options[:no_confirm]
		eputs "Cancelling job: #{@run.job_no}: #{@run.run_name}. \n Press enter to confirm"
# 		puts 'asfda'
		gets
	end
	@run.cancel_job
	if options[:no_confirm] 
		delete =  options[:delete]
	else
		delete = Feedback.get_boolean("Do you want to delete the folder (#{@run.directory}) as well?")
	end
	FileUtils.rm_r(@run.directory) if delete and @run.directory and not @run.directory == ""
	update
	print_out
end

#code_run_environment ⇒ Object 



1219
1220
1221
 
# File 'lib/coderunner/instance_methods.rb', line 1219

def code_run_environment
	run_class.new(self).code_run_environment
end

#continue_in_new_folder(folder, options = {}) ⇒ Object 



1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
 
# File 'lib/coderunner/instance_methods.rb', line 1688

def continue_in_new_folder(folder, options={})
	Dir.chdir(@root_folder) do
		raise "Folder already exists" if FileTest.exist?(folder)
		FileUtils.makedirs("#{folder}/v")
		#FileUtils.makedirs(folder)
		FileUtils.cp(".code_runner_script_defaults.rb", "#{folder}/.code_runner_script_defaults.rb")
		FileUtils.cp(".code-runner-irb-save-history", "#{folder}/.code-runner-irb-save-history")
		FileUtils.cp("#{@defaults_file}_defaults.rb", "#{folder}/#{@defaults_file}_defaults.rb")
		if options[:copy_ids]
			options[:copy_ids].each do |id|
				FileUtils.cp_r(@run_list[id].directory, "#{folder}/v/id_#{id}")
			end
		end

	end
end

#create_archive(options = {}) ⇒ Object

Create a tar archive of the root folder and all the files in it. Options are :compression => true or false :folder => folder in which to place the archive. :verbose => true or false :group => group of new files



1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
 
# File 'lib/coderunner/instance_methods.rb', line 1711

def create_archive(options={})
	verbose = options[:verbose] ? 'v' : ''
	very_verbose = options[:very_verbose] ? 'v' : ''
	comp = options[:compression]
	Dir.chdir(@root_folder) do
		temp_root = ".tmparch/#{File.basename(@root_folder)}"
		FileUtils.makedirs(temp_root)
		system "chgrp  #{options[:group]} #{temp_root}" if options[:group]
		size=@run_list.size
		@run_list.values.each_with_index do |run, index|
			archive_name = "#{File.basename(run.directory)}.tar#{comp ? 
								'.gz' : 
								''}"
			tar_name = archive_name.delsubstr('.gz')
			relative = run.directory.delete_substrings(@root_folder, File.basename(run.directory))
			FileUtils.makedirs(temp_root +  relative)
			unless FileTest.exist? temp_root +  relative + archive_name
			eputs "Archiving #{index} out of #{size}" if options[:verbose]
				Dir.chdir(run.directory + '/..') do
						command =  "tar -cW#{very_verbose}f #{tar_name} #{File.basename(run.directory)}"
						eputs command if options[:verbose]
						unless system command
							raise "Archiving failed"
						end
						break unless comp
						command = "gzip -4 -vf #{tar_name}"
						eputs command if options[:verbose]
						unless system command
							raise "Compression failed"
						end
						command = "gzip -t #{archive_name}"
						eputs command if options[:verbose]
						unless system command
							raise "Compression failed"
						end
						#exit
				end
				FileUtils.mv(relative.delsubstr('/') + archive_name, temp_root + '/' + relative + archive_name)
			end
			
			system "chgrp -R #{options[:group]} #{temp_root}" if options[:group]
		end
		Dir.entries.each do |file|
			case file
			when '.', '..', '.tmparch'
				next
			when /^v/
				next unless File.file? file
			else
				FileUtils.cp_r(file, "#{temp_root}/#{file}")
			end
		end
		Dir.chdir('.tmparch') do
	  	command = "tar -cWv --remove-files -f #{File.basename(@root_folder)}.tar #{File.basename(@root_folder)}"
	  	command = "tar -cWv -f #{File.basename(@root_folder)}.tar #{File.basename(@root_folder)}"
			eputs command if options[:verbose]
			raise "Archiving Failed" unless system command
		end
		FileUtils.mv(".tmparch/#{File.basename(@root_folder)}.tar", "#{File.basename(@root_folder)}.tar")
		#FileUtils.rm_r(".tmparch")
	end
end

#destroy(options = {}) ⇒ Object

Delete the folders of all runs for whom CodeRunner#filter(run) is true. This will permanently erase the runs. This is an interactive method which asks for confirmation.



1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
 
# File 'lib/coderunner/instance_methods.rb', line 1539

def destroy(options={})
	ids = @ids.find_all{|id| filter @run_list[id]}
	unless options[:no_confirm]
		logf(:destroy)			
		puts "About to delete:"
		ids.each{|id| eputs @run_list[id].run_name}
		return unless Feedback.get_boolean("You are about to DESTROY #{ids.size} jobs. There is no way back. All data will be eliminated. Please confirm the delete.")
		#gets
		eputs "Please confirm again. Press Enter to confirm, Ctrl + C to cancel"
		gets
	end
	ids.each{|id| 
		FileUtils.rm_r @run_list[id].directory if @run_list[id].directory and not ["", ".", ".."].include? @run_list[id].directory
		@run_list.delete(id); @ids.delete(id); generate_combined_ids}
	set_max_id(@ids.max || 0)
	save_large_cache
	generate_combined_ids
end

#dup ⇒ Object

Duplicate the runner, trying to be intelligent as far as possible in duplicating instance variables. Not fully correct yet. Avoid using at the moment.



1528
1529
1530
1531
1532
1533
1534
1535
 
# File 'lib/coderunner/instance_methods.rb', line 1528

def dup
	logf(:dup)
	new_one = self.class.new(@code, @root_folder, modlet: @modlet, version: @version)
	new_one.ids = @ids.dup; new_one.phantom_ids = @phantom_ids.dup;
	new_one.run_list = @run_list.dup; new_one.phantom_run_list = @phantom_run_list.dup
	new_one.run_class = @run_class
	return new_one
end

#executable_location ⇒ Object 



1214
1215
1216
1217
 
# File 'lib/coderunner/instance_methods.rb', line 1214

def executable_location
	return '' unless @executable
	File.dirname(@executable)
end

#executable_name ⇒ Object 



1209
1210
1211
1212
 
# File 'lib/coderunner/instance_methods.rb', line 1209

def executable_name
	return 'job_chain' unless @executable
	File.basename(@executable)
end

#film_graphkit(axes, options, film_options = {}) ⇒ Object 



408
409
410
411
 
# File 'lib/coderunner/graphs_and_films.rb', line 408

def film_graphkit(axes, options, film_options = {})
# 		self.class.make_film_multiple_runners([[self,[[[axes, options]],[]]]], film_options)
	film_from_lists([[axes, options]], [], film_options)
end

#film_run_graphkit(name, options, film_options = {}) ⇒ Object 



413
414
415
416
 
# File 'lib/coderunner/graphs_and_films.rb', line 413

def film_run_graphkit(name, options, film_options = {})
# 		self.class.make_film_multiple_runners([[self,[[],[[name, options]]]]], film_options)
		film_from_lists([], [[name, options]], film_options)
end

#filter(run = @run, conditions = @conditions) ⇒ Object

Return true if run.instance_eval(conditions) == true and false if run.instance_eval(conditions) == false. performing some checks on the validity of the variable conditions. For people who are new to Ruby instance_eval means ‘evaluate in the context of the run’. Generally conditions will be something like ‘status == :Complete and height == 4.5 and not width == 20’, where height and width might be some input parameters or results from the diminishing.

Raises:



740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
 
# File 'lib/coderunner/instance_methods.rb', line 740

def filter(run=@run, conditions=@conditions)
	logf(:filter)
	@run = run
# 		to_instance_variables(directory)
	return true unless conditions 
#		p conditions, @run.id, @run.is_phantom
	conditions = conditions.dup	
	raise CRFatal.new("
-----------------------------
Conditions contain a single = sign: #{conditions}
-----------------------------") if conditions =~ /[^!=<>]=[^=~<>]/
	log conditions
	begin
		fil = @run.instance_eval(conditions)
	rescue => err
		eputs run.directory
		eputs conditions
		raise err
	end
	return fil
end

#filtered_ids(conditions = @conditions) ⇒ Object

Return a list of ids, filtered according to conditions. See CodeRunner#filter



727
728
729
730
731
732
 
# File 'lib/coderunner/instance_methods.rb', line 727

def filtered_ids(conditions=@conditions)
	generate_combined_ids
# 		ep @combined_run_list.keys
# 		sort_runs
	return @combined_ids.find_all{|id| filter(@combined_run_list[id], conditions)}
end

#generate_combined_ids(kind = nil) ⇒ Object 



1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
 
# File 'lib/coderunner/instance_methods.rb', line 1611

def generate_combined_ids(kind= nil)
	logf(:generate_combined_ids)
# 		case purpose
# 		when :print_out
# 			@combined_ids = []
# 			@combined_ids += @phantom_ids if @run_class.print_out_phantom_run_list
# 			@combined_ids += @ids if @run_class.print_out_real_run_list
# 		when :readout
# 			@combined_ids = []
# 			@combined_ids += @phantom_ids if @run_class.readout_phantom_run_list
# 			@combined_ids += @ids if @run_class.readout_real_run_list
# 		when :submitting
# 			@combined_ids = @ids
	kind ||= @use_phantom
	case kind
	when :real
		@combined_ids = @ids
	when :phantom
		@combined_ids = @phantom_ids
	when :both
		@combined_ids = @ids + @phantom_ids
	else
		raise CRFatal.new("Bad use phantom variable: #{kind.inspect}")
	end
	log('@phantom_run_list.class', @phantom_run_list.class)
	#puts 'crlist', @phantom_run_list.keys, @run_list.keys
	@combined_run_list = @phantom_run_list + @run_list
	log(:kind, kind)
# 		log(:combined_ids, @combined_ids)
	sort_runs(:combined)
end

#get_all_root_folder_contents ⇒ Object

List every file in the root folder.



806
807
808
809
 
# File 'lib/coderunner/instance_methods.rb', line 806

def get_all_root_folder_contents # :nodoc:
	@root_folder_contents =[]
	Find.find(@root_folder){|file| @root_folder_contents.push file}
end

#get_max(run, variable, sweep, complete = nil) ⇒ Object

def max(variable,sweep=nil, complete=nil) logf(:max) return get_max(variable,sweep, complete) == @run.id end

def get_max(variable,sweep=nil, complete=nil) logf :get_max sweep ||= variable logi @run.maxes @run.maxes ||= {} similar = similar_runs() similar = similar.find_all{|id| @combined_run_list.status == :Complete} if complete logi(:similar, similar, @combined_run_list[similar].send(variable)) @run.maxes[sweep] ||= similar ? similar.max{|id1,id2| @combined_run_list.send(variable) <=> @combined_run_list.send(variable)} : @run.id # puts “got_here” logi(“@run.maxes[#sweep]”, @run.maxes[sweep]) return @run.maxes[sweep] end



1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
 
# File 'lib/coderunner/instance_methods.rb', line 1488

def get_max(run, variable,sweep, complete=nil)
	logf :get_max
	generate_combined_ids
	sweep = [sweep] unless sweep.class == Array
	similar = similar_runs(sweep, run)  
	similar = similar.find_all{|id| @combined_run_list[id].status == :Complete} if complete
	logi(:similar, similar, @combined_run_list[similar[0]].send(variable))
	max = similar[1] ? similar.max{|id1,id2| @combined_run_list[id1].send(variable) <=> @combined_run_list[id2].send(variable)} : similar[0]
# 		puts "got_here"
	return max
end

#get_min(run, variable, sweep, complete = nil) ⇒ Object 



1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
 
# File 'lib/coderunner/instance_methods.rb', line 1500

def get_min(run, variable,sweep, complete=nil)
	logf :get_min
	generate_combined_ids
	sweep = [sweep] unless sweep.class == Array
	similar = similar_runs(sweep, run)  
	similar = similar.find_all{|id| @combined_run_list[id].status == :Complete} if complete
	logi(:similar, similar, @combined_run_list[similar[0]].send(variable))
	min = similar[1] ? similar.min{|id1,id2| @combined_run_list[id1].send(variable) <=> @combined_run_list[id2].send(variable)} : similar[0]
# 		puts "got_here"
	return min
end

#get_run_class ⇒ Object 



206
207
208
 
# File 'lib/coderunner/instance_methods.rb', line 206

def get_run_class
	@run_class = setup_run_class(@code, modlet: @modlet, version: @version, executable: @executable)
end

#get_run_class_name(code, modlet) ⇒ Object

See CodeRunner.get_run_class_name



330
331
332
 
# File 'lib/coderunner/instance_methods.rb', line 330

def get_run_class_name(code, modlet)
	self.class.get_run_class_name(code, modlet)
end

#gets ⇒ Object

No reading from the command line thank you very much!



39
40
41
 
# File 'lib/coderunner.rb', line 39

def gets #No reading from the command line thank you very much!
	$stdin.gets
end

#graphkit(*args) ⇒ Object

call-seq: graphkit(axes, options) Make a general graphkit, i.e. a graph that combines output from lots of runs. The axes should be an array of strings, each string defining what should be plotted on each axis. Each string is actually a fragment of Ruby code which will be evaluated by each run for whom option or @conditions evaulates to true. The most important other option is options. This is a string or an array of strings, each of which will be evaluated by each run; the data will then be grouped for the values of these strings. E.g. graphkit([‘a’, ‘b’], [“c”], conditions: ‘d==1’



254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
 
# File 'lib/coderunner/graphs_and_films.rb', line 254

def graphkit(*args)
	logf(:graphkit)
	if args.size == 1
		axes, options = graphkit_shorthand(args[0])
	elsif args.size == 2
		axes, options = args
	else
		raise ArgumentError
	end
# 		p axes
# 		return named_graphkit(axes, options) if axes.class == String or axes.class == Symbol
#  		ep @sort
	(return sweep_graphkits(options[:sweep], options) do |new_options|
		graphkit(axes, new_options)
	end) if options [:sweep]
	
	intial_conditions = @conditions
	initial_sort = @sort
	@conditions = (options[:conditions] or @conditions)
	@sort = (options[:sort] or @sort)

	sort_runs

	case axes.size
	when 1
		kit = GraphKit.autocreate({x: axiskit(axes[0])})
		kit.data[0].gp.with = "linespoints"
	when 2
		kit = GraphKit.autocreate({x: axiskit(axes[0]), y: axiskit(axes[1])})
		kit.data[0].gp.with = "linespoints"
			
	when 3
		kit = GraphKit.autocreate({x: axiskit(axes[0]), y: axiskit(axes[1]), z: axiskit(axes[2])})
		kit.data[0].gp.with = "linespoints"

	when 4
		kit = GraphKit.autocreate({x: axiskit(axes[0]), y: axiskit(axes[1]), z: axiskit(axes[2]), f: axiskit(axes[3])})
		kit.data[0].gp.with = "linespoints palette"
	else
		raise CRError.new("Bad graph string")
	end

	kit.title += %| for #{@conditions.gsub(/==/, "=").gsub(/\&\&/, " and ").gsub(/\|\|/, " or ").gsub(/\A(.{,40}).*\s/m, '\1')}| if @conditions
	kit.file_name = kit.title.gsub(/=/, ' eq ').gsub(/\&\&/, " and ").gsub(/\|\|/, " or ").gsub(/ /, "_").gsub(/\//, "over").gsub(/\*/, '.')
	kit.modify(options)
	
	
# 		p kit
	kit.data[0].gp.title = "#@code:#{kit.data[0].title}" 
	#Kernel.puts server_dump(kit) if @server 
	return kit




end

#graphkit_from_lists(graphs, run_graphs, extra_options = {}) ⇒ Object 



376
377
378
379
380
381
382
383
384
385
386
387
388
389
 
# File 'lib/coderunner/graphs_and_films.rb', line 376

def graphkit_from_lists(graphs, run_graphs, extra_options = {})
	run_kits = run_graphs.map do |gr|
		name, options = run_graphkit_shorthand(gr)
		options += extra_options
		run_graphkit(name, options.dup)
	 end
	 kits = graphs.map do |gr|
		axes, options = graphkit_shorthand(gr)
		options += extra_options			
		graphkit(axes, options.dup)
	 end

	(run_kits + kits).inject{|kit, k| kit+k}
end

#graphkit_from_lists_with_frame_array(frame_array, graphs, run_graphs, extra_options = {}) ⇒ Object 



390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
 
# File 'lib/coderunner/graphs_and_films.rb', line 390

def graphkit_from_lists_with_frame_array(frame_array, graphs, run_graphs, extra_options = {})
	server = @server; @server = false
# 		ep "Frame array to calculate is", frame_array, "\n\n"
	i=0
	array = frame_array.map do |frame_index|
# 			if print_message
			eputs "\033[2A" # Terminal jargon - go back one line
			eputs sprintf("Fetching graphs: %2.2f", i.to_f/frame_array.size.to_f * 100.0) + "% Complete"
# 			end
		i+=1
		[frame_index, graphkit_from_lists(graphs, run_graphs, extra_options.absorb({(extra_options[:in]||extra_options[:index_name]||:frame_index) => frame_index}))]
	end
# 		eputs Marshal.load(Marshal.dump(array)).pretty_inspect
	#Kernel.puts server_dump(array) if server
	return array
end

#graphkit_shorthand(*gr) ⇒ Object

Parse graphkit shorthand for a general graph and return it as [axes, options]. GraphKit shorthand is: ‘<axis1>[ : <axis2> [ : <axis3 [ : <axis4> ] ] ] [ ; <graph_options> [ ; <conditions> [ ; <sort> ] ] ]’ i.e. all commands for the graph in one string



13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
 
# File 'lib/coderunner/graphs_and_films.rb', line 13

def graphkit_shorthand(*gr)
	return gr if gr.size > 1
	gr = gr[0] 
	return gr if gr.class == Array
	axes, options, cons, srt = gr.split(/;/)
	axes = eval("['#{axes.split(/(?<!\\|:):(?!:)/).map{|s| s.gsub(/\\:/, ':')}.join("','")}']")
	options = (options and options =~ /\S/) ? eval(options): {}
# 		p srt
	cons = nil unless cons and cons=~ /\S/
	srt = nil unless srt and srt=~ /\S/

	options[:conditions] ||= cons
	options[:sort] ||= srt
	return [axes, options]
end

#increment_max_id ⇒ Object

Increase the value of @max_id by 1.



225
226
227
 
# File 'lib/coderunner/instance_methods.rb', line 225

def increment_max_id
	@max_id +=1
end

#job_identifier ⇒ Object 



1230
1231
1232
 
# File 'lib/coderunner/instance_methods.rb', line 1230

def job_identifier
	"#{@id_list[0]}-#{@id_list[-1]}"
end

#make_film_from_lists(graphs, run_graphs, film_options = {}) ⇒ Object 



418
419
420
 
# File 'lib/coderunner/graphs_and_films.rb', line 418

def make_film_from_lists(graphs, run_graphs, film_options = {})
	self.class.make_film_multiple_runners([[self,[graphs, run_graphs]]], film_options)
end

#marshalled_variables ⇒ Object

Dump all the instance variables of the runner to stdout as Marshalled binary data. This is used for RemoteCodeRunner server functions.



932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
 
# File 'lib/coderunner/instance_methods.rb', line 932

def marshalled_variables
	#ep 'marsh1'
	instance_vars = {}
	instance_variables.each do |var|
		instance_vars[var] = instance_variable_get(var)
	end
	instance_vars[:@run_list].values.each{|run| run.runner=nil}
	#Kernel.puts server_dump(instance_vars)
	instance_vars[:@cache]={}
	instance_vars[:@phantom_run_list].values.each{|run| run.runner = nil}
	#ep 'marsh2'
	#eputs instance_vars.pretty_inspect
	#instance_vars.each do |var, val|
		#ep var
		#eputs server_dump(val)
	#end


	instance_vars
end

#merge(*others) ⇒ Object 



3
4
5
 
# File 'lib/coderunner/merged_code_runner.rb', line 3

def merge(*others)
	Merged.new(*others.unshift(self))
end

#merge_method(meth, *args) ⇒ Object 



7
8
9
 
# File 'lib/coderunner/merged_code_runner.rb', line 7

def merge_method(meth, *args)
	send(meth, *args)
end

#new_run ⇒ Object

Create a new instance of the class @run_class



1288
1289
1290
1291
 
# File 'lib/coderunner/instance_methods.rb', line 1288

def new_run
	logf(:new_run)
	@run_class.new(self)
end

#p(*args) ⇒ Object

Here we redefine the inspect function p to raise an error if anything is written to standard out while in server mode. This is because the server mode uses standard out to communicate and writing to standard out can break it. All messages, debug information and so on, should always be written to standard error.



278
279
280
281
282
283
284
 
# File 'lib/coderunner/instance_methods.rb', line 278

def p(*args)
	if @server
		raise "Writing to stdout in server mode will break things!"
	else
		super(*args)
	end
end

#parameter_scan(parameter_scan, parameters, options = {}) ⇒ Object

Raises:



1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
 
# File 'lib/coderunner/instance_methods.rb', line 1396

def parameter_scan(parameter_scan, parameters, options={})
	skip = true unless options[:skip] == false
# 		version= (options[:version] or "")
	nprocs = options[:nprocs]
	logf(:parameter_scan)
	raise CRFatal.new("Wrong directory: parameter scan being conducted in #{Dir.pwd} instead of my root folder: #{@root_folder}") unless Dir.pwd == File.expand_path(@root_folder)
	@skip = skip
	puts parameter_scan.inspect
	
	@nprocs = nprocs; @skip=skip; 
	log '@run_class', @run_class
	@run = @run_class.new(self)
	@run.update_submission_parameters(parameters)
#  		@running_scans = {}; @gammas = {}
	beginning = "catch(:finished)  do \n"
	end_string = "\nend"
	parameter_scan.each do |variable_scan|
		beginning += %[ #{variable_scan[1].inspect}.each do |value|\n\t@run.#{variable_scan[0]} = value\n]
		@run_class.rcp.naming_pars.push variable_scan[0].to_sym; @run_class.rcp.naming_pars.uniq!
		end_string += %[\nend]
	end
	middle = <<EOF
@@psppipe.synchronize(:pssubmit){@running = submit(@run, nprocs: @nprocs, version: @version, skip: @skip); update};  
loop do
# 	@@mutex.synchronize{}
@run_class.update_status(self)
# 	puts run_class.current_status
Dir.chdir(@run_list[@running].directory){@run_list.delete(@running); traverse_directories}
# 	ep @run_list[@running].status, @run_list[@running].id, @run_list[@running].job_no, queue_status
unless @run_list[@running].status == :Incomplete or @run_list[@running].status  == :NotStarted
	
	@run.parameter_transition(@run_list[@running])
	break
end
sleep 3
# 	Thread.pass
#  	puts Thread.current.to_s + " is looping, @run_list[@running].status = " + @run_list[@running].status.to_s + " @running = " + @running.to_s + " @run_list[@running].job_no = " + @run_list[@running].job_no.to_s  
end	
EOF
	command = beginning + middle + end_string
	puts command
	instance_eval(command, 'parameter_scan_code')
# 				puts Thread.current.object_id
# 				puts Thread.current.to_s; print " is finished"
# 		@@psppipe.i_send(:finished, true, tp: 0)

end

#print(*args) ⇒ Object

Here we redefine the function print to raise an error if anything is written to standard out while in server mode. This is because the server mode uses standard out to communicate and writing to standard out can break it. All messages, debug information and so on, should always be written to standard error.



304
305
306
307
308
309
310
 
# File 'lib/coderunner/instance_methods.rb', line 304

def print(*args)
	if @server
		raise "Writing to stdout in server mode will break things!"
	else
		super(*args)
	end
end

#print_out(rewind = nil, options = {}) ⇒ Object

Print out a summary of all the runs in the root folder, formatted in nice pretty colours. Since this is often called in a loop, if called twice without any arguments it will erase the first printout. To stop this happening set rewind to 0. If the command is being issued not in a terminal, so that CodeRunner cannot determine the size of the terminal, the second argument must be passed as an array of [rows, columns].



640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
 
# File 'lib/coderunner/instance_methods.rb', line 640

def print_out(rewind = nil, options={}) # terminal_size = [rows, cols]
	rewind ||= @print_out_size
	terminal_size = options[:terminal_size]
	logf(:print_out)
# 		raise CRFatal.new("terminal size must be given if this is called any where except inside a terminal (for example if yo	u've called this as a subprocess)") unless terminal_size or $stdout.tty?
	terminal_size ||= Terminal.terminal_size
	#lots of gritty terminal jargon in here. Edit at your peril!
	
	unless ENV['CODE_RUNNER_NO_COLOUR']=='true' or ENV['CODE_RUNNER_NO_COLOR']=='true'
		dc= Terminal::WHITE #  .default_colour
		green= Terminal::LIGHT_GREEN
		cyan = Terminal::CYAN
		bblck = Terminal::BACKGROUND_BLACK
	else
		dc= ""# Terminal::WHITE #  .default_colour
		green= "" # Terminal::LIGHT_GREEN
		cyan = "" #Terminal::CYAN
		bblck = "" #Terminal::BACKGROUND_BLACK
	end
	cln = Terminal::CLEAR_LINE
# 		print "\033[#{rewind}A"
	deco = '-'*terminal_size[1]
	
	Terminal.erewind(rewind)
	
	eputs "\n#{cln}\n#{cln}\n#{bblck}#{dc}#{deco}\n#{@run_class.rcp.code_long} Status:#{cln}\n#{deco}"

	i = 0; j=0 # i is no. of actual terminal lines; j is number of results lines
	
	# Group the lines by major sort key
	#@split = @sort && @sort.split(/\s+/).size > 1 ? @sort.split(/\s+/)[0].to_sym : nil 
	@split_point = nil
	
	generate_combined_ids
	@combined_ids.each do |id|
		begin
# 			puts id, :hello; gets
			@run = @combined_run_list[id]
			if filter
# 				puts @run[:id], @id; gets
				#@new_split_point = @split ? @run.send(@split) : nil
				#if @split_point && @split_point != @new_split_point then eputs sprintf(" #{cln}", ""); i+=1 end
				#@split_point = @new_split_point
				eprint j%2==0 ? j%4==0 ? cyan : green : dc
				line = options[:with_comments] ? @run.comment_line : @run.print_out_line.chomp  
				eprint line
				eputs cln
# 				puts (line.size / Terminal.terminal_size[1]).class
# 				puts (line.size / Terminal.terminal_size[1])

				i+=((line.sub(/\w*$/, '').size-1)  / terminal_size[1]).ceiling
				j+=1
			end
# 				raise "monkeys"
		rescue => err
			eputs err
			eputs err.backtrace
			eputs "---------------------\nUnable to print out line for this job:"
			eputs "run_name: #{@run.run_name}"
			eputs "status: #{@run.status}\n-----------------------"
			Terminal.reset
			return
# 				raise CRFatal.new 
		end
	end
	@print_out_size = i+7# + (@run_list.keys.find{|id| not [:Complete, :Failed].include? @run_list[id].status } ? 0 : 1) 
	eprint dc, deco; Terminal.reset; eputs
# 		 puts
# 		puts
# 		puts dc # "\033[1;37m"
# 		print 'rewind size is', rewind

end

#puts(*args) ⇒ Object

Here we redefine the function puts to raise an error if anything is written to standard out while in server mode. This is because the server mode uses standard out to communicate and writing to standard out can break it. All messages, debug information and so on, should always be written to standard error.



291
292
293
294
295
296
297
 
# File 'lib/coderunner/instance_methods.rb', line 291

def puts(*args)
	if @server
		raise "Writing to stdout in server mode will break things!"
	else
		super(*args)
	end
end

#rcp ⇒ Object

def submit



1205
1206
1207
 
# File 'lib/coderunner/instance_methods.rb', line 1205

def rcp
	@run_class.rcp
end

#read_defaults ⇒ Object 



215
216
217
218
219
220
221
 
# File 'lib/coderunner/instance_methods.rb', line 215

def read_defaults
	DEFAULT_RUNNER_OPTIONS.each{|key,value| set(key, value)}
				#ep DEFAULT_RUNNER_OPTIONS, @multiple_processes

	read_folder_defaults
	get_run_class
end

#read_folder_defaults ⇒ Object

Read any default options contained in the file .code_runner_script_defaults.rb in the root folder.



247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
 
# File 'lib/coderunner/instance_methods.rb', line 247

def read_folder_defaults
# 		p @root_folder + '/.code_runner_script_defaults.rb'
	if ENV['CODE_RUNNER_READONLY_DEFAULTS']
		hash = eval(File.read(@root_folder + '/.code_runner_script_defaults.rb'))
		FOLDER_DEFAULTS.each do |var|
			set(var, hash[var]) if hash[var]
		end
	else

		Hash.phoenix(@root_folder + '/.code_runner_script_defaults.rb') do |hash|
# 			ep hash
			FOLDER_DEFAULTS.each do |var|
# 				p send(var), hash[var]
				hash[var] = (send(var) or hash[var])
				hash[:code_runner_version] ||= CodeRunner::CODE_RUNNER_VERSION.to_s
				set(var, hash[var])
			end
			@start_id = hash[:start_id] if hash[:start_id]
# 			ep "start_id: #@start_id"
			hash
		end
	end
	
	raise "No default information exists for this folder. If you are running CodeRunner from the commmand line please run again with the -C <code> and -X <executable> (and -m <modlet>, if required) flag (you only need to specify these flags once in each folder). Else, please specify :code and :executable (and :modlet if required) as options in CodeRunner.new(folder, options)" unless @code and @executable
end

#readout ⇒ Object

Similar to CodeRunner#write_data, except that the readout is written to stdout, and formatted a bit. Will probably be rarely used.



764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
 
# File 'lib/coderunner/instance_methods.rb', line 764

def readout
	logf(:readout)
	generate_combined_ids
	@split = @sort && @sort.split(/\s+/).size > 1 ? @sort.split(/\s+/)[0].to_sym : nil
	@split_point = nil
	string = @combined_ids.inject("") do |s, id|
		run = @combined_run_list[id]
		if run.status =~ /Complete/ and filter(run)  
			@new_split_point = @split ? run.send(@split) : nil
			splitter = (@split_point and @split_point != @new_split_point) ? "\n\n" : ""
			@split_point = @new_split_point
			splitter = "" # comment to put split points back in
# 				puts s.class, splitter.class, data_string(directory)
			data_string = run.data_string
			raise CRFatal.new("data_string did not return a string") unless data_string.class == String
			s + splitter + data_string 
		else 
			s 
		end
	end
# 		puts @max.inspect

	return string
end

#readout_cols(*var_list) ⇒ Object

? Probably can get rid of this one



791
792
793
794
795
796
797
798
799
800
801
802
 
# File 'lib/coderunner/instance_methods.rb', line 791

def readout_cols(*var_list) # :nodoc:
	logf :readout_cols
	ans = [[]] * var_list.size
	generate_combined_ids
	filtered_ids.each do |id|
		run = combined_run_list[id]
		var_list.each_with_index do |var, index|
			ans[index].push run.send(var)
		end
	end
	ans
end

#recheck_filtered_runs(write_status_dots = false) ⇒ Object

Self-explanatory! Call CodeRunner::Run#process_directory for every run for which CodeRunner#filter(run) is true. (Note, for historical reasons traverse_directories is called rather than CodeRunner::Run#process_directory directly but the effect is nearly the same).



995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
 
# File 'lib/coderunner/instance_methods.rb', line 995

def recheck_filtered_runs(write_status_dots=false)
	@write_status_dots = write_status_dots
	logf :recheck_filtered_runs
	@run_class.update_status(self)
	@run_list.each do |id, run|
		if filter(run) and not (run.status == :Complete or run.status == :Failed)
# 				eputs run.directory
			Dir.chdir(run.directory){run.process_directory}
# 				eputs run.status
# 				ep run
		end
	end
	save_large_cache
# 		run_list, @run_list = @run_list, {}; @ids = []; @max_id = 0
# 		run_list.each do |id, run|
# 			if  not filter(run) or run.status == :Complete
# 				@run_list[id] = run
# 				@ids.push id
# 			else
# 				Dir.chdir(run.directory){traverse_directories}
# 			end
# 		end
# 		@ids.sort!
# 		@ids = @run_list.keys
	set_max_id(@ids.max || 0)
	sort_runs
	respond_to_requests
	save_large_cache
end

#recheck_incomplete_runs ⇒ Object

Self-explanatory! Call CodeRunner::Run#process_directory for every run whose status is not either :Complete or :Failed. (Note, for historical reasons traverse_directories is called rather than CodeRunner::Run#process_directory directly but the effect is nearly the same).



968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
 
# File 'lib/coderunner/instance_methods.rb', line 968

def recheck_incomplete_runs
	logf :recheck_incomplete_runs
	@run_class.update_status(self)
	redone_count = 0
	run_list, @run_list = @run_list, {}; @ids = [];
	run_list.each do |id, run|
# 						print id
		if run.status == :Complete or run.status == :Failed
# 							print ".", id, "___"
			@run_list[id] = run
			@ids.push id
		else
# 							print id, "%%%"; 
			redone_count+=1
			Dir.chdir(run.directory){traverse_directories}
		end
	end
	@ids.sort!
# 		@ids = @run_list.keys
	set_max_id(@ids.max || 0)
	sort_runs
	respond_to_requests
	save_large_cache
end

#respond_to_requests ⇒ Object

One of the features of CodeRunner is two way communication between a runner and its runs. The runs can request actions from the runner directly by calling its instance methods, but often the runs want something to happen after the runner has processed every run in the directory. For example if a run wanted to check if it was resolved, it would need to know about all the other runs so it could compare itself with them. In this case it would place an instance method that it wanted to call in the variable @requests in the runner. The runner would then call that instance method on every run after it had finished processing all the runs.

In summary, this method is called after every time the runner has checked through the directory. When it is called, it looks in the variable @requests which contains symbols representing methods. It calls each symbol as an instance method of every run in turn. So if @requests was [:check_converged] it would call run.check_converged for every run.



1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
 
# File 'lib/coderunner/instance_methods.rb', line 1029

def respond_to_requests
	logf(:respond_to_requests)
	logi(:@requests, @requests)
	log('@phantom_run_list.class', @phantom_run_list.class)
	while @requests[0]
		old_requests = @requests.uniq
		@requests = []
		old_requests.each do |request|
			@current_request = request
			if request == :traverse_directories # a special case
				@ids = []
				@run_list = {}
				@phantom_run_list = {}
				@phantom_ids = []

				Dir.chdir(@root_folder){traverse_directories}
			else
				filtered_ids.each do |id|
					run = @run_list[id]
					Dir.chdir(run.directory){run.instance_eval(request.to_s); run.save; run.write_results}
				end
			end
		end
	end
end

#run_graphkit(*args) ⇒ Object

Raises:



345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
 
# File 'lib/coderunner/graphs_and_films.rb', line 345

def run_graphkit(*args)
	if args.size == 1
		name, options = run_graphkit_shorthand(args[0])
	elsif args.size == 2
		name, options = args
	else
		raise ArgumentError
	end
	
	(return sweep_graphkits(options[:sweep], options) do |new_options|
		run_graphkit(name, new_options)
	end) if options [:sweep]
	
	old_sort, old_conditions = @sort, @conditions
	@sort = options[:sort] if options[:sort]
	@conditions = options[:conditions] if options[:conditions]		
	generate_combined_ids
	fids = filtered_ids
	raise CRError.new("No ids match these conditions: #@conditions") unless fids.size > 0
	kit = (fids.map do |id| 
		run = @combined_run_list[id]; 
# 			p run; STDIN.gets
		kit = run.graphkit(name, options.dup); 
		kit.data[0].title ||= run.run_name; 
		kit
	end).inject{|kit, k| kit+k}
	@sort, @conditions = old_sort, old_conditions
	#Kernel.puts server_dump(kit) if @server
	kit
end

#run_graphkit_shorthand(*grs) ⇒ Object 



329
330
331
332
333
334
335
336
337
338
339
340
341
342
 
# File 'lib/coderunner/graphs_and_films.rb', line 329

def run_graphkit_shorthand(*grs)
# 		p grs
	return grs if grs.size > 1
	gr = grs[0] 
# 		p gr
	return gr if gr.class == Array
	name, options, cons, srt = gr.split(/;/)
	options = (options and options =~ /\S/) ? eval(options): {}
	cons = nil unless cons and cons=~ /\S/
	srt = nil unless srt and srt=~ /\S/
	options[:conditions] ||= cons if cons
	options[:sort] ||= srt if srt
	[name, options]
end

#runs ⇒ Object

Return an array of runs (run_list.values)



231
232
233
 
# File 'lib/coderunner/instance_methods.rb', line 231

def runs
	run_list.values
end

#save_all ⇒ Object 



1643
1644
1645
1646
1647
1648
1649
 
# File 'lib/coderunner/instance_methods.rb', line 1643

def save_all
	save_large_cache
	@run_list.values.each do |run|
		run.save
		run.write_results
	end
end

#save_large_cache ⇒ Object

Write the variable @run_list, which contains all information currently known about the simulations in the root folder, as Marshalled binary data in the file “.CODE_RUNNER_TEMP_RUN_LIST_CACHE”. This cache will be used later by CodeRunner#update.



955
956
957
958
959
960
961
962
963
964
 
# File 'lib/coderunner/instance_methods.rb', line 955

def save_large_cache
# 		pp self
# 		ep @run_list
# 		pp @run_list.values.map{|run| run.instance_variables.map{|var| [var, run.instance_variable_get(var).class]}}
	
	generate_combined_ids
	@combined_run_list.each{|id, run| run.runner = nil; run.phantom_runs.each{|pr| pr.runner=nil} if run.phantom_runs}
	File.open(@root_folder + "/.CODE_RUNNER_TEMP_RUN_LIST_CACHE", 'w'){|file| file.puts Marshal.dump @run_list}
	@combined_run_list.each{|id, run| run.runner = self; run.phantom_runs.each{|pr| pr.runner=self} if run.phantom_runs}
end

#server_dump(obj) ⇒ Object 



316
317
318
 
# File 'lib/coderunner/instance_methods.rb', line 316

def server_dump(obj)
	self.class.server_dump(obj)
end

#set_start_id(id) ⇒ Object 



320
321
322
323
324
325
326
 
# File 'lib/coderunner/instance_methods.rb', line 320

def set_start_id(id)
	raise "start_id #{id} lower than max_id #@max_id" if @max_id and id < @max_id 
	Hash.phoenix(@root_folder + '/.code_runner_script_defaults.rb') do |hash|
		@start_id = hash[:start_id] = id
		hash
	end
end

#setup_run_class(code, options) ⇒ Object

See CodeRunner.setup_run_class



336
337
338
339
 
# File 'lib/coderunner/instance_methods.rb', line 336

def setup_run_class(code, options)
	options[:runner] ||= self
	self.class.setup_run_class(code, options)
end

#similar_runs(exclude_variables = [], run = @run) ⇒ Object

Find runs whose input parameters are all the same as those for run, with the exception of those listed in exclude_variables

Raises:



1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
 
# File 'lib/coderunner/instance_methods.rb', line 1446

def similar_runs(exclude_variables=[], run=@run) #all runs for which variables are the same as 'run', with the exception of exclude_variables
	logf(:similar_runs)
	raise CRFatal.new("generate_combined_ids must be called before this function is called") unless (@combined_run_list.size > 0 and @combined_ids.size > 0) or @ids.size ==0
	command = (run.class.rcp.variables+run.class.rcp.run_info-exclude_variables  - [:output_file, :error_file, :runner, :phantom_runs]).inject("@combined_ids.find_all{|id| @combined_run_list[id].class == run.class}"){ |s,v|	
		s + %<.find_all{|id| @combined_run_list[id].#{v}.class == #{run.send(v).inspect}.class and @combined_run_list[id].#{v} == #{run.send(v).inspect}}>} #the second send call retrieves the type conversion

#  		log command
 		#puts command
	begin 
		similar = instance_eval(command)
	rescue => err
		log command
		raise err
	end
	return similar
end

#simple_scan(scan_string, options = {}) ⇒ Object

Submit a series of runs according to scan_string. scan_string specifies a number of scans separated by hashes. Each scan has a number which is the ID of the run to start from followed by a colon. After the colon the user can write some code which modifies the original run. For example:

simple_scan(‘23: @height *= 2; @weight *=1.1’)

will submit a series of runs, where each successive run has height twice the last one, and weight 1.1 times the last one, where we assume that height and weight are input parameters.

Options are the same as CodeRunner#submit



1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
 
# File 'lib/coderunner/instance_methods.rb', line 1303

def simple_scan(scan_string, options={})
	scans = scan_string.split('#')
	ppipe = PPipe.new(scans.size + 1, true, controller_refresh: 0.5, redirect: false)
	pipe_numbers = ppipe.fork(scans.size - 1) 

	#p @run_class.naming_pars
	instructions = (scans[(ppipe.mpn > 0 ? ppipe.mpn - 1 : 0)])
	id = instructions.scan(/^\d+\:/)[0].to_i
	instructions = instructions.sub(/^\d+\:/, '')
	@run= id > 0 ?  @run_list[id] : (run  = @run_class.new(self); run.update_submission_parameters((options[:parameters] or '{}')); run)
        @run_class.rcp.variables.each do |par|
	      #p par, @run_class.naming_pars
	      @run.naming_pars.push par if scan_string =~ Regexp.new(Regexp.escape(par.to_s))
	end
	@run.naming_pars.uniq!
# 		@run.naming_pars +== @run_class.naming_pars
	catch(:finished) do
	loop do #submit loop
		ppipe.synchronize(:submit) do 
		          @running = submit(@run, nprocs: options[:nprocs], version: @run_class.rcp.version)
                                  @conditions = "id == #@running"
			   #print_out
																 #ep 'Running run', @running
		end
		loop do # check finished loop
			dir = @run_list[@running].directory
			@run_list.delete(@running)
			@run_class.update_status(self)
			Dir.chdir(dir) do
				traverse_directories
			end
			unless @@wait and (@run_list[@running].status == :Incomplete or @run_list[@running].status  == :NotStarted or @run_list[@running].status == :Queueing)
				@run.parameter_transition(@run_list[@running])
				@old_run = @run_list[@running]
				break
			end
			#p @running
			ppipe.i_send(:running, @running, tp: 0)
			if ppipe.is_root
				arr = (pipe_numbers + [0]).inject([]) do |arr, pn|
					arr.push ppipe.w_recv(:running, fp: pn)
				end
				#p arr
				@conditions = "#{arr.inspect}.include? id"
				print_out
			end
			sleep 3
		end
		@run.instance_eval(instructions)
	end
	end
	(ppipe.die; exit) unless ppipe.is_root
	ppipe.finish
end

#sort_runs(type = @use_phantom.to_s.sub(/real/, '')) ⇒ Object

Sort the runs according to the variable @sort which can be either whitespace separated string or an array of strings. In the former case, the string is split to form an array of strings, using the separator /\s+/. For example, if

Then the runs will be sorted according to first height, then (descending) weight, then colour. What actually happens is that the variable @ids is sorted, rather than @run_list. For this to work, height, weight and colour must be either variables or results or instance methods of the run class.

Type can be either ” or ‘phantom’, in which case the variable sorted will be either @ids or @phantom_ids respectively.



620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
 
# File 'lib/coderunner/instance_methods.rb', line 620

def sort_runs(type = @use_phantom.to_s.sub(/real/, ''))
	logf(:sort_runs)
	log(:type, type)
	#ep 'sort', @sort
	#sort_list = @sort ? (@sort.class == String ? eval(@sort) : @sort) : []
	run_list_name = [type.to_s, 'run_list'].join('_').gsub(/^_/, '')
	ids_name = [type.to_s, 'ids'].join('_').gsub(/^_/, '')
	log run_list_name 
		set(ids_name, send(ids_name).sort_by do |id|
			run = send(run_list_name)[id]
			sortkey = run.instance_eval((@sort or '[]'))
			#ep 'sortkey', sortkey
			sortkey

			#sort_list.map{|str| run.instance_eval(str)}
		end)
end

#submit(runs, options = {}) ⇒ Object

Start a simulation: submit the run that is passed. What happens is as follows:

  1. Modify the run according to options.

  2. Check if any other processes are submitting runs in the same root folder. In this case there will be a file called ‘submitting’ in the folder. If there is such a file wait until it is gone.

  3. Check if a run with identical parameters has been submitted before. In which case skip submitting the run unless options == false.

  4. Call run.submit

Options can be any one of CodeRunner::SUBMIT_OPTIONS. The options passed here will override values stored as instance variables of the runner with the same name, which will override these values if they are set in the runs itself. For example if run.nprocs == ‘32x4’ runner.nprocs == nil options == nil the number of processes will be 32x4. On the other hand if run.nprocs == ‘32x4’ runner.nprocs == ‘24x4’ options == ‘48x4’ the number of processes will be 48x4.



1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
 
# File 'lib/coderunner/instance_methods.rb', line 1072

def submit(runs, options={})
	eputs "System " + SYS 
	eputs "No. Procs " + @nprocs.inspect

	runs = [runs] unless runs.class == Array #can pass a single run to submit
		outruns = runs.dup
	skip = true unless options[:skip] == false
	SUBMIT_OPTIONS.each do |option|
		set(option, options[option]) if options.keys.include? option
	end
	logf(:submit)
	Dir.chdir(@root_folder) do
		@skip=skip
		mess = false
		while FileTest.exist?("submitting") 
			(eputs " Waiting for another process to finish submitting. If you know that no other CodeRunner processes are submitting in this folder (#@root_folder) then delete the file 'submitting' and try again"; mess = true) unless mess
			sleep rand
		end
# 			old_trap = trap(0)
		#old_trap0 = trap(0){eputs "Aborted Submit (0)!"; File.delete("#@root_folder/submitting"); exit!}
		old_trap2 = trap(2){eputs "Aborted Submit (2)!"; File.delete("#@root_folder/submitting") if FileTest.exist? "#@root_folder/submitting"; trap(2, "DEFAULT"); trap(0, "DEFAULT"); Process.kill(2, 0)}
# 		File.open("submitting", "w"){|file| file.puts ""}
		FileUtils.touch("submitting")
		unless options[:no_update_before_submit]
			@use_large_cache, ulc = false, @use_large_cache; update; @use_large_cache = ulc
		end
		generate_combined_ids(:real)
# 			old_job_nos = queue_status.scan(/^\s*(\d+)/).map{|match| match[0].to_i}
		script = "" if options[:job_chain]
		runs.each_with_index do |run, index|
			similar = similar_runs([], run)
			if @skip and similar[0] and not (options[:replace_existing] or options[:rerun])
				eputs "Found similar run: #{@run_list[similar[0]].run_name}"
				eputs "Skipping submission..."
				runs[index] = nil
				next
			end
			unless options[:replace_existing] or options[:rerun]
				@max_id+=1
				run.id = @max_id
			else
				if options[:replace_existing]
					FileUtils.rm_r run.directory 
			  elsif options[:rerun]
					################# For backwards compatibility
					SUBMIT_OPTIONS.each do |opt|
						 run.set(opt, send(opt)) unless run.send(opt)	
					end
					###########################################
					FileUtils.rm "#{run.directory}/code_runner_results.rb"
					FileUtils.rm "#{run.directory}/.code_runner_run_data"
				end
				@run_list.delete(run.id)
				@ids.delete run.id
				generate_combined_ids
			end

			begin
				
				unless options[:job_chain]
					run.prepare_submission unless options[:rerun]
					next if @test_submission
					Dir.chdir(run.directory) do 
						old_job_nos = queue_status.scan(/^\s*(\d+)/).map{|match| match[0].to_i}
						######################### The big tomale!
						run.job_no = run.execute  # Start the simulation and get the job_number
						#########################
						run.job_no = get_new_job_no(old_job_nos) unless run.job_no.kind_of? Integer # (if the execute command does not return the job number, look for it manually) 
# 							eputs 'run.job_no', run.job_no
						run.output_file = nil # reset the output file
						run.output_file # Sets the output_file on first call
						run.error_file = nil # reset the output file
						run.error_file # Sets the error_file on first call
						run.write_info
						eputs "Submitted run: #{run.run_name}"
					end
				else
					run.prepare_submission unless options[:rerun]
					script << "cd #{run.directory}\n"
					script << "#{run.run_command}\n"
					next if @test_submission
				end
			rescue => err
				File.delete("submitting")
				raise(err)
			end
		end # runs.each
		runs.compact!
		if options[:job_chain] and not @test_submission and runs.size > 0
			FileUtils.makedirs('job_chain_files')
			@id_list = runs.map{|run| run.id}
			
			#@executable ||= runs[0].executable
			@submission_script = script
			# A hook... default is to do nothing
			@submission_script = @run_class.modify_job_script(self, runs, @submission_script)
			# To get out of job_chain_files folder
			@submission_script = "cd .. \n" + @submission_script
			old_job_nos = queue_status.scan(/^\s*(\d+)/).map{|match| match[0].to_i}
			################ Submit the run
			Dir.chdir('job_chain_files'){job_no = execute}
			################
			job_no = get_new_job_no(old_job_nos) unless job_no.kind_of? Integer 	# (if the execute command does not return the job number, look for it manually) 
# 				eputs 'jobchain no', job_no
			#runs.each{|run| run.job_no = job_no}
			runs.each do |run| 
				run.job_no = @job_no = job_no
				run.output_file = run.relative_directory.split("/").map{|d| ".."}.join("/") + "/job_chain_files/" + output_file
				run.error_file = run.relative_directory.split("/").map{|d| ".."}.join("/") + "/job_chain_files/" + error_file
				run.write_info
				eputs "Submitted run: #{run.run_name}"
			end
		end
		@write_status_dots, wsd = false, @write_status_dots
		@run_class.update_status(self)
		runs.each do |run| 
# 				ep run.id, run_list.keys		
			Dir.chdir(run.directory){traverse_directories}
		end
		@write_status_dots = wsd	
		save_large_cache
		File.delete("submitting")
		#trap(0, old_trap0)
		trap(2, old_trap2)

		
	end # Dir.chdir(@root_folder)
#     eputs
    #ep 'runs submitted', outruns
	return outruns[0].id if outruns.size == 1 #used in parameter scans 

end

#sweep_graphkits(sweeps, options, &block) ⇒ Object 



29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
 
# File 'lib/coderunner/graphs_and_films.rb', line 29

def sweep_graphkits(sweeps, options, &block)
	server = @server; @server = false
	intial_conditions = @conditions
	initial_sort = @sort
	@conditions = (options[:conditions] or @conditions)
	@sort = (options[:sort] or @sort)

	sweeps = sweeps.class == Array ? sweeps : [sweeps]
	generate_combined_ids
# 		ep sweeps
	sweep_values = filtered_ids.inject([]) do |arr, id|
		arr.push(sweeps.inject([]) do |sweep_arr, var|
			
			sweep_arr.push @combined_run_list[id].send(var)
# 				  ep sweep_arr, var
			sweep_arr
		end)
		arr.uniq
	end
	old_conditions = @conditions
	new_options = options.dup
	new_options.delete(:sweep) if new_options[:sweep]
# 			p new_options; gets
	kits = sweep_values.sort.map do |values|
		new_cons = (sweeps.zip(values).map do |name, value|
			"#{name}==#{value}"
		end).join(" and ")
		new_options[:conditions] = old_conditions ? old_conditions + " and #{new_cons}" : new_cons
# 				ep new_options[:conditions]
		kit = yield(new_options)
		
			
		kit.data[0].gp.title = new_cons.gsub(/==/, '=')
# 				p kit
		kit
	end
# 		@conditions = old_conditions
	@conditions = intial_conditions 
	@sort = initial_sort 
	if options[:sweep_multid]
		kitdata = kits.map do |kit| 
			kit.data[0].axes.values.map{|axiskit| axiskit.data}
		end
		final_axes = (kitdata[0].size  + sweeps.size - 1).times.inject([]){|arr, i| arr.push []}
		data_hash = {}
		sweep_values.sort.each_with_index do |vals, i| 
# 				ep 'v', vals
			data = kitdata[i]
			length = data[0].size
			vals.each_with_index do |val, j|
				final_axes[j].push val
# 					final_axes[j].uniq!
			end
			function = data.pop
			for j in 0...length
				
				data.each_with_index do |axisdata, k|
# 						ep 'vk', vals.size + k
# 						break if k == data.size - 1
					final_axes[vals.size + k].push axisdata[j]
					final_axes[vals.size + k].uniq!
				end
# 					ep 'h', final_axes, function
				data_hash[vals + data.map{|axisdata| axisdata[j]}] = function[j]
			end
		end
		final_axes.map!{|vals| vals.uniq.sort}
		final_data = SparseTensor.new(final_axes.size)
		data_hash.each do |coordinates, function|
			indices = []
			coordinates.each_with_index do |val, i| 
				indices[i] = final_axes[i].index(val)
			end
			final_data[*indices] = function
		end
		hash = {}
		
# 			pp final_axes, final_data
		
		titles = sweeps + kits[0].data[0].axes.values.map{|axiskit| axiskit[:title]}
		(final_axes + [final_data]).each_with_index do |d, i|
			hash[GraphKit::AXES[i]] = {data: d, title: titles[i].to_s}
		end
		kit = GraphKit.autocreate(hash)
		
					
					
# 				length = data[i].size
# 				array = vals.map{|val| [val] * length} + data
# 				p array
# 			end
# 			final_data = [[]] * (kitdata[0].size  + sweeps.size)
# 			sweep_values.sort.each_with_index do |vals, i|
# 				data = kitdata[0]
# 				length = data[0].size
# 				array = vals.map{|val| [val] * length} + data
# 				p array
# 			end
	else
		kit =  kits.inject{|ans, kit| ans + kit} 
	end
			
	
	#Kernel.puts server_dump(kit) if server 

	return kit

end

#update(write_status_dots = true, use_large_cache = @use_large_cache, use_large_cache_but_recheck_incomplete = @use_large_cache_but_recheck_incomplete) ⇒ Object

Update the information about all the runs stored in the variable @run_list. By default, this is done by calling CodeRunner#traverse_directories. If, on the other hand, @use_large_cache is set to true, this is done by reading the temporary cache maintained in “.CODE_RUNNER_TEMP_RUN_LIST_CACHE” in the root folder. This is much quicker. If in addition @use_large_cache_but_recheck_incomplete is set to true, all runs whose status is not either :Complete or :Failed will be rechecked.



813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
 
# File 'lib/coderunner/instance_methods.rb', line 813

def update(write_status_dots=true, use_large_cache=@use_large_cache, use_large_cache_but_recheck_incomplete=@use_large_cache_but_recheck_incomplete)
	@use_large_cache = use_large_cache
	logf(:update)
	@write_status_dots = write_status_dots

	@run_list={}
	@ids=[]
	@phantom_run_list = {}
	@phantom_ids = []
	@run_store =[]
# 		@multiple_processes_directories = []
	set_max_id 0
	@run_class.update_status(self)
	if @use_large_cache and not @recalc_all and not @reprocess_all
		log("using large cache")
		begin
			begin
				eputs 'Loading large cache...' if @write_status_dots
				Dir.chdir(@root_folder) do 
					@run_list = Marshal.load(File.read(".CODE_RUNNER_TEMP_RUN_LIST_CACHE"))
				end
				@run_list.values.each{|run| run.runner = self}
			rescue ArgumentError => err
				eputs err
				raise err unless err.message =~ /undefined class/
				#NB all code_names have to contain only lowercase letters:
# 					modlet, code = err.message.scan(/CodeRunner\:\:([A-Z][\w+])?([A-Z]\w+)Run/)[0]
				code, modlet = err.message.scan(/CodeRunner\:\:([A-Z][a-z0-9_]+)(?:::([A-Z]\w+))?/)[0]
# 					ep code.downcase, modlet
				modlet.downcase! if modlet
				setup_run_class(code.downcase, modlet: modlet)
				retry
			end
# 				ep @run_list
			@ids = @run_list.keys
				@run_list.each{|id, run| run.runner = self }
			#eputs "Setting max id..."
			set_max_id(@ids.max || 0)
			#eputs "max_id = #@max_id"
# 				puts @max_id; gets
# 				@use_large_cache = false
			@ids.sort!
			redone_count = 0
			
			
# 					puts "hello"
# 				ep @use_large_cache_but_recheck_incomplete; exit
			recheck_incomplete_runs if use_large_cache_but_recheck_incomplete
# 				sort_runs
			eprint "Updating runs..." if @write_status_dots
# 				get_all_root_folder_contents
# 				puts @run_list.values[0].directory, File.expand_path(@root_folder).esc_regex
			fix_directories = (run_list.size > 0 and not @run_list.values[0].directory =~ File.expand_path(@root_folder).esc_regex)
# 				eputs 'fdirectories', fix_directories
# 				exit
			eputs "Fixing Directories..." if fix_directories
			@run_list.each do |id, run|
				eprint '.' if @write_status_dots
				run.directory = File.join(@root_folder, run.relative_directory) if fix_directories
# 					run.directory = "#@root_folder/#{run.relative_directory}"
# 					unless @root_folder_contents.include? run.directory# File.directory? run.directory and run.directory =~ File.expand_path(@root_folder).esc_regex
# 						if @root_folder_contents.include?(rel = File.join(@root_folder, run.relative_directory))
# 							run.directory = rel
# 						else
# 							raise CRFatal.new("Directory #{run.directory} not found")
# 						end
# 					end
# 					eputs @use_phantom
				#run.generate_phantom_runs #if @use_phantom.to_s =~ /phantom/i
				run.phantom_runs.each{|r| add_phantom_run(r)} if run.phantom_runs
			end
			eputs if @write_status_dots
			save_large_cache if fix_directories
# 				puts redone_count
			return self
		rescue Errno::ENOENT, ArgumentError, TypeError => err
			if err.class == ArgumentError and not err.message =~ /marshal data too short/ or err.class == TypeError and not err.message =~ /incompatible marshal file format/
				eputs err
				eputs err.backtrace
				raise CRFatal.new
			end
			eputs err, "Rereading run data"
# 				@use_large_cache = false
		end
	end

	log("not using large cache")
	@run_list={}
	@ids=[]
	@phantom_run_list = {}
	@phantom_ids = []
	@run_store =[]
# 		@multiple_processes_directories = []
	set_max_id 0

	log("traversing directories")
	eprint 'Analysing runs..' if @write_status_dots
	#interrupted = false; 
	#old_trap2 = trap(2){}
	#trap(2){eputs "Interrupt acknowledged... reloading saved cache. Interrupt again to override (may cause file corruption)."; trap(2, old_trap2); update(true, true, false); Process.kill(2,0)}
	Dir.chdir(@root_folder){traverse_directories}
	@max_id ||= 0
	eputs
# 		puts 'before request', @ids, @requests;
	respond_to_requests
# 		@n_checks += 1
# 		exit if ($nruns > 0 && @n_checks > $nruns) 
	sort_runs
	@recalc_all = false
# 		pp @run_list
	save_large_cache
	#@run_list.each{|id, run| run.generate_phantom_runs}
	#trap(2, old_trap2)
	#Process.kill(2, 0) if interrupted
	return self
end

#write_data(filename = "data.txt") ⇒ Object

Write out a simple datafile containing all the inputs and outputs listed in the run class property readout_string. What is actually written out can be customised by redefining the method data_string in the run class, or changing readout_string or both.



597
598
599
600
601
602
603
604
605
606
607
608
609
610
 
# File 'lib/coderunner/instance_methods.rb', line 597

def write_data(filename = "data.txt")
	logf(:write_data)
	generate_combined_ids
	File.open(filename, "w") do |file|
		@combined_ids.each do |id|
			run = @combined_run_list[id]
			if run.status =~ /Complete/
				data_string = run.data_string
				raise CRFatal.new("data_string did not return a string") unless data_string.class == String
				file.puts data_string 
			end
		end
	end
end