Class: SpreadsheetAgent::Agent

Inherits:
Db
  • Object
show all
Defined in:
lib/spreadsheet_agent.rb

Overview

SpreadsheetAgent::Agent is designed to make it easy to create a single task which connects to a field within a record on a page within the configured SpreadsheetAgent compatible Google Spreadsheet, runs, and reports whether the job completed or ended in error. An agent can be configured to only run when certain prerequisite fields have completed. The data in these fields can be filled in by other SpreadsheetAgent::Agents, SpreadsheetAgent::Runners, or humans. Compute node configuration is available to prevent the agent from running more than a certain number of instances of itself, or not run if certain other agents or processes are running on the node. Finally, an agent can be configured to subsume another agent, and fill in the completion field for that agent in addition to its own when it completes successfully.

extends SpreadsheetAgent::Db

Instance Attribute Summary collapse

Attributes inherited from Db

#config, #config_file, #db, #session

Instance Method Summary collapse

Methods inherited from Db

#build_db

Constructor Details

#initialize(attributes) ⇒ Agent

create a new SpreadsheetAgent::Agent with the following:

required configuration parameters:

  • agent_name

  • page_name

  • keys

optional parameters:

  • config_file: (see SpreadsheetAgent::DB)

  • debug

  • prerequisites

  • max_selves

  • conflicts_with

  • subsumes



97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
 
# File 'lib/spreadsheet_agent.rb', line 97

def initialize(attributes)
  @agent_name = attributes[:agent_name]
  @page_name = attributes[:page_name]
  @keys = attributes[:keys].clone
  unless @agent_name && @page_name && @keys
    raise SpreadsheetAgentError, "agent_name, page_name, and keys attributes are required!"
  end
  @config_file = attributes[:config_file]
  build_db()

  @worksheet = @db.worksheet_by_title(@page_name)
  @debug = attributes[:debug]
  if attributes[:prerequisites]
    @prerequisites = attributes[:prerequisites].clone
  end

  @max_selves = attributes[:max_selves]
  if attributes[:conflicts_with]
    @conflicts_with = attributes[:conflicts_with].clone
  end
  if attributes[:subsumes]
    @subsumes = attributes[:subsumes].clone
  end
end

Instance Attribute Details

#agent_name ⇒ Object

The name of the field in the page to which the agent should report status



48
49
50
 
# File 'lib/spreadsheet_agent.rb', line 48

def agent_name
  @agent_name
end

#conflicts_with ⇒ Object

Hash of process_name to number of max_instances. This works on Linux with ps. If the agent detects the specified number of max_instances of the given process (based on a line match), it will not attempt to run



74
75
76
 
# File 'lib/spreadsheet_agent.rb', line 74

def conflicts_with
  @conflicts_with
end

#debug ⇒ Object

Boolean. When true, the agent code will print verbosely to STDERR. When false, and the process! returns a failure status, the agent will email all stdout and stderr to the email specified in the :config send_to value



61
62
63
 
# File 'lib/spreadsheet_agent.rb', line 61

def debug
  @debug
end

#keys ⇒ Object

hash of key-value pairs. The keys are defined in config/agent.conf.yml. The values specify the values for those fields in the record on the page for which the agent is running. All keys configured as ‘required: 1’ in config/agent.conf.yml must be included in the keys hash



56
57
58
 
# File 'lib/spreadsheet_agent.rb', line 56

def keys
  @keys
end

#max_selves ⇒ Object

Optional integer. This works on Linux with ps. The agent will not attempt to run if there are max_selves instances running



69
70
71
 
# File 'lib/spreadsheet_agent.rb', line 69

def max_selves
  @max_selves
end

#page_name ⇒ Object

The name of the Page on the Google Spreadsheet that contains the record to be worked on by the agent



51
52
53
 
# File 'lib/spreadsheet_agent.rb', line 51

def page_name
  @page_name
end

#prerequisites ⇒ Object

Optional array of prerequisite fields that must contain a 1 in them for the record on the page before the agent will attempt to run



65
66
67
 
# File 'lib/spreadsheet_agent.rb', line 65

def prerequisites
  @prerequisites
end

#subsumes ⇒ Object

Array of fields on the record which this agent subsumes. If the agent completes successfully these fields will be updated with a 1 in addition to the field for the agent



78
79
80
 
# File 'lib/spreadsheet_agent.rb', line 78

def subsumes
  @subsumes
end

#worksheet ⇒ Object (readonly)

Readonly access to the GoogleDrive::Worksheet that is being access by the agent.



81
82
83
 
# File 'lib/spreadsheet_agent.rb', line 81

def worksheet
  @worksheet
end

Instance Method Details

#get_entry ⇒ Object

Returns the GoogleDrive::List object for the specified keys



199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
 
# File 'lib/spreadsheet_agent.rb', line 199

def get_entry
  this_entry = nil
  if @worksheet
    @worksheet.list.each do |this_row|
      keep_row = true

      @config['key_fields'].keys.reject { |key_field|
        !(@config['key_fields'][key_field]["required"]) && !(@keys[key_field])
      }.each do |key|
        break unless keep_row
        keep_row = (this_row[key] == @keys[key])
      end
    
      if keep_row
        return this_row
      end
    end
  end
end

#process!(&agent_code) ⇒ Object

If the agent does not have any conflicting processes (max_selves or conflicts_with) and if the entry is ready (field ‘ready’ has a 1), and all prerequisite fields have a 1, gets the GoogleDrive::List record, and passes it to the supplied agent_code PROC as argument. This PROC must return a required boolean field indicating success or failure, and an optional hash of key - value fields that will be updated on the GoogleDrive::List record. Note, the updates are made regardless of the value of success. In fact, the agent can be configured to update different fields based on success or failure. Also, note that any value can be stored in the hash. This allows the agent to communicate any useful information to the google spreadsheet for other agents (SpreadsheetAgent::Agent, SpreadsheetAgent::Runner, or human) to use. The PROC must try at all costs to avoid terminating. If an error is encountered, it should return false for the success field to signal that the process failed. If no errors are encountered it should return true for the success field.

Exits successfully, enters a 1 in the agent_name field
$agent->process! do |entry|
  true
end

Same, but also updates the 'notice' field in the record along with the 1 in the agent_name field
$agent->process! do |entry|
  [true, {:notice => 'There were 30 files processed'}]
end

Fails, enters f:#{hostname} in the agent_name field
$agent->process! do |entry|
  false

Same, but also updates the 'notice' field in the record along with the failure notice
$agent->process! do |entry|
  [false, {:notice => 'There were 10 files left to process!' }]
end

This agent passes different parameters based on success or failure
$agent->process! do |entry|
  if $success
    true
  else
    [ false, {:notice => 'there were 10 remaining files'}]
  end
end



163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
 
# File 'lib/spreadsheet_agent.rb', line 163

def process!(&agent_code)
  @worksheet.reload
  no_problems = true
  capture_output = nil
  unless @debug
    capture_output = CaptureIO.new
    capture_output.start
  end

  begin
    return true if has_conflicts()
    (runnable, entry) = run_entry()
    return false unless entry
    return true unless runnable
  
    success, update_entry = agent_code.call(entry)
    if success
      complete_entry(update_entry)
    else
      fail_entry(update_entry)
    end
  rescue
    $stderr.puts "#{ $! }"
    no_problems = false
  end
  unless capture_output.nil?
    if no_problems
      capture_output.stop
    else
      mail_error(capture_output.stop)
    end
  end
  return no_problems
end