Class: SpreadsheetAgent::Agent

Inherits:
Db
  • Object
show all
Defined in:
lib/spreadsheet_agent/agent.rb

Overview

Note:

The license of this source is “MIT Licence”

SpreadsheetAgent::Agent is designed to make it easy to create a single task which connects to a field within a record on a page within the configured SpreadsheetAgent compatible Google Spreadsheet, runs supplied code, and reports whether the job completed or ended in error. An agent can be configured to only run when certain prerequisite fields have completed. The data in these fields can be filled in by other SpreadsheetAgent::Agents, SpreadsheetAgent::Runners, or humans. Compute node configuration is available to prevent the agent from running more than a certain number of instances of itself, or not run if certain other agents or processes are running on the node. Finally, an agent can be configured to subsume another agent, and fill in the completion field for that agent in addition to its own when it completes successfully.

Author:

  • Darin London Copyright 2013

Instance Attribute Summary collapse

Attributes inherited from Db

#config, #config_file, #db, #session

Instance Method Summary collapse

Methods inherited from Db

#build_db

Constructor Details

#initialize(attributes) ⇒ Agent

create a new SpreadsheetAgent::Agent

Parameters:

  • attributes (Hash) —

    keys are the attribute names, values are their values

Options Hash (attributes):

  • agent_name (String) —

    REQUIRED

  • page_name (String) —

    REQUIRED

  • keys (Hash) —

    REQUIRED

  • config_file (String) — default: see SpreadsheetAgent::DB
  • debug (Boolean)
  • prerequisites (Array)
  • max_selves (Integer)
  • conflicts_with (Hash)
  • subsumes (Array) 


82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/spreadsheet_agent/agent.rb', line 82

def initialize(attributes)
  @agent_name = attributes[:agent_name]
  @page_name = attributes[:page_name]
  @keys = attributes[:keys].clone
  unless @agent_name && @page_name && @keys
    raise SpreadsheetAgentError, "agent_name, page_name, and keys attributes are required!"
  end
  @config_file = attributes[:config_file]
  build_db()

  @worksheet = @db.worksheet_by_title(@page_name)
  @debug = attributes[:debug]
  if attributes[:prerequisites]
    @prerequisites = attributes[:prerequisites].clone
  end

  @max_selves = attributes[:max_selves]
  if attributes[:conflicts_with]
    @conflicts_with = attributes[:conflicts_with].clone
  end
  if attributes[:subsumes]
    @subsumes = attributes[:subsumes].clone
  end
end

Instance Attribute Details

#agent_name ⇒ String

The name of the field in the page to which the agent should report status

Returns:

  • (String) 


19
20
21
 
# File 'lib/spreadsheet_agent/agent.rb', line 19

def agent_name
  @agent_name
end

#conflicts_with ⇒ Hash

Note:

This works on Linux with ps.

List of other processes, and the maximum number of running instances of the process that are allowed before this agent should avoid running on the given server. Hash of process_name => number of max_instances. If specified, each key is treated as a process name in ps. If the agent detects the specified number of max_instances of the given process (based on a line match), it will not attempt to run. If not specified, it will run regardless of the other processes already running on a server.

Returns:

  • (Hash) 


59
60
61
 
# File 'lib/spreadsheet_agent/agent.rb', line 59

def conflicts_with
  @conflicts_with
end

#debug ⇒ Boolean

Specify whether to print debug information (default false). When true, the agent code will print verbosely to STDERR. When false, and the process! returns a failure status, the agent will email all stdout and stderr to the email specified in the :config send_to value

Returns:

  • (Boolean) 


37
38
39
 
# File 'lib/spreadsheet_agent/agent.rb', line 37

def debug
  @debug
end

#keys ⇒ Hash

Hash used to find the entry on the Google Spreadsheet Worksheet Keys are defined in config/agent.conf.yml. All keys configured as ‘required: 1’ must be included in the keys hash. Values specify values for those fields in the record on the page for which the agent is running.

Returns:

  • (Hash) 


30
31
32
 
# File 'lib/spreadsheet_agent/agent.rb', line 30

def keys
  @keys
end

#max_selves ⇒ Integer

Note:

This works on Linux with ps.

Maximum number of instances of this agent to run on any particular server. If specified, newly instantiated agents will not attempt to run process! if there are max_selves instances already running on the same server. If not specified, all instances will attempt to run.

Returns:

  • (Integer) 


51
52
53
 
# File 'lib/spreadsheet_agent/agent.rb', line 51

def max_selves
  @max_selves
end

#page_name ⇒ String

The name of the Page on the Google Spreadsheet that contains the record to be worked on by the agent

Returns:

  • (String) 


23
24
25
 
# File 'lib/spreadsheet_agent/agent.rb', line 23

def page_name
  @page_name
end

#prerequisites ⇒ Array

Optional array of prerequisites. If supplied, each entry is treated as a field name on the Google Worksheet which must contain a 1 in it for the record on the page before this agent will attempt to run.

Returns:

  • (Array) 


43
44
45
 
# File 'lib/spreadsheet_agent/agent.rb', line 43

def prerequisites
  @prerequisites
end

#subsumes ⇒ Array

List of fields (agent or otherwise) that this agent should also complete when it completes successfully. Each entry is treated as a fields on the record which this agent subsumes. If the agent completes successfully these fields will be updated with a 1 in addition to the field for the agent.

Returns:

  • (Array) 


65
66
67
 
# File 'lib/spreadsheet_agent/agent.rb', line 65

def subsumes
  @subsumes
end

#worksheet ⇒ GoogleDrive::Worksheet (readonly)

The GoogleDrive::Worksheet that is being access by the agent.

Returns:

  • (GoogleDrive::Worksheet) 


69
70
71
 
# File 'lib/spreadsheet_agent/agent.rb', line 69

def worksheet
  @worksheet
end

Instance Method Details

#get_entry ⇒ GoogleDrive::List

The GoogleDrive::List for the specified keys

Returns:

  • (GoogleDrive::List) 


188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
 
# File 'lib/spreadsheet_agent/agent.rb', line 188

def get_entry
  this_entry = nil
  if @worksheet
    @worksheet.list.each do |this_row|
      keep_row = true

      @config['key_fields'].keys.reject { |key_field|
        !(@config['key_fields'][key_field]["required"]) && !(@keys[key_field])
      }.each do |key|
        break unless keep_row
        keep_row = (this_row[key] == @keys[key])
      end
    
      if keep_row
        return this_row
      end
    end
  end
end

#process!(&agent_code) {|entry| ... } ⇒ Object

If the agent does not have any conflicting processes (max_selves or conflicts_with) and if the entry field ‘ready’ has a 1, and any supplied prerequisite fields have a 1, gets the GoogleDrive::List record, and passes it to the supplied Proc. This PROC must return a required boolean field indicating success or failure, and an optional hash of key - value fields that will be updated on the GoogleDrive::List record. Note, the updates are made regardless of the value of success. In fact, the agent can be configured to update different fields based on success or failure. Also, note that any value can be stored in the hash. This allows the agent to communicate any useful information to the google spreadsheet for other agents (SpreadsheetAgent::Agent, SpreadsheetAgent::Runner, or human) to use. The Proc must try at all costs to avoid terminating. If an error is encountered, it should return false for the success field to signal that the process failed. If no errors are encountered it should return true for the success field.

Examples:

Exit successfully, enters a 1 in the agent_name field

$agent->process! do |entry|
  true
end

Same, but also updates the ‘notice’ field in the record along with the 1 in the agent_name field

$agent->process! do |entry|
  [true, {:notice => 'There were 30 files processed'}]
end

Fails, enters f:server_hostname in the agent_name field

$agent->process! do |entry|
  false

Same, but also updates the ‘notice’ field in the record along with the failure notice

$agent->process! do |entry|
  [false, {:notice => 'There were 10 files left to process!' }]
end

This agent passes different parameters based on success or failure

$agent->process! do |entry|
  if $success
    true
  else
    [ false, {:notice => 'there were 10 remaining files'}]
  end
end

Parameters:

  • agent_code (Proc) —

    Code to process entry

Yield Parameters:

  • entry (GoogleDrive::List)

Yield Returns:

  • (Boolean, Hash) —

    success, (optional) hash of fields to update and values to update on the fields



151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
 
# File 'lib/spreadsheet_agent/agent.rb', line 151

def process!(&agent_code)
  @worksheet.reload
  no_problems = true
  capture_output = nil
  unless @debug
    capture_output = CaptureIO.new
    capture_output.start
  end

  begin
    return true if has_conflicts()
    (runnable, entry) = run_entry()
    return false unless entry
    return true unless runnable
  
    success, update_entry = agent_code.call(entry)
    if success
      complete_entry(update_entry)
    else
      fail_entry(update_entry)
    end
  rescue
    $stderr.puts "#{ $! }"
    no_problems = false
  end
  unless capture_output.nil?
    if no_problems
      capture_output.stop
    else
      mail_error(capture_output.stop)
    end
  end
  return no_problems
end