Class: REC::State

Inherits:
Object show all
Defined in:
lib/rec/state.rb

Overview

A State is an object that represents the memory of something having happened. For example, “server terra is down”. It also remembers useful statistics about what caused this state to be (the original log entries and the rule they matched), for how long it should remain in memory, what it pertains to (eg. the server called ‘terra’).

A state is also useful for other rules to refer to. For example, a second rule matching “host terra is up” can check if the server is currently down by reference to the state with a title of “host terra is down”.

This is much more useful than matching log entries one by one without any memory of what has gone before. You cannot correlate events without keeping State.

Constant Summary collapse

Generate =

shortcut action to generate a message on each event

Proc.new { |state|
	state.generate()
}
Ignore =

shortcut action to ignore an event, but leave the state to expire

Proc.new { |state|
}
Generate_and_release =

shortcut action to generate a message and release the state immediately

Proc.new { |state|
	state.generate()
	state.release()
}
Generate_first_only =

shortcut action to generate a message on first event only, swallow the rest

Proc.new { |state|
   state.generate(:alert) if state.count == 1
}
Notify_urgently_first_only =

shortcut action to notify the admin immediately (eg. system is down) and wait (eg. for system to come up again)

Proc.new { |state|
  Notify.urgent(state.generate(:alert)) if state.count == 1
}
Notify_urgently_and_release =

shortcut action to notify urgently and release Good for a single event (eg. backup failed) requiring immediate action

Proc.new { |state|
  Notify.urgent(state.generate(:alert))
state.release()
}
Notify_and_release =

shortcut action to notify and release Good for a single event not requiring immediate action

Proc.new { |state|
  Notify.normal(state.generate(:alert))
state.release()
}
Notify_duration = 
Proc.new { |state|
  # We must have only one allstate, which is the original state (eg. "server down")
  original = state.params[:allstates][0]				  # get the original state's key
  duration = state.find(original).age
  # we expect to have :alert containing "%duration$d minutes"
  state.details["duration"] = duration.to_i()/60	# store the outage duration
  Notify.normal(state.generate())					        # send the notice
  state.release(original)							            # forget both
  state.release()
}
Log_duration = 
Proc.new { |state|
  # We must have only one allstate, which is the original state (eg. "process started")
  original = state.params[:allstates][0]				  # get the original state's key
  duration = state.find(original).age
  # we expect to have :alert containing "%duration$d minutes"
  state.details["duration"] = duration.to_i()/60	# store the duration
  state.generate()      					                # record the event
  state.release(original)							            # forget both
  state.release()
}
@@timeouts =

A array of Timeouts. A Timeout struct has two elements:

  • timestamp at which to expire

  • key of the state to be expired

[]
@@states =

A hash of states, keyed on state title

{}
@@eventsOut =

A count of new events sent to output

0

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(title, lifespan, params = {}) ⇒ State

Creates a new state with the given (unique) title, the lifespan before the state is forgotten, and a hash of parameters.

Note that the time is not necessarily ‘now’ because REC can be executed against historical log files. It uses the timestamp of the original log entry, not the current clock time.



181
182
183
184
185
186
187
188
189
190
191
192
 
# File 'lib/rec/state.rb', line 181

def initialize(title, lifespan, params={})
	@created = @updated = Correlator.now()
	@title = title
	@lifespan = lifespan.to_f
	@params = params
	@count = 0
	@rid = 0
	@logs = []		# array of remembered logLines
	@details = {}	# hash of remembered details
	@@states[title] = self
	State.timeout_at(@created + @lifespan, @title)
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(symbol, *args) ⇒ Object

Allow access to any parameter by a convenience method state.capture is more succinct than state.params



281
282
283
 
# File 'lib/rec/state.rb', line 281

def method_missing(symbol, *args)
	@params[symbol]
end

Instance Attribute Details

#alert ⇒ Object (readonly)

An alert message to be sent if subsequent events warrant it



159
160
161
 
# File 'lib/rec/state.rb', line 159

def alert
  @alert
end

#count ⇒ Object (readonly)

Number of times this state has been matched



165
166
167
 
# File 'lib/rec/state.rb', line 165

def count
  @count
end

#created ⇒ Object (readonly)

time when this state was created



167
168
169
 
# File 'lib/rec/state.rb', line 167

def created
  @created
end

#details ⇒ Object (readonly)

Hash of custom fields to be remembered



171
172
173
 
# File 'lib/rec/state.rb', line 171

def details
  @details
end

#final ⇒ Object (readonly)

An alert message to be sent when the state expires



161
162
163
 
# File 'lib/rec/state.rb', line 161

def final
  @final
end

#lifespan ⇒ Object (readonly)

How long this state shoudl live before being automatically forgotten



157
158
159
 
# File 'lib/rec/state.rb', line 157

def lifespan
  @lifespan
end

#logs ⇒ Object (readonly)

Array of original log entries matching this state



173
174
175
 
# File 'lib/rec/state.rb', line 173

def logs
  @logs
end

#params ⇒ Object (readonly)

Hash of parameters of the rule that created this state



163
164
165
 
# File 'lib/rec/state.rb', line 163

def params
  @params
end

#rid ⇒ Object (readonly)

unique ID of the rule which gave rise to this state



153
154
155
 
# File 'lib/rec/state.rb', line 153

def rid
  @rid
end

#title ⇒ Object (readonly)

The unique title for this state (eg. “server earth is down”)



155
156
157
 
# File 'lib/rec/state.rb', line 155

def title
  @title
end

#updated ⇒ Object (readonly)

last time this state was updated



169
170
171
 
# File 'lib/rec/state.rb', line 169

def updated
  @updated
end

Class Method Details

.[](key) ⇒ Object

Returns the state matching the given key (title)



96
97
98
 
# File 'lib/rec/state.rb', line 96

def self.[](key)
	@@states[key]
end

.expire_states ⇒ Object

Deletes all expired states, executing :onexpiry blocks before deletion



115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
 
# File 'lib/rec/state.rb', line 115

def self.expire_states()
	timeout = @@timeouts.first()
	while @@timeouts.length > 0 and timeout.expiry < Correlator.now() do
		state = State[timeout.key]
		if state.nil?
			@@timeouts.shift
			timeout = @@timeouts.first()
			next
		end
		#$stderr.puts("Releasing state #{state.title} with count of #{state.count}")
		final = Rule[state.rid].params[:final]
		final.call(state) if final
		@@states.delete(@@timeouts.shift().key)
		timeout = @@timeouts.first()
	end
end

.find(template, state) ⇒ Object

Returns a matching state or nil.

This is used to locate a state, typically the other state created by a pair of rules. For example, in a rule handling “Server earth is up” we want to find the state corresponding to “Server earth is down”

  • template is a string like “Server %host$s is down”

  • state is the current state



147
148
149
150
 
# File 'lib/rec/state.rb', line 147

def self.find(template, state)
	title = template.sprinth(state.details)
	@@states[title]
end

.stats ⇒ Object

Returns a 2-element array containing:

  • the number of states

  • the number of new events sent to output



135
136
137
138
 
# File 'lib/rec/state.rb', line 135

def self.stats()
	statesCount = @@states.keys.length
	[statesCount, @@eventsOut]
end

.timeout_at(time, title) ⇒ Object

Add a Timeout for the given time and specified state. This timeout is sorted into the correct sequence of timeouts to make State::expire_states more efficient.



102
103
104
105
106
107
108
109
110
111
112
 
# File 'lib/rec/state.rb', line 102

def self.timeout_at(time, title)
	tnew = Struct::Timeout.new(time, title)
	n = @@timeouts.find_index { |to|
		to.expiry > time
	}
	if n.nil?
		@@timeouts = @@timeouts << tnew
	else
		@@timeouts[n..n] = [tnew, @@timeouts[n]]
	end
end

Instance Method Details

#age ⇒ Object

Returns the age of this state



212
213
214
 
# File 'lib/rec/state.rb', line 212

def age
	Correlator.now() - @created
end

#extend_for(dur) ⇒ Object

Resets the expiry time to be dur seconds after current time (this may be shorter than the original lifetime)



232
233
234
235
236
237
238
239
 
# File 'lib/rec/state.rb', line 232

def extend_for(dur)
	n = @@timeouts.find_index { |to|
		to.title == @title
	}
	@@timeouts[n..n] = [] unless n.nil?
	expiry = Correlator.now() + dur
	self.timeout_at(expiry, @title)
end

#find(template) ⇒ Object

Convenience method to find another state, based on parameters in this state



273
274
275
 
# File 'lib/rec/state.rb', line 273

def find(template)
   State::find(template, self)
end

#generate(sym = :alert) ⇒ Object

Creates a new event, writes it to the output log, and returns the event. An event (or ‘log entry’) is a timestamp followed by a message

Returns the message only (without the timestamp).



255
256
257
258
259
260
261
262
263
264
265
 
# File 'lib/rec/state.rb', line 255

def generate(sym = :alert)
	message = @params[sym].sprinth(stats())
	if message.length > 0
		event = "%s %s" % [@updated.iso8601, message] + @logs.join("\n")
		$stdout.print("> ") if $debug
		$stdout.puts(event)
		$stdout.flush()
		@@eventsOut = @@eventsOut + 1
	end
	message
end

#generate_first_only(sym = :alert) ⇒ Object

Creates a new event when this state is created, but ignores later occurrences



268
269
270
 
# File 'lib/rec/state.rb', line 268

def generate_first_only(sym = :alert)
	generate(sym) if @count == 1
end

#release(pattern = nil) ⇒ Object

Forget a state. - if no pattern is provided, forget this state. - if a pattern is provided, use the stats for this state to determine the title of the other state, and remove that from memory. For example, if the server is back up again, we no longer need to remember that the server was down.



222
223
224
225
226
227
228
 
# File 'lib/rec/state.rb', line 222

def release(pattern=nil)
	if pattern.nil?
		@@states.delete(@title)
	else
		@@states.delete(pattern.sprinth(stats()))
	end
end

#stats ⇒ Object

Returns the details of the state (ie. whatever custom fields were defined, eg. userid) merged with the standard statistics:

  • count: number of matches so far

  • age: age of the state

  • created: time when this state was created

  • updated: time last updated (eg. when the latest event matched this state)



247
248
249
 
# File 'lib/rec/state.rb', line 247

def stats()
	@details.merge({"count"=>@count, "age"=>age(), "created"=>@created, "updated"=>@updated})
end

#update(rid, matches, logLine = nil) ⇒ Object

Updates the statistics of this state (following a match). Remembers lots of useful things: - the number of matches so far

  • time last updated

  • age of this state

  • the ID of the rule which last matched

  • more details (custom fields) may be added to memory

  • message templates (for values to be interpolated into)

  • a list of the original log entries pertaining to this state



203
204
205
206
207
208
209
 
# File 'lib/rec/state.rb', line 203

def update(rid, matches, logLine=nil)
	@count = @count.succ
	@updated = Correlator.now()
	@rid = rid
	@details.merge!(matches)
	@logs << logLine if @params[:capture]
end