Class: SemanticLogger::Appender::MongoDB

Inherits:
Base
  • Object
show all
Defined in:
lib/semantic_logger/appender/mongodb.rb

Overview

The Mongo Appender for the SemanticLogger

Mongo Document Schema:

_id        : ObjectId("4d9cbcbf7abb3abdaf9679cd"),
time       : ISODate("2011-04-06T19:19:27.006Z"),
host_name  : "Name of the host on which this log entry originated",
application: 'Name of application or service logging the data - clarity_base, nginx, tomcat',
pid        : process id
thread_name: "name or id of thread",
name       : "com.clarity.MyClass",
level      : 'trace|debug|info|warn|error|fatal'
level_index: 0|1|2|3|4|5
message    : "Message supplied to the logging call",
duration   : ms,  # Set by Logger#benchmark
tags       : "Some tracking id" | ["id1", "id2"]
payload : {
  Optional. Any user supplied data, including any thread specific context variables
  values supplied on a per log entry will override any thread context values
}
# When an exception is supplied as the first or second parameter
# If supplied as the first parameter, message='exception name'
exception: {
  name:        'MyException',
  message:     'Invalid value',
  stack_trace: []
}

Instance Attribute Summary collapse

Attributes inherited from Base

#formatter

Attributes inherited from Base

#level, #name

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Base

colorized_formatter

Methods inherited from Base

default_level, default_level=, #payload, #pop_tags, #push_tags, #tagged, #tags, #with_payload

Constructor Details

#initialize(params = {}, &block) ⇒ MongoDB

Create a MongoDB Appender instance

SemanticLogger::Appender::MongoDB.new(:db => Mongo::Connection.new['database'])

Parameters: :db [Mongo::Database]

The MongoDB database connection to use, not the database name

:collection_name [String]

Name of the collection to store log data in
Default: semantic_logger

:host_name [String]

host_name to include in the document logged to Mongo
Default: first part of host name extracted from Socket

:safe [Boolean]

Whether to use safe write for logging
Not recommended to change this value except to diagnose connection
issues or when log entries are not being written to Mongo
Default: false

:application [String]

Name of the application to include in the document written to mongo
Default: nil (None)

:collection_size [Integer]

 The size of the MongoDB capped collection to create in bytes
 Default: 1 GB

 Some examples
   Prod: 25GB (.5GB per day across 4 servers over 10 days)
   Dev: .5GB
   Test: File
   Release: 4GB

:collection_max [Integer]
  Maximum number of log entries that the capped collection will hold

:level [Symbol]
  Only allow log entries of this level or higher to be written to MongoDB



78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 78

def initialize(params={}, &block)
  @db              = params[:db] || raise('Missing mandatory parameter :db')
  @collection_name = params[:collection_name] || 'semantic_logger'
  @host_name       = params[:host_name] || Socket.gethostname.split('.').first
  @safe            = params[:safe] || false
  @application     = params[:application]

  # Create a collection that will hold the lesser of 1GB space or 10K documents
  @collection_size = params[:collection_size] || 1024**3
  @collection_max  = params[:collection_max]

  # Create the collection and necessary indexes
  create_indexes

  # Set the log level and formatter
  super(params[:level], &block)
end

Instance Attribute Details

#application ⇒ Object

Returns the value of attribute application.



34
35
36
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 34

def application
  @application
end

#collection_name ⇒ Object (readonly)

Returns the value of attribute collection_name.



33
34
35
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 33

def collection_name
  @collection_name
end

#db ⇒ Object (readonly)

Returns the value of attribute db.



33
34
35
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 33

def db
  @db
end

#host_name ⇒ Object

Returns the value of attribute host_name.



34
35
36
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 34

def host_name
  @host_name
end

#safe ⇒ Object

Returns the value of attribute safe.



34
35
36
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 34

def safe
  @safe
end

Class Method Details

.host_name ⇒ Object

Default host_name to use if none is supplied to the appenders initializer



163
164
165
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 163

def self.host_name
  @@host_name ||= Socket.gethostname.split('.').first
end

.host_name=(host_name) ⇒ Object

Override the default host_name



168
169
170
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 168

def self.host_name=(host_name)
  @@host_name = host_name
end

.strip_colorizing(message) ⇒ Object

Strip the standard Rails colorizing from the logged message



158
159
160
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 158

def self.strip_colorizing(message)
  message.to_s.gsub(/(\e(\[([\d;]*[mz]?))?)?/, '').strip
end

Instance Method Details

#collection ⇒ Object

Return the collection being used to write the log document to



120
121
122
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 120

def collection
  @collection ||= db[collection_name]
end

#create_indexes ⇒ Object

Create the required capped collection Features of capped collection:

  • No indexes by default (not even on _id)

  • Documents cannot be deleted,

  • Document updates cannot make them any larger

  • Documents are always stored in insertion order

    • A find will always return the documents in their insertion order

Creates an index based on tags to support faster lookups



105
106
107
108
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 105

def create_indexes
  db.create_collection(collection_name, {:capped => true, :size => @collection_size, :max => @collection_max})
  collection.ensure_index('tags')
end

#default_formatter ⇒ Object

Default log formatter

Replace this formatter by supplying a Block to the initializer



132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 132

def default_formatter
  Proc.new do |log|
    document = {
      :time        => log.time,
      :host_name   => host_name,
      :pid         => $PID,
      :thread_name => log.thread_name,
      :name        => log.name,
      :level       => log.level,
      :level_index => log.level_index,
    }
    document[:application] = application if application
    document[:message]     = self.class.strip_colorizing(log.message) if log.message
    document[:duration]    = log.duration if log.duration
    document[:tags]        = log.tags if log.tags && (log.tags.size > 0)
    document[:payload]     = log.payload if log.payload
    document[:exception]   = {
      :name        => log.exception.class.name,
      :message     => log.exception.message,
      :stack_trace => log.exception.backtrace
    } if log.exception
    document
  end
end

#flush ⇒ Object

Flush all pending logs to disk.

Waits for all sent documents to be written to disk



126
127
128
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 126

def flush
  db.get_last_error
end

#log(log) ⇒ Object

Log the message to MongoDB



173
174
175
176
177
178
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 173

def log(log)
  # Insert log entry into Mongo
  # Use safe=>false so that we do not wait for it to be written to disk, or
  # for the response from the MongoDB server
  collection.insert(formatter.call(log), :safe=>safe) if level_index <= (log.level_index || 0)
end

#purge_all ⇒ Object

Purge all data from the capped collection by dropping the collection and recreating it. Also useful when the size of the capped collection needs to be changed



113
114
115
116
117
 
# File 'lib/semantic_logger/appender/mongodb.rb', line 113

def purge_all
  collection.drop
  @collection = nil
  create_indexes
end