Class: Daru::IO::Importers::Log

Inherits:
Base
  • Object
show all
Defined in:
lib/daru/io/importers/log.rb

Overview

Log Importer Class, that extends read_rails_log method to Daru::DataFrame

Constant Summary collapse

ORDERS = 
{
  rails3: %i[method path ip timestamp line_type lineno source
             controller action format params rendered_file
             partial_duration status duration view db].freeze,

  apache: %i[remote_host remote_logname user timestamp http_method
             path http_version http_status bytes_sent referer
             user_agent line_type lineno source].freeze,

  amazon_s3: %i[bucket_owner bucket timestamp remote_ip requester request_id operation
                key request_uri http_status error_code bytes_sent object_size total_time
                turnaround_time referer user_agent line_type lineno source].freeze
}.freeze
RENAME_FIELDS = 
{
  path: :resource_path
}.freeze

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Base

#optional_gem

Constructor Details

#initialize ⇒ Log

Returns a new instance of Log.



11
12
13
 
# File 'lib/daru/io/importers/log.rb', line 11

def initialize
  optional_gem 'request-log-analyzer', '~> 1.13.4', requires: 'request_log_analyzer'
end

Class Method Details

.read(path, format: :rails3) ⇒ Daru::IO::Importers::Log

Reads data from a log file

Examples:

Reading from rails log file

instance = Daru::IO::Importers::Log.read("rails_test.log")

Reading from apache log file

instance = Daru::IO::Importers::Log.new.read("apache_test.log", format: :apache)

Reading from amazon s3 log file

instance = Daru::IO::Importers::Log.new.read("amazon_s3_test.log", format: :amazon_s3)

Parameters:

  • path (String) —

    Path to log file, where the dataframe is to be imported from.

  • format (Symbol) (defaults to: :rails3) —

    Format of log file, which can be :rails3, :apache or :amazon_s3 default format set to :rails3

Returns:



53
54
55
56
57
58
59
60
61
62
63
 
# File 'lib/daru/io/importers/log.rb', line 53

def read(path, format: :rails3)
  @format = format
  @file_data = RequestLogAnalyzer::Source::LogParser
               .new(RequestLogAnalyzer::FileFormat.load(@format), source_files: path)
               .map do |request|
                 ORDERS
                   .fetch(@format)
                   .map { |attr| request.attributes.include?(attr) ? request.attributes[attr] : nil }
               end
  self
end

Instance Method Details

#call ⇒ Daru::DataFrame

Imports a Daru::DataFrame from a Log Importer instance and log file

Examples:

Reading from a log file

df = instance.call

=> #<Daru::DataFrame(150x17)>
#         method resource_path         ip  timestamp  line_type     lineno     source contr...
#   0        GET             /  127.0.0.1 2018022607  completed          5  /home/roh Rails...
#   1        GET             /  127.0.0.1 2018022716  completed         12  /home/roh Rails...
# ...        ...        ...        ...        ...        ...        ...        ...      ...

Returns:



77
78
79
80
 
# File 'lib/daru/io/importers/log.rb', line 77

def call
  Daru::DataFrame.rows(@file_data, order: ORDERS.fetch(@format)
    .map { |attr| RENAME_FIELDS.fetch(attr, attr) })
end

#read(path, format: :rails3) ⇒ Daru::IO::Importers::Log

Reads data from a log file

Examples:

Reading from rails log file

instance = Daru::IO::Importers::Log.read("rails_test.log")

Reading from apache log file

instance = Daru::IO::Importers::Log.new.read("apache_test.log", format: :apache)

Reading from amazon s3 log file

instance = Daru::IO::Importers::Log.new.read("amazon_s3_test.log", format: :amazon_s3)

Parameters:

  • path (String) —

    Path to log file, where the dataframe is to be imported from.

  • format (Symbol) (defaults to: :rails3) —

    Format of log file, which can be :rails3, :apache or :amazon_s3 default format set to :rails3

Returns:



53
54
55
56
57
58
59
60
61
62
63
 
# File 'lib/daru/io/importers/log.rb', line 53

def read(path, format: :rails3)
  @format = format
  @file_data = RequestLogAnalyzer::Source::LogParser
               .new(RequestLogAnalyzer::FileFormat.load(@format), source_files: path)
               .map do |request|
                 ORDERS
                   .fetch(@format)
                   .map { |attr| request.attributes.include?(attr) ? request.attributes[attr] : nil }
               end
  self
end