Request log analyzer
This is a simple command line tool to analyze request log files of both Rails and Merb. Its purpose is to find what actions are best candidates for optimization.
This tool will parse all requests in the logfile and aggregate the information. Once it is finished parsing the log file, it will show the requests that take op most server time. Different metrics are used (cumulative time, average time, blockers, DB time, etc)
Installation
gem sources -a gems.github.com sudo gem install wvanbergen-request-log-analyzer
Usage
Usage: request-log-analyzer [FILE] [OPTION] Analyze the given log FILE with the given OPTION Example: request-log-analyzer mongrel.log
--fast, -t: Only use completed requests
--guess-database-time, -g: Guesses the database duration of requests if they are not in the log
--output, -o: Comma-separated list of reports to show
--amount, -c: Displays the top <amount> elements in the reports
--colorize, -z: Fancy bash coloring
Example
Note that this example was shortened for your viewing pleasure. $ request-log-analyzer /var/log/my_app.log
Request log analyzer, by Willem van Bergen and Bart ten Brinke
Processing all log lines…
Successfully analyzed 58908 requests from log file
Timestamp first request: 2008-07-13T06:25:58+00:00 Timestamp last request: 2008-07-20T06:18:53+00:00 Total time analyzed: 7 days Methods: DELETE (1%), GET (50%), POST (22%), PUT (25%).
Top 10 most requested actions
/overview/:date/ : 19359 requests /overview/day/:date/ : 6365 requests /overview/:date/set/ : 5589 requests /overview/ : 3985 requests /clients/:id/ : 1976 requests .….…
Top 10 actions by time - cumulative
/overview/:date/ : 9044.582s [19359 requests] /overview/ : 8478.767s [3985 requests] /overview/:date/set/ : 3309.041s [5589 requests] /clients/:id/products/:id/ : 1479.911s [924 requests] /clients/:id/ : 750.080s [1976 requests] .….…
Top 10 actions by time - per request mean
/overview/ : 2.128s [3985 requests] /clients/:id/products/:id/ : 1.602s [924 requests] /overview/:date/set/ : 0.592s [5589 requests] /overview/:date/ : 0.467s [19359 requests] /clients/:id/ : 0.380s [1976 requests] .….…
Top 10 worst DB offenders - cumulative time
/overview/:date/ : 8773.993s [19359 requests] /overview/ : 8394.754s [3985 requests] /overview/:date/set/ : 3307.928s [5589 requests] /clients/:id/products/:id/ : 1425.220s [924 requests] /clients/:id/ : 535.229s [1976 requests] .….…
Top 10 worst DB offenders - mean time
/overview/:id/:id/:id/print/ : 6.994s [448 requests] /overview/ : 2.128s [3985 requests] /clients/:id/products/:id/ : 1.602s [924 requests] /overview/:date/set/ : 0.592s [5589 requests] /overview/:date/ : 0.467s [19359 requests] .….…
Mongrel process blockers (> 1.0 seconds)
/overview/:date/ : 7494.233s [3144 requests] /overview/ : 8320.293s [1549 requests] /overview/:date/set/ : 1149.235s [803 requests] /overview/:id/:id/:id/print/new/ : 613.693s [341 requests] /clients/:id/products/:id/ : 1370.693s [313 requests] .….…
Requests graph - per hour
........
7:00 - 2731 : XXXXXXX
8:00 - 6139 : XXXXXXXXXXXXXXXX
9:00 - 7465 : XXXXXXXXXXXXXXXXXXXX
10:00 - 7118 : XXXXXXXXXXXXXXXXXXX
11:00 - 7409 : XXXXXXXXXXXXXXXXXXX
12:00 - 6450 : XXXXXXXXXXXXXXXXX
13:00 - 5377 : XXXXXXXXXXXXXX
14:00 - 6058 : XXXXXXXXXXXXXXXX
15:00 - 4156 : XXXXXXXXXXX
16:00 - 2767 : XXXXXXX
17:00 - 1598 : XXXX
18:00 - 792 : XX
........
Errors
ArgumentError: [237 requests]
-> invalid date
StaleObjectError: [28 requests]
-> Attempted to update a stale object
RuntimeError: [3 requests]
-> Cannot destroy rule before it was created
StatementError: [2 requests]
-> Mysql::Error: Deadlock found when to get lock; try restarting transaction
NoMethodError: [1 requests]
-> undefined method `code' for nil:NilClass