Class: Maatkit::LogPlayer
- Inherits:
-
Object
- Object
- Maatkit::LogPlayer
- Defined in:
- lib/maatkit-ruby/mk-log-player.rb
Overview
Replay MySQL query logs.
Maatkit::LogPlayer.new( array, str, array)
Instance Attribute Summary collapse
-
#ask_pass ⇒ Object
group: Play Prompt for a password when connecting to MySQL.
-
#base_dir ⇒ Object
type: string; default: ./ Base directory for “–split” session files and “–play” result file.
-
#base_file_name ⇒ Object
type: string; default: session Base file name for “–split” session files and “–play” result file.
-
#charset ⇒ Object
group: Play short form: -A; type: string Default character set.
-
#config ⇒ Object
type: Array Read this comma-separated list of config files; if specified, this must be the first option on the command line.
-
#defaults_file ⇒ Object
short form: -F; type: string Only read mysql options from the given file.
-
#dry_run ⇒ Object
Print which processes play which session files then exit.
-
#filter ⇒ Object
type: string; group: Split Discard “–split” events for which this Perl code doesn’t return true.
-
#help ⇒ Object
Show help and exit.
-
#host ⇒ Object
short form: -h; type: string; group: Play Connect to host.
-
#iterations ⇒ Object
type: int; default: 1; group: Play How many times each thread should play all its session files.
-
#max_sessions ⇒ Object
type: int; default: 5000000; group: Split Maximum number of sessions to “–split”.
-
#only_select ⇒ Object
group: Play Play only SELECT and USE queries; ignore all others.
-
#password ⇒ Object
short form: -p; type: string; group: Play Password to use when connecting.
-
#path_to_mk_log_player ⇒ Object
Sets the executable path, otherwise the environment path will be used.
-
#pid ⇒ Object
type: string Create the given PID file.
-
#play ⇒ Object
type: string; group: Play Play (execute) session files created by “–split”.
-
#port ⇒ Object
short form: -P; type: int; group: Play Port number to use for connection.
-
#print ⇒ Object
group: Play Print queries instead of playing them; requires “–play”.
-
#quiet ⇒ Object
Do not print anything; disables “–verbose”.
-
#results ⇒ Object
default: yes Print “–play” results to files in “–base-dir”.
-
#session_files ⇒ Object
type: int; default: 8; group: Split Number of session files to create with “–split”.
-
#set_vars ⇒ Object
type: string; group: Play; default: wait_timeout=10000 Set these MySQL variables.
-
#socket ⇒ Object
short form: -S; type: string; group: Play Socket file to use for connection.
-
#split ⇒ Object
type: string; group: Split Split log by given attribute to create session files.
-
#split_random ⇒ Object
group: Split Split log without an attribute, write queries round-robin to session files.
-
#threads ⇒ Object
type: int; default: 2; group: Play Number of threads used to play sessions concurrently.
-
#type ⇒ Object
type: string; group: Split The type of log to “–split” (default slowlog).
-
#user ⇒ Object
short form: -u; type: string; group: Play User for login if not current user.
-
#verbose ⇒ Object
short form: -v; cumulative: yes; default: 0 Increase verbosity; can specifiy multiple times.
-
#version ⇒ Object
Show version and exit.
-
#wait_between_sessions ⇒ Object
type: array; default: 0; group: Play Not implemented yet.
-
#warnings ⇒ Object
default: no; group: Play Print warnings about SQL errors such as invalid queries to STDERR.
Instance Method Summary collapse
-
#initialize ⇒ LogPlayer
constructor
Returns a new LogPlayer Object.
-
#start(options = nil) ⇒ Object
Execute the command.
Constructor Details
#initialize ⇒ LogPlayer
Returns a new LogPlayer Object
259 260 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 259 def initialize() end |
Instance Attribute Details
#ask_pass ⇒ Object
group: Play Prompt for a password when connecting to MySQL.
18 19 20 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 18 def ask_pass @ask_pass end |
#base_dir ⇒ Object
type: string; default: ./ Base directory for “–split” session files and “–play” result file.
23 24 25 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 23 def base_dir @base_dir end |
#base_file_name ⇒ Object
type: string; default: session Base file name for “–split” session files and “–play” result file. Each “–split” session file will be saved as <base-file-name>-N.txt, where N is a four digit, zero- padded session ID. For example: session-0003.txt. Each “–play” result file will be saved as <base-file-name>-results-PID.txt, where PID is the process ID of the executing thread. All files are saved in “–base-dir”.
33 34 35 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 33 def base_file_name @base_file_name end |
#charset ⇒ Object
group: Play short form: -A; type: string Default character set. If the value is utf8, sets Perl’s binmode on STDOUT to utf8, passes the mysql_enable_utf8 option to DBD::mysql, and runs SET NAMES UTF8 after connecting to MySQL. Any other value sets binmode on STDOUT without the utf8 layer, and runs SET NAMES after connecting to MySQL.
41 42 43 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 41 def charset @charset end |
#config ⇒ Object
type: Array Read this comma-separated list of config files; if specified, this must be the first option on the command line.
47 48 49 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 47 def config @config end |
#defaults_file ⇒ Object
short form: -F; type: string Only read mysql options from the given file.
52 53 54 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 52 def defaults_file @defaults_file end |
#dry_run ⇒ Object
Print which processes play which session files then exit.
56 57 58 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 56 def dry_run @dry_run end |
#filter ⇒ Object
type: string; group: Split Discard “–split” events for which this Perl code doesn’t return true. This option only works with “–split”. This option is a string of Perl code or a file containing Perl code that gets compiled into a subroutine with one argument: $event. This is a hashref. If the given value is a readable file, then mk-log-player reads the entire file and uses its contents as the code. The file should not contain a shebang (#!/usr/bin/perl) line. If the code returns true, the query is split; otherwise it is discarded. The code is the last statement in the subroutine other than “return $event”. The subroutine template is:
sub { $event = shift; filter && return $event; }
Filters given on the command line are wrapped inside parentheses like like “( filter )”. For complex, multi-line filters, you must put the code inside a file so it will not be wrapped inside parentheses. Either way, the filter must produce syntactically valid code given the template. For example, an if- else branch given on the command line would not be valid:
--filter 'if () { } else { }' # WRONG
Since it’s given on the command line, the if-else branch would be wrapped inside parentheses which is not syntactically valid. So to accomplish something more complex like this would require putting the code in a file, for example filter.txt:
my $event_ok; if (...) { $event_ok=1; } else { $event_ok=0; } $event_ok
Then specify “–filter filter.txt” to read the code from filter.txt. If the filter code won’t compile, mk-log-player will die with an error. If the filter code does compile, an error may still occur at runtime if the code tries to do something wrong (like pattern match an undefined value). mk-log-player does not provide any safeguards so code carefully! An example filter that discards everything but SELECT statements:
--filter '$event->{arg} =~ m/^select/i'
This is compiled into a subroutine like the following:
sub { $event = shift; ( $event->{arg} =~ m/^select/i ) && return $event; }
You can find an explanation of the structure of $event at <code.google.com/p/maatkit/wiki/EventAttributes>.
88 89 90 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 88 def filter @filter end |
#help ⇒ Object
Show help and exit.
92 93 94 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 92 def help @help end |
#host ⇒ Object
short form: -h; type: string; group: Play Connect to host.
97 98 99 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 97 def host @host end |
#iterations ⇒ Object
type: int; default: 1; group: Play How many times each thread should play all its session files.
102 103 104 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 102 def iterations @iterations end |
#max_sessions ⇒ Object
type: int; default: 5000000; group: Split Maximum number of sessions to “–split”. By default, “mk-log-player” tries to split every session from the log file. For huge logs, however, this can result in millions of sessions. This option causes only the first N number of sessions to be saved. All sessions after this number are ignored, but sessions split before this number will continue to have their queries split even if those queries appear near the end of the log and after this number has been reached.
112 113 114 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 112 def max_sessions @max_sessions end |
#only_select ⇒ Object
group: Play Play only SELECT and USE queries; ignore all others.
117 118 119 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 117 def only_select @only_select end |
#password ⇒ Object
short form: -p; type: string; group: Play Password to use when connecting.
122 123 124 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 122 def password @password end |
#path_to_mk_log_player ⇒ Object
Sets the executable path, otherwise the environment path will be used.
254 255 256 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 254 def path_to_mk_log_player @path_to_mk_log_player end |
#pid ⇒ Object
type: string Create the given PID file. The file contains the process ID of the script. The PID file is removed when the script exits. Before starting, the script checks if the PID file already exists. If it does not, then the script creates and writes its own PID to it. If it does, then the script checks the following: if the file contains a PID and a process is running with that PID, then the script dies; or, if there is no process running with that PID, then the script overwrites the file with its own PID and starts; else, if the file contains no PID, then the script dies.
132 133 134 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 132 def pid @pid end |
#play ⇒ Object
type: string; group: Play Play (execute) session files created by “–split”. The argument to play must be a commaxn-separated list of session files created by “–split” or a directory. If the argument is a directory, ALL files in that directory will be played.
139 140 141 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 139 def play @play end |
#port ⇒ Object
short form: -P; type: int; group: Play Port number to use for connection.
144 145 146 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 144 def port @port end |
#print ⇒ Object
group: Play Print queries instead of playing them; requires “–play”. You must also specify “–play” with “–print”. Although the queries will not be executed, “–play” is required to specify which session files to read.
151 152 153 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 151 def print @print end |
#quiet ⇒ Object
Do not print anything; disables “–verbose”.
155 156 157 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 155 def quiet @quiet end |
#results ⇒ Object
default: yes Print “–play” results to files in “–base-dir”.
160 161 162 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 160 def results @results end |
#session_files ⇒ Object
type: int; default: 8; group: Split Number of session files to create with “–split”. The number of session files should either be equal to the number of “–threads” you intend to “–play” or be an even multiple of “–threads”. This number is important for maximum performance because it:
* allows each thread to have roughly the same amount of sessions to play
* avoids having to open/close many session files
* avoids disk IO overhead by doing large sequential reads
You may want to increase this number beyond “–threads” if each session file becomes too large. For example, splitting a 20G log into 8 sessions files may yield roughly eight 2G session files. See also “–max-sessions”.
173 174 175 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 173 def session_files @session_files end |
#set_vars ⇒ Object
type: string; group: Play; default: wait_timeout=10000 Set these MySQL variables. Immediately after connecting to MySQL, this string will be appended to SET and executed.
179 180 181 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 179 def set_vars @set_vars end |
#socket ⇒ Object
short form: -S; type: string; group: Play Socket file to use for connection.
184 185 186 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 184 def socket @socket end |
#split ⇒ Object
type: string; group: Split Split log by given attribute to create session files. Valid attributes are any which appear in the log: Thread_id, Schema, etc.
190 191 192 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 190 def split @split end |
#split_random ⇒ Object
group: Split Split log without an attribute, write queries round-robin to session files. This option, if specified, overrides “–split” and causes the log to be split query-by-query, writing each query to the next session file in round-robin style. If you don’t care about “sessions” and just want to split a lot into N many session files and the relation or order of the queries does not matter, then use this option.
199 200 201 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 199 def split_random @split_random end |
#threads ⇒ Object
type: int; default: 2; group: Play Number of threads used to play sessions concurrently. Specifies the number of parallel processes to run. The default is 2. On GNU/Linux machines, the default is the number of times ‘processor’ appears in /proc/cpuinfo. On Windows, the default is read from the environment. In any case, the default is at least 2, even when there’s only a single processor. See also “–session-files”.
209 210 211 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 209 def threads @threads end |
#type ⇒ Object
type: string; group: Split The type of log to “–split” (default slowlog). The permitted types are binlog
# Split a binary log file.
genlog
# Split a general log file.
slowlog
# Split a log file in any varation of MySQL slow-log format.
220 221 222 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 220 def type @type end |
#user ⇒ Object
short form: -u; type: string; group: Play User for login if not current user.
225 226 227 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 225 def user @user end |
#verbose ⇒ Object
short form: -v; cumulative: yes; default: 0 Increase verbosity; can specifiy multiple times. This option is disabled by “–quiet”.
231 232 233 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 231 def verbose @verbose end |
#version ⇒ Object
Show version and exit.
235 236 237 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 235 def version @version end |
#wait_between_sessions ⇒ Object
type: array; default: 0; group: Play Not implemented yet. The wait time is given in seconds with microsecond precision and can be either a single value or a range. A single value causes an exact wait; example: 0.010 = wait 10 milliseconds. A range causes a random wait between the given value times; example: 0.001,1 = random wait from 1 millisecond to 1 second.
244 245 246 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 244 def wait_between_sessions @wait_between_sessions end |
#warnings ⇒ Object
default: no; group: Play Print warnings about SQL errors such as invalid queries to STDERR.
249 250 251 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 249 def warnings @warnings end |
Instance Method Details
#start(options = nil) ⇒ Object
Execute the command
265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 |
# File 'lib/maatkit-ruby/mk-log-player.rb', line 265 def start( = nil) tmp = Tempfile.new('tmp') command = option_string() + .to_s + " 2> " + tmp.path success = system(command) if success begin while (line = tmp.readline) line.chomp selected_string = line end rescue EOFError tmp.close end return selected_string else tmp.close! return success end end |