Class: Maatkit::TableSync
- Inherits:
-
Object
- Object
- Maatkit::TableSync
- Defined in:
- lib/maatkit-ruby/mk-table-sync.rb
Overview
Synchronize MySQL table data efficiently.
Maatkit::TableSync.new( array, str, array)
Instance Attribute Summary collapse
-
#algorithms ⇒ Object
type: string; default: Chunk,Nibble,GroupBy,Stream Algorithm to use when comparing the tables, in order of preference.
-
#ask_pass ⇒ Object
Prompt for a password when connecting to MySQL.
-
#bidirectional ⇒ Object
Enable bidirectional sync between first and subsequent hosts.
-
#bin_log ⇒ Object
default: yes Log to the binary log (SET SQL_LOG_BIN=1).
-
#buffer_in_mysql ⇒ Object
Instruct MySQL to buffer queries in its memory.
-
#buffer_to_client ⇒ Object
default: yes Fetch rows one-by-one from MySQL while comparing.
-
#charset ⇒ Object
short form: -A; type: string Default character set.
-
#check_master ⇒ Object
default: yes With –sync-to-master, try to verify that the detected master is the real master.
-
#check_privileges ⇒ Object
default: yes Check that user has all necessary privileges on source and destination table.
-
#check_slave ⇒ Object
default: yes Check whether the destination server is a slave.
-
#check_triggers ⇒ Object
default: yes Check that no triggers are defined on the destination table.
-
#chunk_column ⇒ Object
type: string Chunk the table on this column.
-
#chunk_index ⇒ Object
type: string Chunk the table using this index.
-
#chunk_size ⇒ Object
type: string; default: 1000 Number of rows or data size per chunk.
-
#columns ⇒ Object
short form: -c; type: array Compare this comma-separated list of columns.
-
#config ⇒ Object
type: Array Read this comma-separated list of config files; if specified, this must be the first option on the command line.
-
#conflict_column ⇒ Object
type: string Compare this column when rows conflict during a –bidirectional sync.
-
#conflict_comparison ⇒ Object
type: string Choose the –conflict-column with this property as the source.
-
#conflict_error ⇒ Object
type: string; default: warn How to report unresolvable conflicts and conflict errors This option changes how the user is notified when a conflict cannot be resolved or causes some kind of error.
-
#conflict_threshold ⇒ Object
type: string Amount by which one –conflict-column must exceed the other.
-
#conflict_value ⇒ Object
type: string Use this value for certain –conflict-comparison.
-
#databases ⇒ Object
short form: -d; type: hash Sync only this comma-separated list of databases.
-
#defaults_file ⇒ Object
short form: -F; type: string Only read mysql options from the given file.
-
#dry_run ⇒ Object
Analyze, decide the sync algorithm to use, print and exit.
-
#engines ⇒ Object
short form: -e; type: hash Sync only this comma-separated list of storage engines.
-
#execute ⇒ Object
Execute queries to make the tables have identical data.
-
#explain_hosts ⇒ Object
Print connection information and exit.
-
#float_precision ⇒ Object
type: int Precision for FLOAT and DOUBLE number-to-string conversion.
-
#foreign_key_checks ⇒ Object
default: yes Enable foreign key checks (SET FOREIGN_KEY_CHECKS=1).
-
#function ⇒ Object
type: string Which hash function you’d like to use for checksums.
-
#help ⇒ Object
Show help and exit.
-
#hex_blob ⇒ Object
default: yes HEX() BLOB, TEXT and BINARY columns.
-
#host ⇒ Object
short form: -h; type: string Connect to host.
-
#ignore_columns ⇒ Object
type: Hash Ignore this comma-separated list of column names in comparisons.
-
#ignore_databases ⇒ Object
type: Hash Ignore this comma-separated list of databases.
-
#ignore_engines ⇒ Object
type: Hash; default: FEDERATED,MRG_MyISAM Ignore this comma-separated list of storage engines.
-
#ignore_tables ⇒ Object
type: Hash Ignore this comma-separated list of tables.
-
#index_hint ⇒ Object
default: yes Add FORCE/USE INDEX hints to the chunk and row queries.
-
#lock ⇒ Object
type: int Lock tables: 0=none, 1=per sync cycle, 2=per table, or 3=globally.
-
#lock_and_rename ⇒ Object
Lock the source and destination table, sync, then swap names.
-
#password ⇒ Object
short form: -p; type: string Password to use when connecting.
-
#path_to_mk_table_sync ⇒ Object
Sets the executable path, otherwise the environment path will be used.
-
#pid ⇒ Object
type: string Create the given PID file.
-
#port ⇒ Object
short form: -P; type: int Port number to use for connection.
-
#print ⇒ Object
Print queries that will resolve differences.
-
#recursion_method ⇒ Object
type: string Preferred recursion method used to find slaves.
-
#replace ⇒ Object
Write all INSERT and UPDATE statements as REPLACE.
-
#replicate ⇒ Object
type: string Sync tables listed as different in this table.
-
#set_vars ⇒ Object
type: string; default: wait_timeout=10000 Set these MySQL variables.
-
#socket ⇒ Object
short form: -S; type: string Socket file to use for connection.
-
#sync_to_master ⇒ Object
Treat the DSN as a slave and sync it to its master.
-
#tables ⇒ Object
short form: -t; type: hash Sync only this comma-separated list of tables.
-
#timeout_ok ⇒ Object
Keep going if –wait fails.
-
#transaction ⇒ Object
Use transactions instead of LOCK TABLES.
-
#trim ⇒ Object
TRIM() VARCHAR columns in BIT_XOR and ACCUM modes.
-
#unique_checks ⇒ Object
default: yes Enable unique key checks (SET UNIQUE_CHECKS=1).
-
#user ⇒ Object
short form: -u; type: string User for login if not current user.
-
#verbose ⇒ Object
short form: -v; cumulative: yes Print results of sync operations.
-
#version ⇒ Object
Show version and exit.
-
#wait ⇒ Object
short form: -w; type: time How long to wait for slaves to catch up to their master.
-
#where ⇒ Object
type: string WHERE clause to restrict syncing to part of the table.
-
#zero_chunk ⇒ Object
default: yes Add a chunk for rows with zero or zero-equivalent values.
Instance Method Summary collapse
-
#initialize ⇒ TableSync
constructor
Returns a new TableSync Object.
-
#start(options = nil) ⇒ Object
Execute the command.
Constructor Details
#initialize ⇒ TableSync
Returns a new TableSync Object
411 412 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 411 def initialize() end |
Instance Attribute Details
#algorithms ⇒ Object
type: string; default: Chunk,Nibble,GroupBy,Stream Algorithm to use when comparing the tables, in order of preference. For each table, mk-table-sync will check if the table can be synced with the given algorithms in the order that they’re given. The first algorithm that can sync the table is used. See ALGORITHMS.
20 21 22 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 20 def algorithms @algorithms end |
#ask_pass ⇒ Object
Prompt for a password when connecting to MySQL.
25 26 27 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 25 def ask_pass @ask_pass end |
#bidirectional ⇒ Object
Enable bidirectional sync between first and subsequent hosts.
30 31 32 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 30 def bidirectional @bidirectional end |
#bin_log ⇒ Object
default: yes Log to the binary log (SET SQL_LOG_BIN=1). Specifying –no-bin-log will SET SQL_LOG_BIN=0.
37 38 39 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 37 def bin_log @bin_log end |
#buffer_in_mysql ⇒ Object
Instruct MySQL to buffer queries in its memory. This option adds the SQL_BUFFER_RESULT option to the comparison queries. This causes MySQL to execute the queries and place them in a temporary table internally before sending the results back to mk-table-sync. The advantage of this strategy is that mk-table-sync can fetch rows as desired without using a lot of memory inside the Perl process, while releasing locks on the MySQL table (to reduce contention with other queries). The disadvantage is that it uses more memory on the MySQL server instead. You probably want to leave –[no]buffer-to-client enabled too, because buffering into a temp table and then fetching it all into Perl’s memory is probably a silly thing to do. This option is most useful for the GroupBy and Stream algorithms, which may fetch a lot of data from the server.
44 45 46 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 44 def buffer_in_mysql @buffer_in_mysql end |
#buffer_to_client ⇒ Object
default: yes Fetch rows one-by-one from MySQL while comparing. This option enables mysql_use_result which causes MySQL to hold the selected rows on the server until the tool fetches them. This allows the tool to use less memory but may keep the rows locked on the server longer. If this option is disabled by specifying –no-buffer-to-client then mysql_store_result is used which causes MySQL to send all selected rows to the tool at once. This may result in the results “cursor” being held open for a shorter time on the server, but if the tables are large, it could take a long time anyway, and use all your memory. For most non-trivial data sizes, you want to leave this option enabled. This option is disabled when –bidirectional is used.
54 55 56 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 54 def buffer_to_client @buffer_to_client end |
#charset ⇒ Object
short form: -A; type: string Default character set. If the value is utf8, sets Perl’s binmode on STDOUT to utf8, passes the mysql_enable_utf8 option to DBD::mysql, and runs SET NAMES UTF8 after connecting to MySQL. Any other value sets binmode on STDOUT without the utf8 layer, and runs SET NAMES after connecting to MySQL.
60 61 62 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 60 def charset @charset end |
#check_master ⇒ Object
default: yes With –sync-to-master, try to verify that the detected master is the real master.
66 67 68 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 66 def check_master @check_master end |
#check_privileges ⇒ Object
default: yes Check that user has all necessary privileges on source and destination table.
71 72 73 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 71 def check_privileges @check_privileges end |
#check_slave ⇒ Object
default: yes Check whether the destination server is a slave. If the destination server is a slave, it’s generally unsafe to make changes on it. However, sometimes you have to; –replace won’t work unless there’s a unique index, for example, so you can’t make changes on the master in that scenario. By default mk-table-sync will complain if you try to change data on a slave. Specify –no-slave-check to disable this check. Use it at your own risk.
77 78 79 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 77 def check_slave @check_slave end |
#check_triggers ⇒ Object
default: yes Check that no triggers are defined on the destination table. Triggers were introduced in MySQL v5.0.2, so for older versions this option has no effect because triggers will not be checked.
83 84 85 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 83 def check_triggers @check_triggers end |
#chunk_column ⇒ Object
type: string Chunk the table on this column.
88 89 90 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 88 def chunk_column @chunk_column end |
#chunk_index ⇒ Object
type: string Chunk the table using this index.
93 94 95 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 93 def chunk_index @chunk_index end |
#chunk_size ⇒ Object
type: string; default: 1000 Number of rows or data size per chunk. The size of each chunk of rows for the Chunk and Nibble algorithms. The size can be either a number of rows, or a data size. Data sizes are specified with a suffix of k=kibibytes, M=mebibytes, G=gibibytes. Data sizes are converted to a number of rows by dividing by the average row length.
99 100 101 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 99 def chunk_size @chunk_size end |
#columns ⇒ Object
short form: -c; type: array Compare this comma-separated list of columns.
104 105 106 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 104 def columns @columns end |
#config ⇒ Object
type: Array Read this comma-separated list of config files; if specified, this must be the first option on the command line.
109 110 111 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 109 def config @config end |
#conflict_column ⇒ Object
type: string Compare this column when rows conflict during a –bidirectional sync. When a same but differing row is found the value of this column from each row is compared according to –conflict-comparison, –conflict-value and –conflict-threshold to determine which row has the correct data and becomes the source. The column can be any type for which there is an appropriate –conflict-comparison (this is almost all types except, for example, blobs). This option only works with –bidirectional. See BIDIRECTIONAL SYNCING for more information.
116 117 118 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 116 def conflict_column @conflict_column end |
#conflict_comparison ⇒ Object
type: string Choose the –conflict-column with this property as the source. The option affects how the –conflict-column values from the conflicting rows are compared. Possible comparisons are one of these MAGIC_comparisons: newest|oldest|greatest|least|equals|matches COMPARISON CHOOSES ROW WITH
=========================================================
newest #Newest temporal L<–conflict-column> value oldest #Oldest temporal L<–conflict-column> value greatest # Greatest numerical L<–conflict-column> value least # Least numerical L<–conflict-column> value equals #L<–conflict-column> value equal to L<–conflict-value> matches #L<–conflict-column> value matching Perl regex pattern
# #L<--conflict-value>
This option only works with –bidirectional. See BIDIRECTIONAL SYNCING for more information.
133 134 135 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 133 def conflict_comparison @conflict_comparison end |
#conflict_error ⇒ Object
type: string; default: warn How to report unresolvable conflicts and conflict errors This option changes how the user is notified when a conflict cannot be resolved or causes some kind of error. Possible values are:
-
warn: Print a warning to STDERR about the unresolvable conflict
-
die: Die, stop syncing, and print a warning to STDERR
This option only works with –bidirectional. See BIDIRECTIONAL SYNCING for more information.
142 143 144 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 142 def conflict_error @conflict_error end |
#conflict_threshold ⇒ Object
type: string Amount by which one –conflict-column must exceed the other. The –conflict-threshold prevents a conflict from being resolved if the absolute difference between the two –conflict-column values is less than this amount. For example, if two –conflict-column have timestamp values “2009-12-01 12:00:00” and “2009-12-01 12:05:00” the difference is 5 minutes. If –conflict-threshold is set to “5m” the conflict will be resolved, but if –conflict-threshold is set to “6m” the conflict will fail to resolve because the difference is not greater than or equal to 6 minutes. In this latter case, –conflict-error will report the failure. This option only works with –bidirectional. See BIDIRECTIONAL SYNCING for more information.
149 150 151 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 149 def conflict_threshold @conflict_threshold end |
#conflict_value ⇒ Object
type: string Use this value for certain –conflict-comparison. This option gives the value for equals and matches –conflict-comparison. This option only works with –bidirectional. See BIDIRECTIONAL SYNCING for more information.
156 157 158 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 156 def conflict_value @conflict_value end |
#databases ⇒ Object
short form: -d; type: hash Sync only this comma-separated list of databases. A common request is to sync tables from one database with tables from another database on the same or different server. This is not yet possible. –databases will not do it, and you can’t do it with the D part of the DSN either because in the absence of a table name it assumes the whole server should be synced and the D part controls only the connection’s default database.
162 163 164 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 162 def databases @databases end |
#defaults_file ⇒ Object
short form: -F; type: string Only read mysql options from the given file. You must give an absolute pathname.
167 168 169 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 167 def defaults_file @defaults_file end |
#dry_run ⇒ Object
Analyze, decide the sync algorithm to use, print and exit. Implies –verbose so you can see the results. The results are in the same output format that you’ll see from actually running the tool, but there will be zeros for rows affected. This is because the tool actually executes, but stops before it compares any data and just returns zeros. The zeros do not mean there are no changes to be made.
172 173 174 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 172 def dry_run @dry_run end |
#engines ⇒ Object
short form: -e; type: hash Sync only this comma-separated list of storage engines.
177 178 179 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 177 def engines @engines end |
#execute ⇒ Object
Execute queries to make the tables have identical data. This option makes mk-table-sync actually sync table data by executing all the queries that it created to resolve table differences. Therefore, the tables will be changed! And unless you also specify –verbose, the changes will be made silently. If this is not what you want, see –print or –dry-run.
182 183 184 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 182 def execute @execute end |
#explain_hosts ⇒ Object
Print connection information and exit. Print out a list of hosts to which mk-table-sync will connect, with all the various connection options, and exit.
187 188 189 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 187 def explain_hosts @explain_hosts end |
#float_precision ⇒ Object
type: int Precision for FLOAT and DOUBLE number-to-string conversion. Causes FLOAT and DOUBLE values to be rounded to the specified number of digits after the decimal point, with the ROUND() function in MySQL. This can help avoid checksum mismatches due to different floating-point representations of the same values on different MySQL versions and hardware. The default is no rounding; the values are converted to strings by the CONCAT() function, and MySQL chooses the string representation. If you specify a value of 2, for example, then the values 1.008 and 1.009 will be rounded to 1.01, and will checksum as equal.
192 193 194 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 192 def float_precision @float_precision end |
#foreign_key_checks ⇒ Object
default: yes Enable foreign key checks (SET FOREIGN_KEY_CHECKS=1). Specifying –no-foreign-key-checks will SET FOREIGN_KEY_CHECKS=0.
198 199 200 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 198 def foreign_key_checks @foreign_key_checks end |
#function ⇒ Object
type: string Which hash function you’d like to use for checksums. The default is CRC32. Other good choices include MD5 and SHA1. If you have installed the FNV_64 user-defined function, mk-table-sync will detect it and prefer to use it, because it is much faster than the built-ins. You can also use MURMUR_HASH if you’ve installed that user-defined function. Both of these are distributed with Maatkit. See mk-table-checksum for more information and benchmarks.
204 205 206 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 204 def function @function end |
#help ⇒ Object
Show help and exit.
208 209 210 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 208 def help @help end |
#hex_blob ⇒ Object
default: yes HEX() BLOB, TEXT and BINARY columns. When row data from the source is fetched to create queries to sync the data (i.e. the queries seen with –print and executed by –execute), binary columns are wrapped in HEX() so the binary data does not produce an invalid SQL statement. You can disable this option but you probably shouldn’t.
214 215 216 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 214 def hex_blob @hex_blob end |
#host ⇒ Object
short form: -h; type: string Connect to host.
219 220 221 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 219 def host @host end |
#ignore_columns ⇒ Object
type: Hash Ignore this comma-separated list of column names in comparisons. This option causes columns not to be compared. However, if a row is determined to differ between tables, all columns in that row will be synced, regardless. (It is not currently possible to exclude columns from the sync process itself, only from the comparison.)
225 226 227 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 225 def ignore_columns @ignore_columns end |
#ignore_databases ⇒ Object
type: Hash Ignore this comma-separated list of databases.
230 231 232 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 230 def ignore_databases @ignore_databases end |
#ignore_engines ⇒ Object
type: Hash; default: FEDERATED,MRG_MyISAM Ignore this comma-separated list of storage engines.
235 236 237 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 235 def ignore_engines @ignore_engines end |
#ignore_tables ⇒ Object
type: Hash Ignore this comma-separated list of tables. Table names may be qualified with the database name.
241 242 243 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 241 def ignore_tables @ignore_tables end |
#index_hint ⇒ Object
default: yes Add FORCE/USE INDEX hints to the chunk and row queries. By default mk-table-sync adds a FORCE/USE INDEX hint to each SQL statement to coerce MySQL into using the index chosen by the sync algorithm or specified by –chunk-index. This is usually a good thing, but in rare cases the index may not be the best for the query so you can suppress the index hint by specifying –no-index-hint and let MySQL choose the index. This does not affect the queries printed by –print; it only affects the chunk and row queries that mk-table-sync uses to select and compare rows.
248 249 250 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 248 def index_hint @index_hint end |
#lock ⇒ Object
type: int Lock tables: 0=none, 1=per sync cycle, 2=per table, or 3=globally. This uses LOCK TABLES. This can help prevent tables being changed while you’re examining them. The possible values are as follows: VALUE MEANING
=======================================================
0 #Never lock tables. 1 #Lock and unlock one time per sync cycle (as implemented
# by the syncing algorithm). This is the most granular
# level of locking available. For example, the Chunk
# algorithm will lock each chunk of C<N> rows, and then
# unlock them if they are the same on the source and the
# destination, before moving on to the next chunk.
2 #Lock and unlock before and after each table. 3 #Lock and unlock once for every server (DSN) synced, with
# C<FLUSH TABLES WITH READ LOCK>.
A replication slave is never locked if –replicate or –sync-to-master is specified, since in theory locking the table on the master should prevent any changes from taking place. (You are not changing data on your slave, right?) If –wait is given, the master (source) is locked and then the tool waits for the slave to catch up to the master before continuing. If –transaction is specified, LOCK TABLES is not used. Instead, lock and unlock are implemented by beginning and committing transactions. The exception is if –lock is 3. If –no-transaction is specified, then LOCK TABLES is used for any value of –lock. See –[no]transaction.
269 270 271 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 269 def lock @lock end |
#lock_and_rename ⇒ Object
Lock the source and destination table, sync, then swap names. This is useful as a less-blocking ALTER TABLE, once the tables are reasonably in sync with each other (which you may choose to accomplish via any number of means, including dump and reload or even something like mk-archiver). It requires exactly two DSNs and assumes they are on the same server, so it does no waiting for replication or the like. Tables are locked with LOCK TABLES.
273 274 275 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 273 def lock_and_rename @lock_and_rename end |
#password ⇒ Object
short form: -p; type: string Password to use when connecting.
278 279 280 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 278 def password @password end |
#path_to_mk_table_sync ⇒ Object
Sets the executable path, otherwise the environment path will be used.
406 407 408 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 406 def path_to_mk_table_sync @path_to_mk_table_sync end |
#pid ⇒ Object
type: string Create the given PID file. The file contains the process ID of the script. The PID file is removed when the script exits. Before starting, the script checks if the PID file already exists. If it does not, then the script creates and writes its own PID to it. If it does, then the script checks the following: if the file contains a PID and a process is running with that PID, then the script dies; or, if there is no process running with that PID, then the script overwrites the file with its own PID and starts; else, if the file contains no PID, then the script dies.
283 284 285 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 283 def pid @pid end |
#port ⇒ Object
short form: -P; type: int Port number to use for connection.
288 289 290 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 288 def port @port end |
#print ⇒ Object
Print queries that will resolve differences. If you don’t trust mk-table-sync, or just want to see what it will do, this is a good way to be safe. These queries are valid SQL and you can run them yourself if you want to sync the tables manually.
293 294 295 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 293 def print @print end |
#recursion_method ⇒ Object
type: string Preferred recursion method used to find slaves. Possible methods are: METHOD # USES
================
processlist SHOW PROCESSLIST hosts # SHOW SLAVE HOSTS The processlist method is preferred because SHOW SLAVE HOSTS is not reliable. However, the hosts method is required if the server uses a non-standard port (not 3306). Usually mk-table-sync does the right thing and finds the slaves, but you may give a preferred method and it will be used first. If it doesn’t find any slaves, the other methods will be tried.
304 305 306 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 304 def recursion_method @recursion_method end |
#replace ⇒ Object
Write all INSERT and UPDATE statements as REPLACE. This is automatically switched on as needed when there are unique index violations.
309 310 311 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 309 def replace @replace end |
#replicate ⇒ Object
type: string Sync tables listed as different in this table. Specifies that mk-table-sync should examine the specified table to find data that differs. The table is exactly the same as the argument of the same name to mk-table-checksum. That is, it contains records of which tables (and ranges of values) differ between the master and slave. For each table and range of values that shows differences between the master and slave, mk-table-checksum will sync that table, with the appropriate WHERE clause, to its master. This automatically sets –wait to 60 and causes changes to be made on the master instead of the slave. If –sync-to-master is specified, the tool will assume the server you specified is the slave, and connect to the master as usual to sync. Otherwise, it will try to use SHOW PROCESSLIST to find slaves of the server you specified. If it is unable to find any slaves via SHOW PROCESSLIST, it will inspect SHOW SLAVE HOSTS instead. You must configure each slave’s report-host, report-port and other options for this to work right. After finding slaves, it will inspect the specified table on each slave to find data that needs to be synced, and sync it. The tool examines the master’s copy of the table first, assuming that the master is potentially a slave as well. Any table that shows differences there will NOT be synced on the slave(s). For example, suppose your replication is set up as A->B, B->C, B->D. Suppose you use this argument and specify server B. The tool will examine server B’s copy of the table. If it looks like server B’s data in table test.tbl1 is different from server A’s copy, the tool will not sync that table on servers C and D.
320 321 322 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 320 def replicate @replicate end |
#set_vars ⇒ Object
type: string; default: wait_timeout=10000 Set these MySQL variables. Immediately after connecting to MySQL, this string will be appended to SET and executed.
325 326 327 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 325 def set_vars @set_vars end |
#socket ⇒ Object
short form: -S; type: string Socket file to use for connection.
330 331 332 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 330 def socket @socket end |
#sync_to_master ⇒ Object
Treat the DSN as a slave and sync it to its master. Treat the server you specified as a slave. Inspect SHOW SLAVE STATUS, connect to the server’s master, and treat the master as the source and the slave as the destination. Causes changes to be made on the master. Sets –wait to 60 by default, sets –lock to 1 by default, and disables –[no]transaction by default. See also –replicate, which changes this option’s behavior.
335 336 337 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 335 def sync_to_master @sync_to_master end |
#tables ⇒ Object
short form: -t; type: hash Sync only this comma-separated list of tables. Table names may be qualified with the database name.
341 342 343 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 341 def tables @tables end |
#timeout_ok ⇒ Object
Keep going if –wait fails. If you specify –wait and the slave doesn’t catch up to the master’s position before the wait times out, the default behavior is to abort. This option makes the tool keep going anyway. Warning: if you are trying to get a consistent comparison between the two servers, you probably don’t want to keep going after a timeout.
346 347 348 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 346 def timeout_ok @timeout_ok end |
#transaction ⇒ Object
Use transactions instead of LOCK TABLES. The granularity of beginning and committing transactions is controlled by –lock. This is enabled by default, but since –lock is disabled by default, it has no effect. Most options that enable locking also disable transactions by default, so if you want to use transactional locking (via LOCK IN SHARE MODE and FOR UPDATE, you must specify –transaction explicitly. If you don’t specify –transaction explicitly mk-table-sync will decide on a per-table basis whether to use transactions or table locks. It currently uses transactions on InnoDB tables, and table locks on all others. If –no-transaction is specified, then mk-table-sync will not use transactions at all (not even for InnoDB tables) and locking is controlled by –lock. When enabled, either explicitly or implicitly, the transaction isolation level is set REPEATABLE READ and transactions are started WITH CONSISTENT SNAPSHOT.
355 356 357 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 355 def transaction @transaction end |
#trim ⇒ Object
TRIM() VARCHAR columns in BIT_XOR and ACCUM modes. Helps when comparing MySQL 4.1 to >= 5.0. This is useful when you don’t care about the trailing space differences between MySQL versions which vary in their handling of trailing spaces. MySQL 5.0 and later all retain trailing spaces in VARCHAR, while previous versions would remove them.
360 361 362 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 360 def trim @trim end |
#unique_checks ⇒ Object
default: yes Enable unique key checks (SET UNIQUE_CHECKS=1). Specifying –no-unique-checks will SET UNIQUE_CHECKS=0.
366 367 368 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 366 def unique_checks @unique_checks end |
#user ⇒ Object
short form: -u; type: string User for login if not current user.
371 372 373 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 371 def user @user end |
#verbose ⇒ Object
short form: -v; cumulative: yes Print results of sync operations. See OUTPUT for more details about the output.
377 378 379 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 377 def verbose @verbose end |
#version ⇒ Object
Show version and exit.
381 382 383 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 381 def version @version end |
#wait ⇒ Object
short form: -w; type: time How long to wait for slaves to catch up to their master. Make the master wait for the slave to catch up in replication before comparing the tables. The value is the number of seconds to wait before timing out (see also –timeout-ok). Sets –lock to 1 and –[no]transaction to 0 by default. If you see an error such as the following, MASTER_POS_WAIT returned -1 It means the timeout was exceeded and you need to increase it. The default value of this option is influenced by other options. To see what value is in effect, run with –help. To disable waiting entirely (except for locks), specify –wait 0. This helps when the slave is lagging on tables that are not being synced.
391 392 393 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 391 def wait @wait end |
#where ⇒ Object
type: string WHERE clause to restrict syncing to part of the table.
396 397 398 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 396 def where @where end |
#zero_chunk ⇒ Object
default: yes Add a chunk for rows with zero or zero-equivalent values. The only has an effect when –chunk-size is specified. The purpose of the zero chunk is to capture a potentially large number of zero values that would imbalance the size of the first chunk. For example, if a lot of negative numbers were inserted into an unsigned integer column causing them to be stored as zeros, then these zero values are captured by the zero chunk instead of the first chunk and all its non-zero values.
401 402 403 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 401 def zero_chunk @zero_chunk end |
Instance Method Details
#start(options = nil) ⇒ Object
Execute the command
417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 |
# File 'lib/maatkit-ruby/mk-table-sync.rb', line 417 def start( = nil) tmp = Tempfile.new('tmp') command = option_string() + .to_s + " 2> " + tmp.path success = system(command) if success begin while (line = tmp.readline) line.chomp selected_string = line end rescue EOFError tmp.close end return selected_string else tmp.close! return success end end |