Module: Lhm

Extended by:
Lhm, Throttler
Included in:
Lhm
Defined in:
lib/lhm.rb,
lib/lhm/table.rb,
lib/lhm/chunker.rb,
lib/lhm/command.rb,
lib/lhm/invoker.rb,
lib/lhm/printer.rb,
lib/lhm/version.rb,
lib/lhm/migrator.rb,
lib/lhm/entangler.rb,
lib/lhm/migration.rb,
lib/lhm/sql_retry.rb,
lib/lhm/throttler.rb,
lib/lhm/timestamp.rb,
lib/lhm/connection.rb,
lib/lhm/sql_helper.rb,
lib/lhm/table_name.rb,
lib/lhm/chunk_finder.rb,
lib/lhm/chunk_insert.rb,
lib/lhm/intersection.rb,
lib/lhm/test_support.rb,
lib/lhm/throttler/time.rb,
lib/lhm/atomic_switcher.rb,
lib/lhm/cleanup/current.rb,
lib/lhm/locked_switcher.rb,
lib/lhm/proxysql_helper.rb,
lib/lhm/id_set_chunk_finder.rb,
lib/lhm/id_set_chunk_insert.rb,
lib/lhm/throttler/slave_lag.rb,
lib/lhm/throttler/threads_running.rb

Overview

Copyright © 2011 - 2013, SoundCloud Ltd., Rany Keddo, Tobias Bielohlawek, Tobias Schmidt

Defined Under Namespace

Modules: Cleanup, Command, Printer, ProxySQLHelper, SqlHelper, TestInvoker, TestMigrator, Throttler Classes: AtomicSwitcher, ChunkFinder, ChunkInsert, Chunker, Connection, Entangler, Error, IdSetChunkFinder, IdSetChunkInsert, Intersection, Invoker, LockedSwitcher, Migration, Migrator, SqlRetry, Table, TableName, Timestamp

Constant Summary collapse

DEFAULT_LOGGER_OPTIONS = 
{ level: Logger::INFO, file: STDOUT }
VERSION = 
'3.6.4'

Constants included from Throttler

Throttler::CLASSES

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Throttler

format_hosts, setup_throttler, throttler

Class Method Details

.execute_inline! ⇒ Object

Patch LHM to execute ALTER TABLE directly on original tables, without the online migration dance. This mode is designed for local/CI environments where we can speed things up by not invoking “real” LHM logic.



31
32
33
34
 
# File 'lib/lhm/test_support.rb', line 31

def self.execute_inline!
  Lhm::Migrator.prepend(TestMigrator)
  Lhm::Invoker.prepend(TestInvoker)
end

.logger ⇒ Object 



108
109
110
111
112
113
114
115
116
 
# File 'lib/lhm.rb', line 108

def self.logger
  @@logger ||=
    begin
      logger = Logger.new(DEFAULT_LOGGER_OPTIONS[:file])
      logger.level = DEFAULT_LOGGER_OPTIONS[:level]
      logger.formatter = nil
      logger
    end
end

.logger=(new_logger) ⇒ Object 



104
105
106
 
# File 'lib/lhm.rb', line 104

def self.logger=(new_logger)
  @@logger = new_logger
end

Instance Method Details

#change_table(table_name, options = {}) {|Migrator| ... } ⇒ Boolean

Alters a table with the changes described in the block

Options Hash (options):

  • :stride (Integer) —

    Size of a chunk (defaults to: 2,000)

  • :throttle (Integer) —

    Time to wait between chunks in milliseconds (defaults to: 100)

  • :start (Integer) —

    Primary Key position at which to start copying chunks

  • :limit (Integer) —

    Primary Key position at which to stop copying chunks

  • :atomic_switch (Boolean) —

    Use atomic switch to rename tables (defaults to: true) If using a version of mysql affected by atomic switch bug, LHM forces user to set this option (see SqlHelper#supports_atomic_switch?)

  • :reconnect_with_consistent_host (Boolean) —

    Active / Deactivate ProxySQL-aware reconnection procedure (default to: false)

Yields:

  • (Migrator) —

    Yielded Migrator object records the changes

Raises:

  • (Error) —

    Raises Lhm::Error in case of a error and aborts the migration



53
54
55
56
57
58
59
60
61
 
# File 'lib/lhm.rb', line 53

def change_table(table_name, options = {}, &block)
  with_flags(options) do
    origin = Table.parse(table_name, connection)
    invoker = Invoker.new(origin, connection)
    block.call(invoker.migrator)
    invoker.run(options)
    true
  end
end

#cleanup(run = false, options = {}) ⇒ Object

Cleanup tables and triggers

Options Hash (options):

  • :until (Time) —

    Filter to only remove tables up to specified time (defaults to: nil)



69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
 
# File 'lib/lhm.rb', line 69

def cleanup(run = false, options = {})
  lhm_tables = connection.select_values('show tables').select { |name| name =~ /^lhm(a|n)_/ }
  if options[:until]
    lhm_tables.select! do |table|
      table_date_time = Time.strptime(table, 'lhma_%Y_%m_%d_%H_%M_%S')
      table_date_time <= options[:until]
    end
  end

  lhm_triggers = connection.select_values('show triggers').collect do |trigger|
    trigger.respond_to?(:trigger) ? trigger.trigger : trigger
  end.select { |name| name =~ /^lhmt/ }

  drop_tables_and_triggers(run, lhm_triggers, lhm_tables)
end

#cleanup_current_run(run, table_name, options = {}) ⇒ Object 



85
86
87
 
# File 'lib/lhm.rb', line 85

def cleanup_current_run(run, table_name, options = {})
  Lhm::Cleanup::Current.new(run, table_name, connection, options).execute
end

#connection ⇒ Object

Returns DB connection (or initializes it if not created yet)



97
98
99
100
101
102
 
# File 'lib/lhm.rb', line 97

def connection
  @@connection ||= begin
                     raise 'Please call Lhm.setup' unless defined?(ActiveRecord)
                     @@connection = Connection.new(connection: ActiveRecord::Base.connection)
                   end
end

#setup(connection) ⇒ Object

Setups DB connection



92
93
94
 
# File 'lib/lhm.rb', line 92

def setup(connection)
  @@connection = Connection.new(connection: connection)
end