Class: Crawlr::Hooks

Inherits:
Object
  • Object
show all
Defined in:
lib/crawlr/hooks.rb

Overview

Event hook management system for scraping lifecycle customization.

The Hooks class provides a flexible event-driven system that allows users to register custom behavior at specific points during the scraping process. It supports multiple hooks per event and validates event names to ensure consistency across the framework.

Examples:

Basic hook registration

hooks = Crawlr::Hooks.new

hooks.register(:before_visit) do |url, headers|
  puts "About to visit: #{url}"
  headers['X-Custom'] = 'value'
end

Multiple hooks for the same event

hooks.register(:after_visit) do |url, response|
  log_response_time(url, response)
end

hooks.register(:after_visit) do |url, response|
  update_statistics(response.status)
end

Error handling hooks

hooks.register(:on_error) do |url, error|
  error_logger.warn("Failed to scrape #{url}: #{error.message}")
  notify_monitoring_system(url, error)
end

Author:

  • [Your Name]

Since:

  • 0.1.0

Constant Summary collapse

ALLOWED_EVENTS =

Supported lifecycle events for hook registration

  • :before_visit - Triggered before making HTTP request
  • :after_visit - Triggered after receiving HTTP response
  • :on_error - Triggered when an error occurs during scraping

Returns:

  • (Array<Symbol>) —

    Array of valid event names

Since:

  • 0.1.0

%i[before_visit after_visit on_error].freeze

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Hooks

Initializes a new Hooks instance

Creates an empty hook registry with auto-vivifying arrays for each event type.

Examples:

hooks = Crawlr::Hooks.new

Since:

  • 0.1.0



51
52
53
 
# File 'lib/crawlr/hooks.rb', line 51

def initialize
  @hooks = Hash.new { |h, k| h[k] = [] }
end

Instance Method Details

#clear(event = nil) ⇒ void

This method returns an undefined value.

Clears registered hooks for all events or a specific event

Useful for testing, resetting hook configuration, or dynamically changing hook behavior during scraping sessions.

Examples:

Clear all hooks

hooks.clear

Clear hooks for specific event

hooks.clear(:before_visit)

Clear error hooks only

hooks.clear(:on_error)

Parameters:

  • event (Symbol, nil) (defaults to: nil) —

    Specific event to clear, or nil to clear all

Since:

  • 0.1.0



153
154
155
156
157
158
159
 
# File 'lib/crawlr/hooks.rb', line 153

def clear(event = nil)
  if event
    @hooks[event].clear
  else
    @hooks.clear
  end
end

#register(event, &block) {|args| ... } ⇒ void

This method returns an undefined value.

Registers a hook for a specific scraping lifecycle event

Hooks are executed in the order they were registered. Multiple hooks can be registered for the same event, and all will be executed when the event is triggered.

Examples:

Before visit hook for request modification

register(:before_visit) do |url, headers|
  headers['User-Agent'] = 'Custom Bot 1.0'
  headers['Authorization'] = get_auth_token(url)
end

After visit hook for response processing

register(:after_visit) do |url, response|
  response_time = response.headers['X-Response-Time']
  metrics.record_response_time(url, response_time)
end

Error handling hook

register(:on_error) do |url, error|
  if error.is_a?(Timeout::Error)
    retry_queue.add(url, delay: 30)
  end
end

Parameters:

  • event (Symbol) —

    The lifecycle event to hook into

  • block (Proc) —

    The block to execute when the event occurs

Yield Parameters:

  • args (Array) —

    Event-specific arguments passed to the hook

Raises:

  • (ArgumentError) —

    When the event is not in ALLOWED_EVENTS

  • (ArgumentError) —

    When no block is provided

Since:

  • 0.1.0



86
87
88
89
90
91
 
# File 'lib/crawlr/hooks.rb', line 86

def register(event, &block)
  raise ArgumentError, "Invalid event #{event}" unless ALLOWED_EVENTS.include?(event)
  raise ArgumentError, "Block required" unless block

  @hooks[event] << block
end

#stats ⇒ Hash<Symbol, Object>

Returns statistics about registered hooks

Provides metrics about hook registration for monitoring, debugging, and ensuring expected hooks are properly configured.

Examples:

stats = hooks.stats
puts "Total hooks: #{stats[:total_hooks]}"
puts "Before visit hooks: #{stats[:per_event][:before_visit]}"
puts "Error hooks: #{stats[:per_event][:on_error]}"

Parameters:

  • return (Hash) —

    a customizable set of options

Returns:

  • (Hash<Symbol, Object>) —

    Statistics hash containing hook metrics

Since:

  • 0.1.0



132
133
134
135
 
# File 'lib/crawlr/hooks.rb', line 132

def stats
  grouped = @hooks.transform_values(&:size)
  { total_hooks: @hooks.values.flatten.size, per_event: grouped }
end

#trigger(event, *args) ⇒ void

This method returns an undefined value.

Triggers all registered hooks for a specific event

Executes hooks in the order they were registered. If any hook raises an exception, it will be propagated and may prevent subsequent hooks from executing.

Examples:

Trigger before_visit hooks

trigger(:before_visit, 'https://example.com', headers_hash)

Trigger after_visit hooks

trigger(:after_visit, 'https://example.com', response_object)

Trigger error hooks

trigger(:on_error, 'https://example.com', exception_object)

Parameters:

  • event (Symbol) —

    The event to trigger

  • args (Array) —

    Variable arguments to pass to the hook blocks

Raises:

  • (ArgumentError) —

    When the event is not in ALLOWED_EVENTS

Since:

  • 0.1.0



112
113
114
115
116
 
# File 'lib/crawlr/hooks.rb', line 112

def trigger(event, *args)
  raise ArgumentError, "Invalid event #{event}" unless ALLOWED_EVENTS.include?(event)

  @hooks[event].each { |blk| blk.call(*args) }
end