Module: Karafka::Pro::Cleaner

Defined in:

lib/karafka/pro/cleaner.rb,
lib/karafka/pro/cleaner/errors.rb,
lib/karafka/pro/cleaner/messages/message.rb,
lib/karafka/pro/cleaner/messages/messages.rb

Overview

Feature that introduces a granular memory management for each message and messages iterator

It allows for better resource allocation by providing an API to clear payload and raw payload from a message after those are no longer needed but before whole messages are freed and removed by Ruby GC.

This can be useful when processing bigger batches or bigger messages one after another and wanting not to have all of the data loaded into memory.

Can yield significant memory savings (up to 80%).

Defined Under Namespace

Modules: Errors, Messages

Class Method Summary

• .post_setup(_config) ⇒ Object
• .pre_setup(_config) ⇒ Object

Class Method Details

.post_setup(_config) ⇒ Object

Parameters:

• _config (Karafka::Core::Configurable::Node) —

  root node config

# File 'lib/karafka/pro/cleaner.rb', line 35

def post_setup(_config)
  true
end

.pre_setup(_config) ⇒ Object

Parameters:

• _config (Karafka::Core::Configurable::Node) —

  root node config

# File 'lib/karafka/pro/cleaner.rb', line 29

def pre_setup(_config)
  ::Karafka::Messages::Message.prepend(Messages::Message)
  ::Karafka::Messages::Messages.prepend(Messages::Messages)
end