Module: Karafka::Connection::BatchDelegator
- Defined in:
- lib/karafka/connection/batch_delegator.rb
Overview
Class that delegates processing of batch received messages for which we listen to a proper processor
Class Method Summary collapse
-
.call(group_id, kafka_batch) ⇒ Object
Delegates messages (does something with them) It will either schedule or run a proper processor action for messages.
Class Method Details
.call(group_id, kafka_batch) ⇒ Object
Note:
This should be looped to obtain a constant delegating of new messages
Delegates messages (does something with them) It will either schedule or run a proper processor action for messages
14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
# File 'lib/karafka/connection/batch_delegator.rb', line 14 def call(group_id, kafka_batch) topic = Persistence::Topics.fetch(group_id, kafka_batch.topic) consumer = Persistence::Consumers.fetch(topic, kafka_batch.partition) Karafka.monitor.instrument( 'connection.batch_delegator.call', caller: self, consumer: consumer, kafka_batch: kafka_batch ) do # Due to how ruby-kafka is built, we have the metadata that is stored on the batch # level only available for batch consuming consumer. = Params::Builders::BatchMetadata.from_kafka_batch( kafka_batch, topic ) = kafka_batch. # Depending on a case (persisted or not) we might use new consumer instance per # each batch, or use the same one for all of them (for implementing buffering, etc.) if topic.batch_consuming consumer.params_batch = Params::Builders::ParamsBatch.( , topic ) consumer.call else .each do || consumer.params_batch = Params::Builders::ParamsBatch.( [], topic ) consumer.call end end end end |