Class: Observ::DatasetRunnerService

Inherits:
Object
  • Object
show all
Defined in:
app/services/observ/dataset_runner_service.rb

Overview

Service responsible for executing dataset evaluations

This service runs an agent against all items in a dataset run, creating traces for each execution and tracking results.

Usage:

run = DatasetRun.find(1)
DatasetRunnerService.new(run).call

The service:

  • Updates run status to :running at start
  • Processes each dataset item through the AgentExecutorService
  • Creates a session and trace for each item execution
  • Records errors for failed items
  • Updates metrics after completion
  • Sets final status to :completed or :failed

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(dataset_run) ⇒ DatasetRunnerService

Returns a new instance of DatasetRunnerService.



24
25
26
27
 
# File 'app/services/observ/dataset_runner_service.rb', line 24

def initialize(dataset_run)
  @dataset_run = dataset_run
  @dataset = dataset_run.dataset
end

Instance Attribute Details

#dataset ⇒ Object (readonly)

Returns the value of attribute dataset.



22
23
24
 
# File 'app/services/observ/dataset_runner_service.rb', line 22

def dataset
  @dataset
end

#dataset_run ⇒ Object (readonly)

Returns the value of attribute dataset_run.



22
23
24
 
# File 'app/services/observ/dataset_runner_service.rb', line 22

def dataset_run
  @dataset_run
end

Instance Method Details

#call ⇒ Object 



29
30
31
32
33
34
35
36
37
38
39
 
# File 'app/services/observ/dataset_runner_service.rb', line 29

def call
  dataset_run.update!(status: :running)

  process_all_items

  dataset_run.update_metrics!
  determine_final_status
rescue StandardError => e
  handle_run_failure(e)
  raise
end