Class: OpenAI::Resources::Evals::Runs

Inherits:
Object
  • Object
show all
Defined in:
lib/openai/resources/evals/runs.rb,
lib/openai/resources/evals/runs/output_items.rb,
sig/openai/resources/evals/runs.rbs,
sig/openai/resources/evals/runs/output_items.rbs

Overview

Manage and run evals in the OpenAI platform.

Defined Under Namespace

Classes: OutputItems

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(client:) ⇒ Runs

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Runs.

Parameters:



185
186
187
188
 
# File 'lib/openai/resources/evals/runs.rb', line 185

def initialize(client:)
  @client = client
  @output_items = OpenAI::Resources::Evals::Runs::OutputItems.new(client: client)
end

Instance Attribute Details

#output_items ⇒ OpenAI::Resources::Evals::Runs::OutputItems (readonly)

Manage and run evals in the OpenAI platform.

Returns:



10
11
12
 
# File 'lib/openai/resources/evals/runs.rb', line 10

def output_items
  @output_items
end

Instance Method Details

#cancel(run_id, eval_id:, request_options: {}) ⇒ OpenAI::Models::Evals::RunCancelResponse

Cancel an ongoing evaluation run.

Parameters:

  • run_id (String) —

    The ID of the run to cancel.

  • eval_id (String) —

    The ID of the evaluation whose run you want to cancel.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



167
168
169
170
171
172
173
174
175
176
177
178
179
180
 
# File 'lib/openai/resources/evals/runs.rb', line 167

def cancel(run_id, params)
  parsed, options = OpenAI::Evals::RunCancelParams.dump_request(params)
  eval_id = parsed.delete(:eval_id) do
    raise ArgumentError.new("missing required path argument #{_1}")
  end

  @client.request(
    method: :post,
    path: ["evals/%1$s/runs/%2$s", eval_id, run_id],
    model: OpenAI::Models::Evals::RunCancelResponse,
    security: {bearer_auth: true},
    options: options
  )
end

#create(eval_id, data_source:, metadata: nil, name: nil, request_options: {}) ⇒ OpenAI::Models::Evals::RunCreateResponse

Kicks off a new run for a given evaluation, specifying the data source, and what model configuration to use to test. The datasource will be validated against the schema specified in the config of the evaluation.

Parameters:

Returns:

See Also:



40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/openai/resources/evals/runs.rb', line 40

def create(eval_id, params)
  parsed, options = OpenAI::Evals::RunCreateParams.dump_request(params)
  @client.request(
    method: :post,
    path: ["evals/%1$s/runs", eval_id],
    body: parsed,
    model: OpenAI::Models::Evals::RunCreateResponse,
    security: {bearer_auth: true},
    options: options
  )
end

#delete(run_id, eval_id:, request_options: {}) ⇒ OpenAI::Models::Evals::RunDeleteResponse

Delete an eval run.

Parameters:

  • run_id (String) —

    The ID of the run to delete.

  • eval_id (String) —

    The ID of the evaluation to delete the run from.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



137
138
139
140
141
142
143
144
145
146
147
148
149
150
 
# File 'lib/openai/resources/evals/runs.rb', line 137

def delete(run_id, params)
  parsed, options = OpenAI::Evals::RunDeleteParams.dump_request(params)
  eval_id = parsed.delete(:eval_id) do
    raise ArgumentError.new("missing required path argument #{_1}")
  end

  @client.request(
    method: :delete,
    path: ["evals/%1$s/runs/%2$s", eval_id, run_id],
    model: OpenAI::Models::Evals::RunDeleteResponse,
    security: {bearer_auth: true},
    options: options
  )
end

#list(eval_id, after: nil, limit: nil, order: nil, status: nil, request_options: {}) ⇒ OpenAI::Internal::CursorPage<OpenAI::Models::Evals::RunListResponse>

Get a list of runs for an evaluation.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to retrieve runs for.

  • after (String) —

    Identifier for the last run from the previous pagination request.

  • limit (Integer) —

    Number of runs to retrieve.

  • order (Symbol, OpenAI::Models::Evals::RunListParams::Order) —

    Sort order for runs by timestamp. Use asc for ascending order or desc for descending order. Defaults to asc.

  • status (Symbol, OpenAI::Models::Evals::RunListParams::Status) —

    Filter runs by status. One of queued | in_progress | failed | completed | canceled.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



108
109
110
111
112
113
114
115
116
117
118
119
120
 
# File 'lib/openai/resources/evals/runs.rb', line 108

def list(eval_id, params = {})
  parsed, options = OpenAI::Evals::RunListParams.dump_request(params)
  query = OpenAI::Internal::Util.encode_query_params(parsed)
  @client.request(
    method: :get,
    path: ["evals/%1$s/runs", eval_id],
    query: query,
    page: OpenAI::Internal::CursorPage,
    model: OpenAI::Models::Evals::RunListResponse,
    security: {bearer_auth: true},
    options: options
  )
end

#retrieve(run_id, eval_id:, request_options: {}) ⇒ OpenAI::Models::Evals::RunRetrieveResponse

Get an evaluation run by ID.

Parameters:

  • run_id (String) —

    The ID of the run to retrieve.

  • eval_id (String) —

    The ID of the evaluation to retrieve runs for.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



67
68
69
70
71
72
73
74
75
76
77
78
79
80
 
# File 'lib/openai/resources/evals/runs.rb', line 67

def retrieve(run_id, params)
  parsed, options = OpenAI::Evals::RunRetrieveParams.dump_request(params)
  eval_id = parsed.delete(:eval_id) do
    raise ArgumentError.new("missing required path argument #{_1}")
  end

  @client.request(
    method: :get,
    path: ["evals/%1$s/runs/%2$s", eval_id, run_id],
    model: OpenAI::Models::Evals::RunRetrieveResponse,
    security: {bearer_auth: true},
    options: options
  )
end