Class: OpenAI::Resources::Evals

Inherits:
Object
  • Object
show all
Defined in:
lib/openai/resources/evals.rb,
lib/openai/resources/evals/runs.rb,
lib/openai/resources/evals/runs/output_items.rb,
sig/openai/resources/evals.rbs,
sig/openai/resources/evals/runs.rbs,
sig/openai/resources/evals/runs/output_items.rbs

Overview

Manage and run evals in the OpenAI platform.

Defined Under Namespace

Classes: Runs

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(client:) ⇒ Evals

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Evals.

Parameters:



177
178
179
180
 
# File 'lib/openai/resources/evals.rb', line 177

def initialize(client:)
  @client = client
  @runs = OpenAI::Resources::Evals::Runs.new(client: client)
end

Instance Attribute Details

#runs ⇒ OpenAI::Resources::Evals::Runs (readonly)

Manage and run evals in the OpenAI platform.

Returns:



9
10
11
 
# File 'lib/openai/resources/evals.rb', line 9

def runs
  @runs
end

Instance Method Details

#create(data_source_config:, testing_criteria:, metadata: nil, name: nil, request_options: {}) ⇒ OpenAI::Models::EvalCreateResponse

Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the data used in the evaluation. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the Evals guide.

Parameters:

Returns:

See Also:



46
47
48
49
50
51
52
53
54
55
56
 
# File 'lib/openai/resources/evals.rb', line 46

def create(params)
  parsed, options = OpenAI::EvalCreateParams.dump_request(params)
  @client.request(
    method: :post,
    path: "evals",
    body: parsed,
    model: OpenAI::Models::EvalCreateResponse,
    security: {bearer_auth: true},
    options: options
  )
end

#delete(eval_id, request_options: {}) ⇒ OpenAI::Models::EvalDeleteResponse

Delete an evaluation.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to delete.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



164
165
166
167
168
169
170
171
172
 
# File 'lib/openai/resources/evals.rb', line 164

def delete(eval_id, params = {})
  @client.request(
    method: :delete,
    path: ["evals/%1$s", eval_id],
    model: OpenAI::Models::EvalDeleteResponse,
    security: {bearer_auth: true},
    options: params[:request_options]
  )
end

#list(after: nil, limit: nil, order: nil, order_by: nil, request_options: {}) ⇒ OpenAI::Internal::CursorPage<OpenAI::Models::EvalListResponse>

List evaluations for a project.

Parameters:

  • after (String) —

    Identifier for the last eval from the previous pagination request.

  • limit (Integer) —

    Number of evals to retrieve.

  • order (Symbol, OpenAI::Models::EvalListParams::Order) —

    Sort order for evals by timestamp. Use asc for ascending order or desc for descending order.

  • order_by (Symbol, OpenAI::Models::EvalListParams::OrderBy) —

    Evals can be ordered by creation time or last updated time. Use created_at for creation time or updated_at for last updated time.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



138
139
140
141
142
143
144
145
146
147
148
149
150
 
# File 'lib/openai/resources/evals.rb', line 138

def list(params = {})
  parsed, options = OpenAI::EvalListParams.dump_request(params)
  query = OpenAI::Internal::Util.encode_query_params(parsed)
  @client.request(
    method: :get,
    path: "evals",
    query: query,
    page: OpenAI::Internal::CursorPage,
    model: OpenAI::Models::EvalListResponse,
    security: {bearer_auth: true},
    options: options
  )
end

#retrieve(eval_id, request_options: {}) ⇒ OpenAI::Models::EvalRetrieveResponse

Get an evaluation by ID.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to retrieve.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



70
71
72
73
74
75
76
77
78
 
# File 'lib/openai/resources/evals.rb', line 70

def retrieve(eval_id, params = {})
  @client.request(
    method: :get,
    path: ["evals/%1$s", eval_id],
    model: OpenAI::Models::EvalRetrieveResponse,
    security: {bearer_auth: true},
    options: params[:request_options]
  )
end

#update(eval_id, metadata: nil, name: nil, request_options: {}) ⇒ OpenAI::Models::EvalUpdateResponse

Update certain properties of an evaluation.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to update.

  • metadata (Hash{Symbol=>String}, nil) —

    Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard.

    Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

  • name (String) —

    Rename the evaluation.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



103
104
105
106
107
108
109
110
111
112
113
 
# File 'lib/openai/resources/evals.rb', line 103

def update(eval_id, params = {})
  parsed, options = OpenAI::EvalUpdateParams.dump_request(params)
  @client.request(
    method: :post,
    path: ["evals/%1$s", eval_id],
    body: parsed,
    model: OpenAI::Models::EvalUpdateResponse,
    security: {bearer_auth: true},
    options: options
  )
end