Class: Aws::Bedrock::Types::CustomMetricDefinition

Inherits:

Struct

• Object
• Struct
• Aws::Bedrock::Types::CustomMetricDefinition

Includes:

Structure

Defined in:

lib/aws-sdk-bedrock/types.rb

Overview

The definition of a custom metric for use in an Amazon Bedrock evaluation job. A custom metric definition includes a metric name, prompt (instructions) and optionally, a rating scale. Your prompt must include a task description and input variables. The required input variables are different for model-as-a-judge and RAG evaluations.

For more information about how to define a custom metric in Amazon Bedrock, see [Create a prompt for a custom metrics (LLM-as-a-judge model evaluations)] and [Create a prompt for a custom metrics (RAG evaluations)].

[1]: docs.aws.amazon.com/bedrock/latest/userguide/model-evaluation-custom-metrics-prompt-formats.html [2]: docs.aws.amazon.com/bedrock/latest/userguide/kb-evaluation-custom-metrics-prompt-formats.html

See Also:

• AWS API Documentation

Constant Summary

SENSITIVE =

[:name]

Instance Attribute Summary

• #instructions ⇒ String

  The prompt for a custom metric that instructs the evaluator model how to rate the model or RAG source under evaluation.

• #name ⇒ String

  The name for a custom metric.

• #rating_scale ⇒ Array<Types::RatingScaleItem>

  Defines the rating scale to be used for a custom metric.

Instance Attribute Details

#instructions ⇒ String

The prompt for a custom metric that instructs the evaluator model how to rate the model or RAG source under evaluation.

Returns:

• (String)

# File 'lib/aws-sdk-bedrock/types.rb', line 3777

class CustomMetricDefinition < Struct.new(
  :name,
  :instructions,
  :rating_scale)
  SENSITIVE = [:name]
  include Aws::Structure
end

#name ⇒ String

The name for a custom metric. Names must be unique in your Amazon Web Services region.

Returns:

• (String)

# File 'lib/aws-sdk-bedrock/types.rb', line 3777

class CustomMetricDefinition < Struct.new(
  :name,
  :instructions,
  :rating_scale)
  SENSITIVE = [:name]
  include Aws::Structure
end

#rating_scale ⇒ Array<Types::RatingScaleItem>

Defines the rating scale to be used for a custom metric. We recommend that you always define a ratings scale when creating a custom metric. If you don’t define a scale, Amazon Bedrock won’t be able to visually display the results of the evaluation in the console or calculate average values of numerical scores. For more information on specifying a rating scale, see [Specifying an output schema (rating scale)].

[1]: docs.aws.amazon.com/bedrock/latest/userguide/model-evaluation-custom-metrics-prompt-formats.html#model-evaluation-custom-metrics-prompt-formats-schema

Returns:

• (Array<Types::RatingScaleItem>)

# File 'lib/aws-sdk-bedrock/types.rb', line 3777

class CustomMetricDefinition < Struct.new(
  :name,
  :instructions,
  :rating_scale)
  SENSITIVE = [:name]
  include Aws::Structure
end