Class: Aws::Bedrock::Types::HumanEvaluationConfig

Inherits:
Struct
  • Object
show all
Includes:
Structure
Defined in:
lib/aws-sdk-bedrock/types.rb

Overview

Specifies the custom metrics, how tasks will be rated, the flow definition ARN, and your custom prompt datasets. Model evaluation jobs use human workers only support the use of custom prompt datasets. To learn more about custom prompt datasets and the required format, see [Custom prompt datasets].

When you create custom metrics in ‘HumanEvaluationCustomMetric` you must specify the metric’s ‘name`. The list of `names` specified in the `HumanEvaluationCustomMetric` array, must match the `metricNames` array of strings specified in `EvaluationDatasetMetricConfig`. For example, if in the `HumanEvaluationCustomMetric` array your specified the names `“accuracy”, “toxicity”, “readability”` as custom metrics then the `metricNames` array would need to look like the following `[“accuracy”, “toxicity”, “readability”]` in `EvaluationDatasetMetricConfig`.

[1]: docs.aws.amazon.com/bedrock/latest/userguide/model-evaluation-prompt-datasets-custom.html

See Also:

Constant Summary collapse

SENSITIVE = 
[]

Instance Attribute Summary collapse

Instance Attribute Details

#custom_metrics ⇒ Array<Types::HumanEvaluationCustomMetric>

A ‘HumanEvaluationCustomMetric` object. It contains the names the metrics, how the metrics are to be evaluated, an optional description.

Returns:



4943
4944
4945
4946
4947
4948
4949
 
# File 'lib/aws-sdk-bedrock/types.rb', line 4943

class HumanEvaluationConfig < Struct.new(
  :human_workflow_config,
  :custom_metrics,
  :dataset_metric_configs)
  SENSITIVE = []
  include Aws::Structure
end

#dataset_metric_configs ⇒ Array<Types::EvaluationDatasetMetricConfig>

Use to specify the metrics, task, and prompt dataset to be used in your model evaluation job.

Returns:



4943
4944
4945
4946
4947
4948
4949
 
# File 'lib/aws-sdk-bedrock/types.rb', line 4943

class HumanEvaluationConfig < Struct.new(
  :human_workflow_config,
  :custom_metrics,
  :dataset_metric_configs)
  SENSITIVE = []
  include Aws::Structure
end

#human_workflow_config ⇒ Types::HumanWorkflowConfig

The parameters of the human workflow.

Returns:



4943
4944
4945
4946
4947
4948
4949
 
# File 'lib/aws-sdk-bedrock/types.rb', line 4943

class HumanEvaluationConfig < Struct.new(
  :human_workflow_config,
  :custom_metrics,
  :dataset_metric_configs)
  SENSITIVE = []
  include Aws::Structure
end