Class: Aws::SageMaker::Types::Channel
- Inherits:
-
Struct
- Object
- Struct
- Aws::SageMaker::Types::Channel
- Includes:
- Aws::Structure
- Defined in:
- lib/aws-sdk-sagemaker/types.rb
Overview
When making an API call, you may pass Channel data as a hash:
{
channel_name: "ChannelName", # required
data_source: { # required
s3_data_source: { # required
s3_data_type: "ManifestFile", # required, accepts ManifestFile, S3Prefix, AugmentedManifestFile
s3_uri: "S3Uri", # required
s3_data_distribution_type: "FullyReplicated", # accepts FullyReplicated, ShardedByS3Key
attribute_names: ["AttributeName"],
},
},
content_type: "ContentType",
compression_type: "None", # accepts None, Gzip
record_wrapper_type: "None", # accepts None, RecordIO
input_mode: "Pipe", # accepts Pipe, File
shuffle_config: {
seed: 1, # required
},
}
A channel is a named input source that training algorithms can consume.
Instance Attribute Summary collapse
-
#channel_name ⇒ String
The name of the channel.
-
#compression_type ⇒ String
If training data is compressed, the compression type.
-
#content_type ⇒ String
The MIME type of the data.
-
#data_source ⇒ Types::DataSource
The location of the channel data.
-
#input_mode ⇒ String
(Optional) The input mode to use for the data channel in a training job.
-
#record_wrapper_type ⇒ String
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format.
-
#shuffle_config ⇒ Types::ShuffleConfig
A configuration for a shuffle option for input data in a channel.
Instance Attribute Details
#channel_name ⇒ String
The name of the channel.
656 657 658 659 660 661 662 663 664 665 |
# File 'lib/aws-sdk-sagemaker/types.rb', line 656 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) include Aws::Structure end |
#compression_type ⇒ String
If training data is compressed, the compression type. The default value is ‘None`. `CompressionType` is used only in Pipe input mode. In File mode, leave this field unset or set it to None.
656 657 658 659 660 661 662 663 664 665 |
# File 'lib/aws-sdk-sagemaker/types.rb', line 656 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) include Aws::Structure end |
#content_type ⇒ String
The MIME type of the data.
656 657 658 659 660 661 662 663 664 665 |
# File 'lib/aws-sdk-sagemaker/types.rb', line 656 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) include Aws::Structure end |
#data_source ⇒ Types::DataSource
The location of the channel data.
656 657 658 659 660 661 662 663 664 665 |
# File 'lib/aws-sdk-sagemaker/types.rb', line 656 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) include Aws::Structure end |
#input_mode ⇒ String
(Optional) The input mode to use for the data channel in a training job. If you don’t set a value for ‘InputMode`, Amazon SageMaker uses the value set for `TrainingInputMode`. Use this parameter to override the `TrainingInputMode` setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job’s general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use ‘File` input mode. To stream data directly from Amazon S3 to the container, choose `Pipe` input mode.
To use a model for incremental training, choose ‘File` input model.
656 657 658 659 660 661 662 663 664 665 |
# File 'lib/aws-sdk-sagemaker/types.rb', line 656 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) include Aws::Structure end |
#record_wrapper_type ⇒ String
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, Amazon SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don’t need to set this attribute. For more information, see [Create a Dataset Using RecordIO].
In File mode, leave this field unset or set it to None.
[1]: mxnet.incubator.apache.org/architecture/note_data_loading.html#data-format
656 657 658 659 660 661 662 663 664 665 |
# File 'lib/aws-sdk-sagemaker/types.rb', line 656 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) include Aws::Structure end |
#shuffle_config ⇒ Types::ShuffleConfig
A configuration for a shuffle option for input data in a channel. If you use ‘S3Prefix` for `S3DataType`, this shuffles the results of the S3 key prefix matches. If you use `ManifestFile`, the order of the S3 object references in the `ManifestFile` is shuffled. If you use `AugmentedManifestFile`, the order of the JSON lines in the `AugmentedManifestFile` is shuffled. The shuffling order is determined using the `Seed` value.
For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with ‘S3DataDistributionType` of `ShardedByS3Key`, the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch.
656 657 658 659 660 661 662 663 664 665 |
# File 'lib/aws-sdk-sagemaker/types.rb', line 656 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) include Aws::Structure end |