Method: Aws::SageMaker::Client#create_processing_job

Defined in:

lib/aws-sdk-sagemaker/client.rb

#create_processing_job(params = {}) ⇒ Types::CreateProcessingJobResponse

Creates a processing job.

Examples:

Request syntax with placeholder values

resp = client.create_processing_job({
  processing_inputs: [
    {
      input_name: "String", # required
      s3_input: { # required
        s3_uri: "S3Uri", # required
        local_path: "ProcessingLocalPath", # required
        s3_data_type: "ManifestFile", # required, accepts ManifestFile, S3Prefix
        s3_input_mode: "Pipe", # required, accepts Pipe, File
        s3_data_distribution_type: "FullyReplicated", # accepts FullyReplicated, ShardedByS3Key
        s3_compression_type: "None", # accepts None, Gzip
      },
    },
  ],
  processing_output_config: {
    outputs: [ # required
      {
        output_name: "String", # required
        s3_output: { # required
          s3_uri: "S3Uri", # required
          local_path: "ProcessingLocalPath", # required
          s3_upload_mode: "Continuous", # required, accepts Continuous, EndOfJob
        },
      },
    ],
    kms_key_id: "KmsKeyId",
  },
  processing_job_name: "ProcessingJobName", # required
  processing_resources: { # required
    cluster_config: { # required
      instance_count: 1, # required
      instance_type: "ml.t3.medium", # required, accepts ml.t3.medium, ml.t3.large, ml.t3.xlarge, ml.t3.2xlarge, ml.m4.xlarge, ml.m4.2xlarge, ml.m4.4xlarge, ml.m4.10xlarge, ml.m4.16xlarge, ml.c4.xlarge, ml.c4.2xlarge, ml.c4.4xlarge, ml.c4.8xlarge, ml.p2.xlarge, ml.p2.8xlarge, ml.p2.16xlarge, ml.p3.2xlarge, ml.p3.8xlarge, ml.p3.16xlarge, ml.c5.xlarge, ml.c5.2xlarge, ml.c5.4xlarge, ml.c5.9xlarge, ml.c5.18xlarge, ml.m5.large, ml.m5.xlarge, ml.m5.2xlarge, ml.m5.4xlarge, ml.m5.12xlarge, ml.m5.24xlarge, ml.r5.large, ml.r5.xlarge, ml.r5.2xlarge, ml.r5.4xlarge, ml.r5.8xlarge, ml.r5.12xlarge, ml.r5.16xlarge, ml.r5.24xlarge
      volume_size_in_gb: 1, # required
      volume_kms_key_id: "KmsKeyId",
    },
  },
  stopping_condition: {
    max_runtime_in_seconds: 1, # required
  },
  app_specification: { # required
    image_uri: "ImageUri", # required
    container_entrypoint: ["ContainerEntrypointString"],
    container_arguments: ["ContainerArgument"],
  },
  environment: {
    "ProcessingEnvironmentKey" => "ProcessingEnvironmentValue",
  },
  network_config: {
    enable_network_isolation: false,
    vpc_config: {
      security_group_ids: ["SecurityGroupId"], # required
      subnets: ["SubnetId"], # required
    },
  },
  role_arn: "RoleArn", # required
  tags: [
    {
      key: "TagKey", # required
      value: "TagValue", # required
    },
  ],
  experiment_config: {
    experiment_name: "ExperimentConfigName",
    trial_name: "ExperimentConfigName",
    trial_component_display_name: "ExperimentConfigName",
  },
})

Response structure

resp.processing_job_arn #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :processing_inputs (Array<Types::ProcessingInput>) —

  For each input, data is downloaded from S3 into the processing container before the processing job begins running if “S3InputMode” is set to File.

• :processing_output_config (Types::ProcessingOutputConfig) —

  Output configuration for the processing job.

• :processing_job_name (required, String) —

  The name of the processing job. The name must be unique within an AWS Region in the AWS account.

• :processing_resources (required, Types::ProcessingResources) —

  Identifies the resources, ML compute instances, and ML storage volumes to deploy for a processing job. In distributed training, you specify more than one instance.

• :stopping_condition (Types::ProcessingStoppingCondition) —

  The time limit for how long the processing job is allowed to run.

• :app_specification (required, Types::AppSpecification) —

  Configures the processing job to run a specified Docker container image.

• :environment (Hash<String,String>) —

  Sets the environment variables in the Docker container.

• :network_config (Types::NetworkConfig) —

  Networking options for a processing job.

• :role_arn (required, String) —

  The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker can assume to perform tasks on your behalf.

• :tags (Array<Types::Tag>) — default: Optional —

  An array of key-value pairs. For more information, see

  Using Cost Allocation Tags][1

  in the *AWS Billing and Cost

  Management User Guide*.

  [1]: docs-aws.amazon.com/awsaccountbilling/latest/aboutv2/cost-alloc-tags.html#allocation-whatURL

• :experiment_config (Types::ExperimentConfig) —

  Configuration for the experiment.

Returns:

• (Types::CreateProcessingJobResponse) —

  Returns a response object which responds to the following methods:

  • #processing_job_arn => String

See Also:

• AWS API Documentation

# File 'lib/aws-sdk-sagemaker/client.rb', line 2887

def create_processing_job(params = {}, options = {})
  req = build_request(:create_processing_job, params)
  req.send_request(options)
end