Class: Google::Cloud::Dataproc::V1::OrderedJob

Inherits:
Object
  • Object
show all
Extended by:
Protobuf::MessageExts::ClassMethods
Includes:
Protobuf::MessageExts
Defined in:
proto_docs/google/cloud/dataproc/v1/workflow_templates.rb

Overview

A job executed by the workflow.

Defined Under Namespace

Classes: LabelsEntry

Instance Attribute Summary collapse

Instance Attribute Details

Returns Optional. Job is a Flink job.

Note: The following fields are mutually exclusive: flink_job, hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_r_job, spark_sql_job, presto_job, trino_job. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Dataproc::V1::FlinkJob) —

    Optional. Job is a Flink job.

    Note: The following fields are mutually exclusive: flink_job, hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_r_job, spark_sql_job, presto_job, trino_job. If a field in that set is populated, all other fields in the set will automatically be cleared.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#hadoop_job ⇒ ::Google::Cloud::Dataproc::V1::HadoopJob

Returns Optional. Job is a Hadoop job.

Note: The following fields are mutually exclusive: hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_r_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Dataproc::V1::HadoopJob) —

    Optional. Job is a Hadoop job.

    Note: The following fields are mutually exclusive: hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_r_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#hive_job ⇒ ::Google::Cloud::Dataproc::V1::HiveJob

Returns Optional. Job is a Hive job.

Note: The following fields are mutually exclusive: hive_job, hadoop_job, spark_job, pyspark_job, pig_job, spark_r_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Dataproc::V1::HiveJob) —

    Optional. Job is a Hive job.

    Note: The following fields are mutually exclusive: hive_job, hadoop_job, spark_job, pyspark_job, pig_job, spark_r_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#labels ⇒ ::Google::Protobuf::Map{::String => ::String}

Returns Optional. The labels to associate with this job.

Label keys must be between 1 and 63 characters long, and must conform to the following regular expression: [\p{Ll}\p{Lo}][\p{Ll}\p{Lo}\p{N}_-]{0,62}

Label values must be between 1 and 63 characters long, and must conform to the following regular expression: [\p{Ll}\p{Lo}\p{N}_-]{0,63}

No more than 32 labels can be associated with a given job.

Returns:

  • (::Google::Protobuf::Map{::String => ::String}) —

    Optional. The labels to associate with this job.

    Label keys must be between 1 and 63 characters long, and must conform to the following regular expression: [\p{Ll}\p{Lo}][\p{Ll}\p{Lo}\p{N}_-]{0,62}

    Label values must be between 1 and 63 characters long, and must conform to the following regular expression: [\p{Ll}\p{Lo}\p{N}_-]{0,63}

    No more than 32 labels can be associated with a given job.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#pig_job ⇒ ::Google::Cloud::Dataproc::V1::PigJob

Returns Optional. Job is a Pig job.

Note: The following fields are mutually exclusive: pig_job, hadoop_job, spark_job, pyspark_job, hive_job, spark_r_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Dataproc::V1::PigJob) —

    Optional. Job is a Pig job.

    Note: The following fields are mutually exclusive: pig_job, hadoop_job, spark_job, pyspark_job, hive_job, spark_r_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#prerequisite_step_ids ⇒ ::Array<::String>

Returns Optional. The optional list of prerequisite job step_ids. If not specified, the job will start at the beginning of workflow.

Returns:

  • (::Array<::String>) —

    Optional. The optional list of prerequisite job step_ids. If not specified, the job will start at the beginning of workflow.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#presto_job ⇒ ::Google::Cloud::Dataproc::V1::PrestoJob

Returns Optional. Job is a Presto job.

Note: The following fields are mutually exclusive: presto_job, hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_r_job, spark_sql_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Dataproc::V1::PrestoJob) —

    Optional. Job is a Presto job.

    Note: The following fields are mutually exclusive: presto_job, hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_r_job, spark_sql_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#pyspark_job ⇒ ::Google::Cloud::Dataproc::V1::PySparkJob

Returns Optional. Job is a PySpark job.

Note: The following fields are mutually exclusive: pyspark_job, hadoop_job, spark_job, hive_job, pig_job, spark_r_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Dataproc::V1::PySparkJob) —

    Optional. Job is a PySpark job.

    Note: The following fields are mutually exclusive: pyspark_job, hadoop_job, spark_job, hive_job, pig_job, spark_r_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#scheduling ⇒ ::Google::Cloud::Dataproc::V1::JobScheduling

Returns Optional. Job scheduling configuration.

Returns:



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#spark_job ⇒ ::Google::Cloud::Dataproc::V1::SparkJob

Returns Optional. Job is a Spark job.

Note: The following fields are mutually exclusive: spark_job, hadoop_job, pyspark_job, hive_job, pig_job, spark_r_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Dataproc::V1::SparkJob) —

    Optional. Job is a Spark job.

    Note: The following fields are mutually exclusive: spark_job, hadoop_job, pyspark_job, hive_job, pig_job, spark_r_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#spark_r_job ⇒ ::Google::Cloud::Dataproc::V1::SparkRJob

Returns Optional. Job is a SparkR job.

Note: The following fields are mutually exclusive: spark_r_job, hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Dataproc::V1::SparkRJob) —

    Optional. Job is a SparkR job.

    Note: The following fields are mutually exclusive: spark_r_job, hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_sql_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#spark_sql_job ⇒ ::Google::Cloud::Dataproc::V1::SparkSqlJob

Returns Optional. Job is a SparkSql job.

Note: The following fields are mutually exclusive: spark_sql_job, hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_r_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Dataproc::V1::SparkSqlJob) —

    Optional. Job is a SparkSql job.

    Note: The following fields are mutually exclusive: spark_sql_job, hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_r_job, presto_job, trino_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#step_id ⇒ ::String

Returns Required. The step id. The id must be unique among all jobs within the template.

The step id is used as prefix for job id, as job goog-dataproc-workflow-step-id label, and in prerequisiteStepIds field from other steps.

The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.

Returns:

  • (::String) —

    Required. The step id. The id must be unique among all jobs within the template.

    The step id is used as prefix for job id, as job goog-dataproc-workflow-step-id label, and in prerequisiteStepIds field from other steps.

    The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#trino_job ⇒ ::Google::Cloud::Dataproc::V1::TrinoJob

Returns Optional. Job is a Trino job.

Note: The following fields are mutually exclusive: trino_job, hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_r_job, spark_sql_job, presto_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Dataproc::V1::TrinoJob) —

    Optional. Job is a Trino job.

    Note: The following fields are mutually exclusive: trino_job, hadoop_job, spark_job, pyspark_job, hive_job, pig_job, spark_r_job, spark_sql_job, presto_job, flink_job. If a field in that set is populated, all other fields in the set will automatically be cleared.



315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'proto_docs/google/cloud/dataproc/v1/workflow_templates.rb', line 315

class OrderedJob
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # @!attribute [rw] key
  #   @return [::String]
  # @!attribute [rw] value
  #   @return [::String]
  class LabelsEntry
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end