Module: Google::Cloud::Dataproc::V1

Defined in:

lib/google/cloud/dataproc/v1.rb,
lib/google/cloud/dataproc/v1/credentials.rb,
lib/google/cloud/dataproc/v1/job_controller_client.rb,
lib/google/cloud/dataproc/v1/cluster_controller_client.rb,
lib/google/cloud/dataproc/v1/workflow_template_service_client.rb,
lib/google/cloud/dataproc/v1/autoscaling_policy_service_client.rb,
lib/google/cloud/dataproc/v1/doc/google/cloud/dataproc/v1/jobs.rb,
lib/google/cloud/dataproc/v1/doc/google/cloud/dataproc/v1/clusters.rb,
lib/google/cloud/dataproc/v1/doc/google/cloud/dataproc/v1/workflow_templates.rb,
lib/google/cloud/dataproc/v1/doc/google/cloud/dataproc/v1/autoscaling_policies.rb

Overview

Ruby Client for Google Cloud Dataproc API

Google Cloud Dataproc API: Manages Hadoop-based clusters and jobs on Google Cloud Platform.

• Product Documentation

Quick Start

In order to use this library, you first need to go through the following steps:

1. Select or create a Cloud Platform project.
2. Enable billing for your project.
3. Enable the Google Cloud Dataproc API.
4. Setup Authentication.

Installation

$ gem install google-cloud-dataproc

Preview

ClusterControllerClient

require "google/cloud/dataproc"

cluster_controller_client = Google::Cloud::Dataproc::ClusterController.new(version: :v1)
project_id_2 = project_id
region = "global"

# Iterate over all results.
cluster_controller_client.list_clusters(project_id_2, region).each do |element|
  # Process element.
end

# Or iterate over results one page at a time.
cluster_controller_client.list_clusters(project_id_2, region).each_page do |page|
  # Process each page at a time.
  page.each do |element|
    # Process element.
  end
end

Next Steps

• Read the Google Cloud Dataproc API Product documentation to learn more about the product and see How-to Guides.
• View this repository's main README to see the full list of Cloud APIs that we cover.

Enabling Logging

To enable logging for this library, set the logger for the underlying gRPC library. The logger that you set may be a Ruby stdlib Logger as shown below, or a Google::Cloud::Logging::Logger that will write logs to Stackdriver Logging. See grpc/logconfig.rb and the gRPC spec_helper.rb for additional information.

Configuring a Ruby stdlib logger:

require "logger"

module MyLogger
  LOGGER = Logger.new $stderr, level: Logger::WARN
  def logger
    LOGGER
  end
end

# Define a gRPC module-level logger method before grpc/logconfig.rb loads.
module GRPC
  extend MyLogger
end

Defined Under Namespace

Modules: AutoscalingPolicyService, ClusterController, JobController, WorkflowTemplateService Classes: AcceleratorConfig, AutoscalingConfig, AutoscalingPolicy, AutoscalingPolicyServiceClient, BasicAutoscalingAlgorithm, BasicYarnAutoscalingConfig, CancelJobRequest, Cluster, ClusterConfig, ClusterControllerClient, ClusterMetrics, ClusterOperation, ClusterSelector, ClusterStatus, CreateAutoscalingPolicyRequest, CreateClusterRequest, CreateWorkflowTemplateRequest, Credentials, DeleteAutoscalingPolicyRequest, DeleteClusterRequest, DeleteJobRequest, DeleteWorkflowTemplateRequest, DiagnoseClusterRequest, DiagnoseClusterResults, DiskConfig, EncryptionConfig, GceClusterConfig, GetAutoscalingPolicyRequest, GetClusterRequest, GetJobRequest, GetWorkflowTemplateRequest, HadoopJob, HiveJob, InstanceGroupAutoscalingPolicyConfig, InstanceGroupConfig, InstantiateInlineWorkflowTemplateRequest, InstantiateWorkflowTemplateRequest, Job, JobControllerClient, JobPlacement, JobReference, JobScheduling, JobStatus, KerberosConfig, LifecycleConfig, ListAutoscalingPoliciesRequest, ListAutoscalingPoliciesResponse, ListClustersRequest, ListClustersResponse, ListJobsRequest, ListJobsResponse, ListWorkflowTemplatesRequest, ListWorkflowTemplatesResponse, LoggingConfig, ManagedCluster, ManagedGroupConfig, NodeInitializationAction, OrderedJob, ParameterValidation, PigJob, PrestoJob, PySparkJob, QueryList, RegexValidation, ReservationAffinity, SecurityConfig, SoftwareConfig, SparkJob, SparkRJob, SparkSqlJob, SubmitJobRequest, TemplateParameter, UpdateAutoscalingPolicyRequest, UpdateClusterRequest, UpdateJobRequest, UpdateWorkflowTemplateRequest, ValueValidation, WorkflowGraph, WorkflowMetadata, WorkflowNode, WorkflowTemplate, WorkflowTemplatePlacement, WorkflowTemplateServiceClient, YarnApplication