Class: Gcloud::Bigquery::Project

Inherits:
Object
  • Object
show all
Defined in:
lib/gcloud/bigquery/project.rb

Overview

# Project

Projects are top-level containers in Google Cloud Platform. They store information about billing and authorized users, and they contain BigQuery data. Each project has a friendly name and a unique ID.

Gcloud::Bigquery::Project is the main object for interacting with Google BigQuery. Dataset objects are created, accessed, and deleted by Gcloud::Bigquery::Project.

See Gcloud#bigquery

Examples:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(service) ⇒ Project

Creates a new Service instance.

See Gcloud.bigquery



56
57
58
 
# File 'lib/gcloud/bigquery/project.rb', line 56

def initialize service
  @service = service
end

Instance Attribute Details

#service ⇒ Object 



50
51
52
 
# File 'lib/gcloud/bigquery/project.rb', line 50

def service
  @service
end

Class Method Details

.default_project ⇒ Object 



77
78
79
80
81
82
 
# File 'lib/gcloud/bigquery/project.rb', line 77

def self.default_project
  ENV["BIGQUERY_PROJECT"] ||
    ENV["GCLOUD_PROJECT"] ||
    ENV["GOOGLE_CLOUD_PROJECT"] ||
    Gcloud::GCE.project_id
end

Instance Method Details

#create_dataset(dataset_id, name: nil, description: nil, expiration: nil, location: nil) {|access| ... } ⇒ Gcloud::Bigquery::Dataset

Creates a new dataset.

Examples:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

dataset = bigquery.create_dataset "my_dataset"

A name and description can be provided:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

dataset = bigquery.create_dataset "my_dataset",
                                  name: "My Dataset",
                                  description: "This is my Dataset"

Access rules can be provided with the ‘access` option:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

dataset = bigquery.create_dataset "my_dataset",
  access: [{"role"=>"WRITER", "userByEmail"=>"[email protected]"}]

Or, configure access with a block: (See Dataset::Access)

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

dataset = bigquery.create_dataset "my_dataset" do |access|
  access.add_writer_user "[email protected]"
end

Yields:

  • (access) —

    a block for setting rules

Yield Parameters:



308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
 
# File 'lib/gcloud/bigquery/project.rb', line 308

def create_dataset dataset_id, name: nil, description: nil,
                   expiration: nil, location: nil
  ensure_service!

  new_ds = Google::Apis::BigqueryV2::Dataset.new(
    dataset_reference: Google::Apis::BigqueryV2::DatasetReference.new(
      project_id: project, dataset_id: dataset_id))

  # Can set location only on creation, no Dataset#location method
  new_ds.update! location: location unless location.nil?

  updater = Dataset::Updater.new(new_ds).tap do |b|
    b.name = name unless name.nil?
    b.description = description unless description.nil?
    b.default_expiration = expiration unless expiration.nil?
  end

  if block_given?
    yield updater
    updater.check_for_mutated_access!
  end

  gapi = service.insert_dataset new_ds
  Dataset.from_gapi gapi, service
end

#dataset(dataset_id) ⇒ Gcloud::Bigquery::Dataset?

Retrieves an existing dataset by ID.

Examples:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

dataset = bigquery.dataset "my_dataset"
puts dataset.name



244
245
246
247
248
249
250
 
# File 'lib/gcloud/bigquery/project.rb', line 244

def dataset dataset_id
  ensure_service!
  gapi = service.get_dataset dataset_id
  Dataset.from_gapi gapi, service
rescue Gcloud::NotFoundError
  nil
end

#datasets(all: nil, token: nil, max: nil) ⇒ Array<Gcloud::Bigquery::Dataset>

Retrieves the list of datasets belonging to the project.

Examples:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

datasets = bigquery.datasets
datasets.each do |dataset|
  puts dataset.name
end

Retrieve hidden datasets with the ‘all` optional arg:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

all_datasets = bigquery.datasets all: true

Retrieve all datasets: (See Dataset::List#all)

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

datasets = bigquery.datasets
datasets.all do |dataset|
  puts dataset.name
end



376
377
378
379
380
381
 
# File 'lib/gcloud/bigquery/project.rb', line 376

def datasets all: nil, token: nil, max: nil
  ensure_service!
  options = { all: all, token: token, max: max }
  gapi = service.list_datasets options
  Dataset::List.from_gapi gapi, service, all, max
end

#job(job_id) ⇒ Gcloud::Bigquery::Job?

Retrieves an existing job by ID.

Examples:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

job = bigquery.job "my_job"



399
400
401
402
403
404
405
 
# File 'lib/gcloud/bigquery/project.rb', line 399

def job job_id
  ensure_service!
  gapi = service.get_job job_id
  Job.from_gapi gapi, service
rescue Gcloud::NotFoundError
  nil
end

#jobs(all: nil, token: nil, max: nil, filter: nil) ⇒ Array<Gcloud::Bigquery::Job>

Retrieves the list of jobs belonging to the project.

Examples:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

jobs = bigquery.jobs
jobs.each do |job|
  # process job
end

Retrieve only running jobs using the ‘filter` optional arg:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

running_jobs = bigquery.jobs filter: "running"
running_jobs.each do |job|
  # process job
end

Retrieve all jobs: (See Job::List#all)

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

jobs = bigquery.jobs
jobs.all do |job|
  # process job
end



459
460
461
462
463
464
 
# File 'lib/gcloud/bigquery/project.rb', line 459

def jobs all: nil, token: nil, max: nil, filter: nil
  ensure_service!
  options = { all: all, token: token, max: max, filter: filter }
  gapi = service.list_jobs options
  Job::List.from_gapi gapi, service, all, max, filter
end

#project ⇒ Object

The BigQuery project connected to.

Examples:

require "gcloud"

gcloud = Gcloud.new "my-todo-project", "/path/to/keyfile.json"
bigquery = gcloud.bigquery

bigquery.project #=> "my-todo-project"



71
72
73
 
# File 'lib/gcloud/bigquery/project.rb', line 71

def project
  service.project
end

#query(query, max: nil, timeout: 10000, dryrun: nil, cache: true, dataset: nil, project: nil) ⇒ Gcloud::Bigquery::QueryData

Queries data using the [synchronous method](cloud.google.com/bigquery/querying-data).

Examples:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

data = bigquery.query "SELECT name FROM [my_proj:my_data.my_table]"
data.each do |row|
  puts row["name"]
end

Retrieve all rows: (See QueryData#all)

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

data = bigquery.query "SELECT name FROM [my_proj:my_data.my_table]"
data.all do |row|
  puts row["name"]
end



218
219
220
221
222
223
224
225
 
# File 'lib/gcloud/bigquery/project.rb', line 218

def query query, max: nil, timeout: 10000, dryrun: nil, cache: true,
          dataset: nil, project: nil
  ensure_service!
  options = { max: max, timeout: timeout, dryrun: dryrun, cache: cache,
              dataset: dataset, project: project }
  gapi = service.query query, options
  QueryData.from_gapi gapi, service
end

#query_job(query, priority: "INTERACTIVE", cache: true, table: nil, create: nil, write: nil, large_results: nil, flatten: nil, dataset: nil) ⇒ Gcloud::Bigquery::QueryJob

Queries data using the [asynchronous method](cloud.google.com/bigquery/querying-data).

Examples:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

job = bigquery.query_job "SELECT name FROM [my_proj:my_data.my_table]"

job.wait_until_done!
if !job.failed?
  job.query_results.each do |row|
    puts row["name"]
  end
end



146
147
148
149
150
151
152
153
154
155
 
# File 'lib/gcloud/bigquery/project.rb', line 146

def query_job query, priority: "INTERACTIVE", cache: true, table: nil,
              create: nil, write: nil, large_results: nil, flatten: nil,
              dataset: nil
  ensure_service!
  options = { priority: priority, cache: cache, table: table,
              create: create, write: write, large_results: large_results,
              flatten: flatten, dataset: dataset }
  gapi = service.query_job query, options
  Job.from_gapi gapi, service
end