Class: Google::Cloud::Bigquery::Table

Inherits:
Object
  • Object
show all
Defined in:
lib/google/cloud/bigquery/table.rb,
lib/google/cloud/bigquery/table/list.rb,
lib/google/cloud/bigquery/table/async_inserter.rb

Overview

Table

A named resource representing a BigQuery table that holds zero or more records. Every table is defined by a schema that may contain nested and repeated fields.

The Table class can also represent a logical view, which is a virtual table defined by a SQL query (see #view? and Dataset#create_view); or a materialized view, which is a precomputed view that periodically caches results of a query for increased performance and efficiency (see #materialized_view? and Dataset#create_materialized_view).

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"

table = dataset.create_table "my_table" do |schema|
  schema.string "first_name", mode: :required
  schema.record "cities_lived", mode: :repeated do |nested_schema|
    nested_schema.string "place", mode: :required
    nested_schema.integer "number_of_years", mode: :required
  end
end

row = {
  "first_name" => "Alice",
  "cities_lived" => [
    {
      "place" => "Seattle",
      "number_of_years" => 5
    },
    {
      "place" => "Stockholm",
      "number_of_years" => 6
    }
  ]
}
table.insert row

Creating a logical view:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
view = dataset.create_view "my_view",
         "SELECT name, age FROM `my_project.my_dataset.my_table`"
view.view? # true

Creating a materialized view:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
view = dataset.create_materialized_view "my_materialized_view",
                                        "SELECT name, age FROM `my_project.my_dataset.my_table`"
view.materialized_view? # true

See Also:

Direct Known Subclasses

Updater

Defined Under Namespace

Classes: AsyncInserter, List, Updater

Attributes collapse

Data collapse

Lifecycle collapse

Instance Method Details

#api_url ⇒ String?

A URL that can be used to access the table using the REST API.



718
719
720
721
722
 
# File 'lib/google/cloud/bigquery/table.rb', line 718

def api_url
  return nil if reference?
  ensure_full_data!
  @gapi.self_link
end

#buffer_bytes ⇒ Integer?

A lower-bound estimate of the number of bytes currently in this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.



1361
1362
1363
1364
1365
 
# File 'lib/google/cloud/bigquery/table.rb', line 1361

def buffer_bytes
  return nil if reference?
  ensure_full_data!
  @gapi.streaming_buffer&.estimated_bytes
end

#buffer_oldest_at ⇒ Time?

The time of the oldest entry currently in this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.



1395
1396
1397
1398
1399
1400
1401
 
# File 'lib/google/cloud/bigquery/table.rb', line 1395

def buffer_oldest_at
  return nil if reference?
  ensure_full_data!
  return nil unless @gapi.streaming_buffer
  oldest_entry_time = @gapi.streaming_buffer.oldest_entry_time
  Convert.millis_to_time oldest_entry_time
end

#buffer_rows ⇒ Integer?

A lower-bound estimate of the number of rows currently in this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.



1379
1380
1381
1382
1383
 
# File 'lib/google/cloud/bigquery/table.rb', line 1379

def buffer_rows
  return nil if reference?
  ensure_full_data!
  @gapi.streaming_buffer&.estimated_rows
end

#bytes_count ⇒ Integer?

The number of bytes in the table.



794
795
796
797
798
799
800
801
802
 
# File 'lib/google/cloud/bigquery/table.rb', line 794

def bytes_count
  return nil if reference?
  ensure_full_data!
  begin
    Integer @gapi.num_bytes
  rescue StandardError
    nil
  end
end

#clone(destination_table, reservation: nil) {|job| ... } ⇒ Boolean

Clones the data from the table to another table using a synchronous method that blocks for a response. The source and destination table have the same table type, but only bill for unique data. Timeouts and transient errors are generally handled as needed to complete the job. See also #copy_job.

The geographic location for the job ("US", "EU", etc.) can be set via CopyJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
destination_table = dataset.table "my_destination_table"

table.clone destination_table

Passing a string identifier for the destination table:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.clone "other-project:other_dataset.other_table"

Yields:

  • (job) —

    a job configuration object

Yield Parameters:



2048
2049
2050
2051
2052
2053
 
# File 'lib/google/cloud/bigquery/table.rb', line 2048

def clone destination_table, reservation: nil, &block
  copy_job_with_operation_type destination_table,
                               operation_type: OperationType::CLONE,
                               reservation: reservation,
                               &block
end

#clone? ⇒ Boolean?

Checks if the table's type is CLONE, indicating that the table represents a BigQuery table clone.

See Also:



926
927
928
929
 
# File 'lib/google/cloud/bigquery/table.rb', line 926

def clone?
  return nil if reference?
  !@gapi.clone_definition.nil?
end

#clone_definition ⇒ Google::Apis::BigqueryV2::CloneDefinition?

The Information about base table and clone time of the table.



195
196
197
198
 
# File 'lib/google/cloud/bigquery/table.rb', line 195

def clone_definition
  return nil if reference?
  @gapi.clone_definition
end

#clustering? ⇒ Boolean?

Checks if the table is clustered.

See Google::Cloud::Bigquery::Table::Updater#clustering_fields=, #clustering_fields and #clustering_fields=.

See Also:



531
532
533
534
 
# File 'lib/google/cloud/bigquery/table.rb', line 531

def clustering?
  return nil if reference?
  !@gapi.clustering.nil?
end

#clustering_fields ⇒ Array<String>?

One or more fields on which data should be clustered. Must be specified with time partitioning, data in the table will be first partitioned and subsequently clustered. The order of the returned fields determines the sort order of the data.

BigQuery supports clustering for both partitioned and non-partitioned tables.

See Google::Cloud::Bigquery::Table::Updater#clustering_fields=, #clustering_fields= and #clustering?.

See Also:



559
560
561
562
563
 
# File 'lib/google/cloud/bigquery/table.rb', line 559

def clustering_fields
  return nil if reference?
  ensure_full_data!
  @gapi.clustering.fields if clustering?
end

#clustering_fields=(fields) ⇒ Object

Updates the list of fields on which data should be clustered.

Only top-level, non-repeated, simple-type fields are supported. When you cluster a table using multiple columns, the order of columns you specify is important. The order of the specified columns determines the sort order of the data.

BigQuery supports clustering for both partitioned and non-partitioned tables.

See Google::Cloud::Bigquery::Table::Updater#clustering_fields=, #clustering_fields and #clustering?.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.clustering_fields = ["last_name", "first_name"]

See Also:



601
602
603
604
605
606
607
608
609
610
 
# File 'lib/google/cloud/bigquery/table.rb', line 601

def clustering_fields= fields
  reload! unless resource_full?
  if fields
    @gapi.clustering ||= Google::Apis::BigqueryV2::Clustering.new
    @gapi.clustering.fields = fields
  else
    @gapi.clustering = nil
  end
  patch_gapi! :clustering
end

#copy(destination_table, create: nil, write: nil, reservation: nil) {|job| ... } ⇒ Boolean

Copies the data from the table to another table using a synchronous method that blocks for a response. Timeouts and transient errors are generally handled as needed to complete the job. See also #copy_job.

The geographic location for the job ("US", "EU", etc.) can be set via CopyJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
destination_table = dataset.table "my_destination_table"

table.copy destination_table

Passing a string identifier for the destination table:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.copy "other-project:other_dataset.other_table"

Yields:

  • (job) —

    a job configuration object

Yield Parameters:



1984
1985
1986
1987
1988
1989
1990
1991
 
# File 'lib/google/cloud/bigquery/table.rb', line 1984

def copy destination_table, create: nil, write: nil, reservation: nil, &block
  copy_job_with_operation_type destination_table,
                               create: create,
                               write: write,
                               operation_type: OperationType::COPY,
                               reservation: reservation,
                               &block
end

#copy_job(destination_table, create: nil, write: nil, job_id: nil, prefix: nil, labels: nil, dryrun: nil, operation_type: nil, reservation: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::CopyJob

Copies the data from the table to another table using an asynchronous method. In this method, a CopyJob is immediately returned. The caller may poll the service by repeatedly calling Job#reload! and Job#done? to detect when the job is done, or simply block until the job is done by calling #Job#wait_until_done!. See also #copy.

The geographic location for the job ("US", "EU", etc.) can be set via CopyJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
destination_table = dataset.table "my_destination_table"

copy_job = table.copy_job destination_table

Passing a string identifier for the destination table:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

copy_job = table.copy_job "other-project:other_dataset.other_table"

copy_job.wait_until_done!
copy_job.done? #=> true

Yields:

  • (job) —

    a job configuration object

Yield Parameters:



1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
 
# File 'lib/google/cloud/bigquery/table.rb', line 1888

def copy_job destination_table, create: nil, write: nil, job_id: nil, prefix: nil, labels: nil, dryrun: nil,
             operation_type: nil, reservation: nil
  ensure_service!
  options = { create: create,
              write: write,
              dryrun: dryrun,
              labels: labels,
              job_id: job_id,
              prefix: prefix,
              operation_type: operation_type,
              reservation: reservation }
  updater = CopyJob::Updater.from_options(
    service,
    table_ref,
    Service.get_table_ref(destination_table, default_ref: table_ref),
    options
  )
  updater.location = location if location # may be table reference

  yield updater if block_given?

  job_gapi = updater.to_gapi
  gapi = service.copy_table job_gapi
  Job.from_gapi gapi, service
end

#created_at ⇒ Time?

The time when this table was created.



830
831
832
833
834
 
# File 'lib/google/cloud/bigquery/table.rb', line 830

def created_at
  return nil if reference?
  ensure_full_data!
  Convert.millis_to_time @gapi.creation_time
end

#data(token: nil, max: nil, start: nil, format_options_use_int64_timestamp: true) ⇒ Google::Cloud::Bigquery::Data

Retrieves data from the table.

If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the data retrieval.

Examples:

Paginate rows of data: (See Data#next)

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

# Iterate over the first page of results
data.each do |row|
  puts row[:name]
end
# Retrieve the next page of results
data = data.next if data.next?

Retrieve all rows of data: (See Data#all)

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = table.data

data.all do |row|
  puts row[:name]
end



1771
1772
1773
1774
1775
1776
1777
 
# File 'lib/google/cloud/bigquery/table.rb', line 1771

def data token: nil, max: nil, start: nil, format_options_use_int64_timestamp: true
  ensure_service!
  reload! unless resource_full?
  data_json = service.list_tabledata dataset_id, table_id, token: token, max: max, start: start,
format_options_use_int64_timestamp: format_options_use_int64_timestamp
  Data.from_gapi_json data_json, gapi, nil, service, format_options_use_int64_timestamp
end

#dataset_id ⇒ String

The ID of the Dataset containing this table.



144
145
146
147
 
# File 'lib/google/cloud/bigquery/table.rb', line 144

def dataset_id
  return reference.dataset_id if reference?
  @gapi.table_reference.dataset_id
end

#default_collation ⇒ String?

The default collation of the table.



763
764
765
766
767
 
# File 'lib/google/cloud/bigquery/table.rb', line 763

def default_collation
  return nil if reference?
  ensure_full_data!
  @gapi.default_collation
end

#default_collation=(new_default_collation) ⇒ Object

Updates the default collation of the table.

If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.



780
781
782
783
784
 
# File 'lib/google/cloud/bigquery/table.rb', line 780

def default_collation= new_default_collation
  reload! unless resource_full?
  @gapi.update! default_collation: new_default_collation
  patch_gapi! :default_collation
end

#delete ⇒ Boolean

Permanently deletes the table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.delete



3086
3087
3088
3089
3090
3091
3092
 
# File 'lib/google/cloud/bigquery/table.rb', line 3086

def delete
  ensure_service!
  service.delete_table dataset_id, table_id
  # Set flag for #exists?
  @exists = false
  true
end

#description ⇒ String?

A user-friendly description of the table.



732
733
734
735
736
 
# File 'lib/google/cloud/bigquery/table.rb', line 732

def description
  return nil if reference?
  ensure_full_data!
  @gapi.description
end

#description=(new_description) ⇒ Object

Updates the user-friendly description of the table.

If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.



749
750
751
752
753
 
# File 'lib/google/cloud/bigquery/table.rb', line 749

def description= new_description
  reload! unless resource_full?
  @gapi.update! description: new_description
  patch_gapi! :description
end

#enable_refresh=(new_enable_refresh) ⇒ Object

Sets whether automatic refresh of the materialized view is enabled. When true, the materialized view is updated when the base table is updated. See #materialized_view?.



1575
1576
1577
1578
1579
1580
 
# File 'lib/google/cloud/bigquery/table.rb', line 1575

def enable_refresh= new_enable_refresh
  @gapi.materialized_view = Google::Apis::BigqueryV2::MaterializedViewDefinition.new(
    enable_refresh: new_enable_refresh
  )
  patch_gapi! :materialized_view
end

#enable_refresh? ⇒ Boolean?

Whether automatic refresh of the materialized view is enabled. When true, the materialized view is updated when the base table is updated. The default value is true. See #materialized_view?.



1560
1561
1562
1563
1564
1565
 
# File 'lib/google/cloud/bigquery/table.rb', line 1560

def enable_refresh?
  return nil unless @gapi.materialized_view
  val = @gapi.materialized_view.enable_refresh
  return true if val.nil?
  val
end

#encryption ⇒ EncryptionConfiguration?

The EncryptionConfiguration object that represents the custom encryption method used to protect the table. If not set, Dataset#default_encryption is used.

Present only if the table is using custom encryption.

See Also:



1267
1268
1269
1270
1271
1272
 
# File 'lib/google/cloud/bigquery/table.rb', line 1267

def encryption
  return nil if reference?
  ensure_full_data!
  return nil if @gapi.encryption_configuration.nil?
  EncryptionConfiguration.from_gapi(@gapi.encryption_configuration).freeze
end

#encryption=(value) ⇒ Object

Set the EncryptionConfiguration object that represents the custom encryption method used to protect the table. If not set, Dataset#default_encryption is used.

Present only if the table is using custom encryption.

If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.

See Also:



1292
1293
1294
1295
1296
 
# File 'lib/google/cloud/bigquery/table.rb', line 1292

def encryption= value
  reload! unless resource_full?
  @gapi.encryption_configuration = value.to_gapi
  patch_gapi! :encryption_configuration
end

#etag ⇒ String?

The ETag hash of the table.



704
705
706
707
708
 
# File 'lib/google/cloud/bigquery/table.rb', line 704

def etag
  return nil if reference?
  ensure_full_data!
  @gapi.etag
end

#exists?(force: false) ⇒ Boolean

Determines whether the table exists in the BigQuery service. The result is cached locally. To refresh state, set force to true.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table", skip_lookup: true
table.exists? # true



3142
3143
3144
3145
3146
3147
3148
3149
 
# File 'lib/google/cloud/bigquery/table.rb', line 3142

def exists? force: false
  return gapi_exists? if force
  # If we have a value, return it
  return @exists unless @exists.nil?
  # Always true if we have a gapi object
  return true if resource?
  gapi_exists?
end

#expires_at ⇒ Time?

The time when this table expires. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.



846
847
848
849
850
 
# File 'lib/google/cloud/bigquery/table.rb', line 846

def expires_at
  return nil if reference?
  ensure_full_data!
  Convert.millis_to_time @gapi.expiration_time
end

#external ⇒ External::DataSource?

The External::DataSource (or subclass) object that represents the external data source that the table represents. Data can be queried the table, even though the data is not stored in BigQuery. Instead of loading or streaming the data, this object references the external data source.

Present only if the table represents an External Data Source. See #external? and External::DataSource.

See Also:



1315
1316
1317
1318
1319
1320
 
# File 'lib/google/cloud/bigquery/table.rb', line 1315

def external
  return nil if reference?
  ensure_full_data!
  return nil if @gapi.external_data_configuration.nil?
  External.from_gapi(@gapi.external_data_configuration).freeze
end

#external=(external) ⇒ Object

Set the External::DataSource (or subclass) object that represents the external data source that the table represents. Data can be queried the table, even though the data is not stored in BigQuery. Instead of loading or streaming the data, this object references the external data source.

Use only if the table represents an External Data Source. See #external? and External::DataSource.

If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.

See Also:



1343
1344
1345
1346
1347
 
# File 'lib/google/cloud/bigquery/table.rb', line 1343

def external= external
  reload! unless resource_full?
  @gapi.external_data_configuration = external.to_gapi
  patch_gapi! :external_data_configuration
end

#external? ⇒ Boolean?

Checks if the table's type is EXTERNAL, indicating that the table represents an External Data Source. See #external? and External::DataSource.



960
961
962
963
 
# File 'lib/google/cloud/bigquery/table.rb', line 960

def external?
  return nil if reference?
  @gapi.type == "EXTERNAL"
end

#extract(extract_url, format: nil, compression: nil, delimiter: nil, header: nil, reservation: nil) {|job| ... } ⇒ Boolean

Extracts the data from the table to a Google Cloud Storage file using a synchronous method that blocks for a response. Timeouts and transient errors are generally handled as needed to complete the job. See also #extract_job.

The geographic location for the job ("US", "EU", etc.) can be set via ExtractJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.

Examples:

Extract to a JSON file:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.extract "gs://my-bucket/file-name.json", format: "json"

Extract to a CSV file, attaching labels to the job:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.extract "gs://my-bucket/file-name.csv" do |extract|
  extract.labels = { "custom-label" => "custom-value" }
end

Yields:

  • (job) —

    a job configuration object

Yield Parameters:

See Also:



2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
 
# File 'lib/google/cloud/bigquery/table.rb', line 2373

def extract extract_url, format: nil, compression: nil, delimiter: nil, header: nil, reservation: nil, &block
  job = extract_job extract_url,
                    format:      format,
                    compression: compression,
                    delimiter:   delimiter,
                    header:      header,
                    reservation: reservation,
                    &block
  job.wait_until_done!
  ensure_job_succeeded! job
  true
end

#extract_job(extract_url, format: nil, compression: nil, delimiter: nil, header: nil, job_id: nil, prefix: nil, labels: nil, dryrun: nil, reservation: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::ExtractJob

Extracts the data from the table to a Google Cloud Storage file using an asynchronous method. In this method, an ExtractJob is immediately returned. The caller may poll the service by repeatedly calling Job#reload! and Job#done? to detect when the job is done, or simply block until the job is done by calling #Job#wait_until_done!. See also #extract.

The geographic location for the job ("US", "EU", etc.) can be set via ExtractJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will automatically be set to the location of the table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

extract_job = table.extract_job "gs://my-bucket/file-name.json",
                                format: "json"
extract_job.wait_until_done!
extract_job.done? #=> true

Yields:

  • (job) —

    a job configuration object

Yield Parameters:

See Also:



2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
 
# File 'lib/google/cloud/bigquery/table.rb', line 2291

def extract_job extract_url, format: nil, compression: nil, delimiter: nil, header: nil, job_id: nil,
                prefix: nil, labels: nil, dryrun: nil, reservation: nil
  ensure_service!
  options = { format: format, compression: compression, delimiter: delimiter, header: header, dryrun: dryrun,
              job_id: job_id, prefix: prefix, labels: labels, reservation: reservation }
  updater = ExtractJob::Updater.from_options service, table_ref, extract_url, options
  updater.location = location if location # may be table reference

  yield updater if block_given?

  job_gapi = updater.to_gapi
  gapi = service.extract_table job_gapi
  Job.from_gapi gapi, service
end

#fields ⇒ Array<Schema::Field>?

The fields of the table, obtained from its schema.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.fields.each do |field|
  puts field.name
end



1205
1206
1207
1208
 
# File 'lib/google/cloud/bigquery/table.rb', line 1205

def fields
  return nil if reference?
  schema.fields
end

#headers ⇒ Array<Symbol>?

The names of the columns in the table, obtained from its schema.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.headers.each do |header|
  puts header
end



1228
1229
1230
1231
 
# File 'lib/google/cloud/bigquery/table.rb', line 1228

def headers
  return nil if reference?
  schema.headers
end

#id ⇒ String?

The combined Project ID, Dataset ID, and Table ID for this table, in the format specified by the Legacy SQL Query Reference (project-name:dataset_id.table_id). This is useful for referencing tables in other projects and datasets. To use this value in queries see #query_id.



625
626
627
628
 
# File 'lib/google/cloud/bigquery/table.rb', line 625

def id
  return nil if reference?
  @gapi.id
end

#insert(rows, insert_ids: nil, skip_invalid: nil, ignore_unknown: nil) ⇒ Google::Cloud::Bigquery::InsertResponse

Inserts data into the table for near-immediate querying, without the need to complete a load operation before the data can appear in query results.

Simple Ruby types are generally accepted per JSON rules, along with the following support for BigQuery's more complex types:

BigQuery Ruby Notes
NUMERIC BigDecimal BigDecimal values will be rounded to scale 9.
BIGNUMERIC String Pass as String to avoid rounding to scale 9.
DATETIME DateTime DATETIME does not support time zone.
DATE Date
GEOGRAPHY String Well-known text (WKT) or GeoJSON.
JSON String (Stringified JSON) String, as JSON does not have a schema to verify.
TIMESTAMP Time
TIME Google::Cloud::BigQuery::Time
BYTES File, IO, StringIO, or similar
ARRAY Array Nested arrays, nil values are not supported.
STRUCT Hash Hash keys may be strings or symbols.

For GEOGRAPHY data, see Working with BigQuery GIS data.

Because BigQuery's streaming API is designed for high insertion rates, modifications to the underlying table metadata are eventually consistent when interacting with the streaming system. In most cases metadata changes are propagated within minutes, but during this period API responses may reflect the inconsistent state of the table.

The value :skip can be provided to skip the generation of IDs for all rows, or to skip the generation of an ID for a specific row in the array.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

rows = [
  { "first_name" => "Alice", "age" => 21 },
  { "first_name" => "Bob", "age" => 22 }
]
table.insert rows

Avoid retrieving the dataset and table with skip_lookup:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset", skip_lookup: true
table = dataset.table "my_table", skip_lookup: true

rows = [
  { "first_name" => "Alice", "age" => 21 },
  { "first_name" => "Bob", "age" => 22 }
]
table.insert rows

Pass BIGNUMERIC value as a string to avoid rounding to scale 9 in the conversion from BigDecimal:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

row = {
  "my_numeric" => BigDecimal("123456798.987654321"),
  "my_bignumeric" => "123456798.98765432100001" # BigDecimal would be rounded, use String instead!
}
table.insert row

Raises:

  • (ArgumentError)

See Also:



2994
2995
2996
2997
2998
2999
3000
3001
3002
3003
3004
3005
3006
3007
3008
3009
3010
3011
3012
3013
 
# File 'lib/google/cloud/bigquery/table.rb', line 2994

def insert rows, insert_ids: nil, skip_invalid: nil, ignore_unknown: nil
  rows = [rows] if rows.is_a? Hash
  raise ArgumentError, "No rows provided" if rows.empty?

  insert_ids = Array.new(rows.count) { :skip } if insert_ids == :skip
  insert_ids = Array insert_ids
  if insert_ids.count.positive? && insert_ids.count != rows.count
    raise ArgumentError, "insert_ids must be the same size as rows"
  end

  ensure_service!
  gapi = service.insert_tabledata dataset_id,
                                  table_id,
                                  rows,
                                  skip_invalid: skip_invalid,
                                  ignore_unknown: ignore_unknown,
                                  insert_ids: insert_ids,
                                  project_id: project_id
  InsertResponse.from_gapi rows, gapi
end

#insert_async(skip_invalid: nil, ignore_unknown: nil, max_bytes: 10_000_000, max_rows: 500, interval: 10, threads: 4) {|response| ... } ⇒ Table::AsyncInserter

Create an asynchronous inserter object used to insert rows in batches.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
inserter = table.insert_async do |result|
  if result.error?
    log_error result.error
  else
    log_insert "inserted #{result.insert_count} rows " \
      "with #{result.error_count} errors"
  end
end

rows = [
  { "first_name" => "Alice", "age" => 21 },
  { "first_name" => "Bob", "age" => 22 }
]
inserter.insert rows

inserter.stop.wait!

Yields:

  • (response) —

    the callback for when a batch of rows is inserted

Yield Parameters:



3062
3063
3064
3065
3066
3067
3068
 
# File 'lib/google/cloud/bigquery/table.rb', line 3062

def insert_async skip_invalid: nil, ignore_unknown: nil, max_bytes: 10_000_000, max_rows: 500, interval: 10,
                 threads: 4, &block
  ensure_service!

  AsyncInserter.new self, skip_invalid: skip_invalid, ignore_unknown: ignore_unknown, max_bytes: max_bytes,
                          max_rows: max_rows, interval: interval, threads: threads, &block
end

#labels ⇒ Hash<String, String>?

A hash of user-provided labels associated with this table. Labels are used to organize and group tables. See Using Labels.

The returned hash is frozen and changes are not allowed. Use #labels= to replace the entire hash.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

labels = table.labels
labels["department"] #=> "shipping"



1001
1002
1003
1004
1005
1006
 
# File 'lib/google/cloud/bigquery/table.rb', line 1001

def labels
  return nil if reference?
  m = @gapi.labels
  m = m.to_h if m.respond_to? :to_h
  m.dup.freeze
end

#labels=(labels) ⇒ Object

Updates the hash of user-provided labels associated with this table. Labels are used to organize and group tables. See Using Labels.

If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.labels = { "department" => "shipping" }



1045
1046
1047
1048
1049
 
# File 'lib/google/cloud/bigquery/table.rb', line 1045

def labels= labels
  reload! unless resource_full?
  @gapi.labels = labels
  patch_gapi! :labels
end

#last_refresh_time ⇒ Time?

The time when the materialized view was last modified. See #materialized_view?.



1590
1591
1592
 
# File 'lib/google/cloud/bigquery/table.rb', line 1590

def last_refresh_time
  Convert.millis_to_time @gapi.materialized_view&.last_refresh_time
end

#load(files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, autodetect: nil, null_marker: nil, session_id: nil, schema: self.schema, date_format: nil, datetime_format: nil, time_format: nil, timestamp_format: nil, null_markers: nil, source_column_match: nil, time_zone: nil, reference_file_schema_uri: nil, preserve_ascii_control_characters: nil, reservation: nil) {|updater| ... } ⇒ Boolean

Loads data into the table. You can pass a google-cloud storage file path or a google-cloud storage file instance. Or, you can upload a file directly. See Loading Data with a POST Request.

The geographic location for the job ("US", "EU", etc.) can be set via LoadJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

success = table.load "gs://my-bucket/file-name.csv"

Pass a google-cloud-storage File instance:

require "google/cloud/bigquery"
require "google/cloud/storage"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

storage = Google::Cloud::Storage.new
bucket = storage.bucket "my-bucket"
file = bucket.file "file-name.csv"
success = table.load file

Pass a list of google-cloud-storage files:

require "google/cloud/bigquery"
require "google/cloud/storage"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

storage = Google::Cloud::Storage.new
bucket = storage.bucket "my-bucket"
file = bucket.file "file-name.csv"
table.load [file, "gs://my-bucket/file-name2.csv"]

Upload a file directly:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

file = File.open "my_data.json"
success = table.load file do |j|
  j.format = "newline_delimited_json"
end

Yields:

  • (updater) —

    A block for setting the schema of the destination table and other options for the load job. The schema can be omitted if the destination table already exists, or if you're loading data from a Google Cloud Datastore backup.

Yield Parameters:



2873
2874
2875
2876
2877
2878
2879
2880
2881
2882
2883
2884
2885
2886
2887
2888
2889
2890
2891
2892
2893
2894
 
# File 'lib/google/cloud/bigquery/table.rb', line 2873

def load files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil,
         quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil,
         quote: nil, skip_leading: nil, autodetect: nil, null_marker: nil, session_id: nil,
         schema: self.schema, date_format: nil, datetime_format: nil, time_format: nil, timestamp_format: nil,
         null_markers: nil, source_column_match: nil, time_zone: nil, reference_file_schema_uri: nil,
         preserve_ascii_control_characters: nil, reservation: nil, &block
  job = load_job files, format: format, create: create, write: write, projection_fields: projection_fields,
                        jagged_rows: jagged_rows, quoted_newlines: quoted_newlines, encoding: encoding,
                        delimiter: delimiter, ignore_unknown: ignore_unknown, max_bad_records: max_bad_records,
                        quote: quote, skip_leading: skip_leading, autodetect: autodetect,
                        null_marker: null_marker, session_id: session_id, schema: schema,
                        date_format: date_format, datetime_format: datetime_format, time_format: time_format,
                        timestamp_format: timestamp_format, null_markers: null_markers,
                        source_column_match: source_column_match, time_zone: time_zone,
                        reference_file_schema_uri: reference_file_schema_uri,
                        preserve_ascii_control_characters: preserve_ascii_control_characters,
                        reservation: reservation, &block

  job.wait_until_done!
  ensure_job_succeeded! job
  true
end

#load_job(files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, job_id: nil, prefix: nil, labels: nil, autodetect: nil, null_marker: nil, dryrun: nil, create_session: nil, session_id: nil, schema: self.schema, date_format: nil, datetime_format: nil, time_format: nil, timestamp_format: nil, null_markers: nil, source_column_match: nil, time_zone: nil, reference_file_schema_uri: nil, preserve_ascii_control_characters: nil, reservation: nil) {|load_job| ... } ⇒ Google::Cloud::Bigquery::LoadJob

Loads data into the table. You can pass a google-cloud storage file path or a google-cloud storage file instance. Or, you can upload a file directly. See Loading Data with a POST Request.

The geographic location for the job ("US", "EU", etc.) can be set via LoadJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

load_job = table.load_job "gs://my-bucket/file-name.csv"

Pass a google-cloud-storage File instance:

require "google/cloud/bigquery"
require "google/cloud/storage"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

storage = Google::Cloud::Storage.new
bucket = storage.bucket "my-bucket"
file = bucket.file "file-name.csv"
load_job = table.load_job file

Pass a list of google-cloud-storage files:

require "google/cloud/bigquery"
require "google/cloud/storage"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

storage = Google::Cloud::Storage.new
bucket = storage.bucket "my-bucket"
file = bucket.file "file-name.csv"
load_job = table.load_job [file, "gs://my-bucket/file-name2.csv"]

Upload a file directly:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

file = File.open "my_data.csv"
load_job = table.load_job file

Yields:

  • (load_job) —

    a block for setting the load job

Yield Parameters:

  • load_job (LoadJob) —

    the load job object to be updated



2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
2646
2647
2648
2649
2650
2651
2652
2653
2654
2655
2656
2657
2658
2659
 
# File 'lib/google/cloud/bigquery/table.rb', line 2630

def load_job files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil,
             quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil,
             quote: nil, skip_leading: nil, job_id: nil, prefix: nil, labels: nil, autodetect: nil,
             null_marker: nil, dryrun: nil, create_session: nil, session_id: nil, schema: self.schema,
             date_format: nil, datetime_format: nil, time_format: nil, timestamp_format: nil,
             null_markers: nil, source_column_match: nil, time_zone: nil, reference_file_schema_uri: nil,
             preserve_ascii_control_characters: nil, reservation: nil
  ensure_service!

  updater = load_job_updater format: format, create: create, write: write, projection_fields: projection_fields,
                             jagged_rows: jagged_rows, quoted_newlines: quoted_newlines, encoding: encoding,
                             delimiter: delimiter, ignore_unknown: ignore_unknown,
                             max_bad_records: max_bad_records, quote: quote, skip_leading: skip_leading,
                             dryrun: dryrun, job_id: job_id, prefix: prefix, schema: schema, labels: labels,
                             autodetect: autodetect, null_marker: null_marker, create_session: create_session,
                             session_id: session_id, date_format: date_format,
                             datetime_format: datetime_format, time_format: time_format,
                             timestamp_format: timestamp_format, null_markers: null_markers,
                             source_column_match: source_column_match, time_zone: time_zone,
                             reference_file_schema_uri: reference_file_schema_uri,
                             preserve_ascii_control_characters: preserve_ascii_control_characters,
                             reservation: reservation

  yield updater if block_given?

  job_gapi = updater.to_gapi

  return load_local files, job_gapi if local_file? files
  load_storage files, job_gapi
end

#location ⇒ String?

The geographic location where the table should reside. Possible values include EU and US. The default value is US.



973
974
975
976
977
 
# File 'lib/google/cloud/bigquery/table.rb', line 973

def location
  return nil if reference?
  ensure_full_data!
  @gapi.location
end

#materialized_view? ⇒ Boolean?

Checks if the table's type is MATERIALIZED_VIEW, indicating that the table represents a BigQuery materialized view. See Dataset#create_materialized_view.

See Also:



944
945
946
947
 
# File 'lib/google/cloud/bigquery/table.rb', line 944

def materialized_view?
  return nil if reference?
  @gapi.type == "MATERIALIZED_VIEW"
end

#modified_at ⇒ Time?

The date when this table was last modified.



860
861
862
863
864
 
# File 'lib/google/cloud/bigquery/table.rb', line 860

def modified_at
  return nil if reference?
  ensure_full_data!
  Convert.millis_to_time @gapi.last_modified_time
end

#name ⇒ String?

The name of the table.



674
675
676
677
 
# File 'lib/google/cloud/bigquery/table.rb', line 674

def name
  return nil if reference?
  @gapi.friendly_name
end

#name=(new_name) ⇒ Object

Updates the name of the table.

If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.



690
691
692
693
694
 
# File 'lib/google/cloud/bigquery/table.rb', line 690

def name= new_name
  reload! unless resource_full?
  @gapi.update! friendly_name: new_name
  patch_gapi! :friendly_name
end

#param_types ⇒ Hash

The types of the fields in the table, obtained from its schema. Types use the same format as the optional query parameter types.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.param_types



1248
1249
1250
1251
 
# File 'lib/google/cloud/bigquery/table.rb', line 1248

def param_types
  return nil if reference?
  schema.param_types
end

#policy ⇒ Policy

Gets the Cloud IAM access control policy for the table. The latest policy will be read from the service. See also #update_policy.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

policy = table.policy

policy.frozen? #=> true
binding_owner = policy.bindings.find { |b| b.role == "roles/owner" }
binding_owner.role #=> "roles/owner"
binding_owner.members #=> ["user:[email protected]"]
binding_owner.frozen? #=> true
binding_owner.members.frozen? #=> true

Raises:

  • (ArgumentError)

See Also:



1647
1648
1649
1650
1651
1652
 
# File 'lib/google/cloud/bigquery/table.rb', line 1647

def policy
  raise ArgumentError, "Block argument not supported: Use #update_policy instead." if block_given?
  ensure_service!
  gapi = service.get_table_policy dataset_id, table_id
  Policy.from_gapi(gapi).freeze
end

#project_id ⇒ String

The ID of the Project containing this table.



156
157
158
159
 
# File 'lib/google/cloud/bigquery/table.rb', line 156

def project_id
  return reference.project_id if reference?
  @gapi.table_reference.project_id
end

#query ⇒ String?

The query that defines the view or materialized view. See #view? and #materialized_view?.



1412
1413
1414
 
# File 'lib/google/cloud/bigquery/table.rb', line 1412

def query
  view? ? @gapi.view&.query : @gapi.materialized_view&.query
end

#query=(new_query) ⇒ Object

Updates the query that defines the view. (See #view?.) Not supported for materialized views.

This method sets the query using standard SQL. To specify legacy SQL or to use user-defined function resources for a view, use (#set_query) instead.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
view = dataset.table "my_view"

view.query = "SELECT first_name FROM " \
             "`my_project.my_dataset.my_table`"

See Also:



1440
1441
1442
 
# File 'lib/google/cloud/bigquery/table.rb', line 1440

def query= new_query
  set_query new_query
end

#query_id(standard_sql: nil, legacy_sql: nil) ⇒ String

The value returned by #id, wrapped in backticks (Standard SQL) or s quare brackets (Legacy SQL) to accommodate project IDs containing dashes. Useful in queries.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

data = bigquery.query "SELECT first_name FROM #{table.query_id}"



658
659
660
661
662
663
664
 
# File 'lib/google/cloud/bigquery/table.rb', line 658

def query_id standard_sql: nil, legacy_sql: nil
  if Convert.resolve_legacy_sql standard_sql, legacy_sql
    "[#{project_id}:#{dataset_id}.#{table_id}]"
  else
    "`#{project_id}.#{dataset_id}.#{table_id}`"
  end
end

#query_legacy_sql? ⇒ Boolean

Checks if the view's query is using legacy sql. See #view?.



1509
1510
1511
1512
1513
1514
 
# File 'lib/google/cloud/bigquery/table.rb', line 1509

def query_legacy_sql?
  return nil unless @gapi.view
  val = @gapi.view.use_legacy_sql
  return true if val.nil?
  val
end

#query_standard_sql? ⇒ Boolean

Checks if the view's query is using standard sql. See #view?.



1523
1524
1525
1526
 
# File 'lib/google/cloud/bigquery/table.rb', line 1523

def query_standard_sql?
  return nil unless @gapi.view
  !query_legacy_sql?
end

#query_udfs ⇒ Array<String>?

The user-defined function resources used in the view's query. May be either a code resource to load from a Google Cloud Storage URI (gs://bucket/path), or an inline resource that contains code for a user-defined function (UDF). Providing an inline code resource is equivalent to providing a URI for a file containing the same code. See User-Defined Functions. See #view?.



1543
1544
1545
1546
1547
1548
 
# File 'lib/google/cloud/bigquery/table.rb', line 1543

def query_udfs
  return nil unless @gapi.view
  udfs_gapi = @gapi.view.user_defined_function_resources
  return [] if udfs_gapi.nil?
  Array(udfs_gapi).map { |udf| udf.inline_code || udf.resource_uri }
end

#range_partitioning? ⇒ Boolean?

Checks if the table is range partitioned. See Creating and using integer range partitioned tables.



220
221
222
223
 
# File 'lib/google/cloud/bigquery/table.rb', line 220

def range_partitioning?
  return nil if reference?
  !@gapi.range_partitioning.nil?
end

#range_partitioning_end ⇒ Integer?

The end of range partitioning, exclusive. See Creating and using integer range partitioned tables.



281
282
283
284
285
 
# File 'lib/google/cloud/bigquery/table.rb', line 281

def range_partitioning_end
  return nil if reference?
  ensure_full_data!
  @gapi.range_partitioning.range.end if range_partitioning?
end

#range_partitioning_field ⇒ Integer?

The field on which the table is range partitioned, if any. The field must be a top-level NULLABLE/REQUIRED field. The only supported type is INTEGER/INT64. See Creating and using integer range partitioned tables.



235
236
237
238
239
 
# File 'lib/google/cloud/bigquery/table.rb', line 235

def range_partitioning_field
  return nil if reference?
  ensure_full_data!
  @gapi.range_partitioning.field if range_partitioning?
end

#range_partitioning_interval ⇒ Integer?

The width of each interval. See Creating and using integer range partitioned tables.



265
266
267
268
269
270
 
# File 'lib/google/cloud/bigquery/table.rb', line 265

def range_partitioning_interval
  return nil if reference?
  ensure_full_data!
  return nil unless range_partitioning?
  @gapi.range_partitioning.range.interval
end

#range_partitioning_start ⇒ Integer?

The start of range partitioning, inclusive. See Creating and using integer range partitioned tables.



250
251
252
253
254
 
# File 'lib/google/cloud/bigquery/table.rb', line 250

def range_partitioning_start
  return nil if reference?
  ensure_full_data!
  @gapi.range_partitioning.range.start if range_partitioning?
end

#reference? ⇒ Boolean

Whether the table was created without retrieving the resource representation from the BigQuery service.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table", skip_lookup: true

table.reference? # true
table.reload!
table.reference? # false



3170
3171
3172
 
# File 'lib/google/cloud/bigquery/table.rb', line 3170

def reference?
  @gapi.nil?
end

#refresh_interval_ms ⇒ Integer?

The maximum frequency in milliseconds at which the materialized view will be refreshed. See #materialized_view?.



1603
1604
1605
 
# File 'lib/google/cloud/bigquery/table.rb', line 1603

def refresh_interval_ms
  @gapi.materialized_view&.refresh_interval_ms
end

#refresh_interval_ms=(new_refresh_interval_ms) ⇒ Object

Sets the maximum frequency at which the materialized view will be refreshed. See #materialized_view?.



1615
1616
1617
1618
1619
1620
 
# File 'lib/google/cloud/bigquery/table.rb', line 1615

def refresh_interval_ms= new_refresh_interval_ms
  @gapi.materialized_view = Google::Apis::BigqueryV2::MaterializedViewDefinition.new(
    refresh_interval_ms: new_refresh_interval_ms
  )
  patch_gapi! :materialized_view
end

#reload! ⇒ Google::Cloud::Bigquery::Table Also known as: refresh!

Reloads the table with current data from the BigQuery service.

Examples:

Skip retrieving the table from the service, then load it:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table", skip_lookup: true

table.reload!



3112
3113
3114
3115
3116
3117
3118
 
# File 'lib/google/cloud/bigquery/table.rb', line 3112

def reload!
  ensure_service!
  @gapi = service.get_table dataset_id, table_id, metadata_view: metadata_view
  @reference = nil
  @exists = nil
  self
end

#require_partition_filter ⇒ Boolean?

Whether queries over this table require a partition filter that can be used for partition elimination to be specified. See Partitioned Tables.



479
480
481
482
483
 
# File 'lib/google/cloud/bigquery/table.rb', line 479

def require_partition_filter
  return nil if reference?
  ensure_full_data!
  @gapi.require_partition_filter
end

#require_partition_filter=(new_require) ⇒ Object

Sets whether queries over this table require a partition filter. See Partitioned Tables.

If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table" do |t|
  t.require_partition_filter = true
end



508
509
510
511
512
 
# File 'lib/google/cloud/bigquery/table.rb', line 508

def require_partition_filter= new_require
  reload! unless resource_full?
  @gapi.require_partition_filter = new_require
  patch_gapi! :require_partition_filter
end

#resource? ⇒ Boolean

Whether the table was created with a resource representation from the BigQuery service.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table", skip_lookup: true

table.resource? # false
table.reload!
table.resource? # true



3193
3194
3195
 
# File 'lib/google/cloud/bigquery/table.rb', line 3193

def resource?
  !@gapi.nil?
end

#resource_full? ⇒ Boolean

Whether the table was created with a full resource representation from the BigQuery service.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.resource_full? # true



3242
3243
3244
 
# File 'lib/google/cloud/bigquery/table.rb', line 3242

def resource_full?
  @gapi.is_a? Google::Apis::BigqueryV2::Table
end

#resource_partial? ⇒ Boolean

Whether the table was created with a partial resource representation from the BigQuery service by retrieval through Dataset#tables. See Tables: list response for the contents of the partial representation. Accessing any attribute outside of the partial representation will result in loading the full representation.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

dataset = bigquery.dataset "my_dataset"
table = dataset.tables.first

table.resource_partial? # true
table.description # Loads the full resource.
table.resource_partial? # false



3221
3222
3223
 
# File 'lib/google/cloud/bigquery/table.rb', line 3221

def resource_partial?
  @gapi.is_a? Google::Apis::BigqueryV2::TableList::Table
end

#resource_tags ⇒ Hash<String, String>?

The resource tags associated with this table. Tag keys are globally unique.

The returned hash is frozen and changes are not allowed. Use #resource_tags= to replace the entire hash.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

resource_tags = table.resource_tags
resource_tags["12345/environment"] #=> "production"

See Also:



1079
1080
1081
1082
1083
1084
 
# File 'lib/google/cloud/bigquery/table.rb', line 1079

def resource_tags
  return nil if reference?
  m = @gapi.resource_tags
  m = m.to_h if m.respond_to? :to_h
  m.dup.freeze
end

#resource_tags=(resource_tags) ⇒ Object

Updates the resource tags associated with this table. Tag keys are globally unique.

If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.resource_tags = { "12345/environment" => "production" }

See Also:



1116
1117
1118
1119
1120
 
# File 'lib/google/cloud/bigquery/table.rb', line 1116

def resource_tags= resource_tags
  reload! unless resource_full?
  @gapi.resource_tags = resource_tags
  patch_gapi! :resource_tags
end

#restore(destination_table, create: nil, write: nil, reservation: nil) {|job| ... } ⇒ Boolean

Restore the data from the table to another table using a synchronous method that blocks for a response. The source table type is SNAPSHOT and the destination table type is TABLE. Timeouts and transient errors are generally handled as needed to complete the job. See also #copy_job.

The geographic location for the job ("US", "EU", etc.) can be set via CopyJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
destination_table = dataset.table "my_destination_table"

table.restore destination_table

Passing a string identifier for the destination table:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.restore "other-project:other_dataset.other_table"

Yields:

  • (job) —

    a job configuration object

Yield Parameters:



2187
2188
2189
2190
2191
2192
2193
2194
 
# File 'lib/google/cloud/bigquery/table.rb', line 2187

def restore destination_table, create: nil, write: nil, reservation: nil, &block
  copy_job_with_operation_type destination_table,
                               create: create,
                               write: write,
                               operation_type: OperationType::RESTORE,
                               reservation: reservation,
                               &block
end

#rows_count ⇒ Integer?

The number of rows in the table.



812
813
814
815
816
817
818
819
820
 
# File 'lib/google/cloud/bigquery/table.rb', line 812

def rows_count
  return nil if reference?
  ensure_full_data!
  begin
    Integer @gapi.num_rows
  rescue StandardError
    nil
  end
end

#schema(replace: false) {|schema| ... } ⇒ Google::Cloud::Bigquery::Schema?

Returns the table's schema. If the table is not a view (See #view?), this method can also be used to set, replace, or add to the schema by passing a block. See Schema for available methods.

If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table"

table.schema do |schema|
  schema.string "first_name", mode: :required
  schema.record "cities_lived", mode: :repeated do |nested_schema|
    nested_schema.string "place", mode: :required
    nested_schema.integer "number_of_years", mode: :required
  end
end

Load the schema from a file

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table"
table.schema do |schema|
  schema.load File.open("schema.json")
end

Yields:

  • (schema) —

    a block for setting the schema

Yield Parameters:

  • schema (Schema) —

    the object accepting the schema



1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
 
# File 'lib/google/cloud/bigquery/table.rb', line 1172

def schema replace: false
  return nil if reference? && !block_given?
  reload! unless resource_full?
  schema_builder = Schema.from_gapi @gapi.schema
  if block_given?
    schema_builder = Schema.from_gapi if replace
    yield schema_builder
    if schema_builder.changed?
      @gapi.schema = schema_builder.to_gapi
      patch_gapi! :schema
    end
  end
  schema_builder.freeze
end

#set_query(query, standard_sql: nil, legacy_sql: nil, udfs: nil) ⇒ Object

Updates the query that defines the view. (See #view?.) Not supported for materialized views.

Allows setting of standard vs. legacy SQL and user-defined function resources.

Examples:

Update a view:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
view = dataset.table "my_view"

view.set_query "SELECT first_name FROM " \
               "`my_project.my_dataset.my_table`",
               standard_sql: true

See Also:



1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
 
# File 'lib/google/cloud/bigquery/table.rb', line 1491

def set_query query, standard_sql: nil, legacy_sql: nil, udfs: nil
  raise "Updating the query is not supported for Table type: #{@gapi.type}" unless view?
  use_legacy_sql = Convert.resolve_legacy_sql standard_sql, legacy_sql
  @gapi.view = Google::Apis::BigqueryV2::ViewDefinition.new(
    query:                           query,
    use_legacy_sql:                  use_legacy_sql,
    user_defined_function_resources: udfs_gapi(udfs)
  )
  patch_gapi! :view
end

#snapshot(destination_table, reservation: nil) {|job| ... } ⇒ Boolean

Takes snapshot of the data from the table to another table using a synchronous method that blocks for a response. The source table type is TABLE and the destination table type is SNAPSHOT. Timeouts and transient errors are generally handled as needed to complete the job. See also #copy_job.

The geographic location for the job ("US", "EU", etc.) can be set via CopyJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
destination_table = dataset.table "my_destination_table"

table.snapshot destination_table

Passing a string identifier for the destination table:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.snapshot "other-project:other_dataset.other_table"

Yields:

  • (job) —

    a job configuration object

Yield Parameters:



2109
2110
2111
2112
2113
2114
 
# File 'lib/google/cloud/bigquery/table.rb', line 2109

def snapshot destination_table, reservation: nil, &block
  copy_job_with_operation_type destination_table,
                               operation_type: OperationType::SNAPSHOT,
                               reservation: reservation,
                               &block
end

#snapshot? ⇒ Boolean?

Checks if the table's type is SNAPSHOT, indicating that the table represents a BigQuery table snapshot.

See Also:



909
910
911
912
 
# File 'lib/google/cloud/bigquery/table.rb', line 909

def snapshot?
  return nil if reference?
  @gapi.type == "SNAPSHOT"
end

#snapshot_definition ⇒ Google::Apis::BigqueryV2::SnapshotDefinition?

The Information about base table and snapshot time of the table.



182
183
184
185
 
# File 'lib/google/cloud/bigquery/table.rb', line 182

def snapshot_definition
  return nil if reference?
  @gapi.snapshot_definition
end

#table? ⇒ Boolean?

Checks if the table's type is TABLE.



875
876
877
878
 
# File 'lib/google/cloud/bigquery/table.rb', line 875

def table?
  return nil if reference?
  @gapi.type == "TABLE"
end

#table_id ⇒ String

A unique ID for this table.



131
132
133
134
 
# File 'lib/google/cloud/bigquery/table.rb', line 131

def table_id
  return reference.table_id if reference?
  @gapi.table_reference.table_id
end

#test_iam_permissions(*permissions) ⇒ Array<String>

Tests the specified permissions against the Cloud IAM access control policy.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

permissions = table.test_iam_permissions "bigquery.tables.get",
                                         "bigquery.tables.delete"
permissions.include? "bigquery.tables.get"    #=> true
permissions.include? "bigquery.tables.delete" #=> false

See Also:



1716
1717
1718
1719
1720
1721
 
# File 'lib/google/cloud/bigquery/table.rb', line 1716

def test_iam_permissions *permissions
  permissions = Array(permissions).flatten
  ensure_service!
  gapi = service.test_table_permissions dataset_id, table_id, permissions
  gapi.permissions.freeze
end

#time_partitioning? ⇒ Boolean?

Checks if the table is time partitioned. See Partitioned Tables.



297
298
299
300
 
# File 'lib/google/cloud/bigquery/table.rb', line 297

def time_partitioning?
  return nil if reference?
  !@gapi.time_partitioning.nil?
end

#time_partitioning_expiration ⇒ Integer?

The expiration for the time partitions, if any, in seconds. See Partitioned Tables.



422
423
424
425
426
427
428
 
# File 'lib/google/cloud/bigquery/table.rb', line 422

def time_partitioning_expiration
  return nil if reference?
  ensure_full_data!
  return nil unless time_partitioning?
  return nil if @gapi.time_partitioning.expiration_ms.nil?
  @gapi.time_partitioning.expiration_ms / 1_000
end

#time_partitioning_expiration=(expiration) ⇒ Object

Sets the time partition expiration for the table. See Partitioned Tables. The table must also be time partitioned.

See #time_partitioning_type=.

If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table" do |t|
  t.schema do |schema|
    schema.timestamp "dob", mode: :required
  end
  t.time_partitioning_type = "DAY"
  t.time_partitioning_field = "dob"
  t.time_partitioning_expiration = 86_400
end



460
461
462
463
464
465
466
 
# File 'lib/google/cloud/bigquery/table.rb', line 460

def time_partitioning_expiration= expiration
  reload! unless resource_full?
  expiration_ms = expiration * 1000 if expiration
  @gapi.time_partitioning ||= Google::Apis::BigqueryV2::TimePartitioning.new
  @gapi.time_partitioning.expiration_ms = expiration_ms
  patch_gapi! :time_partitioning
end

#time_partitioning_field ⇒ String?

The field on which the table is time partitioned, if any. If not set, the destination table is time partitioned by pseudo column _PARTITIONTIME; if set, the table is time partitioned by this field. See Partitioned Tables.



367
368
369
370
371
 
# File 'lib/google/cloud/bigquery/table.rb', line 367

def time_partitioning_field
  return nil if reference?
  ensure_full_data!
  @gapi.time_partitioning.field if time_partitioning?
end

#time_partitioning_field=(field) ⇒ Object

Sets the field on which to time partition the table. If not set, the destination table is time partitioned by pseudo column _PARTITIONTIME; if set, the table is time partitioned by this field. See Partitioned Tables. The table must also be time partitioned.

See #time_partitioning_type=.

You can only set the time partitioning field while creating a table as in the example below. BigQuery does not allow you to change time partitioning on an existing table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table" do |t|
  t.schema do |schema|
    schema.timestamp "dob", mode: :required
  end
  t.time_partitioning_type  = "DAY"
  t.time_partitioning_field = "dob"
end



405
406
407
408
409
410
 
# File 'lib/google/cloud/bigquery/table.rb', line 405

def time_partitioning_field= field
  reload! unless resource_full?
  @gapi.time_partitioning ||= Google::Apis::BigqueryV2::TimePartitioning.new
  @gapi.time_partitioning.field = field
  patch_gapi! :time_partitioning
end

#time_partitioning_type ⇒ String?

The period for which the table is time partitioned, if any. See Partitioned Tables.



313
314
315
316
317
 
# File 'lib/google/cloud/bigquery/table.rb', line 313

def time_partitioning_type
  return nil if reference?
  ensure_full_data!
  @gapi.time_partitioning.type if time_partitioning?
end

#time_partitioning_type=(type) ⇒ Object

Sets the time partitioning type for the table. See Partitioned Tables. The supported types are DAY, HOUR, MONTH, and YEAR, which will generate one partition per day, hour, month, and year, respectively.

You can only set time partitioning when creating a table as in the example below. BigQuery does not allow you to change time partitioning on an existing table.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table" do |t|
  t.schema do |schema|
    schema.timestamp "dob", mode: :required
  end
  t.time_partitioning_type  = "DAY"
  t.time_partitioning_field = "dob"
end



348
349
350
351
352
353
 
# File 'lib/google/cloud/bigquery/table.rb', line 348

def time_partitioning_type= type
  reload! unless resource_full?
  @gapi.time_partitioning ||= Google::Apis::BigqueryV2::TimePartitioning.new
  @gapi.time_partitioning.type = type
  patch_gapi! :time_partitioning
end

#type ⇒ String?

The type of the table like if its a TABLE, VIEW or SNAPSHOT etc.,



169
170
171
172
 
# File 'lib/google/cloud/bigquery/table.rb', line 169

def type
  return nil if reference?
  @gapi.type
end

#update_policy {|policy| ... } ⇒ Policy

Updates the Cloud IAM access control policy for the table. The latest policy will be read from the service. See also #policy.

Examples:

Update the policy by passing a block.

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

table.update_policy do |p|
  p.grant role: "roles/viewer", members: "user:[email protected]"
  p.revoke role: "roles/editor", members: "user:[email protected]"
  p.revoke role: "roles/owner"
end # 2 API calls

Yields:

  • (policy) —

    A block for updating the policy. The latest policy will be read from the service and passed to the block. After the block completes, the modified policy will be written to the service.

Yield Parameters:

  • policy (Policy) —

    The mutable Policy for the table.

Raises:

  • (ArgumentError)

See Also:



1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
 
# File 'lib/google/cloud/bigquery/table.rb', line 1680

def update_policy
  raise ArgumentError, "A block updating the policy must be provided" unless block_given?
  ensure_service!
  gapi = service.get_table_policy dataset_id, table_id
  policy = Policy.from_gapi gapi
  yield policy
  # TODO: Check for changes before calling RPC
  gapi = service.set_table_policy dataset_id, table_id, policy.to_gapi
  Policy.from_gapi(gapi).freeze
end

#view? ⇒ Boolean?

Checks if the table's type is VIEW, indicating that the table represents a BigQuery logical view. See Dataset#create_view.

See Also:



892
893
894
895
 
# File 'lib/google/cloud/bigquery/table.rb', line 892

def view?
  return nil if reference?
  @gapi.type == "VIEW"
end