Class: Gcloud::Bigquery::Table
- Inherits:
-
Object
- Object
- Gcloud::Bigquery::Table
- Defined in:
- lib/gcloud/bigquery/table.rb,
lib/gcloud/bigquery/table/list.rb,
lib/gcloud/bigquery/table/schema.rb
Overview
# Table
A named resource representing a BigQuery table that holds zero or more records. Every table is defined by a schema that may contain nested and repeated fields.
Defined Under Namespace
Instance Attribute Summary collapse
Attributes collapse
-
#api_url ⇒ Object
A URL that can be used to access the dataset using the REST API.
-
#created_at ⇒ Object
The time when this table was created.
-
#dataset_id ⇒ Object
The ID of the
Datasetcontaining this table. -
#description ⇒ Object
The description of the table.
-
#description=(new_description) ⇒ Object
Updates the description of the table.
-
#etag ⇒ Object
A string hash of the dataset.
-
#expires_at ⇒ Object
The time when this table expires.
-
#fields ⇒ Object
The fields of the table.
-
#headers ⇒ Object
The names of the columns in the table.
-
#id ⇒ Object
The combined Project ID, Dataset ID, and Table ID for this table, in the format specified by the [Query Reference](cloud.google.com/bigquery/query-reference#from):
project_name:datasetId.tableId. -
#location ⇒ Object
The geographic location where the table should reside.
-
#modified_at ⇒ Object
The date when this table was last modified.
-
#name ⇒ Object
The name of the table.
-
#name=(new_name) ⇒ Object
Updates the name of the table.
-
#project_id ⇒ Object
The ID of the
Projectcontaining this table. -
#query_id ⇒ Object
The value returned by #id, wrapped in square brackets if the Project ID contains dashes, as specified by the [Query Reference](cloud.google.com/bigquery/query-reference#from).
-
#schema(replace: false) {|schema| ... } ⇒ Object
Returns the table’s schema as hash containing the keys and values returned by the Google Cloud BigQuery [Rest API ](cloud.google.com/bigquery/docs/reference/v2/tables#resource).
-
#schema=(new_schema) ⇒ Object
Updates the schema of the table.
-
#table? ⇒ Boolean
Checks if the table’s type is “TABLE”.
-
#table_id ⇒ Object
A unique ID for this table.
-
#table_ref ⇒ Object
The gapi fragment containing the Project ID, Dataset ID, and Table ID as a camel-cased hash.
-
#view? ⇒ Boolean
Checks if the table’s type is “VIEW”.
Data collapse
-
#bytes_count ⇒ Object
The number of bytes in the table.
-
#copy(destination_table, create: nil, write: nil, dryrun: nil) ⇒ Gcloud::Bigquery::CopyJob
Copies the data from the table to another table.
-
#data(token: nil, max: nil, start: nil) ⇒ Gcloud::Bigquery::Data
Retrieves data from the table.
-
#extract(extract_url, format: nil, compression: nil, delimiter: nil, header: nil, dryrun: nil) ⇒ Gcloud::Bigquery::ExtractJob
Extract the data from the table to a Google Cloud Storage file.
-
#insert(rows, skip_invalid: nil, ignore_unknown: nil) ⇒ Gcloud::Bigquery::InsertResponse
Inserts data into the table for near-immediate querying, without the need to complete a #load operation before the data can appear in query results.
-
#link(source_url, create: nil, write: nil, dryrun: nil) ⇒ Gcloud::Bigquery::Job
Links the table to a source table identified by a URI.
-
#load(file, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, dryrun: nil, chunk_size: nil) ⇒ Gcloud::Bigquery::LoadJob
Loads data into the table.
-
#rows_count ⇒ Object
The number of rows in the table.
Lifecycle collapse
- .from_gapi(gapi, conn) ⇒ Object
-
#delete ⇒ Boolean
Permanently deletes the table.
-
#reload! ⇒ Object
(also: #refresh!)
Reloads the table with current data from the BigQuery service.
Instance Method Summary collapse
-
#initialize ⇒ Table
constructor
A new instance of Table.
Constructor Details
#initialize ⇒ Table
Returns a new instance of Table.
77 78 79 80 |
# File 'lib/gcloud/bigquery/table.rb', line 77 def initialize @connection = nil @gapi = {} end |
Instance Attribute Details
#connection ⇒ Object
69 70 71 |
# File 'lib/gcloud/bigquery/table.rb', line 69 def connection @connection end |
#gapi ⇒ Object
73 74 75 |
# File 'lib/gcloud/bigquery/table.rb', line 73 def gapi @gapi end |
Class Method Details
.from_gapi(gapi, conn) ⇒ Object
882 883 884 885 886 887 888 |
# File 'lib/gcloud/bigquery/table.rb', line 882 def self.from_gapi gapi, conn klass = class_for gapi klass.new.tap do |f| f.gapi = gapi f.connection = conn end end |
Instance Method Details
#api_url ⇒ Object
A URL that can be used to access the dataset using the REST API.
189 190 191 192 |
# File 'lib/gcloud/bigquery/table.rb', line 189 def api_url ensure_full_data! @gapi["selfLink"] end |
#bytes_count ⇒ Object
The number of bytes in the table.
218 219 220 221 |
# File 'lib/gcloud/bigquery/table.rb', line 218 def bytes_count ensure_full_data! @gapi["numBytes"] end |
#copy(destination_table, create: nil, write: nil, dryrun: nil) ⇒ Gcloud::Bigquery::CopyJob
Copies the data from the table to another table. The destination table argument can also be a string identifier as specified by the [Query Reference](cloud.google.com/bigquery/query-reference#from): project_name:datasetId.tableId. This is useful for referencing tables in other projects and datasets.
507 508 509 510 511 512 513 514 515 516 517 518 |
# File 'lib/gcloud/bigquery/table.rb', line 507 def copy destination_table, create: nil, write: nil, dryrun: nil ensure_connection! = { create: create, write: write, dryrun: dryrun } resp = connection.copy_table table_ref, get_table_ref(destination_table), if resp.success? Job.from_gapi resp.data, connection else fail ApiError.from_response(resp) end end |
#created_at ⇒ Object
The time when this table was created.
238 239 240 241 |
# File 'lib/gcloud/bigquery/table.rb', line 238 def created_at ensure_full_data! Time.at(@gapi["creationTime"] / 1000.0) end |
#data(token: nil, max: nil, start: nil) ⇒ Gcloud::Bigquery::Data
Retrieves data from the table.
443 444 445 446 447 448 449 450 451 452 |
# File 'lib/gcloud/bigquery/table.rb', line 443 def data token: nil, max: nil, start: nil ensure_connection! = { token: token, max: max, start: start } resp = connection.list_tabledata dataset_id, table_id, if resp.success? Data.from_response resp, self else fail ApiError.from_response(resp) end end |
#dataset_id ⇒ Object
The ID of the Dataset containing this table.
98 99 100 |
# File 'lib/gcloud/bigquery/table.rb', line 98 def dataset_id @gapi["tableReference"]["datasetId"] end |
#delete ⇒ Boolean
Permanently deletes the table.
854 855 856 857 858 859 860 861 862 |
# File 'lib/gcloud/bigquery/table.rb', line 854 def delete ensure_connection! resp = connection.delete_table dataset_id, table_id if resp.success? true else fail ApiError.from_response(resp) end end |
#description ⇒ Object
The description of the table.
199 200 201 202 |
# File 'lib/gcloud/bigquery/table.rb', line 199 def description ensure_full_data! @gapi["description"] end |
#description=(new_description) ⇒ Object
Updates the description of the table.
209 210 211 |
# File 'lib/gcloud/bigquery/table.rb', line 209 def description= new_description patch_gapi! description: new_description end |
#etag ⇒ Object
A string hash of the dataset.
179 180 181 182 |
# File 'lib/gcloud/bigquery/table.rb', line 179 def etag ensure_full_data! @gapi["etag"] end |
#expires_at ⇒ Object
The time when this table expires. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.
250 251 252 253 254 |
# File 'lib/gcloud/bigquery/table.rb', line 250 def expires_at ensure_full_data! return nil if @gapi["expirationTime"].nil? Time.at(@gapi["expirationTime"] / 1000.0) end |
#extract(extract_url, format: nil, compression: nil, delimiter: nil, header: nil, dryrun: nil) ⇒ Gcloud::Bigquery::ExtractJob
Extract the data from the table to a Google Cloud Storage file.
599 600 601 602 603 604 605 606 607 608 609 610 |
# File 'lib/gcloud/bigquery/table.rb', line 599 def extract extract_url, format: nil, compression: nil, delimiter: nil, header: nil, dryrun: nil ensure_connection! = { format: format, compression: compression, delimiter: delimiter, header: header, dryrun: dryrun } resp = connection.extract_table table_ref, extract_url, if resp.success? Job.from_gapi resp.data, connection else fail ApiError.from_response(resp) end end |
#fields ⇒ Object
The fields of the table.
385 386 387 388 389 390 |
# File 'lib/gcloud/bigquery/table.rb', line 385 def fields f = schema["fields"] f = f.to_hash if f.respond_to? :to_hash f = [] if f.nil? f end |
#headers ⇒ Object
The names of the columns in the table.
397 398 399 |
# File 'lib/gcloud/bigquery/table.rb', line 397 def headers fields.map { |f| f["name"] } end |
#id ⇒ Object
The combined Project ID, Dataset ID, and Table ID for this table, in the format specified by the [Query Reference](cloud.google.com/bigquery/query-reference#from): project_name:datasetId.tableId. To use this value in queries see #query_id.
130 131 132 |
# File 'lib/gcloud/bigquery/table.rb', line 130 def id @gapi["id"] end |
#insert(rows, skip_invalid: nil, ignore_unknown: nil) ⇒ Gcloud::Bigquery::InsertResponse
Inserts data into the table for near-immediate querying, without the need to complete a #load operation before the data can appear in query results.
825 826 827 828 829 830 831 832 833 834 835 |
# File 'lib/gcloud/bigquery/table.rb', line 825 def insert rows, skip_invalid: nil, ignore_unknown: nil rows = [rows] if rows.is_a? Hash ensure_connection! = { skip_invalid: skip_invalid, ignore_unknown: ignore_unknown } resp = connection.insert_tabledata dataset_id, table_id, rows, if resp.success? InsertResponse.from_gapi rows, resp.data else fail ApiError.from_response(resp) end end |
#link(source_url, create: nil, write: nil, dryrun: nil) ⇒ Gcloud::Bigquery::Job
Links the table to a source table identified by a URI.
547 548 549 550 551 552 553 554 555 556 |
# File 'lib/gcloud/bigquery/table.rb', line 547 def link source_url, create: nil, write: nil, dryrun: nil ensure_connection! = { create: create, write: write, dryrun: dryrun } resp = connection.link_table table_ref, source_url, if resp.success? Job.from_gapi resp.data, connection else fail ApiError.from_response(resp) end end |
#load(file, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, dryrun: nil, chunk_size: nil) ⇒ Gcloud::Bigquery::LoadJob
Loads data into the table. You can pass a gcloud storage file path or a gcloud storage file instance. Or, you can upload a file directly. See [Loading Data with a POST Request]( cloud.google.com/bigquery/loading-data-post-request#multipart).
A chunk_size value can be provided in the options to be used in resumable uploads. This value is the number of bytes per chunk and must be divisible by 256KB. If it is not divisible by 256KB then it will be lowered to the nearest acceptable value.
### A note about large direct uploads
You may encounter a Broken pipe (Errno::EPIPE) error when attempting to upload large files. To avoid this problem, add the [httpclient](rubygems.org/gems/httpclient) gem to your project, and the line (or lines) of configuration shown below. These lines must execute after you require gcloud but before you make your first gcloud connection. The first statement configures [Faraday](rubygems.org/gems/faraday) to use httpclient. The second statement, which should only be added if you are using a version of Faraday at or above 0.9.2, is a workaround for [this gzip issue](github.com/GoogleCloudPlatform/gcloud-ruby/issues/367).
772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 |
# File 'lib/gcloud/bigquery/table.rb', line 772 def load file, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, dryrun: nil, chunk_size: nil ensure_connection! = { format: format, create: create, write: write, projection_fields: projection_fields, jagged_rows: jagged_rows, quoted_newlines: quoted_newlines, encoding: encoding, delimiter: delimiter, ignore_unknown: ignore_unknown, chunk_size: chunk_size, max_bad_records: max_bad_records, quote: quote, skip_leading: skip_leading, dryrun: dryrun } return load_storage(file, ) if storage_url? file return load_local(file, ) if local_file? file fail Gcloud::Bigquery::Error, "Don't know how to load #{file}" end |
#location ⇒ Object
The geographic location where the table should reside. Possible values include EU and US. The default value is US.
290 291 292 293 |
# File 'lib/gcloud/bigquery/table.rb', line 290 def location ensure_full_data! @gapi["location"] end |
#modified_at ⇒ Object
The date when this table was last modified.
261 262 263 264 |
# File 'lib/gcloud/bigquery/table.rb', line 261 def modified_at ensure_full_data! Time.at(@gapi["lastModifiedTime"] / 1000.0) end |
#name ⇒ Object
The name of the table.
161 162 163 |
# File 'lib/gcloud/bigquery/table.rb', line 161 def name @gapi["friendlyName"] end |
#name=(new_name) ⇒ Object
Updates the name of the table.
170 171 172 |
# File 'lib/gcloud/bigquery/table.rb', line 170 def name= new_name patch_gapi! name: new_name end |
#project_id ⇒ Object
The ID of the Project containing this table.
107 108 109 |
# File 'lib/gcloud/bigquery/table.rb', line 107 def project_id @gapi["tableReference"]["projectId"] end |
#query_id ⇒ Object
The value returned by #id, wrapped in square brackets if the Project ID contains dashes, as specified by the [Query Reference](cloud.google.com/bigquery/query-reference#from). Useful in queries.
152 153 154 |
# File 'lib/gcloud/bigquery/table.rb', line 152 def query_id project_id["-"] ? "[#{id}]" : id end |
#reload! ⇒ Object Also known as: refresh!
Reloads the table with current data from the BigQuery service.
869 870 871 872 873 874 875 876 877 |
# File 'lib/gcloud/bigquery/table.rb', line 869 def reload! ensure_connection! resp = connection.get_table dataset_id, table_id if resp.success? @gapi = resp.data else fail ApiError.from_response(resp) end end |
#rows_count ⇒ Object
The number of rows in the table.
228 229 230 231 |
# File 'lib/gcloud/bigquery/table.rb', line 228 def rows_count ensure_full_data! @gapi["numRows"] end |
#schema(replace: false) {|schema| ... } ⇒ Object
Returns the table’s schema as hash containing the keys and values returned by the Google Cloud BigQuery [Rest API ](cloud.google.com/bigquery/docs/reference/v2/tables#resource). This method can also be used to set, replace, or add to the schema by passing a block. See Schema for available methods. To set the schema by passing a hash instead, use #schema=.
329 330 331 332 333 334 335 336 337 338 339 |
# File 'lib/gcloud/bigquery/table.rb', line 329 def schema replace: false ensure_full_data! g = @gapi g = g.to_hash if g.respond_to? :to_hash s = g["schema"] ||= {} return s unless block_given? s = nil if replace schema_builder = Schema.new s yield schema_builder self.schema = schema_builder.schema if schema_builder.changed? end |
#schema=(new_schema) ⇒ Object
Updates the schema of the table. To update the schema using a block instead, use #schema.
376 377 378 |
# File 'lib/gcloud/bigquery/table.rb', line 376 def schema= new_schema patch_gapi! schema: new_schema end |
#table? ⇒ Boolean
Checks if the table’s type is “TABLE”.
271 272 273 |
# File 'lib/gcloud/bigquery/table.rb', line 271 def table? @gapi["type"] == "TABLE" end |
#table_id ⇒ Object
A unique ID for this table. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.
89 90 91 |
# File 'lib/gcloud/bigquery/table.rb', line 89 def table_id @gapi["tableReference"]["tableId"] end |
#table_ref ⇒ Object
The gapi fragment containing the Project ID, Dataset ID, and Table ID as a camel-cased hash.
115 116 117 118 119 |
# File 'lib/gcloud/bigquery/table.rb', line 115 def table_ref table_ref = @gapi["tableReference"] table_ref = table_ref.to_hash if table_ref.respond_to? :to_hash table_ref end |
#view? ⇒ Boolean
Checks if the table’s type is “VIEW”.
280 281 282 |
# File 'lib/gcloud/bigquery/table.rb', line 280 def view? @gapi["type"] == "VIEW" end |