Class: Gcloud::Bigquery::Table
- Inherits:
-
Object
- Object
- Gcloud::Bigquery::Table
- Defined in:
- lib/gcloud/bigquery/table.rb,
lib/gcloud/bigquery/table/list.rb,
lib/gcloud/bigquery/table/schema.rb
Overview
# Table
A named resource representing a BigQuery table that holds zero or more records. Every table is defined by a schema that may contain nested and repeated fields.
Defined Under Namespace
Instance Attribute Summary collapse
Attributes collapse
-
#api_url ⇒ Object
A URL that can be used to access the dataset using the REST API.
-
#created_at ⇒ Object
The time when this table was created.
-
#dataset_id ⇒ Object
The ID of the ‘Dataset` containing this table.
-
#description ⇒ Object
The description of the table.
-
#description=(new_description) ⇒ Object
Updates the description of the table.
-
#etag ⇒ Object
A string hash of the dataset.
-
#expires_at ⇒ Object
The time when this table expires.
-
#fields ⇒ Object
The fields of the table.
-
#headers ⇒ Object
The names of the columns in the table.
-
#id ⇒ Object
The combined Project ID, Dataset ID, and Table ID for this table, in the format specified by the [Query Reference](cloud.google.com/bigquery/query-reference#from): ‘project_name:datasetId.tableId`.
-
#location ⇒ Object
The geographic location where the table should reside.
-
#modified_at ⇒ Object
The date when this table was last modified.
-
#name ⇒ Object
The name of the table.
-
#name=(new_name) ⇒ Object
Updates the name of the table.
-
#project_id ⇒ Object
The ID of the ‘Project` containing this table.
-
#query_id ⇒ Object
The value returned by #id, wrapped in square brackets if the Project ID contains dashes, as specified by the [Query Reference](cloud.google.com/bigquery/query-reference#from).
-
#schema(replace: false) {|schema| ... } ⇒ Object
Returns the table’s schema as hash containing the keys and values returned by the Google Cloud BigQuery [Rest API ](cloud.google.com/bigquery/docs/reference/v2/tables#resource).
-
#schema=(new_schema) ⇒ Object
Updates the schema of the table.
-
#table? ⇒ Boolean
Checks if the table’s type is “TABLE”.
-
#table_id ⇒ Object
A unique ID for this table.
-
#table_ref ⇒ Object
The gapi fragment containing the Project ID, Dataset ID, and Table ID as a camel-cased hash.
-
#view? ⇒ Boolean
Checks if the table’s type is “VIEW”.
Data collapse
-
#bytes_count ⇒ Object
The number of bytes in the table.
-
#copy(destination_table, create: nil, write: nil, dryrun: nil) ⇒ Gcloud::Bigquery::CopyJob
Copies the data from the table to another table.
-
#data(token: nil, max: nil, start: nil) ⇒ Gcloud::Bigquery::Data
Retrieves data from the table.
-
#extract(extract_url, format: nil, compression: nil, delimiter: nil, header: nil, dryrun: nil) ⇒ Gcloud::Bigquery::ExtractJob
Extract the data from the table to a Google Cloud Storage file.
-
#insert(rows, skip_invalid: nil, ignore_unknown: nil) ⇒ Gcloud::Bigquery::InsertResponse
Inserts data into the table for near-immediate querying, without the need to complete a #load operation before the data can appear in query results.
-
#link(source_url, create: nil, write: nil, dryrun: nil) ⇒ Gcloud::Bigquery::Job
Links the table to a source table identified by a URI.
-
#load(file, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, dryrun: nil) ⇒ Gcloud::Bigquery::LoadJob
Loads data into the table.
-
#rows_count ⇒ Object
The number of rows in the table.
Lifecycle collapse
- .from_gapi(gapi, conn) ⇒ Object
-
#delete ⇒ Boolean
Permanently deletes the table.
-
#reload! ⇒ Object
(also: #refresh!)
Reloads the table with current data from the BigQuery service.
Instance Method Summary collapse
-
#initialize ⇒ Table
constructor
A new instance of Table.
Constructor Details
#initialize ⇒ Table
Returns a new instance of Table.
77 78 79 80 |
# File 'lib/gcloud/bigquery/table.rb', line 77 def initialize @connection = nil @gapi = {} end |
Instance Attribute Details
#connection ⇒ Object
69 70 71 |
# File 'lib/gcloud/bigquery/table.rb', line 69 def connection @connection end |
#gapi ⇒ Object
73 74 75 |
# File 'lib/gcloud/bigquery/table.rb', line 73 def gapi @gapi end |
Class Method Details
.from_gapi(gapi, conn) ⇒ Object
857 858 859 860 861 862 863 |
# File 'lib/gcloud/bigquery/table.rb', line 857 def self.from_gapi gapi, conn klass = class_for gapi klass.new.tap do |f| f.gapi = gapi f.connection = conn end end |
Instance Method Details
#api_url ⇒ Object
A URL that can be used to access the dataset using the REST API.
189 190 191 192 |
# File 'lib/gcloud/bigquery/table.rb', line 189 def api_url ensure_full_data! @gapi["selfLink"] end |
#bytes_count ⇒ Object
The number of bytes in the table.
218 219 220 221 |
# File 'lib/gcloud/bigquery/table.rb', line 218 def bytes_count ensure_full_data! @gapi["numBytes"] end |
#copy(destination_table, create: nil, write: nil, dryrun: nil) ⇒ Gcloud::Bigquery::CopyJob
Copies the data from the table to another table. The destination table argument can also be a string identifier as specified by the [Query Reference](cloud.google.com/bigquery/query-reference#from): ‘project_name:datasetId.tableId`. This is useful for referencing tables in other projects and datasets.
492 493 494 495 496 497 498 499 500 501 502 503 |
# File 'lib/gcloud/bigquery/table.rb', line 492 def copy destination_table, create: nil, write: nil, dryrun: nil ensure_connection! = { create: create, write: write, dryrun: dryrun } resp = connection.copy_table table_ref, get_table_ref(destination_table), if resp.success? Job.from_gapi resp.data, connection else fail ApiError.from_response(resp) end end |
#created_at ⇒ Object
The time when this table was created.
238 239 240 241 |
# File 'lib/gcloud/bigquery/table.rb', line 238 def created_at ensure_full_data! Time.at(@gapi["creationTime"] / 1000.0) end |
#data(token: nil, max: nil, start: nil) ⇒ Gcloud::Bigquery::Data
Retrieves data from the table.
428 429 430 431 432 433 434 435 436 437 |
# File 'lib/gcloud/bigquery/table.rb', line 428 def data token: nil, max: nil, start: nil ensure_connection! = { token: token, max: max, start: start } resp = connection.list_tabledata dataset_id, table_id, if resp.success? Data.from_response resp, self else fail ApiError.from_response(resp) end end |
#dataset_id ⇒ Object
The ID of the ‘Dataset` containing this table.
98 99 100 |
# File 'lib/gcloud/bigquery/table.rb', line 98 def dataset_id @gapi["tableReference"]["datasetId"] end |
#delete ⇒ Boolean
Permanently deletes the table.
829 830 831 832 833 834 835 836 837 |
# File 'lib/gcloud/bigquery/table.rb', line 829 def delete ensure_connection! resp = connection.delete_table dataset_id, table_id if resp.success? true else fail ApiError.from_response(resp) end end |
#description ⇒ Object
The description of the table.
199 200 201 202 |
# File 'lib/gcloud/bigquery/table.rb', line 199 def description ensure_full_data! @gapi["description"] end |
#description=(new_description) ⇒ Object
Updates the description of the table.
209 210 211 |
# File 'lib/gcloud/bigquery/table.rb', line 209 def description= new_description patch_gapi! description: new_description end |
#etag ⇒ Object
A string hash of the dataset.
179 180 181 182 |
# File 'lib/gcloud/bigquery/table.rb', line 179 def etag ensure_full_data! @gapi["etag"] end |
#expires_at ⇒ Object
The time when this table expires. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.
250 251 252 253 254 |
# File 'lib/gcloud/bigquery/table.rb', line 250 def expires_at ensure_full_data! return nil if @gapi["expirationTime"].nil? Time.at(@gapi["expirationTime"] / 1000.0) end |
#extract(extract_url, format: nil, compression: nil, delimiter: nil, header: nil, dryrun: nil) ⇒ Gcloud::Bigquery::ExtractJob
Extract the data from the table to a Google Cloud Storage file.
584 585 586 587 588 589 590 591 592 593 594 595 |
# File 'lib/gcloud/bigquery/table.rb', line 584 def extract extract_url, format: nil, compression: nil, delimiter: nil, header: nil, dryrun: nil ensure_connection! = { format: format, compression: compression, delimiter: delimiter, header: header, dryrun: dryrun } resp = connection.extract_table table_ref, extract_url, if resp.success? Job.from_gapi resp.data, connection else fail ApiError.from_response(resp) end end |
#fields ⇒ Object
The fields of the table.
385 386 387 388 389 390 |
# File 'lib/gcloud/bigquery/table.rb', line 385 def fields f = schema["fields"] f = f.to_hash if f.respond_to? :to_hash f = [] if f.nil? f end |
#headers ⇒ Object
The names of the columns in the table.
397 398 399 |
# File 'lib/gcloud/bigquery/table.rb', line 397 def headers fields.map { |f| f["name"] } end |
#id ⇒ Object
The combined Project ID, Dataset ID, and Table ID for this table, in the format specified by the [Query Reference](cloud.google.com/bigquery/query-reference#from): ‘project_name:datasetId.tableId`. To use this value in queries see #query_id.
130 131 132 |
# File 'lib/gcloud/bigquery/table.rb', line 130 def id @gapi["id"] end |
#insert(rows, skip_invalid: nil, ignore_unknown: nil) ⇒ Gcloud::Bigquery::InsertResponse
Inserts data into the table for near-immediate querying, without the need to complete a #load operation before the data can appear in query results.
800 801 802 803 804 805 806 807 808 809 810 |
# File 'lib/gcloud/bigquery/table.rb', line 800 def insert rows, skip_invalid: nil, ignore_unknown: nil rows = [rows] if rows.is_a? Hash ensure_connection! = { skip_invalid: skip_invalid, ignore_unknown: ignore_unknown } resp = connection.insert_tabledata dataset_id, table_id, rows, if resp.success? InsertResponse.from_gapi rows, resp.data else fail ApiError.from_response(resp) end end |
#link(source_url, create: nil, write: nil, dryrun: nil) ⇒ Gcloud::Bigquery::Job
Links the table to a source table identified by a URI.
532 533 534 535 536 537 538 539 540 541 |
# File 'lib/gcloud/bigquery/table.rb', line 532 def link source_url, create: nil, write: nil, dryrun: nil ensure_connection! = { create: create, write: write, dryrun: dryrun } resp = connection.link_table table_ref, source_url, if resp.success? Job.from_gapi resp.data, connection else fail ApiError.from_response(resp) end end |
#load(file, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, dryrun: nil) ⇒ Gcloud::Bigquery::LoadJob
Loads data into the table. You can pass a gcloud storage file path or a gcloud storage file instance. Or, you can upload a file directly. See [Loading Data with a POST Request]( cloud.google.com/bigquery/loading-data-post-request#multipart).
### A note about large direct uploads
You may encounter a Broken pipe (Errno::EPIPE) error when attempting to upload large files. To avoid this problem, add the [httpclient](rubygems.org/gems/httpclient) gem to your project, and the line (or lines) of configuration shown below. These lines must execute after you require gcloud but before you make your first gcloud connection. The first statement configures [Faraday](rubygems.org/gems/faraday) to use httpclient. The second statement, which should only be added if you are using a version of Faraday at or above 0.9.2, is a workaround for [this gzip issue](github.com/GoogleCloudPlatform/gcloud-ruby/issues/367).
748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 |
# File 'lib/gcloud/bigquery/table.rb', line 748 def load file, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, dryrun: nil ensure_connection! = { format: format, create: create, write: write, projection_fields: projection_fields, jagged_rows: jagged_rows, quoted_newlines: quoted_newlines, encoding: encoding, delimiter: delimiter, ignore_unknown: ignore_unknown, max_bad_records: max_bad_records, quote: quote, skip_leading: skip_leading, dryrun: dryrun } return load_storage(file, ) if storage_url? file return load_local(file, ) if local_file? file fail Gcloud::Bigquery::Error, "Don't know how to load #{file}" end |
#location ⇒ Object
The geographic location where the table should reside. Possible values include EU and US. The default value is US.
290 291 292 293 |
# File 'lib/gcloud/bigquery/table.rb', line 290 def location ensure_full_data! @gapi["location"] end |
#modified_at ⇒ Object
The date when this table was last modified.
261 262 263 264 |
# File 'lib/gcloud/bigquery/table.rb', line 261 def modified_at ensure_full_data! Time.at(@gapi["lastModifiedTime"] / 1000.0) end |
#name ⇒ Object
The name of the table.
161 162 163 |
# File 'lib/gcloud/bigquery/table.rb', line 161 def name @gapi["friendlyName"] end |
#name=(new_name) ⇒ Object
Updates the name of the table.
170 171 172 |
# File 'lib/gcloud/bigquery/table.rb', line 170 def name= new_name patch_gapi! name: new_name end |
#project_id ⇒ Object
The ID of the ‘Project` containing this table.
107 108 109 |
# File 'lib/gcloud/bigquery/table.rb', line 107 def project_id @gapi["tableReference"]["projectId"] end |
#query_id ⇒ Object
The value returned by #id, wrapped in square brackets if the Project ID contains dashes, as specified by the [Query Reference](cloud.google.com/bigquery/query-reference#from). Useful in queries.
152 153 154 |
# File 'lib/gcloud/bigquery/table.rb', line 152 def query_id project_id["-"] ? "[#{id}]" : id end |
#reload! ⇒ Object Also known as: refresh!
Reloads the table with current data from the BigQuery service.
844 845 846 847 848 849 850 851 852 |
# File 'lib/gcloud/bigquery/table.rb', line 844 def reload! ensure_connection! resp = connection.get_table dataset_id, table_id if resp.success? @gapi = resp.data else fail ApiError.from_response(resp) end end |
#rows_count ⇒ Object
The number of rows in the table.
228 229 230 231 |
# File 'lib/gcloud/bigquery/table.rb', line 228 def rows_count ensure_full_data! @gapi["numRows"] end |
#schema(replace: false) {|schema| ... } ⇒ Object
Returns the table’s schema as hash containing the keys and values returned by the Google Cloud BigQuery [Rest API ](cloud.google.com/bigquery/docs/reference/v2/tables#resource). This method can also be used to set, replace, or add to the schema by passing a block. See Schema for available methods. To set the schema by passing a hash instead, use #schema=.
329 330 331 332 333 334 335 336 337 338 339 |
# File 'lib/gcloud/bigquery/table.rb', line 329 def schema replace: false ensure_full_data! g = @gapi g = g.to_hash if g.respond_to? :to_hash s = g["schema"] ||= {} return s unless block_given? s = nil if replace schema_builder = Schema.new s yield schema_builder self.schema = schema_builder.schema if schema_builder.changed? end |
#schema=(new_schema) ⇒ Object
Updates the schema of the table. To update the schema using a block instead, use #schema.
376 377 378 |
# File 'lib/gcloud/bigquery/table.rb', line 376 def schema= new_schema patch_gapi! schema: new_schema end |
#table? ⇒ Boolean
Checks if the table’s type is “TABLE”.
271 272 273 |
# File 'lib/gcloud/bigquery/table.rb', line 271 def table? @gapi["type"] == "TABLE" end |
#table_id ⇒ Object
A unique ID for this table. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.
89 90 91 |
# File 'lib/gcloud/bigquery/table.rb', line 89 def table_id @gapi["tableReference"]["tableId"] end |
#table_ref ⇒ Object
The gapi fragment containing the Project ID, Dataset ID, and Table ID as a camel-cased hash.
115 116 117 118 119 |
# File 'lib/gcloud/bigquery/table.rb', line 115 def table_ref table_ref = @gapi["tableReference"] table_ref = table_ref.to_hash if table_ref.respond_to? :to_hash table_ref end |
#view? ⇒ Boolean
Checks if the table’s type is “VIEW”.
280 281 282 |
# File 'lib/gcloud/bigquery/table.rb', line 280 def view? @gapi["type"] == "VIEW" end |