Class: Google::Cloud::Bigquery::Table
- Inherits:
-
Object
- Object
- Google::Cloud::Bigquery::Table
- Defined in:
- lib/google/cloud/bigquery/table.rb,
lib/google/cloud/bigquery/table/list.rb,
lib/google/cloud/bigquery/table/async_inserter.rb
Overview
Table
A named resource representing a BigQuery table that holds zero or more records. Every table is defined by a schema that may contain nested and repeated fields.
The Table class can also represent a view, which is a virtual table defined by a SQL query. BigQuery's views are logical views, not materialized views, which means that the query that defines the view is re-executed every time the view is queried. Queries are billed according to the total amount of data in all table fields referenced directly or indirectly by the top-level query. (See #view?, #query, #query=, and Dataset#create_view.)
Direct Known Subclasses
Defined Under Namespace
Classes: AsyncInserter, List, Updater
Attributes collapse
-
#api_url ⇒ String?
A URL that can be used to access the table using the REST API.
-
#buffer_bytes ⇒ Integer?
A lower-bound estimate of the number of bytes currently in this table's streaming buffer, if one is present.
-
#buffer_oldest_at ⇒ Time?
The time of the oldest entry currently in this table's streaming buffer, if one is present.
-
#buffer_rows ⇒ Integer?
A lower-bound estimate of the number of rows currently in this table's streaming buffer, if one is present.
-
#clustering? ⇒ Boolean?
Checks if the table is clustered.
-
#clustering_fields ⇒ Array<String>?
One or more fields on which data should be clustered.
-
#created_at ⇒ Time?
The time when this table was created.
-
#dataset_id ⇒ String
The ID of the
Dataset
containing this table. -
#description ⇒ String?
A user-friendly description of the table.
-
#description=(new_description) ⇒ Object
Updates the user-friendly description of the table.
-
#encryption ⇒ EncryptionConfiguration?
The EncryptionConfiguration object that represents the custom encryption method used to protect the table.
-
#encryption=(value) ⇒ Object
Set the EncryptionConfiguration object that represents the custom encryption method used to protect the table.
-
#etag ⇒ String?
The ETag hash of the table.
-
#expires_at ⇒ Time?
The time when this table expires.
-
#external ⇒ External::DataSource?
The External::DataSource (or subclass) object that represents the external data source that the table represents.
-
#external=(external) ⇒ Object
Set the External::DataSource (or subclass) object that represents the external data source that the table represents.
-
#external? ⇒ Boolean?
Checks if the table's type is "EXTERNAL", indicating that the table represents an External Data Source.
-
#fields ⇒ Array<Schema::Field>?
The fields of the table, obtained from its schema.
-
#headers ⇒ Array<Symbol>?
The names of the columns in the table, obtained from its schema.
-
#id ⇒ String?
The combined Project ID, Dataset ID, and Table ID for this table, in the format specified by the Legacy SQL Query Reference (
project-name:dataset_id.table_id
). -
#labels ⇒ Hash<String, String>?
A hash of user-provided labels associated with this table.
-
#labels=(labels) ⇒ Object
Updates the hash of user-provided labels associated with this table.
-
#location ⇒ String?
The geographic location where the table should reside.
-
#modified_at ⇒ Time?
The date when this table was last modified.
-
#name ⇒ String?
The name of the table.
-
#name=(new_name) ⇒ Object
Updates the name of the table.
-
#project_id ⇒ String
The ID of the
Project
containing this table. -
#query ⇒ String
The query that executes each time the view is loaded.
-
#query_id(standard_sql: nil, legacy_sql: nil) ⇒ String
The value returned by #id, wrapped in backticks (Standard SQL) or s quare brackets (Legacy SQL) to accommodate project IDs containing dashes.
-
#query_legacy_sql? ⇒ Boolean
Checks if the view's query is using legacy sql.
-
#query_standard_sql? ⇒ Boolean
Checks if the view's query is using standard sql.
-
#query_udfs ⇒ Array<String>
The user-defined function resources used in the view's query.
-
#schema(replace: false) {|schema| ... } ⇒ Google::Cloud::Bigquery::Schema?
Returns the table's schema.
-
#table? ⇒ Boolean?
Checks if the table's type is "TABLE".
-
#table_id ⇒ String
A unique ID for this table.
-
#time_partitioning? ⇒ Boolean?
Checks if the table is time-partitioned.
-
#time_partitioning_expiration ⇒ Integer?
The expiration for the table partitions, if any, in seconds.
-
#time_partitioning_expiration=(expiration) ⇒ Object
Sets the partition expiration for the table.
-
#time_partitioning_field ⇒ String?
The field on which the table is partitioned, if any.
-
#time_partitioning_field=(field) ⇒ Object
Sets the field on which to partition the table.
-
#time_partitioning_type ⇒ String?
The period for which the table is partitioned, if any.
-
#time_partitioning_type=(type) ⇒ Object
Sets the partitioning for the table.
-
#view? ⇒ Boolean?
Checks if the table's type is "VIEW", indicating that the table represents a BigQuery view.
Data collapse
-
#bytes_count ⇒ Integer?
The number of bytes in the table.
-
#copy(destination_table, create: nil, write: nil) {|job| ... } ⇒ Boolean
Copies the data from the table to another table using a synchronous method that blocks for a response.
-
#copy_job(destination_table, create: nil, write: nil, job_id: nil, prefix: nil, labels: nil, dryrun: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::CopyJob
Copies the data from the table to another table using an asynchronous method.
-
#data(token: nil, max: nil, start: nil) ⇒ Google::Cloud::Bigquery::Data
Retrieves data from the table.
-
#extract(extract_url, format: nil, compression: nil, delimiter: nil, header: nil) {|job| ... } ⇒ Boolean
Extracts the data from the table to a Google Cloud Storage file using a synchronous method that blocks for a response.
-
#extract_job(extract_url, format: nil, compression: nil, delimiter: nil, header: nil, job_id: nil, prefix: nil, labels: nil, dryrun: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::ExtractJob
Extracts the data from the table to a Google Cloud Storage file using an asynchronous method.
-
#insert(rows, insert_ids: nil, skip_invalid: nil, ignore_unknown: nil) ⇒ Google::Cloud::Bigquery::InsertResponse
Inserts data into the table for near-immediate querying, without the need to complete a load operation before the data can appear in query results.
-
#insert_async(skip_invalid: nil, ignore_unknown: nil, max_bytes: 10000000, max_rows: 500, interval: 10, threads: 4) {|response| ... } ⇒ Table::AsyncInserter
Create an asynchronous inserter object used to insert rows in batches.
-
#load(files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, autodetect: nil, null_marker: nil) {|updater| ... } ⇒ Boolean
Loads data into the table.
-
#load_job(files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, job_id: nil, prefix: nil, labels: nil, autodetect: nil, null_marker: nil, dryrun: nil) {|load_job| ... } ⇒ Google::Cloud::Bigquery::LoadJob
Loads data into the table.
-
#rows_count ⇒ Integer?
The number of rows in the table.
Lifecycle collapse
-
#delete ⇒ Boolean
Permanently deletes the table.
-
#exists?(force: nil) ⇒ Boolean
Determines whether the table exists in the BigQuery service.
-
#query=(new_query) ⇒ Object
Updates the query that executes each time the view is loaded.
-
#reference? ⇒ Boolean
Whether the table was created without retrieving the resource representation from the BigQuery service.
-
#reload! ⇒ Google::Cloud::Bigquery::Table
(also: #refresh!)
Reloads the table with current data from the BigQuery service.
-
#resource? ⇒ Boolean
Whether the table was created with a resource representation from the BigQuery service.
-
#resource_full? ⇒ Boolean
Whether the table was created with a full resource representation from the BigQuery service.
-
#resource_partial? ⇒ Boolean
Whether the table was created with a partial resource representation from the BigQuery service by retrieval through Dataset#tables.
-
#set_query(query, standard_sql: nil, legacy_sql: nil, udfs: nil) ⇒ Object
Updates the query that executes each time the view is loaded.
Instance Method Details
#api_url ⇒ String?
A URL that can be used to access the table using the REST API.
477 478 479 480 481 |
# File 'lib/google/cloud/bigquery/table.rb', line 477 def api_url return nil if reference? ensure_full_data! @gapi.self_link end |
#buffer_bytes ⇒ Integer?
A lower-bound estimate of the number of bytes currently in this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
951 952 953 954 955 |
# File 'lib/google/cloud/bigquery/table.rb', line 951 def buffer_bytes return nil if reference? ensure_full_data! @gapi.streaming_buffer.estimated_bytes if @gapi.streaming_buffer end |
#buffer_oldest_at ⇒ Time?
The time of the oldest entry currently in this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
985 986 987 988 989 990 991 992 993 994 995 |
# File 'lib/google/cloud/bigquery/table.rb', line 985 def buffer_oldest_at return nil if reference? ensure_full_data! return nil unless @gapi.streaming_buffer oldest_entry_time = @gapi.streaming_buffer.oldest_entry_time begin ::Time.at(Integer(oldest_entry_time) / 1000.0) rescue StandardError nil end end |
#buffer_rows ⇒ Integer?
A lower-bound estimate of the number of rows currently in this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer.
969 970 971 972 973 |
# File 'lib/google/cloud/bigquery/table.rb', line 969 def buffer_rows return nil if reference? ensure_full_data! @gapi.streaming_buffer.estimated_rows if @gapi.streaming_buffer end |
#bytes_count ⇒ Integer?
The number of bytes in the table.
522 523 524 525 526 527 528 529 530 |
# File 'lib/google/cloud/bigquery/table.rb', line 522 def bytes_count return nil if reference? ensure_full_data! begin Integer @gapi.num_bytes rescue StandardError nil end end |
#clustering? ⇒ Boolean?
Checks if the table is clustered.
339 340 341 342 |
# File 'lib/google/cloud/bigquery/table.rb', line 339 def clustering? return nil if reference? !@gapi.clustering.nil? end |
#clustering_fields ⇒ Array<String>?
One or more fields on which data should be clustered. Must be specified with time-based partitioning, data in the table will be first partitioned and subsequently clustered. The order of the returned fields determines the sort order of the data.
See Google::Cloud::Bigquery::Table::Updater#clustering_fields=.
365 366 367 368 369 |
# File 'lib/google/cloud/bigquery/table.rb', line 365 def clustering_fields return nil if reference? ensure_full_data! @gapi.clustering.fields if clustering? end |
#copy(destination_table, create: nil, write: nil) {|job| ... } ⇒ Boolean
Copies the data from the table to another table using a synchronous method that blocks for a response. Timeouts and transient errors are generally handled as needed to complete the job. See also #copy_job.
The geographic location for the job ("US", "EU", etc.) can be set via CopyJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1358 1359 1360 1361 1362 1363 |
# File 'lib/google/cloud/bigquery/table.rb', line 1358 def copy destination_table, create: nil, write: nil, &block job = copy_job destination_table, create: create, write: write, &block job.wait_until_done! ensure_job_succeeded! job true end |
#copy_job(destination_table, create: nil, write: nil, job_id: nil, prefix: nil, labels: nil, dryrun: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::CopyJob
Copies the data from the table to another table using an asynchronous method. In this method, a CopyJob is immediately returned. The caller may poll the service by repeatedly calling Job#reload! and Job#done? to detect when the job is done, or simply block until the job is done by calling #Job#wait_until_done!. See also #copy.
The geographic location for the job ("US", "EU", etc.) can be set via CopyJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 |
# File 'lib/google/cloud/bigquery/table.rb', line 1274 def copy_job destination_table, create: nil, write: nil, job_id: nil, prefix: nil, labels: nil, dryrun: nil ensure_service! = { create: create, write: write, dryrun: dryrun, labels: labels, job_id: job_id, prefix: prefix } updater = CopyJob::Updater.( service, table_ref, Service.get_table_ref(destination_table, default_ref: table_ref), ) updater.location = location if location # may be table reference yield updater if block_given? job_gapi = updater.to_gapi gapi = service.copy_table job_gapi Job.from_gapi gapi, service end |
#created_at ⇒ Time?
The time when this table was created.
558 559 560 561 562 563 564 565 566 |
# File 'lib/google/cloud/bigquery/table.rb', line 558 def created_at return nil if reference? ensure_full_data! begin ::Time.at(Integer(@gapi.creation_time) / 1000.0) rescue StandardError nil end end |
#data(token: nil, max: nil, start: nil) ⇒ Google::Cloud::Bigquery::Data
Retrieves data from the table.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the data retrieval.
1170 1171 1172 1173 1174 1175 1176 1177 |
# File 'lib/google/cloud/bigquery/table.rb', line 1170 def data token: nil, max: nil, start: nil ensure_service! reload! unless resource_full? = { token: token, max: max, start: start } data_json = service.list_tabledata \ dataset_id, table_id, Data.from_gapi_json data_json, gapi, nil, service end |
#dataset_id ⇒ String
The ID of the Dataset
containing this table.
129 130 131 132 |
# File 'lib/google/cloud/bigquery/table.rb', line 129 def dataset_id return reference.dataset_id if reference? @gapi.table_reference.dataset_id end |
#delete ⇒ Boolean
Permanently deletes the table.
2082 2083 2084 2085 2086 2087 2088 |
# File 'lib/google/cloud/bigquery/table.rb', line 2082 def delete ensure_service! service.delete_table dataset_id, table_id # Set flag for #exists? @exists = false true end |
#description ⇒ String?
A user-friendly description of the table.
491 492 493 494 495 |
# File 'lib/google/cloud/bigquery/table.rb', line 491 def description return nil if reference? ensure_full_data! @gapi.description end |
#description=(new_description) ⇒ Object
Updates the user-friendly description of the table.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
508 509 510 511 512 |
# File 'lib/google/cloud/bigquery/table.rb', line 508 def description= new_description reload! unless resource_full? @gapi.update! description: new_description patch_gapi! :description end |
#encryption ⇒ EncryptionConfiguration?
The EncryptionConfiguration object that represents the custom encryption method used to protect the table. If not set, default encryption is used.
Present only if the table is using custom encryption.
855 856 857 858 859 860 861 |
# File 'lib/google/cloud/bigquery/table.rb', line 855 def encryption return nil if reference? ensure_full_data! return nil if @gapi.encryption_configuration.nil? EncryptionConfiguration.from_gapi(@gapi.encryption_configuration) .freeze end |
#encryption=(value) ⇒ Object
Set the EncryptionConfiguration object that represents the custom encryption method used to protect the table. If not set, default encryption is used.
Present only if the table is using custom encryption.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
882 883 884 885 886 |
# File 'lib/google/cloud/bigquery/table.rb', line 882 def encryption= value reload! unless resource_full? @gapi.encryption_configuration = value.to_gapi patch_gapi! :encryption_configuration end |
#etag ⇒ String?
The ETag hash of the table.
463 464 465 466 467 |
# File 'lib/google/cloud/bigquery/table.rb', line 463 def etag return nil if reference? ensure_full_data! @gapi.etag end |
#exists?(force: nil) ⇒ Boolean
Determines whether the table exists in the BigQuery service. The
result is cached locally. To refresh state, set force
to true
.
2138 2139 2140 2141 2142 2143 2144 2145 |
# File 'lib/google/cloud/bigquery/table.rb', line 2138 def exists? force: nil return gapi_exists? if force # If we have a value, return it return @exists unless @exists.nil? # Always true if we have a gapi object return true if resource? gapi_exists? end |
#expires_at ⇒ Time?
The time when this table expires. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.
578 579 580 581 582 583 584 585 586 |
# File 'lib/google/cloud/bigquery/table.rb', line 578 def expires_at return nil if reference? ensure_full_data! begin ::Time.at(Integer(@gapi.expiration_time) / 1000.0) rescue StandardError nil end end |
#external ⇒ External::DataSource?
The External::DataSource (or subclass) object that represents the external data source that the table represents. Data can be queried the table, even though the data is not stored in BigQuery. Instead of loading or streaming the data, this object references the external data source.
Present only if the table represents an External Data Source. See #external? and External::DataSource.
905 906 907 908 909 910 |
# File 'lib/google/cloud/bigquery/table.rb', line 905 def external return nil if reference? ensure_full_data! return nil if @gapi.external_data_configuration.nil? External.from_gapi(@gapi.external_data_configuration).freeze end |
#external=(external) ⇒ Object
Set the External::DataSource (or subclass) object that represents the external data source that the table represents. Data can be queried the table, even though the data is not stored in BigQuery. Instead of loading or streaming the data, this object references the external data source.
Use only if the table represents an External Data Source. See #external? and External::DataSource.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
933 934 935 936 937 |
# File 'lib/google/cloud/bigquery/table.rb', line 933 def external= external reload! unless resource_full? @gapi.external_data_configuration = external.to_gapi patch_gapi! :external_data_configuration end |
#external? ⇒ Boolean?
Checks if the table's type is "EXTERNAL", indicating that the table represents an External Data Source. See #external? and External::DataSource.
646 647 648 649 |
# File 'lib/google/cloud/bigquery/table.rb', line 646 def external? return nil if reference? @gapi.type == "EXTERNAL" end |
#extract(extract_url, format: nil, compression: nil, delimiter: nil, header: nil) {|job| ... } ⇒ Boolean
Extracts the data from the table to a Google Cloud Storage file using a synchronous method that blocks for a response. Timeouts and transient errors are generally handled as needed to complete the job. See also #extract_job.
The geographic location for the job ("US", "EU", etc.) can be set via ExtractJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 |
# File 'lib/google/cloud/bigquery/table.rb', line 1526 def extract extract_url, format: nil, compression: nil, delimiter: nil, header: nil, &block job = extract_job extract_url, format: format, compression: compression, delimiter: delimiter, header: header, &block job.wait_until_done! ensure_job_succeeded! job true end |
#extract_job(extract_url, format: nil, compression: nil, delimiter: nil, header: nil, job_id: nil, prefix: nil, labels: nil, dryrun: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::ExtractJob
Extracts the data from the table to a Google Cloud Storage file using an asynchronous method. In this method, an ExtractJob is immediately returned. The caller may poll the service by repeatedly calling Job#reload! and Job#done? to detect when the job is done, or simply block until the job is done by calling #Job#wait_until_done!. See also #extract.
The geographic location for the job ("US", "EU", etc.) can be set via ExtractJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 |
# File 'lib/google/cloud/bigquery/table.rb', line 1447 def extract_job extract_url, format: nil, compression: nil, delimiter: nil, header: nil, job_id: nil, prefix: nil, labels: nil, dryrun: nil ensure_service! = { format: format, compression: compression, delimiter: delimiter, header: header, dryrun: dryrun, job_id: job_id, prefix: prefix, labels: labels } updater = ExtractJob::Updater. service, table_ref, extract_url, updater.location = location if location # may be table reference yield updater if block_given? job_gapi = updater.to_gapi gapi = service.extract_table job_gapi Job.from_gapi gapi, service end |
#fields ⇒ Array<Schema::Field>?
The fields of the table, obtained from its schema.
813 814 815 816 |
# File 'lib/google/cloud/bigquery/table.rb', line 813 def fields return nil if reference? schema.fields end |
#headers ⇒ Array<Symbol>?
The names of the columns in the table, obtained from its schema.
836 837 838 839 |
# File 'lib/google/cloud/bigquery/table.rb', line 836 def headers return nil if reference? schema.headers end |
#id ⇒ String?
The combined Project ID, Dataset ID, and Table ID for this table, in
the format specified by the Legacy SQL Query
Reference
(project-name:dataset_id.table_id
). This is useful for referencing
tables in other projects and datasets. To use this value in queries
see #query_id.
384 385 386 387 |
# File 'lib/google/cloud/bigquery/table.rb', line 384 def id return nil if reference? @gapi.id end |
#insert(rows, insert_ids: nil, skip_invalid: nil, ignore_unknown: nil) ⇒ Google::Cloud::Bigquery::InsertResponse
Inserts data into the table for near-immediate querying, without the need to complete a load operation before the data can appear in query results.
1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 |
# File 'lib/google/cloud/bigquery/table.rb', line 1992 def insert rows, insert_ids: nil, skip_invalid: nil, ignore_unknown: nil rows = [rows] if rows.is_a? Hash insert_ids = Array insert_ids if insert_ids.count > 0 && insert_ids.count != rows.count raise ArgumentError, "insert_ids must be the same size as rows" end rows = [rows] if rows.is_a? Hash raise ArgumentError, "No rows provided" if rows.empty? ensure_service! = { skip_invalid: skip_invalid, ignore_unknown: ignore_unknown, insert_ids: insert_ids } gapi = service.insert_tabledata dataset_id, table_id, rows, InsertResponse.from_gapi rows, gapi end |
#insert_async(skip_invalid: nil, ignore_unknown: nil, max_bytes: 10000000, max_rows: 500, interval: 10, threads: 4) {|response| ... } ⇒ Table::AsyncInserter
Create an asynchronous inserter object used to insert rows in batches.
2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 |
# File 'lib/google/cloud/bigquery/table.rb', line 2055 def insert_async skip_invalid: nil, ignore_unknown: nil, max_bytes: 10000000, max_rows: 500, interval: 10, threads: 4, &block ensure_service! AsyncInserter.new self, skip_invalid: skip_invalid, ignore_unknown: ignore_unknown, max_bytes: max_bytes, max_rows: max_rows, interval: interval, threads: threads, &block end |
#labels ⇒ Hash<String, String>?
A hash of user-provided labels associated with this table. Labels are used to organize and group tables. See Using Labels.
The returned hash is frozen and changes are not allowed. Use #labels= to replace the entire hash.
687 688 689 690 691 692 |
# File 'lib/google/cloud/bigquery/table.rb', line 687 def labels return nil if reference? m = @gapi.labels m = m.to_h if m.respond_to? :to_h m.dup.freeze end |
#labels=(labels) ⇒ Object
Updates the hash of user-provided labels associated with this table. Labels are used to organize and group tables. See Using Labels.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
724 725 726 727 728 |
# File 'lib/google/cloud/bigquery/table.rb', line 724 def labels= labels reload! unless resource_full? @gapi.labels = labels patch_gapi! :labels end |
#load(files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, autodetect: nil, null_marker: nil) {|updater| ... } ⇒ Boolean
Loads data into the table. You can pass a google-cloud storage file path or a google-cloud storage file instance. Or, you can upload a file directly. See Loading Data with a POST Request.
The geographic location for the job ("US", "EU", etc.) can be set via LoadJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 |
# File 'lib/google/cloud/bigquery/table.rb', line 1918 def load files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, autodetect: nil, null_marker: nil, &block job = load_job files, format: format, create: create, write: write, projection_fields: projection_fields, jagged_rows: jagged_rows, quoted_newlines: quoted_newlines, encoding: encoding, delimiter: delimiter, ignore_unknown: ignore_unknown, max_bad_records: max_bad_records, quote: quote, skip_leading: skip_leading, autodetect: autodetect, null_marker: null_marker, &block job.wait_until_done! ensure_job_succeeded! job true end |
#load_job(files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, job_id: nil, prefix: nil, labels: nil, autodetect: nil, null_marker: nil, dryrun: nil) {|load_job| ... } ⇒ Google::Cloud::Bigquery::LoadJob
Loads data into the table. You can pass a google-cloud storage file path or a google-cloud storage file instance. Or, you can upload a file directly. See Loading Data with a POST Request.
The geographic location for the job ("US", "EU", etc.) can be set via LoadJob::Updater#location= in a block passed to this method. If the table is a full resource representation (see #resource_full?), the location of the job will be automatically set to the location of the table.
1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 |
# File 'lib/google/cloud/bigquery/table.rb', line 1723 def load_job files, format: nil, create: nil, write: nil, projection_fields: nil, jagged_rows: nil, quoted_newlines: nil, encoding: nil, delimiter: nil, ignore_unknown: nil, max_bad_records: nil, quote: nil, skip_leading: nil, job_id: nil, prefix: nil, labels: nil, autodetect: nil, null_marker: nil, dryrun: nil ensure_service! updater = load_job_updater format: format, create: create, write: write, projection_fields: projection_fields, jagged_rows: jagged_rows, quoted_newlines: quoted_newlines, encoding: encoding, delimiter: delimiter, ignore_unknown: ignore_unknown, max_bad_records: max_bad_records, quote: quote, skip_leading: skip_leading, dryrun: dryrun, job_id: job_id, prefix: prefix, schema: schema, labels: labels, autodetect: autodetect, null_marker: null_marker yield updater if block_given? job_gapi = updater.to_gapi return load_local(files, job_gapi) if local_file? files load_storage files, job_gapi end |
#location ⇒ String?
The geographic location where the table should reside. Possible
values include EU
and US
. The default value is US
.
659 660 661 662 663 |
# File 'lib/google/cloud/bigquery/table.rb', line 659 def location return nil if reference? ensure_full_data! @gapi.location end |
#modified_at ⇒ Time?
The date when this table was last modified.
596 597 598 599 600 601 602 603 604 |
# File 'lib/google/cloud/bigquery/table.rb', line 596 def modified_at return nil if reference? ensure_full_data! begin ::Time.at(Integer(@gapi.last_modified_time) / 1000.0) rescue StandardError nil end end |
#name ⇒ String?
The name of the table.
433 434 435 436 |
# File 'lib/google/cloud/bigquery/table.rb', line 433 def name return nil if reference? @gapi.friendly_name end |
#name=(new_name) ⇒ Object
Updates the name of the table.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
449 450 451 452 453 |
# File 'lib/google/cloud/bigquery/table.rb', line 449 def name= new_name reload! unless resource_full? @gapi.update! friendly_name: new_name patch_gapi! :friendly_name end |
#project_id ⇒ String
The ID of the Project
containing this table.
141 142 143 144 |
# File 'lib/google/cloud/bigquery/table.rb', line 141 def project_id return reference.project_id if reference? @gapi.table_reference.project_id end |
#query ⇒ String
The query that executes each time the view is loaded.
1004 1005 1006 |
# File 'lib/google/cloud/bigquery/table.rb', line 1004 def query @gapi.view.query if @gapi.view end |
#query=(new_query) ⇒ Object
Updates the query that executes each time the view is loaded.
This sets the query using standard SQL. To specify legacy SQL or to use user-defined function resources use (#set_query) instead.
1031 1032 1033 |
# File 'lib/google/cloud/bigquery/table.rb', line 1031 def query= new_query set_query new_query end |
#query_id(standard_sql: nil, legacy_sql: nil) ⇒ String
The value returned by #id, wrapped in backticks (Standard SQL) or s quare brackets (Legacy SQL) to accommodate project IDs containing dashes. Useful in queries.
417 418 419 420 421 422 423 |
# File 'lib/google/cloud/bigquery/table.rb', line 417 def query_id standard_sql: nil, legacy_sql: nil if Convert.resolve_legacy_sql standard_sql, legacy_sql "[#{project_id}:#{dataset_id}.#{table_id}]" else "`#{project_id}.#{dataset_id}.#{table_id}`" end end |
#query_legacy_sql? ⇒ Boolean
Checks if the view's query is using legacy sql.
1089 1090 1091 1092 1093 |
# File 'lib/google/cloud/bigquery/table.rb', line 1089 def query_legacy_sql? val = @gapi.view.use_legacy_sql return true if val.nil? val end |
#query_standard_sql? ⇒ Boolean
Checks if the view's query is using standard sql.
1102 1103 1104 |
# File 'lib/google/cloud/bigquery/table.rb', line 1102 def query_standard_sql? !query_legacy_sql? end |
#query_udfs ⇒ Array<String>
The user-defined function resources used in the view's query. May be
either a code resource to load from a Google Cloud Storage URI
(gs://bucket/path
), or an inline resource that contains code for a
user-defined function (UDF). Providing an inline code resource is
equivalent to providing a URI for a file containing the same code. See
User-Defined
Functions.
1120 1121 1122 1123 1124 |
# File 'lib/google/cloud/bigquery/table.rb', line 1120 def query_udfs udfs_gapi = @gapi.view.user_defined_function_resources return [] if udfs_gapi.nil? Array(udfs_gapi).map { |udf| udf.inline_code || udf.resource_uri } end |
#reference? ⇒ Boolean
Whether the table was created without retrieving the resource representation from the BigQuery service.
2166 2167 2168 |
# File 'lib/google/cloud/bigquery/table.rb', line 2166 def reference? @gapi.nil? end |
#reload! ⇒ Google::Cloud::Bigquery::Table Also known as: refresh!
Reloads the table with current data from the BigQuery service.
2108 2109 2110 2111 2112 2113 2114 |
# File 'lib/google/cloud/bigquery/table.rb', line 2108 def reload! ensure_service! @gapi = service.get_table dataset_id, table_id @reference = nil @exists = nil self end |
#resource? ⇒ Boolean
Whether the table was created with a resource representation from the BigQuery service.
2189 2190 2191 |
# File 'lib/google/cloud/bigquery/table.rb', line 2189 def resource? !@gapi.nil? end |
#resource_full? ⇒ Boolean
Whether the table was created with a full resource representation from the BigQuery service.
2238 2239 2240 |
# File 'lib/google/cloud/bigquery/table.rb', line 2238 def resource_full? @gapi.is_a? Google::Apis::BigqueryV2::Table end |
#resource_partial? ⇒ Boolean
Whether the table was created with a partial resource representation from the BigQuery service by retrieval through Dataset#tables. See Tables: list response for the contents of the partial representation. Accessing any attribute outside of the partial representation will result in loading the full representation.
2217 2218 2219 |
# File 'lib/google/cloud/bigquery/table.rb', line 2217 def resource_partial? @gapi.is_a? Google::Apis::BigqueryV2::TableList::Table end |
#rows_count ⇒ Integer?
The number of rows in the table.
540 541 542 543 544 545 546 547 548 |
# File 'lib/google/cloud/bigquery/table.rb', line 540 def rows_count return nil if reference? ensure_full_data! begin Integer @gapi.num_rows rescue StandardError nil end end |
#schema(replace: false) {|schema| ... } ⇒ Google::Cloud::Bigquery::Schema?
Returns the table's schema. If the table is not a view (See #view?), this method can also be used to set, replace, or add to the schema by passing a block. See Schema for available methods.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved.
780 781 782 783 784 785 786 787 788 789 790 791 792 793 |
# File 'lib/google/cloud/bigquery/table.rb', line 780 def schema replace: false return nil if reference? && !block_given? reload! unless resource_full? schema_builder = Schema.from_gapi @gapi.schema if block_given? schema_builder = Schema.from_gapi if replace yield schema_builder if schema_builder.changed? @gapi.schema = schema_builder.to_gapi patch_gapi! :schema end end schema_builder.freeze end |
#set_query(query, standard_sql: nil, legacy_sql: nil, udfs: nil) ⇒ Object
Updates the query that executes each time the view is loaded. Allows setting of standard vs. legacy SQL and user-defined function resources.
1073 1074 1075 1076 1077 1078 1079 1080 |
# File 'lib/google/cloud/bigquery/table.rb', line 1073 def set_query query, standard_sql: nil, legacy_sql: nil, udfs: nil use_legacy_sql = Convert.resolve_legacy_sql standard_sql, legacy_sql @gapi.view = Google::Apis::BigqueryV2::ViewDefinition.new \ query: query, use_legacy_sql: use_legacy_sql, user_defined_function_resources: udfs_gapi(udfs) patch_gapi! :view end |
#table? ⇒ Boolean?
Checks if the table's type is "TABLE".
615 616 617 618 |
# File 'lib/google/cloud/bigquery/table.rb', line 615 def table? return nil if reference? @gapi.type == "TABLE" end |
#table_id ⇒ String
A unique ID for this table.
116 117 118 119 |
# File 'lib/google/cloud/bigquery/table.rb', line 116 def table_id return reference.table_id if reference? @gapi.table_reference.table_id end |
#time_partitioning? ⇒ Boolean?
Checks if the table is time-partitioned. See Partitioned Tables.
166 167 168 169 |
# File 'lib/google/cloud/bigquery/table.rb', line 166 def time_partitioning? return nil if reference? !@gapi.time_partitioning.nil? end |
#time_partitioning_expiration ⇒ Integer?
The expiration for the table partitions, if any, in seconds. See Partitioned Tables.
285 286 287 288 289 290 291 |
# File 'lib/google/cloud/bigquery/table.rb', line 285 def time_partitioning_expiration return nil if reference? ensure_full_data! @gapi.time_partitioning.expiration_ms / 1_000 if time_partitioning? && !@gapi.time_partitioning.expiration_ms.nil? end |
#time_partitioning_expiration=(expiration) ⇒ Object
Sets the partition expiration for the table. See Partitioned Tables. The table must also be partitioned.
If the table is not a full resource representation (see #resource_full?), the full representation will be retrieved before the update to comply with ETag-based optimistic concurrency control.
319 320 321 322 323 324 325 |
# File 'lib/google/cloud/bigquery/table.rb', line 319 def time_partitioning_expiration= expiration reload! unless resource_full? @gapi.time_partitioning ||= \ Google::Apis::BigqueryV2::TimePartitioning.new @gapi.time_partitioning.expiration_ms = expiration * 1000 patch_gapi! :time_partitioning end |
#time_partitioning_field ⇒ String?
The field on which the table is partitioned, if any. If not
set, the destination table is partitioned by pseudo column
_PARTITIONTIME
; if set, the table is partitioned by this field. See
Partitioned Tables.
229 230 231 232 233 |
# File 'lib/google/cloud/bigquery/table.rb', line 229 def time_partitioning_field return nil if reference? ensure_full_data! @gapi.time_partitioning.field if time_partitioning? end |
#time_partitioning_field=(field) ⇒ Object
Sets the field on which to partition the table. If not
set, the destination table is partitioned by pseudo column
_PARTITIONTIME
; if set, the table is partitioned by this field. See
Partitioned Tables.
The table must also be partitioned.
You can only set the partitioning field while creating a table as in the example below. BigQuery does not allow you to change partitioning on an existing table.
267 268 269 270 271 272 273 |
# File 'lib/google/cloud/bigquery/table.rb', line 267 def time_partitioning_field= field reload! unless resource_full? @gapi.time_partitioning ||= \ Google::Apis::BigqueryV2::TimePartitioning.new @gapi.time_partitioning.field = field patch_gapi! :time_partitioning end |
#time_partitioning_type ⇒ String?
The period for which the table is partitioned, if any. See Partitioned Tables.
181 182 183 184 185 |
# File 'lib/google/cloud/bigquery/table.rb', line 181 def time_partitioning_type return nil if reference? ensure_full_data! @gapi.time_partitioning.type if time_partitioning? end |
#time_partitioning_type=(type) ⇒ Object
Sets the partitioning for the table. See Partitioned Tables.
You can only set partitioning when creating a table as in the example below. BigQuery does not allow you to change partitioning on an existing table.
209 210 211 212 213 214 215 |
# File 'lib/google/cloud/bigquery/table.rb', line 209 def time_partitioning_type= type reload! unless resource_full? @gapi.time_partitioning ||= \ Google::Apis::BigqueryV2::TimePartitioning.new @gapi.time_partitioning.type = type patch_gapi! :time_partitioning end |
#view? ⇒ Boolean?
Checks if the table's type is "VIEW", indicating that the table represents a BigQuery view. See Dataset#create_view.
630 631 632 633 |
# File 'lib/google/cloud/bigquery/table.rb', line 630 def view? return nil if reference? @gapi.type == "VIEW" end |