Class: Google::Cloud::Bigquery::QueryJob
- Defined in:
- lib/google/cloud/bigquery/query_job.rb
Overview
QueryJob
A Job subclass representing a query operation that may be performed on a Table. A QueryJob instance is created when you call Project#query_job, Dataset#query_job.
Direct Known Subclasses
Defined Under Namespace
Attributes collapse
-
#clustering? ⇒ Boolean?
Checks if the destination table will be clustered.
-
#clustering_fields ⇒ Array<String>?
One or more fields on which the destination table should be clustered.
-
#data(token: nil, max: nil, start: nil) ⇒ Google::Cloud::Bigquery::Data
(also: #query_results)
Retrieves the query results for the job.
-
#encryption ⇒ Google::Cloud::BigQuery::EncryptionConfiguration
The encryption configuration of the destination table.
-
#time_partitioning? ⇒ Boolean?
Checks if the destination table will be time-partitioned.
-
#time_partitioning_expiration ⇒ Integer?
The expiration for the destination table partitions, if any, in seconds.
-
#time_partitioning_field ⇒ String?
The field on which the destination table will be partitioned, if any.
-
#time_partitioning_require_filter? ⇒ Boolean
If set to true, queries over the destination table will require a partition filter that can be used for partition elimination to be specified.
-
#time_partitioning_type ⇒ String?
The period for which the destination table will be partitioned, if any.
-
#wait_until_done! ⇒ Object
Refreshes the job until the job is
DONE
.
Instance Method Summary collapse
-
#batch? ⇒ Boolean
Checks if the priority for the query is
BATCH
. -
#bytes_processed ⇒ Integer?
The number of bytes processed by the query.
-
#cache? ⇒ Boolean
Checks if the query job looks for an existing result in the query cache.
-
#cache_hit? ⇒ Boolean
Checks if the query results are from the query cache.
-
#ddl? ⇒ Boolean
Whether the query is a DDL statement.
-
#ddl_operation_performed ⇒ String?
The DDL operation performed, possibly dependent on the pre-existence of the DDL target.
-
#ddl_target_table ⇒ Google::Cloud::Bigquery::Table?
The DDL target table, in reference state.
-
#destination ⇒ Table
The table in which the query results are stored.
-
#dml? ⇒ Boolean
Whether the query is a DML statement.
-
#dryrun? ⇒ Boolean
(also: #dryrun, #dry_run, #dry_run?)
If set, don't actually run this job.
-
#flatten? ⇒ Boolean
Checks if the query job flattens nested and repeated fields in the query results.
-
#interactive? ⇒ Boolean
Checks if the priority for the query is
INTERACTIVE
. -
#large_results? ⇒ Boolean
Checks if the the query job allows arbitrarily large results at a slight cost to performance.
-
#legacy_sql? ⇒ Boolean
Checks if the query job is using legacy sql.
-
#maximum_billing_tier ⇒ Integer?
Limits the billing tier for this job.
-
#maximum_bytes_billed ⇒ Integer?
Limits the bytes billed for this job.
-
#num_dml_affected_rows ⇒ Integer?
The number of rows affected by a DML statement.
-
#query_plan ⇒ Array<Google::Cloud::Bigquery::QueryJob::Stage>?
Describes the execution plan for the query.
-
#standard_sql? ⇒ Boolean
Checks if the query job is using standard sql.
-
#statement_type ⇒ String?
The type of query statement, if valid.
-
#udfs ⇒ Array<String>
The user-defined function resources used in the query.
Methods inherited from Job
#cancel, #configuration, #created_at, #done?, #ended_at, #error, #errors, #failed?, #job_id, #labels, #location, #pending?, #project_id, #reload!, #rerun!, #running?, #started_at, #state, #statistics, #status, #user_email
Instance Method Details
#batch? ⇒ Boolean
Checks if the priority for the query is BATCH
.
58 59 60 61 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 58 def batch? val = @gapi.configuration.query.priority val == "BATCH" end |
#bytes_processed ⇒ Integer?
The number of bytes processed by the query.
176 177 178 179 180 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 176 def bytes_processed Integer @gapi.statistics.query.total_bytes_processed rescue StandardError nil end |
#cache? ⇒ Boolean
Checks if the query job looks for an existing result in the query cache. For more information, see Query Caching.
96 97 98 99 100 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 96 def cache? val = @gapi.configuration.query.use_query_cache return false if val.nil? val end |
#cache_hit? ⇒ Boolean
Checks if the query results are from the query cache.
166 167 168 169 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 166 def cache_hit? return false unless @gapi.statistics.query @gapi.statistics.query.cache_hit end |
#clustering? ⇒ Boolean?
Checks if the destination table will be clustered.
490 491 492 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 490 def clustering? !@gapi.configuration.query.clustering.nil? end |
#clustering_fields ⇒ Array<String>?
One or more fields on which the destination table should be clustered. Must be specified with time-based partitioning, data in the table will be first partitioned and subsequently clustered. The order of the returned fields determines the sort order of the data.
See Google::Cloud::Bigquery::QueryJob::Updater#clustering_fields=.
514 515 516 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 514 def clustering_fields @gapi.configuration.query.clustering.fields if clustering? end |
#data(token: nil, max: nil, start: nil) ⇒ Google::Cloud::Bigquery::Data Also known as: query_results
Retrieves the query results for the job.
574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 574 def data token: nil, max: nil, start: nil return nil unless done? if dryrun? return Data.from_gapi_json({ rows: [] }, nil, @gapi, service) end if ddl? || dml? data_hash = { totalRows: nil, rows: [] } return Data.from_gapi_json data_hash, nil, @gapi, service end ensure_schema! = { token: token, max: max, start: start } data_hash = service.list_tabledata \ destination_table_dataset_id, destination_table_table_id, Data.from_gapi_json data_hash, destination_table_gapi, @gapi, service end |
#ddl? ⇒ Boolean
Whether the query is a DDL statement.
263 264 265 266 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 263 def ddl? %w[CREATE_MODEL CREATE_TABLE CREATE_TABLE_AS_SELECT CREATE_VIEW \ DROP_MODEL DROP_TABLE DROP_VIEW].include? statement_type end |
#ddl_operation_performed ⇒ String?
The DDL operation performed, possibly dependent on the pre-existence of the DDL target. (See #ddl_target_table.) Possible values (new values might be added in the future):
- "CREATE": The query created the DDL target.
- "SKIP": No-op. Example cases: the query is
CREATE TABLE IF NOT EXISTS
while the table already exists, or the query isDROP TABLE IF EXISTS
while the table does not exist. - "REPLACE": The query replaced the DDL target. Example case: the
query is
CREATE OR REPLACE TABLE
, and the table already exists. - "DROP": The query deleted the DDL target.
306 307 308 309 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 306 def ddl_operation_performed return nil unless @gapi.statistics.query @gapi.statistics.query.ddl_operation_performed end |
#ddl_target_table ⇒ Google::Cloud::Bigquery::Table?
The DDL target table, in reference state. (See Table#reference?.)
Present only for CREATE/DROP TABLE/VIEW
queries. (See
#statement_type.)
319 320 321 322 323 324 325 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 319 def ddl_target_table return nil unless @gapi.statistics.query ensure_service! table = @gapi.statistics.query.ddl_target_table return nil unless table Google::Cloud::Bigquery::Table.new_reference_from_gapi table, service end |
#destination ⇒ Table
The table in which the query results are stored.
344 345 346 347 348 349 350 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 344 def destination table = @gapi.configuration.query.destination_table return nil unless table retrieve_table table.project_id, table.dataset_id, table.table_id end |
#dml? ⇒ Boolean
Whether the query is a DML statement.
287 288 289 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 287 def dml? %w[INSERT UPDATE MERGE DELETE].include? statement_type end |
#dryrun? ⇒ Boolean Also known as: dryrun, dry_run, dry_run?
If set, don't actually run this job. A valid query will return a mostly empty response with some processing statistics, while an invalid query will return the same error it would if it wasn't a dry run.
111 112 113 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 111 def dryrun? @gapi.configuration.dry_run end |
#encryption ⇒ Google::Cloud::BigQuery::EncryptionConfiguration
The encryption configuration of the destination table.
398 399 400 401 402 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 398 def encryption EncryptionConfiguration.from_gapi( @gapi.configuration.query.destination_encryption_configuration ) end |
#flatten? ⇒ Boolean
Checks if the query job flattens nested and repeated fields in the
query results. The default is true
. If the value is false
,
large_results? should return true
.
126 127 128 129 130 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 126 def flatten? val = @gapi.configuration.query.flatten_results return true if val.nil? val end |
#interactive? ⇒ Boolean
Checks if the priority for the query is INTERACTIVE
.
69 70 71 72 73 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 69 def interactive? val = @gapi.configuration.query.priority return true if val.nil? val == "INTERACTIVE" end |
#large_results? ⇒ Boolean
Checks if the the query job allows arbitrarily large results at a slight cost to performance.
82 83 84 85 86 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 82 def large_results? val = @gapi.configuration.query.allow_large_results return false if val.nil? val end |
#legacy_sql? ⇒ Boolean
Checks if the query job is using legacy sql.
357 358 359 360 361 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 357 def legacy_sql? val = @gapi.configuration.query.use_legacy_sql return true if val.nil? val end |
#maximum_billing_tier ⇒ Integer?
Limits the billing tier for this job. Queries that have resource usage beyond this tier will raise (without incurring a charge). If unspecified, this will be set to your project default. For more information, see High-Compute queries.
142 143 144 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 142 def maximum_billing_tier @gapi.configuration.query.maximum_billing_tier end |
#maximum_bytes_billed ⇒ Integer?
Limits the bytes billed for this job. Queries that will have bytes
billed beyond this limit will raise (without incurring a charge). If
nil
, this will be set to your project default.
154 155 156 157 158 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 154 def maximum_bytes_billed Integer @gapi.configuration.query.maximum_bytes_billed rescue StandardError nil end |
#num_dml_affected_rows ⇒ Integer?
The number of rows affected by a DML statement. Present only for DML
statements INSERT
, UPDATE
or DELETE
. (See #statement_type.)
334 335 336 337 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 334 def num_dml_affected_rows return nil unless @gapi.statistics.query @gapi.statistics.query.num_dml_affected_rows end |
#query_plan ⇒ Array<Google::Cloud::Bigquery::QueryJob::Stage>?
Describes the execution plan for the query.
207 208 209 210 211 212 213 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 207 def query_plan return nil unless @gapi.statistics.query && @gapi.statistics.query.query_plan Array(@gapi.statistics.query.query_plan).map do |stage| Stage.from_gapi stage end end |
#standard_sql? ⇒ Boolean
Checks if the query job is using standard sql.
368 369 370 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 368 def standard_sql? !legacy_sql? end |
#statement_type ⇒ String?
The type of query statement, if valid. Possible values (new values might be added in the future):
- "CREATE_MODEL": DDL statement, see Using Data Definition Language Statements
- "CREATE_TABLE": DDL statement, see Using Data Definition Language Statements
- "CREATE_TABLE_AS_SELECT": DDL statement, see Using Data Definition Language Statements
- "CREATE_VIEW": DDL statement, see Using Data Definition Language Statements
- "DELETE": DML statement, see Data Manipulation Language Syntax
- "DROP_MODEL": DDL statement, see Using Data Definition Language Statements
- "DROP_TABLE": DDL statement, see Using Data Definition Language Statements
- "DROP_VIEW": DDL statement, see Using Data Definition Language Statements
- "INSERT": DML statement, see Data Manipulation Language Syntax
- "MERGE": DML statement, see Data Manipulation Language Syntax
- "SELECT": SQL query, see Standard SQL Query Syntax
- "UPDATE": DML statement, see Data Manipulation Language Syntax
241 242 243 244 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 241 def statement_type return nil unless @gapi.statistics.query @gapi.statistics.query.statement_type end |
#time_partitioning? ⇒ Boolean?
Checks if the destination table will be time-partitioned. See Partitioned Tables.
413 414 415 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 413 def time_partitioning? !@gapi.configuration.query.time_partitioning.nil? end |
#time_partitioning_expiration ⇒ Integer?
The expiration for the destination table partitions, if any, in seconds. See Partitioned Tables.
457 458 459 460 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 457 def time_partitioning_expiration tp = @gapi.configuration.query.time_partitioning tp.expiration_ms / 1_000 if tp && !tp.expiration_ms.nil? end |
#time_partitioning_field ⇒ String?
The field on which the destination table will be partitioned, if any.
If not set, the destination table will be partitioned by pseudo column
_PARTITIONTIME
; if set, the table will be partitioned by this field.
See Partitioned Tables.
442 443 444 445 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 442 def time_partitioning_field return nil unless time_partitioning? @gapi.configuration.query.time_partitioning.field end |
#time_partitioning_require_filter? ⇒ Boolean
If set to true, queries over the destination table will require a partition filter that can be used for partition elimination to be specified. See Partitioned Tables.
473 474 475 476 477 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 473 def time_partitioning_require_filter? tp = @gapi.configuration.query.time_partitioning return false if tp.nil? || tp.require_partition_filter.nil? tp.require_partition_filter end |
#time_partitioning_type ⇒ String?
The period for which the destination table will be partitioned, if any. See Partitioned Tables.
426 427 428 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 426 def time_partitioning_type @gapi.configuration.query.time_partitioning.type if time_partitioning? end |
#udfs ⇒ Array<String>
The user-defined function resources used in the query. May be either a
code resource to load from a Google Cloud Storage URI
(gs://bucket/path
), or an inline resource that contains code for a
user-defined function (UDF). Providing an inline code resource is
equivalent to providing a URI for a file containing the same code. See
User-Defined Functions.
383 384 385 386 387 388 389 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 383 def udfs udfs_gapi = @gapi.configuration.query.user_defined_function_resources return nil unless udfs_gapi Array(udfs_gapi).map do |udf| udf.inline_code || udf.resource_uri end end |
#wait_until_done! ⇒ Object
Refreshes the job until the job is DONE
.
The delay between refreshes will incrementally increase.
533 534 535 536 537 538 539 540 541 542 543 544 545 546 |
# File 'lib/google/cloud/bigquery/query_job.rb', line 533 def wait_until_done! return if done? ensure_service! loop do query_results_gapi = service.job_query_results \ job_id, location: location, max: 0 if query_results_gapi.job_complete @destination_schema_gapi = query_results_gapi.schema break end end reload! end |