Class: Gcloud::Bigquery::Job

Inherits:
Object
  • Object
show all
Defined in:
lib/gcloud/bigquery/job.rb,
lib/gcloud/bigquery/job/list.rb

Overview

Job

Represents a generic Job that may be performed on a Table.

The subclasses of Job represent the specific BigQuery job types: CopyJob, ExtractJob, LoadJob, and QueryJob.

A job instance is created when you call Project#query_job, Dataset#query_job, Table#copy, Table#extract, Table#load, or View#data.

Examples:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery

q = "SELECT COUNT(word) as count FROM publicdata:samples.shakespeare"
job = bigquery.query_job q

job.wait_until_done!

if job.failed?
  puts job.error
else
  puts job.query_results.first
end

See Also:

Direct Known Subclasses

CopyJob, ExtractJob, LoadJob, QueryJob

Defined Under Namespace

Classes: List

Instance Method Summary collapse

Instance Method Details

#configurationObject Also known as: config

The configuration for the job. Returns a hash.

See Also:


156
157
158
159
160
# File 'lib/gcloud/bigquery/job.rb', line 156

def configuration
  hash = @gapi["configuration"] || {}
  hash = hash.to_hash if hash.respond_to? :to_hash
  hash
end

#created_atObject

The time when the job was created.


126
127
128
129
130
# File 'lib/gcloud/bigquery/job.rb', line 126

def created_at
  return nil if @gapi["statistics"].nil?
  return nil if @gapi["statistics"]["creationTime"].nil?
  Time.at(@gapi["statistics"]["creationTime"] / 1000.0)
end

#done?Boolean

Checks if the job's state is DONE. When true, the job has stopped running. However, a DONE state does not mean that the job completed successfully. Use #failed? to detect if an error occurred or if the job was successful.


113
114
115
116
# File 'lib/gcloud/bigquery/job.rb', line 113

def done?
  return false if state.nil?
  "done".casecmp(state).zero?
end

#ended_atObject

The time when the job ended. This field is present when the job's state is DONE.


145
146
147
148
149
# File 'lib/gcloud/bigquery/job.rb', line 145

def ended_at
  return nil if @gapi["statistics"].nil?
  return nil if @gapi["statistics"]["endTime"].nil?
  Time.at(@gapi["statistics"]["endTime"] / 1000.0)
end

#errorHash

The last error for the job, if any errors have occurred. Returns a hash.

See Also:


198
199
200
# File 'lib/gcloud/bigquery/job.rb', line 198

def error
  status["errorResult"]
end

#errorsObject

The errors for the job, if any errors have occurred. Returns an array of hash objects. See #error.


205
206
207
# File 'lib/gcloud/bigquery/job.rb', line 205

def errors
  Array status["errors"]
end

#failed?Boolean

Checks if an error is present.


120
121
122
# File 'lib/gcloud/bigquery/job.rb', line 120

def failed?
  !error.nil?
end

#job_idObject

The ID of the job.


74
75
76
# File 'lib/gcloud/bigquery/job.rb', line 74

def job_id
  @gapi["jobReference"]["jobId"]
end

#pending?Boolean

Checks if the job's state is PENDING.


103
104
105
106
# File 'lib/gcloud/bigquery/job.rb', line 103

def pending?
  return false if state.nil?
  "pending".casecmp(state).zero?
end

#project_idObject

The ID of the project containing the job.


80
81
82
# File 'lib/gcloud/bigquery/job.rb', line 80

def project_id
  @gapi["jobReference"]["projectId"]
end

#reload!Object Also known as: refresh!

Reloads the job with current data from the BigQuery service.


223
224
225
226
227
228
229
230
231
# File 'lib/gcloud/bigquery/job.rb', line 223

def reload!
  ensure_connection!
  resp = connection.get_job job_id
  if resp.success?
    @gapi = resp.data
  else
    fail ApiError.from_response(resp)
  end
end

#rerun!Object

Created a new job with the current configuration.


211
212
213
214
215
216
217
218
219
# File 'lib/gcloud/bigquery/job.rb', line 211

def rerun!
  ensure_connection!
  resp = connection.insert_job configuration
  if resp.success?
    Job.from_gapi resp.data, connection
  else
    fail ApiError.from_response(resp)
  end
end

#running?Boolean

Checks if the job's state is RUNNING.


96
97
98
99
# File 'lib/gcloud/bigquery/job.rb', line 96

def running?
  return false if state.nil?
  "running".casecmp(state).zero?
end

#started_atObject

The time when the job was started. This field is present after the job's state changes from PENDING to either RUNNING or DONE.


136
137
138
139
140
# File 'lib/gcloud/bigquery/job.rb', line 136

def started_at
  return nil if @gapi["statistics"].nil?
  return nil if @gapi["statistics"]["startTime"].nil?
  Time.at(@gapi["statistics"]["startTime"] / 1000.0)
end

#stateObject

The current state of the job. The possible values are PENDING, RUNNING, and DONE. A DONE state does not mean that the job completed successfully. Use #failed? to discover if an error occurred or if the job was successful.


89
90
91
92
# File 'lib/gcloud/bigquery/job.rb', line 89

def state
  return nil if @gapi["status"].nil?
  @gapi["status"]["state"]
end

#statisticsObject Also known as: stats

The statistics for the job. Returns a hash.

See Also:


168
169
170
171
172
# File 'lib/gcloud/bigquery/job.rb', line 168

def statistics
  hash = @gapi["statistics"] || {}
  hash = hash.to_hash if hash.respond_to? :to_hash
  hash
end

#statusObject

The job's status. Returns a hash. The values contained in the hash are also exposed by #state, #error, and #errors.


178
179
180
181
182
# File 'lib/gcloud/bigquery/job.rb', line 178

def status
  hash = @gapi["status"] || {}
  hash = hash.to_hash if hash.respond_to? :to_hash
  hash
end

#wait_until_done!Object

Refreshes the job until the job is DONE. The delay between refreshes will incrementally increase.

Examples:

require "gcloud"

gcloud = Gcloud.new
bigquery = gcloud.bigquery
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

extract_job = table.extract "gs://my-bucket/file-name.json",
                            format: "json"
extract_job.wait_until_done!
extract_job.done? #=> true

250
251
252
253
254
255
256
257
258
# File 'lib/gcloud/bigquery/job.rb', line 250

def wait_until_done!
  backoff = ->(retries) { sleep 2 * retries + 5 }
  retries = 0
  until done?
    backoff.call retries
    retries += 1
    reload!
  end
end