Class: Gcloud::Bigquery::LoadJob

Inherits:
Job
  • Object
show all
Defined in:
lib/gcloud/bigquery/load_job.rb

Overview

LoadJob

A Job subclass representing a load operation that may be performed on a Table. A LoadJob instance is created when you call Table#load.

See Loading Data Into BigQuery and the Jobs API reference for details.

Instance Attribute Summary

Attributes inherited from Job

#connection, #gapi

Instance Method Summary collapse

Methods inherited from Job

#configuration, #created_at, #done?, #ended_at, #error, #errors, #failed?, from_gapi, #initialize, #job_id, #pending?, #project_id, #reload!, #rerun!, #running?, #started_at, #state, #statistics, #status, #wait_until_done!

Constructor Details

This class inherits a constructor from Gcloud::Bigquery::Job

Instance Method Details

#allow_jagged_rows? ⇒ Boolean

Checks if the load operation accepts rows that are missing trailing optional columns. The missing values are treated as nulls. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an error is returned. The default value is false. Only applicable to CSV, ignored for other formats.

Returns:

  • (Boolean) 


144
145
146
147
148
 
# File 'lib/gcloud/bigquery/load_job.rb', line 144

def allow_jagged_rows?
  val = config["load"]["allowJaggedRows"]
  val = false if val.nil?
  val
end

#backup? ⇒ Boolean

Checks if the source data is a Google Cloud Datastore backup.

Returns:

  • (Boolean) 


133
134
135
136
 
# File 'lib/gcloud/bigquery/load_job.rb', line 133

def backup?
  val = config["load"]["sourceFormat"]
  val == "DATASTORE_BACKUP"
end

#csv? ⇒ Boolean

Checks if the format of the source data is CSV. The default is true.

Returns:

  • (Boolean) 


125
126
127
128
129
 
# File 'lib/gcloud/bigquery/load_job.rb', line 125

def csv?
  val = config["load"]["sourceFormat"]
  return true if val.nil?
  val == "CSV"
end

#delimiter ⇒ Object

The delimiter used between fields in the source data. The default is a comma (,).



52
53
54
55
56
 
# File 'lib/gcloud/bigquery/load_job.rb', line 52

def delimiter
  val = config["load"]["fieldDelimiter"]
  val = "," if val.nil?
  val
end

#destination ⇒ Object

The table into which the operation loads data. This is the table on which Table#load was invoked. Returns a Table instance.



41
42
43
44
45
46
47
 
# File 'lib/gcloud/bigquery/load_job.rb', line 41

def destination
  table = config["load"]["destinationTable"]
  return nil unless table
  retrieve_table table["projectId"],
                 table["datasetId"],
                 table["tableId"]
end

#ignore_unknown_values? ⇒ Boolean

Checks if the load operation allows extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned. The default is false.

Returns:

  • (Boolean) 


156
157
158
159
160
 
# File 'lib/gcloud/bigquery/load_job.rb', line 156

def ignore_unknown_values?
  val = config["load"]["ignoreUnknownValues"]
  val = false if val.nil?
  val
end

#input_file_bytes ⇒ Object

The number of bytes of source data.



180
181
182
 
# File 'lib/gcloud/bigquery/load_job.rb', line 180

def input_file_bytes
  stats["load"]["inputFileBytes"]
end

#input_files ⇒ Object

The number of source files.



174
175
176
 
# File 'lib/gcloud/bigquery/load_job.rb', line 174

def input_files
  stats["load"]["inputFiles"]
end

#iso8859_1? ⇒ Boolean

Checks if the character encoding of the data is ISO-8859-1.

Returns:

  • (Boolean) 


78
79
80
81
 
# File 'lib/gcloud/bigquery/load_job.rb', line 78

def iso8859_1?
  val = config["load"]["encoding"]
  val == "ISO-8859-1"
end

#json? ⇒ Boolean

Checks if the format of the source data is newline-delimited JSON. The default is false.

Returns:

  • (Boolean) 


118
119
120
121
 
# File 'lib/gcloud/bigquery/load_job.rb', line 118

def json?
  val = config["load"]["sourceFormat"]
  val == "NEWLINE_DELIMITED_JSON"
end

#max_bad_records ⇒ Object

The maximum number of bad records that the load operation can ignore. If the number of bad records exceeds this value, an error is returned. The default value is 0, which requires that all records be valid.



100
101
102
103
104
 
# File 'lib/gcloud/bigquery/load_job.rb', line 100

def max_bad_records
  val = config["load"]["maxBadRecords"]
  val = 0 if val.nil?
  val
end

#output_bytes ⇒ Object

The number of bytes that have been loaded into the table. While an import job is in the running state, this value may change.



194
195
196
 
# File 'lib/gcloud/bigquery/load_job.rb', line 194

def output_bytes
  stats["load"]["outputBytes"]
end

#output_rows ⇒ Object

The number of rows that have been loaded into the table. While an import job is in the running state, this value may change.



187
188
189
 
# File 'lib/gcloud/bigquery/load_job.rb', line 187

def output_rows
  stats["load"]["outputRows"]
end

#quote ⇒ Object

The value that is used to quote data sections in a CSV file. The default value is a double-quote (+“+). If your data does not contain quoted sections, the value should be an empty string. If your data contains quoted newline characters, #quoted_newlines? should return true.



89
90
91
92
93
 
# File 'lib/gcloud/bigquery/load_job.rb', line 89

def quote
  val = config["load"]["quote"]
  val = "\"" if val.nil?
  val
end

#quoted_newlines? ⇒ Boolean

Checks if quoted data sections may contain newline characters in a CSV file. The default is false.

Returns:

  • (Boolean) 


109
110
111
112
113
 
# File 'lib/gcloud/bigquery/load_job.rb', line 109

def quoted_newlines?
  val = config["load"]["allowQuotedNewlines"]
  val = true if val.nil?
  val
end

#schema ⇒ Object

The schema for the data. Returns a hash. Can be empty if the table has already has the correct schema (see Table#schema= and Table#schema), or if the schema can be inferred from the loaded data.



166
167
168
169
170
 
# File 'lib/gcloud/bigquery/load_job.rb', line 166

def schema
  val = config["load"]["schema"]
  val = {} if val.nil?
  val
end

#skip_leading_rows ⇒ Object

The number of header rows at the top of a CSV file to skip. The default value is 0.



61
62
63
64
65
 
# File 'lib/gcloud/bigquery/load_job.rb', line 61

def skip_leading_rows
  val = config["load"]["skipLeadingRows"]
  val = 0 if val.nil?
  val
end

#sources ⇒ Object

The URI or URIs representing the Google Cloud Storage files from which the operation loads data.



34
35
36
 
# File 'lib/gcloud/bigquery/load_job.rb', line 34

def sources
  Array config["load"]["sourceUris"]
end

#utf8? ⇒ Boolean

Checks if the character encoding of the data is UTF-8. This is the default.

Returns:

  • (Boolean) 


70
71
72
73
74
 
# File 'lib/gcloud/bigquery/load_job.rb', line 70

def utf8?
  val = config["load"]["encoding"]
  return true if val.nil?
  val == "UTF-8"
end