Class: Gcloud::Bigquery::LoadJob

Inherits:
Job
  • Object
show all
Defined in:
lib/gcloud/bigquery/load_job.rb

Overview

# LoadJob

A Job subclass representing a load operation that may be performed on a Table. A LoadJob instance is created when you call Table#load.

See Also:

Instance Attribute Summary

Attributes inherited from Job

#connection, #gapi

Instance Method Summary collapse

Methods inherited from Job

#configuration, #created_at, #done?, #ended_at, #error, #errors, #failed?, from_gapi, #initialize, #job_id, #pending?, #project_id, #reload!, #rerun!, #running?, #started_at, #state, #statistics, #status, #wait_until_done!

Constructor Details

This class inherits a constructor from Gcloud::Bigquery::Job

Instance Method Details

#allow_jagged_rows? ⇒ Boolean

Checks if the load operation accepts rows that are missing trailing optional columns. The missing values are treated as nulls. If ‘false`, records with missing trailing columns are treated as bad records, and if there are too many bad records, an error is returned. The default value is `false`. Only applicable to CSV, ignored for other formats.

Returns:

  • (Boolean) 


139
140
141
142
143
 
# File 'lib/gcloud/bigquery/load_job.rb', line 139

def allow_jagged_rows?
  val = config["load"]["allowJaggedRows"]
  val = false if val.nil?
  val
end

#backup? ⇒ Boolean

Checks if the source data is a Google Cloud Datastore backup.

Returns:

  • (Boolean) 


128
129
130
131
 
# File 'lib/gcloud/bigquery/load_job.rb', line 128

def backup?
  val = config["load"]["sourceFormat"]
  val == "DATASTORE_BACKUP"
end

#csv? ⇒ Boolean

Checks if the format of the source data is CSV. The default is ‘true`.

Returns:

  • (Boolean) 


120
121
122
123
124
 
# File 'lib/gcloud/bigquery/load_job.rb', line 120

def csv?
  val = config["load"]["sourceFormat"]
  return true if val.nil?
  val == "CSV"
end

#delimiter ⇒ Object

The delimiter used between fields in the source data. The default is a comma (,).



51
52
53
 
# File 'lib/gcloud/bigquery/load_job.rb', line 51

def delimiter
  config["load"]["fieldDelimiter"] || ","
end

#destination ⇒ Object

The table into which the operation loads data. This is the table on which Table#load was invoked. Returns a Table instance.



40
41
42
43
44
45
46
 
# File 'lib/gcloud/bigquery/load_job.rb', line 40

def destination
  table = config["load"]["destinationTable"]
  return nil unless table
  retrieve_table table["projectId"],
                 table["datasetId"],
                 table["tableId"]
end

#ignore_unknown_values? ⇒ Boolean

Checks if the load operation allows extra values that are not represented in the table schema. If ‘true`, the extra values are ignored. If `false`, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned. The default is `false`.

Returns:

  • (Boolean) 


151
152
153
154
155
 
# File 'lib/gcloud/bigquery/load_job.rb', line 151

def ignore_unknown_values?
  val = config["load"]["ignoreUnknownValues"]
  val = false if val.nil?
  val
end

#input_file_bytes ⇒ Object

The number of bytes of source data.



175
176
177
 
# File 'lib/gcloud/bigquery/load_job.rb', line 175

def input_file_bytes
  stats["load"]["inputFileBytes"]
end

#input_files ⇒ Object

The number of source files.



169
170
171
 
# File 'lib/gcloud/bigquery/load_job.rb', line 169

def input_files
  stats["load"]["inputFiles"]
end

#iso8859_1? ⇒ Boolean

Checks if the character encoding of the data is ISO-8859-1.

Returns:

  • (Boolean) 


73
74
75
76
 
# File 'lib/gcloud/bigquery/load_job.rb', line 73

def iso8859_1?
  val = config["load"]["encoding"]
  val == "ISO-8859-1"
end

#json? ⇒ Boolean

Checks if the format of the source data is [newline-delimited JSON](jsonlines.org/). The default is ‘false`.

Returns:

  • (Boolean) 


113
114
115
116
 
# File 'lib/gcloud/bigquery/load_job.rb', line 113

def json?
  val = config["load"]["sourceFormat"]
  val == "NEWLINE_DELIMITED_JSON"
end

#max_bad_records ⇒ Object

The maximum number of bad records that the load operation can ignore. If the number of bad records exceeds this value, an error is returned. The default value is ‘0`, which requires that all records be valid.



95
96
97
98
99
 
# File 'lib/gcloud/bigquery/load_job.rb', line 95

def max_bad_records
  val = config["load"]["maxBadRecords"]
  val = 0 if val.nil?
  val
end

#output_bytes ⇒ Object

The number of bytes that have been loaded into the table. While an import job is in the running state, this value may change.



189
190
191
 
# File 'lib/gcloud/bigquery/load_job.rb', line 189

def output_bytes
  stats["load"]["outputBytes"]
end

#output_rows ⇒ Object

The number of rows that have been loaded into the table. While an import job is in the running state, this value may change.



182
183
184
 
# File 'lib/gcloud/bigquery/load_job.rb', line 182

def output_rows
  stats["load"]["outputRows"]
end

#quote ⇒ Object

The value that is used to quote data sections in a CSV file. The default value is a double-quote (‘“`). If your data does not contain quoted sections, the value should be an empty string. If your data contains quoted newline characters, #quoted_newlines? should return `true`.



84
85
86
87
88
 
# File 'lib/gcloud/bigquery/load_job.rb', line 84

def quote
  val = config["load"]["quote"]
  val = "\"" if val.nil?
  val
end

#quoted_newlines? ⇒ Boolean

Checks if quoted data sections may contain newline characters in a CSV file. The default is ‘false`.

Returns:

  • (Boolean) 


104
105
106
107
108
 
# File 'lib/gcloud/bigquery/load_job.rb', line 104

def quoted_newlines?
  val = config["load"]["allowQuotedNewlines"]
  val = true if val.nil?
  val
end

#schema ⇒ Object

The schema for the data. Returns a hash. Can be empty if the table has already has the correct schema (see Table#schema= and Table#schema), or if the schema can be inferred from the loaded data.



161
162
163
164
165
 
# File 'lib/gcloud/bigquery/load_job.rb', line 161

def schema
  val = config["load"]["schema"]
  val = {} if val.nil?
  val
end

#skip_leading_rows ⇒ Object

The number of header rows at the top of a CSV file to skip. The default value is ‘0`.



58
59
60
 
# File 'lib/gcloud/bigquery/load_job.rb', line 58

def skip_leading_rows
  config["load"]["skipLeadingRows"] || 0
end

#sources ⇒ Object

The URI or URIs representing the Google Cloud Storage files from which the operation loads data.



33
34
35
 
# File 'lib/gcloud/bigquery/load_job.rb', line 33

def sources
  Array config["load"]["sourceUris"]
end

#utf8? ⇒ Boolean

Checks if the character encoding of the data is UTF-8. This is the default.

Returns:

  • (Boolean) 


65
66
67
68
69
 
# File 'lib/gcloud/bigquery/load_job.rb', line 65

def utf8?
  val = config["load"]["encoding"]
  return true if val.nil?
  val == "UTF-8"
end