Class: Gcloud::Bigquery::LoadJob

Inherits:
Job
  • Object
show all
Defined in:
lib/gcloud/bigquery/load_job.rb

Overview

LoadJob

A Job subclass representing a load operation that may be performed on a Table. A LoadJob instance is created when you call Table#load.

Instance Method Summary collapse

Methods inherited from Job

#configuration, #created_at, #done?, #ended_at, #error, #errors, #failed?, #job_id, #pending?, #project_id, #reload!, #rerun!, #running?, #started_at, #state, #statistics, #status, #wait_until_done!

Instance Method Details

#allow_jagged_rows?Boolean

Checks if the load operation accepts rows that are missing trailing optional columns. The missing values are treated as nulls. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an error is returned. The default value is false. Only applicable to CSV, ignored for other formats.

Returns:

  • (Boolean)

143
144
145
146
147
# File 'lib/gcloud/bigquery/load_job.rb', line 143

def allow_jagged_rows?
  val = config["load"]["allowJaggedRows"]
  val = false if val.nil?
  val
end

#backup?Boolean

Checks if the source data is a Google Cloud Datastore backup.

Returns:

  • (Boolean)

132
133
134
135
# File 'lib/gcloud/bigquery/load_job.rb', line 132

def backup?
  val = config["load"]["sourceFormat"]
  val == "DATASTORE_BACKUP"
end

#csv?Boolean

Checks if the format of the source data is CSV. The default is true.

Returns:

  • (Boolean)

124
125
126
127
128
# File 'lib/gcloud/bigquery/load_job.rb', line 124

def csv?
  val = config["load"]["sourceFormat"]
  return true if val.nil?
  val == "CSV"
end

#delimiterObject

The delimiter used between fields in the source data. The default is a comma (,).


51
52
53
54
55
# File 'lib/gcloud/bigquery/load_job.rb', line 51

def delimiter
  val = config["load"]["fieldDelimiter"]
  val = "," if val.nil?
  val
end

#destinationObject

The table into which the operation loads data. This is the table on which Table#load was invoked. Returns a Table instance.


40
41
42
43
44
45
46
# File 'lib/gcloud/bigquery/load_job.rb', line 40

def destination
  table = config["load"]["destinationTable"]
  return nil unless table
  retrieve_table table["projectId"],
                 table["datasetId"],
                 table["tableId"]
end

#ignore_unknown_values?Boolean

Checks if the load operation allows extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned. The default is false.

Returns:

  • (Boolean)

155
156
157
158
159
# File 'lib/gcloud/bigquery/load_job.rb', line 155

def ignore_unknown_values?
  val = config["load"]["ignoreUnknownValues"]
  val = false if val.nil?
  val
end

#input_file_bytesObject

The number of bytes of source data.


179
180
181
# File 'lib/gcloud/bigquery/load_job.rb', line 179

def input_file_bytes
  stats["load"]["inputFileBytes"]
end

#input_filesObject

The number of source files.


173
174
175
# File 'lib/gcloud/bigquery/load_job.rb', line 173

def input_files
  stats["load"]["inputFiles"]
end

#iso8859_1?Boolean

Checks if the character encoding of the data is ISO-8859-1.

Returns:

  • (Boolean)

77
78
79
80
# File 'lib/gcloud/bigquery/load_job.rb', line 77

def iso8859_1?
  val = config["load"]["encoding"]
  val == "ISO-8859-1"
end

#json?Boolean

Checks if the format of the source data is newline-delimited JSON. The default is false.

Returns:

  • (Boolean)

117
118
119
120
# File 'lib/gcloud/bigquery/load_job.rb', line 117

def json?
  val = config["load"]["sourceFormat"]
  val == "NEWLINE_DELIMITED_JSON"
end

#max_bad_recordsObject

The maximum number of bad records that the load operation can ignore. If the number of bad records exceeds this value, an error is returned. The default value is 0, which requires that all records be valid.


99
100
101
102
103
# File 'lib/gcloud/bigquery/load_job.rb', line 99

def max_bad_records
  val = config["load"]["maxBadRecords"]
  val = 0 if val.nil?
  val
end

#output_bytesObject

The number of bytes that have been loaded into the table. While an import job is in the running state, this value may change.


193
194
195
# File 'lib/gcloud/bigquery/load_job.rb', line 193

def output_bytes
  stats["load"]["outputBytes"]
end

#output_rowsObject

The number of rows that have been loaded into the table. While an import job is in the running state, this value may change.


186
187
188
# File 'lib/gcloud/bigquery/load_job.rb', line 186

def output_rows
  stats["load"]["outputRows"]
end

#quoteObject

The value that is used to quote data sections in a CSV file. The default value is a double-quote ("). If your data does not contain quoted sections, the value should be an empty string. If your data contains quoted newline characters, #quoted_newlines? should return true.


88
89
90
91
92
# File 'lib/gcloud/bigquery/load_job.rb', line 88

def quote
  val = config["load"]["quote"]
  val = "\"" if val.nil?
  val
end

#quoted_newlines?Boolean

Checks if quoted data sections may contain newline characters in a CSV file. The default is false.

Returns:

  • (Boolean)

108
109
110
111
112
# File 'lib/gcloud/bigquery/load_job.rb', line 108

def quoted_newlines?
  val = config["load"]["allowQuotedNewlines"]
  val = true if val.nil?
  val
end

#schemaObject

The schema for the data. Returns a hash. Can be empty if the table has already has the correct schema (see Table#schema= and Table#schema), or if the schema can be inferred from the loaded data.


165
166
167
168
169
# File 'lib/gcloud/bigquery/load_job.rb', line 165

def schema
  val = config["load"]["schema"]
  val = {} if val.nil?
  val
end

#skip_leading_rowsObject

The number of header rows at the top of a CSV file to skip. The default value is 0.


60
61
62
63
64
# File 'lib/gcloud/bigquery/load_job.rb', line 60

def skip_leading_rows
  val = config["load"]["skipLeadingRows"]
  val = 0 if val.nil?
  val
end

#sourcesObject

The URI or URIs representing the Google Cloud Storage files from which the operation loads data.


33
34
35
# File 'lib/gcloud/bigquery/load_job.rb', line 33

def sources
  Array config["load"]["sourceUris"]
end

#utf8?Boolean

Checks if the character encoding of the data is UTF-8. This is the default.

Returns:

  • (Boolean)

69
70
71
72
73
# File 'lib/gcloud/bigquery/load_job.rb', line 69

def utf8?
  val = config["load"]["encoding"]
  return true if val.nil?
  val == "UTF-8"
end