Class: Opener::Webservice::Server

Inherits:
Sinatra::Base
  • Object
show all
Defined in:
lib/opener/webservice/server.rb

Overview

The meat of the webservices: the actual Sinatra application. Components should extend this class and configure it (e.g. to specify what component class to use).

Constant Summary collapse

INPUT_FIELDS =

List of fields that can contain input to process.

Returns:

  • (Array) 
%w{input input_url}

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.accepted_params ⇒ Array

Returns the accepted component parameters.

Returns:

  • (Array) 


31
32
33
 
# File 'lib/opener/webservice/server.rb', line 31

def self.accepted_params
  return @accepted_params ||= []
end

.accepted_params=(params) ⇒ Object

Sets the accepted component parameters. Parameter names are always stored as symbols.

Parameters:

  • params (Array) 


22
23
24
 
# File 'lib/opener/webservice/server.rb', line 22

def self.accepted_params=(params)
  @accepted_params = params.map(&:to_sym)
end

.text_processor ⇒ Class

Returns the text processor to use.

Returns:

  • (Class) 


49
50
51
 
# File 'lib/opener/webservice/server.rb', line 49

def self.text_processor
  return @text_processor
end

.text_processor=(processor) ⇒ Object

Sets the text processor to use.

Parameters:

  • processor (Class) 


40
41
42
 
# File 'lib/opener/webservice/server.rb', line 40

def self.text_processor=(processor)
  @text_processor = processor
end

Instance Method Details

#/ ⇒ Object

Processes the input using a component.

Data can be submitted in two ways:

  1. As regular POST fields

  2. A single JSON object as the POST body

When submitting data, you can use the following fields (either as POST fields or as the fields of a JSON object):

| Field | Description | |:—————|:——————————————–| | input | The raw input text/KAF to process | | input_url | A URL to a document to download and process | | callbacks | An array of callback URLs | | error_callback | A URL to submit errors to | | request_id | A unique ID to associate with the document | | metadata | A custom metadata object to store in S3 |

In case of a JSON object the input body would look something like the following:

{"input": "Hello world, this is....", request_id: "123abc"}



83
84
85
 
# File 'lib/opener/webservice/server.rb', line 83

get '/' do
  erb :index
end

#add_transaction_parameters(options) ⇒ Object

Parameters:

  • options (Hash) 


277
278
279
280
281
282
283
284
285
286
287
288
 
# File 'lib/opener/webservice/server.rb', line 277

def add_transaction_parameters(options)
  # If raw input is given we'll trim it so the payload isn't too large for
  # Rollbar/New Relic. This uses Hash#merge so we don't modify the
  # original options variable.
  if options['input']
    options = options.merge(
      'input' => options['input'].byteslice(0, 256)
    )
  end

  Transaction.current.add_parameters(options)
end

#analyze(options) ⇒ Array

Analyzes the input and returns an Array containing the output and content type.

Parameters:

  • options (Hash)

Returns:

  • (Array) 


191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
 
# File 'lib/opener/webservice/server.rb', line 191

def analyze(options)
  add_transaction_parameters(options)

  comp_options = InputSanitizer.new.whitelist_options(
    options,
    self.class.accepted_params
  )

  input     = InputExtractor.new.extract(options)
  processor = self.class.text_processor.new(comp_options)
  output    = processor.run(input)

  if processor.respond_to?(:output_type)
    type = processor.output_type
  else
    type = :xml
  end

  return output, type
end

#analyze_async(options, request_id) ⇒ Object

Analyzes the input asynchronously.

Parameters:

  • options (Hash)
  • request_id (String) 


218
219
220
221
222
223
224
225
226
227
228
229
 
# File 'lib/opener/webservice/server.rb', line 218

def analyze_async(options, request_id)
  output, _ = analyze(options)

  submit_output(output, request_id, options)

# Submit the error to the error callback, re-raise so Rollbar can also
# report it.
rescue Exception => error
  ErrorHandler.new.submit(error, request_id) if options['error_callback']

  raise error
end

#async ⇒ Object

Runs the block in a separate thread. When running a test environment the block is instead yielded normally.



328
329
330
331
332
333
334
 
# File 'lib/opener/webservice/server.rb', line 328

def async
  if self.class.environment == :test
    yield
  else
    Thread.new { yield }
  end
end

#authenticate! ⇒ Object

Authenticates the current request.



312
313
314
315
316
317
318
319
320
321
322
 
# File 'lib/opener/webservice/server.rb', line 312

def authenticate!
  token  = Configuration.authentication_token
  secret = Configuration.authentication_secret
  creds  = {token => params[token], secret => params[secret]}

  response = HTTPClient.get(Configuration.authentication_endpoint, creds)

  unless response.ok?
    halt(403, "Authentication failed: #{response.body}")
  end
end

#json_input? ⇒ TrueClass|FalseClass

Returns ‘true` if the input data is in JSON, false otherwise

Returns:

  • (TrueClass|FalseClass) 


305
306
307
 
# File 'lib/opener/webservice/server.rb', line 305

def json_input?
  return request.content_type == 'application/json'
end

#params_from_json ⇒ Hash

Returns a Hash containing the parameters from a JSON payload. The keys of this Hash are returned as strings to prevent Symbol DOS attacks.

Returns:

  • (Hash) 


296
297
298
 
# File 'lib/opener/webservice/server.rb', line 296

def params_from_json
  return JSON.load(request.body.read)
end

#process_async(options) ⇒ Hash

Processes a request asynchronously, results are submitted to the next callback URL.

Parameters:

  • options (Hash)

Returns:

  • (Hash) 


165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
 
# File 'lib/opener/webservice/server.rb', line 165

def process_async(options)
  request_id = options['request_id'] || SecureRandom.hex
  final_url  = options['callbacks'].last

  Core::Syslog.info(
    "Processing asynchronous request with final URL #{final_url}",
    :request_id => request_id
  )

  async { analyze_async(options, request_id) }

  content_type :json

  return JSON.dump(
    :request_id => request_id,
    :output_url => "#{final_url}/#{request_id}"
  )
end

#process_sync(options) ⇒ String

Processes a request synchronously, results are sent as the response upon completion.

Parameters:

  • options (Hash)

Returns:

  • (String) 


148
149
150
151
152
153
154
155
156
 
# File 'lib/opener/webservice/server.rb', line 148

def process_sync(options)
  output, ctype = analyze(options)

  content_type(ctype)

  Transaction.reset_current

  return output
end

#submit_output(output, request_id, options) ⇒ Object

Submits the output to the next callback URL.

Parameters:

  • output (String)
  • request_id (String)
  • options (Hash) 


238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
 
# File 'lib/opener/webservice/server.rb', line 238

def submit_output(output, request_id, options)
  callbacks = options['callbacks'].dup
  next_url  = callbacks.shift

  # Re-use the old payload so that any extra data (e.g. metadata) is kept
  # in place.
  new_payload = options.merge(
    'callbacks'  => callbacks,
    'request_id' => request_id
  )

  # Make sure we don't re-send this to the next component.
  new_payload.delete('input')

  if Configuration.output_bucket
    Core::Syslog.info(
      "Uploading output to s3://#{Configuration.output_bucket}",
      :request_id => request_id
    )

    uploader = Uploader.new
    object   = uploader.upload(request_id, output, options['metadata'])

    new_payload['input_url'] = object.url_for(:read, :expires => 3600)
  else
    new_payload['input'] = output
  end

  Core::Syslog.info(
    "Submitting output to #{next_url}",
    :request_id => request_id
  )

  CallbackHandler.new.post(next_url, new_payload)
end