Class: Mindee::V1::Client

Inherits:
Object show all
Defined in:
lib/mindee/v1/client.rb,
sig/mindee/v1/client.rbs

Overview

Mindee API Client. See: https://developers.mindee.com/docs

Instance Method Summary collapse

Constructor Details

#initialize(api_key: '') ⇒ Client

Returns a new instance of Client.

Parameters:

  • (defaults to: '')
  • (defaults to: '') 


94
95
96
 
# File 'lib/mindee/v1/client.rb', line 94

def initialize(api_key: '')
  @api_key = api_key
end

Instance Method Details

#create_endpoint(endpoint_name: '', account_name: '', version: '') ⇒ Mindee::V1::HTTP::Endpoint

Creates a custom endpoint with the given values. Do not set for standard (off the shelf) endpoints.

Parameters:

  • (defaults to: '')

    For custom endpoints, the "API name" field in the "Settings" page of the API Builder. Do not set for standard (off the shelf) endpoints.

  • (defaults to: '')

    For custom endpoints, your account or organization username on the API Builder. This is normally not required unless you have a custom endpoint which has the same name as a standard (off the shelf) endpoint.

  • (defaults to: '')

    For custom endpoints, version of the product

  • (defaults to: '')
  • (defaults to: '')
  • (defaults to: '')

Returns:



406
407
408
409
410
411
412
413
 
# File 'lib/mindee/v1/client.rb', line 406

def create_endpoint(endpoint_name: '', account_name: '', version: '')
  initialize_endpoint(
    Mindee::V1::Product::Universal::Universal,
    endpoint_name: endpoint_name,
    account_name: ,
    version: version
  )
end

#enqueue(input_source, product_class, endpoint: nil, options: {}) ⇒ Mindee::V1::Parsing::Common::ApiResponse

Enqueue a document for async parsing

Parameters:

  • The source of the input document (local file or URL).

  • The class of the product.

  • (defaults to: {})

    A hash of options to configure the enqueue behavior. Possible keys:

    • :endpoint [V1::HTTP::Endpoint, nil] Endpoint of the API. Doesn't need to be set in the case of OTS APIs.
    • :all_words [bool] Whether to extract all the words on each page. This performs a full OCR operation on the server and will increase response time.
    • :full_text [bool] Whether to include the full OCR text response in compatible APIs. This performs a full OCR operation on the server and may increase response time.
    • :close_file [bool] Whether to close() the file after parsing it. Set to false if you need to access the file after this operation.
    • :page_options [Hash, nil] Page cutting/merge options: 
      - `:page_indexes` [Array<Integer>] Zero-based list of page indexes.
- `:operation` [Symbol] Operation to apply on the document, given the `page_indexes` specified:
  - `:KEEP_ONLY` - keep only the specified pages, and remove all others.
  - `:REMOVE` - remove the specified pages, and keep all others.
- `:on_min_pages` [Integer] Apply the operation only if the document has at least this many pages.
    • :cropper [bool] Whether to include cropper results for each page. This performs a cropping operation on the server and will increase response time.
    • :rag [bool] Whether to enable Retrieval-Augmented Generation. Only works if a Workflow ID is provided.
    • :workflow_id [String, nil] ID of the workflow to use.
  • (defaults to: nil)

    Endpoint of the API.

  • (defaults to: nil)
  • (defaults to: {})

Returns:



195
196
197
198
199
200
201
202
203
204
205
 
# File 'lib/mindee/v1/client.rb', line 195

def enqueue(input_source, product_class, endpoint: nil, options: {})
  opts = normalize_parse_options(options)
  endpoint ||= initialize_endpoint(product_class)
  logger.debug("Enqueueing document as '#{endpoint.url_root}'")

  prediction, raw_http = endpoint.predict_async(
    input_source,
    opts
  )
  Mindee::V1::Parsing::Common::ApiResponse.new(product_class, prediction, raw_http)
end

#enqueue_and_parse(input_source, product_class, endpoint, options, cancellation_token = nil) ⇒ Mindee::V1::Parsing::Common::ApiResponse

rubocop:disable Metrics/CyclomaticComplexity Enqueue a document for async parsing and automatically try to retrieve it

rubocop:disable Metrics/PerceivedComplexity

Parameters:

  • The source of the input document (local file or URL).

  • The class of the product.

  • A hash of options to configure the parsing behavior. Possible keys:

    • :endpoint [V1::HTTP::Endpoint, nil] Endpoint of the API. Doesn't need to be set in the case of OTS APIs.
    • :all_words [bool] Whether to extract all the words on each page. This performs a full OCR operation on the server and will increase response time.
    • :full_text [bool] Whether to include the full OCR text response in compatible APIs. This performs a full OCR operation on the server and may increase response time.
    • :close_file [bool] Whether to close() the file after parsing it. Set to false if you need to access the file after this operation.
    • :page_options [Hash, nil] Page cutting/merge options: 
      - `:page_indexes` [Array<Integer>] Zero-based list of page indexes.
- `:operation` [Symbol] Operation to apply on the document, given the `page_indexes` specified:
  - `:KEEP_ONLY` - keep only the specified pages, and remove all others.
  - `:REMOVE` - remove the specified pages, and keep all others.
- `:on_min_pages` [Integer] Apply the operation only if the document has at least this many pages.
    • :cropper [bool, nil] Whether to include cropper results for each page. This performs a cropping operation on the server and will increase response time.
    • :rag [bool] Whether to enable Retrieval-Augmented Generation. Only works if a Workflow ID is provided.
    • :workflow_id [String, nil] ID of the workflow to use.
    • :initial_delay_sec [Numeric] Initial delay before polling. Defaults to 2.
    • :delay_sec [Numeric] Delay between polling attempts. Defaults to 1.5.
    • :max_retries [Integer] Maximum number of retries. Defaults to 80.

  • Endpoint of the API.

  • (defaults to: nil)

    Token for cancellation.

Returns:

Raises:



254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
 
# File 'lib/mindee/v1/client.rb', line 254

def enqueue_and_parse(
  input_source,
  product_class,
  endpoint,
  options,
  cancellation_token = nil
)
  validate_async_params(options.initial_delay_sec, options.delay_sec, options.max_retries)
  enqueue_res = enqueue(input_source, product_class, endpoint: endpoint, options: options)
  job = enqueue_res.job or raise Error::MindeeAPIError, 'Expected job to be present'
  job_id = job.id

  raise Mindee::Error::MindeeError, 'Enqueueing of the document was canceled.' if cancellation_token&.canceled?

  sleep(options.initial_delay_sec)
  polling_attempts = 1
  logger.debug("Successfully enqueued document with job id: '#{job_id}'")
  queue_res = parse_queued(job_id, product_class, endpoint: endpoint)
  queue_res_job = queue_res.job or raise Error::MindeeAPIError, 'Expected job to be present'
  valid_statuses = [
    Mindee::V1::Parsing::Common::JobStatus::WAITING,
    Mindee::V1::Parsing::Common::JobStatus::PROCESSING,
  ]
  # @type var valid_statuses: Array[(:waiting | :processing | :completed | :failed)]
  while valid_statuses.include?(queue_res_job.status) && polling_attempts < options.max_retries
    logger.debug("Polling server for parsing result with job id: '#{job_id}'. Attempt #{polling_attempts}")
    raise Mindee::Error::MindeeError, 'Enqueueing of the document was canceled.' if cancellation_token&.canceled?

    sleep(options.delay_sec)
    queue_res = parse_queued(job_id, product_class, endpoint: endpoint)
    queue_res_job = queue_res.job or raise Error::MindeeAPIError, 'Expected job to be present'
    polling_attempts += 1
  end

  if queue_res_job.status != Mindee::V1::Parsing::Common::JobStatus::COMPLETED
    elapsed = options.initial_delay_sec + (polling_attempts * options.delay_sec.to_f)
    raise Error::MindeeAPIError,
          "Asynchronous parsing request timed out after #{elapsed} seconds (#{polling_attempts} tries)"
  end

  queue_res
end

#execute_workflow(input_source, workflow_id, options: {}) ⇒ Mindee::V1::Parsing::Common::WorkflowResponse

rubocop:enable Metrics/CyclomaticComplexity Sends a document to a workflow.

Accepts options either as a Hash or as a WorkflowOptions struct.

requiring authentication.

  • page_options [Hash, nil] Page cutting/merge options:
    • :page_indexes Zero-based list of page indexes.
      • :operation Operation to apply on the document, given the `page_indexes specified:
        • :KEEP_ONLY - keep only the specified pages, and remove all others.
        • :REMOVE - remove the specified pages, and keep all others.
    • :on_min_pages Apply the operation only if document has at least this many pages.

Parameters:

  • (defaults to: {})

    Options to configure workflow behavior. Possible keys:

    • document_alias [String, nil] Alias to give to the document.

    • priority [Symbol, nil] Priority to give to the document.

    • full_text [bool] Whether to include the full OCR text response in compatible APIs.

    • rag [bool, nil] Whether to enable Retrieval-Augmented Generation.

    • public_url [String, nil] A unique, encrypted URL for accessing the document validation interface without

  • (defaults to: {})

Returns:



320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
 
# File 'lib/mindee/v1/client.rb', line 320

def execute_workflow(input_source, workflow_id, options: {})
  opts = options.is_a?(WorkflowOptions) ? options : WorkflowOptions.new(params: options)
  if opts.respond_to?(:page_options) && input_source.is_a?(Input::Source::LocalInputSource)
    process_pdf_if_required(input_source, opts)
  end

  workflow_endpoint = V1::HTTP::WorkflowEndpoint.new(workflow_id, api_key: @api_key.to_s)
  logger.debug("Sending document to workflow '#{workflow_id}'")

  prediction, raw_http = workflow_endpoint.execute_workflow(
    input_source,
    opts
  )

  Mindee::V1::Parsing::Common::WorkflowResponse.new(V1::Product::Universal::Universal, prediction, raw_http)
end

#fix_account_name(account_name) ⇒ String

Parameters:

Returns:



462
463
464
465
466
467
468
469
 
# File 'lib/mindee/v1/client.rb', line 462

def ()
  if .nil? || .empty?
    logger.info("No account name provided, #{OTS_OWNER} will be used by default.")
    return OTS_OWNER
  end

  
end

#fix_endpoint_name(product_class, endpoint_name) ⇒ String

Parameters:

Returns:



458
459
460
 
# File 'lib/mindee/v1/client.rb', line 458

def fix_endpoint_name(product_class, endpoint_name)
  endpoint_name.nil? || endpoint_name.empty? ? product_class.endpoint_name.to_s : endpoint_name.to_s
end

#fix_version(product_class, version) ⇒ String

Parameters:

Returns:



471
472
473
474
475
476
477
478
479
 
# File 'lib/mindee/v1/client.rb', line 471

def fix_version(product_class, version)
  return version unless version.nil? || version.empty?

  if product_class.endpoint_version.nil? || product_class.endpoint_version.to_s.empty?
    logger.debug('No version provided for a custom build, will attempt to poll version 1 by default.')
    return '1'
  end
  product_class.endpoint_version || ''
end

#initialize_endpoint(product_class, endpoint_name: '', account_name: '', version: '') ⇒ Mindee::V1::HTTP::Endpoint

Creates an endpoint with the given values. Raises an error if the endpoint is invalid.

Parameters:

  • class of the product

  • (defaults to: '')

    For custom endpoints, the "API name" field in the "Settings" page of the API Builder. Do not set for standard (off the shelf) endpoints.

  • (defaults to: '')

    For custom endpoints, your account or organization username on the API Builder. This is normally not required unless you have a custom endpoint which has the same name as a standard (off the shelf) endpoint.

  • (defaults to: '')

    For custom endpoints, version of the product.

  • (defaults to: '')
  • (defaults to: '')
  • (defaults to: '')

Returns:



446
447
448
449
450
451
452
453
454
455
456
 
# File 'lib/mindee/v1/client.rb', line 446

def initialize_endpoint(product_class, endpoint_name: '', account_name: '', version: '')
  if (endpoint_name.nil? || endpoint_name.empty?) && product_class == Mindee::V1::Product::Universal::Universal
    raise Mindee::Error::MindeeConfigurationError, 'Missing argument endpoint_name when using custom class'
  end

  endpoint_name = fix_endpoint_name(product_class, endpoint_name)
   = ()
  version = fix_version(product_class, version)

  V1::HTTP::Endpoint.new(, endpoint_name, version, api_key: @api_key.to_s)
end

#load_prediction(product_class, local_response) ⇒ Mindee::V1::Parsing::Common::ApiResponse

Load a prediction.

Parameters:

  • class of the product

Returns:



342
343
344
345
346
347
348
349
350
351
 
# File 'lib/mindee/v1/client.rb', line 342

def load_prediction(product_class, local_response)
  raise Error::MindeeAPIError, 'Expected LocalResponse to not be nil.' if local_response.nil?

  response_hash = local_response.as_hash || {}
  raise Error::MindeeAPIError, 'Expected LocalResponse#as_hash to return a hash.' if response_hash.nil?

  Mindee::V1::Parsing::Common::ApiResponse.new(product_class, response_hash, response_hash.to_json)
rescue KeyError, Error::MindeeAPIError
  raise Error::MindeeInputError, 'No prediction found in local response.'
end

#logger ⇒ Logging

Returns:



39
 
# File 'sig/mindee/v1/client.rbs', line 39

def logger: () -> Logging

#normalize_parse_options(options) ⇒ ParseOptions

If needed, converts the parsing options provided as a hash into a proper ParseOptions object.

Parameters:

  • Options.

Returns:



484
485
486
487
488
 
# File 'lib/mindee/v1/client.rb', line 484

def normalize_parse_options(options)
  return options if options.is_a?(ParseOptions)

  ParseOptions.new(params: options)
end

#parse(input_source, product_class, endpoint: nil, options: {}, enqueue: true) ⇒ Mindee::V1::Parsing::Common::ApiResponse

Enqueue a document for parsing and automatically try to retrieve it if needed.

Accepts options either as a Hash or as a ParseOptions struct.

Parameters:

  • The class of the product.

  • (defaults to: nil)

    Endpoint of the API.

  • (defaults to: {})

    A hash of options to configure the parsing behavior. Possible keys:

    • :all_words [bool] Whether to extract all the words on each page. This performs a full OCR operation on the server and will increase response time.
    • :full_text [bool] Whether to include the full OCR text response in compatible APIs. This performs a full OCR operation on the server and may increase response time.
    • :close_file [bool] Whether to close() the file after parsing it. Set to false if you need to access the file after this operation.
    • :page_options [Hash, nil] Page cutting/merge options: 
      - `:page_indexes` [Array<Integer>] Zero-based list of page indexes.
- `:operation` [Symbol] Operation to apply on the document, given the `page_indexes` specified:
  - `:KEEP_ONLY` - keep only the specified pages, and remove all others.
  - `:REMOVE` - remove the specified pages, and keep all others.
- `:on_min_pages` [Integer] Apply the operation only if the document has at least this many pages.
    • :cropper [bool, nil] Whether to include cropper results for each page. This performs a cropping operation on the server and will increase response time.
    • :initial_delay_sec [Numeric] Initial delay before polling. Defaults to 2.
    • :delay_sec [Numeric] Delay between polling attempts. Defaults to 1.5.
    • :max_retries [Integer] Maximum number of retries. Defaults to 80.
  • (defaults to: true)

    Whether to enqueue the file.

  • (defaults to: nil)
  • (defaults to: {})

Returns:



125
126
127
128
129
130
131
132
133
134
135
 
# File 'lib/mindee/v1/client.rb', line 125

def parse(input_source, product_class, endpoint: nil, options: {}, enqueue: true)
  opts = normalize_parse_options(options)
  process_pdf_if_required(input_source, opts) if input_source.is_a?(Input::Source::LocalInputSource)
  endpoint ||= initialize_endpoint(product_class)

  if enqueue && product_class.has_async
    enqueue_and_parse(input_source, product_class, endpoint, opts)
  else
    parse_sync(input_source, product_class, endpoint, opts)
  end
end

#parse_queued(job_id, product_class, endpoint: nil) ⇒ Mindee::V1::Parsing::Common::ApiResponse

Parses a queued document

Doesn't need to be set in the case of OTS APIs.

Parameters:

  • ID of the job (queue) to poll from

  • class of the product

  • (defaults to: nil)

    Endpoint of the API

  • (defaults to: nil)

Returns:



215
216
217
218
219
220
 
# File 'lib/mindee/v1/client.rb', line 215

def parse_queued(job_id, product_class, endpoint: nil)
  endpoint = initialize_endpoint(product_class) if endpoint.nil?
  logger.debug("Fetching queued document as '#{endpoint.url_root}'")
  prediction, raw_http = endpoint.parse_async(job_id)
  Mindee::V1::Parsing::Common::ApiResponse.new(product_class, prediction, raw_http)
end

#parse_sync(input_source, product_class, endpoint, options) ⇒ Mindee::V1::Parsing::Common::ApiResponse

Call prediction API on a document and parse the results.

Parameters:

  • class of the product

  • Endpoint of the API.

  • A hash of options to configure the parsing behavior. Possible keys:

    • :all_words [bool] Whether to extract all the words on each page. This performs a full OCR operation on the server and will increase response time.
    • :full_text [bool] Whether to include the full OCR text response in compatible APIs. This performs a full OCR operation on the server and may increase response time.
    • :close_file [bool] Whether to close() the file after parsing it. Set to false if you need to access the file after this operation.
    • :page_options [Hash, nil] Page cutting/merge options: 
      - `:page_indexes` [Array<Integer>] Zero-based list of page indexes.
- `:operation` [Symbol] Operation to apply on the document, given the `page_indexes` specified:
  - `:KEEP_ONLY` - keep only the specified pages, and remove all others.
  - `:REMOVE` - remove the specified pages, and keep all others.
- `:on_min_pages` [Integer] Apply the operation only if the document has at least this many pages.
    • :cropper [bool, nil] Whether to include cropper results for each page. This performs a cropping operation on the server and will increase response time.

Returns:



158
159
160
161
162
163
164
165
166
167
 
# File 'lib/mindee/v1/client.rb', line 158

def parse_sync(input_source, product_class, endpoint, options)
  logger.debug("Parsing document as '#{endpoint.url_root}'")

  prediction, raw_http = endpoint.predict(
    input_source,
    options
  )

  Mindee::V1::Parsing::Common::ApiResponse.new(product_class, prediction, raw_http)
end

#process_pdf_if_required(input_source, opts) ⇒ void

This method returns an undefined value.

Processes a PDF if parameters were provided.

Parameters:



493
494
495
496
497
498
499
 
# File 'lib/mindee/v1/client.rb', line 493

def process_pdf_if_required(input_source, opts)
  return unless input_source.is_a?(Mindee::Input::Source::LocalInputSource) &&
                opts.page_options.on_min_pages &&
                input_source.pdf?

  input_source.process_pdf(opts.page_options)
end

#source_from_b64string(base64_string, filename, repair_pdf: false) ⇒ Mindee::Input::Source::Base64InputSource

Load a document from a base64 encoded string.

Parameters:

  • Input to parse as base64 string

  • The name of the file (without the path)

  • (defaults to: false)

    Attempts to fix broken pdf if true

  • (defaults to: false)

Returns:



375
376
377
 
# File 'lib/mindee/v1/client.rb', line 375

def source_from_b64string(base64_string, filename, repair_pdf: false)
  Input::Source::Base64InputSource.new(base64_string, filename, repair_pdf: repair_pdf)
end

#source_from_bytes(input_bytes, filename, repair_pdf: false) ⇒ Mindee::Input::Source::BytesInputSource

Load a document from raw bytes.

Parameters:

  • Encoding::BINARY byte input

  • The name of the file (without the path)

  • (defaults to: false)

    Attempts to fix broken pdf if true

  • (defaults to: false)

Returns:



366
367
368
 
# File 'lib/mindee/v1/client.rb', line 366

def source_from_bytes(input_bytes, filename, repair_pdf: false)
  Input::Source::BytesInputSource.new(input_bytes, filename, repair_pdf: repair_pdf)
end

#source_from_file(input_file, filename, repair_pdf: false) ⇒ Mindee::Input::Source::FileInputSource

Load a document from a normal Ruby File.

Parameters:

  • Input file handle

  • The name of the file (without the path)

  • (defaults to: false)

    Attempts to fix broken pdf if true

  • (defaults to: false)

Returns:



384
385
386
 
# File 'lib/mindee/v1/client.rb', line 384

def source_from_file(input_file, filename, repair_pdf: false)
  Input::Source::FileInputSource.new(input_file, filename, repair_pdf: repair_pdf)
end

#source_from_path(input_path, repair_pdf: false) ⇒ Mindee::Input::Source::PathInputSource

Load a document from an absolute path, as a string.

Parameters:

  • Path of file to open

  • (defaults to: false)

    Attempts to fix broken pdf if true

  • (defaults to: false)

Returns:



357
358
359
 
# File 'lib/mindee/v1/client.rb', line 357

def source_from_path(input_path, repair_pdf: false)
  Input::Source::PathInputSource.new(input_path, repair_pdf: repair_pdf)
end

#source_from_url(url) ⇒ Mindee::Input::Source::URLInputSource

Load a document from a secure remote source (HTTPS).

Parameters:

  • URL of the file

Returns:



391
392
393
 
# File 'lib/mindee/v1/client.rb', line 391

def source_from_url(url)
  Input::Source::URLInputSource.new(url)
end

#validate_async_params(initial_delay_sec, delay_sec, max_retries) ⇒ void

This method returns an undefined value.

Validates the parameters for async auto-polling

Parameters:

  • initial delay before polling

  • delay between polling attempts

  • maximum amount of retries.

Raises:



419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
 
# File 'lib/mindee/v1/client.rb', line 419

def validate_async_params(initial_delay_sec, delay_sec, max_retries)
  min_delay_sec = 1
  min_initial_delay_sec = 1
  min_retries = 2

  if delay_sec < min_delay_sec
    raise ArgumentError,
          "Cannot set auto-poll delay to less than #{min_delay_sec} second(s)"
  end
  if initial_delay_sec < min_initial_delay_sec
    raise ArgumentError,
          "Cannot set initial parsing delay to less than #{min_initial_delay_sec} second(s)"
  end
  raise ArgumentError, "Cannot set auto-poll retries to less than #{min_retries}" if max_retries < min_retries
end