Class: ParallelParser
- Inherits:
-
Object
- Object
- ParallelParser
- Defined in:
- lib/biodiversity/parser.rb
Overview
Public: Parser which runs in parallel.
Examples
parser = ParallelParser.new(4) parser.parse([“Betula L.”, “Pardosa moesta”])
Instance Method Summary collapse
-
#cpu_num ⇒ Object
Public: Returns the number of cores/CPUs.
-
#initialize(processes_num = nil) ⇒ ParallelParser
constructor
Public: Initialize ParallelParser.
-
#parse(names_list) ⇒ Object
Public: Parses an array of scientific names using several processes in parallel.
Constructor Details
#initialize(processes_num = nil) ⇒ ParallelParser
Public: Initialize ParallelParser.
processes_num - an Integer to setup the number of processes (default: nil).
If processes number is not set it will be determined
automatically.
51 52 53 54 55 56 57 58 59 |
# File 'lib/biodiversity/parser.rb', line 51 def initialize(processes_num = nil) require "parallel" cpu_num if processes_num.to_i > 0 @processes_num = [processes_num, cpu_num - 1].min else @processes_num = cpu_num > 3 ? cpu_num - 2 : 1 end end |
Instance Method Details
#cpu_num ⇒ Object
Public: Returns the number of cores/CPUs.
Returns Integer of cores/CPUs.
87 88 89 |
# File 'lib/biodiversity/parser.rb', line 87 def cpu_num @cpu_num ||= Parallel.processor_count end |
#parse(names_list) ⇒ Object
Public: Parses an array of scientific names using several processes in parallel.
Scientific names are deduplicated in the process, so every string is parsed only once.
names_list - takes an Array of scientific names,
each element should be a String.
Examples
parser = ParallelParser.new(4) parser.parse([“Homo sapiens L.”, “Quercus quercus”])
Returns a Hash with scientific names as a key, and parsing results as a value.
77 78 79 80 81 82 |
# File 'lib/biodiversity/parser.rb', line 77 def parse(names_list) parsed = Parallel.map(names_list.uniq, in_processes: @processes_num) do |n| [n, parse_process(n)] end parsed.inject({}) { |res, x| res[x[0]] = x[1]; res } end |