Module: RelatonIetf::Scrapper

Extended by:
Scrapper
Included in:
Scrapper
Defined in:
lib/relaton_ietf/scrapper.rb

Overview

Scrapper module

Constant Summary collapse

IDS = 
"https://raw.githubusercontent.com/ietf-ribose/relaton-data-ids/main/data/"
RFC = 
"https://raw.githubusercontent.com/ietf-ribose/relaton-data-rfcs/main/data/"
RSS = 
"https://raw.githubusercontent.com/ietf-ribose/relaton-data-rfcsubseries/main/data/"

Instance Method Summary collapse

Instance Method Details

#scrape_page(text, is_relation: false) ⇒ RelatonIetf::IetfBibliographicItem

Parameters:

  • text (String)
  • is_relation (TrueClass, FalseClass) (defaults to: false)

Returns:



15
16
17
18
19
20
21
22
23
24
 
# File 'lib/relaton_ietf/scrapper.rb', line 15

def scrape_page(text, is_relation: false)
  # Remove initial "IETF " string if specified
  ref = text.gsub(/^IETF /, "")
  ref.sub!(/(?<=^(?:RFC|BCP|FYI|STD))\s(\d+)/) { $1.rjust 4, "0" }
  rfc_item ref, is_relation
rescue Timeout::Error, Errno::EINVAL, Errno::ECONNRESET, EOFError,
       Net::HTTPBadResponse, Net::HTTPHeaderSyntaxError,
       Net::ProtocolError, SocketError
  raise RelatonBib::RequestError, "No document found for #{ref} reference"
end