Module: NdrImport::Helpers::File::XmlStreaming

Includes:
UTF8Encoding
Included in:
File::Xml
Defined in:
lib/ndr_import/helpers/file/xml_streaming.rb

Overview

This mixin adds XML streaming functionality, to support more performant handling of large files by Nokogiri. Uses the ‘XML::Reader` API, and maintains a temporary DOM as the XML is streamed to allow XPath querying from the root node.

If the system has ‘iconv` available, will attempt to verify the encoding of the file being read externally, so it can be streamed in to Ruby. Otherwise, will load the raw data in to check the encoding, but still stream it through Nokogiri’s parser.

Defined Under Namespace

Classes: Cursor, Error, NestingError

Instance Method Summary collapse

Instance Method Details

#each_node(safe_path, xpath, &block) ⇒ Object

Streams the contents of the given ‘safe_path`, and yields each element matching `xpath` as they’re found.

In the case of dodgy encoding, may fall back to slurping the file, but will still use stream parsing for XML.



119
120
121
122
123
124
125
126
127
 
# File 'lib/ndr_import/helpers/file/xml_streaming.rb', line 119

def each_node(safe_path, xpath, &block)
  return enum_for(:each_node, safe_path, xpath) unless block

  require 'nokogiri'

  with_encoding_check(safe_path) do |stream, encoding|
    stream_xml_nodes(stream, xpath, encoding, &block)
  end
end