Class: Html2rss::Html::Navigator

Inherits:
Object
  • Object
show all
Defined in:
lib/html2rss/html/navigator.rb,
lib/html2rss/html/navigator/text_extractor.rb

Overview

Navigator owns DOM chrome recognition and node traversal helpers.

Defined Under Namespace

Classes: TextExtractor

Constant Summary collapse

HEADING_TAGS =

Heading tags used to prioritize title extraction and container assessment.

%w[h1 h2 h3 h4 h5 h6].freeze
IGNORED_CONTAINER_TAGS =

Element tags that indicate ignored DOM chrome when found in a container path.

%w[nav footer header svg script style].to_set.freeze
UTILITY_LANDMARK_TAGS =

Ancestor tags that usually indicate navigation/utility regions inside a content container.

%w[nav aside footer menu].to_set.freeze
CARD_WALK_STOP_TAGS =

Immediate parent walk stops here — not a usable article card.

(UTILITY_LANDMARK_TAGS | IGNORED_CONTAINER_TAGS | %w[html body]).freeze
WRAPPING_ANCHOR_CHILD_TAGS =

Inner tags that mean a wrapping is a card, not a span-styled list link.

(HEADING_TAGS + %w[p]).freeze
MAIN_ANCHOR_SELECTOR =

Anchor selector used to identify the canonical article link element.

begin
  buf = +'a[href]:not([href=""])'
  %w[# javascript: mailto: tel: file:// sms: data:].each do |prefix|
    buf << %[:not([href^="#{prefix}"])]
  end
  buf.freeze
end

Class Method Summary collapse

Class Method Details

.descendant_of?(child_node, parent_node) ⇒ Boolean

Returns true if child_node is a descendant of parent_node. Walks up using parent pointers to avoid NodeSet allocations.

Parameters:

  • potential descendant

  • potential ancestor

Returns:

  • true when child_node is a descendant of parent_node



117
118
119
120
121
122
123
124
125
 
# File 'lib/html2rss/html/navigator.rb', line 117

def descendant_of?(child_node, parent_node)
  curr = child_node.respond_to?(:parent) ? child_node.parent : nil
  while curr
    return true if curr == parent_node

    curr = curr.respond_to?(:parent) ? curr.parent : nil
  end
  false
end

.extract_visible_text(tag, separator: ' ', exclude_nodes: nil) ⇒ String?

Extracts visible text from a given node and its children.

Parameters:

  • the node from which to extract visible text

  • (defaults to: ' ')

    separator used to join text fragments (default is a space)

  • (defaults to: nil)

    nodes to exclude from extraction

Returns:

  • the concatenated visible text, or nil if none is found



40
41
42
 
# File 'lib/html2rss/html/navigator.rb', line 40

def extract_visible_text(tag, separator: ' ', exclude_nodes: nil)
  TextExtractor.call(tag, separator:, exclude_nodes:)
end

.find_closest_selector_upwards(current_tag, selector) ⇒ Nokogiri::XML::Node?

Think of it as css_upwards method. It searches for the closest parent that matches the given selector.

Parameters:

  • starting node

  • CSS selector to search upwards for

Returns:

  • first matching node in upward traversal



87
88
89
90
91
92
93
94
95
96
 
# File 'lib/html2rss/html/navigator.rb', line 87

def find_closest_selector_upwards(current_tag, selector)
  while current_tag
    found = current_tag.at_css(selector)
    return found if found

    return nil unless current_tag.respond_to?(:parent)

    current_tag = current_tag.parent
  end
end

.find_tag_in_ancestors(current_tag, tag_name) ⇒ Nokogiri::XML::Node?

Searches for the closest parent that matches the given tag name.

Parameters:

  • starting node

  • tag name to find in ancestors

Returns:

  • matching ancestor node



104
105
106
107
108
 
# File 'lib/html2rss/html/navigator.rb', line 104

def find_tag_in_ancestors(current_tag, tag_name)
  return current_tag if Probe.tag(current_tag) == Probe.fold(tag_name)

  current_tag.ancestors(tag_name).first
end

.main_anchor_for(article_tag) ⇒ Nokogiri::XML::Node?

Returns first eligible descendant anchor.

Parameters:

  • article-like container to search within

Returns:

  • first eligible descendant anchor



47
48
49
50
51
 
# File 'lib/html2rss/html/navigator.rb', line 47

def main_anchor_for(article_tag)
  return article_tag if Probe.tag(article_tag) == 'a' && article_tag.matches?(MAIN_ANCHOR_SELECTOR)

  article_tag.at_css(MAIN_ANCHOR_SELECTOR)
end

.parent_until_condition(node, condition) ⇒ Nokogiri::XML::Node?

Returns the first parent that satisfies the condition. If the condition is met, it returns the node itself.

Parameters:

  • The node to start the search from.

  • The condition to be met.

Returns:

  • The first parent that satisfies the condition.



60
61
62
63
64
65
66
 
# File 'lib/html2rss/html/navigator.rb', line 60

def parent_until_condition(node, condition)
  while node && !node.document? && Probe.tag(node) != 'html'
    return node if condition.call(node)

    node = node.parent
  end
end

.usable_card_parent?(node) ⇒ Boolean

Immediate parent is a usable article card (not html/body/landmark chrome).

Parameters:

Returns:



73
74
75
76
77
78
 
# File 'lib/html2rss/html/navigator.rb', line 73

def usable_card_parent?(node)
  return false unless node
  return false if node.respond_to?(:document?) && node.document?

  !CARD_WALK_STOP_TAGS.include?(Probe.tag(node))
end