Class: Html2rss::PageRecon

Inherits:
Object
  • Object
show all
Defined in:
lib/html2rss/page_recon.rb,
lib/html2rss/page_recon/diagnostics.rb

Overview

Shared page recon for MCP Inspect, curation Recon, and FeedResolution probes.

Owns surface class, native feed hints, segment stats, and a cheap AutoSource article count. Diagnostic fetch for Inspect/Recon lives on PageRecon.probe — not twin fetch helpers in those callers.

Defined Under Namespace

Modules: Diagnostics Classes: Assessment, Probe, Result

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(response:, url:) ⇒ PageRecon

Returns a new instance of PageRecon.

Parameters:



164
165
166
167
 
# File 'lib/html2rss/page_recon.rb', line 164

def initialize(response:, url:)
  @response = response
  @url = url
end

Class Method Details

.assess(response:, url:) ⇒ Assessment

Cheap page assessment for policy gates and probe scoring (fixed AutoSource limit).

Parameters:

Returns:



136
137
138
 
# File 'lib/html2rss/page_recon.rb', line 136

def self.assess(response:, url:)
  new(response:, url:).assess
end

.call(response:, url:, strategy: nil) ⇒ Result

Parameters:

  • requested entry URL

  • (defaults to: nil)

    unused (reserved for callers that already chose a strategy)

Returns:



78
79
80
 
# File 'lib/html2rss/page_recon.rb', line 78

def self.call(response:, url:, strategy: nil) # rubocop:disable Lint/UnusedMethodArgument
  new(response:, url:).call
end

.discover_segments(sst, url) ⇒ Array

Parameters:

Returns:



154
155
156
157
158
159
 
# File 'lib/html2rss/page_recon.rb', line 154

def self.discover_segments(sst, url)
  link_resolver = Scoring::LinkResolver.new(url)
  AutoSource::Segmenter.call(sst, base_url: url, strategy: :list, link_resolver:)
rescue StandardError
  []
end

.probe(url, strategy: :auto) ⇒ Probe

Builds a request session, fetches the URL once, and runs full page recon.

Parameters:

  • (defaults to: :auto)

    request strategy (:auto resolves to a concrete diagnostic strategy)

  • a customizable set of options

Returns:



90
91
92
93
94
95
96
97
98
99
100
101
 
# File 'lib/html2rss/page_recon.rb', line 90

def self.probe(url, strategy: :auto, **)
  url_obj = Url.from_absolute(url)
  resolved = FeedPipeline::StrategyPlan.concrete_for_diagnostic(strategy)
  session = build_probe_session(url_obj, resolved, **)
  response = session.fetch_initial_response
  Probe.new(
    session:,
    response:,
    result: call(response:, url: url_obj, strategy: resolved),
    strategy: resolved
  )
end

.surface_category_for(response:, url:) ⇒ Symbol

Surface class only — no AutoSource extract (for empty-extract error labels).

Parameters:

Returns:



146
147
148
 
# File 'lib/html2rss/page_recon.rb', line 146

def self.surface_category_for(response:, url:)
  new(response:, url:).surface_category_for
end

Instance Method Details

#assess ⇒ Assessment

Returns:



171
172
173
174
175
176
177
178
179
180
181
182
 
# File 'lib/html2rss/page_recon.rb', line 171

def assess
  return feed_assessment if response.feed_response?

  parsed = html_parsed_body
  articles_count, admission_drops = cheap_articles
  Assessment.new(
    surface_category: surface_category(parsed),
    articles_count:,
    admission_drops:,
    html_response: true
  )
end

#call ⇒ Result

Returns:



194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
 
# File 'lib/html2rss/page_recon.rb', line 194

def call # rubocop:disable Metrics/AbcSize, Metrics/MethodLength -- assemble recon Result
  requested = Url.from_absolute(url)
  final = response.url
  parsed = html_parsed_body
  assessment = assess
  sst_payload, segment_stats = sst_payload_and_segments(requested)

  Result.new(
    requested_url: requested.to_s,
    final_url: final.to_s,
    status: response.status,
    scheme_downgrade: scheme_downgrade?(requested, final),
    alternate_feeds: alternate_feeds_from(parsed),
    surface_category: assessment.surface_category,
    articles_count: assessment.articles_count,
    admission_drops: assessment.admission_drops,
    segment_stats:,
    html_response: response.html_response?,
    content_type: response.content_type,
    blocked_surface: blocked_surface_key,
    sst: sst_payload
  )
end

#surface_category_for ⇒ Symbol

Returns:



186
187
188
189
190
 
# File 'lib/html2rss/page_recon.rb', line 186

def surface_category_for
  return :unsupported_surface if response.feed_response?

  surface_category(html_parsed_body)
end