Class: CMSScanner::Finders::InterestingFindings::RobotsTxt

Inherits:
Finder
  • Object
show all
Defined in:
app/finders/interesting_findings/robots_txt.rb

Overview

Robots.txt finder

Constant Summary

Constants inherited from Finder

Finder::DIRECT_ACCESS

Instance Attribute Summary

Attributes inherited from Finder

#progress_bar, #target

Instance Method Summary collapse

Methods inherited from Finder

#browser, #create_progress_bar, #found_by, #hydra, #initialize, #passive, #titleize

Constructor Details

This class inherits a constructor from CMSScanner::Finders::Finder

Instance Method Details

#aggressive(_opts = {}) ⇒ InterestingFinding

Returns:

  • (InterestingFinding) 


9
10
11
12
13
14
15
16
 
# File 'app/finders/interesting_findings/robots_txt.rb', line 9

def aggressive(_opts = {})
  path = 'robots.txt'
  res  = target.head_and_get(path)

  return unless res&.code == 200 && res.body =~ /(?:user-agent|(?:dis)?allow):/i

  NS::Model::RobotsTxt.new(target.url(path), confidence: 100, found_by: found_by)
end