Class: CMSScanner::Finders::InterestingFindings::RobotsTxt

Inherits:
Finder
  • Object
show all
Defined in:
app/finders/interesting_findings/robots_txt.rb

Overview

Robots.txt finder

Constant Summary

Constants inherited from Finder

Finder::DIRECT_ACCESS

Instance Attribute Summary

Attributes inherited from Finder

#target

Instance Method Summary collapse

Methods inherited from Finder

#browser, #found_by, #hydra, #initialize, #passive, #progress_bar, #titleize

Constructor Details

This class inherits a constructor from CMSScanner::Finders::Finder

Instance Method Details

#aggressive(_opts = {}) ⇒ InterestingFinding

Returns:



12
13
14
15
16
17
18
 
# File 'app/finders/interesting_findings/robots_txt.rb', line 12

def aggressive(_opts = {})
  res = NS::Browser.get(url)

  return unless res && res.code == 200 && res.body =~ /(?:user-agent|(?:dis)?allow):/i

  NS::RobotsTxt.new(url, confidence: 100, found_by: found_by)
end

#url ⇒ String

Returns The url of the robots.txt file.

Returns:

  • (String) —

    The url of the robots.txt file



7
8
9
 
# File 'app/finders/interesting_findings/robots_txt.rb', line 7

def url
  target.url('robots.txt')
end