Class: Roboto::ContentProvider

Inherits:
Object
  • Object
show all
Defined in:
lib/roboto/content_provider.rb

Overview

provides the content of effective robots.txt file

Instance Method Summary collapse

Instance Method Details

#contents ⇒ String

Reads the contents of the effective robots.txt file

Returns:

  • (String) —

    the contents of the effective robots.txt file



6
7
8
 
# File 'lib/roboto/content_provider.rb', line 6

def contents
  @contents ||= File.read(path)
end

#path ⇒ Path

Determines the most relevant robots.txt file.

It checks for the paths in the following order

  1. Rails.root/config/robots/.txt (ie production.txt)
  2. Rails.root/config/robots/default.txt
  3. The default, blank robots.txt provided by the gem

Returns:

  • (Path) —

    the path of the effective robots.txt file



18
19
20
21
22
23
24
25
26
27
 
# File 'lib/roboto/content_provider.rb', line 18

def path
  lookup_paths.each do |f|
    if FileTest.exist?(f)
      return f
    end
  end

  #this should never occur because we define a default in the gem
  raise "Robots file not found"
end