Class: Firecrawl::MapRequest

Inherits:
Request
  • Object
show all
Defined in:
lib/firecrawl/map_request.rb

Overview

The MapRequest class encapsulates a ‘/map’ POST request to the Firecrawl API. After creating a new MapRequest instance you can make the request by calling the map method to crawl the site and retrieve links

examples

require ‘firecrawl’

request = Firecrawl::MapRequest.new( api_key: ENV[ ‘FIRECRAWL_API_KEY’ )

response = request.submit( ‘example.com’, { limit: 100 } ) if response.success?

result = response.result 
if result.success? 
  result.links.each do | link |
    puts link 
  end 
end

else

puts response.result.error_description

end

Constant Summary

Constants inherited from Request

Request::BASE_URI

Instance Method Summary collapse

Methods inherited from Request

#initialize

Constructor Details

This class inherits a constructor from Firecrawl::Request

Instance Method Details

#submit(url, options = nil, &block) ⇒ Object

The submit method makes a Firecrawl ‘/map’ POST request which will scrape the site with given url and return links to all hosted pages related to that url.

The response is always an instance of Faraday::Response. If response.success? is true, then response.result will be an instance MapResult. If the request is not successful then response.result will be an instance of ErrorResult.



35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
 
# File 'lib/firecrawl/map_request.rb', line 35

def submit( url, options = nil, &block )    
  if options
    options = options.is_a?( MapOptions ) ? options : MapOptions.build( options.to_h ) 
    options = options.to_h
  else 
    options = {}
  end
  options[ :url ] = url.to_s     

  response = post( "#{BASE_URI}/map", options, &block )
  result = nil 
  if response.success?  
    attributes = ( JSON.parse( response.body, symbolize_names: true ) rescue nil )
    result = MapResult.new( attributes )
  else 
    result = ErrorResult.new( response.status, attributes ) 
  end
  
  ResponseMethods.install( response, result )     
end