Class: RestScrapyd

Inherits:
Object
  • Object
show all
Includes:
RestCore
Defined in:
lib/rest-scrapyd.rb

Overview

Don’t define a ‘RestScrapyd::Client` and include it in the class to avoid yardoc choking

Instance Attribute Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#project ⇒ Object

Default project name (can also be passed as hash argument to constructor).



26
27
28
 
# File 'lib/rest-scrapyd.rb', line 26

def project
  @project
end

Instance Method Details

#addversion(version, egg, project = self.project) ⇒ Hash<String, Object>

Add a version to a project, creating the project if it doesn’t exist.

Parameters:

  • version (String) —

    Project version

  • egg (String, IO) —

    Filename or file to upload, must be a Scrapy egg

Returns:

  • (Hash<String, Object>) —

    Response, right now just {“spiders”: <n>}

See Also:



33
34
35
36
 
# File 'lib/rest-scrapyd.rb', line 33

def addversion(version, egg, project=self.project)
  egg = File.open(egg, 'rb') if egg.is_a? String
  post('addversion.json', project: project, version: version, egg: egg)
end

#cancel(job, project = self.project) ⇒ String

Cancel a scheduled or running job

Parameters:

  • job (String) —

    Jobid to cancel

  • project (String) (defaults to: self.project) —

    Project name

Returns:

  • (String) —

    Previous state of job

See Also:



55
56
57
 
# File 'lib/rest-scrapyd.rb', line 55

def cancel(job, project=self.project)
  post('cancel.json', project: project, job: job)['prevstate']
end

#delproject(project) ⇒ Object

Delete a project and all its uploaded versions.

Parameters:

  • project (String) —

    Project name

See Also:



101
102
103
 
# File 'lib/rest-scrapyd.rb', line 101

def delproject(project)
  post('delproject.json', project: project)
end

#delversion(version, project = self.project) ⇒ Object

Delete a project version.

Parameters:

  • version (String) —

    Project version

  • project (String) (defaults to: self.project) —

    Project name

See Also:



94
95
96
 
# File 'lib/rest-scrapyd.rb', line 94

def delversion(version, project=self.project)
  post('delversion.json', project: project, version: version)
end

#listjobs(project = self.project) ⇒ Hash<String, Array<Hash>>

Get the list of pending, running and finished jobs of some project.

Parameters:

  • project (String) (defaults to: self.project) —

    Project name

Returns:

  • (Hash<String, Array<Hash>>) —

    Jobs for project

See Also:



86
87
88
 
# File 'lib/rest-scrapyd.rb', line 86

def listjobs(project=self.project)
  get('listjobs.json', project: project).reject{|k,v| k=='status'}
end

#listprojects ⇒ Array<String>

Get the list of projects uploaded to this Scrapy server.

Returns:

  • (Array<String>) —

    Available projects

See Also:



62
63
64
 
# File 'lib/rest-scrapyd.rb', line 62

def listprojects
  get('listprojects.json')['projects']
end

#listspiders(project = self.project) ⇒ Array<String>

Get the list of spiders available in the last version of some project.

Parameters:

  • project (String) (defaults to: self.project) —

    Project name

Returns:

  • (Array<String>) —

    Available spiders for project

See Also:



78
79
80
 
# File 'lib/rest-scrapyd.rb', line 78

def listspiders(project=self.project)
  get('listspiders.json', project: project)['spiders']
end

#listversions(project = self.project) ⇒ Array<String>

Get the list of versions available for some project.

Parameters:

  • project (String) (defaults to: self.project) —

    Project name

Returns:

  • (Array<String>) —

    Available versions for project

See Also:



70
71
72
 
# File 'lib/rest-scrapyd.rb', line 70

def listversions(project=self.project)
  get('listversions.json', project: project)['versions']
end

#schedule(spider, version, settings = {}) ⇒ String

Schedule a spider run (also known as a job).

Parameters:

  • spider (String) —

    Spider name

  • version (String) —

    Project version

  • settings (Hash<String, String>) (defaults to: {}) —

    Additional Scrapy settings

Options Hash (settings):

  • :project (String) — default: {self.project} —

    Project name

Returns:

  • (String) —

    Job id

See Also:



45
46
47
48
 
# File 'lib/rest-scrapyd.rb', line 45

def schedule(spider, version, settings={})
  settings = {project: self.project, spider: spider}.merge(settings)
  post('schedule.json', settings)['jobid']
end