Class: RestScrapyd

Inherits:

Object

• Object
• RestScrapyd

Includes:

RestCore

Defined in:

lib/rest-scrapyd.rb

Instance Attribute Summary

• #project ⇒ Object

  Default project name (can also be passed as hash argument to constructor).

Instance Method Summary

• #addversion(version, egg, project = self.project) ⇒ Hash<String, Object>

  Add a version to a project, creating the project if it doesn’t exist.

• #cancel(job, project = self.project) ⇒ String

  Cancel a scheduled or running job.

• #delproject(project) ⇒ Object

  Delete a project and all its uploaded versions.

• #delversion(version, project = self.project) ⇒ Object

  Delete a project version.

• #listjobs(project = self.project) ⇒ Hash<String, Array<Hash>>

  Get the list of pending, running and finished jobs of some project.

• #listprojects ⇒ Array<String>

  Get the list of projects uploaded to this Scrapy server.

• #listspiders(project = self.project) ⇒ Array<String>

  Get the list of spiders available in the last version of some project.

• #listversions(project = self.project) ⇒ Array<String>

  Get the list of versions available for some project.

• #schedule(spider, version, settings = {}) ⇒ String

  Schedule a spider run (also known as a job).

Instance Attribute Details

#project ⇒ Object

Default project name (can also be passed as hash argument to constructor).

# File 'lib/rest-scrapyd.rb', line 31

def project
  @project
end

Instance Method Details

#addversion(version, egg, project = self.project) ⇒ Hash<String, Object>

Add a version to a project, creating the project if it doesn’t exist.

Parameters:

• version (String) —

  Project version

• egg (String, IO) —

  Filename or file to upload, must be a Scrapy egg

Returns:

• (Hash<String, Object>) —

  Response, right now just {“spiders”: <n>}

See Also:

• http://scrapyd.readthedocs.org/en/latest/api.html#addversion-json

# File 'lib/rest-scrapyd.rb', line 38

def addversion(version, egg, project=self.project)
  egg = File.open(egg, 'rb') if egg.is_a? String
  post('addversion.json', project: project, version: version, egg: egg)
end

#cancel(job, project = self.project) ⇒ String

Cancel a scheduled or running job

Parameters:

• job (String) —

  Jobid to cancel

• project (String) (defaults to: self.project) —

  Project name

Returns:

• (String) —

  Previous state of job

See Also:

• http://scrapyd.readthedocs.org/en/latest/api.html#cancel-json

# File 'lib/rest-scrapyd.rb', line 60

def cancel(job, project=self.project)
  post('cancel.json', project: project, job: job)['prevstate']
end

#delproject(project) ⇒ Object

Delete a project and all its uploaded versions.

Parameters:

• project (String) —

  Project name

See Also:

• http://scrapyd.readthedocs.org/en/latest/api.html#delproject-json

# File 'lib/rest-scrapyd.rb', line 106

def delproject(project)
  post('delproject.json', project: project)
end

#delversion(version, project = self.project) ⇒ Object

Delete a project version.

Parameters:

• version (String) —

  Project version

• project (String) (defaults to: self.project) —

  Project name

See Also:

• http://scrapyd.readthedocs.org/en/latest/api.html#delversion-json

# File 'lib/rest-scrapyd.rb', line 99

def delversion(version, project=self.project)
  post('delversion.json', project: project, version: version)
end

#listjobs(project = self.project) ⇒ Hash<String, Array<Hash>>

Get the list of pending, running and finished jobs of some project.

Parameters:

• project (String) (defaults to: self.project) —

  Project name

Returns:

• (Hash<String, Array<Hash>>) —

  Jobs for project

See Also:

• http://scrapyd.readthedocs.org/en/latest/api.html#listjobs-json

# File 'lib/rest-scrapyd.rb', line 91

def listjobs(project=self.project)
  get('listjobs.json', project: project).reject{|k,v| k=='status'}
end

#listprojects ⇒ Array<String>

Get the list of projects uploaded to this Scrapy server.

Returns:

• (Array<String>) —

  Available projects

See Also:

• http://scrapyd.readthedocs.org/en/latest/api.html#listprojects-json

# File 'lib/rest-scrapyd.rb', line 67

def listprojects
  get('listprojects.json')['projects']
end

#listspiders(project = self.project) ⇒ Array<String>

Get the list of spiders available in the last version of some project.

Parameters:

• project (String) (defaults to: self.project) —

  Project name

Returns:

• (Array<String>) —

  Available spiders for project

See Also:

• http://scrapyd.readthedocs.org/en/latest/api.html#listspiders-json

# File 'lib/rest-scrapyd.rb', line 83

def listspiders(project=self.project)
  get('listspiders.json', project: project)['spiders']
end

#listversions(project = self.project) ⇒ Array<String>

Get the list of versions available for some project.

Parameters:

• project (String) (defaults to: self.project) —

  Project name

Returns:

• (Array<String>) —

  Available versions for project

See Also:

• http://scrapyd.readthedocs.org/en/latest/api.html#listversions-json

# File 'lib/rest-scrapyd.rb', line 75

def listversions(project=self.project)
  get('listversions.json', project: project)['versions']
end

#schedule(spider, version, settings = {}) ⇒ String

Schedule a spider run (also known as a job).

Parameters:

• spider (String) —

  Spider name

• version (String) —

  Project version

• settings (Hash<String, String>) (defaults to: {}) —

  Additional Scrapy settings

Options Hash (settings):

• :project (String) — default: {self.project} —

  Project name

Returns:

• (String) —

  Job id

See Also:

• http://scrapyd.readthedocs.org/en/latest/api.html#schedule-json

# File 'lib/rest-scrapyd.rb', line 50

def schedule(spider, version, settings={})
  settings = {project: self.project, spider: spider}.merge(settings)
  post('schedule.json', settings)['jobid']
end