Class: Wuclan::Lastfm::Scrape::Base

Inherits:
Object
  • Object
show all
Includes:
Monkeyshines::RawJsonContents, Monkeyshines::ScrapeRequestCore, Monkeyshines::ScrapeRequestCore::Paginated, Monkeyshines::ScrapeRequestCore::PaginatedWithLimit, Monkeyshines::ScrapeRequestCore::Paginating
Defined in:
lib/wuclan/lastfm/scrape/base.rb,
lib/wuclan/open_social/scrape/base.rb

Overview

Base class for Lastfm API requests

Direct Known Subclasses

AlbumInfoRequest, AlbumTagsRequest, ArtistEventsRequest, ArtistImagesRequest, ArtistInfoRequest, ArtistPodcastRequest, ArtistShoutsRequest, ArtistSimilarRequest, ArtistTagsRequest, ArtistTopAlbumsRequest, ArtistTopFansRequest, ArtistTopTagsRequest, ArtistTopTracksRequest, EventAttendeesRequest, EventInfoRequest, EventShoutsRequest, GeoEventsRequest, GeoTopArtistsRequest, GeoTopTracksRequest, GroupMembersRequest, GroupWeeklyAlbumChartRequest, GroupWeeklyArtistChartRequest, GroupWeeklyChartListRequest, GroupWeeklyTrackChartRequest, LastfmAlbumInfoRequest, LastfmAlbumTagsRequest, LastfmArtistEventsRequest, LastfmArtistImagesRequest, LastfmArtistInfoRequest, LastfmArtistPodcastRequest, LastfmArtistShoutsRequest, LastfmArtistSimilarRequest, LastfmArtistTagsRequest, LastfmArtistTopAlbumsRequest, LastfmArtistTopFansRequest, LastfmArtistTopTagsRequest, LastfmArtistTopTracksRequest, LastfmEventAttendeesRequest, LastfmEventInfoRequest, LastfmEventShoutsRequest, LastfmGeoEventsRequest, LastfmGeoTopArtistsRequest, LastfmGeoTopTracksRequest, LastfmGroupMembersRequest, LastfmGroupWeeklyAlbumChartRequest, LastfmGroupWeeklyArtistChartRequest, LastfmGroupWeeklyChartListRequest, LastfmGroupWeeklyTrackChartRequest, LastfmPlaylistfetchRequest, LastfmTagSimilarRequest, LastfmTagTopAlbumsRequest, LastfmTagTopArtistsRequest, LastfmTagTopTagsRequest, LastfmTagTopTracksRequest, LastfmTagWeeklyArtistChartRequest, LastfmTagWeeklyChartListRequest, LastfmTasteometercompareRequest, LastfmTrackInfoRequest, LastfmTrackSimilarRequest, LastfmTrackTagsRequest, LastfmTrackTopFansRequest, LastfmTrackTopTagsRequest, LastfmUserEventsRequest, LastfmUserFriendsRequest, LastfmUserInfoRequest, LastfmUserLovedTracksRequest, LastfmUserNeighboursRequest, LastfmUserPastEventsRequest, LastfmUserPlaylistsRequest, LastfmUserRecentStationsRequest, LastfmUserRecentTracksRequest, LastfmUserRecommendedArtistsRequest, LastfmUserRecommendedEventsRequest, LastfmUserShoutsRequest, LastfmUserTopAlbumsRequest, LastfmUserTopArtistsRequest, LastfmUserTopTagsRequest, LastfmUserTopTracksRequest, LastfmUserWeeklyAlbumChartRequest, LastfmUserWeeklyArtistChartRequest, LastfmUserWeeklyChartListRequest, LastfmUserWeeklyTrackChartRequest, LastfmVenueEventsRequest, LastfmVenuePastEventsRequest, PlaylistfetchRequest, TagSimilarRequest, TagTopAlbumsRequest, TagTopArtistsRequest, TagTopTagsRequest, TagTopTracksRequest, TagWeeklyArtistChartRequest, TagWeeklyChartListRequest, TasteometercompareRequest, TrackInfoRequest, TrackSimilarRequest, TrackTagsRequest, TrackTopFansRequest, TrackTopTagsRequest, UserEventsRequest, UserFriendsRequest, UserInfoRequest, UserLovedTracksRequest, UserNeighboursRequest, UserPastEventsRequest, UserPlaylistsRequest, UserRecentStationsRequest, UserRecentTracksRequest, UserRecommendedArtistsRequest, UserRecommendedEventsRequest, UserShoutsRequest, UserTopAlbumsRequest, UserTopArtistsRequest, UserTopTagsRequest, UserTopTracksRequest, UserWeeklyAlbumChartRequest, UserWeeklyArtistChartRequest, UserWeeklyChartListRequest, UserWeeklyTrackChartRequest, VenueEventsRequest, VenuePastEventsRequest

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(*args) ⇒ Base

call with fields in order:

identifier page moreinfo url scraped_at response_code response_message contents

you only have to fill in what you know – in fact, it’s typical to pass in only the identifier



37
38
39
40
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 37

def initialize *args
  super *args
  self.page = (page.to_i < 1 ? 1 : page.to_i)
end

Class Method Details

.from_identifier_hash(name, hsh = {}) ⇒ Object

Call with a hash of identifier attr-val pairs (including an optional page)

Ex.

LastfmAlbumInfoRequest.from_identifier_hash(
  :album => 'Rum+Sodomy+%26+the+Lash',
  :artist => 'The+Pogues',
  :mbid => 'ba1f6641-9085-36a0-8962-65ad6e48afd1')

Calls

LastfmAlbumInfoRequest.new(
  'album=Rum+Sodomy+%26+the+Lash&artist=The+Pogues&mbid=ba1f6641-9085-36a0-8962-65ad6e48afd1')

FIXME – this is stupid in taking a special ‘id’ and a just-as-necessary hash



76
77
78
79
80
81
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 76

def self.from_identifier_hash name, hsh={}
  name = url_encode(name)
  page = hsh.delete(:page)
  rest = self.class.make_url_query(hsh)
  new("#{name}&#{rest}", page)
end

Instance Method Details

#healthy? ⇒ Boolean

Healthy if it’s well-formed and not an error

Returns:

  • (Boolean) 


89
90
91
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 89

def healthy?
  super && ( (contents !~ %r{^\{"error":}) )
end

#items ⇒ Object

Extract the actual search items returned



126
127
128
129
130
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 126

def items
  # [obj_or_array].flatten makes this always be an array (lastfm gives
  # the single object when it's not an array)
  [main_result[response_type]].flatten.compact rescue []
end

#main_result ⇒ Object

Last.fm responses typically have the form

{ "requested_objs": {
    "requested_type": [ { ... obj ... }, { ... obj ... } ... ],
    "@attr":{ "key": "val" }
  }
}

e.g.

{"similarartists":{
   "artist": [
     ...
     {"name":"Pillar","mbid":"..","match":"5.66",..."streamable":"1"}],
  "@attr":{"artist":"Dead by Sunrise"}
  }}



111
112
113
114
115
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 111

def main_result
  return @main_result if @main_result
  return unless healthy?
  @main_result = parsed_contents[main_fieldname] rescue {}
end

#make_url ⇒ Object

Generate request URL from other attributes



56
57
58
59
60
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 56

def make_url
  [ "http://ws.audioscrobbler.com/2.0/?method=#{api_method}#{identifier}",
    "&limit=#{max_items}&api_key=#{self.class.api_key}&format=json",
    "&page=#{page}" ].join("")
end

#max_pages ⇒ Object

Max pages for this resource



158
159
160
161
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 158

def max_pages
  return 1 if (result_attrs.blank? || result_attrs['totalPages'].blank?)
  attrs['totalPages'].to_i.clamp(1,self.hard_request_limit)
end

#recursive_requests(&block) ⇒ Object 



180
181
182
183
184
185
186
187
188
189
190
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 180

def recursive_requests &block
  return unless healthy?
  #
  # requestables
  #
  requestables.each do |klass|
    req = klass.new(identifier)
    req.generation = generation.to_i + 1
    yield req
  end
end

#request_for_page(page, pageinfo = nil) ⇒ Object

creates the paginated request



151
152
153
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 151

def request_for_page page, pageinfo=nil
  (page.to_i > 1) ? self.class.new(identifier, page) : self
end

#result_attrs ⇒ Object

Information about the request itself.

(see doc for result_attrs)



121
122
123
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 121

def result_attrs
  main_result["@attr"] rescue {}
end

#span ⇒ Object

Span of IDs. Assumes the response has the ids in sort order oldest to newest (which the twitter API provides)



169
170
171
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 169

def span
  [items.last['id'], items.first['id']] rescue nil
end

#timespan ⇒ Object

Span of created_at times covered by this request. Useful for rate estimation.



174
175
176
 
# File 'lib/wuclan/lastfm/scrape/base.rb', line 174

def timespan
  [Time.parse(items.last['created_at']).utc, Time.parse(items.first['created_at']).utc] rescue nil
end