ScrapeCreators
A zero-dependency Ruby client for the Scrape Creators API.
Installation
Add this line to your application's Gemfile:
gem 'scrape_creators', '0.8.0'
And then execute:
$ bundle install
Configuration
Configure your API key in an initializer (e.g., config/initializers/scrape_creators.rb):
ScrapeCreators.configure do |config|
config.api_key = ENV['SCRAPE_CREATORS_API_KEY']
end
Usage
Initialize a client:
scraper = ScrapeCreators::Client.new
Fetch Instagram User Posts & Reels
posts = scraper.posts("zuck")
Fetch TikTok Profile Videos
posts = scraper.posts("zuck", source: 'tiktok')
Fetch YouTube Channel Shorts
posts = scraper.posts("zuck", source: 'youtube')
Cursor-based pagination
All three platforms support cursor-based pagination. By default posts returns the first page only. Pass pages: to fetch more (e.g. 5 for the first five pages of a creator's feed):
posts = scraper.posts("zuck", pages: 5) # Instagram
posts = scraper.posts("zuck", source: 'tiktok', pages: 5) # TikTok
posts = scraper.posts("zuck", source: 'youtube', pages: 5) # YouTube Shorts
If you need finer control, posts_page returns { items:, cursor:, has_more: } so you can page through the full timeline yourself:
page = scraper.posts_page("zuck", source: 'tiktok')
loop do
page[:items].each { |item| puts item['desc'] }
break unless page[:has_more] && page[:cursor]
page = scraper.posts_page("zuck", source: 'tiktok', cursor: page[:cursor])
end
Fetch Instagram Post / Reel Info
info = scraper.post("https://www.instagram.com/p/DKSMEpKRd6h/", download_media: true)
Fetch Instagram Post / Reel Comments
comments = scraper.comments("https://www.instagram.com/p/DKSMEpKRd6h/")
Comment pagination
comments returns the first page by default. Pass pages: to collect several pages:
comments = scraper.comments(video_url, source: 'instagram', pages: 5)
Use comments_page for an initial batch followed by a Load more button. It returns
{ comments:, cursor:, has_more: } for Instagram, TikTok, and YouTube:
page = scraper.comments_page(video_url, source: 'instagram')
# When the user asks for more:
if page[:has_more]
page = scraper.comments_page(video_url, source: 'instagram', cursor: page[:cursor])
end
Pass the returned cursor unchanged. The client translates it to YouTube's
continuationToken parameter and uses cursor for Instagram and TikTok. TikTok's
has_more flag is respected; missing or unchanged cursors stop pagination.
Multi-page collection also stops if a cursor repeats. Failed API responses raise
ScrapeCreators::APIError so callers can retry without losing their place.
API documentation: Instagram, TikTok, YouTube.
License
The gem is available as open source under the terms of the MIT License.
Account search
client.search_users("sean walker", source: "instagram")
client.search_users("sean walker", source: "tiktok")
client.search_users("sean walker", source: "youtube")
# => [{ handle: "seanwalker", name: "Sean Walker", avatar_url: "https://..." }]
Returns one page of accounts, deduplicated by handle. YouTube searches channels,
whose handles can be passed to posts_page to fetch Shorts. Results without a
usable handle are omitted. Blank queries and unsupported sources raise
ArgumentError; unsuccessful API responses raise ScrapeCreators::APIError.