Module: Forki
- Extended by:
- Configuration
- Defined in:
- lib/forki.rb,
lib/forki/post.rb,
lib/forki/user.rb,
lib/forki/version.rb,
lib/forki/scrapers/scraper.rb,
lib/forki/scrapers/post_scraper.rb,
lib/forki/scrapers/user_scraper.rb
Defined Under Namespace
Classes: BlockedCredentialsError, ContentUnavailableError, Error, InvalidUrlError, MissingCredentialsError, Post, PostScraper, RetryableError, Scraper, UnhandledContentError, User, UserScraper
Constant Summary collapse
- VERSION =
"0.2.5"
- @@forki_logger =
Logger.new(STDOUT)
Class Method Summary collapse
- .create_temp_storage_location ⇒ Object
-
.extract_file_extension_from_url(url) ⇒ Object
Extract the file extension from a media URL E.g.
-
.retrieve_media(url) ⇒ Object
Get an image from a URL and save to a temp folder set in the configuration under temp_storage_location.
- .set_logger_level ⇒ Object
Methods included from Configuration
Class Method Details
.create_temp_storage_location ⇒ Object
86 87 88 89 90 |
# File 'lib/forki.rb', line 86 def self.create_temp_storage_location return if File.exist?(Forki.temp_storage_location) && File.directory?(Forki.temp_storage_location) FileUtils.mkdir_p Forki.temp_storage_location end |
.extract_file_extension_from_url(url) ⇒ Object
Extract the file extension from a media URL E.g. “.png” from scontent-atl3-2.xx.fbcdn.net/v/t39.30808-1.png?stp=dst-png_p148x148
55 56 57 58 59 60 61 62 |
# File 'lib/forki.rb', line 55 def self.extract_file_extension_from_url(url) stripped_url = url.split("?").first # remove URL query params extension = stripped_url.split(".").last extension = nil unless /^[a-zA-Z0-9]{3}$/.match?(extension) # extension must be a 3-character alphanumeric string extension = ".#{extension}" unless extension.nil? extension end |
.retrieve_media(url) ⇒ Object
Get an image from a URL and save to a temp folder set in the configuration under temp_storage_location
66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 |
# File 'lib/forki.rb', line 66 def self.retrieve_media(url) @@forki_logger.debug("Forki download started: #{url}") start_time = Time.now response = Typhoeus.get(url) extension = Forki.extract_file_extension_from_url(url) temp_file = "#{Forki.temp_storage_location}/facebook_media_#{SecureRandom.uuid}#{extension}" # We do this in case the folder isn't created yet, since it's a temp folder we'll just do so create_temp_storage_location File.binwrite(temp_file, response.body) @@forki_logger.debug("Forki download finished") @@forki_logger.debug("Save Location: #{temp_file}") @@forki_logger.debug("Size: #{(File.size("./#{temp_file}").to_f / 1024 / 1024).round(4)} MB") @@forki_logger.debug("Time to Download: #{(Time.now - start_time).round(3)} seconds") temp_file end |
.set_logger_level ⇒ Object
92 93 94 95 96 97 98 |
# File 'lib/forki.rb', line 92 def self.set_logger_level if ENV["RAILS_ENV"] == "test" || ENV["RAILS_ENV"] == "development" @@forki_logger.level = Logger::INFO else @@forki_logger.level = Logger::WARN end end |