Class: RocketJob::DirmonEntry
- Inherits:
-
Object
- Object
- RocketJob::DirmonEntry
- Includes:
- Plugins::Document, Plugins::StateMachine
- Defined in:
- lib/rocket_job/dirmon_entry.rb
Class Method Summary collapse
-
.add_whitelist_path(path) ⇒ Object
Add a path to the whitelist Raises: Errno::ENOENT: No such file or directory.
-
.counts_by_state ⇒ Object
Returns [Hash<String:Integer>] of the number of dirmon entries in each state.
-
.delete_whitelist_path(path) ⇒ Object
Deletes a path from the whitelist paths Raises: Errno::ENOENT: No such file or directory.
-
.get_whitelist_paths ⇒ Object
Security Settings.
Instance Method Summary collapse
-
#each ⇒ Object
Passes each filename [Pathname] found that matches the pattern into the supplied block.
-
#job_class ⇒ Object
Returns the Job to be created.
-
#later(iopath) ⇒ Object
Archives the file, then kicks off a file upload job to upload the archived file.
-
#set_exception(worker_name, exc_or_message) ⇒ Object
Set exception information for this DirmonEntry and fail it.
Class Method Details
.add_whitelist_path(path) ⇒ Object
Add a path to the whitelist Raises: Errno::ENOENT: No such file or directory
143 144 145 146 147 148 149 |
# File 'lib/rocket_job/dirmon_entry.rb', line 143 def self.add_whitelist_path(path) # Confirms that path exists path = IOStreams.path(path).realpath.to_s whitelist_paths << path whitelist_paths.uniq! path end |
.counts_by_state ⇒ Object
Returns [Hash<String:Integer>] of the number of dirmon entries in each state. Note: If there are no workers in that particular state then the hash will not have a value for it.
Example dirmon entries in every state:
RocketJob::DirmonEntry.counts_by_state
# => {
:pending => 1,
:enabled => 37,
:failed => 1,
:disabled => 3
}
Example no dirmon entries:
RocketJob::Job.counts_by_state
# => {}
176 177 178 179 180 181 182 |
# File 'lib/rocket_job/dirmon_entry.rb', line 176 def self.counts_by_state counts = {} collection.aggregate([{"$group" => {_id: "$state", count: {"$sum" => 1}}}]).each do |result| counts[result["_id"].to_sym] = result["count"] end counts end |
.delete_whitelist_path(path) ⇒ Object
Deletes a path from the whitelist paths Raises: Errno::ENOENT: No such file or directory
153 154 155 156 157 158 159 |
# File 'lib/rocket_job/dirmon_entry.rb', line 153 def self.delete_whitelist_path(path) # Confirms that path exists path = IOStreams.path(path).realpath.to_s whitelist_paths.delete(path) whitelist_paths.uniq! path end |
.get_whitelist_paths ⇒ Object
Security Settings
A whitelist of paths from which to process files. This prevents accidental or malicious ‘pattern`s from processing files from anywhere in the system that the user under which Dirmon is running can access.
All resolved ‘pattern`s must start with one of the whitelisted path, otherwise they will be rejected
Note:
-
If no whitelist paths have been added, then a whitelist check is not performed
-
Relative paths can be used, but are not considered safe since they can be manipulated
-
These paths should be assigned in an initializer and not editable via the Web UI to ensure that they are not tampered with
Default: [] ==> Do not enforce whitelists
Returns [Array<String>] a copy of the whitelisted paths
137 138 139 |
# File 'lib/rocket_job/dirmon_entry.rb', line 137 def self.get_whitelist_paths whitelist_paths.dup end |
Instance Method Details
#each ⇒ Object
Passes each filename [Pathname] found that matches the pattern into the supplied block
185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 |
# File 'lib/rocket_job/dirmon_entry.rb', line 185 def each SemanticLogger.named_tagged(dirmon_entry: id.to_s) do # Case insensitive filename matching IOStreams.each_child(pattern) do |path| path = path.realpath # Skip archive directories next if path.to_s.include?(archive_directory || self.class.default_archive_directory) # Security check? if whitelist_paths.size.positive? && whitelist_paths.none? { |whitepath| path.to_s.start_with?(whitepath) } logger.warn "Skipping file: #{path} since it is not in any of the whitelisted paths: #{whitelist_paths.join(', ')}" next end # File must be writable so it can be removed after processing if path.respond_to?(:writable?) && !path.writable? logger.warn "Skipping file: #{file_name} since it is not writable by the current user. Must be able to delete/move the file after queueing the job" next end yield(path) end end end |
#job_class ⇒ Object
Returns the Job to be created.
225 226 227 228 229 230 231 |
# File 'lib/rocket_job/dirmon_entry.rb', line 225 def job_class return if job_class_name.nil? job_class_name.constantize rescue NameError nil end |
#later(iopath) ⇒ Object
Archives the file, then kicks off a file upload job to upload the archived file.
234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 |
# File 'lib/rocket_job/dirmon_entry.rb', line 234 def later(iopath) job_id = BSON::ObjectId.new archive_path = archive_iopath(iopath).join("#{job_id}_#{iopath.basename}") iopath.move_to(archive_path) job = RocketJob::Jobs::UploadFileJob.create!( job_class_name: job_class_name, properties: properties, description: "#{name}: #{iopath.basename}", upload_file_name: archive_path.to_s, original_file_name: iopath.to_s, job_id: job_id ) logger.info( message: "Created RocketJob::Jobs::UploadFileJob", payload: { dirmon_entry_name: name, upload_file_name: archive_path.to_s, original_file_name: iopath.to_s, job_class_name: job_class_name, job_id: job_id.to_s, upload_job_id: job.id.to_s } ) job end |
#set_exception(worker_name, exc_or_message) ⇒ Object
Set exception information for this DirmonEntry and fail it
210 211 212 213 214 215 216 217 218 219 220 221 222 |
# File 'lib/rocket_job/dirmon_entry.rb', line 210 def set_exception(worker_name, ) if .is_a?(Exception) self.exception = JobException.from_exception() exception.worker_name = worker_name else build_exception( class_name: "RocketJob::DirmonEntryException", message: , backtrace: [], worker_name: worker_name ) end end |