Class: RocketJob::DirmonEntry

Inherits:
Object
  • Object
show all
Includes:
Plugins::Document, Plugins::StateMachine
Defined in:
lib/rocket_job/dirmon_entry.rb

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.add_whitelist_path(path) ⇒ Object

Add a path to the whitelist Raises: Errno::ENOENT: No such file or directory



143
144
145
146
147
148
149
 
# File 'lib/rocket_job/dirmon_entry.rb', line 143

def self.add_whitelist_path(path)
  # Confirms that path exists
  path = IOStreams.path(path).realpath.to_s
  whitelist_paths << path
  whitelist_paths.uniq!
  path
end

.counts_by_state ⇒ Object

Returns [Hash<String:Integer>] of the number of dirmon entries in each state. Note: If there are no workers in that particular state then the hash will not have a value for it.

Example dirmon entries in every state:

RocketJob::DirmonEntry.counts_by_state
# => {
       :pending => 1,
       :enabled => 37,
       :failed => 1,
       :disabled => 3
     }

Example no dirmon entries:

RocketJob::Job.counts_by_state
# => {}



176
177
178
179
180
181
182
 
# File 'lib/rocket_job/dirmon_entry.rb', line 176

def self.counts_by_state
  counts = {}
  collection.aggregate([{"$group" => {_id: "$state", count: {"$sum" => 1}}}]).each do |result|
    counts[result["_id"].to_sym] = result["count"]
  end
  counts
end

.delete_whitelist_path(path) ⇒ Object

Deletes a path from the whitelist paths Raises: Errno::ENOENT: No such file or directory



153
154
155
156
157
158
159
 
# File 'lib/rocket_job/dirmon_entry.rb', line 153

def self.delete_whitelist_path(path)
  # Confirms that path exists
  path = IOStreams.path(path).realpath.to_s
  whitelist_paths.delete(path)
  whitelist_paths.uniq!
  path
end

.get_whitelist_paths ⇒ Object

Security Settings

A whitelist of paths from which to process files. This prevents accidental or malicious ‘pattern`s from processing files from anywhere in the system that the user under which Dirmon is running can access.

All resolved ‘pattern`s must start with one of the whitelisted path, otherwise they will be rejected

Note:

  • If no whitelist paths have been added, then a whitelist check is not performed

  • Relative paths can be used, but are not considered safe since they can be manipulated

  • These paths should be assigned in an initializer and not editable via the Web UI to ensure that they are not tampered with

Default: [] ==> Do not enforce whitelists

Returns [Array<String>] a copy of the whitelisted paths



137
138
139
 
# File 'lib/rocket_job/dirmon_entry.rb', line 137

def self.get_whitelist_paths
  whitelist_paths.dup
end

Instance Method Details

#each ⇒ Object

Passes each filename [Pathname] found that matches the pattern into the supplied block



185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
 
# File 'lib/rocket_job/dirmon_entry.rb', line 185

def each
  SemanticLogger.named_tagged(dirmon_entry: id.to_s) do
    # Case insensitive filename matching
    IOStreams.each_child(pattern) do |path|
      path = path.realpath
      # Skip archive directories
      next if path.to_s.include?(archive_directory || self.class.default_archive_directory)

      # Security check?
      if whitelist_paths.size.positive? && whitelist_paths.none? { |whitepath| path.to_s.start_with?(whitepath) }
        logger.warn "Skipping file: #{path} since it is not in any of the whitelisted paths: #{whitelist_paths.join(', ')}"
        next
      end

      # File must be writable so it can be removed after processing
      if path.respond_to?(:writable?) && !path.writable?
        logger.warn "Skipping file: #{file_name} since it is not writable by the current user. Must be able to delete/move the file after queueing the job"
        next
      end
      yield(path)
    end
  end
end

#job_class ⇒ Object

Returns the Job to be created.



225
226
227
228
229
230
231
 
# File 'lib/rocket_job/dirmon_entry.rb', line 225

def job_class
  return if job_class_name.nil?

  job_class_name.constantize
rescue NameError
  nil
end

#later(iopath) ⇒ Object

Archives the file, then kicks off a file upload job to upload the archived file.



234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
 
# File 'lib/rocket_job/dirmon_entry.rb', line 234

def later(iopath)
  job_id       = BSON::ObjectId.new
  archive_path = archive_iopath(iopath).join("#{job_id}_#{iopath.basename}")
  iopath.move_to(archive_path)

  job = RocketJob::Jobs::UploadFileJob.create!(
    job_class_name:     job_class_name,
    properties:         properties,
    description:        "#{name}: #{iopath.basename}",
    upload_file_name:   archive_path.to_s,
    original_file_name: iopath.to_s,
    job_id:             job_id
  )

  logger.info(
    message: "Created RocketJob::Jobs::UploadFileJob",
    payload: {
      dirmon_entry_name:  name,
      upload_file_name:   archive_path.to_s,
      original_file_name: iopath.to_s,
      job_class_name:     job_class_name,
      job_id:             job_id.to_s,
      upload_job_id:      job.id.to_s
    }
  )
  job
end

#set_exception(worker_name, exc_or_message) ⇒ Object

Set exception information for this DirmonEntry and fail it



210
211
212
213
214
215
216
217
218
219
220
221
222
 
# File 'lib/rocket_job/dirmon_entry.rb', line 210

def set_exception(worker_name, exc_or_message)
  if exc_or_message.is_a?(Exception)
    self.exception        = JobException.from_exception(exc_or_message)
    exception.worker_name = worker_name
  else
    build_exception(
      class_name:  "RocketJob::DirmonEntryException",
      message:     exc_or_message,
      backtrace:   [],
      worker_name: worker_name
    )
  end
end