Class: RocketJob::DirmonEntry

Inherits:
Object
  • Object
show all
Includes:
Plugins::Document, Plugins::StateMachine
Defined in:
lib/rocket_job/dirmon_entry.rb

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.add_whitelist_path(path) ⇒ Object

Add a path to the whitelist Raises: Errno::ENOENT: No such file or directory



137
138
139
140
141
142
143
 
# File 'lib/rocket_job/dirmon_entry.rb', line 137

def self.add_whitelist_path(path)
  # Confirms that path exists
  path = IOStreams.path(path).realpath.to_s
  whitelist_paths << path
  whitelist_paths.uniq!
  path
end

.counts_by_state ⇒ Object

Returns [Hash<String:Integer>] of the number of dirmon entries in each state. Note: If there are no workers in that particular state then the hash will not have a value for it.

Example dirmon entries in every state:

RocketJob::DirmonEntry.counts_by_state
# => {
       :pending => 1,
       :enabled => 37,
       :failed => 1,
       :disabled => 3
     }

Example no dirmon entries:

RocketJob::Job.counts_by_state
# => {}



170
171
172
173
174
175
176
 
# File 'lib/rocket_job/dirmon_entry.rb', line 170

def self.counts_by_state
  counts = {}
  collection.aggregate([{"$group" => {_id: "$state", count: {"$sum" => 1}}}]).each do |result|
    counts[result["_id"].to_sym] = result["count"]
  end
  counts
end

.delete_whitelist_path(path) ⇒ Object

Deletes a path from the whitelist paths Raises: Errno::ENOENT: No such file or directory



147
148
149
150
151
152
153
 
# File 'lib/rocket_job/dirmon_entry.rb', line 147

def self.delete_whitelist_path(path)
  # Confirms that path exists
  path = IOStreams.path(path).realpath.to_s
  whitelist_paths.delete(path)
  whitelist_paths.uniq!
  path
end

.get_whitelist_paths ⇒ Object

Security Settings

A whitelist of paths from which to process files. This prevents accidental or malicious ‘pattern`s from processing files from anywhere in the system that the user under which Dirmon is running can access.

All resolved ‘pattern`s must start with one of the whitelisted path, otherwise they will be rejected

Note:

  • If no whitelist paths have been added, then a whitelist check is not performed

  • Relative paths can be used, but are not considered safe since they can be manipulated

  • These paths should be assigned in an initializer and not editable via the Web UI to ensure that they are not tampered with

Default: [] ==> Do not enforce whitelists

Returns [Array<String>] a copy of the whitelisted paths



131
132
133
 
# File 'lib/rocket_job/dirmon_entry.rb', line 131

def self.get_whitelist_paths
  whitelist_paths.dup
end

Instance Method Details

#each ⇒ Object

Yields [IOStreams::Path] for each file found that matches the current pattern.



179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
 
# File 'lib/rocket_job/dirmon_entry.rb', line 179

def each
  SemanticLogger.named_tagged(dirmon_entry: id.to_s) do
    # Case insensitive filename matching
    IOStreams.each_child(pattern) do |path|
      path = path.realpath
      # Skip archive directories
      next if path.to_s.include?(archive_directory || self.class.default_archive_directory)

      # Security check?
      if whitelist_paths.size.positive? && whitelist_paths.none? { |whitepath| path.to_s.start_with?(whitepath) }
        logger.warn "Skipping file: #{path} since it is not in any of the whitelisted paths: #{whitelist_paths.join(', ')}"
        next
      end

      # File must be writable so it can be removed after processing
      if path.respond_to?(:writable?) && !path.writable?
        logger.warn "Skipping file: #{file_name} since it is not writable by the current user. Must be able to delete/move the file after queueing the job"
        next
      end
      yield(path)
    end
  end
end

#job_class ⇒ Object

Returns the Job to be created.



219
220
221
222
223
224
225
 
# File 'lib/rocket_job/dirmon_entry.rb', line 219

def job_class
  return if job_class_name.nil?

  job_class_name.constantize
rescue NameError
  nil
end

#later(iopath) ⇒ Object

Archives the file, then kicks off a file upload job to upload the archived file.



228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
 
# File 'lib/rocket_job/dirmon_entry.rb', line 228

def later(iopath)
  job_id       = BSON::ObjectId.new
  archive_path = archive_iopath(iopath).join("#{job_id}_#{iopath.basename}")
  iopath.move_to(archive_path)

  job = RocketJob::Jobs::UploadFileJob.create!(
    job_class_name:     job_class_name,
    properties:         properties,
    description:        "#{name}: #{iopath.basename}",
    upload_file_name:   archive_path,
    original_file_name: iopath.to_s,
    job_id:             job_id
  )

  logger.info(
    message: "Created RocketJob::Jobs::UploadFileJob",
    payload: {
      dirmon_entry_name:  name,
      upload_file_name:   archive_path,
      original_file_name: iopath.to_s,
      job_class_name:     job_class_name,
      job_id:             job_id.to_s,
      upload_job_id:      job.id.to_s
    }
  )
  job
end

#set_exception(worker_name, exc_or_message) ⇒ Object

Set exception information for this DirmonEntry and fail it



204
205
206
207
208
209
210
211
212
213
214
215
216
 
# File 'lib/rocket_job/dirmon_entry.rb', line 204

def set_exception(worker_name, exc_or_message)
  if exc_or_message.is_a?(Exception)
    self.exception        = JobException.from_exception(exc_or_message)
    exception.worker_name = worker_name
  else
    build_exception(
      class_name:  "RocketJob::DirmonEntryException",
      message:     exc_or_message,
      backtrace:   [],
      worker_name: worker_name
    )
  end
end