Class: RocketJob::DirmonEntry

Inherits:
Object
  • Object
show all
Includes:
Plugins::Document, Plugins::StateMachine
Defined in:
lib/rocket_job/dirmon_entry.rb

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.add_whitelist_path(path) ⇒ Object

Add a path to the whitelist Raises: Errno::ENOENT: No such file or directory



135
136
137
138
139
140
141
 
# File 'lib/rocket_job/dirmon_entry.rb', line 135

def self.add_whitelist_path(path)
  # Confirms that path exists
  path = IOStreams.path(path).realpath.to_s
  whitelist_paths << path
  whitelist_paths.uniq!
  path
end

.counts_by_state ⇒ Object

Returns [Hash<String:Integer>] of the number of dirmon entries in each state. Note: If there are no workers in that particular state then the hash will not have a value for it.

Example dirmon entries in every state:

RocketJob::DirmonEntry.counts_by_state
# => {
       :pending => 1,
       :enabled => 37,
       :failed => 1,
       :disabled => 3
     }

Example no dirmon entries:

RocketJob::Job.counts_by_state
# => {}



168
169
170
171
172
173
174
 
# File 'lib/rocket_job/dirmon_entry.rb', line 168

def self.counts_by_state
  counts = {}
  collection.aggregate([{"$group" => {_id: "$state", count: {"$sum" => 1}}}]).each do |result|
    counts[result["_id"].to_sym] = result["count"]
  end
  counts
end

.delete_whitelist_path(path) ⇒ Object

Deletes a path from the whitelist paths Raises: Errno::ENOENT: No such file or directory



145
146
147
148
149
150
151
 
# File 'lib/rocket_job/dirmon_entry.rb', line 145

def self.delete_whitelist_path(path)
  # Confirms that path exists
  path = IOStreams.path(path).realpath.to_s
  whitelist_paths.delete(path)
  whitelist_paths.uniq!
  path
end

.get_whitelist_paths ⇒ Object

Security Settings

A whitelist of paths from which to process files. This prevents accidental or malicious ‘pattern`s from processing files from anywhere in the system that the user under which Dirmon is running can access.

All resolved ‘pattern`s must start with one of the whitelisted path, otherwise they will be rejected

Note:

  • If no whitelist paths have been added, then a whitelist check is not performed

  • Relative paths can be used, but are not considered safe since they can be manipulated

  • These paths should be assigned in an initializer and not editable via the Web UI to ensure that they are not tampered with

Default: [] ==> Do not enforce whitelists

Returns [Array<String>] a copy of the whitelisted paths



129
130
131
 
# File 'lib/rocket_job/dirmon_entry.rb', line 129

def self.get_whitelist_paths
  whitelist_paths.dup
end

Instance Method Details

#each ⇒ Object

Yields [IOStreams::Path] for each file found that matches the current pattern.



177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
 
# File 'lib/rocket_job/dirmon_entry.rb', line 177

def each
  SemanticLogger.named_tagged(dirmon_entry: id.to_s) do
    # Case insensitive filename matching
    IOStreams.each_child(pattern) do |path|
      path = path.realpath
      # Skip archive directories
      next if path.to_s.include?(archive_directory || self.class.default_archive_directory)

      # Security check?
      if whitelist_paths.size.positive? && whitelist_paths.none? { |whitepath| path.to_s.start_with?(whitepath) }
        logger.warn "Skipping file: #{path} since it is not in any of the whitelisted paths: #{whitelist_paths.join(', ')}"
        next
      end

      # File must be writable so it can be removed after processing
      if path.respond_to?(:writable?) && !path.writable?
        logger.warn "Skipping file: #{file_name} since it is not writable by the current user. Must be able to delete/move the file after queueing the job"
        next
      end
      yield(path)
    end
  end
end

#job_class ⇒ Object

Returns the Job to be created.



217
218
219
220
221
222
223
 
# File 'lib/rocket_job/dirmon_entry.rb', line 217

def job_class
  return if job_class_name.nil?

  job_class_name.constantize
rescue NameError
  nil
end

#later(iopath) ⇒ Object

Archives the file, then kicks off a file upload job to upload the archived file.



226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
 
# File 'lib/rocket_job/dirmon_entry.rb', line 226

def later(iopath)
  job_id       = BSON::ObjectId.new
  archive_path = archive_iopath(iopath).join("#{job_id}_#{iopath.basename}")
  iopath.move_to(archive_path)

  job = RocketJob::Jobs::UploadFileJob.create!(
    job_class_name:     job_class_name,
    properties:         properties,
    description:        "#{name}: #{iopath.basename}",
    upload_file_name:   archive_path,
    original_file_name: iopath.to_s,
    job_id:             job_id
  )

  logger.info(
    message: "Created RocketJob::Jobs::UploadFileJob",
    payload: {
      dirmon_entry_name:  name,
      upload_file_name:   archive_path,
      original_file_name: iopath.to_s,
      job_class_name:     job_class_name,
      job_id:             job_id.to_s,
      upload_job_id:      job.id.to_s
    }
  )
  job
end

#set_exception(worker_name, exc_or_message) ⇒ Object

Set exception information for this DirmonEntry and fail it



202
203
204
205
206
207
208
209
210
211
212
213
214
 
# File 'lib/rocket_job/dirmon_entry.rb', line 202

def set_exception(worker_name, exc_or_message)
  if exc_or_message.is_a?(Exception)
    self.exception        = JobException.from_exception(exc_or_message)
    exception.worker_name = worker_name
  else
    build_exception(
      class_name:  "RocketJob::DirmonEntryException",
      message:     exc_or_message,
      backtrace:   [],
      worker_name: worker_name
    )
  end
end