Class: Puppet::Provider::ParsedFile

Inherits:
Puppet::Provider show all
Extended by:
Util::FileParsing
Defined in:
lib/puppet/provider/parsedfile.rb

Overview

This provider can be used as the parent class for a provider that parses and generates files. Its content must be loaded via the ‘prefetch’ method, and the file will be written when ‘flush’ is called on the provider instance. At this point, the file is written once for every provider instance.

Once the provider prefetches the data, it’s the resource’s job to copy that data over to the @is variables.

NOTE: The prefetch method swallows FileReadErrors by treating the corresponding target as an empty file. If you would like to turn this behavior off, then set the raise_prefetch_errors class variable to true. Doing so will error all resources associated with the failed target.

Constant Summary

Constants included from Util

Util::AbsolutePathPosix, Util::AbsolutePathWindows, Util::DEFAULT_POSIX_MODE, Util::DEFAULT_WINDOWS_MODE, Util::RFC_3986_URI_REGEX

Constants included from Util::POSIX

Util::POSIX::LOCALE_ENV_VARS, Util::POSIX::USER_ENV_VARS

Constants included from Util::SymbolicFileMode

Util::SymbolicFileMode::SetGIDBit, Util::SymbolicFileMode::SetUIDBit, Util::SymbolicFileMode::StickyBit, Util::SymbolicFileMode::SymbolicMode, Util::SymbolicFileMode::SymbolicSpecialToBit

Constants inherited from Puppet::Provider

Confine

Constants included from Util::Docs

Util::Docs::HEADER_LEVELS

Class Attribute Summary collapse

Instance Attribute Summary collapse

Attributes included from Util::FileParsing

#line_separator, #trailing_separator

Attributes inherited from Puppet::Provider

#resource

Attributes included from Util::Docs

#doc, #nodoc

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Util::FileParsing

clear_records, fields, handle_record_line, handle_text_line, lines, parse, parse_line, record_line, records?, text_line, to_file, to_line, valid_attr?

Methods included from Util

absolute_path?, benchmark, chuser, clear_environment, default_env, deterministic_rand, deterministic_rand_int, exit_on_fail, get_env, get_environment, logmethods, merge_environment, path_to_uri, pretty_backtrace, replace_file, safe_posix_fork, set_env, symbolizehash, thinmark, uri_encode, uri_query_encode, uri_to_path, which, withenv, withumask

Methods included from Util::POSIX

#get_posix_field, #gid, groups_of, #idfield, #methodbyid, #methodbyname, #search_posix_field, #uid

Methods included from Util::SymbolicFileMode

#normalize_symbolic_mode, #symbolic_mode_to_int, #valid_symbolic_mode?

Methods inherited from Puppet::Provider

#<=>, #clear, #command, command, commands, declared_feature?, default?, default_match, defaultfor, execfail, #execfail, execpipe, #execpipe, execute, #execute, fact_match, feature_match, #get, has_command, #inspect, #name, notdefaultfor, optional_commands, post_resource_eval, #set, some_default_match, specificity, supports_parameter?, #to_s

Methods included from Util::Logging

#clear_deprecation_warnings, #debug, #deprecation_warning, #format_exception, #get_deprecation_offender, #log_and_raise, #log_deprecations_to_file, #log_exception, #puppet_deprecation_warning, #send_log, setup_facter_logging!, #warn_once

Methods included from Util::Docs

#desc, #dochook, #doctable, #markdown_definitionlist, #markdown_header, #nodoc?, #pad, scrub

Methods included from Util::Warnings

clear_warnings, debug_once, maybe_log, notice_once, warnonce

Methods included from Confiner

#confine, #confine_collection, #suitable?

Methods included from Util::Errors

#adderrorcontext, #devfail, #error_context, error_location, error_location_with_space, error_location_with_unknowns, #exceptwrap, #fail

Constructor Details

#initialize(record) ⇒ ParsedFile

Returns a new instance of ParsedFile.



452
453
454
455
456
457
458
459
# File 'lib/puppet/provider/parsedfile.rb', line 452

def initialize(record)
  super

  # The 'record' could be a resource or a record, depending on how the provider
  # is initialized.  If we got an empty property hash (probably because the resource
  # is just being initialized), then we want to set up some defaults.
  @property_hash = self.class.record?(resource[:name]) || {:record_type => self.class.name, :ensure => :absent} if @property_hash.empty?
end

Class Attribute Details

.default_targetObject



23
24
25
# File 'lib/puppet/provider/parsedfile.rb', line 23

def default_target
  @default_target
end

.raise_prefetch_errorsObject



23
24
25
# File 'lib/puppet/provider/parsedfile.rb', line 23

def raise_prefetch_errors
  @raise_prefetch_errors
end

.targetObject



23
24
25
# File 'lib/puppet/provider/parsedfile.rb', line 23

def target
  @target
end

Instance Attribute Details

#property_hashObject



26
27
28
# File 'lib/puppet/provider/parsedfile.rb', line 26

def property_hash
  @property_hash
end

Class Method Details

.backup_target(target) ⇒ Object

Make sure our file is backed up, but only back it up once per transaction. We cheat and rely on the fact that @records is created on each prefetch.



86
87
88
89
90
91
92
93
94
# File 'lib/puppet/provider/parsedfile.rb', line 86

def self.backup_target(target)
  return nil unless target_object(target).respond_to?(:backup)

  @backup_stats ||= {}
  return nil if @backup_stats[target] == @records.object_id

  target_object(target).backup
  @backup_stats[target] = @records.object_id
end

.clean(hash) ⇒ Object



28
29
30
31
32
33
34
35
# File 'lib/puppet/provider/parsedfile.rb', line 28

def self.clean(hash)
  newhash = hash.dup
  [:record_type, :on_disk].each do |p|
    newhash.delete(p) if newhash.include?(p)
  end

  newhash
end

.clearObject



37
38
39
40
# File 'lib/puppet/provider/parsedfile.rb', line 37

def self.clear
  @target_objects.clear
  @records.clear
end

.default_modeObject

This method is abstract.

Providers inheriting parsedfile can override this method to provide a mode. The value should be suitable for File.chmod

The mode for generated files if they are newly created. No mode will be set on existing files.



346
347
348
# File 'lib/puppet/provider/parsedfile.rb', line 346

def self.default_mode
  nil
end

.drop_native_headerObject

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method is abstract.

Providers based on ParsedFile that make use of the support for third party headers may override this method to return true. When this is done, headers that are matched by the native_header_regex are not written back to disk.

How to handle third party headers.



146
147
148
# File 'lib/puppet/provider/parsedfile.rb', line 146

def self.drop_native_header
  false
end

.filetypeObject



42
43
44
# File 'lib/puppet/provider/parsedfile.rb', line 42

def self.filetype
  @filetype ||= Puppet::Util::FileType.filetype(:flat)
end

.filetype=(type) ⇒ Object



46
47
48
49
50
51
52
53
54
# File 'lib/puppet/provider/parsedfile.rb', line 46

def self.filetype=(type)
  if type.is_a?(Class)
    @filetype = type
  elsif klass = Puppet::Util::FileType.filetype(type)
    @filetype = klass
  else
    raise ArgumentError, _("Invalid filetype %{type}") % { type: type }
  end
end

.flush(record) ⇒ Object

Flush all of the targets for which there are modified records. The only reason we pass a record here is so that we can add it to the stack if necessary – it’s passed from the instance calling ‘flush’.



59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
# File 'lib/puppet/provider/parsedfile.rb', line 59

def self.flush(record)
  # Make sure this record is on the list to be flushed.
  unless record[:on_disk]
    record[:on_disk] = true
    @records << record

    # If we've just added the record, then make sure our
    # target will get flushed.
    modified(record[:target] || default_target)
  end

  return unless defined?(@modified) and ! @modified.empty?

  flushed = []
  begin
    @modified.sort { |a,b| a.to_s <=> b.to_s }.uniq.each do |target|
      Puppet.debug "Flushing #{@resource_type.name} provider target #{target}"
      flushed << target
      flush_target(target)
    end
  ensure
    @modified.reject! { |t| flushed.include?(t) }
  end
end

.flush_target(target) ⇒ Object

Flush all of the records relating to a specific target.



97
98
99
100
101
102
103
104
105
106
107
108
109
# File 'lib/puppet/provider/parsedfile.rb', line 97

def self.flush_target(target)
  if @raise_prefetch_errors && @failed_prefetch_targets.key?(target)
    raise Puppet::Error, _("Failed to read %{target}'s records when prefetching them. Reason: %{detail}") % { target: target, detail: @failed_prefetch_targets[target] }
  end

  backup_target(target)

  records = target_records(target).reject { |r|
    r[:ensure] == :absent
  }

  target_object(target).write(to_file(records))
end

.headerObject

Return the header placed at the top of each generated file, warning users that modifying this file manually is probably a bad idea.



113
114
115
116
117
# File 'lib/puppet/provider/parsedfile.rb', line 113

def self.header
%{# HEADER: This file was autogenerated at #{Time.now}
# HEADER: by puppet.  While it can still be managed manually, it
# HEADER: is definitely not recommended.\n}
end

.initvarsObject

Add another type var.



151
152
153
154
155
156
157
158
159
160
161
162
163
164
# File 'lib/puppet/provider/parsedfile.rb', line 151

def self.initvars
  @records = []
  @target_objects = {}

  # Hash of <target> => <failure reason>.
  @failed_prefetch_targets = {}
  @raise_prefetch_errors = false

  @target = nil

  # Default to flat files
  @filetype ||= Puppet::Util::FileType.filetype(:flat)
  super
end

.instancesObject

Return a list of all of the records we can find.



167
168
169
170
171
172
173
# File 'lib/puppet/provider/parsedfile.rb', line 167

def self.instances
  targets.collect do |target|
    prefetch_target(target)
  end.flatten.reject { |r| skip_record?(r) }.collect do |record|
    new(record)
  end
end

.match_providers_with_resources(resources) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Match a list of catalog resources with provider instances

Parameters:

  • resources (Array<Puppet::Resource>)

    A list of resources using this class as a provider



233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
# File 'lib/puppet/provider/parsedfile.rb', line 233

def self.match_providers_with_resources(resources)
  return unless resources
  matchers = resources.dup
  @records.each do |record|
    # Skip things like comments and blank lines
    next if skip_record?(record)

    if (resource = resource_for_record(record, resources))
      resource.provider = new(record)
    elsif respond_to?(:match)
      if resource = match(record, matchers)
        matchers.delete(resource.title)
        record[:name] = resource[:name]
        resource.provider = new(record)
      end
    end
  end
end

.mk_resource_methodsObject

Override the default method with a lot more functionality.



176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
# File 'lib/puppet/provider/parsedfile.rb', line 176

def self.mk_resource_methods
  [resource_type.validproperties, resource_type.parameters].flatten.each do |attr|
    attr = attr.intern
    define_method(attr) do
      # If it's not a valid field for this record type (which can happen
      # when different platforms support different fields), then just
      # return the should value, so the resource shuts up.
      if @property_hash[attr] or self.class.valid_attr?(self.class.name, attr)
        @property_hash[attr] || :absent
      else
        if defined?(@resource)
          @resource.should(attr)
        else
          nil
        end
      end
    end

    define_method(attr.to_s + "=") do |val|
      mark_target_modified
      @property_hash[attr] = val
    end
  end
end

.modified(target) ⇒ Object

Mark a target as modified so we know to flush it. This only gets used within the attr= methods.



209
210
211
212
# File 'lib/puppet/provider/parsedfile.rb', line 209

def self.modified(target)
  @modified ||= []
  @modified << target unless @modified.include?(target)
end

.native_header_regexObject

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method is abstract.

Providers based on ParsedFile may implement this to make it possible to identify a header maintained by a third party tool. The provider can then allow that header to remain near the top of the written file, or remove it after composing the file content. If implemented, the function must return a Regexp object. The expression must be tailored to match exactly one third party header.

Note:

When specifying regular expressions in multiline mode, avoid greedy repetitions such as ‘.*’ (use .*? instead). Otherwise, the provider may drop file content between sparse headers.

An optional regular expression matched by third party headers.

For example, this can be used to filter the vixie cron headers as erroneously exported by older cron versions.

See Also:



135
136
137
# File 'lib/puppet/provider/parsedfile.rb', line 135

def self.native_header_regex
  nil
end

.prefetch(resources = nil) ⇒ Object

Retrieve all of the data from disk. There are three ways to know which files to retrieve: We might have a list of file objects already set up, there might be instances of our associated resource and they will have a path parameter set, and we will have a default path set. We need to turn those three locations into a list of files, prefetch each one, and make sure they’re associated with each appropriate resource instance.



221
222
223
224
225
226
# File 'lib/puppet/provider/parsedfile.rb', line 221

def self.prefetch(resources = nil)
  # Reset the record list.
  @records = prefetch_all_targets(resources)

  match_providers_with_resources(resources)
end

.prefetch_all_targets(resources) ⇒ Object



267
268
269
270
271
272
273
# File 'lib/puppet/provider/parsedfile.rb', line 267

def self.prefetch_all_targets(resources)
  records = []
  targets(resources).each do |target|
    records += prefetch_target(target)
  end
  records
end

.prefetch_target(target) ⇒ Object

Prefetch an individual target.

Raises:



276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
# File 'lib/puppet/provider/parsedfile.rb', line 276

def self.prefetch_target(target)
  begin
    target_records = retrieve(target)
  rescue Puppet::Util::FileType::FileReadError => detail
    if @raise_prefetch_errors
      # We will raise an error later in flush_target. This way,
      # only the resources linked to our target will fail
      # evaluation.
      @failed_prefetch_targets[target] = detail.to_s
    else
      puts detail.backtrace if Puppet[:trace]
      Puppet.err _("Could not prefetch %{resource} provider '%{name}' target '%{target}': %{detail}. Treating as empty") % { resource: self.resource_type.name, name: self.name, target: target, detail: detail }
    end

    target_records = []
  end

  target_records.each do |r|
    r[:on_disk] = true
    r[:target] = target
    r[:ensure] = :present
  end

  target_records = prefetch_hook(target_records) if respond_to?(:prefetch_hook)

  raise Puppet::DevError, _("Prefetching %{target} for provider %{name} returned nil") % { target: target, name: self.name } unless target_records

  target_records
end

.record?(name) ⇒ Boolean

Is there an existing record with this name?

Returns:

  • (Boolean)


307
308
309
310
# File 'lib/puppet/provider/parsedfile.rb', line 307

def self.record?(name)
  return nil unless @records
  @records.find { |r| r[:name] == name }
end

.resource_for_record(record, resources) ⇒ Puppet::Resource?

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Look up a resource based on a parsed file record

Parameters:

Returns:



260
261
262
263
264
265
# File 'lib/puppet/provider/parsedfile.rb', line 260

def self.resource_for_record(record, resources)
  name = record[:name]
  if name
    resources[name]
  end
end

.resource_type=(resource) ⇒ Object

Always make the resource methods.



202
203
204
205
# File 'lib/puppet/provider/parsedfile.rb', line 202

def self.resource_type=(resource)
  super
  mk_resource_methods
end

.retrieve(path) ⇒ Object

Retrieve the text for the file. Returns nil in the unlikely event that it doesn’t exist.



314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
# File 'lib/puppet/provider/parsedfile.rb', line 314

def self.retrieve(path)
  # XXX We need to be doing something special here in case of failure.
  text = target_object(path).read
  if text.nil? or text == ""
    # there is no file
    return []
  else
    # Set the target, for logging.
    old = @target
    begin
      @target = path
      return self.parse(text)
    rescue Puppet::Error => detail
      detail.file = @target if detail.respond_to?(:file=)
      raise detail
    ensure
      @target = old
    end
  end
end

.skip_record?(record) ⇒ Boolean

Should we skip the record? Basically, we skip text records. This is only here so subclasses can override it.

Returns:

  • (Boolean)


337
338
339
# File 'lib/puppet/provider/parsedfile.rb', line 337

def self.skip_record?(record)
  record_type(record[:record_type]).text?
end

.target_object(target) ⇒ Object

Initialize the object if necessary.



351
352
353
354
355
356
357
358
359
360
361
362
# File 'lib/puppet/provider/parsedfile.rb', line 351

def self.target_object(target)
  # only send the default mode if the actual provider defined it,
  # because certain filetypes (e.g. the crontab variants) do not
  # expect it in their initialize method
  if default_mode
    @target_objects[target] ||= filetype.new(target, default_mode)
  else
    @target_objects[target] ||= filetype.new(target)
  end

  @target_objects[target]
end

.target_records(target) ⇒ Object

Find all of the records for a given target



365
366
367
# File 'lib/puppet/provider/parsedfile.rb', line 365

def self.target_records(target)
  @records.find_all { |r| r[:target] == target }
end

.targets(resources = nil) ⇒ Object

Find a list of all of the targets that we should be reading. This is used to figure out what targets we need to prefetch.

Raises:



371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
# File 'lib/puppet/provider/parsedfile.rb', line 371

def self.targets(resources = nil)
  targets = []
  # First get the default target
  raise Puppet::DevError, _("Parsed Providers must define a default target") unless self.default_target
  targets << self.default_target

  # Then get each of the file objects
  targets += @target_objects.keys

  # Lastly, check the file from any resource instances
  if resources
    resources.each do |name, resource|
      if value = resource.should(:target)
        targets << value
      end
    end
  end

  targets.uniq.compact
end

.to_file(records) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Compose file contents from the set of records.

If self.native_header_regex is not nil, possible vendor headers are identified by matching the return value against the expression. If one (or several consecutive) such headers, are found, they are either moved in front of the self.header if self.drop_native_header is false (this is the default), or removed from the return value otherwise.



401
402
403
404
405
406
407
408
409
410
411
412
413
# File 'lib/puppet/provider/parsedfile.rb', line 401

def self.to_file(records)
  text = super
  if native_header_regex and (match = text.match(native_header_regex))
    if drop_native_header
      # concatenate the text in front of and after the native header
      text = match.pre_match + match.post_match
    else
      native_header = match[0]
      return native_header + header + match.pre_match + match.post_match
    end
  end
  header + text
end

Instance Method Details

#createObject



415
416
417
418
419
420
421
422
423
# File 'lib/puppet/provider/parsedfile.rb', line 415

def create
  @resource.class.validproperties.each do |property|
    if value = @resource.should(property)
      @property_hash[property] = value
    end
  end
  mark_target_modified
  (@resource.class.name.to_s + "_created").intern
end

#destroyObject



425
426
427
428
429
# File 'lib/puppet/provider/parsedfile.rb', line 425

def destroy
  # We use the method here so it marks the target as modified.
  self.ensure = :absent
  (@resource.class.name.to_s + "_deleted").intern
end

#exists?Boolean

Returns:

  • (Boolean)


431
432
433
# File 'lib/puppet/provider/parsedfile.rb', line 431

def exists?
  !(@property_hash[:ensure] == :absent or @property_hash[:ensure].nil?)
end

#flushObject

Write our data to disk.



436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
# File 'lib/puppet/provider/parsedfile.rb', line 436

def flush
  # Make sure we've got a target and name set.

  # If the target isn't set, then this is our first modification, so
  # mark it for flushing.
  unless @property_hash[:target]
    @property_hash[:target] = @resource.should(:target) || self.class.default_target
    self.class.modified(@property_hash[:target])
  end
  @resource.class.key_attributes.each do |attr|
    @property_hash[attr] ||= @resource[attr]
  end

  self.class.flush(@property_hash)
end

#prefetchObject

Retrieve the current state from disk.

Raises:



462
463
464
465
# File 'lib/puppet/provider/parsedfile.rb', line 462

def prefetch
  raise Puppet::DevError, _("Somehow got told to prefetch with no resource set") unless @resource
  self.class.prefetch(@resource[:name] => @resource)
end

#record_typeObject



467
468
469
# File 'lib/puppet/provider/parsedfile.rb', line 467

def record_type
  @property_hash[:record_type]
end