Class: Parser::Source::TreeRewriter

Inherits:
Object
  • Object
show all
Defined in:
lib/parser/source/tree_rewriter.rb

Overview

TreeRewriter performs the heavy lifting in the source rewriting process. It schedules code updates to be performed in the correct order.

For simple cases, the resulting source will be obvious.

Examples for more complex cases follow. Assume these examples are acting on the source 'puts(:hello, :world). The methods #wrap, #remove, etc. receive a Range as first argument; for clarity, examples below use english sentences and a string of raw code instead.

Overlapping ranges:

Any two rewriting actions on overlapping ranges will fail and raise a ClobberingError, unless they are both deletions (covered next).

  • wrap ':hello, ' with '(' and ')'
  • wrap ', :world' with '(' and ')' => CloberringError

Overlapping deletions:

  • remove ':hello, '
  • remove ', :world'

The overlapping ranges are merged and ':hello, :world' will be removed. This policy can be changed. :crossing_deletions defaults to :accept but can be set to :warn or :raise.

Multiple actions at the same end points:

Results will always be independent on the order they were given. Exception: rewriting actions done on exactly the same range (covered next).

Example:

  • replace ', ' by ' => '
  • wrap ':hello, :world' with 'and ''
  • replace ':world' with ':everybody'
  • wrap ':world' with '[', ']'

The resulting string will be 'puts({:hello => [:everybody]})' and this result is independent on the order the instructions were given in.

Note that if the two "replace" were given as a single replacement of ', :world' for ' => :everybody', the result would be a ClobberingError because of the wrap in square brackets.

Multiple wraps on same range:

  • wrap ':hello' with '(' and ')'
  • wrap ':hello' with '[' and ']'

The wraps are combined in order given and results would be 'puts([(:hello)], :world)'.

Multiple replacements on same range:

  • replace ':hello' by ':hi', then
  • replace ':hello' by ':hey'

The replacements are made in the order given, so the latter replacement supersedes the former and ':hello' will be replaced by ':hey'.

This policy can be changed. :different_replacements defaults to :accept but can be set to :warn or :raise.

Swallowed insertions:

wrap 'world' by '', '' replace ':hello, :world' with ':hi'

A containing replacement will swallow the contained rewriting actions and ':hello, :world' will be replaced by ':hi'.

This policy can be changed for swallowed insertions. :swallowed_insertions defaults to :accept but can be set to :warn or :raise

Implementation

The updates are organized in a tree, according to the ranges they act on (where children are strictly contained by their parent), hence the name.

Constant Summary collapse

DEPRECATION_WARNING = 
[
  'TreeRewriter#insert_before_multi and insert_before_multi exist only for legacy compatibility.',
  'Please update your code to use `wrap`, `insert_before` or `insert_after` instead.'
].join("\n").freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(source_buffer, crossing_deletions: :accept, different_replacements: :accept, swallowed_insertions: :accept) ⇒ TreeRewriter

Returns a new instance of TreeRewriter.

Parameters:



98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
 
# File 'lib/parser/source/tree_rewriter.rb', line 98

def initialize(source_buffer,
               crossing_deletions: :accept,
               different_replacements: :accept,
               swallowed_insertions: :accept)
  @diagnostics = Diagnostic::Engine.new
  @diagnostics.consumer = -> diag { $stderr.puts diag.render }

  @source_buffer = source_buffer
  @in_transaction = false

  @policy = {crossing_deletions: crossing_deletions,
             different_replacements: different_replacements,
             swallowed_insertions: swallowed_insertions}.freeze
  check_policy_validity

  @enforcer = method(:enforce_policy)
  # We need a range that would be jugded as containing all other ranges,
  # including 0...0 and size...size:
  all_encompassing_range = @source_buffer.source_range.adjust(begin_pos: -1, end_pos: +1)
  @action_root = TreeRewriter::Action.new(all_encompassing_range, @enforcer)
end

Instance Attribute Details

#diagnostics ⇒ Diagnostic::Engine (readonly)

Returns:



91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
 
# File 'lib/parser/source/tree_rewriter.rb', line 91

class TreeRewriter
  attr_reader :source_buffer
  attr_reader :diagnostics

  ##
  # @param [Source::Buffer] source_buffer
  #
  def initialize(source_buffer,
                 crossing_deletions: :accept,
                 different_replacements: :accept,
                 swallowed_insertions: :accept)
    @diagnostics = Diagnostic::Engine.new
    @diagnostics.consumer = -> diag { $stderr.puts diag.render }

    @source_buffer = source_buffer
    @in_transaction = false

    @policy = {crossing_deletions: crossing_deletions,
               different_replacements: different_replacements,
               swallowed_insertions: swallowed_insertions}.freeze
    check_policy_validity

    @enforcer = method(:enforce_policy)
    # We need a range that would be jugded as containing all other ranges,
    # including 0...0 and size...size:
    all_encompassing_range = @source_buffer.source_range.adjust(begin_pos: -1, end_pos: +1)
    @action_root = TreeRewriter::Action.new(all_encompassing_range, @enforcer)
  end

  ##
  # Returns true iff no (non trivial) update has been recorded
  #
  # @return [Boolean]
  #
  def empty?
    @action_root.empty?
  end

  ##
  # Merges the updates of argument with the receiver.
  # Policies of the receiver are used.
  #
  # @param [Rewriter] with
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def merge!(with)
    raise 'TreeRewriter are not for the same source_buffer' unless
      source_buffer == with.source_buffer

    @action_root = @action_root.combine(with.action_root)
    self
  end

  ##
  # Returns a new rewriter that consists of the updates of the received
  # and the given argument. Policies of the receiver are used.
  #
  # @param [Rewriter] with
  # @return [Rewriter] merge of receiver and argument
  # @raise [ClobberingError] when clobbering is detected
  #
  def merge(with)
    dup.merge!(with)
  end

  ##
  # Replaces the code of the source range `range` with `content`.
  #
  # @param [Range] range
  # @param [String] content
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def replace(range, content)
    combine(range, replacement: content)
  end

  ##
  # Inserts the given strings before and after the given range.
  #
  # @param [Range] range
  # @param [String, nil] insert_before
  # @param [String, nil] insert_after
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def wrap(range, insert_before, insert_after)
    combine(range, insert_before: insert_before.to_s, insert_after: insert_after.to_s)
  end

  ##
  # Shortcut for `replace(range, '')`
  #
  # @param [Range] range
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def remove(range)
    replace(range, ''.freeze)
  end


  ##
  # Shortcut for `wrap(range, content, nil)`
  #
  # @param [Range] range
  # @param [String] content
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def insert_before(range, content)
    wrap(range, content, nil)
  end

  ##
  # Shortcut for `wrap(range, nil, content)`
  #
  # @param [Range] range
  # @param [String] content
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def insert_after(range, content)
    wrap(range, nil, content)
  end

  ##
  # Applies all scheduled changes to the `source_buffer` and returns
  # modified source as a new string.
  #
  # @return [String]
  #
 def process
    source     = @source_buffer.source

    chunks = []
    last_end = 0
    @action_root.ordered_replacements.each do |range, replacement|
      chunks << source[last_end...range.begin_pos] << replacement
      last_end = range.end_pos
    end
    chunks << source[last_end...source.length]
    chunks.join
  end

  ##
  # Provides a protected block where a sequence of multiple rewrite actions
  # are handled atomically. If any of the actions failed by clobbering,
  # all the actions are rolled back. Transactions can be nested.
  #
  # @raise [RuntimeError] when no block is passed
  #
  def transaction
    unless block_given?
      raise "#{self.class}##{__method__} requires block"
    end

    previous = @in_transaction
    @in_transaction = true
    restore_root = @action_root

    yield

    restore_root = nil

    self
  ensure
    @action_root = restore_root if restore_root
    @in_transaction = previous
  end

  def in_transaction?
    @in_transaction
  end

  ##
  # @api private
  # @deprecated Use insert_after or wrap
  #
  def insert_before_multi(range, text)
    self.class.warn_of_deprecation
    insert_before(range, text)
  end

  ##
  # @api private
  # @deprecated Use insert_after or wrap
  #
  def insert_after_multi(range, text)
    self.class.warn_of_deprecation
    insert_after(range, text)
  end

  DEPRECATION_WARNING = [
    'TreeRewriter#insert_before_multi and insert_before_multi exist only for legacy compatibility.',
    'Please update your code to use `wrap`, `insert_before` or `insert_after` instead.'
  ].join("\n").freeze

  extend Deprecation

  protected

  attr_reader :action_root

  private

  ACTIONS = %i[accept warn raise].freeze
  def check_policy_validity
    invalid = @policy.values - ACTIONS
    raise ArgumentError, "Invalid policy: #{invalid.join(', ')}" unless invalid.empty?
  end

  def combine(range, attributes)
    range = check_range_validity(range)
    action = TreeRewriter::Action.new(range, @enforcer, **attributes)
    @action_root = @action_root.combine(action)
    self
  end

  def check_range_validity(range)
    if range.begin_pos < 0 || range.end_pos > @source_buffer.source.size
      raise IndexError, "The range #{range} is outside the bounds of the source"
    end
    range
  end

  def enforce_policy(event)
    return if @policy[event] == :accept
    return unless (values = yield)
    trigger_policy(event, **values)
  end

  POLICY_TO_LEVEL = {warn: :warning, raise: :error}.freeze
  def trigger_policy(event, range: raise, conflict: nil, **arguments)
    action = @policy[event] || :raise
    diag = Parser::Diagnostic.new(POLICY_TO_LEVEL[action], event, arguments, range)
    @diagnostics.process(diag)
    if conflict
      range, *highlights = conflict
      diag = Parser::Diagnostic.new(POLICY_TO_LEVEL[action], :"#{event}_conflict", arguments, range, highlights)
      @diagnostics.process(diag)
    end
    raise Parser::ClobberingError, "Parser::Source::TreeRewriter detected clobbering" if action == :raise
  end
end

#source_buffer ⇒ Source::Buffer (readonly)

Returns:



91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
 
# File 'lib/parser/source/tree_rewriter.rb', line 91

class TreeRewriter
  attr_reader :source_buffer
  attr_reader :diagnostics

  ##
  # @param [Source::Buffer] source_buffer
  #
  def initialize(source_buffer,
                 crossing_deletions: :accept,
                 different_replacements: :accept,
                 swallowed_insertions: :accept)
    @diagnostics = Diagnostic::Engine.new
    @diagnostics.consumer = -> diag { $stderr.puts diag.render }

    @source_buffer = source_buffer
    @in_transaction = false

    @policy = {crossing_deletions: crossing_deletions,
               different_replacements: different_replacements,
               swallowed_insertions: swallowed_insertions}.freeze
    check_policy_validity

    @enforcer = method(:enforce_policy)
    # We need a range that would be jugded as containing all other ranges,
    # including 0...0 and size...size:
    all_encompassing_range = @source_buffer.source_range.adjust(begin_pos: -1, end_pos: +1)
    @action_root = TreeRewriter::Action.new(all_encompassing_range, @enforcer)
  end

  ##
  # Returns true iff no (non trivial) update has been recorded
  #
  # @return [Boolean]
  #
  def empty?
    @action_root.empty?
  end

  ##
  # Merges the updates of argument with the receiver.
  # Policies of the receiver are used.
  #
  # @param [Rewriter] with
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def merge!(with)
    raise 'TreeRewriter are not for the same source_buffer' unless
      source_buffer == with.source_buffer

    @action_root = @action_root.combine(with.action_root)
    self
  end

  ##
  # Returns a new rewriter that consists of the updates of the received
  # and the given argument. Policies of the receiver are used.
  #
  # @param [Rewriter] with
  # @return [Rewriter] merge of receiver and argument
  # @raise [ClobberingError] when clobbering is detected
  #
  def merge(with)
    dup.merge!(with)
  end

  ##
  # Replaces the code of the source range `range` with `content`.
  #
  # @param [Range] range
  # @param [String] content
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def replace(range, content)
    combine(range, replacement: content)
  end

  ##
  # Inserts the given strings before and after the given range.
  #
  # @param [Range] range
  # @param [String, nil] insert_before
  # @param [String, nil] insert_after
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def wrap(range, insert_before, insert_after)
    combine(range, insert_before: insert_before.to_s, insert_after: insert_after.to_s)
  end

  ##
  # Shortcut for `replace(range, '')`
  #
  # @param [Range] range
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def remove(range)
    replace(range, ''.freeze)
  end


  ##
  # Shortcut for `wrap(range, content, nil)`
  #
  # @param [Range] range
  # @param [String] content
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def insert_before(range, content)
    wrap(range, content, nil)
  end

  ##
  # Shortcut for `wrap(range, nil, content)`
  #
  # @param [Range] range
  # @param [String] content
  # @return [Rewriter] self
  # @raise [ClobberingError] when clobbering is detected
  #
  def insert_after(range, content)
    wrap(range, nil, content)
  end

  ##
  # Applies all scheduled changes to the `source_buffer` and returns
  # modified source as a new string.
  #
  # @return [String]
  #
 def process
    source     = @source_buffer.source

    chunks = []
    last_end = 0
    @action_root.ordered_replacements.each do |range, replacement|
      chunks << source[last_end...range.begin_pos] << replacement
      last_end = range.end_pos
    end
    chunks << source[last_end...source.length]
    chunks.join
  end

  ##
  # Provides a protected block where a sequence of multiple rewrite actions
  # are handled atomically. If any of the actions failed by clobbering,
  # all the actions are rolled back. Transactions can be nested.
  #
  # @raise [RuntimeError] when no block is passed
  #
  def transaction
    unless block_given?
      raise "#{self.class}##{__method__} requires block"
    end

    previous = @in_transaction
    @in_transaction = true
    restore_root = @action_root

    yield

    restore_root = nil

    self
  ensure
    @action_root = restore_root if restore_root
    @in_transaction = previous
  end

  def in_transaction?
    @in_transaction
  end

  ##
  # @api private
  # @deprecated Use insert_after or wrap
  #
  def insert_before_multi(range, text)
    self.class.warn_of_deprecation
    insert_before(range, text)
  end

  ##
  # @api private
  # @deprecated Use insert_after or wrap
  #
  def insert_after_multi(range, text)
    self.class.warn_of_deprecation
    insert_after(range, text)
  end

  DEPRECATION_WARNING = [
    'TreeRewriter#insert_before_multi and insert_before_multi exist only for legacy compatibility.',
    'Please update your code to use `wrap`, `insert_before` or `insert_after` instead.'
  ].join("\n").freeze

  extend Deprecation

  protected

  attr_reader :action_root

  private

  ACTIONS = %i[accept warn raise].freeze
  def check_policy_validity
    invalid = @policy.values - ACTIONS
    raise ArgumentError, "Invalid policy: #{invalid.join(', ')}" unless invalid.empty?
  end

  def combine(range, attributes)
    range = check_range_validity(range)
    action = TreeRewriter::Action.new(range, @enforcer, **attributes)
    @action_root = @action_root.combine(action)
    self
  end

  def check_range_validity(range)
    if range.begin_pos < 0 || range.end_pos > @source_buffer.source.size
      raise IndexError, "The range #{range} is outside the bounds of the source"
    end
    range
  end

  def enforce_policy(event)
    return if @policy[event] == :accept
    return unless (values = yield)
    trigger_policy(event, **values)
  end

  POLICY_TO_LEVEL = {warn: :warning, raise: :error}.freeze
  def trigger_policy(event, range: raise, conflict: nil, **arguments)
    action = @policy[event] || :raise
    diag = Parser::Diagnostic.new(POLICY_TO_LEVEL[action], event, arguments, range)
    @diagnostics.process(diag)
    if conflict
      range, *highlights = conflict
      diag = Parser::Diagnostic.new(POLICY_TO_LEVEL[action], :"#{event}_conflict", arguments, range, highlights)
      @diagnostics.process(diag)
    end
    raise Parser::ClobberingError, "Parser::Source::TreeRewriter detected clobbering" if action == :raise
  end
end

Instance Method Details

#empty? ⇒ Boolean

Returns true iff no (non trivial) update has been recorded

Returns:

  • (Boolean) 


125
126
127
 
# File 'lib/parser/source/tree_rewriter.rb', line 125

def empty?
  @action_root.empty?
end

#in_transaction? ⇒ Boolean

Returns:

  • (Boolean) 


263
264
265
 
# File 'lib/parser/source/tree_rewriter.rb', line 263

def in_transaction?
  @in_transaction
end

#insert_after(range, content) ⇒ Rewriter

Shortcut for wrap(range, nil, content)

Parameters:

  • range (Range)
  • content (String)

Returns:

Raises:



214
215
216
 
# File 'lib/parser/source/tree_rewriter.rb', line 214

def insert_after(range, content)
  wrap(range, nil, content)
end

#insert_before(range, content) ⇒ Rewriter

Shortcut for wrap(range, content, nil)

Parameters:

  • range (Range)
  • content (String)

Returns:

Raises:



202
203
204
 
# File 'lib/parser/source/tree_rewriter.rb', line 202

def insert_before(range, content)
  wrap(range, content, nil)
end

#merge(with) ⇒ Rewriter

Returns a new rewriter that consists of the updates of the received and the given argument. Policies of the receiver are used.

Parameters:

Returns:

  • (Rewriter) —

    merge of receiver and argument

Raises:



153
154
155
 
# File 'lib/parser/source/tree_rewriter.rb', line 153

def merge(with)
  dup.merge!(with)
end

#merge!(with) ⇒ Rewriter

Merges the updates of argument with the receiver. Policies of the receiver are used.

Parameters:

Returns:

Raises:



137
138
139
140
141
142
143
 
# File 'lib/parser/source/tree_rewriter.rb', line 137

def merge!(with)
  raise 'TreeRewriter are not for the same source_buffer' unless
    source_buffer == with.source_buffer

  @action_root = @action_root.combine(with.action_root)
  self
end

#process ⇒ String

Applies all scheduled changes to the source_buffer and returns modified source as a new string.

Returns:

  • (String) 


224
225
226
227
228
229
230
231
232
233
234
235
 
# File 'lib/parser/source/tree_rewriter.rb', line 224

def process
  source     = @source_buffer.source

  chunks = []
  last_end = 0
  @action_root.ordered_replacements.each do |range, replacement|
    chunks << source[last_end...range.begin_pos] << replacement
    last_end = range.end_pos
  end
  chunks << source[last_end...source.length]
  chunks.join
end

#remove(range) ⇒ Rewriter

Shortcut for replace(range, '')

Parameters:

Returns:

Raises:



189
190
191
 
# File 'lib/parser/source/tree_rewriter.rb', line 189

def remove(range)
  replace(range, ''.freeze)
end

#replace(range, content) ⇒ Rewriter

Replaces the code of the source range range with content.

Parameters:

  • range (Range)
  • content (String)

Returns:

Raises:



165
166
167
 
# File 'lib/parser/source/tree_rewriter.rb', line 165

def replace(range, content)
  combine(range, replacement: content)
end

#transaction ⇒ Object

Provides a protected block where a sequence of multiple rewrite actions are handled atomically. If any of the actions failed by clobbering, all the actions are rolled back. Transactions can be nested.

Raises:

  • (RuntimeError) —

    when no block is passed



244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
 
# File 'lib/parser/source/tree_rewriter.rb', line 244

def transaction
  unless block_given?
    raise "#{self.class}##{__method__} requires block"
  end

  previous = @in_transaction
  @in_transaction = true
  restore_root = @action_root

  yield

  restore_root = nil

  self
ensure
  @action_root = restore_root if restore_root
  @in_transaction = previous
end

#wrap(range, insert_before, insert_after) ⇒ Rewriter

Inserts the given strings before and after the given range.

Parameters:

  • range (Range)
  • insert_before (String, nil)
  • insert_after (String, nil)

Returns:

Raises:



178
179
180
 
# File 'lib/parser/source/tree_rewriter.rb', line 178

def wrap(range, insert_before, insert_after)
  combine(range, insert_before: insert_before.to_s, insert_after: insert_after.to_s)
end