Class: Pry::Indent

Inherits:
Object show all
Includes:
Helpers::BaseHelpers
Defined in:
lib/pry/indent.rb

Overview

Pry::Indent is a class that can be used to indent a number of lines containing Ruby code similar as to how IRB does it (but better). The class works by tokenizing a string using CodeRay and then looping over those tokens. Based on the tokens in a line of code that line (or the next one) will be indented or un-indented by correctly.

Defined Under Namespace

Classes: UnparseableNestingError

Constant Summary collapse

SPACES =

The amount of spaces to insert for each indent level.

'  '
OPEN_TOKENS =

Hash containing all the tokens that should increase the indentation level. The keys of this hash are open tokens, the values the matching tokens that should prevent a line from being indented if they appear on the same line.

{
  'def'    => 'end',
  'class'  => 'end',
  'module' => 'end',
  'do'     => 'end',
  'if'     => 'end',
  'unless' => 'end',
  'while'  => 'end',
  'until'  => 'end',
  'for'    => 'end',
  'case'   => 'end',
  'begin'  => 'end',
  '['      => ']',
  '{'      => '}',
  '('      => ')'
}
SINGLELINE_TOKENS =

Which tokens can either be open tokens, or appear as modifiers on a single-line.

%w(if while until unless rescue)
OPTIONAL_DO_TOKENS =

Which tokens can be followed by an optional “do” keyword.

%w(for while until)
IGNORE_TOKENS =

Collection of token types that should be ignored. Without this list keywords such as “class” inside strings would cause the code to be indented incorrectly.

:pre_constant and :preserved_constant are the CodeRay 0.9.8 and 1.0.0 classifications of “true”, “false”, and “nil”.

[:space, :content, :string, :method, :ident,
:constant, :pre_constant, :predefined_constant]
STATEMENT_END_TOKENS =

Tokens that indicate the end of a statement (i.e. that, if they appear directly before an “if” indicates that that if applies to the same line, not the next line)

:reserved and :keywords are the CodeRay 0.9.8 and 1.0.0 respectively classifications of “super”, “next”, “return”, etc.

IGNORE_TOKENS + [:regexp, :integer, :float, :keyword,
:delimiter, :reserved]
MIDWAY_TOKENS =

Collection of tokens that should appear dedented even though they don’t affect the surrounding code.

%w(when else elsif ensure rescue)

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Helpers::BaseHelpers

colorize_code, command_dependencies_met?, find_command, heading, highlight, jruby?, jruby_19?, mri?, mri_19?, mri_20?, mri_21?, mri_2?, not_a_real_file?, rbx?, #safe_send, safe_send, silence_warnings, stagger_output, use_ansi_codes?, windows?, windows_ansi?

Constructor Details

#initialize ⇒ Indent

Returns a new instance of Indent.



102
103
104
 
# File 'lib/pry/indent.rb', line 102

def initialize
  reset
end

Instance Attribute Details

#indent_level ⇒ String (readonly)

Returns String containing the spaces to be inserted before the next line.

Returns:

  • (String) —

    String containing the spaces to be inserted before the next line.



18
19
20
 
# File 'lib/pry/indent.rb', line 18

def indent_level
  @indent_level
end

#stack ⇒ Array<String> (readonly)

Returns The stack of open tokens.

Returns:

  • (Array<String>) —

    The stack of open tokens.



21
22
23
 
# File 'lib/pry/indent.rb', line 21

def stack
  @stack
end

Class Method Details

.indent(str) ⇒ String

Clean the indentation of a fragment of ruby.

Parameters:

  • str (String)

Returns:

  • (String) 


80
81
82
 
# File 'lib/pry/indent.rb', line 80

def self.indent(str)
  new.indent(str)
end

.nesting_at(str, line_number) ⇒ Array<String>

Get the module nesting at the given point in the given string.

NOTE If the line specified contains a method definition, then the nesting at the start of the method definition is used. Otherwise the nesting from the end of the line is used.

Parameters:

  • str (String) —

    The ruby code to analyze

  • line_number (Fixnum) —

    The line number (starting from 1)

Returns:

  • (Array<String>) 


93
94
95
96
97
98
99
100
 
# File 'lib/pry/indent.rb', line 93

def self.nesting_at(str, line_number)
  indent = new
  lines = str.split("\n")
  n = line_number - 1
  to_indent = lines[0...n] << (lines[n] || "").split("def").first(1)
  indent.indent(to_indent.join("\n") << "\n")
  indent.module_nesting
end

Instance Method Details

#correct_indentation(prompt, code, overhang = 0) ⇒ String

Return a string which, when printed, will rewrite the previous line with the correct indentation. Mostly useful for fixing ‘end’.

Parameters:

  • prompt (String) —

    The user’s prompt

  • code (String) —

    The code the user just typed in.

  • overhang (Fixnum) (defaults to: 0) —

    (0) The number of chars to erase afterwards (i.e., the difference in length between the old line and the new one).

Returns:

  • (String) 


385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
 
# File 'lib/pry/indent.rb', line 385

def correct_indentation(prompt, code, overhang=0)
  prompt = prompt.delete("\001\002")
  line_to_measure = Pry::Helpers::Text.strip_color(prompt) << code
  whitespace = ' ' * overhang

  _, cols = Terminal.screen_size

  cols = cols.to_i
  lines = (cols != 0 ? (line_to_measure.length / cols + 1) : 1).to_i

  if Pry::Helpers::BaseHelpers.windows_ansi?
    move_up   = "\e[#{lines}F"
    move_down = "\e[#{lines}E"
  else
    move_up   = "\e[#{lines}A\e[0G"
    move_down = "\e[#{lines}B\e[0G"
  end

  "#{move_up}#{prompt}#{colorize_code(code)}#{whitespace}#{move_down}"
end

#current_prefix ⇒ Object

Get the indentation for the start of the next line.

This is what’s used between the prompt and the cursor in pry.

Returns:

  • String The correct number of spaces



177
178
179
 
# File 'lib/pry/indent.rb', line 177

def current_prefix
  in_string? ? '' : indent_level
end

#end_of_statement?(last_token, last_kind) ⇒ Boolean

If the code just before an “if” or “while” token on a line looks like the end of a statement, then we want to treat that “if” as a singleline, not multiline statement.

Returns:

  • (Boolean) 


252
253
254
 
# File 'lib/pry/indent.rb', line 252

def end_of_statement?(last_token, last_kind)
  (last_token =~ /^[)\]}\/]$/ || STATEMENT_END_TOKENS.include?(last_kind))
end

#in_string? ⇒ Boolean

Are we currently in the middle of a string literal.

This is used to determine whether to re-indent a given line, we mustn’t re-indent within string literals because to do so would actually change the value of the String!

Returns:

  • (Boolean) —

    Boolean



263
264
265
 
# File 'lib/pry/indent.rb', line 263

def in_string?
  !open_delimiters.empty?
end

#indent(input) ⇒ String

Indents a string and returns it. This string can either be a single line or multiple ones.

Examples:

str = <<TXT
class User
attr_accessor :name
end
TXT

# This would result in the following being displayed:
#
# class User
#   attr_accessor :name
# end
#
puts Pry::Indent.new.indent(str)

Parameters:

  • input (String) —

    The input string to indent.

Returns:

  • (String) —

    The indented version of input.



139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
 
# File 'lib/pry/indent.rb', line 139

def indent(input)
  output = ''
  prefix = indent_level

  input.lines.each do |line|

    if in_string?
      tokens = tokenize("#{open_delimiters_line}\n#{line}")
      tokens = tokens.drop_while{ |token, type| !(String === token && token.include?("\n")) }
      previously_in_string = true
    else
      tokens = tokenize(line)
      previously_in_string = false
    end

    before, after = indentation_delta(tokens)

    before.times{ prefix.sub! SPACES, '' }
    new_prefix = prefix + SPACES * after

    line = prefix + line.lstrip unless previously_in_string

    output += line

    prefix = new_prefix
  end

  @indent_level = prefix

  return output
end

#indentation_delta(tokens) ⇒ Array[Integer]

Get the change in indentation indicated by the line.

By convention, you remove indent from the line containing end tokens, but add indent to the line after that which contains the start tokens.

This method returns a pair, where the first number is the number of closings on this line (i.e. the number of indents to remove before the line) and the second is the number of openings (i.e. the number of indents to add after this line)

Parameters:

  • tokens (Array) —

    A list of tokens to scan.

Returns:

  • (Array[Integer]) 


194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
 
# File 'lib/pry/indent.rb', line 194

def indentation_delta(tokens)

  # We need to keep track of whether we've seen a "for" on this line because
  # if the line ends with "do" then that "do" should be discounted (i.e. we're
  # only opening one level not two) To do this robustly we want to keep track
  # of the indent level at which we saw the for, so we can differentiate
  # between "for x in [1,2,3] do" and "for x in ([1,2,3].map do" properly
  seen_for_at = []

  # When deciding whether an "if" token is the start of a multiline statement,
  # or just the middle of a single-line if statement, we just look at the
  # preceding token, which is tracked here.
  last_token, last_kind = [nil, nil]

  # delta keeps track of the total difference from the start of each line after
  # the given token, 0 is just the level at which the current line started for
  # reference.
  remove_before, add_after = [0, 0]

  # If the list of tokens contains a matching closing token the line should
  # not be indented (and thus we should return true).
  tokens.each do |token, kind|
    is_singleline_if  = (SINGLELINE_TOKENS.include?(token)) && end_of_statement?(last_token, last_kind)
    is_optional_do = (token == "do" && seen_for_at.include?(add_after - 1))

    last_token, last_kind = token, kind unless kind == :space
    next if IGNORE_TOKENS.include?(kind)

    track_module_nesting(token, kind)

    seen_for_at << add_after if OPTIONAL_DO_TOKENS.include?(token)

    if kind == :delimiter
      track_delimiter(token)
    elsif OPEN_TOKENS.keys.include?(token) && !is_optional_do && !is_singleline_if
      @stack << token
      add_after += 1
    elsif token == OPEN_TOKENS[@stack.last]
      popped = @stack.pop
      track_module_nesting_end(popped)
      if add_after == 0
        remove_before += 1
      else
        add_after -= 1
      end
    elsif MIDWAY_TOKENS.include?(token)
      if add_after == 0
        remove_before += 1
        add_after += 1
      end
    end
  end

  return [remove_before, add_after]
end

#module_nesting ⇒ Array<String>

Return a list of strings which can be used to re-construct the Module.nesting at the current point in the file.

Returns nil if the syntax of the file was not recognizable.

Returns:

  • (Array<String>) 


369
370
371
372
373
374
375
 
# File 'lib/pry/indent.rb', line 369

def module_nesting
  @module_nesting.map do |(kind, token)|
    raise UnparseableNestingError, @module_nesting.inspect if token.nil?

    "#{kind} #{token}"
  end
end

#open_delimiters ⇒ String

All the open delimiters, in the order that they first appeared.

Returns:

  • (String) 


303
304
305
 
# File 'lib/pry/indent.rb', line 303

def open_delimiters
  @heredoc_queue + [@string_start].compact
end

#open_delimiters_line ⇒ Object

Return a string which restores the CodeRay string status to the correct value by opening HEREDOCs and strings.

Returns:

  • String



311
312
313
 
# File 'lib/pry/indent.rb', line 311

def open_delimiters_line
  "puts #{open_delimiters.join(", ")}"
end

#reset ⇒ Object

reset internal state



107
108
109
110
111
112
113
114
115
116
 
# File 'lib/pry/indent.rb', line 107

def reset
  @stack = []
  @indent_level = ''
  @heredoc_queue = []
  @close_heredocs = {}
  @string_start = nil
  @awaiting_class = false
  @module_nesting = []
  self
end

#tokenize(string) ⇒ Array

Given a string of Ruby code, use CodeRay to export the tokens.

Parameters:

  • string (String) —

    The Ruby to lex

Returns:

  • (Array) —

    An Array of pairs of [token_value, token_type]



271
272
273
274
275
 
# File 'lib/pry/indent.rb', line 271

def tokenize(string)
  tokens = CodeRay.scan(string, :ruby)
  tokens = tokens.tokens.each_slice(2) if tokens.respond_to?(:tokens) # Coderay 1.0.0
  tokens.to_a
end

#track_delimiter(token) ⇒ Object

Update the internal state about what kind of strings are open.

Most of the complication here comes from the fact that HEREDOCs can be nested. For normal strings (which can’t be nested) we assume that CodeRay correctly pairs open-and-close delimiters so we don’t bother checking what they are.

Parameters:

  • token (String) —

    The token (of type :delimiter)



284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
 
# File 'lib/pry/indent.rb', line 284

def track_delimiter(token)
  case token
  when /^<<-(["'`]?)(.*)\\1/
    @heredoc_queue << token
    @close_heredocs[token] = /^\s*$2/
  when @close_heredocs[@heredoc_queue.first]
    @heredoc_queue.shift
  else
    if @string_start
      @string_start = nil
    else
      @string_start = token
    end
  end
end

#track_module_nesting(token, kind) ⇒ Object

Update the internal state relating to module nesting.

It’s responsible for adding to the @module_nesting array, which looks something like:

[“class”, “Foo”], [“module”, “Bar::Baz”], [“class <<”, “self”

]

A nil value in the @module_nesting array happens in two places: either when @awaiting_class is true and we’re still waiting for the string to fill that space, or when a parse was rejected.

At the moment this function is quite restricted about what formats it will parse, for example we disallow expressions after the class keyword. This could maybe be improved in the future.

Parameters:

  • token (String) —

    a token from Coderay

  • kind (Symbol) —

    the kind of that token



332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
 
# File 'lib/pry/indent.rb', line 332

def track_module_nesting(token, kind)
  if kind == :keyword && (token == "class" || token == "module")
    @module_nesting << [token, nil]
    @awaiting_class = true
  elsif @awaiting_class
    if kind == :operator && token == "<<" && @module_nesting.last[0] == "class"
      @module_nesting.last[0] = "class <<"
      @awaiting_class = true
    elsif kind == :class && token =~ /\A(self|[A-Z:][A-Za-z0-9_:]*)\z/
      @module_nesting.last[1] = token if kind == :class
      @awaiting_class = false
    else
      # leave @module_nesting[-1]
      @awaiting_class = false
    end
  end
end

#track_module_nesting_end(token, kind = :keyword) ⇒ Object

Update the internal state relating to module nesting on ‘end’.

If the current ‘end’ pairs up with a class or a module then we should pop an array off of @module_nesting

Parameters:

  • token (String) —

    a token from Coderay

  • kind (Symbol) (defaults to: :keyword) —

    the kind of that token



357
358
359
360
361
 
# File 'lib/pry/indent.rb', line 357

def track_module_nesting_end(token, kind=:keyword)
  if kind == :keyword && (token == "class" || token == "module")
    @module_nesting.pop
  end
end