Class: Oga::CSS::Lexer Private

Inherits:

Object

• Object
• Oga::CSS::Lexer

Defined in:

lib/oga/css/lexer.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

Lexer for turning CSS expressions into a sequence of tokens. Tokens are returned as arrays with every array having two values:

1. The token type as a Symbol
2. The token value, or nil if there is no value.

Thread Safety

Similar to the XPath lexer this lexer keeps track of an internal state. As a result it's not safe to share the same instance of this lexer between multiple threads. However, no global state is used so you can use separate instances in threads just fine.

Class Attribute Summary

• ._css_lexer_eof_trans ⇒ Object private
• ._css_lexer_from_state_actions ⇒ Object private
• ._css_lexer_index_offsets ⇒ Object private
• ._css_lexer_indicies ⇒ Object private
• ._css_lexer_key_spans ⇒ Object private
• ._css_lexer_to_state_actions ⇒ Object private
• ._css_lexer_trans_actions ⇒ Object private
• ._css_lexer_trans_keys ⇒ Object private
• ._css_lexer_trans_targs ⇒ Object private
• .css_lexer_en_main ⇒ Object private
• .css_lexer_en_predicate ⇒ Object private
• .css_lexer_en_pseudo_args ⇒ Object private
• .css_lexer_error ⇒ Object private
• .css_lexer_first_final ⇒ Object private
• .css_lexer_start ⇒ Object private

Instance Method Summary

• #advance(&block) ⇒ Object private

  Advances through the input and generates the corresponding tokens.

• #initialize(data) ⇒ Lexer constructor private

  A new instance of Lexer.

• #lex ⇒ Array private

  Gathers all the tokens for the input and returns them as an Array.

Constructor Details

#initialize(data) ⇒ Lexer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Lexer.

Parameters:

• data (String) —

  The data to lex.

# File 'lib/oga/css/lexer.rb', line 278

def initialize(data)
  @data = data
end

Class Attribute Details

._css_lexer_eof_trans ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 234

def _css_lexer_eof_trans
  @_css_lexer_eof_trans
end

._css_lexer_from_state_actions ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 223

def _css_lexer_from_state_actions
  @_css_lexer_from_state_actions
end

._css_lexer_index_offsets ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 58

def _css_lexer_index_offsets
  @_css_lexer_index_offsets
end

._css_lexer_indicies ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 69

def _css_lexer_indicies
  @_css_lexer_indicies
end

._css_lexer_key_spans ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 47

def _css_lexer_key_spans
  @_css_lexer_key_spans
end

._css_lexer_to_state_actions ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 212

def _css_lexer_to_state_actions
  @_css_lexer_to_state_actions
end

._css_lexer_trans_actions ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 197

def _css_lexer_trans_actions
  @_css_lexer_trans_actions
end

._css_lexer_trans_keys ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 27

def _css_lexer_trans_keys
  @_css_lexer_trans_keys
end

._css_lexer_trans_targs ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 182

def _css_lexer_trans_targs
  @_css_lexer_trans_targs
end

.css_lexer_en_main ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 266

def css_lexer_en_main
  @css_lexer_en_main
end

.css_lexer_en_predicate ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 262

def css_lexer_en_predicate
  @css_lexer_en_predicate
end

.css_lexer_en_pseudo_args ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 258

def css_lexer_en_pseudo_args
  @css_lexer_en_pseudo_args
end

.css_lexer_error ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 253

def css_lexer_error
  @css_lexer_error
end

.css_lexer_first_final ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 249

def css_lexer_first_final
  @css_lexer_first_final
end

.css_lexer_start ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

# File 'lib/oga/css/lexer.rb', line 245

def css_lexer_start
  @css_lexer_start
end

Instance Method Details

#advance(&block) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Advances through the input and generates the corresponding tokens. Each token is yielded to the supplied block.

This method stores the supplied block in @block and resets it after the lexer loop has finished.

See Also:

• Oga::CSS::Lexer.[[#add_token]

# File 'lib/oga/css/lexer.rb', line 307

def advance(&block)
  @block = block

  data  = @data # saves ivar lookups while lexing.
  ts    = nil
  te    = nil
  stack = []
  top   = 0
  cs    = self.class.css_lexer_start
  act   = 0
  eof   = @data.bytesize
  p     = 0
  pe    = eof

  _css_lexer_eof_trans          = self.class.send(:_css_lexer_eof_trans)
  _css_lexer_from_state_actions = self.class.send(:_css_lexer_from_state_actions)
  _css_lexer_index_offsets      = self.class.send(:_css_lexer_index_offsets)
  _css_lexer_indicies           = self.class.send(:_css_lexer_indicies)
  _css_lexer_key_spans          = self.class.send(:_css_lexer_key_spans)
  _css_lexer_to_state_actions   = self.class.send(:_css_lexer_to_state_actions)
  _css_lexer_trans_actions      = self.class.send(:_css_lexer_trans_actions)
  _css_lexer_trans_keys         = self.class.send(:_css_lexer_trans_keys)
  _css_lexer_trans_targs        = self.class.send(:_css_lexer_trans_targs)

  
# line 333 "lib/oga/css/lexer.rb"
begin
	testEof = false
	_slen, _trans, _keys, _inds, _acts, _nacts = nil
	_goto_level = 0
	_resume = 10
	_eof_trans = 15
	_again = 20
	_test_eof = 30
	_out = 40
	while true
	if _goto_level <= 0
	if p == pe
		_goto_level = _test_eof
		next
	end
	if cs == 0
		_goto_level = _out
		next
	end
	end
	if _goto_level <= _resume
	case _css_lexer_from_state_actions[cs] 
	when 10 then
# line 1 "NONE"
		begin
ts = p
		end
# line 361 "lib/oga/css/lexer.rb"
	end
	_keys = cs << 1
	_inds = _css_lexer_index_offsets[cs]
	_slen = _css_lexer_key_spans[cs]
	_trans = if (   _slen > 0 && 
			_css_lexer_trans_keys[_keys] <= ( (data.getbyte(p) || 0)) && 
			( (data.getbyte(p) || 0)) <= _css_lexer_trans_keys[_keys + 1] 
) then
			_css_lexer_indicies[ _inds + ( (data.getbyte(p) || 0)) - _css_lexer_trans_keys[_keys] ] 
		 else 
			_css_lexer_indicies[ _inds + _slen ]
		 end
	end
	if _goto_level <= _eof_trans
	cs = _css_lexer_trans_targs[_trans]
	if _css_lexer_trans_actions[_trans] != 0
	case _css_lexer_trans_actions[_trans]
	when 29 then
# line 251 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin  add_token(:T_NTH)  end
		end
	when 3 then
# line 253 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin  add_token(:T_ODD)  end
		end
	when 2 then
# line 254 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin  add_token(:T_EVEN)  end
		end
	when 26 then
# line 235 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin 
    add_token(:T_RPAREN)

    cs = 13;
   end
		end
	when 27 then
# line 162 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin 
    emit(:T_IDENT, ts, te)
   end
		end
	when 30 then
# line 246 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1;		end
	when 32 then
# line 252 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1; begin  add_token(:T_MINUS)  end
		end
	when 31 then
# line 195 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1; begin 
    value = slice_input(ts, te).to_i

    add_token(:T_INT, value)
   end
		end
	when 33 then
# line 162 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1; begin 
    emit(:T_IDENT, ts, te)
   end
		end
	when 1 then
# line 1 "NONE"
		begin
	case act
	when 0 then
	begin	begin
		cs = 0
		_goto_level = _again
		next
	end
end
	when 3 then
	begin begin p = ((te))-1; end
 add_token(:T_MINUS) end
end 
			end
	when 34 then
# line 292 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin  add_token(:T_EQ)  end
		end
	when 8 then
# line 293 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin  add_token(:T_SPACE_IN)  end
		end
	when 6 then
# line 294 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin  add_token(:T_STARTS_WITH)  end
		end
	when 5 then
# line 295 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin  add_token(:T_ENDS_WITH)  end
		end
	when 38 then
# line 296 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin  add_token(:T_IN)  end
		end
	when 7 then
# line 297 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin  add_token(:T_HYPHEN_IN)  end
		end
	when 35 then
# line 274 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin 
    add_token(:T_RBRACK)

    cs = 13;
   end
		end
	when 4 then
# line 214 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin 
    emit(:T_STRING, ts + 1, te - 1)
   end
		end
	when 36 then
# line 282 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1;		end
	when 37 then
# line 162 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1; begin 
    emit(:T_IDENT, ts, te)
   end
		end
	when 14 then
# line 268 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin 
    add_token(:T_LBRACK)

    cs = 28;
   end
		end
	when 15 then
# line 147 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin 
    add_token(:T_PIPE)
   end
		end
	when 12 then
# line 229 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin 
    add_token(:T_LPAREN)

    cs = 23;
   end
		end
	when 13 then
# line 162 "lib/oga/css/lexer.rl"
		begin
te = p+1
 begin 
    emit(:T_IDENT, ts, te)
   end
		end
	when 11 then
# line 317 "lib/oga/css/lexer.rl"
		begin
te = p+1
		end
	when 22 then
# line 306 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1; begin  add_token(:T_GREATER)  end
		end
	when 19 then
# line 307 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1; begin  add_token(:T_PLUS)  end
		end
	when 24 then
# line 308 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1; begin  add_token(:T_TILDE)  end
		end
	when 17 then
# line 151 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1; begin 
    add_token(:T_COMMA)
   end
		end
	when 16 then
# line 137 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1; begin 
    add_token(:T_SPACE)
   end
		end
	when 23 then
# line 162 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1; begin 
    emit(:T_IDENT, ts, te)
   end
		end
	when 18 then
# line 141 "lib/oga/css/lexer.rl"
		begin
 add_token(:T_HASH) 		end
# line 304 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1;		end
	when 20 then
# line 142 "lib/oga/css/lexer.rl"
		begin
 add_token(:T_DOT) 		end
# line 304 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1;		end
	when 21 then
# line 143 "lib/oga/css/lexer.rl"
		begin
 add_token(:T_COLON) 		end
# line 304 "lib/oga/css/lexer.rl"
		begin
te = p
p = p - 1;		end
	when 28 then
# line 1 "NONE"
		begin
te = p+1
		end
# line 252 "lib/oga/css/lexer.rl"
		begin
act = 3;		end
# line 642 "lib/oga/css/lexer.rb"
	end
	end
	end
	if _goto_level <= _again
	case _css_lexer_to_state_actions[cs] 
	when 9 then
# line 1 "NONE"
		begin
ts = nil;		end
	when 25 then
# line 1 "NONE"
		begin
ts = nil;		end
# line 1 "NONE"
		begin
act = 0
		end
# line 660 "lib/oga/css/lexer.rb"
	end

	if cs == 0
		_goto_level = _out
		next
	end
	p += 1
	if p != pe
		_goto_level = _resume
		next
	end
	end
	if _goto_level <= _test_eof
	if p == eof
	if _css_lexer_eof_trans[cs] > 0
		_trans = _css_lexer_eof_trans[cs] - 1;
		_goto_level = _eof_trans
		next;
	end
	end

	end
	if _goto_level <= _out
		break
	end
end
	end

# line 83 "lib/oga/css/lexer.rl"

  # % fix highlight
ensure
  @block = nil
end

#lex ⇒ Array

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Gathers all the tokens for the input and returns them as an Array.

Returns:

• (Array)

See Also:

• Oga::CSS::Lexer.[[#advance]

# File 'lib/oga/css/lexer.rb', line 288

def lex
  tokens = []

  advance do |type, value|
    tokens << [type, value]
  end

  return tokens
end