Method: CharacterCleaner#process_entities_and_utf

Defined in:
lib/character_cleaner.rb

#process_entities_and_utf(text) ⇒ Object

Curses doesn’t handle UTF8 well, so we provide utility functions that handle unprintable (by Curses) UTF8 characters. We also convert HTML entity codes into UTF codes first



8
9
10
11
12
13
14
15
16
 
# File 'lib/character_cleaner.rb', line 8

def process_entities_and_utf(text)
  coder = HTMLEntities.new
  text = coder.decode(text)

  input_encoding = "UTF-8"
  output_encoding = 'ASCII//IGNORE//TRANSLIT'

  text = Iconv.new(output_encoding, input_encoding).iconv(text)
end