Unicode String Scanner

A Unicode-aware implementation of Ruby's StringScanner.

Author Tim Morgan
Version 1.0 (Jul 11, 2012)
License Released under the MIT license.


Did you know that StringScanner splits codepoints? Neither did I. This one doesn't.

When would I want to use this? When you want to use StringScanner on a Unicode (UTF-n) string.

When would I not want to use this? If you're interested in speed. This is slower than StringScanner because a) it's not written in native C, and b) it's slower to traverse Unicode strings anyway because characters can have varying byte sizes.


Simply add this gem to your project's Gemfile:

gem 'unicode_scanner'


The UnicodeScanner object responds to exactly the same API as StringScanner, with the exception of the following methods:

  • getbyte
  • any obsolete methods

For more information, see the UnicodeScanner class documentation.