Module: RubyTUI::Unicode

Defined in:
lib/rubytui/unicode.rb

Overview

Unicode text width utilities for terminal rendering.

Handles display width calculation for wide characters (CJK, emoji), combining marks (Devanagari, diacritics), variation selectors (FE0F), and grapheme cluster segmentation.

Examples:

Measure display width

Unicode.string_width("Hello")   # => 5
Unicode.string_width("Hi🔥")   # => 4

Truncate to fit

Unicode.truncate_to_width("Hello World", 5)  # => "Hello"

Constant Summary collapse

EMOJI_PRESENTATION =

Unicode Emoji_Presentation codepoints below U+3000. These render as 2-wide by default (without FE0F) in modern terminals. Source: Unicode 16.0 emoji-data.txt

Set[
  0x231A, 0x231B,                                           # watch, hourglass
  0x23E9, 0x23EA, 0x23EB, 0x23EC, 0x23F0, 0x23F3,         # fast-forward etc.
  0x25FD, 0x25FE,                                           # medium small squares
  0x2614, 0x2615,                                           # umbrella, hot beverage
  0x2648, 0x2649, 0x264A, 0x264B, 0x264C, 0x264D,          # zodiac signs
  0x264E, 0x264F, 0x2650, 0x2651, 0x2652, 0x2653,
  0x267F,                                                    # wheelchair
  0x2693,                                                    # anchor
  0x26A1,                                                    # high voltage
  0x26AA, 0x26AB,                                           # circles
  0x26BD, 0x26BE,                                           # soccer, baseball
  0x26C4, 0x26C5,                                           # snowman, sun behind cloud
  0x26CE,                                                    # ophiuchus
  0x26D4,                                                    # no entry
  0x26EA,                                                    # church
  0x26F2, 0x26F3,                                           # fountain, golf
  0x26F5,                                                    # sailboat
  0x26FA,                                                    # tent
  0x26FD,                                                    # fuel pump
  0x2705,                                                    # check mark
  0x270A, 0x270B,                                           # fists
  0x2728,                                                    # sparkles
  0x274C, 0x274E,                                           # cross marks
  0x2753, 0x2754, 0x2755,                                   # question/exclamation
  0x2757,                                                    # exclamation
  0x2795, 0x2796, 0x2797,                                   # plus/minus/division
  0x27B0, 0x27BF,                                           # curly loops
  0x2B1B, 0x2B1C,                                           # large squares
  0x2B50,                                                    # star
  0x2B55,                                                    # circle
].freeze

Class Method Summary collapse

Class Method Details

.char_width(ch) ⇒ Integer

Estimate the display width of a single character.

Parameters:

  • ch (String) —

    a single character

Returns:

  • (Integer) —

    0 for control characters and combining marks, 2 for wide characters, 1 otherwise



118
119
120
121
122
123
124
125
 
# File 'lib/rubytui/unicode.rb', line 118

def char_width(ch)
  cp = ch.ord
  return 1 if cp >= 0x20 && cp < 0x7F
  return 0 if cp < 0x20 || cp == 0x7F
  return 0 if combining_mark?(cp)
  return 2 if wide_char?(cp)
  1
end

.cluster_width(cluster) ⇒ Integer

Calculate display width of a single grapheme cluster.

Parameters:

  • cluster (String) —

    a grapheme cluster (one or more codepoints)

Returns:

  • (Integer) —

    0, 1, or 2



73
74
75
76
77
78
79
80
81
82
83
84
85
86
 
# File 'lib/rubytui/unicode.rb', line 73

def cluster_width(cluster)
  return char_width(cluster) if cluster.bytesize == 1

  codepoints = cluster.codepoints
  has_fe0f = codepoints.include?(0xFE0F)

  codepoints.each do |cp|
    next if combining_mark?(cp)
    return 2 if wide_char?(cp)
    return 2 if has_fe0f
  end

  1
end

.combining_mark?(cp) ⇒ Boolean

Returns true if codepoint is a combining mark (zero-width).

Parameters:

  • cp (Integer) —

    Unicode codepoint

Returns:

  • (Boolean) —

    true if codepoint is a combining mark (zero-width)



129
130
131
132
133
134
135
136
137
138
139
140
 
# File 'lib/rubytui/unicode.rb', line 129

def combining_mark?(cp)
  (cp >= 0x0300 && cp <= 0x036F) ||   # Combining Diacritical Marks
  (cp >= 0x0900 && cp <= 0x097F &&     # Devanagari dependent vowels, virama, anusvara
    ((cp >= 0x0901 && cp <= 0x0903) || (cp >= 0x093A && cp <= 0x094F) ||
     (cp >= 0x0951 && cp <= 0x0957) || cp == 0x0962 || cp == 0x0963)) ||
  (cp >= 0x1AB0 && cp <= 0x1AFF) ||   # Combining Diacritical Marks Extended
  (cp >= 0x1DC0 && cp <= 0x1DFF) ||   # Combining Diacritical Marks Supplement
  (cp >= 0x20D0 && cp <= 0x20FF) ||   # Combining Marks for Symbols
  (cp >= 0xFE00 && cp <= 0xFE0F) ||   # Variation Selectors
  (cp >= 0xFE20 && cp <= 0xFE2F) ||   # Combining Half Marks
  (cp >= 0xE0100 && cp <= 0xE01EF)    # Variation Selectors Supplement
end

.string_width(string) ⇒ Integer

Calculate display width of a string in terminal columns.

Operates on grapheme clusters so that combining sequences (Devanagari matras/virama, Latin diacritics, emoji ZWJ sequences) are measured as a single unit. Wide characters (CJK, emoji) count as 2 columns.

Parameters:

  • string (String) —

    the string to measure

Returns:

  • (Integer) —

    display width in terminal columns



65
66
67
 
# File 'lib/rubytui/unicode.rb', line 65

def string_width(string)
  string.grapheme_clusters.sum { |gc| cluster_width(gc) }
end

.truncate_to_width(string, max_width) ⇒ String

Truncate a string to fit within a maximum display width.

Unlike simple character slicing, this respects grapheme clusters and wide characters that take 2 columns.

Parameters:

  • string (String) —

    the string to truncate

  • max_width (Integer) —

    maximum display width in columns

Returns:

  • (String) —

    truncated string



96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
 
# File 'lib/rubytui/unicode.rb', line 96

def truncate_to_width(string, max_width)
  return "" if max_width <= 0

  result = String.new
  width = 0

  string.grapheme_clusters.each do |gc|
    w = cluster_width(gc)
    break if width + w > max_width

    result << gc
    width += w
  end

  result
end

.wide_char?(cp) ⇒ Boolean

Returns true if codepoint is a wide character (2-column).

Parameters:

  • cp (Integer) —

    Unicode codepoint

Returns:

  • (Boolean) —

    true if codepoint is a wide character (2-column)



144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
 
# File 'lib/rubytui/unicode.rb', line 144

def wide_char?(cp)
  (cp >= 0x1100 && cp <= 0x115F) ||   # Hangul Jamo
  EMOJI_PRESENTATION.include?(cp) ||   # Emoji with default 2-wide presentation
  (cp >= 0x2E80 && cp <= 0x303E) ||   # CJK Radicals, Kangxi, CJK Symbols
  (cp >= 0x3040 && cp <= 0x33BF) ||   # Hiragana, Katakana, CJK Compat
  (cp >= 0x3400 && cp <= 0x4DBF) ||   # CJK Unified Extension A
  (cp >= 0x4E00 && cp <= 0x9FFF) ||   # CJK Unified Ideographs
  (cp >= 0xA960 && cp <= 0xA97F) ||   # Hangul Jamo Extended-A
  (cp >= 0xAC00 && cp <= 0xD7AF) ||   # Hangul Syllables
  (cp >= 0xF900 && cp <= 0xFAFF) ||   # CJK Compatibility Ideographs
  (cp >= 0xFE10 && cp <= 0xFE19) ||   # Vertical Forms
  (cp >= 0xFE30 && cp <= 0xFE6F) ||   # CJK Compatibility Forms
  (cp >= 0xFF01 && cp <= 0xFF60) ||   # Fullwidth Forms
  (cp >= 0xFFE0 && cp <= 0xFFE6) ||   # Fullwidth Sign
  (cp >= 0x1F000 && cp <= 0x1FAFF) || # Emoji & Symbols
  (cp >= 0x20000 && cp <= 0x2FA1F)    # CJK Unified Extensions B-F
end