wmii

git clone git://oldgit.suckless.org/wmii/
Log | Files | Refs | README | LICENSE

utf.7 (2338B)

      1 .deEX
      2 .ift .ft5
      3 .nf
      4 ..
      5 .deEE
      6 .ft1
      7 .fi
      8 ..
      9 .TH UTF 7
     10 .SH NAME
     11 UTF, Unicode, ASCII, rune \- character set and format
     12 .SH DESCRIPTION
     13 The Plan 9 character set and representation are
     14 based on the Unicode Standard and on the ISO multibyte
     15 .SM UTF-8
     16 encoding (Universal Character
     17 Set Transformation Format, 8 bits wide).
     18 The Unicode Standard represents its characters in 16
     19 bits;
     20 .SM UTF-8
     21 represents such
     22 values in an 8-bit byte stream.
     23 Throughout this manual,
     24 .SM UTF-8
     25 is shortened to
     26 .SM UTF.
     27 .PP
     28 In Plan 9, a
     29 .I rune
     30 is a 16-bit quantity representing a Unicode character.
     31 Internally, programs may store characters as runes.
     32 However, any external manifestation of textual information,
     33 in files or at the interface between programs, uses a
     34 machine-independent, byte-stream encoding called
     35 .SM UTF.
     36 .PP
     37 .SM UTF
     38 is designed so the 7-bit
     39 .SM ASCII
     40 set (values hexadecimal 00 to 7F),
     41 appear only as themselves
     42 in the encoding.
     43 Runes with values above 7F appear as sequences of two or more
     44 bytes with values only from 80 to FF.
     45 .PP
     46 The
     47 .SM UTF
     48 encoding of the Unicode Standard is backward compatible with
     49 .SM ASCII\c
     50 :
     51 programs presented only with
     52 .SM ASCII
     53 work on Plan 9
     54 even if not written to deal with
     55 .SM UTF,
     56 as do
     57 programs that deal with uninterpreted byte streams.
     58 However, programs that perform semantic processing on
     59 .SM ASCII
     60 graphic
     61 characters must convert from
     62 .SM UTF
     63 to runes
     64 in order to work properly with non-\c
     65 .SM ASCII
     66 input.
     67 See
     68 .IR rune (3).
     69 .PP
     70 Letting numbers be binary,
     71 a rune x is converted to a multibyte
     72 .SM UTF
     73 sequence
     74 as follows:
     75 .PP
     76 01.   x in [00000000.0bbbbbbb] → 0bbbbbbb
     77 .br
     78 10.   x in [00000bbb.bbbbbbbb] → 110bbbbb, 10bbbbbb
     79 .br
     80 11.   x in [bbbbbbbb.bbbbbbbb] → 1110bbbb, 10bbbbbb, 10bbbbbb
     81 .br
     82 .PP
     83 Conversion 01 provides a one-byte sequence that spans the
     84 .SM ASCII
     85 character set in a compatible way.
     86 Conversions 10 and 11 represent higher-valued characters
     87 as sequences of two or three bytes with the high bit set.
     88 Plan 9 does not support the 4, 5, and 6 byte sequences proposed by X-Open.
     89 When there are multiple ways to encode a value, for example rune 0,
     90 the shortest encoding is used.
     91 .PP
     92 In the inverse mapping,
     93 any sequence except those described above
     94 is incorrect and is converted to rune hexadecimal 0080.
     95 .SH "SEE ALSO"
     96 .IR ascii (1),
     97 .IR tcs (1),
     98 .IR rune (3),
     99 .IR "The Unicode Standard" .