ASCII vs Unicode vs UTF-8

What is the difference between ASCII, Unicode and UTF-8?

ASCII is a 128-character table; Unicode is a 1,114,112-code-point table that contains ASCII; UTF-8 is the byte encoding of Unicode, identical to ASCII for codes 0-127 and 2-4 bytes for everything else.

The three sit in a line rather than side by side. Unicode says which character gets which number, UTF-8 says how to write that number as bytes, and ASCII is the small set both of them agree with.

ASCII defines 128 characters, codes 0–127, using 7 bits; in practice each code is stored in one 8-bit byte with a leading 0.

This page compares the three; the ASCII table this converter reads from carries all 128 rows.

What is a Unicode code point?

A code point is the number Unicode assigns to a character, written with a U+ prefix and hex digits: U+0041 is A and U+20AC is the euro sign. It says nothing about how the character is stored.

Unicode holds 1,114,112 code points, U+0000 to U+10FFFF, arranged in 17 planes. The first plane, U+0000 to U+FFFF, holds almost every character in daily use, and its first 128 entries are ASCII in the same order.

A code point is a number like any other, so it can be written in any base — how number base conversion works covers that part.

How does UTF-8 encode a character in 1 to 4 bytes?

UTF-8 picks a byte count from the size of the code point, then spreads the code point across a lead byte and its continuation bytes.

1. Find the code point. é is U+00E9, which is 233 as a number.
2. Pick the byte count from the range the code point falls in: one byte up to U+007F, two up to U+07FF, three up to U+FFFF, four above that.
3. Write the lead byte pattern for that count and one 10xxxxxx continuation byte for every byte after the first.
4. Fill the x positions with the bits of the code point, highest first. é becomes 11000011 10101001, which is C3 A9.

The lead byte announces the length, so a decoder never has to guess and a byte lost in transit cannot corrupt the rest of the line. RFC 3629 defines the rule; The Unicode Standard assigns the code points.

UTF-8 encodes each character in 1 to 4 bytes. Characters 0–127 use one byte identical to ASCII, so ASCII text is valid UTF-8.

é is U+00E9 and encodes as C3 A9 (2 bytes); € is U+20AC and encodes as E2 82 AC (3 bytes); 😀 is U+1F600 and encodes as F0 9F 98 80 (4 bytes).

Why is the byte count different from the character count?

Outside ASCII one character is not one byte, so a string of five characters can be five, ten or twenty bytes depending on which characters it holds.

é takes 2 bytes, the euro sign takes 3 and the grinning face takes 4, but each of them is one character. A length measured in bytes and a length measured in characters answer different questions, and only the first tells you how much storage the text needs.

convert text to hexadecimal shows the byte count for any input you paste, and turn text into binary code shows the bits behind it.

What is extended ASCII?

Extended ASCII is the loose name for the 8-bit sets that fill codes 128 to 255 — Latin-1 (ISO 8859-1), Windows-1252 and CP437 among them. Each assigns different characters to the same 128 spare values.

Codes 128–255 are not part of ASCII. They belong to extended sets such as Latin-1 (ISO 8859-1), where 255 is ÿ; in pure ASCII a value above 127 is out of range.

That is why an old file can open as the wrong characters: the bytes are intact, but the set they were written in was never recorded. In Latin-1 code 233 is also é, which is why Latin-1 and UTF-8 look interchangeable until a byte above 127 arrives with no label attached.