Character Encoding of HTML Pages

The character set or encoding of HTML pages only is identified by Tika’s AutoDetectReader. The table shows the percentage how character sets have been used to encode HTML pages crawled by the latest monthly crawls. The underlying data including page counts is provided in charsets.csv.

crawl	CC-MAIN-2026-25	CC-MAIN-2026-30	CC-MAIN-2026-34
charset
<other>	0.0000	0.0001	0.0001
<unknown>	1.9748	2.0062	2.1044
Big5	0.0209	0.0202	0.0257
Big5-HKSCS	0.0001	0.0000	0.0000
EUC-JP	0.1248	0.1231	0.1220
EUC-KR	0.0746	0.0731	0.0754
GB18030	0.0104	0.0089	0.0093
GB2312	0.1750	0.1643	0.1684
GBK	0.0867	0.0853	0.0870
IBM420	0.0053	0.0046	0.0050
IBM424	0.0009	0.0012	0.0013
IBM500	0.0007	0.0005	0.0006
IBM855	0.0000	0.0000	0.0000
IBM866	0.0001	0.0001	0.0001
ISO-2022-JP	0.0012	0.0012	0.0011
ISO-8859-1	2.6514	2.6022	2.4618
ISO-8859-13	0.0000	0.0000	0.0001
ISO-8859-15	0.0377	0.0402	0.0411
ISO-8859-16	0.0004	0.0003	0.0003
ISO-8859-2	0.0766	0.0747	0.0769
ISO-8859-3	0.0003	0.0003	0.0003
ISO-8859-4	0.0007	0.0006	0.0007
ISO-8859-5	0.0006	0.0006	0.0005
ISO-8859-6	0.0000	0.0000	0.0000
ISO-8859-7	0.0041	0.0039	0.0040
ISO-8859-8	0.0008	0.0007	0.0007
ISO-8859-9	0.0211	0.0216	0.0233
KOI8-R	0.0075	0.0075	0.0081
KOI8-U	0.0001	0.0001	0.0001
Shift_JIS	0.1502	0.1478	0.1440
TIS-620	0.0031	0.0031	0.0029
US-ASCII	0.0257	0.0264	0.0241
UTF-16	0.0017	0.0016	0.0016
UTF-16BE	0.0003	0.0003	0.0002
UTF-16LE	0.0035	0.0054	0.0040
UTF-32	0.0001	0.0001	0.0001
UTF-32LE	0.0002	0.0002	0.0002
UTF-8	93.7782	93.8283	93.8314
windows-1250	0.0677	0.0663	0.0651
windows-1251	0.4864	0.4793	0.4998
windows-1252	0.1554	0.1502	0.1605
windows-1253	0.0021	0.0019	0.0020
windows-1254	0.0111	0.0111	0.0107
windows-1255	0.0058	0.0053	0.0051
windows-1256	0.0223	0.0215	0.0203
windows-1257	0.0034	0.0035	0.0034
windows-31j	0.0002	0.0003	0.0002
x-iso-8859-11	0.0000	0.0000	0.0000
x-windows-874	0.0057	0.0061	0.0057
x-windows-949	0.0000	0.0000	0.0000