Character Encoding of HTML Pages

The character set or encoding of HTML pages only is identified by Tika’s AutoDetectReader. The table shows the percentage how character sets have been used to encode HTML pages crawled by the latest monthly crawls. The underlying data including page counts is provided in charsets.csv.

crawl	CC-MAIN-2026-21	CC-MAIN-2026-25	CC-MAIN-2026-30
charset
<other>	0.0001	0.0000	0.0001
<unknown>	2.0799	1.9748	2.0062
Big5	0.0221	0.0209	0.0202
Big5-HKSCS	0.0000	0.0001	0.0000
EUC-JP	0.1283	0.1248	0.1231
EUC-KR	0.0757	0.0746	0.0731
GB18030	0.0120	0.0104	0.0089
GB2312	0.1915	0.1750	0.1643
GBK	0.0905	0.0867	0.0853
IBM420	0.0052	0.0053	0.0046
IBM424	0.0009	0.0009	0.0012
IBM500	0.0008	0.0007	0.0005
IBM855	0.0000	0.0000	0.0000
IBM866	0.0001	0.0001	0.0001
ISO-2022-JP	0.0011	0.0012	0.0012
ISO-8859-1	2.8610	2.6514	2.6022
ISO-8859-13	0.0000	0.0000	0.0000
ISO-8859-15	0.0446	0.0377	0.0402
ISO-8859-16	0.0003	0.0004	0.0003
ISO-8859-2	0.0794	0.0766	0.0747
ISO-8859-3	0.0003	0.0003	0.0003
ISO-8859-4	0.0007	0.0007	0.0006
ISO-8859-5	0.0007	0.0006	0.0006
ISO-8859-6	0.0000	0.0000	0.0000
ISO-8859-7	0.0038	0.0041	0.0039
ISO-8859-8	0.0006	0.0008	0.0007
ISO-8859-9	0.0220	0.0211	0.0216
KOI8-R	0.0076	0.0075	0.0075
KOI8-U	0.0001	0.0001	0.0001
Shift_JIS	0.1567	0.1502	0.1478
TIS-620	0.0031	0.0031	0.0031
US-ASCII	0.0241	0.0257	0.0264
UTF-16	0.0020	0.0017	0.0016
UTF-16BE	0.0003	0.0003	0.0003
UTF-16LE	0.0036	0.0035	0.0054
UTF-32	0.0001	0.0001	0.0001
UTF-32LE	0.0001	0.0002	0.0002
UTF-8	93.3986	93.7782	93.8283
windows-1250	0.0698	0.0677	0.0663
windows-1251	0.4885	0.4864	0.4793
windows-1252	0.1684	0.1554	0.1502
windows-1253	0.0025	0.0021	0.0019
windows-1254	0.0117	0.0111	0.0111
windows-1255	0.0059	0.0058	0.0053
windows-1256	0.0241	0.0223	0.0215
windows-1257	0.0051	0.0034	0.0035
windows-31j	0.0002	0.0002	0.0003
x-iso-8859-11	0.0000	0.0000	0.0000
x-windows-874	0.0059	0.0057	0.0061
x-windows-949	0.0000	0.0000	0.0000