Statistics of Common Crawl Monthly Archives

Number of pages, distribution of top-level domains, crawl overlaps, etc. - basic metrics about Common Crawl Monthly Crawl Archives
Latest crawl: CC-MAIN-2026-04

View the Project on GitHub

MIME Types

The crawled content is dominated by HTML pages and contains only a small percentage of other document formats. The tables show the percentage of the top 100 media or MIME types of the latest monthly crawls.

While the first table is based the Content-Type HTTP header, the second uses the MIME type detected by Apache Tika based on the actual content. The underlying data including page counts is provided in mimetypes.csv resp. mimetypes_detected.csv.

crawl CC-MAIN-2025-47 CC-MAIN-2025-51 CC-MAIN-2026-04
mimetype
<other> 0.0779 0.0720 0.0669
application/atom+xml 0.1434 0.1495 0.1338
application/calendar 0.0004 0.0004 0.0003
application/download 0.0043 0.0032 0.0012
application/epub+zip 0.0018 0.0016 0.0015
application/force-download 0.0114 0.0113 0.0066
application/gpx+xml 0.0007 0.0007 0.0007
application/ics 0.0005 0.0004 0.0004
application/javascript 0.0007 0.0008 0.0007
application/json 0.0279 0.0279 0.0242
application/ld+json 0.0018 0.0017 0.0016
application/marc 0.0004 0.0004 0.0003
application/msword 0.0022 0.0024 0.0022
application/octet-stream 0.0410 0.0506 0.0423
application/octetstream 0.0002 0.0002 0.0002
application/pdf 0.6816 0.6939 0.6211
application/pgp-encrypted 0.0001 0.0001 0.0001
application/pgp-signature 0.0014 0.0018 0.0014
application/postscript 0.0002 0.0002 0.0001
application/rdf+xml 0.0044 0.0042 0.0036
application/rss+xml 0.0439 0.0439 0.0417
application/rtf 0.0010 0.0010 0.0007
application/save-to-disk 0.0000 0.0000 0.0000
application/text 0.0002 0.0002 0.0001
application/unknown 0.0002 0.0002 0.0002
application/vnd.android.package-archive 0.0000 0.0000 0.0000
application/vnd.google-earth.kml+xml 0.0011 0.0011 0.0011
application/vnd.google-earth.kmz 0.0003 0.0002 0.0003
application/vnd.ms-excel 0.0014 0.0016 0.0014
application/vnd.ms-powerpoint 0.0002 0.0002 0.0001
application/vnd.ms-word 0.0002 0.0002 0.0002
application/vnd.oasis.opendocument.text 0.0007 0.0006 0.0006
application/vnd.openxmlformats-officedocument.presentationml.presentation 0.0002 0.0002 0.0002
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet 0.0016 0.0017 0.0017
application/vnd.openxmlformats-officedocument.wordprocessingml.document 0.0023 0.0023 0.0022
application/vnd.wap.xhtml+xml 0.0006 0.0002 0.0003
application/x-bibtex 0.0077 0.0083 0.0081
application/x-bittorrent 0.0001 0.0002 0.0006
application/x-bzip2 0.0001 0.0000 0.0000
application/x-debian-package 0.0000 0.0000 0.0000
application/x-download 0.0021 0.0025 0.0025
application/x-endnote-refer 0.0008 0.0005 0.0004
application/x-gzip 0.0001 0.0001 0.0001
application/x-httpd-php 0.0005 0.0003 0.0002
application/x-java-jnlp-file 0.0001 0.0002 0.0001
application/x-javascript 0.0001 0.0001 0.0001
application/x-json 0.0000 0.0000 0.0000
application/x-mobipocket-ebook 0.0006 0.0006 0.0004
application/x-msdownload 0.0003 0.0003 0.0002
application/x-netcdf 0.0003 0.0002 0.0003
application/x-research-info-systems 0.0107 0.0114 0.0108
application/x-shockwave-flash 0.0001 0.0000 0.0001
application/x-tar 0.0000 0.0000 0.0001
application/x-tex 0.0001 0.0002 0.0001
application/x-troff-man 0.0002 0.0004 0.0004
application/x-zip-compressed 0.0000 0.0000 0.0000
application/xhtml+xml 0.0119 0.0119 0.0101
application/xml 0.0245 0.0250 0.0234
application/zip 0.0002 0.0002 0.0001
audio/mpeg 0.0001 0.0001 0.0001
audio/x-mpegurl 0.0005 0.0004 0.0005
audio/x-scpls 0.0001 0.0001 0.0001
audio/x-wav 0.0000 0.0000 0.0000
binary/octet-stream 0.0003 0.0003 0.0005
image/gif 0.0004 0.0004 0.0005
image/jp2 NaN NaN 0.0000
image/jpeg 0.0003 0.0003 0.0003
image/jpg 0.0000 0.0000 0.0000
image/pjpeg 0.0000 0.0000 0.0000
image/png 0.0003 0.0002 0.0002
image/svg+xml 0.0000 0.0000 0.0000
image/tiff 0.0000 0.0000 0.0000
image/vnd.djvu 0.0003 0.0006 0.0002
image/webp 0.0000 0.0000 0.0000
message/rfc822 0.0004 0.0003 0.0002
text/calendar 0.0400 0.0386 0.0344
text/css 0.0007 0.0005 0.0006
text/csv 0.0032 0.0032 0.0032
text/directory 0.0002 0.0002 0.0002
text/enriched 0.0001 0.0001 0.0000
text/html 98.7109 98.6844 98.8254
text/javascript 0.0003 0.0004 0.0003
text/pdf 0.0000 0.0000 0.0000
text/plain 0.0496 0.0510 0.0457
text/prs.lines.tag 0.0025 0.0024 0.0022
text/tab-separated-values 0.0005 0.0005 0.0004
text/turtle 0.0017 0.0017 0.0013
text/vcard 0.0012 0.0012 0.0010
text/x-bibtex 0.0004 0.0004 0.0004
text/x-c 0.0001 0.0001 0.0001
text/x-csrc 0.0001 0.0002 0.0001
text/x-diff 0.0002 0.0002 0.0003
text/x-patch 0.0002 0.0002 0.0002
text/x-perl 0.0000 0.0000 0.0000
text/x-vcalendar 0.0004 0.0004 0.0003
text/x-vcard 0.0018 0.0020 0.0017
text/xml 0.0663 0.0704 0.0616
unknown/unknown 0.0000 0.0000 0.0000
video/mp4 0.0000 0.0000 0.0000
video/webm 0.0000 0.0000 0.0000
video/x-ms-asf 0.0001 0.0001 0.0001

crawl CC-MAIN-2025-47 CC-MAIN-2025-51 CC-MAIN-2026-04
mimetype_detected
<other> 0.0104 0.0119 0.0101
application/atom+xml 0.1450 0.1510 0.1353
application/epub+zip 0.0020 0.0025 0.0017
application/gpx+xml 0.0007 0.0007 0.0007
application/gzip 0.0000 0.0000 0.0000
application/javascript 0.0008 0.0009 0.0009
application/json 0.0282 0.0278 0.0243
application/marc 0.0009 0.0009 0.0007
application/mbox 0.0012 0.0013 0.0010
application/msword 0.0017 0.0018 0.0016
application/octet-stream 0.0112 0.0191 0.0152
application/pdf 0.6951 0.7084 0.6335
application/pgp-signature 0.0014 0.0015 0.0014
application/pkcs7-signature 0.0004 0.0004 0.0003
application/postscript 0.0003 0.0003 0.0002
application/rdf+xml 0.0085 0.0084 0.0076
application/rss+xml 0.0698 0.0709 0.0661
application/rtf 0.0015 0.0014 0.0011
application/text 0.0000 0.0000 0.0000
application/vnd.android.package-archive 0.0000 0.0000 0.0000
application/vnd.google-earth.kml+xml 0.0023 0.0023 0.0024
application/vnd.google-earth.kmz 0.0003 0.0002 0.0003
application/vnd.ms-excel 0.0007 0.0007 0.0006
application/vnd.ms-powerpoint 0.0002 0.0002 0.0001
application/vnd.oasis.opendocument.spreadsheet 0.0003 0.0003 0.0003
application/vnd.oasis.opendocument.text 0.0009 0.0007 0.0007
application/vnd.openxmlformats-officedocument.presentationml.presentation 0.0002 0.0002 0.0002
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet 0.0016 0.0016 0.0016
application/vnd.openxmlformats-officedocument.wordprocessingml.document 0.0023 0.0023 0.0022
application/x-bibtex-text-file 0.0135 0.0145 0.0128
application/x-bittorrent 0.0002 0.0002 0.0007
application/x-dosexec 0.0000 NaN NaN
application/x-endnote-refer 0.0020 0.0020 0.0014
application/x-mobipocket-ebook 0.0008 0.0008 0.0006
application/x-ms-asx 0.0001 0.0001 0.0001
application/x-msdownload 0.0000 0.0000 0.0000
application/x-pds 0.0016 0.0014 0.0013
application/x-rar-compressed 0.0000 0.0000 NaN
application/x-research-info-systems 0.0000 0.0000 0.0000
application/x-sh 0.0008 0.0007 0.0008
application/x-shockwave-flash 0.0001 0.0001 0.0001
application/x-stata-do 0.0004 0.0004 0.0004
application/x-tex 0.0003 0.0003 0.0002
application/x-tex-tfm 0.0001 0.0002 0.0003
application/x-tika-msoffice 0.0023 0.0021 0.0020
application/x-tika-ooxml 0.0015 0.0017 0.0016
application/x-wais-source 0.0001 0.0001 0.0001
application/x-xz NaN NaN 0.0000
application/xhtml+xml 8.8559 9.2123 8.4126
application/xml 0.0699 0.0738 0.0661
application/xspf+xml 0.0002 0.0002 0.0002
application/zip 0.0000 0.0000 0.0000
application/zlib 0.0001 0.0001 0.0001
audio/mp4 0.0000 NaN NaN
audio/mpeg 0.0000 0.0000 0.0000
audio/vnd.wave 0.0000 0.0000 0.0000
audio/x-mpegurl 0.0000 0.0000 0.0000
image/gif 0.0000 0.0000 0.0000
image/jpeg 0.0001 0.0001 0.0001
image/png 0.0000 0.0000 0.0000
image/svg+xml 0.0000 0.0000 0.0000
image/tiff 0.0000 NaN 0.0000
image/vnd.djvu 0.0005 0.0008 0.0004
image/webp 0.0000 0.0000 0.0000
message/rfc822 0.0007 0.0008 0.0006
text/asp NaN NaN 0.0000
text/calendar 0.0475 0.0460 0.0410
text/css 0.0007 0.0005 0.0006
text/csv 0.0032 0.0032 0.0032
text/html 89.8990 89.5096 90.4431
text/plain 0.0866 0.0869 0.0773
text/prs.lines.tag 0.0048 0.0048 0.0040
text/tab-separated-values 0.0005 0.0006 0.0004
text/troff 0.0004 0.0006 0.0005
text/turtle 0.0017 0.0017 0.0013
text/vtt 0.0007 0.0008 0.0010
text/x-c++src 0.0001 0.0001 0.0001
text/x-chdr 0.0008 0.0006 0.0005
text/x-csrc 0.0005 0.0005 0.0005
text/x-diff 0.0009 0.0009 0.0009
text/x-jsp 0.0000 0.0000 0.0000
text/x-log 0.0020 0.0015 0.0015
text/x-matlab 0.0018 0.0020 0.0008
text/x-perl 0.0011 0.0010 0.0010
text/x-php 0.0030 0.0024 0.0039
text/x-python 0.0002 0.0003 0.0004
text/x-vcalendar 0.0004 0.0004 0.0003
text/x-vcard 0.0036 0.0037 0.0031
text/x-web-markdown 0.0008 0.0008 0.0012
text/x-yaml 0.0004 0.0005 0.0005
video/mp4 0.0000 0.0000 0.0000
video/quicktime 0.0000 0.0000 NaN
video/webm 0.0000 NaN 0.0000