Geoindex JISC UK Web Domain Dataset (1996-2010) aggregated by postcode, year and domain
Datasets usually provide raw data for analysis. This raw data often comes in spreadsheet form, but can be any collection of data, on which analysis can be performed.
Data is aggregated to full UK postcode from: Geoindex JISC UK Web Domain Dataset. Counts of postcodes are summed by year of archive.org instance and sub-domain e.g. .ac.uk. Uncompressed CSV is 650mb and contains 21,000,000 records
About the Geoindex http://dx.doi.org/10.5259/ukwa.ds.2/geo/1
The ~2.5 billion 200 OK responses in the JISC UK Web Domain Dataset (1996-2010) dataset have been scanned for geographic references - specifically postcodes. This set of postcode citations, found at particular URLs, crawled at particular times, forms an historical geoindex of the UK web. For more details about how the data was created, its format, and how to use it, see here.
The geoindex is composed of some 700,641,549 lines of TSV data, each asserting that a given web page, crawled at a given data, containe