DANSK DS/ISO 28500 PDF

This international standard specifies the WARC file format: - to store both the payload content and control information from mainstream Internet application layer protocols, such as HTTP, DNS, and FTP; - to store arbitrary metadata linked to other stored data (e.g., subject classifier, discovered language, encoding); - to support data compression and maintain data record integrity; - to store all control information from the harvesting protocol (e.g., request headers), not just response information; - to store the results of data transformations linked to other stored data; - to store a duplicate detection event linked to other stored data (to reduce storage in the presence of identical or substantially similar resources); - to be extended without disruption to existing functionality; - to support handling of overly long records by truncation or segmentation where desired.