Archiving Internet Information
1.Overview of the Web Archiving Project
(1) Archiving websites for the future
Internet information is updated frequently, and websites may disappear quickly. Also, in recent years, important documents such as reports issued by public agencies have shifted from paper media to the Internet. In order for users to use these contents, the NDL regularly collects and archives relevant websites.
- Web Archiving Project (WARP)
* In this project, archived websites are preserved in their original condition. - The National Diet Library Digital Collections (Online Publications)
* This collection preserves publications such as books and magazines in the archived websites.
(2) History of the Web Archiving Project
The NDL first implemented the NDL Web Archiving Project (WARP) for archiving, preserving and providing Internet information in Japan upon approval from each publisher in 2002.
Since then, the NDL has conducted the following research to study archiving methods and subjects, as well as the challenges anticipated: Research on comprehensive archives, collection and preservation of Japanese websites (October 2004-March 2005)(in Japanse, PDF: 193KB) and Collecting opinions on systematic implementation of archiving and using Internet information (April 2005)(in Japanese). In light of the results, the NDL decided to archive websites of public agencies in Japan comprehensively after relevant legislation. On July 10, 2009, an amendment to part of the National Diet Library Law was promulgated, enabling the NDL to archive and preserve Internet information published by public agencies. With the enforcement of the law on April 1, 2010, the NDL started to comprehensively archive the websites of public agencies.
2.Archiving Based on the National Diet Library Law
(1) Information to be archived
Information released by the following organizations is to be archived based on the National Diet Library Law.
Organizations specified in Article 24 of the National Diet Library Law
- National organizations such as legislative bodies, administrative bodies and judicial bodies, including their local branch offices.
- Independent administrative agencies
- National university corporations (including inter-university research institute corporations)
- Some special corporations
Organizations specified in Article 24-2 of the National Diet Library Law
- Local public bodies including the Legal Absorption Conference
- Some local public corporations
(2) Archive frequency
Type of Organization | Frequency |
---|---|
National organizations | Monthly |
Local public bodies, local public corporations, etc. | Quarterly |
Independent administrative agencies, special corporations, etc. | Quarterly |
National, prefectural and other public universities and colleges | Quarterly |
(3) Archiving method
Websites are collected through an automatic archiving program called Web Crawler. Then, the data of the collected websites is organized and archived in the NDL system. Among the information which is not able to be archived automatically, a certain type of work files defined by the NDL needs to be sent by each publisher.
For details, please see Archiving Internet Information Based on The National Diet Library Law (in Japanese, PDF: 531KB).
(4) Requests for cooperation for archiving
- Change or cancel the settings which prevent automatic archiving by the NDL.
* For changing these settings, please see Archiving Internet Information Based on The National Diet Library Law (in Japanese, PDF: 531KB). - Send work files that are not able to be archived by the automatic archiving program.
* For the types of work files which are to be sent to the NDL, please see Archiving Internet Information Based on The National Diet Library Law (in Japanese, PDF: 531KB). The NDL will contact the publishers individually to discuss this matter.
(5) Release of the archived data
Internet information archived based on the National Diet Library Law is available in the NDL (the Tokyo Main Library, the Kansai-kan of the NDL, and the International Library of Children's Literature).
Also, information for which permission has been obtained from each publisher will be released on the Internet. The copying service (printing on paper) in the NDL will be available based on the publishers' permission.
3.Archiving Based on Permission
Websites excluding those specified by the National Diet Library Law, such as those of public interest corporations', private universities', political parties', international/cultural events' websites, and those related to the Great East Japan Earthquake, are archived, preserved and released by the NDL upon obtaining permission from the publishers of the information.
4.Contact Us
Digital Library Division, Kansai-kan of the NDL
E-mail: warp