Donna Bergmark, Research Programmer in the Digital Library Research Group, will speak about her work on building digital collections by crawling the web. The invention of the hyperlink and the HTTP transmission protocol caused an amazing new structure to appear on the Internet: the World Wide Web. With the Web, there came spiders, robots, and Web crawlers, which go from one link to the next checking web health, ferreting out information and resources, and imposing organization on the huge collection of information (and dross) residing on the net. This talk will consider crawlers and their history, and report on the use of one such crawler to synthesize document collections on various topics in science and mathematics. Such collections could be part of a digital library.

None