DROPS

Document

DOI: 10.4230/LIPIcs.STACS.2009.1809

A Comparison of Techniques for Sampling Web Pages

Authors: Eda Baykan, Monika Henzinger, Stefan F. Keller, Sebastian de Castelberg, and Markus Kinzler

Published in: LIPIcs, Volume 3, 26th International Symposium on Theoretical Aspects of Computer Science (2009)

Abstract

As the World Wide Web is growing rapidly, it is getting increasingly challenging to gather representative information about it. Instead of crawling the web exhaustively one has to resort to other techniques like sampling to determine the properties of the web. A uniform random sample of the web would be useful to determine the percentage of web pages in a specific language, on a topic or in a top level domain. Unfortunately, no approach has been shown to sample the web pages in an unbiased way. Three promising web sampling algorithms are based on random walks. They each have been evaluated individually, but making a comparison on different data sets is not possible. We directly compare these algorithms in this paper. We performed three random walks on the web under the same conditions and analyzed their outcomes in detail. We discuss the strengths and the weaknesses of each algorithm and propose improvements based on experimental results.

Cite as

Eda Baykan, Monika Henzinger, Stefan F. Keller, Sebastian de Castelberg, and Markus Kinzler. A Comparison of Techniques for Sampling Web Pages. In 26th International Symposium on Theoretical Aspects of Computer Science. Leibniz International Proceedings in Informatics (LIPIcs), Volume 3, pp. 13-30, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2009)

Copy BibTex To Clipboard

@InProceedings{baykan_et_al:LIPIcs.STACS.2009.1809,
  author =	{Baykan, Eda and Henzinger, Monika and Keller, Stefan F. and de Castelberg, Sebastian and Kinzler, Markus},
  title =	{{A Comparison of Techniques for Sampling Web Pages}},
  booktitle =	{26th International Symposium on Theoretical Aspects of Computer Science},
  pages =	{13--30},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-939897-09-5},
  ISSN =	{1868-8969},
  year =	{2009},
  volume =	{3},
  editor =	{Albers, Susanne and Marion, Jean-Yves},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.STACS.2009.1809},
  URN =		{urn:nbn:de:0030-drops-18091},
  doi =		{10.4230/LIPIcs.STACS.2009.1809},
  annote =	{Keywords: Random walks, Sampling web pages}
}

Search Results

Documents authored by de Castelberg, Sebastian

A Comparison of Techniques for Sampling Web Pages

Abstract

Cite as

Thanks for your feedback!

Could not send message