Visible to the public Hybrid Focused Crawling for Homemade Explosives Discovery on Surface and Dark Web

TitleHybrid Focused Crawling for Homemade Explosives Discovery on Surface and Dark Web
Publication TypeConference Paper
Year of Publication2016
AuthorsIliou, C., Kalpakis, G., Tsikrika, T., Vrochidis, S., Kompatsiaris, I.
Conference Name2016 11th International Conference on Availability, Reliability and Security (ARES)
Keywordsclassifier-guided hyperlink selection strategy, Context, Crawlers, dark web, Darknets, explosives, focused crawling, Freenet, government data processing, homemade explosives discovery, Human Behavior, human factors, hybrid focused crawling, I2P, information retrieval, Internet, pubcrawl, search engines, surface Web, Terrorism, Tor, Uniform resource locators, Web pages, Web resource discovery
AbstractThis work proposes a generic focused crawling framework for discovering resources on any given topic that reside on the Surface or the Dark Web. The proposed crawler is able to seamlessly traverse the Surface Web and several darknets present in the Dark Web (i.e. Tor, I2P and Freenet) during a single crawl by automatically adapting its crawling behavior and its classifier-guided hyperlink selection strategy based on the network type. This hybrid focused crawler is demonstrated for the discovery of Web resources containing recipes for producing homemade explosives. The evaluation experiments indicate the effectiveness of the proposed ap-proach both for the Surface and the Dark Web.
Citation Keyiliou_hybrid_2016