Home

Welcome to the storm-crawler wiki!

Getting Started

TBD

Components

• Protocols: Network protocols that are usable in storm-crawler
• SiteMapParserBolt: how to handle sitemaps
• JSoupParserBolt: parse HTML documents
• URLFilters: how to filter or normalise outlinks
• Configuration: how to configure the storm-crawler

Resources

• Presentations
• Powered By
• External plugins