Crawler - AFS

AFS Installation and Administration Guide

Product
AFS
AFS_Version
7.10
Platform
RHEL
Category
Reference Guide

A crawler scans specified web sites and follows hyperlinks simulating users behavior, extracts the desired information and detects duplicates. The crawling process consists of two steps: content retrieval and post processing.