crawling

a.k.a. collection frequency, term frequency

In search engine technology, crawling refers to finding new online resources, such as new Web pages. Crawlers find these resources in a number of ways including collection frequency, which refers to the number of documents a term appears in compared to total number of documents; and term frequency, which is the number of occurrences of the search term in a document.