Information Retrieval

AI-generatedWhy compress, lossy and lossless, Reuters RCV1 statistics, effect of preprocessing on index size, Heaps law, Zipf law, dictionary compression, fixed-width arrays, dictionary as a string, blocking, front coding, postings compression, gap storage, variable length encoding, VB code, nibble and word-aligned codes, unary code, gamma (γ) code, prefix-free, universal, parameter-free, γ vs VB, Reuters index sizes

AI-generatedRanked retrieval, feast or famine, scoring, the Jaccard coefficient and its problems, binary and count matrices, bag of words, term frequency, log-frequency weighting, df and idf, cf vs df, tf-idf, the vector space model, angles instead of distance, length normalization, cosine similarity, CosineScore, SMART notation, lnc.ltn

AI-generatedWhy ranking matters, eye-tracking studies, clicks on rank 1, the problem with cosine normalization, pivot normalization, tiered indexes, components of a search system, phrase, Boolean and wildcard queries with vector space retrieval, storing tf in postings, selecting the top k with a min heap, kNN, PageRank-ordered postings and early termination, document-at-a-time, term-at-a-time, weight-sorted postings, accumulators, conjunctive retrieval