Information Retrieval

IIR chapters and topics, exam scope (ch1–7), Boolean retrieval, phrase and proximity queries, k-gram indexes, index construction and compression, Zipf's law, the vector space model, ranked retrieval, language models, text classification, clustering, link analysis

Definition of information retrieval, structured vs unstructured data, the Boolean retrieval model, how Google searches and ranks, limits of grep, the term-document incidence matrix and bit operations, tokens and terms, sparse matrices, the inverted index (dictionary, postings, postings lists), steps to build an inverted index and df, processing AND queries, the postings intersection algorithm, evaluating the Boolean model, Westlaw proximity operators, query optimization (increasing df), estimating OR query sizes

Parsing documents (format, language, character set), document units, word, term, token and type, normalization and equivalence classes, tokenization, numbers and dates, case folding, stop words, lemmatization and stemming, the Porter algorithm, comparing stemmers, recall and precision, skip pointers and IntersectWithSkips, skip spacing (√P), phrase queries, biword indexes, positional indexes, proximity search, PositionalIntersect, combination schemes

Term vocabulary and dictionary, fixed-width arrays, hashes and trees, binary trees, B-trees, wildcard queries, reversed B-trees, permuterm index, k-gram index and postfiltering, edit distance, Levenshtein and Damerau-Levenshtein, dynamic programming, weighted edit distance, spelling correction (isolated word and context-sensitive), k-gram overlap, hit-based correction, the Peter Norvig corrector, Soundex

AI-generatedHardware constraints in IR, disk seeks, Reuters RCV1, BSBI, external sorting, merging blocks, SPIMI, per-block hash dictionaries, distributed indexing, master, parsers and inverters, splits, term partitions, segment files, MapReduce, term-partitioned vs document-partitioned indexes, dynamic indexing, main + auxiliary index, invalidation bit-vector, logarithmic merge, building positional indexes