logo
Published on

CH0. Course Overview

Authors
  • avatar
    Name
    seren-wib
    Twitter
Contents

0. Course overview

  • This semester covers 18–20 of the 21 IIR chapters. The exam covers ch1–7
  • Take-away: the design and data structures of a simple IR system (Boolean retrieval) + the topics this course will cover
IIR chapterTopicIn one lineCourse
01Boolean retrievalInverted index, postings intersection, query optimizationch1
02The term vocabulary and postings listsPhrase queries "Stanford University" and proximity queries Gates near Microsoft → the index must store word positions tooch2
03Dictionaries and tolerant retrievalk-gram index: bo → aboard, about, boardroom, border. Find words even when misspelledch3
04Index constructionSplit a large collection across many machines (parser → inverter, map·reduce)ch4
05Index compressionZipf's law: log frequency vs log rank is a straight linech5
06Scoring, term weighting, the vector space modelBoolean only includes/excludes → rank by query-document proximity. Needs a function that turns occurrence counts into scoresch6
07Computing scores in a complete search systemThe whole structure: parser, index, spelling correction, scoring, rankingch7
08Evaluation, dynamic summariesOut of scope
09Relevance feedback, query expansionOut of scope
12Language modelsUnigram model: multiply word probabilities to get a string's probabilityOut of scope
13Text classification, Naive BayesAutomatically assign documents to predefined classes (language, adult content, region)Out of scope
14Vector classificationOut of scope
15SVMOut of scope
16Flat clusteringOut of scope
17Hierarchical clusteringOut of scope
18Latent semantic indexing (LSI)Out of scope
19The web and its challengesDiverse documents and users, using link analysis and user dataOut of scope
21Link analysis, PageRankOut of scope
  • What continues from ch1: 1.1 ranking → ch6 / 1.2 index construction and compression → ch4·ch5 / 1.3 Westlaw proximity operators → ch2 proximity queries