All Posts

HTCondor main and worker on AWS EC2, building a worker AMI, an EC2 control program in Python boto3 (list, zones, start, regions, stop, create, reboot, images), watching the Condor pool go 2→1→2→3→2→1 nodes with start, stop and create, IAM user keys, sharing the pool password, fixing late re-registration and SigV4 clock skew, Chapter 8 and 9 exercises (AMI types, instance types, running condor_status from the tool, VM APIs of Korean cloud providers)

Term vocabulary and dictionary, fixed-width arrays, hashes and trees, binary trees, B-trees, wildcard queries, reversed B-trees, permuterm index, k-gram index and postfiltering, edit distance, Levenshtein and Damerau-Levenshtein, dynamic programming, weighted edit distance, spelling correction (isolated word and context-sensitive), k-gram overlap, hit-based correction, the Peter Norvig corrector, Soundex

AI-generatedHardware constraints in IR, disk seeks, Reuters RCV1, BSBI, external sorting, merging blocks, SPIMI, per-block hash dictionaries, distributed indexing, master, parsers and inverters, splits, term partitions, segment files, MapReduce, term-partitioned vs document-partitioned indexes, dynamic indexing, main + auxiliary index, invalidation bit-vector, logarithmic merge, building positional indexes

AI-generatedWhy compress, lossy and lossless, Reuters RCV1 statistics, effect of preprocessing on index size, Heaps law, Zipf law, dictionary compression, fixed-width arrays, dictionary as a string, blocking, front coding, postings compression, gap storage, variable length encoding, VB code, nibble and word-aligned codes, unary code, gamma (γ) code, prefix-free, universal, parameter-free, γ vs VB, Reuters index sizes

AI-generatedRanked retrieval, feast or famine, scoring, the Jaccard coefficient and its problems, binary and count matrices, bag of words, term frequency, log-frequency weighting, df and idf, cf vs df, tf-idf, the vector space model, angles instead of distance, length normalization, cosine similarity, CosineScore, SMART notation, lnc.ltn