statistical-language-modeling

Research project investigating whether training-only context sparsity can predict the marginal held-out benefit of increasing n-gram order.

This thesis presents an empirical comparison between PPM★ (Prediction by Partial Matching), a state-of-the-art statistical language modeling algorithm, and IDyOT (Information Dynamics of Thinking), a cognitive architecture designed to model human-like hierarchical learning and prediction.

Implementation of classic N-Gram Language Models with Laplace smoothing, interpolation backoff, and perplexity evaluation — built from scratch in Python.