An empirical relationship observed in text corpora that the number of distinct word types V(N) grows sublinearly with the total token count N and is commonly modelled as V(N) ≈ K · N^β with 0 < β < 1 and constant K depending on language and tokenization. It characterizes diminishing rates of new vocabulary discovery as a corpus grows.

Principle

Because new tokens increasingly repeat existing types as corpora expand, incremental addition of text yields progressively fewer novel types; therefore vocabulary size should be forecasted with a sublinear growth model rather than a linear extrapolation.

Demonstration

Illustrative scenario → Indexing a news-article collection: after processing 1 million tokens the indexer observes 70,000 distinct types; after doubling tokens, type growth is noticeably less than double, consistent with V(N) ≈ K·N^β. Recognition → Indexers use a fitted K and β to estimate dictionary size and plan storage and stemming. Action → Tokenization rules and stopword lists are refined to control index growth. Consequence → Indexing and compression budgets are set realistically, and experimental designs account for lower marginal vocabulary discovery.

Misapplication

Assuming a single universal β or treating the relation as exact for any tokenization. The error is ignoring that β and K vary with language, genre, morphological richness, preprocessing (e.g., lemmatization), and corpus heterogeneity.

Consequence

Properly used, Heaps’ law guides capacity planning for indexes, vocabulary thresholds in search systems, and expectations for corpus annotation effort; misused, it leads to under‑ or overprovisioning when model parameters are not empirically estimated for the specific corpus and preprocessing pipeline.

Reversal

In artificially generated or adversarially constructed token streams (e.g., many unique identifiers) or corpora with extreme morphological productivity and no normalization, β may approach 1 and vocabulary may grow nearly linearly, so Heaps’ sublinearity assumption fails.

Boundary

Clearly within: large natural language corpora with standard tokenization and minimal synthetic identifiers. Boundary case: agglutinative languages without morphological normalization where apparent type growth is higher. Clearly outside: datasets composed mainly of randomly generated unique strings or database keys.

Semantic Tension

Completeness (exhaustively capturing all types for lexicographic purposes) ↔ Practicality (indexing and storage limits that require token filtering or normalization).

Synthesis

Heaps’ Law is a practical, parametrized expectation about vocabulary discovery that must be empirically fitted and interpreted relative to tokenization and linguistic morphology; it predicts diminishing returns but does not determine which tokens are meaningful for downstream tasks.