Min-p and Typical Sampling

Nucleus sampling improved on top-k by truncating on cumulative probability instead of a fixed count (see nucleus-sampling-top-p), but cumulative probability is itself only one way to define "the reasonable part of the distribution." Two later methods, min-p and locally typical sampling, each argue that top-p's own definition of "reasonable" is subtly wrong in a specific, fixable way.

Min-p: scale the floor to the top token

Top-p's failure mode appears at high temperature. Temperature flattens a distribution before top-p ever sees it (implementation order varies, but conceptually), and a flattened distribution can push the cumulative-probability threshold p deep enough into the tail to admit tokens that would have been obviously implausible at the model's original, unflattened confidence. The nucleus grows not because the model became more genuinely uncertain, but because temperature mechanically stretched the gaps.

Min-p sidesteps this by defining the candidate threshold relative to the most probable token rather than as an absolute cumulative sum. A token is admitted if its probability is at least min_p times the top token's probability:

threshold = min_p * p_max
keep token i  if  p_i >= threshold

If the model is very confident (p_max close to 1), the threshold is high in absolute terms, so only genuinely competitive tokens survive, min-p stays tight exactly when top-p's temperature-inflated nucleus would loosen. If the model is uncertain (p_max modest), the threshold scales down proportionally, allowing a wider, appropriately diverse set. Because the threshold is always defined relative to p_max, min-p remains well-behaved across a much wider range of temperatures than top-p does, which is its practical selling point: it lets you push temperature higher for genuine creative diversity without the output collapsing into the incoherence that high-temperature top-p is prone to. This method (min-p sampling) was proposed by Nguyen et al., 2024, who report it holding up better than top-p specifically in the high-temperature regime.

Locally typical sampling: truncate by information content, not rank

Typical sampling, from Meister et al., 2022, starts from a different premise entirely. Both top-k and top-p implicitly assume "most probable" is the right criterion for "reasonable to generate." Typical sampling argues this is not what human language does: human text is not composed of a sequence of maximally-probable-given-context words. It contains a mix of expected and mildly surprising words, and the amount of information (measured in bits, via entropy) each word carries tends to sit near the local expected value, neither maximally predictable nor maximally surprising.

Formally, typical sampling keeps tokens whose information content, -log p(token), is close to the conditional entropy of the distribution at that step, H = -sum_i p_i * log p_i, rather than keeping tokens purely by probability rank:

keep token i  if  | -log(p_i) - H |  is small