Frequency-dependent Selection

Frequency-dependent selection occurs when a phenotype's fitness depends on how common or rare it is in a population, rather than being fixed. Unlike other types of selection, the advantage of a trait changes based on its frequency. This creates dynamic selection pressures where the most beneficial phenotype shifts as population frequencies change, making it a unique mechanism within natural selection and adaptive evolution.

In positive frequency-dependent selection, common phenotypes gain fitness advantages. Heliconius butterflies with toxic morphs display bright color patterns that birds learn to avoid. When one pattern is common, birds recognize and avoid it, increasing that morph's survival. Conversely, rare color patterns go unrecognized, and birds attack those butterflies until learning they are toxic, then their frequency and fitness increase.

Viceroy butterflies demonstrate negative frequency-dependent selection through mimicry of toxic monarchs. When viceroys are rare, birds avoid them, mistaking them for monarchs. However, as viceroys become more common, birds learn that many are safe to eat, reducing their fitness. This shows how increasing frequency can disadvantage a phenotype when predators recognize it as a distinct, edible species.

Apostatic selection is a form of negative frequency-dependent selection where predators focus on common prey phenotypes. Grove snails display polymorphic shell patterns, but thrush predators concentrate on one or two common forms. These abundant phenotypes experience stronger negative selection pressure, while rare shell patterns escape predation, increasing their relative fitness in the population.

In angiosperms, the S-locus controls self-incompatibility to prevent self-fertilization. Plants with common S-locus alleles frequently encounter false "selves," blocking reproduction with compatible mates. Rarer S-locus forms avoid this problem and experience positive selection, while common forms are selected against. This maintains genetic diversity and prevents inbreeding depression in plant populations.

Fitness changes with phenotype frequency because predator or mate recognition depends on familiarity and learning. When a phenotype is common, predators or competitors become familiar with it, altering survival or reproductive success. In positive frequency-dependent selection, commonness increases recognition and protection; in negative frequency-dependent selection, rarity provides an advantage because predators haven't learned to exploit it.

Müllerian mimicry involves multiple toxic species sharing warning signals, strengthening predator learning. Heliconius cydno and H. sapho both display black and white aposematic coloring in Central America. As this common pattern becomes more frequent, predators learn the warning more effectively, increasing fitness for all individuals displaying it. Hybrid offspring with rare white-red-black patterns lack this protection and have reduced fitness.