11 A final word: the problem of replication
Every biologist is well aware of the importance of replicating their experiments in order to be confident in their conclusions. This is a lot more tricky when we consider evolution. To test our hypotheses on evolution, the ideal approach would be to rewind the “tape of evolution” (S. J. Gould) and let the story repeats itself several times to see what happens. This is unfortunately not possible, although some experimental evolution studies do manage to replicate evolutionary experiments.
The phylogenetic comparative method introduced in this tutorial is one appropriate approach to protect ourselves from reaching conclusions that are not strongly supported when considered in an evolutionary context. However, even these approach can sometimes fail. This is why extra care is needed in such studies.
When interpreting their results, biologists should first ask whether they have sufficient replication in their data to allow strong conclusions to be made. And by replication, I mean evolutionary replication. Consider the seed plant example introduced above.
If there are multiple species with ovules enclosed or not and that perform double fertilization or not, the most parsimonious scenario for both characters is that each evolved once along the branch of the tree that leads to the flowering plants. In other words, there has been only one transition between the states of each character in the evolution of this group.
So even if there seems to be replication when we look at the species (several species with each character state was sampled), there is no evolutionary replication! So even if the likelihood that those two events occurred on the same branch is very small and even if a contingency test to calculate the likelihood of such an event is significant, this is a little bit like an experiment with one replicate. Therefore, even when a test that accounts for the phylogeny is significant, a lot of caution is needed when interpreting these results. Ideally, a study should have a decent number of evolutionary replications for the results to be biologically meaningful. I encourage you to read the very nice paper of Maddison and Fitzjohn on the subject (Maddison and FitzJohn 2015).
Ideally, before planning an experiment, one should make sure that there is sufficient replication in the evolution of the traits under study among the species considered to have greater confidence in the results. For instance, it would be much better if each character would have evolved 5-6 times each in the previous example, especially if the two characters were always evolving simultaneously!