How Many Replicates Are Enough?
Buddleia leaf weevil. Image - Scion
The buddleia leaf weevil (Cleopus japonicus) is now a well-established biocontrol agent for buddleia (Buddleja davidii). Since its first release in 2006 it has spread throughout both the North and South Islands. First identified as a possible control agent in the early 1990s, testing by Scion showed it could suppress buddleia growth and even kill the plants.
An essential step in the introduction of a new biocontrol agent is an assessment of the risk it poses to non-target species. Thoughtful experimental design and statistical analyses are needed to ensure valid results are produced and that their meaning can be interpreted correctly so that suitable agents are not rejected unnecessarily or unsuitable agents released.
“Of special concern in the buddleia weevil testing was the potential threat to Hebe speciosa, an endangered plant and a taonga quite closely related to buddleia,” explained Toni Withers of Scion. The risk to H. speciosa was initially assessed by placing five larvae on leaves and monitoring their progress. In one of six replicates, one of the five larvae continued to feed and develop into a pupa, long after all the others died, but the ultimate fate of this pupa was lost. A second trial with double the number of replicates was then carried out. No larvae survived to pupation and approval was then given by the Environmental Risk Management Authority to release the weevil.
Plant species that do not support development of a candidate biocontrol agent during host-range testing are considered to be outside the fundamental host-range of that agent and not hosts. However, when a biocontrol agent is capable of rearing through to adult on a test plant it can be difficult to determine the risk of non-target attack. Clearly, if only a tiny proportion of individuals rear through on a test plant, compared to the target weed (as in the case of the buddleia weevil), then the test plant is unlikely to be a suitable host, but how much replication do you need before you can be certain that you aren’t missing or dismissing a potentially important result? The more replication undertaken, the more the testing costs and the longer it takes – so it is important to find the right balance here.
Ideally, the number of replicates needed for any experiment should be established during the design phase. “One way to do this is to carry out a power analysis – a statistical technique to determine the sample size required for the results of an experiment to be statistically valid,” explained Toni. Selecting a biologically relevant effect size (in this case the “effect” is percentage survival to pupation) can be a major limitation to power analysis. An alternative is to use a range of effect sizes to better understand the relationship between effect and sample sizes. Figure 1 provides an example showing that to confidently detect something that occurs often, or 10 times in 100, only 30 replicates may be necessary. Detecting a rarer event, something that occurs twice in 100, would require 300 replicates, while the detection of extremely rare events would require thousands of replicates.
Buddleia weevil on H. speciosa provides a useful retrospective case study to test the power analysis approach. The first trial could be viewed as providing a preliminary look for effects and effect size. With development to pupation observed in one out of six replicates, the effect could be assumed to be large. A second trial could then be designed using power analysis to ensure the inclusion of enough replicates (20–30) to ascertain whether the first trial had returned a false positive and whether pupation on the non-target was not a frequent event. If doubt still remained, a third trial using a higher sample size, say 100 replicates, could be run to confirm that pupation was indeed a rare event.
Other retrospective analyses have shown that unexpected non-target damage from weed biocontrol agents is extremely rare. However, given that with host specificity testing there is always a danger of false positive and false negative results, it is good to have additional tools that increase confidence in our predictions, which in turn help to maintain this safety record.
This research was supported by the Better Border Biosecurity collaboration (B3) www.b3nz.org.
Contact
Toni Withers
toni.withers@scionresearch.com
Further information
Withers TM, Carlson CA, Gresham BA 2013. Statistical tools to interpret risks that arise from rare events in host specificity testing. Biological Control 64: 177–185.
