See also the letters of Dr Roger Breeze and Dr Alex Donaldson on warmwell
24 December 2006
In the interest of factual accuracy I write to refute many of the misleading statements made by Dr Breeze in his letter of 02 November 2006 to ProMed Mail concerning the portable PCR diagnostic system he tried to promote at the height of the UK 2001 foot-and-mouth disease (FMD) epidemic and to correct some of the wild speculation he made about the impact that the system would have had on disease diagnosis, control strategy and the number of animals culled.
During the UK 2001 FMD epidemic Dr Breeze first made contact with me by 'phone from the United State's Plum Island Laboratory during the evening of Friday 9 March 2001 and asked if it would be possible for him to bring a team to the UK the following week to test a real-time PCR diagnostic system on a farm suspected to have cases of FMD. I pointed out that it would be necessary for the Americans to obtain official permission from the Ministry of Agriculture, Fisheries and Food (MAFF's) before they went near suspected premises and that if he forwarded a request I would be pleased to pass it to Jim Scudamore, Chief Veterinary Officer. During our 'phone conversation I informed Dr Breeze that regardless of the response from Jim Scudamore we would be willing to test the American system under experimental conditions at Pirbright and that I would discuss with colleagues the practicalities of this and let him know our views. The following day I received a letter by fax from Dr Breeze in which he set out his proposal for a collaborative project between the United States Department of Agriculture (USDA), IAH (Pirbright) and MAFF, the main objective of which would be the detection and identification of important diseases with FMD being the first. In that letter Dr Breeze stated "we have not validated and optimized our PCR assay for FMDV on this device as yet but we are close" and "if this assay system can be found useful and validated by MAFF and yourselves in this epidemic, you will have saved a great deal of time and effort and moved FMDV control forward". The references to "validation" in these quoted statements should be noted. It is curious, to say the least, that in his letter to ProMed Mail of 02 November 2006 Dr Breeze wrote "A state of the art real time PCR test was developed based upon this unprecedented genetic information and this test was validated in vitro and in vivo during 2000 and early 2001 by the Plum Island team: these data were later published (1)". The reference cited by Dr Breeze is a paper by Callaghan et al. which was published in June 2002, but more about that later.
On Monday 12 March I advised Dr Breeze that during the weekend I had forwarded his request to Jim Scudamore and assured him that we would be very pleased to collaborate with the USDA. I explained that as we were hosting an inspection from the European Community at Pirbright in the week he proposed to visit with his team it would not be convenient to accommodate his group as well. I informed Dr Breeze that we were planning with MAFF to intensively sample a number of sheep flocks so that a detailed picture could be established of how the virus was behaving during the acute and convalescent phases of the disease and that I was prepared to suggest to Jim Scudamore that the USDA group be involved in these investigations. I asked Dr Breeze how the USDA system performed with blood samples, pointing out that a major problem in the epidemic was silent infection among sheep i.e. the replication and shedding of FMD virus without the development of clinical signs. An assay that could rapidly identify sub-clinically infected animals would obviously be valuable.
As is now well known, Jim Scudamore's decision, conveyed to me on 14 March, was negative. He stated that as MAFF were already inundated with work he didn't want to divert valuable resources to extra studies. He underlined that this did not imply that he was not interested in the work but merely that this was not the time to undertake it. He stated "there will be opportunities for the work to be carried out at a later date and I would like to suggest that you reply to Roger Breeze along the lines that we are interested in this work but now is not appropriate".
I immediately advised Dr Breeze about Jim Scudamore's response and on 16 March received an e-mail from him. In it he stated "Dr Dan Rock from
heads our pathogen detection and identification effort: I have copied him on this e-mail and I will advise him to communicate directly with you on the science and logistics [of collaboration]". Strangely, no communication was received from Dr Rock and the only scientific data we could glean from the Americans - despite several requests - was that with the USDA system it had been found possible to dilute FMD virus grown in cell culture 10,000-fold (i.e. 5 logs) and get a positive result. Colleagues at Pirbright didn't find this claim at all impressive since the analytical sensitivity of a real-time PCR they had developed was 100-fold greater. When I pointed this out to Dr Breeze he replied (23 March) "I seem to have given some wrong numbers on the assay sensitivity and so I will ask Dan to provide the correct data". Again, no communication or scientific data was received from Dr Rock. Plum Island
The next element of the attempted collaboration was an offer from Dr Breeze to demonstrate the USDA system over the internet to representatives from MAFF and Pirbright. I pointed out in my reply that we were working flat out at Pirbright - around the clock in many cases - and we could not spare the time for an activity that was not going to answer the essential question of comparative analytical sensitivity. As an alternative I proposed that a demonstration of the USDA system be given to one of my staff who was due to visit
two days later. I explained that the scientist was familiar with the application of PCR to investigations of FMD pathogenesis and persistence and asked Dr Breeze if this would be possible. No reply to this suggestion was received from Dr Breeze. Instead, almost 6 years later, in his letter to ProMed he berates my colleague for not probing more aggressively for information during his visit to Plum Island . The more relevant question is why should my colleague, a relatively junior scientist, have been expected to seek out the people who were supposed to be working on the PCR system at Plum Island when through the courtesy of a reply from Dr Breeze a demonstration could easily have been arranged between my colleague and the relevant scientists? By the same token, why was it that neither Dr Breeze nor anyone else from the USDA approached Pirbright later on to arrange a proper collaborative scientific evaluation of their test system? The door was still open and indeed, acting in good faith and anticipation, colleagues at Pirbright had carried out our agreement to collect and store duplicate samples for this purpose. Unfortunately, the time and effort they spent doing this during an extraordinarily busy period was wasted. Plum Island
When one compares the chronology of progress made to develop real-time PCR technology for FMD diagnosis at Pirbright with the claims made by Dr Breeze for
the results are revealing and may have a bearing on the failure of the collaborative process during 2001. Plum Island
In his letter of 02 November 2006 to ProMed Mail Dr Breeze makes a great play of the molecular sequencing work done at Plum Island and how this led to the development of a real-time PCR test which "was validated in vitro and vivo during 2000 and early 2001". In the paper resulting from this work (1), for which incidentally, Dr Breeze was not a co-author, it is striking to note that there is no mention of a much earlier paper from Pirbright (2), submitted on 27 June 2000 and published in 2001, which describes the development of a real-time PCR for FMD. Neither is there any mention in the American paper of an even earlier Pirbright paper submitted to the (American) Journal of Clinical Microbiology in March 2000 and published later that year (3). Significantly, this paper paved the way for the use of the 5' UTR as the target for the real-time PCR assay developed at Pirbright. Even more significant is that the American paper of 2002 does not mention a report, published in 2001, which describes the evaluation by Pirbright of a portable real-time PCR machine and in which concerns were raised about its analytical sensitivity and some of the practical limitations of its use in the field (4). In fairness, the American paper of 2002 does mention a Pirbright paper submitted in October 2000 and published in 2001 (5) which describes real-time PCR for FMD diagnosis in detail, but only fleetingly in the Introduction and without further comment or discussion.
The American paper of 2002 reports laboratory results with a portable real-time system for the diagnosis of FMD but the study does not constitute a proper scientific validation of the assay. Since Dr Breeze has stated that it did one begins to wonder whether he actually understands what scientific validation means! The scientific validation of assays is obviously crucially important in diseases such as FMD where the economic and social consequences of a positive result are so grave. Development and validation of an assay is an incremental process consisting of at least five stages which includes, critically, the establishment of diagnostic sensitivity and specificity (6). By no stretch of the imagination do the results presented in the American paper of 2002 constitute a proper scientific validation of the assay presented. For example, no statistical results are provided for its sensitivity or specificity. Indeed the authors state, entirely subjectively; "our data suggest that the assay is as sensitive or even more sensitive than the presently accepted standard method of virus isolation". However, aside from the absence of any statistical data to support the claim, even this statement can be challenged because the comparator virus isolation system they used as a "gold standard" was a continuous bovine kidney cell line (LK-BK) which is neither "standard" nor the most sensitive system for the isolation of FMD virus. It has long been know that primary bovine thyroid cell cultures are more sensitive than established cell lines (7).
By contrast, the authors of the paper from Pirbright, which evaluated the portable Cepheid SmartCycler real-time PCR machine for the rapid diagnosis of foot-and-mouth disease (4), recognising the importance of test validation, stated: "this was a preliminary, small-scale investigation and, although the results are encouraging, they must be validated by the analysis of large numbers of samples from many serotypes and isolates of FMD virus before the assay could be considered for routine diagnostic use". It is worth noting firstly, that this paper was published in May 2002, i.e. before the American paper and secondly, that for much of the time when colleagues at Pirbright were refining real-time PCR technology and evaluating the portable system hundreds of samples were being investigated each day and so the working environment was very difficult - a situation that did not prevail at Plum Island.
In his letter to ProMed Mail of
02 November 2006Dr Breeze not only contradicts the statements he made in March 2001 about the validation of the American PCR system, he downplays the topic, confusing scientific validation with state authorisation which is an entirely different issue. Both scientific validation and state authorisation of tests were relevant issues during the 2001 UK FMD epidemic. In brief, MAFF (later DEFRA) and IAH, Pirbright are separate organisations; MAFF being a department within the UK Government and Pirbright being part of the Institute for Animal Health (IAH) within the Biotechnology and Biological Sciences Research Council. During and after the UK 2001 epidemic the diagnostic and surveillance tests carried out at Pirbright were performed under a contract with MAFF which specified that the laboratory activities had to be performed under an accredited quality assurance scheme and that the tests employed had to be scientifically validated and/or internationally approved. Taking a test into the field for statutory use which did not satisfy those requirements, as proposed by Dr Breeze, was therefore a complete non-starter. Furthermore, it was wild speculation for him to claim that the deployment of the USDA test to the field during 2001 would have resulted in the abandonment of the contiguous cull policy and thereby saved the lives of millions of uninfected animals.
Although he downplays validation in the case of the PCR assay, Dr Breeze goes on to decry the use in 2001 of the non-validated policy of pre-emptive contiguous culling driven by mathematical modellers. Significantly, one of the main reasons why many of us at Pirbright and elsewhere argued against the contiguous cull in 2001 was that it was based on poor science and not scientifically validated, although, of course, state approved - QED!
Returning for a moment to the topic of laboratory tests and the investigation of suspected cases of FMD, there is no doubt that lessons can be learnt from experiences during the epidemic of 2001. It is incontestable that speed of diagnosis, both clinical and laboratory, is of paramount importance. Unfortunately, confirmatory laboratory diagnosis during the UK 2001 epidemic was largely abandoned in March 2001 when slaughter on suspicion, a policy based on clinical judgement only, came into operation. This was because the combined time taken to transport and investigate samples at Pirbright often exceeded the target time of culling within 24 hours. Since suggestions of more rapid methods of transport, such as the use of helicopters, fell on deaf ears, the frequency of laboratory diagnosis declined. In many cases testing was not done at all or else samples were received after animals had been culled. Consequently, the responsibility for diagnosis fell entirely on field veterinarians, most of whom had never seen FMD. We now know that this led to many herds and flocks being needlessly slaughtered with errors being compounded by the model-driven contiguous cull policy which meant that when disease was diagnosed the animals on five neighbouring premises, on average, were slaughtered for every infected premises diagnosed.
The Royal Society inquiry "Infectious diseases in livestock (2002)" examined how the
might prevent and combat further invasions of highly infectious livestock diseases such as FMD and stated: "recent developments offer good prospects for a 'pen-side' test that could be used by veterinarians in the field. Working devices exist, but still need to be internationally validated and further developed so that they are sufficiently cheap and robust for regular use". I believe that two different diagnostic regimens could be considered: (i) an immuno-diffusion assay for detecting the presence of FMD viral antigen; and (ii) a real-time assay that tests for the presence of FMD viral RNA. The former is probably more suitable for on-farm testing in a situation where cases with vesicular lesions are evident since the system is low tech, cheap and the immuno-diffusion plate could be placed in disinfectant and left on the farm after use. On the other hand, a portable real-time PCR system could be used to identify animals in the late stages of incubation before lesions have developed but when blood, nasal, milk and air samples are likely to be positive. The components of the latter system, a laptop computer and a portable real-time machine, each cost several hundred pounds, and require expertise to operate. While I can see these being used by experienced operators near a farm, perhaps in a local or regional animal health office, I do not see that it would be necessary or desirable to take these items onto a farm, and more especially, onto a series of farms. It would be preferable to transport the samples under secure conditions to where the sampling equipment is located. United Kingdom
Obviously, those expected to use on-farm or near farm tests in any future outbreak or epidemic will have to be familiar with the equipment and the interpretation of results. These skills must be learnt in advance of any episode and not during it. This was clearly demonstrated in 2001 when, as part of a collaborative project between Pirbright and the Central Science Laboratory, York around 300 immuno-diffusion plates for the detection of FMD viral antigen (8) were sent to the field on a trial basis. Unfortunately, very few of these were actually used by veterinarians in the field. This was for a variety of reasons but the most unexpected was that field veterinarians did not want to have their clinical judgement challenged in the face of owners! One has to wonder, therefore, if the USDA system had been taken to the field in 2001 whether veterinarians would have been content to have the results "beamed back to base" and their clinical assessment subjected to a second opinion, as proposed by Dr Breeze?
In conclusion, it is clear that the diagnostic equipment which the Americans wanted to use in the field during the UK FMD epidemic of 2001 was not scientifically fit for purpose and so the claim made by Dr Breeze that its use would probably have altered disease control policy and saved the lives of millions of animals was fanciful conjecture.
Alex I Donaldson
(Head of the UK Institute for Animal Health, Pirbright Laboratory [housing the FAO/OIE World Reference Laboratory for FMD]) from 1989 to 2002.
1. Callahan J D, Brown F, Osorio F A, Sur J H, Kramer E, Long G W, Lubroth J, Ellis S J, Shoulars K S, Gaffney K L, Rock D L and Nelson W M (2002) Use of a portable real-time reverse transcriptase-polymerase chain reaction assay for rapid detection of foot-and-mouth disease virus. Journal of the American Veterinary Medical Association 220, 1636-1642.
2. Oleksiewicz M B, Donaldson A I and Alexandersen S (2001) Development of a novel real-time RT-PCR assay for quantitation of foot-and-mouth disease in diverse porcine tissues. Journal of Virological Methods 92, 23-35.
3. Alexandersen A, Forsyth M A, Reid S C and Belsham G J (2000) Development of RT-PCR (oligo-probing) ELISAs for the diagnosis and typing of foot-and-mouth disease: a new system using Simple and Aqueous Phase hybridization. Journal of Clinical Microbiology 38, 4604-4613.
4. Hearps A, Zhang Z and Alexandersen S (2002) Evaluation of the portable Cepheid SmartCycler real-time PCR machine for the rapid diagnosis of foot-and-mouth disease. Veterinary Record 150, 625-628.
5. Alexandersen S, Oleksiewicz M B and Donaldson A I (2001) The early pathogenesis of foot-and-mouth disease in pigs infected by contact: a quantitative time course study using TaqMan RT-PCR. Journal of General Virology 82, 747-755.
6. Principles of validation of diagnostic assays for infectious diseases (2004). Chapter 1.1.3. In "Manual of diagnostic tests and vaccines for terrestrial animals (mammals, birds and bees)". Fifth edition, 2004. OIE, Paris. pp 21-29.
7. Foot and Mouth Disease (2004). Chapter 2.1.1. In "Manual of diagnostic tests and vaccines for terrestrial animals (mammals, birds and bees)". Fifth edition, 2004. OIE, Paris. pp 111-128.
8. Reid S M, Ferris N P, Bruning A, Hutchings G H, Kowalska Z and Akerblom L (2001) Development of a rapid chromatographic strip test for the pen-side detection of foot-and-mouth disease virus antigen. Journal of Virological Methods 96, 189-202.
See also the letters of Dr Breeze and Dr Alex Donaldson on warmwell