Why Kibenge’s lab lost its “reference” status

We made reference to this in our blog post yesterday, but today we realize we have never published this with any context.

Here’s the full story. Earlier this year Dr. Fred Kibenge’s lab at the University of PEI lost its OIE reference status. The lab had been been accredited by the OIE (World Organization for Animal Health) as a reference lab for ISA virus.

Despite what anti-salmon farming activists claim, the reasons why this status was pulled are crystal clear.

“OIE Reference Laboratories have a responsibility to provide high quality disease diagnostic services as global references to all Member Countries of the OIE particularly in the case of doubts or controversies about animal sample analyses,” states a press release from the OIE in Paris last year.

The OIE is therefore keen to maintain the highest technical and operational standards of all Reference Laboratories.

After different Member Countries pointed out questionable diagnostic results emanating from an OIE Reference Laboratory for Infectious Salmon Anaemia located at Atlantic Veterinary College (AVC) in Canada, the OIE decided to conduct an audit of the Reference Laboratory with independent OIE experts from 31 July to 2 August 2012.

Conclusions of the audit were unfavourable and showed that a series of weaknesses in the system have a direct impact on the quality of diagnoses conducted by the OIE Reference Laboratory at AVC.

But wait, there’s more.

After the OIE delegates from all member countries met for their annual general meeting in May this year, they agreed unanimously to pull the reference status from Kibenge’s lab.

The OIE published the full audit report online.

The conclusions showed there were many serious problems with the lab (referred to below as “OIE-RL” for “OIE Reference Lab”)and how it was run.

Principal Conclusions

  • No evidence for any meaningful Quality System in the OIE‐RL
  • OIE‐RL falls well short of acceptable Quality Standards
  • OIE‐RL uses theOIE template for validation of tests, but needs to involve other laboratories in the process
  • Competent laboratory analyst but lack of comprehensive records of continuing professional training and development
  • The OIE expert recognises the importance of laboratory organisation and work flow patterns to avoid contamination, but has not optimised the system. Incompatible pre‐  and post‐PCR procedures were undertaken in the same laboratory.
  • The OIE expert is clearly knowledgeable about ISA, with many peer reviewed publications, but has a research focus, rather than that of a diagnostician
  • Results of tests reported without contextual interpretation
  • No detailed investigation of apparently anomalous results
  • No evidence of multidisciplinary engagement for disease investigation
  • There is a need for much wider involvement with proficiency test/ring test schemes, either as participant or organiser
  • Lack of contact and networking with other laboratories working on ISA
  • Lack of understanding of general philosophy of mutual international support within which OIE operates

The biggest concern raised in the audit was that the lab was ignoring discrepancies in test results and reporting them with little or no interpretation. As we saw in BC, this left the data wide open to abuse and misinterpretation by Alexandra Morton, especially because she did not publish and share the actual lab reports.

As well, the audit raised concerns that the lab was using in-house methods instead of methods described in the OIE Aquatic Manual, and was ignoring obvious opportunities for follow-up tests that would have provided essential information.

Final reports are provided with limited interpretation even when the results of different assays were conflicting.

In an example provided to the panel, seven wild caught cutthroat trout were tested by PCR. The results showed all samples to be negative by real‐time RT‐PCR, while all fish were positive by the conventional segment 8 RT‐PCR and also with an in‐house conventional segment 6 RT‐PCR. In the absence of a real‐time RT‐PCR result to support the conventional RT‐PCR results the panel considers these results to be highly dubious, and that the results should be reported as inconclusive pending further investigation. As a general principle, no further testing should be necessary if a primary screening test (in this case real‐time RT‐PCR) gives a negative result. The rationale for carrying out “confirmatory” testing (conventional RT‐PCR) on negative samples is not clear.

We would regard it as a duty of the OIE‐RL to seek an explanation for these discrepancies. We consider there could be several explanations, including cross‐contamination in the laboratory, but also that the assays were indeed detecting a new genetic variant of ISAV not picked up by the real‐time RT‐PCR used. An obvious part of such an investigation would be the use of alternative real‐time assays as described in the OIE Aquatic Manual, but the panel understands that Dr F. Kibenge used an alternative in‐house real‐time assay to confirm the presence of ISAV and that these results were not shared nor was an explanation provided for the apparent failure of an assay recommended in the OIE manual.

Sequencing followed by a phylogenetic analysis would most probably provide essential information, however Dr Kibenge considered this to be a research issue and had simply reported his findings, even though we were told that further investigation had been initiated.

In conclusion, the panel had concerns that Dr F. Kibenge may deviate from his standard procedures that follow recommendations in the Aquatic Manual, and use alternative non‐validated in‐house methodologies. There seemed to be a lack of appreciation of the obligations on a Reference Laboratory forthorough investigation of dubious orillogicalresults,together with the need for evaluation of the biological significance of results.

There you have it. No conspiracy, just bad science.

Advertisements

3 thoughts on “Why Kibenge’s lab lost its “reference” status”

  1. The results have great and negative significance not only for the lab but for those who draw conclusions based on results from the lab.

    That is why I was surprised that the OIE did not identify the experts who did the study and that there is no statutory declaration by the experts that they are not in any real or perceived conflict of interest.

    That said, their observations that there are deficiencies at the PEI lab is an unsatisfactory situation for all of us who place some faith in the scientific process -for or against aquaculture.

    My experience has nothing to do with aquaculture or laboratory analysis, but any audits I undertook were all done in the full knowledge that my name and qualifications would be public knowledge and that I would be expected to declare any real or perceived conflicts of interest. If that was good enough for public interest regulators, it should have been the same standard for the OIE.

    Is there any consistency left in marine industries?

  2. Ousdale, I’m not clear on the flaws on both sides? As you say, the OIE report is damaging to the UPEI lab for many, many reasons. Bad science and reckless promotion of the bad science – and that’s a polite summary. But the only “flaw” for OIE is that the auditors remain anon?

    I don’t see much balance in these flaws. What am I missing?

  3. The OIE report is not pleasant reading since the lab formerly had a distinguished reputation.

    My one cavail with the OIE report is that it does not list the names of the independent experts which is normal in a scientific audit. It is usual for the experts to list any conflicts.

    It is also usual to list the offending lab’s undertakings before removal. That is conspicuously absent.

    There are clearly flaws on both sides.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s