Food Safety and Security Documents Download

Dr. Roy Betts

Roy Betts is the Science Fellow at Campden BRI, an independent international food consultancy and research organisation based in the UK. His role is to ensure that the industry maintains a full awareness of all microbiological concerns and to help companies respond to and manage microbiological issues relevant to them and their products. Prior to taking this role, Roy managed a large microbiology department at Campden BRI, with particular interests in test method development and validation, and incident and crisis management.

Roy has published widely in the area of food microbiology and is a member of many committees including:

• British Standards Institute AW 9 Committee, Methods for the Microbiological of Foods and Animal Feeds
• Former Chairman of the Board of Directors of the Association of Analytical Chemists (AOAC) Research Institute
• Chairman of the MicroVal General Committee for European microbiological method validation.
• Expert reviewer for the AOAC RI for microbiological test methods
• Member of ISO TC34/SC9 WG3 – Validation of microbiological methods
• Member of International Life Sciences Institute (ILSI) , "Risk Analysis in Food Microbiology Task Force"
• Member of the ILSI sub group on "Industrial Microbiological Risk Assessment”
• Member of the FDF Food Hygiene Committee.
• Member of the UK Advisory Committee for the Microbiological Safety for Foods

With an International reputation as an expert in food microbiology he has received personal invitations to speak at international symposia in the US, Australia, New Zealand, India, Korea, China and throughout the EU and regularly prepares papers for leading publications. Roy has presented over 100 publications/papers on food microbiology in journals and at Symposia.

Hello there. In this talk, I want to cover a microbiological test method validation and verification, what they are, how they differ when you use them. My name is Roy Betts. I'm a research fellow at Camden BRI, an independent food technology and consultancy organisation in the UK. I also chair the MicroVal general committee. MicroVal is an organisation that validates new microbiological test methods. And that allows me to talk about validation from that particular role.

So let's look at microbiological validation, test method validation and verification, and their differences. Well, test method validation is measuring the performance characteristics of a test method. It's done once by a single expert laboratory. And basically, it shows that the method works. It compares the results of that new method with the results that are obtained from a standard reference method. And it's showing that the method actually works in the hands of the expert laboratory. It gives confidence that the method is fit for use. Verification, on the other hand, is comparing the performance of a new method in a single laboratory, they use a laboratory, to performance that’s seen during the validation of the method. So this verification should be done by all laboratories that are using methods for routine testing. And it shows if that one single user laboratory, the lab using the method, can use the method and get the correct results. And validation is done once by an expert laboratory to prove a method works. Verification is done by all user laboratories to show that they can use the method and get the correct results.

So let's go talk about validation. Now, validation that tends to be done using a very standardised validation protocol. And such a protocol is this international standard 16140-2, which covers the validation of proprietary or kit-based microbiological test methods. And what validation is doing is comparing the new method against a known reference method, for example, an international standard, and it's trying to prove that the new method is giving equivalent results to the standard reference method. As I said before, it's done only once by a single expert laboratory, such as Camden BRI. Camden BRI, are an expert validation laboratory for ISO 16140 validations by MicroVal. And it's managed by an independent validation certification body and an example of that—that certification validation certification body—at MicroVal, independent of the laboratory doing the work, independent of the manufacturer of the test methods, and that independence gives the credibility behind the method validation.

The validation according to ISO 16140 is divided into two separate parts. The first part is known as the methods comparison study. Methods comparison studies are designed to compare the results of the new method with those of the reference method that is designed to do the same thing. That methods comparison study is done by a single expert laboratory. As I said, Camden are an expert laboratory for doing micro validations. Once that methods comparison study is complete and we know the results, we move on to doing the interlaboratory study. Now the interlaboratory study is a study that's done by a number of collaborating laboratories. It's organised by the single expert laboratory. And basically, what an expert lab does is it prepares food samples containing the microorganisms or the group of microorganisms that have been tested by the methods. And it sends the inoculated food products around to those eight to ten collaborating laboratories. They test those foods with the reference method and the new method, and they send the results back to the expert laboratory. It's known as a blind study because the labs do not know what samples they're handling and whether they're contaminated or not. And results will then be analysed by the expert laboratory.

So, what sort of things do we look for within the methods comparison study that's done by the expert lab when doing validations? Well, in qualitative methods, presence or absence tests, we're looking for things like the test method sensitivity, the relative level of detection, and the inclusivity and the exclusivity within the test, which organisms does it detect, and which does it exclude from the test. In quantitative methods, in enumeration tests, we're looking for things like the relative trueness, the accuracy profile, the limit of quantification, the lowest level that organisms can be counted. And again, the inclusivity and the exclusivity, which organisms or groups are included, and which are excluded from the test results.

In the interlaboratory study, as I said before, the expert lab prepares contaminated samples, and sends them to the collaborating laboratories. Those laboratories analyse those contaminated samples with both the reference method and the new method that's been validated. And they send the results back to the expert laboratory for analysis. And the statistical analysis is done by an expert lab. And the results will then be presented to the MicroVal Technical Committee, for them to look at the results and to confirm or not whether extra work needs to be done, or the method is indeed valid for use.

As I said, the expert lab does the statistical analysis. The validation certification body, for example, MicroVal, then checks those results, and then a certification body. If the results are correct, the certification body will assess and check that the results are able to meet the requirements of ISO 16140 and will certify that the method is valid and gives equivalent results to the reference method. One thing to note when you're looking at validation certificates and validations is that all validations have a scope. And the scope tells you what food types are included in the validation. So what foods is it valid to test and what organisms are going to be detected, and what sample sizes can be used within, when using the new method.

This is an example of a validation certificate. It's a MicroVal validation certificate, the certification body is Lloyd's Register, and basically, it will tell you all the details you need to know about that certification. So in this particular one, it tells you that it's a MicroVal certificate, it tells you that the reference method used was the ISO method for detecting a presumptive Bacillus cereus. The scope is a broad range of foods. So it's not one food or another. It's a broad range of food. And it tells you that it's been validated in accordance with ISO 16140. So the validation certificate that tells you all you need to know about which food types can be tested and which organisms are detected using that particular new method.

And moving away from validation, the second thing I was going to cover in this talk was verification. I said verification is or should be used within every single laboratory that's going to use a method to test food samples and give microbiological results. Now, up until recently, we didn't have any standard protocol in food microbiology for doing verification in user laboratories. That changed in January 2021 when a new international standard, a new ISO standard, was published for method verification, and the standard was 16140-3. And although this hasn't been used up to now for verification, clearly, in future, now that this is a published standard, it will begin to be used by laboratories to do method verifications in their use. So it's a protocol for the verification of reference methods and validated methods in a single user laboratory.

So what does it involve? Well, what verification actually is involving and what it does, is to show that any testing laboratory can use that method to get the correct results. So it's proving that the user laboratories, the laboratories that are testing foods, can generate the correct results with the method. Verification is always and has always been required for any laboratory that's accredited under ISO 17025 accreditation. But in the past, there's not been a standard way of doing this verification. That's what this new ISO standard is giving us.

So, the standard, as I said beforehand was ISO 16140-3. And the verification process that it describes, is divided into two parts. And those two parts are called, firstly, implementation verification. And secondly, food item verification. I'm going to talk about what those two parts are all about in the next slides.

Implementation verification is basically used to show that the single user laboratory can get a result that is very close to the results that were obtained by the validation expert laboratory during validation. So what happens is that the laboratory that's verifying the method will take or will use one food that was included within the validation that was done on that method by the expert laboratory. And using inoculated studies, it takes that food, it inoculates them with the organism or group that it is trying to detect. And it's looking to be able to get very similar results in those studies, to those that were obtained by the expert laboratory during the validation study. And what implementation verification is doing is indicating the competency of the user laboratory in using that method. So it's comparing its results, really to the results that are obtained by the expert laboratory in the validation process.

The second part of the verification is called food item verification. Now we do food item verification because we know that during validation, it's impossible to test every single food type and show that they're valid. So we test a range of foods to give a feeling that the method works. However, when it comes to individual laboratories, every individual testing lab knows very well the type of methods and the type of foods that they will be testing. So what food item verification is doing is it's demonstrating the ability of the user to use the method with the types of food sample that they commonly use and test. So, they have to use the foods that they are commonly testing in their hands. And the number of foods that are included in food item verification depends upon the frames of the laboratory and what it can use that method for. So, is it only testing dairy samples? Or is it testing dairy and meat and produce? So, it depends on how many food types the laboratory claim they can use the method for, but the maximum number that they will end up verifying is five different food types.

So, when all the results are produced according to the standard, they will be subjected to a statistical analysis and the statistical analysis is given within the standard approach, the ISO 16140-3. And indeed, in the near future, there will be an Excel spreadsheet on the ISO website that allows very simple calculation of the statistics behind the results analysis. And once the statistics have been done, it will show whether the method is valid for use or not.

So in the final slide, how do you ensure that you get reliable and trusted results from any testing laboratory for microbiological methods? Well, firstly, make sure that methods used are validated methods. The new methods have been validated according to standards such as ISO 16140-2, and have they been independently certified by an organisation such as MicroVal or indeed AFNOR or AOAC. Secondly, when you look at that validation certificate, are the foods that are being tested included in the scope of the validation? If you're testing dairy products, are you sure that dairy is included in the scope of validation, or meat, or whatever. So make sure that the methods are included in the scope. And when it comes to the testing laboratory, make sure that the lab has verified that they can use those methods in their own hands. Now any accredited laboratory will have had to have done verification. This will have been done in a variety of ways in the past, but in the future, we're going to see a greater use of that other standard, ISO 16140-3, the verification of methods in single laboratories. And that's going to be the way it will be done in the future. So make sure methods have been verified, to show that the user laboratory can use them properly and get correct results. And finally, of course, I've talked about laboratory accreditation. Are the laboratories accredited according to the requirements of ISO 17025, the laboratory accreditation standard? And that standard will show that the quality systems and equipment etc. are in place and be used properly in those labs. And indeed, that the methods are included in the scope of the labs’ accreditation. So, are they accredited for those particular methods you wish them to use? By employing those four different approaches, you will be able to ensure that you get reliable, trusted and correct results from microbiological testing laboratories. Thank you very much.