Good Data vs. Bad Data: Know the Difference


Brandon Griffin, CEO, Ingredient Identity

What is the real foundation of Quality Control? In short, it is data. Boiling it all down to the fundamentals though, there are two conceptual types of data when it comes to quality control before you decide what to do next. These being either good data or as you might be able to guess, bad data. So what makes for good data?

Good data often is accompanied by adjectives such as “traceable,” “robust,” “repeatable,” “accurate,” and “precise.” These are earned, not just assumed or made up. Good data is not only “yes, this is what I expected” but rather a validation of one’s original hypothesis and process with a resulting outcome that can be confirmed, enabling one to make critical decisions on how to proceed or not. Often data are still “good data” even though they don’t yield what we hoped for. Instead, the scientific process continues. The hypothesis is refined further, as part of the scientific process, or more specifically a Scientific Method. A Scientific Method can be thought of as having two main parts, the first being the procedural process steps involved in the preparation of a given sample for testing while the second is the testing technique and associated parameters of the technique itself applied to the sample, generating a result. The better defined, controlled and tracked these steps and parameters are, in both parts of a Scientific Method being executed, the more “good data” results.

As you can imagine, “bad data” comes from a lack of some, any or all the above. Even worse, the “bad data” may be entirely misrepresented – literally stuff just being made up. To slightly complicate the discussion, just because one may obtain the results desired on the first try, such does not necessarily mean that it is actually the result from “good data” or that the Scientific Method used is even appropriate i.e. “Fit for Purpose”, if it was not appropriately qualified first. In fact, rather, a tell-tail sign of either bad data being used, or the likelihood of non-compliance persisting is the issuance of a “results-only” Certificate of Analysis. These types of reports are typically not supported by any actual data or method details because it either cannot be readily confirmed by any sort of traceable documentation in the report itself or, upon further inspection at the lab facility of how the method was actually qualified or followed. Not always, but quite often, this gives way to the phenomenon of “drylabbing” that unfortunately remains active still today in the Industry; a lab generates a report stating some result only, with no actual data or method details that can be made available to the customer, that magically reports what the client was expecting or needing it to be. The concept and practice of upholding full transparency is essential to countering this grossly negligent behavior regardless of whether the laboratory is in-house or third-party.

Why is this critical to know if you are a brand owner?

For starters, the companies needing to test their products and obtain data are using this information to release product for people (most often) to consume. These are products people put in/on their bodies. Companies use the test data received to make critical business decisions on a daily basis that can affect the general safety of the consumer as well as regulatory compliance of the product, inclusive of label claims. It is imperative that the data be, as one might imagine, the good kind.

Not testing at all, testing incorrectly or limiting transparency creates significant risks to brand owners by way of either “bad data” being used to make decisions or worse, no data at all – ignorance is not an excuse for lack of compliance. Risks include not detecting otherwise preventable contaminations resulting in unacceptable products entering the market. There are also potential risks of adverse events resulting from consumers being harmed by a product; there is the further downstream risk of a lawsuit being filed for failing to meet the quality or regulatory requirements, often resulting in personal injury/damages, breaching of agreements, etc. Companies taking such risks should make sure their Product Liability Insurance is paid up, because it is only a matter of doing a simple investigation and asking key questions as part of a discovery process to quickly determine what kind of data is at play.

Lastly, there is the risk of facing a Regulatory Agency action against your brand resulting from the “bad data.” These are not some type of rare unicorn-sighting risky events, but rather real day-in-day-out occurrences impacting businesses with tremendous downstream negative implications throughout the nation. And guess what, contract laboratories aren’t directly liable. It is because they aren’t actually authorizing the release of any products or ingredients on your behalf no matter what is stated on the report – you are! Think about that for a minute. There are no regulatory and hardly any legal consequences to a lab for providing you with bad data, but there are very, very real consequences to a brand releasing products for sale based upon bad data. The price is high for not ensuring the data you get from a testing lab is good data. Awareness and transparency are vital throughout the supply chain when it comes to this issue.

From a regulatory perspective FDA has been focused heavily on the overall compliance of manufacturing operations since the cGMP’s went it effect. However, they are increasingly inspecting non-clinical laboratories of late, including both third-party contract labs and in-house lab operations of contract manufacturers. Just as you must qualify your vendors whether they be manufacturers, distributors, ingredient suppliers, etc., you also must qualify, or ensure your manufacturer has qualified on your behalf, any laboratories involved. If you are using a third-party contract lab directly, be sure to audit them properly, which should not take more than one day to do. If you or personnel in your company do not have the combination of experience, education and training required to properly conduct an analytical laboratory audit, retain someone who does. Do not be fooled into thinking a 3rd-party GMP Certification of any laboratory or manufacturing facility means that you do not have to audit that facility still, because you must; the FDA does not recognize any GMP Certifications as evidence of compliance or acceptance as evidence of vendor qualification.

It is not only required but is also in your best interest to absolutely ensure that you are getting “good data” and not just a report with a result for your specific product that has little-to-no traceability to a qualified Scientific Method being utilized and the actual data/records to support it. Be mindful. Be vigilant. Be compliant.

Categories: News & Events