Author Name: Florence Dalley; BSc (Hons) Veterinary Nursing Science
With hyperthyroidism currently being the most common endocrine disorder in cats, correct diagnosis of the illness is arguably an important topic within the veterinary profession. With convenience, cost and accuracy being strong factors determining choices made in today’s society, having a screening test for thyroid illness that satisfies all these factors would be ideal. Some researchers believe that their study provides evidence to suggest the in-house ELISA test does just this. However, other studies have provided research to suggest otherwise, leading to differing results over the accuracy of the ELISA test. In order to determine whether the ELISA test is accurate enough for routine use in veterinary practices, validity of the research behind these results must be evaluated. For many studies both supporting and opposing the use of ELISA testing, the validity of the research is questionable, resulting in a need for further reliable and independent research in the field to be conducted.
Hyperthyroidism in cats was first recognised as a disorder during the 1970s (Peterson, 2012). With an increasing number of diagnoses being made, it is now the most common endocrine disease among cats (Peterson, 2012). Cats over ten years old are most at risk, with an estimate of over 10% of elderly cats developing the disease (Druce, 2015; Moore, 2011). Hyperthyroidism is associated with an over-active thyroid gland, producing excess of the hormone thyroxine (T4) and tri-iodothyronine (T3) (Bloor, 2013). These hormones are important for stimulating many metabolic functions within the body including thermoregulation, gastrointestinal movement and metabolic rate (Caney, 2012). If untreated, multiple symptoms can develop with weight loss (88%), excessive eating (49%) and vomiting (44%) being the most commonly observed symptoms in cats (Baral et al. 2012).
When hyperthyroidism was first recognised as a disease, diagnosis was relatively straightforward as it was usually carried out on cats with severe to late stage hyperthyroidism (Peterson, 2013). With advances in diagnostic technology, cats with mild or early stage hyperthyroidism are now being diagnosed more readily, leading to uncertainties in the performance of hyperthyroid screening tests (Mardell, 2013; Peterson, 2013; Kemppainen et al. 2006). There are many screening tests available to veterinary surgeons, with those measuring total T4 (TT4) concentrations currently the most common method in practice (Bloor, 2013; Peterson, 2013; Caney, 2012). In 2014, diagnosis of hyperthyroidism was confirmed in 91% of cats in UK veterinary practices on the basis of a TT4 test (Higgs et al. 2014). Compared to measurements such as T3 or free T4, measurements of TT4 are economical, reliable and convenient (Peterson, 2013). Cats measured for TT4 concentration are generally categorised in the low, normal, boardline high or high band (IDEXX Laboratories, 2015). Low or high results indicate thyroid illness, with low TT4 levels suggesting hypothyroidism and high levels suggesting hyperthyroidism (Peterson et al. 2001). TT4 tests available include Radioimmunoassay (RIA), Chemiluminescent Enzyme Immunoassays (CEIA), Enzyme Immunoassay (EIA) and the Enzyme Linked Immunosorbent assay (ELISA) (Peterson, 2013). Excluding the ELISA test, these require sending samples to a laboratory which is time-consuming and costly to process (Peterson, 2013). The importance of cost is reflected in a study by Higgs et al. (2014) where 488 (80%) of veterinary surgeons regarded cost as an important factor in the management of hyperthyroidism.
Currently, UK veterinary practices are increasingly using the in-house ELISA test (Peterson, 2013). This commercial test kit requires only a blood sample and a running time of fifteen minutes, making it an exciting development in the veterinary industry (IDEXX Laboratories, 2015). However, there is controversy as to the accuracy of this test with some studies showing discordant results for only 13 (11%) of feline samples (Cote et al. 2010), and others showing discordant results for up to 28 (56%) of feline samples (Lurye et al. 2002). Treatments for hyperthyroidism are often invasive, so misdiagnosis can prove fatal (Peterson, 2013). By evaluating the validity of research on ELISA testing for hyperthyroidism in cats, the accuracy of the test can be better determined.
2.0 Literature Review
2.1 Consideration of Sample Selection
2.1.1 Age, Breed, Gender and Diet
Vital to any clinical study is the consideration of participant characteristics, such as age, breed and gender, which can potentially influence data (Feldman et al. 2004). It is currently believed that while breed and gender have no influence on TT4 concentrations in cats, age does have an influence, with TT4 levels increased in younger cats and decreasing as cats grow older (Paterson, 2008; Abaxis, 2006). As hyperthyroidism affects mainly cats over ten years old, the age of cats chosen for research on ELISA testing could have ranged anything from ten to twenty years or more (Moore, 2011). Therefore, TT4 levels could have been decreased to varying degrees dependent on the range of ages of the cats involved. None of the relevant studies on ELISA tests, such as Lurye et al. (2002) and Kemppainen et al. (2006), considered age when interpreting their results, potentially leading to misleading conclusions on the accuracy of ELISA testing.
While excess iodine in some cat foods has shown to be associated with an increase in the development of hyperthyroidism (Edinboro et al. 2004), a deficiency of iodine is proven to manage hyperthyroidism, with 90% of hyperthyroid cats on iodine deficient diets having reduced TT4 concentrations (Peterson, 2015; Kooij et al. 2014; Peterson, 2012). Therefore, TT4 concentrations can be influenced by diet, making it possible that the diets of the cats involved in ELISA research could have also affected results.
2.1.2 Thyroid function, Stress and Concurrent Disease
Cats which present most difficulty in diagnosis are those with either mild or early stage hyperthyroidism, with up to 10% of these cases showing normal TT4 levels (Plotnick, 2015; Peterson, 2006). This is increasingly the case, leading to reduced performance in TT4 tests (Peterson, 2013). Therefore, in order to conclude whether the ELISA test is useful for current practice, studies determining the accuracy of the test should use cats with differing degrees of thyroid function. The only study which specifically used samples from a wide range of TT4 concentrations was Lurye et al. (2002), which concluded that 25 (50%) of feline samples would have led to inappropriate clinical decisions. Other studies, such as Peterson et al. (2003), with results showing only 5 (5%) of samples would have led to inappropriate clinical decisions, failed to specify the thyroid function of the cats involved. It is therefore possible that their results showed few inaccuracies because they only involved cats with severe to late stage hyperthyroidism, improving the reliability of the ELISA test.
Stress caused by obtaining blood samples can result in reduced TT4 levels, while sedation has a similar effect (Kendall, 2015; Ranabir et al. 2011; Paterson, 2008). These factors should therefore be considered in studies regarding accuracy of ELISA tests. Sheldon (2009) and Cote et al. (2010) both support ELISA testing, yet neither study shows evidence suggesting they considered the effect of stress on their results.
Furthermore, some cats in Peterson et al. (2003) study had suspected thyroid disease only. The variable in this study was the method used to measure TT4 levels. In Peterson et al. (2001) 70% of cats with non-thyroidal disease had low TT4 levels. Therefore, with the cats’ condition unproven in Peterson et al. (2003) study, this is an additional variable which may affect the test results, reducing their validity in support of the reliability of the ELISA test.
Although most studies in support of ELISA testing have not adequately considered these factors above, it is unknown how significantly they affect TT4 levels. More research into the quantitative effect of such factors on TT4 levels would be beneficial for indicating how significantly they affect the studies’ validity.
2.2 Statistical Analysis
Many method-comparison studies use correlation coefficients for analysis of agreement (Bland et al. 1986). However, Bland et al. (1986) and Mukaka (2012) stated that using correlation when assessing agreement between methods of clinical assessment is misleading. While Kemppainen et al. (2006), Sheldon (2009) and Cote et al. (2010) all provide data which presents good correlation between the ELISA and other tests, their conclusions are significantly less valid as they all use correlation to compare agreement of TT4 values. However, these studies also created bias plots to analyse agreement; a method recommended by researchers (Mukaka, 2012; Bland et al. 1986). Lurye et al. (2002) used linear regression and bias plots, but not correlation, to analyse agreement, with results which do not support the ELISA test. Therefore, studies supporting ELISA testing use a less valid method of analysis than the study against ELISA testing. It is therefore tempting to believe that the Lurye et al. (2002) study has a more convincing argument against ELISA testing than those studies supporting the method.
2.3 The Importance of a Valid Methodology
2.3.1 Repeatability and Replicability
Repeatability and replicability are terms used when determining the validity of a research study (Winstanley, 2010). For a study to be replicable, the methodology must be recreatable (Winstanley, 2010). Withholding information from a methodology leaves researchers questioning why information was withheld, and the possible effect on results. Both Sheldon (2009) and Cote et al. (2010), in support of ELISA testing, present methodologies lacking information such as sample collection method, exclusion criteria and thyroid function of the cats involved, therefore reducing their validity. Lurye et al. (2002), not in support, and Kemppainen et al. (2006) in support of ELISA testing, both deliver a replicable methodology, not withholding information.
In order for a research study to claim reliability, researchers must take at least two measurements from each participant using both methods, in this case, using both the ELISA test and the comparative screening method (Bartlett et al. 2008). Positively, Lurye et al. (2002) and Kemppainen et al. (2006) carried out precision tests in their methodologies, increasing their reliability. Hanneman (2008) and Chesher (2008) state that precision is justified if a method is shown to produce the same results repeatedly. On this basis, Peterson et al. (2003), Sheldon, (2009) and Cote et al. (2010), all of whom support the ELISA test, have results which are deemed ‘meaningless’ by Hanneman (2008) and Chesher (2008) as evidence of repeatability tests are not shown. Conducting further research on ELISA testing with improved replicability and repeatability would be beneficial in order that more valid evidence for the accuracy of ELISA tests can be presented.
2.3.2 Choice of comparison method: Accuracy
Accuracy is defined as ‘the degree of which an instrument measures the real value of a variable’, and is assessed by ‘comparing the measurement method with a gold standard’ method (Hanneman, 2008). In the case of thyroid screening tests, the RIA test is considered gold standard within the field as it best measures TT4 values (Peterson, 2013). It is therefore logical for researchers to compare the ELISA test to RIA (Feldman et al. 2004), despite the fact that RIA is often not used in practice due to cost and use of radioactivity (Peterson, 2013). Positively, Lurye et al. (2002), Peterson (2003), Kemppainen et al. (2006) and Sheldon (2009) all use RIA to compare TT4 results against the ELISA test, improving the accuracy of each study. As some of the above mentioned studies support ELISA testing and some oppose it, it can be argued that both evidence for and against the routine use of ELISA tests is strong in this regard.
2.3.3 Assay performance
All 198 ELISA test samples in the study by Kemppainen et al. (2006) were performed by a veterinary student. While the veterinary student had experience on running these tests, it is possible the exacting standards may have differed to that of the laboratory technician who performed the RIA samples. In addition, performing tests on 198 samples is likely to be a monotonous and tiring task. As a result of fatigue, reduced performance standards from those running the tests could have led to inaccurate results (Rogers, 2008).
Furthermore, RIA samples in this study were mailed to the laboratory for testing (Kemppainen et al. 2006). Serum samples must be kept at optimum temperature and in plastic ampules, otherwise concentration of TT4 can be affected (Feldman et al. 2004). Kemppainen et al. (2006) provides no information on methods of transport and handling of samples, leaving it to the reader to assume that these factors were taken into account. The fact that certain important information is left to the readers’ assumption reduces the validity of the conclusion that ELISA tests are suitable for routine use in veterinary practice.
2.4 Influence of the Researchers
Peer-reviewed articles are often associated with higher quality as they have been processed through a ‘quality control filter’ by experts in the field (Novella, 2008). Therefore, it could be argued that the studies by Sheldon (2009) and Cote et al. (2010), in support of the ELISA test, lack quality as they are not peer-reviewed. Lurye et al. (2002), Peterson et al. (2003) and Kemppainen et al. (2006) on the other hand, are all peer-reviewed studies and have therefore been subject to expert criticism and recommendations of improvements (Novella, 2008). As such, it could be said that Kemppainen et al. (2006) and Peterson et al. (2003) studies supporting ELISA tests possess equal quality to Lurye et al. (2002) study against ELISA tests.
Conversely, there is evidence to support the idea that peer-reviewed research is subject to bias (Novella, 2008). For example, peer-reviewers may be biased if a study conflicts with their beliefs or criticises their own research (Novella, 2008). Information on who reviewed the research in each study is unknown, making it difficult to justify the level of reviewer bias. For this reason, the fact that some studies regarding ELISA testing have been peer-reviewed and some have not, does not necessarily make a substantial difference in regards to the validity of the studies.
2.4.2 The Researchers
All of the studies in support of ELISA testing have been written or supported by IDEXX, the company producing the ELISA tests (Cote et al. 2010; Sheldon, 2009; Kemppainen et al. 2006). It could be argued that having IDEXX participating with the research is more reliable as running of ELISA tests would have been performed to exacting standards in order to increase their chance of consistent results. Researchers independent of IDEXX may have been less aware of conditions in which the test performs best. However, research carried out by pharmaceutical companies are often heavily criticised for a lack of scientific rigour and are therefore subject to a high level of bias (Goldacre, 2012). Lexchin et al. (2003) found research by pharmaceutical companies were four times more likely to present a positive conclusion than independent studies. Consequently, supporting arguments in these three studies are significantly weakened.
Although there are fewer studies opposing the use of ELISA tests, for the reasons given above, the validity of these studies is greater than those in support of ELISA tests. For this reason, it is hard to justify the routine use of ELISA thyroid tests in UK veterinary practices at this time. If further, more valid research were carried out, the evidence to support ELISA testing would be greatly improved. More independent research is also required to get unbiased evidence either in support or against ELISA testing. In addition, it would be beneficial if up-to-date studies were carried out regularly so the latest research can be applied to veterinary practice, a fast growing industry.
Furthermore, thyroid illness is a condition which affects animals and humans. It is therefore beneficial for experts working both in human and veterinary medicine to share and create new research in order to establish the most accurate and valid screening test for thyroid illness. Until further valid and independent research is carried out to discover the reasons for conflicting results, and to provide practitioners with a more proven and accurate thyroid test, it is fair to say a holistic approach should be used, with laboratory tests, such as the ELISA test, being used as a diagnostic aid only.
Abaxis (2006) VetScan Thyroxine (T4) / Cholesterol Panel [online]. California: Abaxis. Available from: http://www.abaxis.com/pdf/T4-Cholesterol.pdf [Accessed 4 November 2015].
Baral, R. and Peterson, M.E. (2012) Thyroid Gland Disorders. In: Little, S.E., Ed. (2012) The Cat: Clinical Medicine and Management [online]. Philadelphia: Elsevier Saunders. [Accessed 13 November 2015].
Bartlett, J.W. and Frost, C. (2008) Reliability, Repeatability and Reproducibility: Analysis of Measurement Errors in Continuous Variables. Ultrasound in Obstetrics and Gynecology [online]. 31, pp. 466-475.
Bland, J.M. and Altman, D.G. (1986) Statistical Methods for Assessing Agreement between Two Methods of Clinical Measurement. The Lancet [online]. pp. 307-310. [Accessed 25 October 2015].
Bloor, C. (2013) Hyperthyroidism in Cats: What’s New for VNs and Owners? Veterinary Nursing Times. 13 (12), pp. 8–10.
Caney, S. (2009) Caring for a Cat with Hyperthyroidism. Midlotian: Cat Professional Ltd.
Chesher, D. (2008) Evaluating Assay Precision. The Clinical Biochemists Review [online]. 29 (1), pp. 23-26.
Cote, K. and Kintzer, P. (2010) IDEXX SNAPshot Dx Analyzer: SNAP T4 Performance. IDEXX Laboratories [online]. pp, 1-4.
Druce, K. (2015) Feline Hyperthyroidism Part One: Diagnosis. Veterinary Nursing Journal. 30, pp. 229-231.
Edinboro, C.H., Scott-Monocrieff, J.C., Janovitz, E., Thacker, H.L. and Glickman, L.T. (2004) Epidemiologic Study of Relationships Between Consumption of Commercial Canned Food and Risk of Hyperthyroidism in Cats. Journal of the American Veterinary Medicine Association. 224 (6), pp. 879-886.
Feldman, E. C. and Nelson, R.W. (2004) Canine and Feline Endocrinology and Reproduction [online]. 3rd ed. USA: Elsevier Science. [Accessed 4 November 2015].
Goldacre, B. (2012) Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients [online]. United Kingdom: Fourth Estate. [Accessed 5 November 2015].
Hanneman, S. (2008) Design, Analysis and Interpretation of Method-Comparison Studies. Advanced Critical Care Journal [online]. 19 (2), pp. 223-234.
Higgs, P., Murray, J.K. and Hibbert, A. (2014) Medical Management and Monitoring of the Hyperthyroid Cat: A Survey of UK General Practitioners. Journal of Feline Medicine and Surgery. 16 (10), pp. 788-795.
IDEXX Laboratories. (2015) SNAP Total T4 Test. Available from: http://www.idexx.co.uk/smallanimal/inhouse/snap/sah-snap-total-t4-test.html [Accessed 25 October 2015].
Kemppainen, R.J. and Birchfield, J.R. (2006) Measurement of Total Thyroxine Concentration in Serum from Dogs and Cats by Use of Various Methods. American Journal of Veterinary Research. 67 (2), pp. 259-265.
Kendall, K. (2015) Low Stress Methods of Collecting Cat Blood. Available from: http://www.felinefriendlycare.com/cat-blood.html [Accessed 6 November 2015].
Kooij, M., Becvarova, I., Meyer, H.P., Teske, E. and Kooistra, H.S. (2014) Effects of an Iodine Restricted Food on Client Owned Cats with Hyperthyroidism. Journal of Feline Medicine and Surgery [online]. 16 (6), pp.491-498. [Accessed 24 September 2015].
Lexchin, J., Bero, L.A., Djulbegovic, B. and Clark, O. (2003) Pharmaceutical Industry Sponsorship and Research Outcome and Quality: Systematic Review. British Medical Journal. 326, pp. 1-10.
Lurye, J.C., Behrend, E.N. and Kemppainen, R.J. (2002) Evaluation of an In-House Enzyme-Linked Immunosorbent Assay for Quantitative Measurement of Serum Total Thyroxine Concentration in Dogs and Cats. Journal of the American Veterinary Medical Association. 221 (2), pp. 243-249.
Mardell, E. (2013) Diagnosis and Management of Feline Hyperthyroidism. In Practice. 35, pp. 162-170.
Moore, P.H. (2011) Common Medicale Conditions of the Body Systems. In: Aspinall, V. (2011) The Complete Textbook of Veterinary Nursing. Oxford: Butterworth-Heinemann Elsevier. pp, 355.
Mukaka, M.M. (2012) A Guide to Appropriate Use of Correlation Coefficient in Medical Research. Malawi Medical Journal [online]. 24 (3), pp. 69-71.
Novella, S. (2008) Science Based Medicine: The Importance and Limitation of Peer-Review. Available from: https://www.sciencebasedmedicine.org/the-importance-and-limitations-of-peer-review/ [Accessed 25 October 2015].
Paterson, S. (2008) Manual of Skin Diseases of the Dog and Cat [online]. 2nd ed. Oxford: Blackwell Publishing. [Accessed 5 November 2015].
Peterson, M. (2012) Hyperthyroidism in Cats: What’s Causing this Epidemic of Thyroid Disease and Can we Prevent it? Journal of Feline Medicine and Surgery. 14, pp. 804–818.
Peterson, M. E. (2013) More than just T4: Diagnostic Testing for Hyperthyroidism in Cats. Journal of Feline Medicine and Surgery [online]. 15, pp. 765-777. [Accessed 25 September 2015].
Peterson, M.E. (2011) Treating Hyperthyroid Cats with an Iodine Deficient Diet (Hills y/d): Does it Really Work? Available from: http://www.endocrinevet.info/2011/09/treating-hyperthyroid-cats-with-iodine.html [Accessed 13 November 2015].
Peterson, M.E. (2015) Nutritional Management of Feline Hyperthyroidism: What’s the Best Diet to feed these Cats? Available from: http://www.animalendocrine.com/wp-content/uploads/2011/01/Nutritional-Management-of-Feline-Hyperthyroidism-VMA-NYC.pdf [Accessed 13 November 2015]
Peterson, M.E., Demarco, C.L. and Sheldon, K.M. (2003) Total Thyroxine Testing: Comparison of an In-House Test Kit with Radioimmuno and Chemiluminescent Assays. American College of Veterinary Internal Medicine [online]. pp, 396.
Peterson, M.E., Melian, C. and Nichols, R. (2001) Measurement of Serum Concentrations of Free Thyroxine, and Total Triiodothyronine in Cats with Hyperthyroidism and Cats with Non-Thyroidal Disease. Journal of the American Veterinary Medical Association [online]. 218 , pp. 529-536. [Accessed 5 October 2015].
Plotnick, A. (2015) Thyroid Disease and Cats. Available from: https://www.petfinder.com/cats/cat-health/cat-thyroid-disease/ [Accessed 13 November 2015].
Ranabir, S. and Reetu, K. (2011) Stress and Hormones. Indian Journal of Endocrinology and Metabolism [online]. 15 (1), pp. 18-22. [Accessed 25 October 2015].
Rogers, A.E. (2008) The Effects of Fatigue and Sleepiness on Nurse Performance and Patient Safety. In: Hughes, R.G., Ed. (2011) Patient Safety and Quality: An Evidence-Based Handbook for Nurses [online]. USA: Agency for Healthcare Research and Quality, Robert Wood Johnson Foundation. [Accessed 6 November 2015].
Sheldon, K. (2009) IDEXX SNAPshot Dx Analyzer: SNAP T4 Performance. IDEXX Laboratories [online]. pp, 1-4.
Skinner, N.D. (1998) Thyroid Hormone Levels in Cats: Colony Average and the Decrease with Age. American Society for Nutritional Sciences [online]. 128, pp. 2636-2638. [Accessed 13 November 2015].
Winstanley, C. (2010) Writing a Dissertation for Dummies. Chichester: John Wiley and Sons.