Next Article in Journal
Role of CD123 (+) Plasmacytoid Dendritic Cells in Etiologically Different Variants of Erythema Multiforme: A Monocentric Retrospective Study
Previous Article in Journal
Modulation of Gene Expression in a Sterile Atopic Dermatitis Model and Inhibition of Staphylococcus aureus Adhesion by Fucoidan
Article

Inter-Observer and Intra-Observer Variations in the Assessment of Epithelial Dysplasia in Oral Lichenoid Diseases

1
Department of Dermatology, Inselspital Bern University Hospital, University of Bern, CH-3010 Bern, Switzerland
2
Department of Dermatology, Andrology and STDs, Mansoura University Hospitals, Mansoura University, Mansoura 35516, Egypt
3
Centro Studi GISED, 24121 Bergamo, Italy
4
Department of Oral Surgery and Stomatology, School of Dental Medicine, University of Bern, CH-3010 Bern, Switzerland
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Academic Editor: Gürkan Kaya
Received: 25 February 2021 / Revised: 18 March 2021 / Accepted: 26 March 2021 / Published: 1 April 2021
(This article belongs to the Section Clinico-Pathological Correlation in Dermatopathology)

Abstract

Oral lichen planus (OLP) and oral lichenoid lesions (OLL) can both present with histological dysplasia. Despite the presence of WHO-defined criteria for the evaluation of epithelial dysplasia, its assessment is frequently subjective (inter-observer variability). The lack of reproducibility in the evaluation of dysplasia is even more complex in the presence of a lichenoid inflammation. We evaluated dysplasia in 112 oral biopsies with lichenoid inflammation in order to study the inter-observer and the intra-observer variability.
Keywords: oral lichen planus; oral lichenoid lesion; oral lichenoid disease; dysplasia; inter-observer; intra-observer oral lichen planus; oral lichenoid lesion; oral lichenoid disease; dysplasia; inter-observer; intra-observer

1. Introduction

Oral lichen planus (OLP) is the oral manifestation of lichen planus, a common inflammatory dermatologic condition involving the skin, hair follicles, nails, and mucous membranes [1]. The diagnosis is based on the combination of clinical and histopathological criteria; the latter include a band-like subepithelial inflammation with the hydropic degeneration of the basal keratinocytes. Twenty percent of patients with an OLP also have skin lesions [2], and these are very important to distinguish OLP from oral lichenoid lesions (OLL). OLL are a group of diseases characterized clinically by a stomatitis, and histologically by a lichenoid reaction similar to OLP, but typically associated with an attributable etiology [1]: allergic lichenoid contact mucositis, oral lichenoid drug reactions, lichenoid graft versus host disease, and other lichenoid reactions [3,4]. OLP and OLL can present clinically with plaque formation, atrophy, erosions or bullae.
Despite the presence of WHO-defined criteria to evaluate epithelial dysplasia—such as the presence of several architectural and cellular changes—their assessment is frequently subjective (inter-observer variability), and is therefore inconsistent [5,6,7,8,9]. Moreover, the significance of these alterations may also receive a subjective interpretation from different experts: some see them as reactive changes, other as a true sign of (pre-)malignancy [5,6]. There is also an intra-observer variability, in which the same pathologist may have a different opinion about the presence of dysplasia assessed at two different times [7]. The lack of reproducibility in the evaluation of dysplasia is even more complex in the presence of a lichenoid inflammation, when the keratinocytes show reactive alterations which may appear as dysplastic changes. However, we did not find any study about intra- and inter-observer variability in the specific assessment of dysplasia in OLP in comparison to OLL. We therefore evaluated dysplasia in 112 oral biopsies with lichenoid inflammation in order to study the inter-observer and the intra-observer variability.

2. Materials and Methods

In a monocentric, retrospective, interdisciplinary study (dermatology, dental medicine, and dermatopathology), four board-certified dermatopathologists who routinely sign out oral mucosal biopsies for the evaluation of dysplasia evaluated the grade of epithelial dysplasia in 112 biopsies with an oral lichenoid inflammation (all were patients with a confirmed diagnosis of OLP or OLL). All of the consecutive cases were retrieved from the archives of our tertiary referral center during the 34-month study period. The clinical and histological criteria to differentiate OLP from OLL were described in a previous study [10]. Each of the four dermatopathologists evaluated, on one selected slide, the grade of dysplasia (0 = no dysplasia, 1 = mild, 2 = moderate, 3 = severe dysplasia = carcinoma in situ), without information about the original histopathological assessment, clinical data, or patient identity. The dysplasia was diagnosed and graded according to the latest WHO classification, and included the changes in the basal (grade 1) and suprabasal (grade 2) epithelium, and in the entire thickness of the epithelium in grade 3. The examined criteria were architectural and cellular changes [11,12]. The same histological evaluation of dysplasia was repeated, blinded, three months later.
We statistically assessed the intra- and the inter-observer variation of this evaluation according to Cohen’s kappa, with quadratic weights to account for the ordered scores. For the inter-observer reliability, the weighted kappa was computed based on two-way random, single measures intra-class correlation. The kappa was reported along with its 95% confidence interval (CI), and can be interpreted as follows: <0, poor; 0–0.20, slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and >0.80, almost perfect agreement. The comparison between the OLL and OLP patients was based on the median values of all of the examiners’ assessments, and was tested by using a Mann–Whitney U test. All of the tests were considered significant at a p value < 0.05. The analyses were performed in SPSS v.26 (IBM Corp. Armonk, NY, USA).

3. Results

In a previous study, we clinically identified 84 patients with the criteria of OLP, and 28 with OLL [10]. We found a great variability of the interpretation of dysplasia, with a low inter-observer reliability among the four examiners. The kappa was rated very low, measuring 0.05 (95% CI: 0.01, 0.13) and 0.11 (0, 0.25), respectively, in the two repeated assessments. The intra-observer reliability varied among the examiners: the kappa was 0.34 (0.14, 0.54) and 0.36 (0.19, 0.53) (fair agreement) for two examiners, 0.52 (0.30, 0.75) for another examiner (moderate), and 0.65 (0.50, 0.80) for the last examiner (substantial). Despite the variability of the assessment of dysplasia between the experts, the degree of dysplasia was significantly higher in the OLL than the OLP cases (p = 0.03) (Figure 1 and Table 1).
Analyzing the amount of variability in the evaluation of the grade of dysplasia in OLP and OLL, we found a significant inconsistency among the experts (inter-observer variations), and in three of four experts a significant intra-observer discrepancy (poor to moderate agreement) when examining the same slide at two different times. One of the four experts consistently recorded a higher dysplasia grade compared to the other three, and this persisted in the second evaluation. Experts A, B, and C never (but in one case) evaluated dysplasia as grade 3, whereas expert D evaluated dysplasia as grade 3 in 24 (29%) of the cases. Moreover, experts A, B, and C did not record any dysplasia in 66–88% of the cases, but the expert D found some dysplasia in all of the cases. Furthermore, experts A, B, and C found a moderate dysplasia (grade 2) in 1–4% of the cases, but expert D saw it in 61% of the cases (Table 2). The intra-observer variability did not vary according to the number of years in practice. Despite these inter-observer variations, the degree of dysplasia was significantly higher in the OLL than the OLP cases, as was previously demonstrated in other studies [13].

4. Discussion

Both OLP and OLL may histologically show some morphological reactive changes of the keratinocytes, which may be interpreted as dysplasia, but this interpretation is a controversial, debated issue among the experts. Krutchkoff and Eisenberg [14] proposed a list of histologic criteria for the so called ‘lichenoid dysplasia’, a lesion with the histopathologic characteristics of OLP with additional presence of dysplasia. This term was not adopted by other authors; in fact, according to van der Meij and van der Waal, the absence of epithelial dysplasia should be required for the histological diagnosis of OLP [5]. Furthermore, cases with a true epithelial dysplasia and an additional lichenoid reaction (a reactive lichenoid inflammation due to the neoplasia) may bring some additional confusion [5]. Finally, from the pathophysiological point of view, it is well known that in a cytotoxic tissue reaction like a lichenoid dermatitis/stomatitis, some epithelial cells may appear atypical/dysplastic [15].
In the literature, there are some examples of inter-observer and/or intra-observer variation in the histological evaluation of oral dysplasia [6,7,8,9,16,17]. Most authors conclude that the potential bias induced by the subjectivity of the examiner can only be reduced with the help of clearer diagnostic criteria, and with a systematic histological evaluation. An interesting solution is the use of a binary system with low/high risk lesions, as it is applied in the evaluation of female and male intra-epithelial neoplasia of the genital area. Küffer and Lombardi 2002 proposed the term of oral intraepithelial neoplasia, with a grading into low and high grade [18]. Taking into consideration the results of this study, we strongly encourage other specialists dealing with the evaluation of dysplasia in oral lichenoid diseases to use a binary system. Ideally, all of the specialists dealing with patients with an oral lichenoid disease should discuss together the therapeutic consequences of the currently-used histological analysis [8,9].

5. Conclusions

In conclusion, we demonstrated that, in OLL, there is significantly more epithelial dysplasia than in OLP. We also showed that the grading of epithelial dysplasia in OLP/OLL is partly subjective, showing both significant inter-observer and intra-observer variations. In order to avoid this confusing variability, we strongly encourage specialists to use a binary system.

Author Contributions

Conceptualization, H.B. and L.F.; methodology, H.B. and L.F.; formal analysis, H.B. and S.C.; statistics, S.C.; investigation, H.N., R.B., V.G.A.S., L.F., H.B.; writing—original draft preparation, M.Z.; writing—review and editing, H.B., L.F. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and reviewed and approved by the ethics committee of the Canton of Bern (KEK-192/10).

Informed Consent Statement

Patient consent was waived due to the retrospective descriptive study design.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Schlosser, B.J. Lichen planus and lichenoid reactions of the oral mucosa. Dermatol. Ther. 2010, 23, 251–267. [Google Scholar] [CrossRef] [PubMed]
  2. Silverman, S.; Gorsky, M.; Lozada-Nur, F. A prospective follow-up study of 570 patients with oral lichen planus: Persistence, remission, and malignant association. Oral Surg. Oral Med. Oral Pathol. 1985, 60, 30–34. [Google Scholar] [CrossRef]
  3. Van den Haute, V.; Antoine, J.L.; Lachapelle, J.M. Histopathological discriminant criteria between lichenoid drug eruption and idiopathic lichen planus: Retrospective study on selected samples. Dermatologica 1989, 179, 10–13. [Google Scholar] [CrossRef] [PubMed]
  4. Rotim, Z.; Bolanca, Z.; Rogulj, A.A.; Andabak, M.; Boras, V.V.; Vrdoljak, D.V. Oral lichen planus and oral lichenoid reaction—An update. Acta Clin. Croat. 2015, 54, 516–520. [Google Scholar] [PubMed]
  5. Van der Meij, E.H.; van der Waal, I. Lack of clinicopathologic correlation in the diagnosis of oral lichen planus based on the presently available diagnostic criteria and suggestions for modifications. J. Oral Pathol. Med. 2003, 32, 507–512. [Google Scholar] [CrossRef] [PubMed]
  6. Warnakulasuriya, S.; Reibel, J.; Bouquot, J.; Dabelsteen, E. Oral epithelial dysplasia classification systems: Predictive value, utility, weaknesses and scope for improvement. J. Oral Pathol. Med. 2008, 37, 127–133. [Google Scholar] [CrossRef] [PubMed]
  7. Van der Meij, E.H.; Reibel, J.; Slootweg, P.J.; van der Wal, J.E.; de Jong, W.F.; van der Waal, I. Interobserver and intraobserver variability in the histologic assessment of oral lichen planus. J. Oral Pathol. Med. 1999, 28, 274–277. [Google Scholar] [CrossRef]
  8. Kujan, O.; Khattab, A.; Oliver, R.J.; Roberts, S.A.; Thakker, N.; Sloan, P. Why oral histopathology suffers inter-observer variability on grading oral epithelial dysplasia: An attempt to understand the sources of variation. Oral Oncol. 2007, 43, 224–231. [Google Scholar] [CrossRef] [PubMed]
  9. Speight, P.M.; Abram, T.J.; Floriano, P.N.; James, R.; Vick, J.; Thornhill, M.H.; Murdoch, C.; Freeman, C.; Hegarty, A.M.; D’Apice, K.; et al. Interobserver agreement in dysplasia grading: Toward an enhanced gold standard for clinical pathology trials. Oral Surg. Oral Med. Oral Pathol. Oral Radiol. 2015, 120, 474–482.e2. [Google Scholar] [CrossRef] [PubMed]
  10. Feldmeyer, L.; Suter, V.; Oeschger, C.; Cazzaniga, S.; Bornstein, M.; Simon, D.; Borradori, L.; Beltraminelli, H. Oral lichen planus and oral lichenoid lesions—An analysis of clinical and histopathological features. J. Eur. Acad. Dermatol. Venereol. 2020, 34, e104–e107. [Google Scholar] [CrossRef] [PubMed]
  11. El-Naggar, A.; Chan, J.; Grandis, J.; Takata, T.; Slootweg, P. WHO Classification of Head and Neck Tumours [Internet]. Available online: https://publications.iarc.fr/Book-And-Report-Series/Who-Classification-Of-Tumours/WHO-Classification-Of-Head-And-Neck-Tumours-2017 (accessed on 10 February 2021).
  12. Ranganathan, K.; Kavitha, L. Oral epithelial dysplasia: Classifications and clinical relevance in risk assessment of oral potentially malignant disorders. J. Oral Maxillofac. Pathol. 2019, 23, 19–27. [Google Scholar] [CrossRef] [PubMed]
  13. Van der Meij, E.H.; Mast, H.; van der Waal, I. The possible premalignant character of oral lichen planus and oral lichenoid lesions: A prospective five-year follow-up study of 192 patients. Oral Oncol. 2007, 43, 742–748. [Google Scholar] [CrossRef] [PubMed]
  14. Krutchkoff, D.J.; Eisenberg, E. Lichenoid dysplasia: A distinct histopathologic entity. Oral Surg. Oral Med. Oral Pathol. 1985, 60, 308–315. [Google Scholar] [CrossRef]
  15. Lodi, G.; Scully, C.; Carrozzo, M.; Griffiths, M.; Sugerman, P.B.; Thongprasom, K. Current controversies in oral lichen planus: Report of an international consensus meeting. Part 1. Viral infections and etiopathogenesis. Oral Surg. Oral Med. Oral Pathol. Oral Radiol. Endod. 2005, 100, 40–51. [Google Scholar] [CrossRef] [PubMed]
  16. Ranganathan, K.; Kavitha, L.; Sharada, P.; Bavle, R.M.; Rao, R.S.; Pattanshetty, S.M.; Hazarey, V.K.; Madhura, M.G.; Nagaraj, T.; Lingappa, A.; et al. Intra-observer and inter-observer variability in two grading systems for oral epithelial dysplasia: A multi-centre study in India. J. Oral Pathol. Med. 2020, 49, 948–955. [Google Scholar] [CrossRef] [PubMed]
  17. Kujan, O.; Oliver, R.J.; Khattab, A.; Roberts, S.A.; Thakker, N.; Sloan, P. Evaluation of a new binary system of grading oral epithelial dysplasia for prediction of malignant transformation. Oral Oncol. 2006, 42, 987–993. [Google Scholar] [CrossRef]
  18. Küffer, R.; Lombardi, T. Premalignant lesions of the oral mucosa. A discussion about the place of oral intraepithelial neoplasia (OIN). Oral Oncol. 2002, 38, 125–130. [Google Scholar] [CrossRef]
Figure 1. Pictures of an OLL, with magnification on different zones of the same lesion. (a) Overview, (b) detail of an area with no atypia, (c) detail of an area with the presence of atypical keratinocytes. The dysplasia grading among the four experts for the whole slide was as follows (first evaluation/second evaluation at 3 months. Expert 1: dysplasia grading 0/2; Expert 2: dysplasia grading 1/2; Expert 3: dysplasia grading 2/2; Expert 4: dysplasia grading 2/3. Dysplasia grading: 0 = none, 1 = mild, 2 = moderate, 3 = severe.
Figure 1. Pictures of an OLL, with magnification on different zones of the same lesion. (a) Overview, (b) detail of an area with no atypia, (c) detail of an area with the presence of atypical keratinocytes. The dysplasia grading among the four experts for the whole slide was as follows (first evaluation/second evaluation at 3 months. Expert 1: dysplasia grading 0/2; Expert 2: dysplasia grading 1/2; Expert 3: dysplasia grading 2/2; Expert 4: dysplasia grading 2/3. Dysplasia grading: 0 = none, 1 = mild, 2 = moderate, 3 = severe.
Dermatopathology 08 00013 g001
Table 1. Intra-rater reliability of the dysplasia assessment for each rater, and overall.
Table 1. Intra-rater reliability of the dysplasia assessment for each rater, and overall.
First AssessmentSecond AssessmentWeighted Kappa *
(95% CI)
NoneIIIIII
NRow%NRow%NRow%NRow%
Rater: Pathologist 1None6573.0%2427.0%00.0%00.0%0.36 (0.19, 0.53)
I731.8%1463.6%14.5%00.0%
II00.0%1100.0%00.0%00.0%
III00.0%00.0%00.0%00.0%
Rater: Pathologist 2None6875.6%1516.7%77.8%00.0%0.34 (0.14, 0.54)
I735.0%945.0%420.0%00.0%
II00.0%150.0%00.0%150.0%
III00.0%00.0%00.0%00.0%
Rater: Pathologist 3None8494.4%55.6%00.0%00.0%0.52 (0.30, 0.75)
I1473.7%210.5%315.8%00.0%
II125.0%125.0%125.0%125.0%
III00.0%00.0%00.0%00.0%
Rater: Pathologist 4None00.0%00.0%00.0%00.0%0.65 (0.50, 0.80)
I00.0%738.9%1055.6%15.6%
II00.0%45.9%5783.8%710.3%
III00.0%13.8%13.8%2492.3%
CI: confidence interval; * weighted kappa with quadratic weights.
Table 2. Inter-rater reliability of the dysplasia assessment.
Table 2. Inter-rater reliability of the dysplasia assessment.
Dysplasia AssessementPath. 1Path. 2Path. 3Path. 4Weighted Kappa *
(95% CI)
Weighted Kappa
(w/o Path. 4) (95% CI)
NCol%NCol%NCol%NCol%
First dysplasia
assessment
None8979.5%9080.4%8979.5%00.0%0.05 (−0.005, 0.13) 0.13 (0.02, 0.25)
I2219.6%2017.9%1917.0%1816.1%
II10.9%21.8%43.6%6860.7%
III00.0%00.0%00.0%2623.2%
Second dysplasia
assessment
None7264.3%7567.0%9988.4%00.0%0.11 (0.002, 0.25)0.37 (0.25, 0.49)
I3934.8%2522.3%87.1%1210.7%
II10.9%119.8%43.6%6860.7%
III00.0%10.9%10.9%3228.6%
* Weighted kappa with quadratic weights computed based on two-way random, single measures, absolute agreement intra-class correlation. Path.: pathologist.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Back to TopTop