Assessing the inter-rater reliability for nominal, categorical and ordinal data in medical sciences

Background: Inter-rater Reliability (IRR) measures the reliability of raters. The number of raters, the type of variable evaluated, and some other constraints have a direct impact on the use of this tool. Failure to pay attention to these points will increase the error and will harm the results of the research. Materials and Methods: The present study is an analytical one. The samples of this research were randomly selected from students and interns of faculties and hospitals of Mazandaran University of Medical Sciences. The samples of this research were selected from Sari Faculty of Health and Sari Accident and Burn hospital. Results: This study shows how the agreement between the two raters is calculated concerning a nominal variable with and without the factor of chance. Besides, by understanding the amount of agreement between the two raters in the presence of an ordinal variable, we will become acquainted with the calculation of the agreement between more than two raters. Discussion: Some limitations, such as the quantitative and qualitative variables, the number of raters, the existence of missing data directly affects the size of the IRR; according to these points, the appropriate IRR for data to measure the agreement between raters is used.