Fuzzy classification of dichotomous test items and social indicators differentiation property
Abstract
In many fields of human activities such as economics, sustainable development, construction, human resources management etc., dichotomous tests are employed to measure some observed property, for example knowledge level in a specific field or applicant’s eligibility for a job position. Fuzzy classification method for dichotomous test items is proposed in this paper. Depending on the observed property, each test item may well differentiate all testees or only the testees who are strong or weak at that property. Also, the test item may badly differentiate all testees and be inappropriate for that purpose. The method presented in the paper may be applied for small groups of testees with known estimates of the investigated property, for example raw test scores. The proposed method for dichotomous test item classification is based on the fuzzy set theory. Though the tests were originally constructed for knowledge measurement, their mathematical models can be applied for social indicators and wide range of other areas.
Keyword : mathematical modelling, fuzzy sets, dichotomous tests, social indicators, least squares method
How to Cite
Krylovas, A., Kosareva, N., & Karaliūnaitė, J. (2018). Fuzzy classification of dichotomous test items and social indicators differentiation property. Technological and Economic Development of Economy, 24(4), 1755-1775. https://doi.org/10.3846/tede.2018.5213
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Ames, A. J., & Samonte, K. (2015). Using SAS PROC MCMC for Item Response Theory Models. Educational and Psychological Measurement, 75(4), 585-609. https://doi.org/10.1177/0013164414551411
Anderson, J. A. (2004). Discrete mathematics mathematics with combinatorics (2nd ed.). University of South Carolina – Spartanburg, Prentice Hall, New Jersey.
Andreis, F., & Ferrari, P. A. (2014). Multidimensional item response theory models for dichotomous data in customer satisfaction evaluation. Journal of Applied Statistics, 41(9), 2044-2055. https://doi.org/10.1080/02664763.2014.907395
Bretscher, O. (1995). Linear algebra with applications (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
Chen, C.-M., & Duh, L.-J. (2008). Personalized web-based tutoring system based on fuzzy item response theory. Expert Systems with Applications, 34(4), 2298-2315. https://doi.org/10.1016/j.eswa.2007.03.010
Dadelo, S., Turskis, Z., Zavadskas, E. K., Kačerauskas, T., & Dadelienė, R. (2016). Is the evaluation of the students’ values possible? An integrated approach to determining the weights of students’ personal goals using multiple-criteria methods. EURASIA Journal of Mathematics, Science and Technology Education, 12(11), 2771-2781. https://doi.org/10.12973/eurasia.2016.02303a
De Champlain, A. F., Boulais, A.-P., & Dallas, A. (2016). Calibrating the Medical Council of Canada’s Qualifying Examination Part I using an integrated item response theory framework: a comparison of models and designs. Journal of Educational Evaluation for Health Professions, 13(6). https://doi.org/10.3352/jeehp.2016.13.6
Finn, J. A., Ben-Porath, Y. S., & Tellegen, A. (2015). Dichotomous Versus Polytomous response options in psychopathology assessment: Method or meaningful variance?, Psychological Assessment, 27(1), 184-193. https://doi.org/10.1037/pas0000044
Ghorabaee, M. K., Amiri, M, Zavadskas, E. K., & Antucheviciene, J. (2017). Supplier evaluation and selection in fuzzy environments: A review of MADM approaches. Economic Research-Ekonomska Istraživanja 30(1), 1073-1118. https://doi.org/10.1080/1331677X.2017.1314828
Hambleton, R. K., Swaminathan, H., & Rogers, J. H. (1991). Fundamentals of item response theory. New York: Sage Publications.
Huang, H.-Y. (2015). A Multilevel higher order item response theory model for measuring latent growth in longitudinal data. Applied Psychological Measurement, 39(5), 362-372. https://doi.org/10.1177/0146621614568112
Huang, J.-H., & Peng, K.-H. (2012). Fuzzy Rasch model in TOPSIS: A new approach for generating fuzzy numbers to assess the competitiveness of the tourism industries in Asian countries. Tourism Management 33(2), 456-465. https://doi.org/10.1016/j.tourman.2011.05.006
Jin, K.-Y., & Wang, W.-C. (2014). Item response theory models for performance decline during testing. Journal of Educational Measurement, 51(2), 178-200. https://doi.org/10.1111/jedm.12041
Kohli, N., Koran, J., & Henn, L. (2015). Relationships among classical test theory and item response theory frameworks via factor analytic models. Educational and Psychological Measurement, 75(3), 389-405. https://doi.org/10.1177/0013164414559071
Kosareva, N., & Krylovas, A. (2011). A numerical experiment on mathematical model of forecasting the results of knowledge testing. Technological and Economic Development of Economy, 17(1), 42-61. https://doi.org/10.3846/13928619.2011.553994
Krylovas, A., & Kosareva, N. (2008). Mathematical modelling of forecasting the results of knowledge testing. Technological and Economic Development of Economy, 14(3), 388-401. https://doi.org/10.3846/1392-8619.2008.14.388-401
Krylovas, A., & Kosareva, N. (2011). Item response theory applications for social phenomena modeling. Societal Studies, 3(1), 77-93.
Krylovas, A., Kosareva, N., & Navickienė, O. (2013). Economic and social phenomena indicators design Methodology based on averaging values of dichotomous operators. In 3rd International Scientific Conference Whither Our Economies, 24–25 October 2013, Vilnius, Lithuania.
Lee, C.-S., Wang, M.-H., Lin, K.-H., Yang, S.-C., & Lin, T.-T. (2016). Genetic Fuzzy markup language-based item response theory agent for online self-learning platform construction. In IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 24–29 July 2016, Vancouver, Canada. https://doi.org/10.1109/FUZZ-IEEE.2016.7737806
Lee, W.-C. (2010). Classification consistency and accuracy for complex assessments using item response theory. Journal of Educational Measurement, 47(1), 1-17. https://doi.org/10.1111/j.1745-3984.2009.00096.x
Levy, R., Mislevy, R. J., & Sinharay, S. (2009). Posterior predictive model checking for multidimensionality in item response theory. Applied Psychological Measurement, 33(7), 519-537. https://doi.org/10.1177/0146621608329504
Liu, P., Li, Y., & Antuchevičienė, J. (2016). Multi-criteria decision-making method based on intuitionistic trapezoidal fuzzy prioritised OWA operator. Technological and Economic Development of Economy, 22(3), 453-469. https://doi.org/10.3846/20294913.2016.1171262
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley Publishing Company.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Mardani, A., Zavadskas, E. K., Khalifah, Z., Zakuan, N., Jusoh, A., Md Nor, K., & Khoshnoudic, M. (2017). A review of multi-criteria decision-making applications to solve energy management problems: Two decades from 1995 to 2015. Renewable and Sustainable Energy Reviews 71: 216-256. https://doi.org/10.1016/j.rser.2016.12.053
Noventa, S., Stefanutti, L., & Vidotto, G. (2014). An analysis of item response theory and Rasch models based on the most probable distribution method. Psychometrica, 79(3), 377-402. https://doi.org/10.1007/s11336-013-9348-y
Paek, I., & Han, K. T. (2013). IRTPRO 2.1 for Windows (Item response theory for patient–reported outcomes). Applied Psychological Measurement, 37(3), 242-252. https://doi.org/10.1177/0146621612468223
Raileanu, M. S. (2008). Exploring the extension of item response theory models to the economic and social measurement. Proceedings of the 12th WSEAS International Conference on COMPUTERS, 23–25 July 2008, Heraklion, Greece.
Rasch, G. (1960). Probabilistic Models for some Intelligence and Attainment Tests. Copenhagen: Danish Institute for Educational Research. Expanded edition: 1980, Chicago: The University of Chicago Press. 199 p.
Szeles, M. R., & Fusco, A. (2013). Item response theory and the measurement of deprivation: evidence from Luxembourg data. Quality & Quantity, 47(3), 1545-1560. https://doi.org/10.1007/s11135-011-9607-x
Tay, L., & Drasgow, F. (2012). Adjusting the Adjusted chi(2)/df Ratio Statistic for Dichotomous Item Response Theory Analyses: Does the Model Fit?. Educational and Psychological Measurement, 72(3), 510-527. https://doi.org/10.1177/0013164411416976
Toribio, S. G., & Albert, J. H. (2011). Discrepancy measures for item fit analysis in item response theory. Journal of Statistical Computation and Simulation, 81(10): 1345-1360. https://doi.org/10.1080/00949655.2010.4851311
Vetterlein, T., & Zamansky, A. (2016). Reasoning with graded information: The case of diagnostic rating scales in healthcare. Fuzzy Sets and Systems, 298, 207-221. https://doi.org/10.1016/j.fss.2015.11.002
Weissman, A. (2013). Optimizing information using the EM algorithm in item response theory. Annals of Operations Research, 206(1), 627-646. https://doi.org/10.1007/s10479-012-1204-4
Wong, C. C. (2015). Asymptotic standard errors for item response theory true score equating of polytomous items. Journal of Educational Measurement, 52(1), 106-120. https://doi.org/10.1111/jedm.12065
Zadeh, L. A. (1965) Fuzzy sets. Information and Control 8(3), 338–353. https://doi.org/10.1016/S0019-9958(65)90241-X
Zimmermann, H. J. (2001). Fuzzy set theory – and its applications (4th ed.). Kluwer Academinc Publisher. https://doi.org/10.1007/978-94-010-0646-0
Anderson, J. A. (2004). Discrete mathematics mathematics with combinatorics (2nd ed.). University of South Carolina – Spartanburg, Prentice Hall, New Jersey.
Andreis, F., & Ferrari, P. A. (2014). Multidimensional item response theory models for dichotomous data in customer satisfaction evaluation. Journal of Applied Statistics, 41(9), 2044-2055. https://doi.org/10.1080/02664763.2014.907395
Bretscher, O. (1995). Linear algebra with applications (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
Chen, C.-M., & Duh, L.-J. (2008). Personalized web-based tutoring system based on fuzzy item response theory. Expert Systems with Applications, 34(4), 2298-2315. https://doi.org/10.1016/j.eswa.2007.03.010
Dadelo, S., Turskis, Z., Zavadskas, E. K., Kačerauskas, T., & Dadelienė, R. (2016). Is the evaluation of the students’ values possible? An integrated approach to determining the weights of students’ personal goals using multiple-criteria methods. EURASIA Journal of Mathematics, Science and Technology Education, 12(11), 2771-2781. https://doi.org/10.12973/eurasia.2016.02303a
De Champlain, A. F., Boulais, A.-P., & Dallas, A. (2016). Calibrating the Medical Council of Canada’s Qualifying Examination Part I using an integrated item response theory framework: a comparison of models and designs. Journal of Educational Evaluation for Health Professions, 13(6). https://doi.org/10.3352/jeehp.2016.13.6
Finn, J. A., Ben-Porath, Y. S., & Tellegen, A. (2015). Dichotomous Versus Polytomous response options in psychopathology assessment: Method or meaningful variance?, Psychological Assessment, 27(1), 184-193. https://doi.org/10.1037/pas0000044
Ghorabaee, M. K., Amiri, M, Zavadskas, E. K., & Antucheviciene, J. (2017). Supplier evaluation and selection in fuzzy environments: A review of MADM approaches. Economic Research-Ekonomska Istraživanja 30(1), 1073-1118. https://doi.org/10.1080/1331677X.2017.1314828
Hambleton, R. K., Swaminathan, H., & Rogers, J. H. (1991). Fundamentals of item response theory. New York: Sage Publications.
Huang, H.-Y. (2015). A Multilevel higher order item response theory model for measuring latent growth in longitudinal data. Applied Psychological Measurement, 39(5), 362-372. https://doi.org/10.1177/0146621614568112
Huang, J.-H., & Peng, K.-H. (2012). Fuzzy Rasch model in TOPSIS: A new approach for generating fuzzy numbers to assess the competitiveness of the tourism industries in Asian countries. Tourism Management 33(2), 456-465. https://doi.org/10.1016/j.tourman.2011.05.006
Jin, K.-Y., & Wang, W.-C. (2014). Item response theory models for performance decline during testing. Journal of Educational Measurement, 51(2), 178-200. https://doi.org/10.1111/jedm.12041
Kohli, N., Koran, J., & Henn, L. (2015). Relationships among classical test theory and item response theory frameworks via factor analytic models. Educational and Psychological Measurement, 75(3), 389-405. https://doi.org/10.1177/0013164414559071
Kosareva, N., & Krylovas, A. (2011). A numerical experiment on mathematical model of forecasting the results of knowledge testing. Technological and Economic Development of Economy, 17(1), 42-61. https://doi.org/10.3846/13928619.2011.553994
Krylovas, A., & Kosareva, N. (2008). Mathematical modelling of forecasting the results of knowledge testing. Technological and Economic Development of Economy, 14(3), 388-401. https://doi.org/10.3846/1392-8619.2008.14.388-401
Krylovas, A., & Kosareva, N. (2011). Item response theory applications for social phenomena modeling. Societal Studies, 3(1), 77-93.
Krylovas, A., Kosareva, N., & Navickienė, O. (2013). Economic and social phenomena indicators design Methodology based on averaging values of dichotomous operators. In 3rd International Scientific Conference Whither Our Economies, 24–25 October 2013, Vilnius, Lithuania.
Lee, C.-S., Wang, M.-H., Lin, K.-H., Yang, S.-C., & Lin, T.-T. (2016). Genetic Fuzzy markup language-based item response theory agent for online self-learning platform construction. In IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 24–29 July 2016, Vancouver, Canada. https://doi.org/10.1109/FUZZ-IEEE.2016.7737806
Lee, W.-C. (2010). Classification consistency and accuracy for complex assessments using item response theory. Journal of Educational Measurement, 47(1), 1-17. https://doi.org/10.1111/j.1745-3984.2009.00096.x
Levy, R., Mislevy, R. J., & Sinharay, S. (2009). Posterior predictive model checking for multidimensionality in item response theory. Applied Psychological Measurement, 33(7), 519-537. https://doi.org/10.1177/0146621608329504
Liu, P., Li, Y., & Antuchevičienė, J. (2016). Multi-criteria decision-making method based on intuitionistic trapezoidal fuzzy prioritised OWA operator. Technological and Economic Development of Economy, 22(3), 453-469. https://doi.org/10.3846/20294913.2016.1171262
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading MA: Addison-Wesley Publishing Company.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Mardani, A., Zavadskas, E. K., Khalifah, Z., Zakuan, N., Jusoh, A., Md Nor, K., & Khoshnoudic, M. (2017). A review of multi-criteria decision-making applications to solve energy management problems: Two decades from 1995 to 2015. Renewable and Sustainable Energy Reviews 71: 216-256. https://doi.org/10.1016/j.rser.2016.12.053
Noventa, S., Stefanutti, L., & Vidotto, G. (2014). An analysis of item response theory and Rasch models based on the most probable distribution method. Psychometrica, 79(3), 377-402. https://doi.org/10.1007/s11336-013-9348-y
Paek, I., & Han, K. T. (2013). IRTPRO 2.1 for Windows (Item response theory for patient–reported outcomes). Applied Psychological Measurement, 37(3), 242-252. https://doi.org/10.1177/0146621612468223
Raileanu, M. S. (2008). Exploring the extension of item response theory models to the economic and social measurement. Proceedings of the 12th WSEAS International Conference on COMPUTERS, 23–25 July 2008, Heraklion, Greece.
Rasch, G. (1960). Probabilistic Models for some Intelligence and Attainment Tests. Copenhagen: Danish Institute for Educational Research. Expanded edition: 1980, Chicago: The University of Chicago Press. 199 p.
Szeles, M. R., & Fusco, A. (2013). Item response theory and the measurement of deprivation: evidence from Luxembourg data. Quality & Quantity, 47(3), 1545-1560. https://doi.org/10.1007/s11135-011-9607-x
Tay, L., & Drasgow, F. (2012). Adjusting the Adjusted chi(2)/df Ratio Statistic for Dichotomous Item Response Theory Analyses: Does the Model Fit?. Educational and Psychological Measurement, 72(3), 510-527. https://doi.org/10.1177/0013164411416976
Toribio, S. G., & Albert, J. H. (2011). Discrepancy measures for item fit analysis in item response theory. Journal of Statistical Computation and Simulation, 81(10): 1345-1360. https://doi.org/10.1080/00949655.2010.4851311
Vetterlein, T., & Zamansky, A. (2016). Reasoning with graded information: The case of diagnostic rating scales in healthcare. Fuzzy Sets and Systems, 298, 207-221. https://doi.org/10.1016/j.fss.2015.11.002
Weissman, A. (2013). Optimizing information using the EM algorithm in item response theory. Annals of Operations Research, 206(1), 627-646. https://doi.org/10.1007/s10479-012-1204-4
Wong, C. C. (2015). Asymptotic standard errors for item response theory true score equating of polytomous items. Journal of Educational Measurement, 52(1), 106-120. https://doi.org/10.1111/jedm.12065
Zadeh, L. A. (1965) Fuzzy sets. Information and Control 8(3), 338–353. https://doi.org/10.1016/S0019-9958(65)90241-X
Zimmermann, H. J. (2001). Fuzzy set theory – and its applications (4th ed.). Kluwer Academinc Publisher. https://doi.org/10.1007/978-94-010-0646-0