Harms and Ethical Risks of AI Detection Tools in Education: False Positives, Bias, Surveillance, and Student Rights

Louie  Giray; Halime Nuran Caner; Azadeh  Amoozegar

doi:10.59045/nalans.2026.109

Authors

Louie Giray Mapua University, Philippines Author https://orcid.org/0000-0002-1940-035X
H. Nuran Caner Akdeniz University, Turkiye Corresponding Author https://orcid.org/0000-0002-3942-1131
Azadeh Amoozegar INTI International University, Malaysia Author https://orcid.org/0000-0003-4988-5113

DOI:

https://doi.org/10.59045/nalans.2026.109

Keywords:

AI detection, academic integrity, algorithmic bias, educational surveillance, quality education

Abstract

Since 2022, generative AI has become deeply embedded in educational contexts. This prompts universities and schools to deploy AI detection tools in the name of academic integrity. This paper accentuates that the dependence on AI detection tools reflects a broader tendency toward procedural outsourcing, whereby complex evaluative responsibilities are delegated to algorithms rather than addressed through pedagogical judgment and institutional governance. When deified as an instrument in academic integrity processes, AI detectors generate recurring harms: they misclassify legitimate student work, fail to reliably identify misconduct, and disproportionately disadvantage non-native English speakers and other linguistically marginalized learners. Their opacity compounds these issues, producing probabilistic scores without interpretable evidence and undermining procedural fairness. This paper also explains that AI detection tools flag non-native writing at higher rates, effectively penalizing linguistic variation and reinforcing dominant language norms. At the same time, while some bad-faith users can evade detection, high-achieving, original, and neurodivergent students are disproportionately accused. Routine reliance on these AI detection tools also normalizes surveillance, which shifts the educational relationship toward suspicion rather than trust. Framed within the United Nations' Sustainable Development Goals, particularly SDG 4 (Quality Education) and SDG 10 (Reduced Inequalities), this dependence on AI detection tools threatens both educational quality and equity. This paper contributes to scholarship on AI in education by demonstrating that detection-centered approaches to academic integrity are structurally flawed and to discussions on student rights by showing how these tools erode procedural fairness and disproportionately harm marginalized learners.

Author Biographies

Louie Giray, Mapua University, Philippines

Louie Giray is an assistant professor in the Department of Liberal Arts at the School of Foundational Studies and Education, Mapúa University. He is actively engaged in both community and international initiatives, serving as a trustee of the Simbayanan ni Maria Community Foundation and as an advisory board member of the Research Center for International and Global Higher Education at Khazar University, Azerbaijan. He is also a research fellow at Shinawatra University in Thailand and a member of the United Nations University Global AI Network as well as the Artificial Intelligence in Education initiative at the University of Oxford. His scholarly work centers on artificial intelligence in and for education. Prior to his academic career, he worked as a grade school teacher, an experience that continues to inform his commitment to teaching and learning across all levels.
H. Nuran Caner , Akdeniz University, Turkiye

H. Nuran Caner, Ph.D., is an academic and doctoral researcher specializing in Curriculum and Instruction. Her research centers on pre-service teacher education, professional development, and the integration of artificial intelligence in educational contexts, with particular attention to the use and implications of AI detection tools. Her broader scholarly interests include teaching practice, in-service teacher training, and emerging trends in teacher education.
Azadeh Amoozegar , INTI International University, Malaysia

Dr. Azadeh Amoozegar holds a Ph.D. in Educational Technology from Universiti Putra Malaysia (2018) and is currently a Senior Lecturer at the Faculty of Education and Liberal Arts, INTI International University. She is an active contributor to the academic community, serving as a peer reviewer for several high-impact journals and publishing extensively in reputable outlets. Dr. Amoozegar has presented at numerous national and international conferences and has received research awards and competitive grants in recognition of her scholarly contributions. She also supervises postgraduate students and conducts workshops on data analytics, academic writing, and research methods. Her research interests include online learning, artificial intelligence in education, education for sustainable development, adaptive learning technologies, and instructional design.

References

Akhilesh, P., Chennupati, A., Prahas, B., & Venugopalan, M. (2025). AI generated text detection using LORA based fine-tuning of LLM models. In 2025 3rd World Conference on Communication and Computing (pp. 1–7). IEEE. https://doi.org/10.1109/WCONF64849.2025.11233554

Al Najdawi, M. H., Shwedeh, F., Abdelmoghies, M. M., & Ali, A. (2024). Applying artificial intelligence in predicting educational excellence in higher education institutions: A case study in Jordanian universities. Edelweiss Applied Science and Technology, 8(6), 7273–7289. https://doi.org/10.55214/25768484.v8i6.3579

Albeihi, H. H. M., & Rice, M. F. (2025). Generative AI and language diversity: Implications for teachers and learners. Arab World English Journal, 16(1), 43–54. https://dx.doi.org/10.24093/awej/vol16no1.3

Alsharefeen, R. (2025). Faculty as street-level bureaucrats: Discretionary decision-making in the era of generative AI. Frontiers in Education, 10. https://doi.org/10.3389/feduc.2025.1662657

Amjad, A. I. (2026). Redesigning AI for inclusion: University students' experiences with cultural exclusion in AI learning systems. In S. Maqbool, H. Zafeer, & A. Tariq (Eds.), AI-powered cognitive tutors and the new frontier of personalized learning (pp. 139–164). IGI Global. https://doi.org/10.4018/979-8-3373-4217-7.ch006

Angeles, C. N., Samson, B. D., Mama, B. R. Z. I., Luriaga, R. L., Delizo, J. P. D., & Ching, M. R. D. (2024). Students’ perception of the use of AI detector system by faculty members in determining the originality of submitted academic requirements. In Proceedings of the 2024 8th International Conference on E-Commerce, E-Business, and E-Government (pp. 56–61). ACM. https://doi.org/10.1145/3675585.3675587

Baron, P. (2024). Are AI detection and plagiarism similarity scores worthwhile in the age of ChatGPT and other generative AI? Scholarship of Teaching and Learning in the South, 8(2), 151–179. https://doi.org/10.36615/sotls.v8i2.411

Bassett, M. A., Bradshaw, W., Bornsztejn, H., Hogg, A., Murdoch, K., Pearce, B., & Webber, C. (2026). Heads we win, tails you lose: AI detectors in education. Journal of Higher Education Policy and Management, 1–16. https://doi.org/10.1080/1360080X.2026.2622146

Begum, R. S., Mahesan, S., Min, J., & Sakkan, T. (2025). Decolonizing the digital classroom: A critical analysis of power, privilege, and algorithmic bias in AI-mediated learning environments. Asian Journal of Interdisciplinary Research, 8(4), 301–330. https://doi.org/10.54392/ajir25417

Caner, H. N. (2025). AI detection tools: A systematic review of empirical evidence and their implications for education. Khazar Journal of Humanities and Social Sciences, 28(4), 234–246. https://doi.org/10.5782/2223-2621.1580

Casal, J. E., & Kessler, M. (2023). Can linguists distinguish between ChatGPT/AI and human writing? Research Methods in Applied Linguistics, 2(3), Article 100068. https://doi.org/10.1016/j.rmal.2023.100068

Chaka, C. (2024). Accuracy pecking order: How 30 AI detectors stack up in detecting generative artificial intelligence content in university English L1 and English L2 student essays. Journal of Applied Learning and Teaching, 7(1), 127–139. https://doi.org/10.37074/jalt.2024.7.1.33

Chan, C. K. Y., & Colloton, T. (2024). Generative AI in higher education: The ChatGPT effect. Routledge. https://doi.org/10.4324/9781003459026

Chaudhary, G. (2024). Unveiling the black box: Bringing algorithmic transparency to AI. Masaryk University Journal of Law and Technology, 18(1), 93–122. https://doi.org/10.5817/MUJLT2024-1-4

Che, L., Guo, P., Isleem, H. F., & Wang, Z. (2025). The necessity of multimodal feedback for learning effective pedagogical policies with reinforcement learning. Scientific Reports, 16(1), Article 454. https://doi.org/10.1038/s41598-025-29892-5

Dai, R., Thomas, M. K. E., & Rawolle, S. (2025). Revisiting Foucault’s panopticon: How does AI surveillance transform educational norms? British Journal of Sociology of Education, 46(5), 650–668. https://doi.org/10.1080/01425692.2025.2501118

Dakshit, S., Mokhtari, K., & Khalid, A. (2026). Designing understandable and fair AI for learning: The PEARL framework for human-centered educational AI. Education Sciences, 16(2), Article 198. https://doi.org/10.3390/educsci16020198

Deep, P. D., Edgington, W. D., Ghosh, N., & Rahaman, M. S. (2025). Evaluating the effectiveness and ethical implications of AI detection tools in higher education. Information, 16(10), Article 905. https://doi.org/10.3390/info16100905

Elkhatat, A. M., Elsaid, K., & Almeer, S. (2023). Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text. International Journal for Educational Integrity, 19, Article 17. https://doi.org/10.1007/s40979-023-00140-5

Farooqi, S. A. (2026). Ethical framework for AI in education: Navigating challenges and building solutions. In AI in education, governance, and leadership: Adoption, impact, and ethics (pp. 153–188). IGI Global. https://doi.org/10.4018/979-8-3373-5550-4.ch007

Fishchuk, V., & Braun, D. (2024). Robustness of generative AI detection: Adversarial attacks on black-box neural text detectors. International Journal of Speech Technology, 27(4), 861–874. https://doi.org/10.1007/s10772-024-10144-2

Fisk, G. D. (2025). AI or human? Finding and responding to artificial intelligence in student work. Teaching of Psychology, 52(3), 314–318. https://doi.org/10.1177/00986283241251855

Fleckenstein, J., Meyer, J., Jansen, T., Keller, S. D., Köller, O., & Möller, J. (2024). Do teachers spot AI? Evaluating the detectability of AI-generated texts among student essays. Computers and Education: Artificial Intelligence, 6, Article 100209. https://doi.org/10.1016/j.caeai.2024.100209

Gallano, G., Giglio, A., & Ferre, A. (2025). Artificial intelligence in speech-language pathology and dysphagia: A review from Latin American perspective and pilot test of LLMs for rehabilitation planning. Journal of Voice. Advance online publication. https://doi.org/10.1016/j.jvoice.2025.04.010

Giray, L. (2024). The problem with false positives: AI detection unfairly accuses scholars of AI plagiarism. The Serials Librarian, 85(5–6), 181–189. https://doi.org/10.1080/0361526X.2024.2433256

Giray, L. (2026). AI humanizers and the crisis of information integrity: Implications for scientific writing. Naunyn-Schmiedeberg’s Archives of Pharmacology. Advance online publication. https://doi.org/10.1007/s00210-026-05200-4

Giray, L., Sevnarayan, K., & Ranjbaran Madiseh, F. (2025). Beyond policing: AI writing detection tools, trust, academic integrity, and their implications for college writing. Internet Reference Services Quarterly, 29(1), 83–116. https://doi.org/10.1080/10875301.2024.2437174

Hyatt, J. P. K., Bienenstock, E. J., Firetto, C. M., Woods, E. R., & Comus, R. C. (2025). Using aggregated AI detector outcomes to eliminate false positives in STEM-student writing. Advances in Physiology Education, 49(2), 486–495. https://doi.org/10.1152/advan.00235.2024

Ibrahim, K. (2023). Using AI-based detectors to control AI-assisted plagiarism in ESL writing: The terminator versus the machines. Language Testing in Asia, 13(1), Article 20. https://doi.org/10.1186/s40468-023-00260-2

Isaeva, R., Caner, H. N., Caner, M., Giray, L., & Karadag, E. (2026). Students’ engagement with generative AI in academic learning: A self-determination theory and epistemic network analysis study. Computers and Education: Artificial Intelligence, 10, Article 100606. https://doi.org/10.1016/j.caeai.2026.100606

Islam, M. N., Ahmad, S., Aqil, M., Hu, G., Ashiq, M., Abusharhah, M. M., & Saky, S. A. T. M. (2025). Application of artificial intelligence in academic libraries: A bibliometric analysis and knowledge mapping. Discover Artificial Intelligence. Advance online publication. https://doi.org/10.1007/s44163-025-00295-9

Ismail, I. A., & Aloshi, J. M. (2025). Data privacy in AI-driven education: An in-depth exploration into the data privacy concerns and potential solutions. In AI in education: Governance, ethics, and emerging technologies (pp. 223–252). IGI Global. https://doi.org/10.4018/979-8-3693-5443-8.ch008

Ismail, I. A., Alkhateeb, M. O., & Alkaabi, M. H. (2025). AI-driven learning: Current applications, challenges, and future prospects. In Examining AI disruption in educational settings: Challenges and opportunities (pp. 1–32). IGI Global. https://doi.org/10.4018/979-8-3693-7873-1.ch001

Jackson, S., Beekhuizen, B., Zhao, Z., & McEwen, R. (2025). GPT-4-Trinis: Assessing GPT-4's communicative competence in the English-speaking majority world. AI and Society, 40, 1785–1801. https://doi.org/10.1007/s00146-024-01945-9

Jiang, Y., Hao, J., Fauss, M., & Li, C. (2024). Detecting ChatGPT-generated essays in a large-scale writing assessment: Is there a bias against non-native English speakers? Computers & Education, 217, Article 105070. https://doi.org/10.1016/j.compedu.2024.105070

Jones, K. O. (2008). Practical issues for academics using the Turnitin plagiarism detection software. In Proceedings of the 9th International Conference on Computer Systems and Technologies (Article 50). ACM. https://doi.org/10.1145/1500879.1500935

Kangwa, D., Msafiri, M. M., & Fute, A. (2025). Exploring the factors that promote a balance between academic integrity and the effective use of GenAI tools in higher education: A systematic review. Journal of Computer Assisted Learning, 41(5), Article e70109. https://doi.org/10.1111/jcal.70109

Khalkho, R., Singh, S., Gupta, N., & Srivastava, P. (2024). Impact of educational AI on students' studying habits and academic performance. In 2024 International Conference on Artificial Intelligence and Quantum Computation-Based Sensor Application (pp. 1–6). IEEE. https://doi.org/10.1109/ICAIQSA64000.2024.10882178

Khari, M. (2026). Beyond the algorithm: Educational justice for plurilingual students in AI-assisted classrooms. In AI in education: Governance, ethics, and emerging technologies (pp. 405–444). IGI Global. https://doi.org/10.4018/979-8-3373-2670-2.ch015

Koutropoulos, A., Honeychurch, S., Bali, M., Singh, L., Nerantzi, C., Stewart, B., Abegglen, S., Sinfield, S. F., Lazou, C., & Moore, S. (2026). AI surveillance in education: Unpacking ethical dilemmas and the snake-oil promises of AI-infused technosolutionism. In A. Bozkurt (Ed.), Policy and practice for ethical AI integration in education (pp. 69–108). IGI Global. https://doi.org/10.4018/979-8-3373-7569-4.ch003

Kundu, A., & Bej, T. (2025a). AI policies in school education: A comparative study on China, Singapore, Finland, and the US. Journal of Science and Technology Policy Management. Advance online publication. https://doi.org/10.1108/JSTPM-06-2024-0218

Kundu, A., & Bej, T. (2025b). Psychological impacts of AI use on school students: A systematic scoping review of the empirical literature. Research and Practice in Technology Enhanced Learning, 20, Article 030. https://doi.org/10.58459/rptel.2025.20030

Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7), Article 100779. https://doi.org/10.1016/j.patter.2023.100779

Liang, X., Sun, R., & Wu, Q. (2025). Research on the protection and governance of education data privacy in the era of artificial intelligence. In Second International Conference on Optical Communication and Optoelectronic Technology (Vol. 13985, pp. 133–138). SPIE. https://doi.org/10.1117/12.3078222

Maheshwari, K. K., Hajano, S., Baloch, S., Zafar, Z., & Fatima, T. (2025). AI based surveillance system. In 2025 International Conference on Emerging Technologies in Electronics, Computing, and Communication (pp. 1–6). IEEE. https://doi.org/10.1109/ICETECC65365.2025.11070296

Manerba, M. M. (2023). Fairness auditing, explanation and debiasing in linguistic data and language models. In L. Longo (Ed.), Joint proceedings of the xAI-2023 late-breaking work, demos and doctoral consortium (pp. 241–248). CEUR-WS. https://ceur-ws.org/Vol-3554/paper39.pdf

Männiste, M., & Siibak, A. (2025). AI magnified inequalities: Bias, (un)fairness and discrimination resulting from the use of AI-based technologies in the education sector. In Young people in digital environments (pp. 219–233). Edward Elgar Publishing. https://doi.org/10.4337/9781035329250.00025

Martin, N., Wray, M., & Krupa, J. (2025). Universal design for learning: A critical approach. Routledge. https://doi.org/10.4324/9781041057796

Mustapha, R., Ibrahim, N., Ayub, M. N., Jaafar, A. B., Jusoh, M. K. A. B., & Mahmud, M. (2025). From detection to prevention: A human-centered design (HCD) approach to mitigating AI misuse in learning. International Journal of Basic and Applied Sciences, 14(4), 159–167. https://doi.org/10.14419/3t4ghz09

Olsen, H. P., Hildebrandt, T. T., Wiesener, C., Larsen, M. S., & Flügge, A. W. A. (2024). The right to transparency in public governance: Freedom of information and the use of artificial intelligence by public agencies. Digital Government: Research and Practice, 5(1), 1–15. https://doi.org/10.1145/3632753

Pariyanti, E., Wibowo, M., Sultan, Z., & Siolemba Patiro, S. P. (2025). Navigating the ethical dilemma in digital learning: Balancing artificial intelligence with spiritual integrity to address cheating. Journal of Applied Research in Higher Education. Advance online publication. https://doi.org/10.1108/JARHE-10-2024-0522

Perkins, M. (2023). Academic integrity considerations of AI large language models in the post-pandemic era. Journal of University Teaching and Learning Practice, 20(2), Article 07. https://doi.org/10.53761/1.20.02.07

Perkins, M., Roe, J., Vu, B. H., Postma, D., Hickerson, D., & McGaughran, J. (2024). Simple techniques to bypass GenAI text detectors: Implications for inclusive education. International Journal of Educational Technology in Higher Education, 21, Article 53. https://doi.org/10.1186/s41239-024-00487-w

Prajapati, M., Baliarsingh, S. K., & Dev, P. P. (2026). Detecting AI-generated text in high-resource languages: Developing a RoBERTa-CNN hybrid model for academic integrity challenge. International Journal of Machine Learning and Cybernetics, 17(2), Article 40. https://doi.org/10.1007/s13042-025-02868-7

Pratama, A. R. (2025). The accuracy-bias trade-offs in AI text detection tools and their impact on fairness in scholarly publication. PeerJ Computer Science, 11, Article e2953. https://doi.org/10.7717/peerj-cs.2953

Prykhodchenko, S. D., Prykhodchenko, O. Y., Shevtsova, O. S., & Olishevskiy, I. H. (2025). Software detection of Ukrainian-language texts generated by AI: Methods, estimations, challenges. Naukovyi Visnyk Natsionalnoho Hirnychoho Universytetu, 4, 150–159. https://doi.org/10.33271/nvngu/2025-4/150

Rahman, M. I. (2026). Algorithmic bias and transparency in artificial intelligence tools for nursing education: A scoping review. Nurse Education in Practice, 74, Article 104756. https://doi.org/10.1016/j.nepr.2026.104756

Rai, R. D., Pawar, M., & Rajendran, R. (2023). Ethical challenges and best practices for transparency in AIED: A literature review and learner-centric guidelines. In J. L. Shih, A. Kashihara, W. Chen, & H. Ogata (Eds.), Proceedings of the 31st International Conference on Computers in Education (ICCE 2023). Asia-Pacific Society for Computers in Education. https://doi.org/10.58459/icce.2023.1438

Ray, P. P. (2026). A review of TRiSM frameworks in artificial intelligence systems: Fundamentals, taxonomy, use cases, key challenges and future directions. Expert Systems, 43(3), Article e70213. https://doi.org/10.1111/exsy.70213

Saarela, M., Gunasekara, S., & Karimov, A. (2025). The EU AI act: Implications for ethical AI in education. In International Conference on Design Science Research in Information Systems and Technology (pp. 36–50). Springer. https://doi.org/10.1007/978-3-031-93979-2_3

Sangka, K. B., Pratama, R. D., & Hidayat, R. W. (2025). Redefining education in the digital age: The role of GPT models in academic fraud and logistics. Human Behavior and Emerging Technologies, 2025(1), Article 7118205. https://doi.org/10.1155/hbe2/7118205

Shafi, T. B., Siddique, T., Tasnim, F., Chowdhury, S. J., Chowdhury, M. S. H., Mahmud, T., & Hanip, A. (2025). Detecting AI-generated text using machine learning: A comparative study of MLP and RoBERTa-based models. In International Conference on Data Science, AI and Applications (pp. 101–114). Springer. https://doi.org/10.1007/978-3-032-11355-9_8

Stillman, T. (2025). Maintaining academic integrity through AI-trap questions. Journal of Educators Online, 22(4), Article n4. https://doi.org/10.9743/JEO.2025.22.4.14

Swartz, M., & McElroy, K. (2023). The “Academicon”: AI and surveillance in higher education. Surveillance & Society, 21(3), 276–281. https://doi.org/10.24908/ss.v21i3.16105

Torrisi-Steele, G. (2026). AI and the ethics of student data privacy. In V. Wang (Ed.), Foundations and frameworks for AI in education (pp. 53–76). IGI Global Scientific Publishing. https://doi.org/10.4018/979-8-3373-2397-8.ch003

Türkmen, G. (2025). The review of studies on explainable artificial intelligence in educational research. Journal of Educational Computing Research, 63(2), 277–310. https://doi.org/10.1177/07356331241310915

Waked, A. N., Abdelsalam, H., Ashraf, M. W., Pilotti, M., & El Alaoui, K. (2024). The impact of falsely detecting AI-generated text on academic assessment. In M. Shelley & O. T. Ozturk (Eds.), Proceedings of ICRES 2024 (pp. 573–586). ISTES. https://files.eric.ed.gov/fulltext/ED673127.pdf

Walters, W. H. (2023). The effectiveness of software designed to detect AI-generated writing. Open Information Science, 7(1), 1–24. https://doi.org/10.1515/opis-2022-0158

Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., & Waddington, L. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1), 1–39. https://doi.org/10.1007/s40979-023-00146-z

Williams, F. W., & Jowallah, R. (2025). The intersection of AI in higher education: Pros and cons for cultural inclusion and engagement. In International Conference on Human-Computer Interaction (pp. 200–213). Springer. https://doi.org/10.1007/978-3-031-93567-1_14

Xue, Y., Chinapah, V., & Zhu, C. (2025). A comparative analysis of AI privacy concerns in higher education: News coverage in China and Western countries. Education Sciences, 15(6), Article 650. https://doi.org/10.3390/educsci15060650

Yahaya, W. A. J. W., & Muhammad, Y. (2025). AI-driven innovations for comprehensive campus-wide monitoring support services. In A practical guide to artificial intelligence in higher education (pp. 99–105). Springer. https://doi.org/10.1007/978-3-031-56848-0_11

Yandrapalli, V. K., & Sharma, S. (2025). Unveiling synergy: Explainable AI's role in enhancing cloud-based collaborative learning in the cloud era. In U. Shrawankar & P. Mishra (Eds.), Cloud computing for smart education and collaborative learning (pp. 159–169). Chapman and Hall/CRC. https://doi.org/10.1201/9781003472537-14