Algorithmic Biases in Artificial Intelligence Systems Implemented in Public Administration
A Case Analysis of Risk Prediction Models
DOI:
https://doi.org/10.35362/issn.1850-0013-1052Keywords:
risk prediction models, algorithmic biases, algorithmic discrimination, algorithmic justice, e-governmentAbstract
The use of artificial intelligence (AI) in public administration has grown significantly in the last decade due to the multiple benefits of this technology in government functions. However, this has also revealed a series of adverse effects such as the presence of algorithmic biases that generate results and decisions that can be discriminatory and unfair for some individuals or social groups. The objective of this article is to identify and analyze the causes of algorithmic biases present in AI systems in public administration, as well as their ethical, political, and social consequences. To do so, four internationally representative cases of AI-based risk prediction models that are used in different areas of public administration are analyzed. The result of this analysis shows that algorithmic biases occur at the design and training stage of the algorithms, generating significant impacts on access to fundamental rights, public services, and social programs. Civil society plays a fundamental role in exposing algorithmic biases and their consequences, as well as in contributing to the generation of strategies for the ethical implementation of this technology.
Downloads
References
Allhutter, D., Cech, F., Fischer, F., Grill, G. & Mager, A. (2020). Algorithmic profiling of job seekers in Austria: How austerity politics are made effective. Frontiers in Big Data, 3, 502780. DOI: https://doi.org/10.3389/fdata.2020.00005.
Angwin, J., Larson, J., Mattu, S. & Kirchner, L. (2016). Machine Bias: There’s Software Used across the Country to Predict Future Criminals. And It’s Biased against Blacks. ProPublica, 23 de mayo. Recuperado de: https://www.propublica.org/article/machinebias-risk-assessments-in-criminal-sentencing.
Aróstegui, L. A. (2023). El uso de RISCANVI en la toma de decisiones penitenciarias. Estudios Penales y Criminológicos, 44(Ext.), 1-43. DOI: https://doi.org/10.15304/epc.44.8884.
Arts, J. & Van den Berg, M. (2024). What the Dutch benefits scandal and policy’s focus on ‘fraud’ can teach us about the endurance of empire. Critical Social Policy, 45(1), 177-187. DOI: https://doi.org/10.1177/02610183241281346.
Avella, M. D. P. R., Sanabria-Moyano, J. E. & Dinas-Hurtado, K. (2022). Uso del algoritmo COMPAS en el proceso penal y los riesgos a los derechos humanos. Revista Brasileira de Direito Processual Penal, 8(1), 275-310. DOI: https://doi.org/10.22197/rbdpp.v8i1.615.
Baker, R. & Hawn, A. (2022). Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32, 1052-1092. DOI: https://doi.org/10.1007/s40593-021-00285-9.
Berning, P. A. D. (2023). El uso de sistemas basados en inteligencia artificial por las Administraciones públicas: estado actual de la cuestión y algunas propuestas ad futurum para un uso responsable. Revista de Estudios de la Administración Local y Autonómica, (20), 165-185. DOI: https://doi.org/10.24965/reala.11247.
Blanchard, M. (2022). Predictive Analytics in Child Welfare: Five Principles for Regulating Algorithmic Accountability in a New Wave of Predictive Models. University of Baltimore Law Review, 51(3), 421-448. Recuperado de: https://scholarworks.law.ubalt.edu/ublr/vol51/iss3/5.
Blomberg, T., Bales, W., Mann, K., Meldrum. R. & Nedelec Bales. J. (2010). Validation of the COMPAS risk assessment classification instrument. Tallahassee: Florida State University. Recuperado de: http://criminology.fsu.edu/wp-content/uploads/Validationof-the-COMPASRisk-Assessment-Classification-Instrument.pdf.
CAF (2021). EXPERIENCIA. Datos e Inteligencia Artificial en el sector público. Caracas: CAF. Recuperado de: https://scioteca.caf.com/handle/123456789/1793.
Chouldechova, A., Benavides-Prado, D., Fialko, O. & Vaithianathan, R. (2018). A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. En Conference on Fairness, Accountability and Transparency (134-148). Recuperado de: https://proceedings.mlr.press/v81/chouldechova18a.html.
Eubanks, V (2017). Automating Inequality. How High-Tech Tools Profile, Police, and Punish the Poor. Nueva York: St. Martin’s Press.
Feathers, T. (2023). False Alarm: How Wisconsin Uses Race and Income to Label Students “High Risk”. The Markup, 27 de abril. Recuperado de: https://themarkup.org/machine-learning/2023/04/27/false-alarm-how-wisconsin-uses-race-and-incometo-label-students-high-risk.
Fuchs, D. J. (2018). The dangers of human-like bias in machine-learning algorithms. Missouri S&T’s Peer to Peer, 2(1), 1-14. Recuperado de: https://scholarsmine.mst.edu/peer2peer/vol2/iss1/1.
Gerchick, M., Jegede, T., Shah, T., Gutierrez, A., Beiers, S., Shemtov, N., Xu, K., Samant, A. & Horowitz, A. (2023). The devil is in the details: Interrogating values embedded in the allegheny family screening tool. En Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (1292-1310). DOI: https://doi.org/10.1145/3593013.3594081.
Hagerty, A., Aranda, F. & Jemio, D. (2023). Predictive puericulture in Argentina: The Plataforma Tecnológica de Intervención Social and the reproduction of Latin eugenics. BJHS Themes, 8, 221-235. DOI: https://doi.org/10.1017/bjt.2023.2.
Hamilton, M. (2019). The sexist algorithm. Behavioral sciences & the law, 37(2), 145- 157. DOI: https://doi.org/10.1002/bsl.2406.
Hamilton, M. (2019). The biased algorithm: Evidence of disparate impact on Hispanics. American Criminal Law Review, 56(4), 1553-1577. Recuperado de: https://www.law.georgetown.edu/american-criminal-law-review/wp-content/uploads/sites/15/2019/06/56-4-the-biased-algorithm-evidence-of-disparate-impact-on-hispanics.pdf.
Johansen, J., Pedersen, T. & Johansen, C. (2020). Studying the transfer of biases from programmers to programs. ArXiv preprint arXiv:2005.08231. DOI: https://doi.org/10.48550/arXiv.2005.08231.
Koniakou, V. (2023). From the “rush to ethics” to the “race for governance” in Artificial Intelligence. Information Systems Frontiers, 25(1), 71-102. DOI: https://doi.org/10.1007/s10796-022-10300-6.
Kordzadeh, N. & Ghasemaghaei, M. (2022). Algorithmic bias: review, synthesis, and future research directions. European Journal of Information Systems, 31(3), 388-409. DOI: https://doi.org/10.1080/0960085X.2021.1927212.
Lappalainen, K. F. (2021). Protecting Children from Maltreatment with the Help of Artificial Intelligence: A Promise or a Threat to Children’s Rights? En de Vries, K. & Dahlberg, M. (Eds.), De Lege. Law, AI and Digitalisation (431-466). DOI: https://doi.org/10.33063/dl.vi.433.
Lazcoz Moratinos, G., & Castillo Parrilla, J. A. (2020). Valoración algorítmica ante los derechos humanos y el Reglamento General de Protección de Datos: el caso SyRI. Revista chilena de derecho y tecnología, 9(1), 207-225. DOI: http://dx.doi.org/10.5354/0719-2584.2020.56843.
Marinucci, L., Mazzuca, C. & Gangemi, A. (2023). Exposing implicit biases and stereotypes in human and artificial intelligence: state of the art and challenges with a focus on gender. AI & Society, 38(2), 747-761. DOI: https://doi.org/10.1007/s00146-022-01474-3.
Martín, J. (2022). Inteligencia artificial, sesgos y no discriminación en el ámbito de la inspección tributaria. Crónica Tributaria, (182/2022), 51-89. DOI: https://doi.org/10.47092/CT.22.1.2.
Nadeem, A., Olivera, M. & Babak, A. (2022). Gender bias in AI-based decision-making systems: a systematic literature review. Australasian Journal of Information Systems, 26, 1-34. DOI: https://doi.org/10.3127/ajis.v26i0.3835.
Ortiz Freuler, J. & Iglesias, C. (2018). Algorithms and Artificial Intelligence in Latin America: A Study of Implementation by Governments in Argentina and Uruguay. Washington DC: World Wide Web Foundation. Recuperado de: https://webfoundation.org/docs/2018/09/WF_AI-in-LA_Report_Screen_AW.pdf.
Pasquinelli, M., Cafassi, E., Monti, C., Peckaitis, H. & Zarauza, G. (2022). Cómo una máquina aprende y falla. Una gramática del error para la Inteligencia Artificial. Revista Hipertextos, 10(17), 13-29. DOI: https://doi.org/10.24215/23143924e054.
Pedace, K., Schleider, T. J. & Balmaceda, T. (2023). Inteligencia artificial y sesgos. El caso de la predicción del embarazo adolescente en Salta. Revista Iberoamericana de Ciencia, Tecnología y Sociedad, 18(53), 9-26. DOI: https://doi.org/10.52712/issn.1850-0013-359.
Peeters, R. & Widlak, A. (2023). Administrative exclusion in the infrastructure‐level bureaucracy: The case of the Dutch daycare benefit scandal. Public Administration Review, 83(4), 863-877. DOI: https://doi.org/10.1111/puar.13615.
Red en Defensa de los Derechos Digitales (R3D) (2020). Países Bajos suspende uso de algoritmo por discriminar grupos vulnerables. Red en Defensa de los Derechos Digitales, 17 de febrero. Recuperado de: https://r3d.mx/2020/02/17/paises-bajossuspende-uso-de-algoritmo-por-discriminar-grupos-vulnerables/.
Rittenhouse, K., Putnam-Hornstein, E. & Vaithianathan, R. (2023). Algorithms, Humans and Racial Disparities in Child Protective Systems: Evidence from the Allegheny Family Screening Tool. Documento de trabajo.
Rinta-Kahila, T., Someh, I., Gillespie, N., Indulska, M. & Gregor, S. (2022). Algorithmic decision-making and system destructiveness: A case of automatic debt recovery. European Journal of Information Systems, 31(3), 313-338. DOI: https://doi.org/10.1080/0960085X.2021.1960905.
Savage, M. (2021). DWP urged to reveal algorithm that ‘targets’ disabled for benefit fraud. The Guardian, 21 de noviembre. Recuperado de: https://www.theguardian.com/society/2021/nov/21/dwp-urged-to-reveal-algorithm-that-targets-disabled-for-benefit.
Simon, J., Hang Wong, P. & Rieder. G. (2020). Algorithmic bias and the Value Sensitive Design approach. Internet Policy Review, 9(4), 1-16. DOI: https://doi.org/10.14763/2020.4.1534.
Skeem, J., Monahan, J. & Lowenkamp, C. (2016). Gender, risk assessment, and sanctioning: The cost of treating women like men. Law and Human Behavior, 40(5), 580-593. DOI: https://doi.org/10.1037/lhb0000206.
Sued, G. (2022). Culturas algorítmicas: conceptos y métodos para su estudio social. Revista Mexicana de Ciencias Políticas y Sociales, 67(246), 43-73. DOI: https://doi.org/10.22201/fcpys.2448492xe.2022.246.78422.
Valle-Cruz, D., García-Contreras, R. & Gil-Garcia, J. R. (2024). Exploring the negative impacts of artificial intelligence in government: the dark side of intelligent algorithms and cognitive machines. International Review of Administrative Sciences, 90(2), 353-368. DOI: https://doi.org/10.1177/0020852323118705.
Washington, A. L. (2018). How to argue with an algorithm: Lessons from the COMPASProPublica debate. Colorado Technology Law Journal, 17, 131-160. Recuperado de: https://ctlj.colorado.edu/wp-content/uploads/2021/02/17.1_4-Washington_3.18.19.pdf.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 CC Attribution 4.0

This work is licensed under a Creative Commons Attribution 4.0 International License.
All CTS's issues and academic articles are under a CC-BY license.
Since 2007, CTS has provided open and free access to all its contents, including the complete archive of its quarterly edition and the different products presented in its electronic platform. This decision is based on the belief that offering free access to published materials helps to build a greater and better exchange of knowledge.
In turn, for the quarterly edition, CTS allows institutional and thematic repositories, as well as personal web pages, to self-archive articles in their post-print or editorial version, immediately after the publication of the final version of each issue and under the condition that a link to the original source will be incorporated into the self-archive.








