CDU research paper warns about AI risks in higher education

A recent Charles Darwin University (CDU) research paper highlights the risks of AI within higher educating, with the author noting it “discriminates” and is “susceptible to racial and sexist bias”.

The research paper, the critique of AI as a foundation for judicious use in higher education, urges society to look beyond the hype of AI and analyse the risks associated with adopting the technology in education.

Dr Stefan Popenici, author and AI expert at CDU discussed the two most dangerous myths about AI in education.

Firstly, the belief AI is objective, factual and unbiased when it is in fact directly related to specific values, beliefs and biases. Secondly, the belief AI doesn't discriminate when it is inherently discriminatory, referencing also the lack of gender diversity in the growing field.

He said if people think about how technology actually operates, they realise that there is not one point in the history of humanity when technology is not directly related to specific cultures and values, beliefs and biases, religious beliefs or gender stances.

“There is consistent research and books that are providing examples of AI algorithms that discriminate, grotesquely amplify injustice and inequality, targeting and victimising the most vulnerable and exposing us all to unseen mechanisms of decision where we have no transparency and possibility of recourse,” he explained.

Popenici noted the lack of concern for AI in education is affecting the use of students’ data, and impacts on their privacy and ability to think critically and creatively.

He pointed the finger towards “big tech”.

“Big Tech is driven by the aims of profits and power, control and financial gain,” he said.

“Institutions of education and teachers have very different aims: the advancement of knowledge and to nurture educated, responsible, and active citizens that are able to live a balanced life and bring a positive contribution to their societies.”

He said it is “deceiving” to say and “dangerous” to believe that AI is intelligent.

“There is no creativity, no critical thinking, no depth or wisdom in what generative AI gives users after a prompt,” Popenici said.

“Intelligence, as a human trait, is a term that describes a very different set of skills and abilities, much more complex and harder to separate, label, measure and manipulate than any computing system associated with the marketing label of AI.”

He added, “If universities and educators want to remain relevant in the future and have a real chance to reach the aims of education, it is important to consider the ethical and intellectual implications of AI.”