NSW gov employers 'should not' use AI for hiring decisions

NSW government employers have been instructed not to use artificial intelligence to screen or make decisions about candidates applying for roles over concerns of bias and potential hallucinations.

The NSW Office of the Public Service Commissioner has issued guidelines stating that AI "should not" be used for making recruitment-based decisions, with a particular emphasis on avoiding its use for ‘highly specialised roles.’"

According to the commissioner's Use of AI in Recruitment guidance [pdf], AI can be appropriately used for tasks such as developing recruitment documents, preparing materials and communications, summarising candidate responses or interviewers’ notes, handling applicant inquiries—especially when volumes are high—and scheduling interviews.

“Agencies that choose to use AI during recruitment processes must ensure AI use is lawful, clearly and transparently documented, and that risks are appropriately identified and mitigated,” it said.

“Critically, AI must not replace humans in making decisions.”

The key concerns about AI in hiring decisions largely focused on bias.

These include automation bias, where hiring teams may unknowingly over-rely on AI outputs; inherent biases built into AI tools; video interview technologies making inferences based on candidates’ emotions, personality, or appearance; and unreliable assessment criteria.

Additionally, the report highlights errors or fabricated information, known as ‘hallucinations,’ as another significant risk.

From a technology perspective, the report noted that while “purchasing off-the-shelf products can appear attractive, it isn’t necessarily appropriate in every agency environment.

“While a product might work in one agency, it might create unforeseen risks in another,” the report added.

“When procuring an AI product, it is important to closely review the marketing and contract material to ensure it meets the [NSW AI Assessment Framework] requirements.

“This is particularly important given that suppliers may be reluctant to provide any information on the algorithmic decision-making used by the AI product, and also because the AI product may have only been tested in limited ways.”