Responsible AI in the public service will be measured by what happens after the rules are written

Canada is moving quickly to embrace artificial intelligence, which offers real opportunities to improve productivity and modernize public services. The harder question is what happens after the technology is deployed.

The Public Service Commission’s new guidance on AI in federal hiring provides a useful test. AI can now assist hiring managers in assessing, scoring, sorting, and ranking candidates, which are all activities that can determine who advances and who does not.

The commission has identified many of the right safeguards. Hiring managers remain accountable. Candidates must be informed when and how AI will be used. Departments must mitigate biases and barriers, provide information about accommodations, and explain how AI contributed to an assessment or decision.

That is a strong starting point. But responsible AI will be measured by what happens when those rules encounter real people. A safeguard that is never tested or audited risks becoming a good intention rather than a meaningful protection.

Consider human oversight. It requires more than placing a person at the end of an automated process. The decision-maker must understand what the system measured, recognize unreliable output, and be empowered to question and explain the result.

Disability provides an especially useful stress test. I was born with profound hearing loss, and throughout my career, I have seen how systems designed around an assumed “average” person can exclude someone without anyone intending to discriminate. A speech pattern, response time, or communication style may look like data to an algorithm; for someone like me, it can reflect disability rather than ability. That is why this question is not abstract to me. When technology helps decide who gets through the door, we need to know what it is actually measuring.

A hiring system does not have to know that someone has a disability to discriminate. Discrimination does not always announce itself as discrimination. What can be considered neutral characteristics can interact with disability to produce adverse effects.

Aggregate testing may not reveal the problem. Disability is extraordinarily diverse, and a statistically acceptable outcome can still prevent an individual with a particular disability from fairly demonstrating that they can do the job. This distinction matters beyond disability. It exposes a broader challenge in governing artificial intelligence: systems designed for the average can fail the individual.

Accommodation presents another challenge. A person cannot request accommodation for a barrier they cannot identify. If candidates do not know what an AI system measures or how it interacts with their individual characteristics, they may not realize they need an accommodation until after being assessed.

Transparency, therefore, has to mean more than disclosure that AI was used. As I argued earlier this year in Canadian Lawyer, disclosure is where accountability begins; it cannot be where it ends. An unsuccessful candidate needs enough information to understand what was assessed, how AI affected the result, and how to challenge an error.

These are not arguments against AI, but rather arguments for better AI governance. As Canada expands AI across government, we can distinguish ourselves not by how quickly we adopt it, but by how responsibly we deploy it.

That means moving from principles to evidence. The Treasury Board's Directive on Automated Decision-Making requires departments to complete and publish an algorithmic impact assessment before putting a system into production. 

Yet, as The Hill Times reported, only one of the published assessments so far relates to employment. Are systems independently evaluated for barriers? Is human oversight meaningful? Can candidates obtain understandable explanations and effective recourse? These are not hypothetical questions—AI tools are already assisting federal hiring decisions.

Accountability cannot be outsourced to an algorithm or to the company that built it. The federal government is right to establish guardrails before AI becomes embedded in public administration. The harder task is demonstrating that they work.

Parliament also has a role in examining that evidence. Responsible innovation requires both ambition and scrutiny; the two are not adversaries, and each makes the other stronger.

Canada has begun writing the rules for responsible AI. Leadership will be measured by whether we can prove they work.

Lorin MacDonald is a human rights lawyer, educator, and governance advisor whose work spans artificial intelligence accountability, accessibility, human rights, and public policy. She serves on Accessibility Standards Canada's Technical Committee on Accessible Justice and has taught disability law at Western University and Toronto Metropolitan University.

The Hill Times