Vector Institute releases AI tool to detect and rewrite biased text in clinical notes

Have you ever written a clinical note, referral or letter you regretted and wished you could erase or magically rescind? If you already pressed “send,” you’re out of luck, but a new AI tool released in June by the Toronto-based Vector Institute for Artificial Intelligence may be able to alert you to at least some objectionable content in your prose before it gets you in hot water.UnBias-Plus is a free, open-source AI tool designed to detect, explain and re-write biased language in written content and AI training datasets in seconds. The tool analyzes written text for biased language relating to a patient’s race, gender, age, behaviour and political opinions.If a referral note describes a patient as ‘non-compliant and refusing medication,’ or ‘difficult and missing appointments,’ UnBias-Plus will flag it, explain why it’s objectionable or problematic, and rewrite it for your consideration. Instead, it might say: “patient declined medication.” However, the final decision on any suggested edits always rests with the user.“A patient labelled as ‘difficult,’ or ‘non-compliant’ may receive less thorough follow-up, have their reported symptoms taken less seriously or face delays in referral,” said Shaina Raza, an applied machine learning scientist at the Vector Institute. “These framing effects touch treatment decisions, triage priority and even eligibility determinations for insurance coverage.“One area that concerns me deeply,” she added, “is the documented pattern in how patients from racialized communities, particularly Black, Indigenous and people of colour, have been described in clinical notes,” claiming “research shows these patients are more likely to be characterized in subjective, judgmental terms as difficult, rigid or non-adherent, rather than having their clinical situation described factually and neutrally.”The problem of bias isn’t limited to the language doctors use in their clinical notes. It’s even more detrimental when it finds its way into data used train AI models because “it gets learned, replicated and deployed at scale across entire organizations,” said Raza. “Addressing bias at the source before it reaches people or gets built into a model is far more effective than trying to correct it downstream.”A browser-based version of the tool designed for solo practitioners and small clinics allows clinicians to either upload a file or copy and paste a clinical note into UnBias-Plus. Assuming evidence of bias is detected, the clinician reviews the re-written text, and copy-pastes it back in the EMR. A developer installer version of the software is designed for integration with other applications, including EMRs and AI scribes. “The closest analogy I would use,” said Raza, “is a spell checker, but for bias. Once installed, it can run alongside whatever system it operates within, identify potentially biased language and surface suggestions as content is processed.” This version of the tool requires IT support and is designed for larger organizations that want to screen text at scale and embed bias detection into existing clinical systems and workflows.There is a limit of 750 words for use of the browser-based version, but no word limit for using the developer installer version of the software.The Vector Institute describes UnBias-Plus as especially timely given the June release of Canada’s National AI Strategy that warns against algorithmic bias as a requirement for responsible AI development.UnBias-Plus isn’t limited to use in healthcare. It’s also proposed as a tool that can be used by the media and human resources.Here are the links for accessing the browser-based and developer installer versions of UnBias-Plus.…Author note: UnBias-Plus reviewed this story for bias and only took exception to the very first sentence, suggesting the deletion of “erase” and “magically rescind,” claiming they add “dramatic framing."

Vector Institute releases AI tool to detect and rewrite biased text in clinical notes

Register for free access or login.

Related Topics