AI in Digital Transformation, Part 2

Part one of this two-part series discussed the common types of artificial intelligence (AI) and why the technology is important to a company’s digital transformation. Part 2 discusses the challenges companies experience with AI, including data bias, ethical considerations, AI hallucinations, the balance between humans and AI technology, and AI governance, as well as how AI experts are striving to remedy these issues to make AI a valuable contributor to the digital infrastructure.

In 2023, a software engineer in California convinced the chatbot at a local auto dealership to sell him a new Chevy Tahoe for $1. It agreed. Of course, the arrangement was not legally binding, so it didn’t become the deal of the century. Nevertheless, news of that exchange triggered many attempts to goad the chatbot into acting against the dealership’s best interest. The AI unit resisted most of them, but this still raises the question of how much responsibility we should be handing over to AI technology.

While the auto dealership scenario was a relatively simple prank, there are cases involving AI where the stakes are higher, demonstrating that, despite its capabilities, AI does have its share of challenges. A study on the use of AI in healthcare, published in ScienceDirect in 2025, found that the technology has enabled faster disease detection and better personalized healthcare management. However, the study went on to reveal that the clinical AI methods developed and deployed in hospitals exhibit algorithmic, data-driven biases due to insufficient representation of specific races, genders, and age groups, leading to disparities and erroneous outcomes.

Algorithmic bias occurs when computers unintentionally make unfair choices because of the information they were trained on. This revelation suggests that AI is not fully ready to function in all capacities without human supervision.

Limited Data

The latest catchphrase is “AI is only as good as the data it's fed.” AI models are trained on historical data, enabling them to recognize patterns, automate processes, and forecast future outcomes. However, AI’s reliance on historical data may exacerbate existing biases and injustices. An AI model trained on biased data will produce biased results, regardless of how accurate its predictions appear to be.

If the data used to teach an AI system reflects historical prejudices or inequalities, the AI will likely learn and perpetuate those biases. For example, AI systems are increasingly used in corporate settings for hiring decisions, performance evaluations, and promotions. If these systems rely solely on accurate but incomplete data, they risk reinforcing biases and overlooking critical human factors, leading to ineffective or unfair decisions.

Lack of Data

Another issue is that the absence of data (fragmented, outdated, low-quality, or missing) can be just as harmful as the inclusion of overtly biased data or the use of models trained on biased training data. This concept, called algorithmic exclusion, involves an AI system that lacks enough data on an individual or organization to return sufficient output about them. This can happen when data is pulled from narrow sources or reflects past exclusion. Essentially, the AI algorithm fails to recognize individuals, or does so incorrectly, in its analysis.

As AI systems grow more sophisticated, their role in analyzing data and making decisions becomes more critical and influential. That said, bias in AI raises significant ethical concerns about its use, challenging the fairness and justice of the technology’s autonomy.

Intellectual Property and Copyrighted Material

AI promises faster data gathering and analytics, as well as unprecedented possibilities for innovation. It also presents some complex legal issues, particularly in intellectual property and copyright law. There have already been numerous infringement lawsuits filed for using copyrighted materials to train AI models. Currently, the U.S. Copyright Office maintains that human authorship is a prerequisite for copyright protection. If individuals use AI to create copyrighted material, the AI is simply a tool of human creativity. Copyright does not protect facts, ideas, systems, or methods of operation, although it may protect the way these things are expressed. As for copyrighted material used in training AI, there are more variables, which makes the laws more complicated. 

Lack of Explainability

In a nutshell, explainable AI (XAI) is a set of techniques and processes that allow humans to understand the rationale behind the results and outputs of machine learning algorithms. The concept of AI algorithms is commonly referred to as a “black box,” where AI models arrive at conclusions or decisions without providing any explanations for how the outputs were reached. The AI workflows and contributing factors are not visible to human users.

There are two schools of thought regarding XAI. One argues that AI models are simply too complex or that knowing the rationale isn’t necessary for humans. For example, people don’t need to know the molecular constructs and manufacturing processes of pharmaceuticals to reap the benefits. The other contends that understanding how an AI system reaches an output can help developers ensure it’s working as expected. Also, full transparency is required for organizations needing to comply with regulatory guidelines.

To further the argument, MIT Lincoln Laboratory published a paper developed from reviews of 18,254 papers on XAI. In theory, XAI is designed so that AI will spell out the decisions it makes in a way that is interpretable by humans, enabling humans to check that systems are behaving as expected. AI systems require a certain amount of trust from users, and one way to gain that trust is for people to understand how the systems work. However, according to the paper, researchers discovered that the AI decisions are not interpretable by humans. Hosea Siu, a researcher in the MIT Lincoln Laboratory Group, asserted that interpretability has long been a challenge in the field of AI and autonomy. “The machine learning process happens in a ‘black box,’ so model developers often can't explain why or how a system came to a certain decision,” he said. 

AI hallucinations are outputs produced by generative AI models that are linguistically plausible yet entirely fabricated, factually incorrect, or not eligible for verification. Hallucinations can manifest as invented events, fake citations, misquoted archival material, or unsubstantiated interpretations, which AI presents without qualifiers or disclaimers.

The most common reason for hallucinations is that AI produces outputs based on flawed training data—or it simply lacks a clear answer, so it fakes it. Most AI models generate outputs based on the publicly available data they were trained on, rather than a particular knowledge base. Without grounding large language model (LLM) outputs in trusted data sources, they can’t always be relied on for precise information. During a “60 Minutes” report, a few developers at Google gave an example of AI hallucination, citing that Google’s Bard was tasked with writing an essay on economics and recommending five books. Bard produced five seemingly realistic titles, but the books were nonexistent.

While AI is rapidly becoming commonplace in our lives, it doesn’t automatically or seamlessly become woven into them. AI makes its most valuable contributions when embedded in core workflows, decision-making processes, and operating models, reshaping how organizations incorporate the technology into their business models. However, it is imperative that companies redefine the relationship between people and intelligent technology systems, ensuring that human judgment, responsibility, and oversight remain firmly at the center of the collaboration.

Customer Priority Policy

Customer relations is a prime example. A SaaS company that was an early adopter of AI quickly turned over a variety of customer administrative tasks (client call agendas, reporting, data analysis, etc.) to the AI tools. Over time, staff members were unable to recall details from customer conversations and ultimately lost sight of what mattered to customers. Their job descriptions essentially changed to operators of AI.

Seeing Eye-to-Eye in Healthcare

Organizations are proactively working to unlock value from AI use cases. Still, more thought needs to go into embedding AI into the core processes of decision-making and defining how work gets done. There is also the need to answer the burning question, “Who is responsible if something goes wrong?” For example, the technology is trained to support physicians in decision-making and even to recommend actions independently of its human colleagues. Although, whether in collaboration or independent, AI-recommended actions are still always validated by humans. Despite this working relationship, the parameters for its integration into clinical processes are currently in the spotlight. Apparently, not all humans and AI tools work and play well together. 

One example is presented in a paper published in 2022 that explored the role and resolution of disagreements between physicians and their diagnostic AI-based decision support systems (DSS).

Conflicts between physicians and DSS can arise from different forms of friction or incongruence. Diagnostics are designed to operate with wide margins of uncertainty, given our limited knowledge of diseases in general, as well as their formation in individual patients. Proposing different diagnoses can be based on three different reasons:

• The physicians’ assessments are incorrect.

• The DSS assessments are incorrect.

• Both the physician and the DSS are within a range of probable results.

When a human is in a disagreement with another human about a decision and unilaterally decides to break the tie and proceed, that person is usually comfortable with being responsible for the decision's success or failure. However, in human-DSS disagreements, there is no equivalent understanding. Therefore, it is advised that the human physician remains the sole decision-maker since responsibility can be assured in that scenario. Because physicians remain responsible for the outcomes of their decisions, they can reject DSS recommendations if they disagree.

The deployment of AI across industries, such as healthcare, manufacturing, education, government, and others, will continue at an accelerated pace. As with all technologies, AI is a work in progress. To keep it from spiraling out of control, stakeholders are advised to put up guardrails. The AI paradigm is forcing the hand of organizations to establish methods of AI governance, which include necessary practices, protocols, safeguards, systems, and tools to ensure the responsible use of the technology. AI professionals actively involved in AI governance are establishing methods and policies for developing and training new AI models, operating guidelines for applying the models, and software and other technologies to create safety measures with appropriate human oversight. AI governance provides a structured approach to mitigate the potential risks associated with AI systems.

Organizations that succeed in this endeavor keep humans firmly in the lead. They redesign operating models around trust and transparency, deploy disciplined experimentation, while humans remain responsible for direction, trade-offs, decision-making, and outcomes. Companies achieve this through established cross-functional governance and continuous model oversight, complete with audit cycles.

Despite all of its hype, capabilities, and potential for reimagining work and productivity, AI isn’t an independent entity apart from your digital infrastructure. This AI technology is specifically designed to improve operational efficiency by taking on and speeding up the completion of repetitive tasks. 

Your company’s IT systems are enhanced by AI through the embedding of intelligent systems into the infrastructure, enabling it to leverage your organization’s business-specific data. Gaining the full, unbiased potential of AI requires comprehensive, structured data, which is data that fits into fixed tables with consistent rows and columns, and is readily usable for AI.

One area where AI is proving its worth is intelligent automation. For example, although organizations in all industries are shifting to digital operations, printing will always remain a part of the daily business functions. That said, Vasion Intelligent Print Automation (IPA) technology enables your company to modernize all print management tasks, elevate security, unify print and output management, and automate document workflows.

IPA is AI technology that enables users to get documents to the printer, route them beyond the printer or scanner, get signatures on them, then deliver them to storage or elsewhere. Typically, when documents are printed, the data immediately becomes unstructured. With IPA, instead of pressing File>Print and sending the content to a printer, those same steps enable users to send content to a workflow—keeping the data structured. AI’s agentic workflows can activate on every document already flowing through the platform. The data previously existing as flat files, and invisible to every downstream system, starts feeding your organization’s AI instead. 

Ensure you’re gaining the most value from your AI technology and enhance the efficiency of all business units using Vasion Intelligent Print Automation (IPA).