Rahul Kamath THUMB

Rahul Kamath

Insights & Strategy

Design Strategy

The book <a href="https://en.wikipedia.org/wiki/Zen_and_the_Art_of_Motorcycle_Maintenance" target="_blank" rel="noreferrer noopener">Zen and the Art of Motorcycle Maintenance</a>&nbsp;starts with a road trip in which the author Robert Pirsig and his friend John travel into rural areas of the American West on motorcycle.
During this trip, the author notes how differently he and his traveling companion experience and&nbsp;treat their vehicles.

Delve's experts in insights, strategy, design, and engineering can guide you through an end-to-end innovation process that delivers results.

Our Services Page: Product Design and Development (text only) (2)

Trust ai healthcare inline 1 335px

<h2>Magic vs. Machine</h2>
Robert observes that John essentially sees his motorcycle as “magic.” He does not understand how it works, why it works, or how to fix it if it ceases to work. He knows he must add fuel, but that’s about it. For John, the motorcycle is a black box, but as long as it generates movement, he does not care that he has no idea how it works, just that it does. The author describes John’s relationship to his motorcycle as the “romantic” view of the world. 
In contrast, to Robert&nbsp;the motorcycle is a predictable and carefully constructed machine—one governed by physical principles that he understands, and which were harnessed to create a usable tool. There is nothing “woo” about it. He can dissect it, fix it, and put it back together using reason and his understanding of physics. He describes this as the “classical” view of the world.

Gray 50

Continue this conversation on LinkedIn! Follow Rahul's <a href="https://www.linkedin.com/in/rahulkamath/" target="_blank">LinkedIn page</a> and share your thoughts about this blog post <a href="https://www.linkedin.com/posts/rahulkamath_healthcare-artificialintelligence-technology-activity-7075479821882949633-UNk6" target="_blank">here.</a>

John’s perspective is presented as frustrating to Robert, but I think most of us can relate to John in some way or another. After all, this is often how we ourselves perceive and experience technology. 
Think of computers, cell phones, plumbing, electricity, cars—and, of course, motorcycles. We use products like these all the time, but unless we are professionals, we generally remain indifferent to how and why technological sorcery works, primarily focusing on what can be done with them, and outsourcing any technical work to others.

Given the above, it is no surprise that the general public’s experience of artificial intelligence (AI) aligns with John's experience of motorcycles. It's a viewpoint no doubt driven by the impressive text and image generation tools that recently seem to have sprouted up from nowhere. 
For many, the experience of using common AI tools is one of alchemy; mysterious and beautiful, a source of wonder. Just consider the remarkable simplicity of querying, an experience in which we are shielded from the complex inner clockwork occurring behind the scenes.
I relate this here because it is important to draw attention to this divide as a focal point.

Trust ai healthcare inline 2 310px

<h2>An Important&nbsp;Difference Between Motorcycles and AI</h2>
There is an important philosophical difference between an information-dispensing AI tool and a motorcycle. With a motorcycle, movement self-evidently demonstrates to a user the machine’s proper functionality and efficacy.&nbsp;
A motorcycle’s movement is the value, and that value is easily observed and can be exploited by a user even if they are ignorant of how a motorcycle works on a technical level. 
In contrast to the visible, physical output of a motorcycle, AIs are very often epistemic tools. They dispense information and knowledge—or more accurately, the appearance of knowledge. It is the information’s relevance and accuracy in the situation that embodies its value.
The problem is that, unlike with motorcycles, we cannot easily, independently assess or validate the value of AI outputs via simple observation or empiricism. 
Here enters the issue of trust.&nbsp;
We tacitly—I would say blindly at this point—rely on AI to provide validated information. This is the whole point of it. If we believed AI dispensed primarily garbage, we would not be using it at all.
 Medical AI, in particular, requires validation to be safe, and trust to be valuable. However, this is also where I perceive a dangerous shell game going on in AI-land.

Ai medical device webinar youtube thumbnail

We had a great discussion about the ways AI will impact the medical device industry.

<h2>The One-Way Mirror of the AI Assembly Line</h2>
Developers of AI tools are surely in Robert’s “classical” camp. They know how AI inputs work and where outputs come from. They must, in order to create AI tools in the first place.&nbsp;
But many AI companies seem to be&nbsp;avoiding discussing data inputs in a meaningful way. Perhaps they&nbsp;prefer that the&nbsp;public&nbsp; buy into the “woo” side of the offering: “It doesn’t matter how it works, just look at these amazing outputs!” 
Whatever their intentions, generally speaking,&nbsp;AI companies have made very little effort to educate the public on the value-chain of AI’s raw ingredients. They avoid publicly discussing the critical elements that drive a reliable AI, such as data collection processes, validation, and training.&nbsp;
They have even deliberately misdirected the public’s perception <a href="https://www.technologyreview.com/2023/03/14/1069823/gpt-4-is-bigger-and-better-chatgpt-openai/" target="_blank">by implying that the more massive the quantity of input data, the more astounding the outputs</a>—sidestepping any conversation on the process of ensuring data quality or training that data. In doing so, they also gloss over questions about the kinds of limitations or risks that may be in play in AI outputs.

The motivation for this is obvious. If AI companies&nbsp;were transparent about how all this works—educating users on all those unsexy, finicky backend technical processes that directly impact the reliability of AI outputs—demand for the product would drop.
The principle here is roughly the same one that buoys restaurant demand. No one&nbsp;running a restaurant wants customers to&nbsp;see the state of the kitchen or how the food is made. As long as customers are happily ignorant, restaurant proprietors and diners alike are content to&nbsp;barrel ahead without being bogged down by the gory (albeit important) details of sausage-making.

<h2>Building an Institutional Bedrock of Trust</h2>
For millennia, our medical models were rooted in “woo” and beliefs about the supernatural. The leap to empiricism in medicine via germ theory and other scientific advents was a monumental achievement for human civilization, and has led us to much more powerful and predictable understandings of how medicine works and how we can drive better healthcare outcomes.
For that reason, there are many reasons to be excited about AI’s role in evolving this modern medical model through statistical competencies and machine learning.

But AI technology could also pose a curious and ironic risk: surreptitiously inviting health care professionals to move away from the “classical” practice of medicine (where clinical value is arrived at through processes of observation, understanding cause/effect, tracking measurements, critical reasoning, etc), towards a “romantic” practice of medicine (answers obtained quickly but without an observable thread of reason, intuition, or data to support them). 
In fact, without pointed intervention, this seems like a likely outcome, since healthcare resources are expensive, in limited supply, and medical organizations are increasingly in the business of maximizing patient throughput rather than optimizing patient outcomes.

Robot Thinking – Responsible AI Development

Designers have a responsibility to leverage AI safely and mindfully when developing new products and services, especially in situations where the results can impact the physical and mental well-being of others.

This is why, if we are to develop the societal level of trust needed to move responsibly into an AI-enabled healthcare future, four elements need to be in place:
<ol><li>Regulation We need strong regulatory constraints around how AI is used, what kind of medical claims can be made, and how those claims will be validated. This will establish important, high-level guardrails around the usage and risk of AI tools in the clinical setting. </li><li>Data Validation Currently, product claims around clinical value must be validated through controlled studies that ensure safety and efficacy. AIs geared around outputting clinical advice should likewise be vetted through a rigorous approval process to ensure efficacy and minimize risk in specific areas of claimed value. </li><li>Data Transparency Consumers and healthcare providers need a high-level of visibility to AI data inputs, provenance, quality, and training. This will provide the consumer or healthcare provider with the ability to independently gauge the quality of AI inputs for intended outputs and ensure that consumers and healthcare providers can meaningfully “shop around” regarding AI tools to be used in patient care. </li><li>User Education We need widespread efforts to technically educate those responsible for implementing AI in a healthcare setting about what AI is, where outputs come from, how to evaluate the validity of outputs, and how to understand sources of risk. This will build consumer/HCP intuition around limitations and risks, as well as introduce a sense of healthy criticality around results whenever AI is used.</li></ol>
There is no question that the addition of AI has the power to help us tremendously in the continued development of our modern model of medicine.&nbsp;
The peril is that the tech world often encourages the public to run before it can walk—generally because it is strategically advantageous for gaining commercial relevance and market share, not because it is ideal or safe for the public.
But healthcare is a unique product with unique implications. We cannot allow profitability and speed-to-market to be the primary drivers of our healthcare future.&nbsp;
We must view the emerging AI revolution with cautious and sober eyes and take the necessary action to ensure that AI reinforces and enhances, rather than undermines, the rationality embodied by modern medicine. 
I’d love to hear what you think! Let me know on my <a href="https://www.linkedin.com/posts/rahulkamath_healthcare-artificialintelligence-technology-activity-7075479821882949633-UNk6/" target="_blank">LinkedIn post</a>.

Learn more about <a href="https://www.delve.com/services" target="_blank">Product Development & Design Services at Delve.</a>

SEO Text: Our Services - Product Design and Development at Delve (3)

Trust ai healthcare motorcycle hero 352px

To achieve societal trust and move responsibly into an AI-enabled healthcare future, we need to put four elements in place.

Contact Us | Technology

Health & Wellness

Technology

Home

Insights

Four Elements to Achieve Trust in AI-Enabled Healthcare

Breadcrumbs

A full-service product design and strategic innovation consultancy.

Delve

To achieve societal trust and move responsibly into an AI-enabled healthcare future, we need to put four elements in place.

description

generator

kSwNn6qD4B-4HloWnWMUBWQNpBx2_mPi3nMqgsh6ato

google-site-verification

trust, responsibly, ai-enabled, healthcare, future, need, societal, elements, achieve, place

keywords

https://cms.delve.com/assets/images/_1200x630_crop_center-center_82_none/trust-ai-healthcare-motorcycle_hero-352px.jpeg?mtime=1686259063

https://linkedin.com/company/delvedesign/

https://www.delve.com/insights/4-elements-to-achieve-trust-ai-enabled-healthcare

referrer

robots

twitter:card

twitter:creator

twitter:description

https://cms.delve.com/assets/images/_800x418_crop_center-center_82_none/trust-ai-healthcare-motorcycle_hero-352px.jpeg?mtime=1686259063

twitter:image

twitter:image:alt

twitter:image:height

twitter:image:width

twitter:site

twitter:title

{"title":{"title":"Four Elements to Achieve Trust in AI-Enabled Healthcare | Delve"}}

Predict the Future

Privacy Policy

FCOI Policy

Services

Case Studies

Testimonials

What We Do

About Us

Our People

Join Our Team

Careers

Newsroom

Who We Are

Boston

Madison

Philadelphia

San Francisco

Our Studios

Partner with Delve

Contact Us

Contact

Vamsee Pamula, Founder &amp; President, Baebies

Read informative articles from Delve's team of experts. Discover innovative strategies and insights about product design, engineering, and development.

Delve Insights: Product Innovation & Development Blog

innovation, product, design, advice, development

https://cms.delve.com/assets/images/_1200x630_crop_center-center_82_none/Delve_titlecard.png?mtime=1654035558

https://cms.delve.com/assets/images/_800x418_crop_center-center_82_none/Delve_titlecard.png?mtime=1654035558

Four Elements to Achieve Trust in AI-Enabled Healthcare

Four Elements to Achieve Trust in AI-Enabled Healthcare

Related Insights