Defining the Undefinable: Opening the Black Box of AI in HealthCare

The processes that go on inside the black box are so complex, we can’t always inherently understand or explain them. The question is: Do we need to? In the following podcast, Sava and Nemanja will discuss critical questions around Black Box of AI in Health.

Podcast guest: Nemanja Kovacev, MedTech Expert at HTEC Group  

Podcast host: Sava Marinkovich, Head of HealthTech/MedTech at HTEC Group  

“Black boxes” are increasingly becoming a focal point in technological developments in the healthcare arena. Research in deep learning often combines various health data streams and is creating the potential to redefine healthcare delivery and improve patient care. Such solutions, however, can struggle with the issue of explainability and thereby also adaption. Let’s take AI, for instance. Resistance to the utilization of medical AI is driven by both the subjective difficulty of understanding algorithms, an occurrence also known as a ‘Black Box”, and by an illusory subjective understanding of human medical decision-making.  

In other words, we have difficulties understanding complex algorithms, and we as humans often think we understand complex systems more than we really do. For instance, diagnoses are categorical representations of various medical conditions — although successful in classifying, they are often devoid of clinical context, so the judgment made by a such system is not trusted by clinicians. The processes that go on inside the black box are so complex, we can’t always inherently understand or explain them. The question is: Do we need to? 

In the following podcast, Sava and Nemanja will dive deeper into the problem of Black Box of AI in Health and discuss these critical questions:  

• There are still areas in AI that remain in the Wild West. What do organisations need to do in practice to achieve quality data? 
• How valuable is the data once you create a model? 
• AI applications are in black boxes – closed systems which give no insight into how they reach their outcomes. How can this black box be unlocked? 
• Why is white box important and to whom? Does the white box serve to convince the practitioners? What are the reasons we should have a white box at the first place? 
• To what extent can we have explainable AI? 
• Will practitioners have to accept the fact that some things cannot be explained? 
• Will development teams have to make one step more to maybe build the white box in parallel?