Dr Rob Lavis – Group Medical Director
25 years on from my first attendance at the Royal College of Surgeons to sit my Membership exam viva, I had the pleasure of chairing the launch of Medica’s joint white paper with Qure.ai in that very same venue! 25 years ago I was handed an IVU to comment on as a surgical SHO in my MRCS exam. In Medica’s 20th year, I was privileged to chair an eminent panel of radiology AI specialists discussing topics from research to ethics. What a contrast!
The Medica and Qure.ai white paper was presented by Surabhi Srivastava from Qure.ai. 1315 non contrast CT Head studies with acute presentation histories were analysed by the qER algorithm with feedback on performance by radiologists in terms of true or false positive/negative and classification of any haemorrhage (SDH, IPH, EDH, SAH). 112 studies were classified as showing acute haemorrhage. Overall agreement between the radiologists and the algorithm was seen in 93.5% of cases with a negative percent agreement of 94.3%. 9 of the 16 false negative cases related to subtle subarachnoid haemorrhage. The commonest cause of a false positive study was misinterpretation of movement artefact. The qER algorithm will continue to be used as a bleed detection and prioritisation tool as Medica deploy further algorithms for which qER has accreditation. The performance data suggest that it is performing at a level similar to a human expert, but with specific weaknesses that need to be recognised by users.
Dr Neelan Das described an epic 4-year hike to the top of his AI Everest – from concept to deployment and assessment in the real world. He guided us through the essential five steps to success so that fast followers only have to climb Helvellyn! Those steps are: identifying a specific need that an algorithm could answer (as opposed to choosing an algorithm and fitting it to your system) engaging all stakeholders, identifying goals, pre-deployment validation of an algorithm and go live/surveillance. He showed the qXR CXR tool from Qure to have a high sensitivity of 0.93 but a specificity of .51. Neelan noted that this would mean that in those cases identified as normal by the algorithm, 9% would actually be abnormal. Knowing how an algorithm behaves in your population is mandatory in order for radiologists and clinicians to put it in context.
Dr Sarim Ather spoke of his experience in the Research and Development aspects of AI and the importance of post deployment monitoring. Whilst careful choice of a tool is paramount, it must be assessed in the intended environment in order to assure users of reproducibility and performance. Sarim also highlighted the potential for AI tools to undertake healthcare screening on studies including osteoporosis assessment and coronary artery calcification.
In a captivating session Dr Amrita Kumar took us on a tour through the principles of a comprehensive governance framework in which to wrap AI deployment. From stakeholder engagement, through steering group oversight and monitoring, to public engagement and ethical considerations and staff training all facets need to be considered. One key message was that a tool should be robust. The characteristic of robustness in AI means that an algorithm must recognise when it is challenged with incorrect data without ‘falling over’, allowing continued service.
Following the keynote presentations Andrew Cannon, Medica CEO chaired a lively Q and A session exploring a wide range of topics. The panel explored the issues of potential legal challenges to the use of AI outputs and the future of autonomous AI tool reporting. There is no formal legal answer yet on the subject of liability related to an algorithm missing a pathology or a radiologist overruling an AI flagged finding as a false positive. We will watch this space with interest. There are legislative measures in the form of IRMER (2017) in place currently that prevent autonomous AI radiology reporting but this could change in the future. The audience challenged the panel for a timeline of when it would happen. The answers were interesting and ranged from ‘why should it not happen now if it is acceptable to have some X-rays unreported/auto reported’ to somewhere in the region of three to five years. It was also pointed out by the panel that we use a human as the gold standard with which to compare AI. I will leave you to consider whether that is a legitimate standpoint!
Thank you to the Medica team for arranging a thought-provoking evening, to Qure.ai for their support in generating the white paper, to our speakers for some excellent insights into their AI experience and to the audience for joining us and participating in a fantastic evening.
Dr Robert Lavis is a consultant radiologist and is the Chief Medical Officer of Medica. He has interests in medical education, oncology imaging, research imaging and AI. He has a background of surgical training (and notes that he passed his MRCS exam at first sitting!).
