We'll go through a number of recent papers that discuss multimodal approaches to understanding biomedical images. This will include research that combines multiple image domains like this one
as well as, with higher priority, papers that integrate images with genomic or EHR information. We'll start with this nice survey which lays out different approaches:
Deep Multimodal Representation Learning: A Survey, Guo et al. 2019.
Xue et al. combine x-rays and doctor notes, and report superior note generation performance:
Dou et al . report improved prediction through combining genomic and EHR information:
We'll also step out of the biomedical image domain and look at joint learning of text and video clips:
And perhaps a few more papers if there's time!
We will also take a look at the PADCHEST-SJ dataset, which contains high-resolution chest X-rays along with corresponding EHR information,
and see if we can draw any inspiration from it.
Lastly, quality pizza as usual will be provided by our generous host, LifeOmic.
• Important to know
Ground floor, first door on the right as you come in the main entrance of 3800 Paramount Pkwy, Morrisville.----