Summer of Multimodal: Biomedical images!

2019-07-09
Organizer: Triangle Computer Vision and Machine Learning Group
Click here for registration info

We'll go through a number of recent papers that discuss multimodal approaches to understanding biomedical images. This will include research that combines multiple image domains like this one

http://arxiv.org/abs/1804.10916

as well as, with higher priority, papers that integrate images with genomic or EHR information. We'll start with this nice survey which lays out different approaches:

Deep Multimodal Representation Learning: A Survey, Guo et al. 2019.
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8715409

Xue et al. combine x-rays and doctor notes, and report superior note generation performance:
https://faculty.ist.psu.edu/suh972/Xue-MICCAI2018.pdf

Dou et al . report improved prediction through combining genomic and EHR information:
https://www.nature.com/articles/s41598-018-36745-x

We'll also step out of the biomedical image domain and look at joint learning of text and video clips:
https://arxiv.org/abs/1906.03327

And perhaps a few more papers if there's time!

We will also take a look at the PADCHEST-SJ dataset, which contains high-resolution chest X-rays along with corresponding EHR information,

https://arxiv.org/pdf/1901.07441.pdf

and see if we can draw any inspiration from it.

Lastly, quality pizza as usual will be provided by our generous host, LifeOmic.

• Important to know

Ground floor, first door on the right as you come in the main entrance of 3800 Paramount Pkwy, Morrisville.

----

Poster: triangletech