Optical microscopy imaging is one of the most powerful approaches for probing the organization of brain tissue at a (sub)cellular level. Several distinct microscopy techniques are available, each with their own advantages and disadvantages in terms of spatial resolution, field-of-view, molecular specificity, and invasiveness of sample preparation. The SMART BRAIN project seeks to advance the complementary measurement of neuronal tissue by different optical imaging technologies, and to develop the in silico integration of images collected by different methods by means of data-driven multimodal image fusion. This process will combine advantages from different source modalities into a single predictive imaging modality. Specifically, we will integrate 3-D multiphoton microscopy (MPM) data with 3-D light-sheet microscopy (LSM) measurements, to deliver a single ‘fused’ predictive imaging modality that combines the molecular specificity of LSM with the non-invasive sample preparation of MPM. The fused modality will deliver LSM-grade information while avoiding LSM’s invasive clearing procedure, in case tissue needs to be safeguarded for other analyses or for broad LSM based molecular targeting post-MPM. Furthermore, we will explore fusion of MPM with superresolution microscopy (STED), to predict 2-D multiphoton tissue observations at up to 10 times the diffraction limit. This project’s unique multidisciplinary approach has the potential to provide an organizational description of brain tissue that surpasses what any single imaging technique can provide. The proposed approach is based on advanced machine learning algorithms originally developed for integration between 2-D imaging mass spectrometry and standard 2-D stained microscopy, with proof of-concepts demonstrated in murine brain tissue. The SMART BRAIN project will extend these methods for fusion of multiphoton, light-sheet, and STED microscopy. The goal is to discover relationships between observations in the different modalities, to mathematically model them, and then to use them for prediction of morphological information at spatial resolutions beyond instrumental limitations or in tissue areas where not all data types are measured. Fusion of different datasets will be initiated on murine brain tissue, and subsequently extended to human brain samples. Specimens, provided by the neurosurgery partners in the consortium will be obtained from patients showing temporal lobe epilepsy and block of hippocampus and amygdala, as well as from autopsy of healthy subjects, yielding precious information on human brain morphology. The SMART BRAIN project will (i) provide a new multi-modal approach exceeding the state of the art in single-modality optical microscopy, (ii) support the HBP consortium with a previously unavailable fused data type and with unprecedented information on human brain morphology, and (iii) provide a novel methodology for combination with clinical imaging for advanced diagnostics.
