StabilityAI is working on tech to generate images just by thinking; no prompts needed: MindEye

Just a year ago, image generation via prompts was the biggest thing in the world. Everyone was talking about it, and now it seems like it’s soon going to be a thing of the past. StabilityAI’s MindEye is a new approach for reconstructing and retrieving images directly from fMRI brain activity. On the other hand, Stability AI’s new AI model can turn 2D images into 3D videos.

Functional magnetic resonance imaging (fMRI) is a neuroimaging technique that measures brain activity by detecting changes in blood flow. This technology’s role is to map brain function and evaluate potential treatments for neurological conditions.

MindEye builds upon a dataset of brain activity recordings from participants who viewed a series of static images within an MRI scanner. The research team trained the system to analyze these recordings and either retrieve the original image from a pool of candidates (retrieval) or generate a reconstruction of the viewed image.

The researchers demonstrated MindEye’s ability to outperform previous methods in image retrieval tasks, achieving over 90% accuracy in identifying the original image from a pool of candidates. For reconstruction, MindEye uses pre-trained generative models

MindEye can be applied across various fields. In the medical field, its ability to reconstruct visual perception from brain activity might allow diagnostic and assessment methods, particularly in cases where patients struggle with communication. MindEye’s potential for real-time analysis looks for improved brain-computer interface performance.

The team highlights limitations associated with data collection, including the lengthy scan times required and the potential for noisy data due to participant movement or inattentiveness.

More here.