In a groundbreaking move towards revolutionizing image editing, Apple has unveiled an upcoming feature that promises to bring unprecedented capabilities to iPhones. The tech, known as MGIE (MLLM-Guided Image Editing), is the result of a collaborative effort between Apple and researchers from the University of California, Santa Barbara.
The recently published research paper sheds light on MGIE’s capabilities, describing it as an advanced editing software capable of “Photoshop-style modifications.” Unlike traditional image editing tools, MGIE leverages a multimodal large language model (MLLM) that processes both text and images simultaneously. This innovative approach allows users to provide explicit instructions through text prompts for various modifications, from simple tweaks like cropping to more complex edits such as object removal.
VentureBeat’s report highlights the remarkable cross-model understanding exhibited by MLLMs, emphasizing their effectiveness in image editing. Despite their potential, such models have not been widely implemented in mainstream editing software until now.
The MGIE workflow is straightforward – users upload an image to the AI engine and provide clear, concise instructions for desired changes. For instance, users can instruct MGIE to “make the sky more blue” for a picture of a sunny day. However, users may need to provide explicit guidance to achieve the desired results.
Currently available as an open-source project on GitHub, MGIE offers users access to code, data, pre-trained models, and a tutorial notebook for utilizing the AI in editing tasks. Additionally, a public web demo is available on the collaborative tech platform Hugging Face, allowing users to experience the capabilities of MGIE firsthand.
During our evaluation, we selected an image of a cat sourced from relaythat and directed MGIE to implement various modifications. The outcomes were satisfactory for the most part. For example, when tasked with altering the background to pink, MGIE demonstrated acceptable performance in our test.
While MGIE is generating significant buzz, the big question is whether this latest technology will find its way into future iPhones or iOS 18. Apple CEO Tim Cook has hinted at the integration of AI tools into Apple devices later this year, without providing specific details. Observers speculate that MGIE could evolve into Apple’s version of Google’s Magic Editor, a feature capable of completely transforming the contents of an image.
It’s important to note that MGIE is still a work in progress, and its outputs may not be flawless. As demonstrated in sample images, there is room for improvement, but the potential for this AI-powered editing tool to redefine the landscape of image editing is evident. As tech enthusiasts eagerly await further developments, the public demo on Hugging Face provides a glimpse into the exciting possibilities that Apple’s latest innovation brings to the world of image editing.
