FRIDA was developed at Carnegie Mellon’s Robotics Institute.

Carnegie Mellon University’s Robotics Institute has developed a new project called FRIDA, which stands for Framework and Robotics Initiative for Developing Arts. FRIDA is an AI-powered robotic painting system that collaborates with humans to create works of art.

FRIDA uses AI models similar to those used by OpenAI’s ChatGPT and DALL-E 2 to generate text or images in response to prompts. Users give instructions to FRIDA through a text description, submitting other artworks to guide its style, or a photograph, and tell it to paint its AI version of it. The team is also experimenting with other inputs, such as audio. (Carnegie Mellon Univ)

FRIDA is not an artist but a system that artists can collaborate with. The robot simulates how it would paint an image with brush strokes and uses machine learning to evaluate its progress as it works. However, FRIDA’s final outputs are often impressionistic and whimsical, with bold, somewhat imprecise brushstrokes that are not expected from an AI guided robotic device.

FRIDA uses AI and machine learning several times during its artistic process.

First, it spends an hour or more learning how to use its paintbrush. Then, it uses large vision-language models trained on massive data sets that pair text and images scraped from the internet, such as OpenAI’s Contrastive Language-Image Pre-training (CLIP), to understand the input. FRIDA’s team also seeks to address some of the limitations in current large vision-language models by continually refining the ones they use.

FRIDA’s team includes students and faculty members from various departments and backgrounds, and they seek to promote human creativity through FRIDA rather than replace artists. The team is working on developing automatic paint mixing and reducing the simulation-to-real gap to improve the robot’s physical capabilities.

“FRIDA is a project exploring the intersection of human and robotic creativity,” said a faculty member in a statement. “FRIDA is using the kind of AI models that have been developed to do things like caption images and understand scene content and applying it to this artistic generative problem.”

Related Story:  Microsoft To Integrate ChatGPT’s Tech Into Bing For “Conversational” Searchv