The Museum of Art Pudong (MAP) in Shanghai has been noticeably more crowded than usual in recent days.
Inside the galleries, some visitors hold up their phones to conduct live video calls with Doubao, listening as the artificial intelligence chatbot offers real-time interpretations of ceramic motifs from the Louvre’s collection. Others photograph paintings by Pablo Picasso and send the images through, receiving layered written explanations just seconds later.
When an AI app with more than 100 million daily active users meets 300 cross-continental works from the Louvre and 80 representative paintings by Picasso, a cross-industry experiment is unfolding at MAP that brings together technological capability, cultural empowerment, and user growth.
On January 20, ByteDance’s Doubao announced a partnership with MAP, becoming the official AI docent for two major international exhibitions: “The Wonder of Patterns: Masterpieces from India, Iran and the Ottoman World,” and “Picasso Through the Eyes of Paul Smith.” Through exclusive data collaboration and targeted search optimization, the partnership further improves Doubao’s accuracy in artwork recognition and interpretation.
This marks the first time an AI product has served as an official interpretation tool for an art museum.
As the official AI docent for the two flagship exhibitions, Doubao’s arrival reflects a longstanding industry reality: high-quality art interpretation has always been scarce. Museums and galleries have limited numbers of professional docents, and guided tours often require dozens of visitors to listen together, leaving little room for personalization. Expert-level art historical interpretation is even harder to access. Faced with artworks that span centuries and civilizations, many visitors are left with little more than a cursory, walkthrough experience. Cultural accessibility is not only about bringing more people into museums, but also about ensuring that everyone can obtain high-quality interpretation at an affordable cost.
Museum interpretation has long been a challenging scenario. It requires distinguishing between highly similar objects, such as a 15th-century Iranian peony dish and a Ming-era Yongle blue-and-white peony plate. It also demands accurate identification of niche artifacts with limited public documentation, while coping with constantly shifting viewing angles, shaky footage, and changes in lighting as visitors move through galleries.
Through exclusive data collaboration and targeted search optimization with MAP, Doubao draws on the visual reasoning and stable recognition capabilities of its Seed 1.8 model to achieve what the team describes as “detective-style” precision. The system can even respond accurately to softly spoken questions, addressing many of the long-standing pain points of traditional guided tours.
According to a project lead, Seed 1.8 has reached state-of-the-art performance in video perception, streaming, and video understanding. It moves beyond the traditional “take a photo, send it, ask a question, then repeat” workflow. Instead, it is said to continuously digest and analyze the viewer’s changing perspective and surroundings, enabling sustained, natural, humanlike dialogue.
Zhu Jun, a vice president at ByteDance, said interaction between AI and users is fundamentally conversational. During exhibitions, he said, Doubao is designed to engage visitors through empathetic prompts and heuristic dialogue, drawing out their existing feelings and experiences to create a more participatory process of understanding.
While touring the exhibitions, visitors can ask Doubao to interpret works from multiple angles, including artistic style, historical context, creative techniques, and cultural significance. For instance, when standing in front of the Picasso gallery, a visitor might ask how a painting conveys its sense of calm.
Doubao responds by situating the work within its 1932 context, explaining how soft curves and bold color blocks work together to create an intimate and tranquil reading scene. It also discusses how Picasso’s muse at the time, Marie-Therese Walter, influenced his style during this period, and how he struck a balance between figuration and deformation.
Doubao’s exploration of the arts did not happen overnight. Previously, it had partnered with seven national-level museums, including the National Museum of China and the Henan Museum. Through exclusive data collaboration and targeted search optimization, Doubao steadily improved the accuracy of its recognition and interpretation capabilities.
For the two exhibitions at MAP, Doubao further refined its algorithmic models. It can now reportedly identify 80 representative works by Picasso and 300 artifacts from the Louvre’s collection, while also factoring in exhibition design and visitor behavior to deliver more context-aware interpretation. Whether through real-time interaction during video calls or detailed text explanations after image recognition, the aim is to make art interpretation readily accessible.
From museums to art galleries, from ancient artifacts to modern art, Doubao’s AI-guided interpretation is helping build a more inclusive and accessible public cultural space. It not only addresses the shortage of art interpretation resources, but also reshapes the relationship between art and the public. Art is no longer a rarefied pursuit reserved for a select few, but a form of cultural nourishment available to everyone.
When technology breaks down barriers to understanding and art interpretation becomes truly accessible, cultural inclusion begins to take on tangible form.
KrASIA Connection features translated and adapted content that was originally published by 36Kr. This article was written by Li Xiaoxia for 36Kr.