Next Rembrandt

01 GATHERING THE DATA

To distill the artistic DNA of Rembrandt, an extensive database of his paintings was built and analyzed, pixel by pixel.

FUN FACT:

150 Gigabytes of digitally rendered graphics

BUILDING AN EXTENSIVE POOL OF DATA

t’s been almost four centuries since the world lost the talent of one its most influential classical painters, Rembrandt van Rijn. To bring him back, we distilled the artistic DNA from his work and used it to create The Next Rembrandt.

We examined the entire collection of Rembrandt’s work, studying the contents of his paintings pixel by pixel. To get this data, we analyzed a broad range of materials like high resolution 3D scans and digital files, which were upscaled by deep learning algorithms to maximize resolution and quality. This extensive database was then used as the foundation for creating The Next Rembrandt.

“Data is used by many people today to help them be more efficient and knowledgeable about their daily work, and about the decisions they need to make. But in this project it’s also used to make life itself more beautiful. It really touches the human soul.”

– Ron Augustus, Microsoft

02 DETERMINING THE SUBJECT

Data from Rembrandt’s body of work showed the way to the subject of the new painting.

FUN FACT:

346 Paintings were studied

DELVING INTO REMBRANDT VAN RIJN

• 49% FEMALE
• 51% MALE

Throughout his life, Rembrandt painted a great number of self-portraits, commissioned portraits and group shots, Biblical scenes, and even a few landscapes. He’s known for painting brutally honest and unforgiving portrayals of his subjects, utilizing a limited color palette for facial emphasis, and innovating the use of light and shadows.

“There’s a lot of Rembrandt data available — you have this enormous amount of technical data from all these paintings from various collections. And can we actually create something out of it that looks like Rembrandt? That’s an appealing question.”

- Joris Dik, Technical University Delft

BREAKING DOWN THE DEMOGRAPHICS IN REMBRANDT’S WORK

To create new artwork using data from Rembrandt’s paintings, we had to maximize the data pool from which to pull information. Because he painted more portraits than any other subject, we narrowed down our exploration to these paintings.

Then we found the period in which the majority of these paintings were created: between 1632 and 1642. Next, we defined the demographic segmentation of the people in these works and saw which elements occurred in the largest sample of paintings. We funneled down that selection starting with gender and then went on to analyze everything from age and head direction, to the amount of facial hair present.

After studying the demographics, the data lead us to a conclusive subject: a portrait of a Caucasian male with facial hair, between the ages of thirty and forty, wearing black clothes with a white collar and a hat, facing to the right.

03 GENERATING THE FEATURES

A software system was designed to understand Rembrandt’s style and generate new features.

FUN FACT:

500+ Hours of rendering

MASTERING THE STYLE OF REMBRANDT

In creating the new painting, it was imperative to stay accurate to Rembrandt’s unique style. As “The Master of Light and Shadow,” Rembrandt relied on his innovative use of lighting to shape the features in his paintings. By using very concentrated light sources, he essentially created a “spotlight effect” that gave great attention to the lit elements and left the rest of the painting shrouded in shadows. This resulted in some of the features being very sharp and in focus and others becoming soft and almost blurry, an effect that had to be replicated in the new artwork.

“When you want to make a new painting you have some idea of how it’s going to look. But in our case we started from basically nothing — we had to create a whole painting using just data from Rembrandt’s paintings.”

- Ben Haanstra, Developer

GENERATING FEATURES BASED ON DATA

To master his style, we designed a software system that could understand Rembrandt based on his use of geometry, composition, and painting materials. A facial recognition algorithm identified and classified the most typical geometric patterns used by Rembrandt to paint human features. It then used the learned principles to replicate the style and generate new facial features for our painting.

CONSTRUCTING A FACE OUT OF THE NEW FEATURES

Once we generated the individual features, we had to assemble them into a fully formed face and bust according to Rembrandt’s use of proportions. An algorithm measured the distances between the facial features in Rembrandt’s paintings and calculated them based on percentages. Next, the features were transformed, rotated, and scaled, then accurately placed within the frame of the face. Finally, we rendered the light based on gathered data in order to cast authentic shadows on each feature.

04 BRINGING IT TO LIFE

CREATING ACCURATE DEPTH AND TEXTURE

Analyses

We now had a digital file true to Rembrandt’s style in content, shapes, and lighting. But paintings aren’t just 2D — they have a remarkable three-dimensionality that comes from brushstrokes and layers of paint. To recreate this texture, we had to study 3D scans of Rembrandt’s paintings and analyze the intricate layers on top of the canvas.

“We looked at a number of Rembrandt paintings, and we scanned their surface texture, their elemental composition, and what kinds of pigments were used. That’s the kind of information you need if you want to generate a painting by Rembrandt virtually.”

- Joris Dik, Technical University Delft

USING A HEIGHT MAP TO PRINT IN 3D

We created a height map using two different algorithms that found texture patterns of canvas surfaces and layers of paint. That information was transformed into height data, allowing us to mimic the brushstrokes used by Rembrandt.

We then used an elevated printing technique on a 3D printer that output multiple layers of paint-based UV ink. The final height map determined how much ink was released onto the canvas during each layer of the printing process. In the end, we printed thirteen layers of ink, one on top of the other, to create a painting texture true to Rembrandt’s style.

ORIGINAL: Next Rembrandt