Physics and art in 2½D

We live in three spatial dimensions – a good thing, since 2D and 1D are less interesting than 3D. The only problem is in knowing how to artistically portray the 3D world on 2D paper or canvas. No-one understood how to do this until about 1415, when the Renaissance architect Filippo Brunelleschi invented linear perspective to display 3D scenes on 2D surfaces. But even “flat” artworks extend slightly into 3D. The underlying surface may impart texture, and artists create a complex coated surface when they build up layers of pigment on their images.

This texture is a material record of the artist’s effort that makes an original work unique, but does not show up well in ordinary photos and screen images. Now, though, applied optics and sophisticated computation make it possible to directly experience the surface topography of an artwork on a flat display screen. Loïc Baboulaz, an imaging scientist at the Swiss Federal Institute of Technology (EPFL) in Lausanne, has led a project since 2012 to improve the digital display of artworks.

Digital display in general has made huge strides in applications such as computer-generated imagery (CGI), which creates fantastic, but seemingly real scenes in films. This is done by developing the “bidirectional scattering distribution function” (BSDF) – for every point in a scene. The BSDF shows how reflection and scattering affect a light ray entering from an arbitrary direction, to produce the outgoing ray that enters a viewer’s eye. The function can even show how the scene would look if the light source is shifted.

In CGI, the BSDF is derived from optical theory, from mathematical models of the shapes, and from the optical properties of objects in the imaginary scene. The EPFL scientists, however, wanted a BSDF based wholly on actual data from an artwork. In 2014 they showed how to construct the light transport matrix (LTM), a digitized BSDF, for a real nearly planar object – in this case, a pane of stained glass. They illuminated it from different directions and recorded the results with a high-resolution camera. In principle this process picks up all the object’s interactions with light through its surface, as well as subsurface reflections, scattering and refraction (including irregularities). This could create an LTM with a staggering number of elements. But in practice, a flat object with shallow surface topography generates a sparse matrix with relatively few elements, hugely reducing the computational power needed to manage the data.

This approach is used by Artmyn – a start-up company in Saint-Sulpice, Switzerland, co-founded by Baboulaz, who is now its chief technology officer. Artmyn produces “2½D” reproductions of artworks by illuminating them with light-emitting diodes at different visible and ultraviolet wavelengths, generating thousands of images at high resolution and colour fidelity. Its proprietary software turns up to a terabyte of data into an image that can be displayed and manipulated in real time, on a computer or mobile device. The key feature is that the lighting can be varied to produce 3D-like highlighted peaks and shadowed valleys in the surface topography.

To illustrate exactly what this technology can do, Artmyn collaborated with Sotheby’s auction house to digitize Marc Chagall’s Le Printemps, a 1975, 46 × 55 cm oil painting of a couple embracing, flowers, a donkey and a village in shades of red, green and blue. The interactive online display can be adjusted such that you can view everything from the full painting, down to every fleck of colour Chagall applied as it clings to the weave of the canvas – illustrating that, as Picasso noted, Chagall truly “understands what colour really is”. Other Artmyn displays, from the Bodmer Foundation in Switzerland, show a Sumerian clay tablet circa 2000 BC covered with cuneiform script, and Papyrus 66, an Egyptian manuscript of the New Testament from 200–300 AD. Changing the lighting emphasizes the symbols impressed into both artifacts and shows cracks and variations that give a sense of their materials and their great ages.

Surprisingly, another digitization from Bodmer is a physics image – page 48 of Isaac Newton’s original Philosophiæ Naturalis Principia Mathematica (1687) with text and a diagram related to centripetal force. Zooming in and changing the light direction shows faint reversed images of what is printed on the underside of the page, its texture, and distortions in the paper that indicate its age and use. This particular copy belonged to Newton’s contemporary Gottfried Leibniz, who independently invented calculus. His cramped but clear notes in the margin hint at his presence, along with several reddish spots. Leibniz was known to read while smoking in bed, and the marks are burns left by cinders from his pipe.

Would copies that are identical down to microns devalue the magic of the work? Will owning an “original” take on a whole new meaning?

For all the pleasure these images give to art lovers and scholars, the owners of the original works need to know that the Artmyn process is safe. To reassure them, Artmyn measured the temperature of artworks while being scanned and found no increases. Even then, Baboulaz told me, some owners are not ready to share their digitized art with the world. I wonder what will happen if, in the future, the stored digital data are used to create exact physical copies via 3D printing. Then we would have to ask if copies that are identical down to microns would devalue the magic of the work that felt the artist’s touch, and if owning an “original” would take on a whole new meaning.