Pupila is a project done in early 2018 by three people, Elaine Wang, Chenyu Zhang and me. I’m the group leader, responsible for overall planing of the project and solving technical problems, including programming and mechanical design.

Pupila is  a painting robot as well as an interaction installation. The installation gathers iris and eye contour information from users. It combines those data with existing painting styles of masterpieces in the database to generate a customized abstract painting. Spray guns driven by motors and a microcomputer are used for real-time painting on large rice paper.


Nowadays, appreciating artworks are no longer the privilege of the nobility. In arts galleries and museums,  everyone can have easy access to the most outstanding masterpieces of Van Gogh and Monet. However, few people without specialized education in art are engaged in artistic creation.

Thus, we come up with the idea of designing an interaction installation. Users can cooperate with it to create a unique artwork during the interaction process. The development of artificial intelligence enables us to use existed  algorithms to make our imagination come true.


We conduct online surveys to find out how interviewees think about painting robot and their dreaming artworks. Most of them show great interest in the combination of technology and art. Furthermore, we offer a series of masterpieces of different styles for interviewees to choose their preferences. Out of our expectation, most Chinese prefer Chinese monochromes (a kind of traditional Chinese wash painting) to oil paintings artworks. We also summarize some top wishes for the artwork:

  • the painting need to be enjoyable
  • a unified artistic style is preferred
  • the painting should have certain relationship with users
  • the painting should be understandable

Then, we study a couple of existing drawing robots. Some of them can draw very realistically just like literary sketching, others generate rather abstract paintings consisting solely of dots and patterns.


After taking existing products into consideration, we decide to mix the existing two kinds of painting style together. Our painting robot should be able to paint abstract paintings depicting concrete things. As for the artistic style, we decide to use spray guns instead of painting brushes to create similar artistic conception with Chinese wash paintings.

In order to make the painting closely related to users, we select humans’ eyes and pupils as the painting theme. Everyone has his eyes, beautiful and unique, which perfectly satisfies our need.

We name our robot ‘Pupila‘.


To collect information of eye contours as well as iris, we inserted a IR camera in a dark box. The box is equipped with IR distance sensors to tell if their is a user nearby. Once the sensor detects a passer-by, it’ll send a signal to the microcomputer. The signal drives the box to move up and down,  showing smiling faces and emitting sound, to attract the passer-by’s attention.

Once the user look into the box, the camera which is controlled by a light sensor will automatically take photos of the user’s eyes. The pictures are sent back to the computer for painting generation.


Pictures of eyes are color clustered to extract the shape and contour. The clustered pictures are then being compared with the masterpieces in the database to search for a most similar part in existing paintings. The brushing strokes of the matched masterpiece is then used for reference.In addition, to make the artwork unique for every user, the colors used and the embellishment in the painting are generated according to the standard code of iris recognition. 


We desire the painting process of the robot to be elegant and dramatic. So we design a 1-meter-high over hanging beam to load the spray gun. The spray gun can move horizontally on the beam with the help of a motor, while the beam itself can move vertically. The working principle is much like printers in offices.

The structure is mainly built with lego bricks. We make a few wooden parts to make the structure work more smoothly. A microcomputer remotes the motors to print the generated picture in the computer.


The following picture refers to the style of Zhao Wuji, a splash-ink artist, with a eye in the middle of the picture scroll and several embellishment around it.

Art creation requires both rationality and sensibility. Pupila is a preliminary attempt to find the balance between the two. As an interdisciplinary student, I’ll go on exploring the combination of  science and art.