Disney app turns coloured drawings into 3D characters

Disney app turns coloured drawings into 3D characters

A colouring book app devised by researchers at Disney can cause characters from drawings to leap from the page in 3D with the help of augmented reality.
A child colours a character, such as an elephant, on the book page normally, while a tablet or smartphone running the new app monitors the drawing.
Based on the child’s colouring, the app fills in colours in real-time on an animated 3D version of the elephant that is visible on the device’s screen and integrated into the video.
The app keeps the core focus on the traditional activity of colouring while offering a magical digital overlay that enhances engagement.
In user testing ¬ performed with adults rather than children in the study ¬ the researchers found that most users said the app increased their motivation to draw in colouring books and 80 per cent said the app increased their feeling of connection to a character.
To create the new experience, the researchers first created animated 3D virtual characters and then used custom software to generate 2D line-art representations of the characters for a colouring book.
The app, operating on a device with a camera viewing the user and the colouring book, automatically detects the character the user is colouring and displays the 3D version.
As the child applies colour to the 2D drawing, the app applies the same colour to the 3D character ¬ both to the areas visible in the 2D drawing and to the remainder of the 3D form not visible in the book.
Because the colouring occurs in real-time, the illusion is created that the user is also colouring the occluded areas, with similar texturing of the colour.
Determining how to apply colour to the occluded areas was one of the tougher problems to solve, said Robert W Sumner, a principal research scientist who leads the group on animation and interactive graphics at Disney Research.
Simply mirroring the user’s coloured strokes doesn’t work because the pattern of colours used for, say, a character’s face will not be appropriate for the back of the character’s head.
The colour also has to be continuous, so no seams can be seen between the visible areas and the occluded areas or where disparate portions of the textures meet. (AGENCIES)