CIN COLORiT AR
Paint your walls with AR
Like in many things, there’s no need to create something from scratch, as it has already been done before. In the beginning, we hoped this would happen for this project, so we confidently searched for AR components in react libraries looking for the perfect base to start.
Well… it was not the case. The problem we ran into when searching through react libraries is that most of the elements were discontinued/outdated. We also found a version of it done (and shared publicly) by the software development community, but it also wasn't working as we wished… so we ruled that option out.
As the first hypothesis had some limitations, we needed to go in a different direction. We decided to do it from scratch (oh yeah!).
After some brainstorming, we came up with a new approach: accessing the resolutions of the smartphone’s camera and opting for taking continuous screenshots. From there, every screenshot would be worked natively by a function of ours.
How does the algorithm work, step by step:
- We understood that when we selected the surface we wanted to paint, it would paint better (more accurately) if we blurred it first. For this reason, we decided to blur the images, which made the different shades of the same color blend into one (avoiding generating too much noise when generating the mask);
- Recognizing the different colors present in the image, we made a mask that identified the objects through their colors. This way, when coloring the surfaces the color would go around them and not over them;
- After making these two versions of the image (blur and mask) we created a third version. This one is for the painted zone of the image.
This third version is the part that we show the user.
When they are happy with the result and click “save” the image goes through some more color and shadow validations, we join this version with the others and … ta-da! In the end, we just save it on the user’s device gallery.
Basically, that’s the whole process. Of course, we have little tweaks here and there, for example, the painting surface. Sometimes it painted more than necessary, sometimes less… Here we had a proccess of trial and error, till we were really happy with the end result.
One of our biggest concerns was the performance, as we wanted the image to constantly update/refresh. But we quickly understood that speed was the enemy to perfection. So we decided, together with the client, that the best solution was to make the updating of the image slower, and do an upgrade on the code. Having the possibility to make everything slower, we were now able to increase the blur (as it takes some time to do on the mobile phone).
Finally. Here’s our happy ending to this story. And you can test it yourself: CIN COLORiT now with AR is live!
