Virtual try-on for beauty and cosmetics: how it works
beauty virtual try-on detects the user's face, lips, eyes, hair or nails with the camera and applies the product (lipstick, eyeshadow, foundation, blusher, hair colour, nail varnish) in real time. For it to be believable, it has to reproduce the colour and the finish (matte, glossy, satin) well, and work with different skin tones and lighting conditions.
What can be tried on
| Category | Products |
|---|---|
| Lips | Lipsticks, glosses, lip liners |
| Eyes | Eyeshadows, eyeliners, lashes, brows |
| Face | Foundations, blushers, concealers, highlighters |
| Hair | Hair colour, highlights, hairstyles |
| Nails | Nail varnish, designs |
| Skincare | Diagnosis and routine recommendations (facial analysis) |
How it works
- Facial detection: the camera identifies dozens of points on the face and segments areas (lips, eyelids, cheeks).
- Real-time tracking: the application follows the movements of the face.
- Product application: a layer of colour is placed, with texture and finish.
- Adaptation to the light: the colour adjusts to the lighting so that it looks natural.
- Interaction: the user changes shade, compares, takes screenshots and buys.
What determines whether it looks good
Colour fidelity
The main challenge: the colour on screen must resemble that of the real product. It is calibrated with colour data and tests against physical samples.
Finish
Matte, satin, metallic, glitter or gloss are simulated with different material properties. Some finishes are harder to reproduce.
Inclusivity
The result depends on skin tone, face shape, facial hair, glasses and lighting. A good project tests with a diverse sample of people and conditions.
Performance
It must run smoothly on mid-range phones.
Privacy
The camera captures facial data, which are sensitive. Processing must happen on the device whenever possible, people must be clearly informed, and the GDPR must be complied with.
Benefits for beauty brands
- Try without opening the product: there are no contaminated testers or limited samples.
- Explore the full range (50 foundation or lipstick shades) with ease.
- Fewer doubts when buying online.
- Personalised recommendations based on facial analysis.
- An entertaining experience, and a shareable one.
- Data on the most tried shades and on trends.
- Lower sample cost in store.
Where to offer it
- Product page on the online store.
- Campaigns with QR codes on packaging, posters and magazines.
- Physical store: mirrors or tablets next to the shelf.
- Social media: brand filters.
- Events and launches.
Good implementation practices
- Start with the categories that have the most impact (lipsticks, foundations, eyeshadows).
- Gather colour data for each product (spectrophotometry or samples).
- Define finishes and check their fidelity.
- Test with a diversity of skin tones, faces and lighting.
- Integrate with purchase: "Add to basket" from the virtual try-on.
- Add comparison ("before and after", several shades).
- Measure use and conversion.
Limits to bear in mind
- The colour on screen depends on the phone's calibration.
- Real texture (creamy, dry) is not perceived.
- Real application can vary by skin type.
- It is a visual approximation, not a guarantee of the result.
Key ideas
- Beauty virtual try-on detects areas of the face and applies the product in real time.
- Colour fidelity, finish and inclusivity are the main challenges.
- It works on the web, in a physical store, on social media and in campaigns.
- It must respect the privacy of facial data.
- It is best implemented in phases and with measurement.
Frequently asked questions
How accurate is the colour?
It can be very good with calibration, but it is always an approximation.
Does it work with all skin tones?
It should. Ask for tests with diverse samples before you launch.
Does it replace in-store testers?
It complements them, and it reduces shared use of samples.
Can I offer personalised recommendations?
Yes, by combining the virtual try-on with facial analysis and recommendation rules.
More guides on this topic
How to add AR product visualisation to an online store
A guide to adding AR product visualisation to an online store: 3D models, formats, integration on the product page, platforms, performance, analytics and best practice.
How to create an interactive retail campaign with QR codes and augmented reality
A guide to creating an interactive retail campaign with QR codes and augmented reality: concept, touchpoints, mechanic, reward, legal points, launch and measurement.
How to measure virtual try-on performance
Metrics for measuring a virtual try-on: usage rate, time, conversion, order value, returns and satisfaction. A/B test, attribution and how to avoid bias.