In mid-May, Google announced a partnership with Ellis Monk, an associate professor at Harvard, unveiling a more inclusive skin tone scale to better train Google's artificial intelligence systems and improve racial and color representation in its products.

Using the more diverse 10-shade scale is designed to allow Google's services to better see and understand images that feature people with darker skin tones. It's a response to problems in the system that Google and other companies have used to classify skin tones for people of color.

Read more: Google Adopts 10-Step Skin Tone Scale So Its AI Will Understand Diversity

You can see one example of this AI technology at work in the US -- the option to refine makeup-related queries by skin tone within Google search. Separately, the company also added filters tailored to darker skin in Google Photos. And that's just the start.

As Google broadens its use of the Monk Skin Tone Scale to new services and other parts of the world, there's a lot to learn about Google. Here's what you need to know about how it's trying to improve skin tone representation and how Google got there.

Photography can be biased toward lighter skin tones

In the 1950s Kodak dominated the film industry, selling the majority of color film in the US. If you used Kodak film to shoot, you would have to go to a Kodak photo lab to get it developed and printed, where you would inevitably buy more Kodak film to use.

Eventually, the US federal government stepped in and broke up its monopoly, and so Kodak developed a small printer that any film lab across the country could use in their stores to print Kodak film. As part of these kits, Kodak would also supply labs with reference prints to help film lab employees calibrate colors, shadows and light. These references were called Shirley cards.

These Shirley cards featured mostly brunette white women, and excluded pretty much everyone else. At this time, lighter skin was the chemical baseline for photo technology, and so these cards showcased that. As a result, darker skin tones were neglected, and so they were harder to expose properly, which helped to perpetuate the myth that darker skin is more difficult to photograph than lighter skin.

Google's Real Tone feature on the Pixel 6

Now, our digital smartphones have largely superseded analog film. Yet, the racial biases that existed back then are still around. Even in your camera app, lighter skin tones are still favored -- computational photography on smartphones is known to still overexpose and desaturate darker skin.

And that's why last year Google released its first AI-powered tool to help combat the racial biases in photography. Real Tone, exclusive for the Pixel 6 and Pixel 6 Pro, improves how darker skin tones show up in photos and videos. Google worked with a team of diverse image experts to break down the process of properly shooting darker skin tones, and better train the decision-making algorithms used on their phones.

For Real Tone, Google focused on six areas to improve the camera for the Pixel phones, to make it more inclusive and create more authentic, accurate portraits for darker skin tones:

• Face detection: The camera app is better at detecting faces with darker skin tones.
• Auto-white balance: Darker skin tones are shown with more nuanced color temperature.
• Auto-exposure: When adjusting exposure, darker skin tones won't appear too bright or too dark.
• Stray light: This reduces negative effects of stray light, which can make darker skin tones appear washed out.
• Face Unblur: Cut down image blur for people with darker skin tones in low light.
• Google Photos: Google Photos' auto-enhance feature works better for darker skin tones.

What is the Monk Skin Tone Scale?

During Google I/O 2022, Google announced that it's begun incorporating the Monk Skin Tone Scale into services, such as Google Search and Google Photos. The plan is to use the Monk Skin Tone Scale to help address the skin tone bias by better representing historically underrepresented skin tones.

The Monk Skin Tone Scale is a newer approach to categorize a more diverse range of skin tones. Google is using it during product development to create more representative datasets for training its AI models so they are more nuanced at detecting darker skin tones. 

Ellis Monk, a Harvard associate professor in sociology, developed the skin tone scale a decade ago, and Google adapted it for its own digital use. Monk has extensively researched how technology intersects with race and ethnicity, in particular areas of artificial intelligence, machine learning and computer vision.

At Google, the 10-shade Monk Skin Tone Scale replaces the Fitzpatrick scale, a dermatologist-created system designed for reading ultraviolet radiation exposure. The Fitzpatrick scale has only got six skin tones, and most of them are for lighter skin. Google and other tech companies have relied on the Fitzpatrick scale for years to classify skin tones for their algorithms and to assess how well those algorithms are working.

How is the Monk Skin Tone Scale being used?

For now, Google is using the Monk Skin Tone Scale to improve skin representation in two of its popular services: Google Search and Google Photos.

In Search, the Monk Skin Tone Scale will help find and present more relevant results for those with darker skin tones. For example, if you search for a makeup-related query, Google will present color chips and a variety of faces that let you narrow down your search results based on skin tone. That should help you find something like a "winged eyeliner tutorial" photo for darker skin tones.

Google also used the Monk Skin Tone Scale to improve Google Photos with a series of Real Tone filters designed specifically for darker skin tones. These filters were created with the help of image creators who have experience in accurately depicting darker skin tones in photographs, and are now available for use in the Google Photos app for iOS, Android and the web. If you go to the Filters tab in the Google Photos editor, you'll see new choices like Desert, Honey, Isla and Playa.

How else will Google use the new skin color scale?

Google hopes others will incorporate the Monk Skin Tone Scale as they label photos that appear on the web for image attributes like hair color and texture. Creators, brands and publishers can use these metadata options on their images and other web material so that search engines can better recognize them and surface them in searches when it's relevant.

Google released its Monk Skin Tone Scale classification system under liberal licensing terms, open to any researcher or company. Google hopes other tech companies will incorporate the scale into their own development processes.

And Google will continue working with Monk on improving the scale as well, including with tests to validate its use in other countries like Mexico, India, Nigeria and Brazil.