Saturation Masks 1: The problem with Photoshop’s HSB/HSL filter

In addition to my work advancing luminosity masks, I’ve also been a big proponent of saturation masks. I’ve used them since 2007 when I first started writing about saturation masks and saturation painting. The way I originally recommended making them was Adobe’s HSB/HSL filter. However, Adobe stopped including the filter in Photoshop for a number of years, so I changed to a different and ultimately better method to make saturation masks. Adobe has again included the HSB/HSL filter in the Filter menu of Photoshop CC. Now that it’s returned, this is probably a good time to review why it should NOT be used to make saturation masks.

The way to run the HSB/HSL filter is to create a stamp-visible layer of the image and then convert it to HSL (Hue, Saturation, Lightness) with the filter. Once this is done, the new “Green” channel is used as a saturation mask. Sounds easy, but it’s flawed. A saturation mask created by this process poorly matches the saturation in the image, especially for dark colors.

This first blog covering saturation masks will take a closer look at the HSL/HSB filter and the asymmetry problem associated with it. It’s helpful to understand the problem before moving on to the solution. To do this, I’ll use a Granger Chart, shown below, for illustration since it makes the HSB/HSL problem quite obvious. A copy can be downloaded here. (NOTE: There are some color artifacts in the image on this blog as a result of converting from ProPhoto to sRGB for display on the web.)

In the Granger Chart, the most saturated colors zigzag across the vertical center of the image. Saturation will vary somewhat depending on hue, but moving vertically, either up or down away from the center, leads to colors with decreased saturation. The saturated colors in the middle gradually transition to 100% white on the top and 100% black at the very bottom. Black and white have no color at all and so, by definition, are completely unsaturated (saturation = 0%). In other words, pixels close to the center are highly saturated, and saturation decreases in a gradual manner as you move towards the top and bottom edges.

And this is indeed what happens according to Photoshop’s Color Picker when moving vertically from the center to the top edge. The image below shows saturation values from the Color Picker at evenly spaced points along the vertical axis. As you move up from the center (Saturation = 100%), saturation values decrease progressively as distance from the center increases. Exactly as expected.

But look what happens as colors get darker. Saturation values, again extracted from the Color Picker, do NOT decrease when moving downward from the center. As colors get more black and contain less color, saturation values don’t change. Saturation remains at 100% even though the amount of color present is gradually decreasing. Even the very bottom pixel of the Granger Chart still has a saturation value of 98% according to the Adobe Color Picker. Strange, huh? Why does saturation remain unchanged as color intensity decreases and black increases? And why are the dark colors different than the light colors?

The actual saturation mask produced using the HSB/HSL filter (the “Green” channel after converting to HSL) matches values from the Color Picker. It is shown below. As a quick review, a saturation mask shows white or light gray in saturated colors and shows black or dark gray in unsaturated colors. For light colors, the HSB/HSL-produced saturation mask shows a smooth decrease in saturation, fading from white to black when moving from the center towards the top edge. This is exactly as expected. The central intensely colored pixels are highly saturated and show as white in the saturation mask. When moving towards the top, saturation decreases, so the mask turns gradually darker gray until it is entirely black at the top border, which is white in the image (and therefore 0% saturated).

But look what happens with this HSL mask in the dark colors. The dark colors in the image remain pure white in the mask (indicating very high saturation) even though they contain less and less color when moving from the center towards the bottom. Yes, this matches the saturation values derived from the Color Picker, but it ignores the obvious fact that color saturation actually decreases gradually as you move lower from the center of the Granger Chart. There is no corresponding smooth transition in the saturation mask of the dark colors like in the light colors as would be expected.

This is the asymmetry problem that comes from using the HSB/HSL filter to produce saturation masks. Light colors show appropriate saturation and feathering in the HSB/HSL saturation mask, but dark colors show exaggerated saturation and no feathering at all.

The important thing to take away in all this is that a saturation masks created using the HSB/HSL filter significantly (and sometimes grossly) overstate saturation in dark colors of the image. Dark colors end up being mapped as more saturated than they really are. This is also demonstrated in the image below. Its HSB/HSL saturation mask is shown beside it. Notice how the letters and numbers, even though they are nearly black, are showing as a very light gray in the saturation mask created using the HSB/HSL filter. This indicates they contain highly saturated colors. But this is obviously wrong. No way does a color this dark have the degree of color saturation indicated by the mask. Black is 100% UNsaturated, and any colors approaching black should be very dark in a mask that accurately maps color saturation. But in this mask, the characters in “OPEN” and “7 TO 7” are very light. Based on these tones in the mask, their color saturation in the image should exceed that of the deep blue sky (which is a darker gray) and approach that of the reds and yellows in “GRILL” (which are also very light gray). Even worse, the black word “GEARHART” is mapped in the mask as being more saturated than the magenta background behind the hamburger bun. These characters are clearly less saturated than this. So this isn’t just a theoretical concern that happens in a mathematical image like the Granger Chart. The same problem of the mask showing over-saturation of dark colors is also quite obvious and quite extreme in an actual image.

NOTES:

1. The question can fairly be asked as to why these dark colors aren’t actually pure white in this saturation mask of the image as the dark colors are in the Granger Chart? I’m not entirely sure, but I think this probably relates to the way the Granger Chart is created. It’s made from two gradients: one of completely saturated colors and the other of black-to-white tones. I have a feeling that the colors it produces are only a subset of all colors (even though it creates what appears to be a spectrum of all color) and it’s these specific dark colors that get mapped to 100% saturation by Adobe’s Color Picker. Although, it may also be related to how Photoshop converts from RGB to HSL. Or it could be some other reason entirely. The Granger Chart happened to be what I was using when experimenting with saturation masks, and was what tipped me off to the HSB/HSL problem of dark colors appearing more saturated than they really are. But the over-saturation of dark colors has been apparent in all “real” photos that I’ve made saturation masks for using the HSB/HSL filter. The over-saturation of dark colors is seldom 100%, like with the Granger Chart, but it is always very obvious once you’re aware of it and start comparing the saturation of dark colors with other, truly more saturated colors in the image.
2. It is also worth noting that there is still some self-feathering of the of the over-saturation in HSB/HSL-produced saturation masks in “real” images. There is no abrupt cutoff like with the Granger Chart HSB/HSL saturation mask. So using these HSB/HSL masks would still lead to a smooth blending of whatever is adjusted through such a mask. However, visually evaluating these HSB/HSL-created masks to decide where saturation is truly present in the image is nearly impossible since the dark colors end up being so light in the mask that they frequently outshine actual saturated colors. The bottom line is that any adjustment through such a mask intended to affect actual saturated colors in the image disproportionately affect darker colors in the image instead.

With the problem described, the question becomes how to make a more accurate saturation mask, one that treats light and dark colors equally so that the saturation in darker colors gradually tapers off as saturated colors are replaced with unsaturated black. What that mask looks like for the Granger Chart is shown below and was generated using the “SAT” mask option in the TK RapidMask2 panel. It shows perfect saturation feathering in both light and dark tones. This is easily achieved when using a better method than HSB/HSL to generate TRUE saturation masks.

Below is the test image along with the more accurate saturation mask generated from the RapidMask2 panel. The most colorful areas are now the brightest shades of gray in the mask and elements with little or no color (the words “OPEN”, “7 to 7”, and “GEAHART”) are very dark gray. This is exactly the way it should be in a pixel-based mask that accurately maps color saturation.

I’ve been using this more advanced method for making saturation (and vibrance) masks in my panels and developing workflow for close to a decade and wouldn’t return to using the old HSB/HSL filter at this point even though Adobe once again makes it available. I’ll explain how to create these TRUE saturation masks in an upcoming blog.

SUMMARY: Saturation masks produced using Photoshop’s HSB/HSL filter display asymmetrical saturation when comparing light and dark colors. Light colors show a perfectly feathered saturation transition as colors get lighter and less saturated while dark colors get mapped as being more saturated than their color would suggest. A better method for making saturation masks would take into account pixel brightness as well as saturation.

FINAL NOTE: I’m not an expert in color models and do not know why the HSL/HSB filter maps saturation the way it does or why the Saturation values in the Color Picker remain at 100% for dark colors. I just know that pixel-based masks need to accurately display the pixel-based values I’m looking to adjust if they are to be useful. That way I can properly evaluate the mask and modify it, if need be, to match my image. In the case of saturation masks, the HSB/HSL filter is definitely substandard for creating the type of masks I want to use.