Q&A about HDR and Math – ask your questions

The new RGB Math components generate a lot of questions. The questions are currently scattered all over the forum, so I'm creating this thread to answer them all here. If you have a question about a Math component, ask it here and we'll answer it.

Before we proceed to questions, you need to understand two key things about the math components.

First, Math components are channelwise. That is, they accept RGB colors and perform the same operation on all their channels. For example, when the Add component is given two colors, R:0.5 G:1 B:2 and R:10 G:20 B:30, the resulting color will be R:10.5 G:21 B:32. Alpha, in most cases, is not processed channelwise – most components just use the intact alpha from the first operand. When answering questions, we'll most likely omit the channelwise nature of these components and explain them as if they had just one channel.

And second, Math components operate on real numbers. Not some computer-graphicky numbers remapped into some esoteric range from 0 to 1 or 0 to 255 or -1000 to 1000, but real numbers. If you see an answer that mentions numbers like -1200 or 0.00032 or 750000, this means that you can literally enter those numbers into math components (via the RGB channels in the color picker in HDR mode), and the math components will treat these numbers exactly as your pocket calculator would.

For a start, I'll answer some of the questions posted in other threads:

Quote
CorvusCroax wrote: So, how do the math nodes compare in performance to, say, tone curves or blending? Is it better to create things using the math functions? (vs. the old way, using elaborate constructions offsets and blends?) I assume the math nodes are better, being more direct...?

I can safely say that RGB Math components are the fastest map components in Filter Forge. By all means use them for optimization whenever possible.

Good morning Everyone,

Hey Vladimir,

We ALL thank you for your help! You are the man!

Have a GREAT day!

Ron

ah! thanks, vlad! the Lerp is one i've been waiting for and didnt even know it. sweet!

Vlad: this is a great idea for a thread, thanks for grouping it all together for easy reference!

Quote
Vladimir Golovin wrote: I can safely say that RGB Math components are the fastest map components in Filter Forge. By all means use them for optimization whenever possible.

Most excellent.

Quote
Kraellin wrote: So my question is what can we do with some of the math functions that we couldn't do before. For example your comment on the Lerp showed when and how we would use Lerp in a simple functional way.

Quote
Redcap wrote: If you could maybe show just like one new math component a day and a practical application of it, that would be the most helpful to me.

I agree; having a 'maker's intent' sort of brief description for a practical applications of each would probably help demystify the math components and help them be more widely adopted by the community. Examples help a lot.

Quote
However, Lerp is not the same as the Normal blending mode – they are identical only when both images don't have transparent areas.

I didn't think about translucency, that's useful!

by the way guys, math components are quite useful for a number of blendings. in paint shop pro we have relatively the same thing in 'arithmetic' filters and even somewhat similar (though not nearly as close as 'arithmetic') in the user made filters of psp. one use is in detecting parts of a very damaged image. by using blend modes and arithmetic in psp it is sometimes possible to detect, a' la convolution methods, parts of an image that are normally unreadable. scanned handwriting is one example of this. by applying arithmetic values to a scanned image with handwriting that is badly faded or marred in some way, one can often pull out that damaged or almost missing info.

another use is in photo art, but i wont go into that that much right now until i've had more of a chance to play

My question refers more to the HDR in FFv2 than these new math components, but I think it may be something others are wondering about as well.

I think I understand the idea behind the math components, they basically do whatever operation is applied to each of the RGB channels then spit out the result of the two, (Sum, quotient, root, trigonometric function, etc).

They seem to mostly operate on a "per pixel" sort of method (I know FF doesn't use pixels, but for sake of understanding) where FF looks at only the local area; in other words, THIS "pixel" plus THIS "pixel" = THIS.
(The exception would be, Derivative which I'm guessing looks at the nearby "pixels" and interprets rate of change, as derivatives are always intended to do)

ANYWAY: What I'm having a hard time understanding is the usefulness of values beyond the 0-1 range. Unless exporting to an HDR format, won't values that extend beyond the normal 0-1 range simply appear to be clipped when saved? If I have a vertical gradient that goes from 2 to 0 and save it as a LDR PNG, it will simply look like a lesser smooth gradient with a ugly bigger white area. What does HDR (within the confines of creating exported LDR outputs in FF) enable us to do that we couldn't otherwise?

Feel free to shift this post to a new topic if it's too divergent, but I'm guessing that I'm not the only one here wanting to understand this a bit more.

Quote
Sphinx. wrote: Can you explain a little about how the Derivative component work and how many samples it fetches?

To produce one output sample, Derivative takes two samples of its Source input. The samples are located on a line passing through the original sample point (shown as "Target Point" in the picture below), to the right and to the left of it (shown as "Sample 1" and "Sample 2" in the picture below). The distance between these two samples is very small.

Each output color channel is calculated as a difference between the two sampled channel values divided by the distance between the sample points. As other Math components, Derivative is channelwise – all these calculations are performed for each RGB channel separately.

The resulting Alpha channel is calculated by averaging Alpha channels of the two samples.

Quote
CorvusCroax wrote: You'll have the help file written in no time like this!

Well, that's part of the reason why I started this thread

Everyone, please keep the questions coming. If you find my wording unclear, please do point it out!

Quote

i guess i dont understand how HDR is 0...1. so, if HDR is that, then all your floating point values inbetween (which is essentially infinite) get assigned a color value? and then, if FF's HDR breaks out of the 0...1 range, up to trillions like you say, then isnt there still going to be a white threshold, where anything above a certain value is going to be white and if so, then what is that value/threshold?

HDR isn't limited to 0 to 1, it can be any floating point value (including negatives). How filterforge "chooses" a color all depends on what you are saving the file as. If you save it as a 16bit PNG, you only get 16bit color precision.

Lets throw out some imaginary numbers to use as an example: Lets say after all my magic mathy math is done, I have these values at one point: (INTERNAL FF)

R: 3000.0000000
B: 0.2123456
G:-2.7784857

If I export to a HDR format with a bitrate that corresponds to 2 decimal places, FF would export an HDR image like this: (EXPORTED/SAVED AS HDR FORMAT)

R: 3000.000
B: 0.212
G:-2.778

Now I export to an LDR format. (LDR means that any values below 0 or above 1 will be clipped correspondingly. Assuming 16bit PNG This would be something like: (EXPORTED/SAVED LDR FORMAT LOW BIT RATE)

R: 1
B: 0.212
G: 0

If I saved it again but using 32bit PNG, I simply get more precision:(EXPORTED/SAVED LDR FORMAT HIGH BIT RATE)

R: 1
B: 0.212345
G: 0


Aaaannnd hopefully that answers your questions. Please correct me if I err

Quote
Kraellin wrote: wouldnt you have to re-scale the entire color spectrum in a different mode and system and value package (not rgb, not hsb, not Lab, and not all the others) to accomplish any real display value to a number like 123,498,532?

Yes, to display an HDR color on a LDR device, we would have to somehow fit it into the LDR range. The technical term for this process is Tone Mapping.

But bear with me. Displaying is not the only thing you can do with colors in Filter Forge

Quote
Kraellin wrote: what value is a number in the trillions if you cant display it as anything but white?

Yes, Filter Forge can't display colors brighter than 1 as anything but white, and colors darker than 0 as anything but black (without performing tone-mapping on them).

However, let's remember that Filter Forge also uses colors for purposes other than just display.

The key concept behind Filter Forge's power and flexibility is Parameter Mapping -- that is, the ability to to control a parameter of a component with an image produced by another component. Essentially, a component whose parameter is being mapped interprets the color it gets from the map-providing component as a numeric value.

For example, when you're mapping a Roughness parameter in a Perlin noise with a gradient (as shown in this picture), the Perlin noise component converts the brightness of a gradient at a particular pixel to a number, which is then used for calculating the Perlin noise function for the corresponding pixel of the output image. In that example, the the leftmost pixels where the gradient is black correspond to a numeric value of 0, and the rightmost pixels where the gradient is white to a numeric value of 1.

(Actually, the Roughness slider in Perlin Noise component is ranged from 0...100 but that was made for user convenience, its actual internal range is 0...1).

Mapping a finite ranged parameter, e.g. a slider, is simple. Black pixels of the map correspond to the leftmost slider position, and white pixels to its rightmost position.

But what if a parameter you want to map has no range?

Consider, for example, the parameters for gradient endpoint coordinates in the new Free Gradient component: Start X, Start Y, End X and End Y. All four parameters are represented in component's properties as big edit boxes with dials near them. They aren't represented by sliders -- you can't fully represent an unlimited parameter with a slider because sliders always have a minimum value and a maximum value.

Now, suppose that you're trying to map one of these unlimited, rangeless parameters.

If there were no HDR colors in Filter Forge, you can only vary the target parameter within the range of 0 to 1 (remember, black areas of the map correspond to 0, white to 1, and grays to values in between 0 and 1) -- you would have no way to set values lower than 0 and higher than 1 via parameter mapping. With HDR colors, you can map rangeless parameters with any values you want.

Quote
Kraellin wrote: what it's doing is processing the HDR numbers into LDR numbers for the sake of viewing?

Exactly.

Tone-mapping of HDR photos into LDR range gives you practical benefits. For example, your short-exposure shots capture detail in well-lit areas but lose detail in dimly-lit zones, and vice versa with long-exposure shots -- they capture more detail in dark corners, but bright areas get overblown.

When you assemble an HDR picture from several shots taken with different exposures, the resulting HDR retains all the detail captured by all the shots you feed into it. Then a tonemapper can compress the dynamic range of the final picture in such a way that it retains details captured by different exposures.

(For some reason I considered wedding photography as something mundane and usually mediocre, so I was quite amazed when I saw a wedding photographer here in Russia shooting to RAW with bracketing, assembling the shots into HDR and outputting a tone-mapped result -- the quality was superb.)

Quote
CorvusCroax wrote: What's the best way to convert down from HDR back down to LDR?

"Best" for what purpose?

I think for that, you just do the maths. If you have -5..5 range and you want it to be 0..1, you do

Quote
y = (x + 5) * 0.1

If you wanted it to have more fidelity on the numbers that were (say) originally from -2..2 you could then use a tone curve.

Here's the filter:

answer.ffxml

Here's how a compressed Derivative looks when its source is a Perlin Noise with an Electricity profile (the angle is 45 degrees):

Dilla, there's a conceptual problem with that. To implement an automatic range compressor, you need to know the maximum amplitude of the image to be compressed. To know the amplitude, in turn, you need to have the entire image in question already rendered so you can cycle through its pixels to determine the maximum.

Unfortunately, there's no such thing as a "fully rendered image" at any stage of the Filter Forge's rendering process. That's why we still don't have components such as Auto-Levels or Minimum Level or Maximum Level that accept source images via inputs, and that's why we provide these components only for the source image loaded into FF, which we have available right from the start, at any rendering stage.

Quote
Kraellin wrote: have you got a web page for that photographer? would love to see his results.

Nope -- I believe he doesn't even have a web page I'm sure you can find a lot of good HDR-to-LDR examples on the web -- just don't look for gimmicky ones (there's a lot of them, and they're dime a dozen), but look for those who use HDR specifically to combine detail from different exposures in a single image.

I do have one example, but it's a fake one (I made it in 3DS Max). It's the Earth from the promo page for Transform components. The point of the example is that you physically can't capture both the daylight and night lights in a single exposure -- the level of our civilization's night lighting is nothing compared to the daylight levels. So a physically correct shot of the Earth would either have the night side totally black while retaining detail on a daylight side, or the daylight side would be compeltely overblown into hot white while the night lights are visible.

Quote
Kraellin wrote: that i have a camera that shoots RAW

You should also consider exposure bracketing -- RAW formats still have a limited dynamic range, which is usually wider than LDR but still might be insufficient for your HDR purposes.

(Hey, we're hijacking the thread )

Quote
CorvusCroax wrote: there might be some simple way to clip the image back down to LDR.

We'll add a couple of simple tonemappers (Clip and Knee) in the next beta.

thanks, vlad.

and yes, i've done a tiny bit of exposure bracketing and will be getting into it more. i've been slowly moving to setting up to do some HDR photography and i'm close now. if i get any decent results i'll post here on the forums or over on retouchpro and link to it here.

Another help fragment on the relationship between HDR colors and RGB Math components:

Quote

How RGB Math components deal with HDR colors? Essentially, each RGB channel in a Filter Forge color is floating-point number. Channels in LDR colors are limited to the range of 0…1 (displayed as 0…255 by the Color Picker, see the section on ranges above), and HDR colors can have unlimited channel values ranging from tiny fractions to trillions, positive or negative. You can literally enter numbers like -1200 or 0.00032 or 750000 into math components (via the RGB channels in the color picker in HDR mode), and the math components will treat these numbers exactly as your pocket calculator would. There's no capping or wrapping when the result of calculations exceeds some arbitrary threshold, be it 255 or whatever else. When working with any components that treat RGB channels as numbers, it is important to understand that RGB channels in LDR colors, despite having the display range of 0…255, have actual values in the 0…1 range – and Filter Forge components use this latter representation, ignoring the display range. For example, suppose that you’re adding an LDR color with the RGB channels values of R=0, G=128.5, B=255, and an HDR color with RGB channels of R=1000, G=2000 and B=3000. The result will be R=1000, G=2000.5 B=30001. If you expected the answer to be R=1000, G=2128.5 and B=3255, you’re confusing the display range of 0…255 with the actual range of 0…1 – read the section titled "RGB ranges of 0…1 and 0…255 – what’s the difference?" above to clear the confusion.