The Rabbit Hole

Mareck

00:05

Thanks for your feedback @Gez, I'm really happy to have spent so much time on the topic since a week and a half. It's a shift for the brain but it really worth it.
I stick with the CDL node for my nexts tests, i swear^^.
Do you know or @troy_s how the desaturation look works? At which value it start to desaturate? Or the formula?
So much question about the colors too but, later^^.
I'm not @home for the week end but I'll read you next week.
See you.

troy_s

00:23

@mHurley Sort of. The problem is you lit it under a very limited dynamic range, so the LUT will only reveal what you already had there for data, somewhat limiting the impact. Part of using a wider dynamic range view is that you need to push more light at the scene, more bounces, more SSS, etc

@Mareck The original desaturation was a purely mathematical weighting of grey for 709, starting at the low value and working towards high. The problem was that it wouldn't blow out all values, so led to some anomalies, including desaturating highly luminous colour mixes early (greeny yellows)

@Mareck This one starts two and a half stops over middle grey, and is basically a linear convergence (might be eased, can't remember) of each primary. This results in an almost filmish crosstalk as it isn't purely according to luminance weights. Interesting path towards white for various mixes of primaries.

cegaton

02:56

@ToddMcIntosh when using the -10-+6.5 LUT don't use curves. blender.stackexchange.com/a/54026/1853

troy_s

03:49

Damn proud of every single person that has set aside their previous thinking and dove into colour and management of pixels in here.

It is up to imagers to take this upon themselves. It doesn't surprise me at all that @Mareck made that lovely jewelry render as a first effort given that it is precisely the types of imagers that are willing to dive in that can see and understand the benefits.

Rant is brought to you by this horrible and pathetic cancerous thread: mail.gnome.org/archives/gimp-developer-list/2016-May/…

The fewer imagers that are willing to band together and learn things, the more risk of software designed by and for idiots.

@cegaton What lost are you referring to?

(there are times curves are excellent tools, but their current implementation makes them utterly worthless in Blender, in addition to reducing the power of CDL and some other useful techniques that are lurking on the horizon once everyone comes to terms with the concepts at hand.)

cegaton

04:32

@troy_s lost...?

@troy_s I guess the user edited the answer and took the node setup down... but nevertheless the word is out.

troy_s

04:49

Huh????

@cegaton I am lost. What?

cegaton

@troy_s the answer included a setup using curves before the CDL node. Hence my comment. The answer was edited since...

@troy_s blender.stackexchange.com/questions/54025/… If you have access to the edit versions you should be able so see what I meant

troy_s

That's a shame.

I hope more imagers start grabbing on. It is deep territory to mine so many new things (rightfully Todd pointed out just one other fascinating aspect)

Would have been better to seed the answer with the question.

I am actually shocked at how few folks realize the curves have been rather uh... broken since Matt Ebb introduced the sRGB linearizarion workflow.

Goes to show that I suppose the vast bulk of the folks futzing with Blender don't really understand what the curves represent nor do.

cegaton

@troy_s I guess I'll post a question about curves, CDL and color correction... stay tuned then...

troy_s

More rabbit holes

It will be fun seeing all of the "answers" however.

I am sure many likely don't even realize how busted up the AdobePDF spec blends or LGG is.

cegaton

05:04

@troy_s AdobePDF? Portable document format? or PSD?

troy_s

@cegaton PDF.

@cegaton adobe.com/devnet/pdf/pdf_reference.html

@cegaton Table 136, see if you can spot how many blend modes don't work with scene referred models. ;)

There is a secret to spotting them... look for the magic value 1.0

(that old adage that the moment a pixel operation makes an assumption about data is the moment the colour is broken)

@cegaton The sad part is you can probably guess what those blend mode formulas look like in the compositor, and equally why and how broken they are.

@cegaton If you try Nuke non commercial, you will see a (horrifically titled) "Video Colorspace" toggle on the blend node. If you select that, you get display referred AdobePDF formulas. If it is not toggled, you get scene referred versions.

(In the end, the AdobePDF stupidity of blend modes are actually all hacks designed to work around the fact that the entire imaging model was display referred. Dodge and burn were simply exposure chemical processes in photography for example to increase and decrease exposure. In a scene referred model, it is basic math. Display referred? Magic sauce to try and fake it.)

Overlay? Hard light? Etc. All hacks.

Yet another red pill "The more you thought you knew the more you realize how much you don't."

I just assume I am wrong now.

It is easier. Less painful on my psyche.

gandalf3

05:25

I am sure glad I just discovered this room by chance. I've been reading the transcript, and I've already learned things

cegaton

@gandalf3 welcome!

gandalf3

:D

Is there a goto source for red pills?

As in, a reference or guide that covers all the best practices and pitfalls?

Or am I already there

cegaton

@gandalf3 start with the links listed at the bottom of this answer: blender.stackexchange.com/a/46940/1853

gandalf3

Thanks

cegaton

From there on, we're trying to figure out the map as it unfolds...

gandalf3

05:30

I suspect my monitors are likely horribly out of whack

They are wildly different from each other to begin with

But colorimeters seem expensive :/

How important is having a calibrated monitor?

cegaton

@gandalf3 that's about the only way to make sure that what you is going to translate to other people's monitor.

gandalf3

I wonder if I could "calibrate" them with a camera

I could at least try and make them look like eachother

cegaton

@gandalf3 No, you can't really use a camera... I'm sure you've read this... but just in case you haven't : blender.stackexchange.com/questions/31068/…

gandalf3

I hadn't seen that, thanks

cegaton

05:47

@gandalf3 got to go. Good night.

gandalf3

o/

Thanks again

I have my reading cut out for me :)

troy_s

06:00

Ugh blenderartists.org/forum/…

@gandalf3 Not dreadfully given that most displays are pretty decent in the modern era.

@gandalf3 I also cringe a little when someone begins by thinking that colour management is about display profiling etc. Colour management does encompass that, but in truth profiling displays for colour precision is perhaps 2% of what colour management is actually about. I tend to try and get imagers to focus on ignoring that aspect to at least kindle the question "Well what is it then?"

gandalf3

Well what is it then?

I hear different terms such as "color grading", "color management", "color profiling"

troy_s

@gandalf3 It is a deep rabbit hole. Every inch you dive you realize you have to unwind what you know and trade it for unknowns. The upside is that once you start grabbing on, there are many answers that are "Gee that is so damn obvious now". The trade off is realizing that the hole is damn deep, and that there is so much more to imaging that you can leverage by diving deeper.

gandalf3

I suspect they refer to different things, but I don't really understand what the differences are

troy_s

Hrm. Play the slide show. I sort of tried to peak interest in that.

Colour management is really about wrapping your head around different models.

The slide show is here:

docs.google.com/presentation/d/…

Make sure you press present though, as it was designed as a presentation.

I can assure you at the start of that presentation, every single audience member was yawning. By the end of it I had most of them come up and actually express how excited they were.

gandalf3

Presenting it now :)

troy_s

06:09

@gandalf3 For the record: "colour grading" is the act of creatively adjusting an image for colour and contrast. "Colour management" is the management of data values across and through a pipeline. "Colour profiling" is the gathering of data about a device to make the colours it displays or captures relevant.

2

Welcome to the hole.

Enjoy forgetting.

gandalf3

aha, that helps clarify things

@troy_s hehe

troy_s

Only a few rules in Red Pill Club. Ask questions. Try things. Spread what you learn.

If questions don't have clear answers, they will be found.

(the slides for example, were a presentation so there are some gaps in explanations. Some of those gaps are now filled with answers on BSE. Others need stating.)

@gandalf3 The results are rather incredible from people with very little experience with the concepts. Here is @Mareck's early attempt:

gandalf3

I was just looking at it, incredible

troy_s

pasteall.org/pic/show.php?id=103818

There are a good number of examples in the dynamic range thread.

And each of those people are just coming to terms with it. Some brokenness in their pipes still, but the results are hard to believe.

Part of the whole embracing of concept is the quote by Alex Fry I put in the slides.

Creative imagers can make anything look solid given enough time and learned ability. The beauty of the concept however is that so much of what requires cheating in a display referred model concept you get for free simply by embracing a scene referred series of concepts.

(There are many images I can't show anyone that are quite mind ripping but sadly not permitted to be shown thanks to NDAs.)

I truly hope more imagers take up an interest in it. The potential to unlock elevated work is tremendous.

gandalf3

My mind has already been blown by many of the images in your wide dynamic range answer

troy_s

06:22

I must sleep. I will say I have been chasing the colour dragon for longer than I care to acknowledge. My number one obstacle to learning was misinformation. The hardest thing to cut past is the plethora of absolute crap out there from 'reputable' web postings.

Please ask questions. I promise you the answers will make sense, and if I can't answer your particular question, I will find someone that can.

A healthy culture is built one mind at a time.

gandalf3

Thank you, you've already opened my eyes to a world I didn't truly realize existed

This room's name is not is not an exaggeration in the slightest

troy_s

Looking forward to the questions and hopefully experiments. :)

Night for now.

If you ping me or someone else here, the notification happens.

jaska

So this room is about 3d graphics ?

gandalf3

@jaska This room is specifically about color management

For more general blender discussion, try the renderfarm

troy_s

@gandalf3 The funny part is that certain imagers are drawn to the results that they can see and as a result it is a pretty good sign that they are wanting to engage with the concepts. It is like an automatic filtration system.

OK. Sleeps. Ciao.

gandalf3

06:37

Good night

iKlsR

13:36

@troy_s Do you have any recommendations for papers or books on raytracing?

troy_s

13:53

@iKlsR Not the best person to ask. Ask @jtheninja

iKlsR

Damn, I haven't heard that name in years

Good ol' 2.49 luxrender days

troy_s

14:59

@iKlsR He's darn well versed on rendering and details, I suspect he would have a good breakdown as to how shaders and closures work.

mHurley

18:59

@troy_s So, if I'm using the "bassam-test" LUTs and whatnot, what looks and views should I be using to correct my lighting? I'm having a hard time understanding what the role the views, looks, and grading play in adjusting the scene (as opposed to adjusting the render). Should I adjust my lighting with the -10-+6.5 View and one of the Team Argentina Looks, and then grade that?

troy_s

20:21

@mHurley The view is the basic transformation. The looks could be baked in, but I would not do that with the ones I provided as they shift the middle grey too far. In terms of looks, the only one I would consider useful for look is the desaturation look, because it is 1:1 with the default look, with the only difference being the desaturation

The Sharps are just there as temporary glances to see how your lighting responds under a more contrasted look.

The false colours are great for checking the data range under the basic view.

So the idea is light to the basic view, or basic view with desaturation look. Flip on and off the Sharps to get a sense of what some more contrasty curves might look like. Once happy, put a CDL node in and use only the basic view or the basic view plus desaturation for your grade.

mHurley

Maybe I didn't ask the right question. When I'm in Blender, trying to get my lighting correct, and I'm looking at a rendered result deciding which light to dim or bump, how should I be looking at that render? Should I be using a "Look" on that render, or just the View?

Once I'm satisfied with the lighting, will I grade the result that comes from one of the Views, or a View plus a Look, just a Look, or none of the above?

I'm just not clear on what the Looks and Views do to my image, and what part they play in the final result.

mHurley

20:39

@troy_s

troy_s

20:55

@mHurley Think like a photograph.

@mHurley When you take photography to the next level, you typically shoot knowing you are going to grade the image. In doing so, what you are trying to capture in the photo becomes different in terms of data. For example, when Fincher shot Seven, they wanted to thick dark blacks.

@mHurley The only way to get that is to print / grade it down. That means exposing your regions in question a little heavier knowing you are going to pull down the print.

Same goes for a render I would suggest.

So your base view is your entry point (or base view with desaturation as above) and while lighting, you can leverage the false colour go check your data ranges etc. and even test fly some of the Sharps to see how steeper contrast might look in areas (again, they are not perfect as to how I would design them with more time)

Once you are satisfied with your lighting, and tested a few looks to build confidence with your overall illumination qualia and such, flip to grading with only the basic view or basic view + desat and then use the CDL node to dial in the values to where you were going with the render itself.

Best advice is to always treat grading as a separate creative process to the render. Render to EXR. Then commence a grade. That way you can generate many different looks off of your native archival frame

mHurley

21:15

Interesting... So I'm really thinking about the render as "how can I get the most usable data from this scene?" And then I use grading to make that data look good. Right?

I mean, obviously a terrible render won't be saved by grading, but a "useful" render might not look very good until it's been graded. Right?

Gez

22:54

@mHurley @mHurley: Exactly

The log view will give you a wide dynamic range to work with. Your shadows won't be too close to the black cut, highlights won't be too close to the white cut. You'll see a lot of detail.
You'll bring contrast back and choose where the cuts are placed as a creative decision, instead of being fighting with them as you'd do with the sRGB view.

The first thing you'll notice with this view is that you need more light intensity. But, as Troy said, when you start adding more light you get better bounces and reflections for free.

You don't have to cheat them or compensate with extra lamps. Things start to behave more realistically.

So it's very important to keep in mind that during the lighting stage you're not looking for final look, you're looking for data, for correct exposure of your scene.

4

Once you get middle gray locked and your shadows are highlights aren't blown out (the false color look will help you to judge that) you can be sure that your scene is properly exposed, then you can proceed to grading, which is the stage where you produce the final look.

@mHurley: regarding "a useful render might not look very good until is graded", it's sort of true, but the sharpener looks help you there.

Personally I find the 2.0 look quite pleasing. It' provides a comfortable contrast without being too agressive with the shadows and highlights. It's an approximation to the contrast you'll want with your grade, but still gives you some information about high and lows.

@Mareck I'm still watching at your render. I love it. How many samples did you use? Did it take too long to render? The result is really clean, almost noise-free

And the little noise it has feels like grain.

Oh @Mareck nevermind. Just saw your screen capture. It's all there.

Under 5 minutes?

troy_s

23:34

@mHurley Bingo. Of course quality of light and position and intensity still matter tremendously, and developing that ability to see how the light is landing in the scene versus how you intend to grade it will lead to a very wide spectrum of differences and nuances.

@mHurley It is a skill to begin to analyse imagery that way. Hence why having a quick flip Look that you design with a CDL or using one of the canned ones I quickly mocked up is a very useful toggle to get a sense as to where certain regions are shaping up.

@mHurley The default View is a log based encode, which while sort of gaspingly ghastly when you first look at it, and difficult to evaluate, log-like encodes are bent in such a way to make every piece of data you are looking at very clear and visible. You can see with X-Ray vision into your shadows and Welder's Glasses into the highlights; it's like a magic vision superpower.

When you put away your aesthetic vision and evaluate the log view as data that you are going to refine, you will begin to see the value in the 'basic' view.

So to reiterate, concern yourself with quality of light (how diffuse and how far away your emission meshes or area planes are), the shape of the light (how far around a lamp is on an object), and other qualia, but don't get too hung up on contrast transfer curve. You will begin to develop a good sense as to where your sweet spot is for a grade that you can delicately curve off to floor and roll off shoulder to highlight.

The biggest problem with the Sharp looks is that I did a really, really quick solution: I simply did a power curve on the already shaped data. The downside of this is that it pushes the middle grey value around. In an ideal world, those power curves would have the overall shape, a gentle roll to floor, and a nice shoulder, but each would firmly anchor 0.18 at say, 0.5 display referred.

@gandalf3 How goes the rabbit hole?

gandalf3

The presentation helped, now I want to try something

troy_s

@gandalf3 Good for you. Great way to dive in.

@gandalf3 Getting a feel for shape of the lighting under a wide dynamic range and how much detail you can choose to toss in your shadows (versus the default sRGB over-crush in the shadow area)

In most instances, sRGB is the wrong transform. The reason is that Blender dumps display values through OCIO and then direct to the display. What most imagers don't realize is that a modern LCD display is actually a linear response, and to behave as a CRT display, the modern displays have a baked in curve at the hardware level. That means that when you dump an sRGB curve to the display

you are seeing an sRGB curve under a 2.2 power hardware curve. Which means the shadow area isn't quite right because sRGB has a small linear segment near the floor. For most instances, the correct curve would be a 2.2.

(Sounds completely backwards, but alas...)

gandalf3

hum

troy_s

Basically the real key issue is that in your scene value, which are all strictly linearized, you have to figure out A) what range you want to see in your display value and B) how the hell to get them there.

That's what the basic -10-+6.5 view does.

It's a basic concept for getting that particular range into the smaller range.

@gandalf3 Make sense?

gandalf3

23:50

At a conceptual level, more or less.

troy_s

Sounds like you have hesitations or questions?

Transcript for