Tbilateral Writeup

Useful information.

Tbilateral Writeup

Postby TheProfileth » Thu Apr 28, 2011 5:51 pm

Don't use Deen, it destroys details and especially any detail on gradients , so things like soft details like trees or grass or other background details get washed out and disappear.

For today I think I will talk about Tbilateral.
Tbilateral is a filter that most people are afraid to use, mainly because they tried it at default settings and got their source raped then gave up on it.

Tbilateral's parameters

diameterL/diameterC - (spatial diameters)
This is basically the official definition, with a few tweaks since the official definition was pretty decent.
This sets the size of the diameter of the filtering window. Larger values mean more pixels will be included in the average meaning that the strength will be higher, exponentially, higher numbers also slow down the filter. Must be an odd number greater than 1. diameterL is for luma and diameterC is for chroma if it is being processed. The defaults are 5 for both, but I will talk more about this parameter later, because its important really differs based on the kernels you use.
default - diameterL = 5 (int)
diameterC = 5 (int)


sDevL/sDevC - (spatial (domain) deviations)

These set the spatial deviations. The larger sDev is, the less effect distance will have in the weighting of pixels in the average. That is, as you increase sDev distant pixels will have more weight. sDevL is for luma and sDevC is for chroma. These must be greater than 0. To get a better idea of what these settings do try setting iDevL/iDevC to high values, and then gradually increase sDevL/sDevC from 0 on up while keeping iDevL/iDevC constant. Increasing these settings will increase the strength of the smoothing.

default - sDev = 1.4 (float)
sDevC = 1.4 (float)

That is the official definition but here is what I think, its best to set these to 1, or less than 1, as pixel weighting is USUALLY used for limiting but in this case its used to enhance the smoothing, not necessarily a good idea if you ask me, setting it to 1 will provide MUCH more consistent detail protection. Lowering this can help to keep a lot of detail especially when you are using restype 0 but it also helps a lot when filtering HD content as you do not want the extra pixels to cause weighting fluctuations IE not fairly weighting details and treating them as noise, this can help to prevent that, as I detail later. Also its a BAD idea to raise the sDevC to high numbers as it can start lowering color saturation.

iDevL/iDevC - (pixel intensity (range) deviations)

These set the pixel intensity deviations (or color deviations in the case of the chroma planes).The larger iDev is, the less effect pixel difference will have in the weighting of pixels in the average. That is, as you increase iDev pixels that are very different from the current pixel will have more weight. iDevL is for luma and iDevC is for chroma. These must be greater than 0. Try increasing these settings while keeping sDev/sDevC constant to get a better idea of what these do. Increasing these settings will increase the strength of the smoothing.

default - iDev = 7.0 (float)
iDevC = 7.0 (float)


My thoughts on this, keep them at around 5, its a BAD idea to raise the iDevC to high numbers as it can start lowering color saturation, but that is only at higher numbers. This parameter is relatively unimportant, keep it at 5 to be safe, the default is 7.

csL/csC - (center pixel weight multipliers)

These values are multiplied to the center pixel's spatial weight value. A value of 1 does nothing, < 1 means the center pixel will have less weight than normal, > 1 means the center pixel will have more weight than nomral, 0 gives the center pixel no weight. These must be at least 0. Setting csL/csC to 0 will give you SUSAN denoising.

default - csL = 1.0 (float)
csC = 1.0 (float)

So basically this is one of the more important parameters, it basically massively increases the limiting that we mentioned earlier. It multiplies the weighting of the pixels, and more weight on a pixel means less filtering is done on it, if you set these to 10, then Tbilateral pretty much only goes after things its relatively sure are noise, this parameter becomes even more important after we introduce kernels and restypes.

d2 - (second derivative or first derivative)

This setting makes TBilateral use the second derivative instead of the first when doing the intensity calculations. Using d2 should give better results on smooth gradients or anything that fits the piecewise linear model. Setting d2 to false will give better results on images that have uniformly colored areas with sharp edges (anything that fits the piecewise constant model). The actual difference between the two is usually not big for most sources. The effect is rather subtle.

default - false (bool)

I strongly disagree with the official statement, this parameter is EXTREMELY IMPORTANT. If you are encoding anime,you MUST USE D2!!! If you neglect to do this, tbilateral will basically not react nicely to gradients and other non uniform details, ALWAYS set this to true

chroma - (enable chroma processing)
If set to true the chroma planes are processed. If set to false the chroma planes from the source are simply copied to the final output frame.

default - true (bool)

Basically do you want to filter the chroma or not, its faster to not filter chroma, but it can be a good idea, it all depends on what you want.

Ok now for the complex section
kernS/kernI - (domain (spatial) and range (intensity) kernels)

These specify what kernel is used for the domain (kernS) weights and range (kernI) weights. The
possible choices are:

0 - Andrews' wave
1 - El Fallah Ford
2 - Gaussian
3 - Huber’s mini-max
4 - Lorentzian
5 - Tukey bi-weight
6 - Linear descent
7 - Cosine
8 - Flat
9 - Inverse

See the following paper for a description of all the kernels and their properties:

http://www.mediafire.com/?8st7o7p8xojgmpo

Gaussian (option 2) is the kernel used by the default (first proposed) bilateral filter.

default - 2 (kernS) (int)
2 (kernI)


resType -

This specifies how the weights and pixel values are combined to obtain the final result.
Possible options:

0 - Mean (weighted average)
1 - Median (weighted median)
2 - CW-Median (weighted median + extra center pixel weight)
3 - MLR (multiple linear regression) (best fit plane)

Option 0 (weighted average) is the type used by the default bilateral filter.

default - 0 (int)

Now, you most likely will not understand what that PDF was talking about or any of that shit, so let me spend some more time writing up about the different kernels and their combinations.

Alright so now that I have had enough time to experiment and completely assess everything I now have a better understanding of what the restype and the KernS and KernI do and I can explain them. Your KernS KernI and restype are all very closely tied together, so I am going to explain them all at the same time. To put it simply KernS is your smoothing technique/mode and KernI is your limiter or as the official manual calls it your "intensity". The restype is how the filter interprets your KernS and KernI. Restype 0 pretty much gives all the strength to KernS and KernI does very minimal limiting. This is why KernS has the most impact on the effect of the filter when you are using Restype 0. When you use Restype 1 it takes into account the KernI input and uses KernI to limit KernS, which is why instead of a blur you get a degrain usually when you use Restype 1 or 2. The only difference between restype 1 and restype 2 is that in restype 2, the middle pixel is given extra weight, which helps to preserve lines and edges as seen here.
this would be restype 1
o o o o x o o o o o
o o o o x x o o o o
o o o x x o o o o o
o o o x x o o o o o
o o o x x x o o o o
o o o o x x o o o o

this would be restype 2
o o o o xx o o o o o
o o o o xx x o o o o
o o o x xx o o o o o
o o o x xx o o o o o
o o o x xx x o o o o
o o o o xx x o o o o

Ok so lets say that the o's are where tbilateral has less weight meaning noise or grain and the x's are used to show what tbilateral sees as details or edges, the more x's you have the more weight that area has meaning it is less affected by tbilateral. With a greater amount of x's in the center area, tbilateral gives more weight to lines and edges and details keeping them from being filtered, this is why I suggest you use restype 2 almost always over restype 1

My suggested base settings are this
[size=x-small]tbilateral(diameterL=5,diameterC=5,sdevL=1,sdevC=1,idevl=5,idevc=5,csl=5,csc=5,d2=true,chroma=true,kernS=4,kernI=3,restype=2)[/size]

OK so a quick update, let me list a few different combinations of kernels and restypes that yielded ok results. Use the same suggested base settings as above only change the kernel
Ok now I have added comparisons

some general ideas, never use restype=3 its 50 times slower (literally) than restype 2 and almost never yields different results, never use kernels 0 or 2, another rule of thumb is that a higher restype* almost always yields more details, not in every case though, if you have 4 as your kernS, then restype=0 will almost always generate a blur rather than a selective smooth. A word of warning, do not use KernS 0 or 2, they are pretty terrible, and when KernI 0 is used for restype 1 or 2 it basically breaks the filter.

kernS=4 kernI=3 restype=0
pretty generic gaussian blur,I think of it as the baseline blur method of tbilateral. Maintains more details than a gaussian blur and is faster, using a very high csL/csC with this (somewhere between 6-40) will help maintain more detail and shape structure
http://screenshotcomparison.com/comparison/14185
http://screenshotcomparison.com/comparison/14220

kernS=8 kernI=5 restype=0
very good light denoising with detail retention
http://screenshotcomparison.com/comparison/14186
http://screenshotcomparison.com/comparison/14222

kernS=4 kernI=3 restype=2
less of a blur more of a selective denoising, still it's best to use a moderate csL/csC to maintain small lines, I suggest within the range of 3-8
http://screenshotcomparison.com/comparison/14187
http://screenshotcomparison.com/comparison/14219


kernS=4 kernI=9 restype=0
still not a blur, more selective than 4 3 0 but less than 8 5 0
again I suggest a moderate csL/csC of 3-8
http://screenshotcomparison.com/comparison/14189
http://screenshotcomparison.com/comparison/14223

kernS=4 kernI=6 restype= 0 or 1/2
If the restype is 0 then you get a fairly standard blur, but more details than 4 3 0, standard warning about making sure to use a high csL/csC when you use a blur method. If you use 1 or 2 as the restype it's a relatively light denoising mode, very very similar results to 1 6 2, recommended to use a lower csL/csC of 1-4
http://screenshotcomparison.com/comparison/14190
http://screenshotcomparison.com/comparison/14239
http://screenshotcomparison.com/comparison/14252

kernS=1 kernI=6 restype=0 1/2
Again as it is a restype 0 you get a selective blur, it's a bit stronger blur than 4 6 0 but less than 4 3 0 , when you use restype 1 or 2 it's about the same result, varies per source. Recommended to use a low to high csL/csC of 1-15 (the number needed will vary based on the source)
http://screenshotcomparison.com/comparison/14191
http://screenshotcomparison.com/comparison/14229
http://screenshotcomparison.com/comparison/14233

kernS=8 kernI=3 restype= 0 or 1/2
When zero, is the same as 1 6 0, same recommendations apply. When using restype 1 or 2 its a fairly strong denoise, I suggest using a
using a low to moderate csL/csC of 1-6
http://screenshotcomparison.com/comparison/14192

kernS=3 kernI=8 restype= 0 or 1/2
When restype is zero, it creates a pretty good blur, its strong but its also pretty selective, I suggest a csL/csC from 3-10. When restype is 1 or 2 it does from light to heavy denoising, I suggest a csL/csC from 2-10, depending on how much denoising you want.
http://screenshotcomparison.com/comparison/14270
http://screenshotcomparison.com/comparison/14269
http://screenshotcomparison.com/comparison/14260
http://screenshotcomparison.com/comparison/14262

More kernel combos later

Some advice when dealing with HD content, tbilateral is a spacial filter which weighs and compares pixels that are nearby in order to make decisions. When using HD content you have more pixels that are similar next to each other, this tends to lead tbilateral to be less selective about how it filters. There are a couple of ways to combat this. The first is as most of you probably guessed, to turn up your csL/csC, this can work, but a better way is to lower your sDevL/sDevC , this enhances the weighting of pixels making tbilateral more sensitive to detail, and can be EXTREMELY effective on HD sources.
I suggest lowering it to something around 0.5-0.95 depending on how much the filter is killing your details, note that this is not a replacement for csL/csC , that should be set appropriately depending on the source.
Here are some good examples of using tbilateral on HD content
http://screenshotcomparison.com/comparison/15969
http://screenshotcomparison.com/comparison/15970
Notice how in the last one it leaves the actual detail alone and only goes after the noise, that is a result of a good kernel combo, a moderate csL/csC and a slightly lowered sDevL/sDevC
User avatar
TheProfileth
Dual-Duality Leader
 
Posts: 22
Joined: Wed Feb 16, 2011 7:01 pm

Return to FAQs and Articles

Who is online

Users browsing this forum: No registered users and 1 guest

cron