YUV to RGB conversi...
 
Notifications
Clear all

YUV to RGB conversion issues

27 Posts
6 Users
0 Likes
6,965 Views
0
Topic starter

Following the recent Optimize Playback thread, I seem to have opened Pandora's box when it comes to Color Model in Cinelerra.

I work mostly with 8 bit YUV footage from Panasonic and DJI cameras. Up until now, I have always set the color model to RGB-FLOAT for my projects, and this, combined with setting YUV color space to BT709 and YUV color range to MPEG has meant that what I see in the compositor looks the same as my final rendered video.

However, I have noticed that if I set the color model to YUV-8 Bit, then my footage has far more dynamic range (highlights are hard-clipped when set to RGB-FLOAT), and is more saturated. In fact, when set to YUV 8 Bit, the footage looks the same as it does in VLC player.

On the other hand, if I use YUV-8 Bit as the color model, the final rendered video does not look the same in VLC player as it does in the Cinelerra compositor - it is a bit de-saturated compared to the compositor.

What I would like to achieve is to retain the vibrant colour and highlights that I see in the compositor when the color range is set to YUV-8 Bit in my final rendered video. Can anyone point me in the right direction?

17 Answers
0

@glitterball3 

In my little experience after rendering for 3 weeks to get the compositor color.
RGB-8 Bit > Colors equal to the original video.
YUV-8 Bit > Limited colors (saturated). different rendering than compositor.

JPEG > full color
MPEG > limited colors

You can try using Shotcut to do your color tests.

YUV-8 Bit + MPEG looks beautiful in the compositor but that's because the compositor uses X11 and X11 is RGB.

I could be wrong.

Using RGB + JPEG will have colors equal to the compositor.

The YUV of the compositor is not the same as the YUV of the render.

glitterball3 Topic starter 23/01/2022 9:08 pm

@sparkill Your observations are mostly the same as mine - having looked at it again, I would say that in YUV mode, the compositor looks more saturated than it does in VLC player.

However, RGB + JPEG, definitely does not show the same colours as the original video in the compositor for me - the image becomes quite washed-out looking. Also RGB+JPEG when rendered, does not look the same as it did compositor either - it looks similar to the results from RGB+MPEG (close to the original).

Is there any way to prevent the RGB-FLOAT conversion from clipping the highlights?

0

In theory RGB-FLOAT should be the only mode that does not create clipping, so I'm more and more confused...
Anyway, if you can find some certainty about CinGG's behavior with colors, let us know and we'll put it in the manual.

 

EDIT:

The only note to make is to use RGB-FLOAT with BT2020 and not RC709.

What we see in the Compositor depends also on the color space of the monitor.

0

If it's of any help, what I have found is;

If shooting restricted range video as most camcorders do, the colour range needs to be set to MPEG. If shooting full range video as many mirrorless cameras do, it needs to be set to JPEG. Analogy: Putting restricted range video into JPEG colour range is like trying to fill a bucket with a single glass full of water and trying to put full range into MPEG is like trying to get the bucket full into the glass. JPEG into MPEG clips at both ends, losing detail, MPEG into JPEG needs stretching, causing distortion. Assuming, of course, the video scopes are being used as a guide.

I always use RGBA-FLOAT whether shooting 10-bit full range or 8-bit restricted range. I get the impression (correct me if I am wrong!) that the plugins work in RGB, so it is better to set the timeline to RGB at the start to avoid converting YUV to RGB for processing and then back to YUV again.

I use BT709 colour space, even for UHD video, as I output 1080p which will become BT709 on export anyway.

In case you use it, I find there is a problem with ProRes in Cin, I started a thread about it a couple of years ago, whereby rendering loses highlight detail no matter what variation of ProRes I use. I got round it by using DNXHR instead. 

phylsmith2004 28/01/2022 9:07 pm
This post was modified 3 years ago 2 times by phylsmith2004

@dejay 

"In case you use it, I find there is a problem with ProRes in Cin, I started a thread about it a couple of years ago, whereby rendering loses highlight detail no matter what variation of ProRes I use. I got round it by using DNXHR instead."

 

I also finally logged a BugTracker ticket #606 so it can be fixed when a "pro" (pun intended) comes along.

 

0
Topic starter

The easiest way to see the clipping that I am talking about, is to load a video file with some over-exposed highlights, and add a blue banana effect.

Then, switch between RGB-FLOAT and YUV-8 Bit. The mini histogram in the 'value' slider of Blue Banana will show the out-of-range values when in YUV mode, but these will be clipped when in RGB-FLOAT.

Be careful when switching between MPEG and JPEG in the Preferences dialog, as this may change YUV to RGB in the background (if the format dialog window is also open, it will not show the updated value until you close and re-open it).

In case you use it, I find there is a problem with ProRes in Cin, I started a thread about it a couple of years ago, whereby rendering loses highlight detail no matter what variation of ProRes I use. I got round it by using DNXHR instead.

I was indeed rendering in ProRes, I will try rendering in DNXHR instead.

Edit: I think that the JPEG vs MPEG YUV color range may well be a factor here - in my camera, I have the range set to 0-255 (the other option is 16-255). However, if I set the YUV color range to JPEG in Cinelerra, the final render does not match what I see in the compositor.

 

Edit 2: If YUV color range is set to JPEG, then the exported video will be much darker than in the compositor. If I import the rendered video back into cinelerra, then it looks exactly the same in the compositor as the original video. However, the timeline preview will be much darker (just like in VLC)!
So, there's likely a clue there as to why the rendered video looks darker than the original.

Is it possible that MPEG is assumed as the color range in many areas of the software as well as in external video players and tools? If this is the case, is there a way to squeeze that extra range back into the MPEG range in Cinelerra without just discarding it via hard clipping?

This post was modified 3 years ago 4 times by glitterball3
0
Topic starter

I think that I'm getting closer to understanding the issue - if anyone knows better, then please correct me.

1. My videos are recorded using the 0-255 range.

2. 0-255 corresponds to JPEG in the "YUV color range:" setting, and with this set, I can see the full range of black to white in the compositor.

3. When I render a video with "YUV color range:" set to JPEG, the outputed file is usually interpreted as 16-235 and blacks appear crushed and highlights are clipped - both players such as VLC and even other video editors such as Kdenlive interpret the rendered video as being 16-235. However, if I import the rendered file back into cinelerra it is interpreted as being 0-255 without the crushed blacks and clipped highlights.

4. When I set "YUV color range:" to MPEG, what I see in the compositor is exactly the same as how the rendered video is interpreted by other players, however any values outside 16-235 are clipped by Cinelerra and cannot be recovered using any of the color correction effects.

At present, I can either:

a. Set "YUV color range:" to MPEG and the final output will match what I see in the compositor, with the caveat that I may lose some shadows and highlight information from the source media.

or

b. Set "YUV color range:" to JPEG and accept that the final output will be misinterpreted and that my shadows and highlights will be crushed and clipped.

Another question that I have is about the pipeline in Cinelerra:

Is the "YUV color range:" setting in Appearance a project-wide setting, something that affects the final rendered output, or just the compositor?

It seems to be a setting that affects how all media is interpreted - so what happens if there are mixed sources?

I think that a better way of working would be to allow the values outside of 16-235 to continue to exist (rather than being hard-clipped) in a RGB-FLOAT project with color range set to MPEG. These values would be outside the legal range in the scopes, but at least we could pull them back into range and recover highlights/shadows during colour correction

0

@glitterball3

Another question that I have is about the pipeline in Cinelerra:

Is the "YUV color range:" setting in Appearance a project-wide setting, something that affects the final rendered output, or just the compositor?

It seems to be a setting that affects how all media is interpreted - so what happens if there are mixed sources?

As far as I know, the "Project Setting" is used for everything.  So if there are mixed sources, it uses the same current Project setting for all sources.  So for advanced usage, it is always best to set up your project settings first.

https://cinelerra-gg.org/download/CinelerraGG_Manual/Automatic_Best_Model_Media_.html

0

@glitterball3
To see the rendered video in a player without color problems (I can't tell in other NLEs), it is advisable to write in the wrench options the 3 information about the colors you are using as metadata. They are as follows:
colorspace= ... (color matrix: RGB, YUV,...)
color_trc= ... (transfer characteristic or gamma: sRGB, Rec709, ...)
color_primaries= ... (gamut: sRGB, Rec709, BT2020, ...)
These parameters do not affect the rendering you do in CinGG, but are read as metadata by players (Vlc, mpv, ...) that take them into account to play the video in an optimal way.
https://cinelerra-gg.org/download/CinelerraGG_Manual/Extra_acin_a_Options_Render.html

In case of different and non-uniform assets it is advisable to standardize all sources to the same model color, gamma and gamut (and that coincide with those of the project). So render/Transcode inside CinGG or, even better, externally from terminal with ffmpeg, etc. In any case, as Phyllis said, the rendering is only affected by the Project settings (and wrench parameters).

0
Topic starter

@phylsmith2004  & @andreapaz

Thanks for all of the suggestions, however after trying even more things out, it seems that the main problem that I have is that what I see in the compositor is simply not the same as I see in other editors or players unless MPEG is selected as the YUV color range. This comes with the caveat that anything outside the 16-235 range is hard clipped.

Is there any way that this functionality could be changed, or an option added to prevent everything being hard-clipped when using MPEG as the YUV color range? Perhaps only hard-clip the range closer to the output after video effects have been applied?

phylsmith2004 28/01/2022 9:05 pm
This post was modified 3 years ago 2 times by phylsmith2004

@glitterball3 

"Is there any way that this functionality could be changed, or an option added to prevent everything being hard-clipped when using MPEG as the YUV color range? Perhaps only hard-clip the range closer to the output after video effects have been applied?"

 

Not that this will help but I logged a BugTracker ticket #605 because I do not want this to get lost if someone comes along to address the issue.

glitterball3 Topic starter 22/04/2022 8:37 am

@phylsmith2004 I have added a note to the BugTracker - Davinci Resolve has a quite logical way of handling this, perhaps something similar could be implemented?

0

I have to say that things are easier for me now I have changed my main camera.

I was shooting 10-bit 420 full range (full range cannot be changed), 2160p at various compression settings i.e. long-GOP or all-i at anything up to 400mbps, sometimes in F-log, sometimes in Eterna profile, with my X-T3. Now I am shooting 10-bit 422 MPEG long-GOP at 50mbps, using Cine D profile or occasionally F1, with my HC-X1500 and I prefer the image quality slightly, this has made most of my Cin troubles disappear. I am now importing 16-235 rather than the 0-255 I was, I set it to MPEG BT709 and use RGBA-FLOAT, now everything looks good from first import into Cin to the final render. I'm damned if I can see any difference in the outputs between full range in and restricted range in and in any case, output will be restricted range BT709 so no I longer see any logic in importing full range by choice.

glitterball3 Topic starter 28/01/2022 9:43 am

@dejay Your settings do make sense for 10-bit, though I'm still recording in 8-bit, and that extra range does make a difference in 8-bit.

0

@glitterball3

Could you do a little test with the settings below to see if the rendering viewed in VLC match with the Compositor, please?

Because it creates a huge file it is better highlight 5-6 seconds in the timeline and using Selection checkbox in the Render window.

 

- Preferences:
  - Playback A TAB
    - Video Out section
        Play everyframe = unchecked
        Video Driver: X11
                      Use direct x11 render if possible = checked
  - Appearance TAB
    - Color section
        YUV color space: BT709
        YUV color range: JPEG


- Format:
  - Video section
      Frame rate: ---
      Width: ---
      Height: ---
      Color model: RGBA-8bit (also RGBA-FLOAT)


- Render:
  File Format: FFMPEG | qt
  Video Preset
    Compression: png.qt
    Bitrate: 0
    Quality: -1
    Pixels: rgb24 (also rgba64be)
glitterball3 Topic starter 30/01/2022 8:55 am
This post was modified 3 years ago 2 times by glitterball3

@igorbeg 

"Could you do a little test with the settings below to see if the rendering viewed in VLC match with the Compositor, please?

Because it creates a huge file it is better highlight 5-6 seconds in the timeline and using Selection checkbox in the Render window."

 

I did the tests with the following combinations of format settings (RGB-8Bit, RGBA-8Bit and RGBA-FLOAT) and render settings (rgb24 and rgba64be), with the following results comparing the compositor to VLC player:

RGBA-8bit + rgba64be is very different to compositor - very dark
RGB-8bit + rgba64be is slightly different (colour hue)
RGBA-FLOAT + rgba64be is very different to compositor - very dark
RGBA-FLOAT + rgb24 looks identical to me
RGBA-8bit + rgb24 looks identical to me

 

BTW, there is a bug in .css of this forum that causes blockquotes not to be formatted properly when it is a "wpforo-qa-comments".

IgorBeg 30/01/2022 10:53 am

@glitterball3

Mmh, in my case it works also for rgba64be.

Could you convert your RGBA-8bit+rgb24 output (.qt) to .mp4 with ffmpeg and see if it still match VLC vs Compositor? I am not an expert on ffmpeg, but in this test let's see how ffmpeg convert by default. I used the command below and in my case it matched.

ffmpeg -i input.qt output.mp4

Could you post the outputs of the ffprobe info about your source.xx, RGBA-8bit+rgb24.qt, RGBA-8bit+rgb24_byFFmpeg.mp4 files? I use this command:

ffprobe -hide_banner -i input.qt

My tests are on a .mov file source: mjpeg (720p yuvj422).

I hope don't waste your time.

glitterball3 Topic starter 30/01/2022 12:38 pm

@igorbeg The result of that conversion looks exactly the same in VLC player (the same as in the compositor).

Output of ffprobe:

RGBA-8bit+rgb24.qt

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'TestPngQt':
Metadata:
major_brand : qt
minor_version : 512
compatible_brands: qt
encoder : Lavf58.76.100
Duration: 00:00:08.60, start: 0.000000, bitrate: 2007414 kb/s
Stream #0:0: Video: png (png / 0x20676E70), rgb24(pc), 3840x2160 [SAR 1:1 DAR 16:9], 2007280 kb/s, 25 fps, 25 tbr, 12800 tbn, 12800 tbc (default)
Metadata:
handler_name : VideoHandler
vendor_id : FFMP
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]

FFMPEG Output.mp4 file:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'output.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.76.100
  Duration: 00:00:08.62, start: 0.000000, bitrate: 33723 kb/s
  Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv444p, 3840x2160 [SAR 1:1 DAR 16:9], 33662 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]

 

 

 

 

0
Topic starter

Here's the output of ffprobe for the original video:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55673f3d3ac0] st: 0 edit list: 1 Missing key frame while searching for timestamp: 3600
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55673f3d3ac0] st: 0 edit list 1 Cannot find an index entry before timestamp: 3600.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '18-surfer-wide.MP4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: mp42avc1
    creation_time   : 2021-12-23T17:01:38.000000Z
  Duration: 00:00:22.56, start: 0.000000, bitrate: 92715 kb/s
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 92496 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2021-12-23T17:01:38.000000Z
      vendor_id       : [0][0][0][0]
  Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      creation_time   : 2021-12-23T17:01:38.000000Z
      vendor_id       : [0][0][0][0]
0

Thanks @glitterball3 for the last information.
So your source video file is .MP4 h264 yuvj420p(pc, bt709).
My tests are on .AVI mjpeg yuvj422p(pc, bt470bg) and .MP4 h264 yuv420p files,
using CinGG-20211231-x86_64-older_distros.AppImage and CinGG-20201031.

Below the Render settings that seem to match good VLC vs Compositor
Could you do a test with these Render, please?

- Render #1:
  File Format: FFMPEG | mp4
  Video Preset
    Compression: h264.mp4
    Bitrate: 0
    Quality: -1
    Pixels: yuv422p (ffprobe will show it is changed to yuvj422p)
            yuvj422p
- Render #2:
  File Format: FFMPEG | webm
  Video Preset
    Compression: webm.webm
    Bitrate: 0
    Quality: -1
    Pixels: gbrp

Settings used in CinelerraGG:

- Preferences:
  - Playback A TAB
    - Video Out section
        Play everyframe = unchecked
        Video Driver: X11
                      Use direct x11 render if possible = checked
  - Appearance TAB
    - Color section
        YUV color space: BT709
        YUV color range: JPEG


- Format:
  - Video section
      Frame rate: ---
      Width: ---
      Height: ---
      Color model: RGBA-8bit

It is strange,
the Render below is playable in VLC if it has been created from CinelerraGG_20201031,
NOT if it has been created from CinelerraGG_20211231. Is it just my problem?

- Render #3:
  File Format: FFMPEG | mp4
  Video Preset
    Compression: h264.mp4
    Bitrate: 0
    Quality: -1
    Pixels: yuv444p (ffprobe will show it is changed to yuvj444p)
            yuvj444p

Thanks!

0
Topic starter

@igorbeg

For me, a webm video using your settings looks very similar to the compositor, however the mp4 video does not. The mp4 video suffers from the same dark/more contrast difference as before. Webm is very slow to render!

the Render below is playable in VLC if it has been created from CinelerraGG_20201031,
NOT if it has been created from CinelerraGG_20211231. Is it just my problem?

Works fine for me using CinelerraGG_20211231 (with the usual too dark/too much contrast).

Also, I also tested CinGG-20201031-x86_64-older_distros.AppImage, but didn't see any different results.

0

@glitterball3

It is good webm-gbrp works also for you.
I don't know why You and me are seeing different results in some cases.
May be it depends on my old software versions and my screen.
To compare with te Compositor I am using both VLC and ffplay.
The info about.
- VLC media player version 2.2.2 Weatherwax
- ffplay version 2.8.6
- My screen is calibrated on sRGB with an external probe.

However I can see that if we use a Rendering format with "Pixels: gbrp" (see Render #2 of my previous post) or "Pixels: rgb" and using ffmpeg by command line to convert the CinelerraGG output to h264 yuv420p/yuv422p/yuv444p, the conversion seems good for you and for me. It might be a good but "undesiderable" workaround (strategy).

Another good Render is...

- Render #4:
  File Format: FFMPEG | mkv
  Video Preset
    Compression: user_ffv1.mkv (or ffvhuff)
    Bitrate: 0
    Quality: -1
    Pixels: bgra

Converted by ffmpeg command line to...

ffmpeg -i input.mkv output.mp4

(default pix_fmt will be yuv444p)

or for yuv420p ...

ffmpeg -i input.mkv -pix_fmt yuv420p output.mp4

My ideas are on a dead line, now. Sorry.

Page 1 / 2
Share: