wrong color-space metadata after encode from RGB

[MartinPulec]

this refers to CESNET/UltraGrid#341

hi, when QSV is compressing from a RGB format (BGRA for now; but applies also to X2RGB), it converts unconditionally to limited-range YUV BT.601, but sets stream metadata according to AVCodecContext::color_primaries and AVCodecContext::color_range. If those are not set explicitly, it sets full-range to metadata, anyways.

Please see following test case:

1. input
   ffmpeg -y -f lavfi -i color=#ff0000 -t 1 -pix_fmt bgra [params] -c:v hevc_qsv out.h265

2. output evaluation

   mediainfo --Language=raw --Full --Inform='Video;%matrix_coefficients%,%colour_range%' out.h265 &&
    ffmpeg -y -loglevel quiet -i out.h265 -strict -1 -frames 1 out.y4m; sed '1,2d' < out.y4m | od -t x1 
   

input RGB=[0xff,0x00,0x00] corresponds¹ to full-range YCbCr BT.601 [76,85,255]=[0x4c,0x55,0xff], limited range [65,91,237]=[0x51,0x5b,0xed] ²

¹ computed using this coefficient; Cb,Cr coefficients are offset by 128
² or Y=67=0x53 if using Y range 16-240, which I think is correct

evaluation:

1. no params for 1. above - [0x51,0x5b,0xef], mediainfo: ",Full"
2. -colorspace bt709 - [0x51,0x5b,0xef], mediainfo: "BT.709,Full"
3. -colorspace bt709 -color_range tv- (ditto), mediainfo: "BT.709,Limited"
4. -color_range tv - (ditto), mediainfo: ",Limited"
5. -colorspace smpte170m -color_range tv - (ditto), mediainfo: "BT.601,Limited"

So, in all cases, the output is in limitted-range YCbCr 601, but metadata are correct only in case 5 (and also 4, assuming that implicit value is OK).

PS: -vf lavfi -i color=#ff0000 -t 1 -pix_fmt bgra seems to convert to [0x00,0x00,0xfd,0xff] - the swscale conversion is not entirely accurate but I don't know how to produce single-color RGB directly, without conversion from YUV (-vf lavfi -i color produces YUV)

[xhaihao]

Thanks for catching this issue, and I'll look into it.

[alatteri]

Hi @xhaihao

Any update on this issue?

Thanks.

[xhaihao]

-colorspace <arg>, -color_range <arg> etc are used to specify the color properties only, there is no color conversion here.

For example:

$ ffmpeg -y -loglevel verbose -f lavfi -i color=#ff0000 -t 1 -pix_fmt bgra out.0.rgb

The output is bgra(pc, gbr/unknown/unknown, progressive)

$ ffmpeg -y -loglevel verbose -f lavfi -i color=#ff0000 -t 1 -pix_fmt bgra -colorspace bt709 out.1.rgb

The output is bgra(pc, bt709/unknown/unknown, progressive)

$ ffmpeg -y -loglevel verbose -f lavfi -i color=#ff0000 -t 1 -pix_fmt bgra -colorspace bt709 -color_range tv out.2.rgb

The output is bgra(tv, bt709/unknown/unknown, progressive)

The color properties are different in the above examples, however out.0.rgb, out.1.rgb and out.2.rgb are same.

$ md5sum out.*
ef62d4f82747b3211118b905c649b5b0  out.0.rgb
ef62d4f82747b3211118b905c649b5b0  out.1.rgb
ef62d4f82747b3211118b905c649b5b0  out.2.rgb

[xhaihao]

PS: -vf lavfi -i color=#ff0000 -t 1 -pix_fmt bgra seems to convert to [0x00,0x00,0xfd,0xff] - the swscale conversion is not entirely accurate but I don't know how to produce single-color RGB directly, without conversion from YUV (-vf lavfi -i color produces YUV)

You may use -f lavfi -i color=#ff0000,format=bgra to produce single-color RGB directly

[MartinPulec]

thank you much for looking into this

-colorspace , -color_range etc are used to specify the color properties only, there is no color conversion here.

I am aware of the fact that it just sets input metadata.

Anyways, if I deduced it correctly, so QSV does unconditional conversion from (full-range) RGB to limited-range BT.601 YCbCr (please correct me, if I am wrong). In this case, I believe that in the HEVC metadata there should be limited-range YCbCr, not full-range (optionally also BT.601). (The conversion will be correct only for input RGB pc/bt601, of course.)

Currently even though the input RGB using pc/bt601 is converted to limited-range YCbCr, while in codestream metadata remains PC/full range (seems to be just copied from the AVCodecContext::color_range). Well, as a workaround, I could set the source color range to AVCOL_RANGE_MPEG (limited) to enforce correct output codec metadata, but this will be just pretended. Please also note that not setting anything explicitly causes the output to have undefined color primaries (which is OK) but still telling to be full-range YCbCr.

[alatteri]

Hi @xhaihao,

Any thoughts on Martin's latest information?

Thanks,
Alan

[xhaihao]

@alatteri Yes the driver does an internal conversion, however the default range (full range) is set by FFmpeg, not QSV. Note the two commands below have different color ranges, it is full range in 1 however it is limited range in 2

1. $ ffmpeg -y -f lavfi -i color=#ff0000 -t 1 -pix_fmt bgra -c:v hevc_qsv out.h265

2. $ ffmpeg -y -f lavfi -i color=#ff0000,format=bgra -t 1 -c:v hevc_qsv out.h265

@XinfengZhang Any thought about this issue ? Is it possible to expose the capability in VA-API level so that user may control the conversion ?

[XinfengZhang]

plan to add a new interfaces to report the color space support for a given config + format
something like vaGetSurfaceFormatAttributes [refer: https://github.com/intel/libva/pull/589/files]
then add a field to specify the color space in SPS.

TBH, because no requirement previously , I defer it.

[alatteri]

Hi @XinfengZhang

When do you think you'll be able to get this merged? I see it has been pending for a very long time.

Thanks,
Alan

TBH, because no requirement previously , I defer it.

@MartinPulec Do you think this is the fix we need? ->

plan to add a new interfaces to report the color space support for a given config + format
something like vaGetSurfaceFormatAttributes [refer: https://github.com/https://github.com/intel/libva/pull/589/files]
then add a field to specify the color space in SPS.