Spatial Media RFC registered user_data()

[Chad Fogg]

Hello spatial-media developers,

A few questions...

The following sentence, the v1 Spherical Video RFC document...

   https://github.com/google/spatial-media/blob/master/docs/spherical-video-rfc.md

..suggests that user_data_unregistered  SEI carries the v1 Spherical Video RFC payload in AVC (H.264) bitstreams:

  "User data unregistered SEI message syntax from the ISO-14496-10:2005 (see D.1.6). This message is an SEI message of Payload Type 5.”

The latest VLC Player source code blob appears to only parse the spherical indicator, projection mode (equirectangular)  from unregistered user data in the file container (MP4, MKV), but not from the user_data_unregistered SEI in the elementary bitstream itself. Also, the Spherical Media RFC v1 initial view elements (yaw, pitch, roll) are packed into a separate "prod" MP4 metadata block :

 

https://github.com/videolan/vlc/blob/master/modules/demux/mp4/libmp4.c

The spatial media injector python scripts also only appear to insert Spherical Media RFC metadata into MP4 and MKV headers... and not any elementary bitstream SEI's. 

(1) Does anybody here have a document reference for the UUID  ? 

I cannot find a uuid_iso_iec_11578  (128-bit) code and payload syntax defined anywhere other than FFMPEG,  VLC player...

https://github.com/videolan/vlc/blob/master/modules/demux/mp4/libmp4.h

  static const UUID_t XML360BoxUUID = {

                { 0xff, 0xcc, 0x82, 0x63, 0xf8, 0x55, 0x4a, 0x93,

                  0x88, 0x14, 0x58, 0x7a, 0x02, 0x52, 0x1f, 0xdd } };

...and in your injector tool:  

https://github.com/google/spatial-media/blob/master/spatialmedia/metadata_utils.py

SPHERICAL_UUID_ID = (

    "\xff\xcc\x82\x63\xf8\x55\x4a\x93\x88\x14\x58\x7a\x02\x52\x1f\xdd")

(2) is it the intent to apply initial yaw, pitch roll offsets to rotate the output of the decoded video frame from a decoder, or is that metadata just passed onto the player and HMD  ? 

 

VLC seems to apply the parsed Spherical Media RFC metadata as an initial viewpoint in the player that the user can override with keyboard buttons and mouse movements. 

In the HEVC standards development group, we are trying to start an SEI amendment for carriage of essential,  universally practiced equi-rectangular indicators in AVC and HEVC elementary bitstreams.

Regards,

--Chad

(MovieLabs)