Handling of 0 length atoms

[Dan Hinsley]

While the Apple spec says that a zero length top level atoms indicates that it takes up the rest of the file, all players (including iTunes) correctly play a file that has an atom with 0 length and a type of all 0's.  However Mp4V2 will corrupt a file like this when tagging.

I've done quite a bit of debugging and have found that if you take the 8 bytes of 0's and turn then into a free atom with datasize of 0, then MP4V2 will tag it correctly.  But when I try to "force" this in ReadAtom by checking in the if dataSize == 0 leg for a type of all 0's as well, and then setting the dataSize to 0 and the type to free, it still trashes the file (I've debugged through this function both with the patched file and without, and it appears to me to be doing the same thing, but something upstream from the call, or downstream from the write, must still be confused).

I've uploaded the file to http://www.danhinsley.com/downloads/MP4V2 Test Data/test1.mp4.  The structure of the file before the write is:

                                                                                       i.      Moov atom @x20 for 0x227d57

                                                                                     ii.      Free atom at 0x227d77 for 0xb4969

                                                                                    iii.      2 words of 0’s at 0x2dc630

                                                                                   iv.      Mdat atom at 0x2dc6e8 for 1b3b3f25 (goes to end of file)

and after

                                                                                       i.      Free atom at 0x20 for 0x227d57

                                                                                     ii.      Free atom at 0x227d77 for b4969

                                                                                    iii.      2 words of 0’s at 0x2dc6e0

                                                                                   iv.      Mdat atom at 0x2dc6e8 for 1b3b3f25

                                                                                     v.      Mdat atom at 0x1b69060d for 0x1f

                                                  vi.  Moov atom at 0x1b69062c for 0x228003 (goes to end of file)

What I tried to do is replace the if (dataSize == 0) with:

    if (dataSize == 0) {
        // extends to EOF
        if ((int32_t) *type == 0) {
            // even though it doesn't match the spec, some files have two 32 bit words of
            // 0's as a top level atom.  These should just be turned into a zero size free atom and skipped
            dataSize = 8; // so it will turn into 0 after hdrsize subtracted
            type[0] = 'f';
            type[1] = 'r';
            type[2] = 'e';
            type[3] = 'e';
        } else {
            dataSize = file.GetSize() - pos;
        }
    }

Now when I debug through with the patched and unpatched file, it looks to me that they have the same behavior.  I'm thinking there must be something in the write leg, but I can't see where to go from here.

If anyone can give me any tips, I'll be glad to try to finish debugging this and provide a patch.

Thanks,

Dan

[dam...@gmail.com]

iTunes and QuickTime will plays it because the 0 length atom is just before the mdat atom and after the moov atom. After tagging the file with mp4v2, the 0 length atom will remain unchanged (because you are treating it like a free atom, existing free atom aren't rewritten by mp4v2), but the moov atom will be after the 0 length atom, and so nothing will work.

The right fix might be to create a null type atom in mp4v2 that is automatically rewritten to a free atom when closing the file.

[Dan Hinsley]

Is there any atom that is rewritten by MP4V2 but won't impeded the stream (and is only the length and type fields)?  I'd like to be able to fix this.  I can recognize the bad fields, and return an atom to cover them, but now I understand that free won't do it. 

[Dan Hinsley]

OK, what I tried was turning the 8 bytes of 0's into an Xtra atom with length 0 by using the following code in ReadAtom after the check for extendedType:

    if (dataSize == 0) {
        // extends to EOF
        if ((int32_t) *type == 0) {

             //even though it doesn't match the spec, some files have two 32 bit words of
             //0's as a top level atom.  These should just be turned into a zero size free atom and skipped

            dataSize = 8; // so it will turn into 0 after hdrsize subtracted

            type[0] = 'X';
            type[1] = 't';
            type[2] = 'r';
            type[3] = 'a';

        } else {
            dataSize = file.GetSize() - pos;
        }
    }

So if it's an atom with a zero size, it's treated as before, but if it's 8 bytes of 0's, then it's turned into an Xtra atom of length 0.  But when I do this, after I write to the file the initial moov atom gets turned into a free atom and the file is unreadable by anything (MP4V2 or any video player).

Now if I go into the file in question and turn the 8 bytes of 0's into an Xtra atom of length 0, it reads, tags and plays fine.

So I'm at a loss of how to next attack this.

Dan

[christ...@cbau.wanadoo.co.uk]

I'd say the file was bad in the first place, but all kinds of players played it nevertheless. 

Most likely the various players don't care at all where the 'mdat' atom is. 'trak' atoms will have byte offsets into the file and read from those offsets, and they probably don't care whether the data is in an mdat atom or not - of course the data still has to be correct!

Your original file has a 'moov', a 'free', and a '0000' atom with the '0000' atom (I mean all bytes zero) extending to the end of the file. The 'mdat' atom is missing, but the players don't care. After the change, you have a 'free', a 'free', and a '0000' atom. The 'moov' is missing and the players care very much!

I'd probably hack the atom reading portion - if both type and size are 0 for a top level atom, then I'd probably scan how many further zero bytes there are, and pretend it is all a 'free' atom with a size of 8, or more if there were more zero bytes.