developing a nginx sdch module using xdelta

[Liangbin Li]

I'm developing a nginx sdch module using xdelta, but I found that the standard VCDIFF format (rfc3284) generated by xdelta does not work with chrome.

Can anybody tell me what extension of rfc3284 has been made by chrome in sdch protocol, and what can I do to make xdelta work correctly with chrome? 

[Sreekumar Prathap]

Did you try http://code.google.com/p/open-vcdiff/wiki/HowToUseOpenVcdiff? 

Sree

On Sat, Dec 7, 2013 at 6:46 PM, Liangbin Li <lilb....@gmail.com> wrote:

I'm developing a nginx sdch module using xdelta, but I found that the standard VCDIFF format (rfc3284) generated by xdelta does not work with chrome.

Can anybody tell me what extension of rfc3284 has been made by chrome in sdch protocol, and what can I do to make xdelta work correctly with chrome? 

--
You received this message because you are subscribed to the Google Groups "SDCH" group.
To unsubscribe from this group and stop receiving emails from it, send an email to SDCH+uns...@googlegroups.com.
To post to this group, send email to SD...@googlegroups.com.
Visit this group at http://groups.google.com/group/SDCH.
For more options, visit https://groups.google.com/groups/opt_out.

[Liangbin Li]

As much as I know， “streaming encode” is not supported by open-vcdiff curently，https://code.google.com/p/open-vcdiff/issues/detail?id=17

so I chose xdelta，but I tested the delta data returned by some well worked sdch server, by using the open-vcdiff command line tool, sadly, I doesn't work either. 

the command I used as follow:

$ vcdiff decode -interleaved -dictionary dict < delta > target

before doing this, I have removed the "domain,path.." head in dict file and the "server_id\0" header in delta file.

error string is:

ERROR: The length of the delta encoding does not match the size of the header plus the sizes of the data sections

Error trying to decode data chunk of length 11372

在 2013年12月8日星期日UTC+8下午10时32分24秒，nprathap写道：

[Josh MacDonald]

Hi all,
I've been worrying this would come up -- I've received several reports of an incompatibility between xdelta and open-vcdiff  and I'm not sure whether one or the other or both are responsible. Unfortunately, knowing which is responsible doesn't help much, since both have established users. If we could pinpoint the problem, it might be possible to make a strategic compatibility fix.

FYI you can remove the xdelta application header on the command line with the -A flag.

Josh

[openvcdiff]

Hi Liangbin:

Josh has provided an excellent explanation.  xdelta3 uses some non-standard VCDIFF extensions by default.  The following works for me:

xdelta3 -A -n encode -s source_file target_file delta_file

vcdiff decode -dictionary source_file -delta delta_file -target reconstructed_target

Both the -A (disable application header) and -n (disable checksum) options are necessary.

After executing these commands, the contents of reconstructed_target match those of target_file.

The need for the -A and -n options was documented in this announcement:

http://xdelta.org/2008/09/re-googles-open-vcdiff-google-released.html

Saludos,

Lincoln

[Liangbin Li]

Hi, Josh, Saludos:

Thanks for both of you!

I was tring to decode the delta file which I got from a well worked sdch server using xdelta or open-vcdiff command line tool, so I can infer the extensions made by sdch protocol on rfc3284.

Just one minute ago, I succeeded by using open-vcdiff command line tool, the command is as follow:

$ vcdiff decode -checksum -interleaved -dictionary dict_file -delta delta_file > target_file                  [1]

So we can infer that for a well worked sdch server (I mean, works well with chrome), the vcdiff encode process can be realized as the command blow:

$ vcdiff encode -checksum -interleaved -dictionary dict_file -target target_file > delta_file                 [2]

Now I'd like to know, can I do the same work as command [1] and [2] with xdelta command line tool, maybe by adding some parameters or make some change to the source code?

在 2013年12月10日星期二UTC+8上午8时54分43秒，openvcdiff写道：

[openvcdiff]

Hi Liangbin:

You shouldn't need to specify -interleaved and -checksum to use "vcdiff decode" with a payload that contains checksums and interleaved format.  The decoder automatically enables those features based on the version code of the encoded delta.  However, the xdelta3 tool cannot decode delta files that use vcdiff's checksum or interleaved format.  These are not part of the VCDIFF standard.

You may need to remove the first three lines of the downloaded dictionary (the domain and path information) and the first 9 bytes of the SDCH-encoded HTTP response (the dictionary checksum) in order to run "vcdiff decode" against those two files.  The commands "tail --lines +4 dictionary_response > dictionary" and "tail --bytes +10 sdch_response > sdch_delta" will do this for you.

Cheers,

Lincoln