Downloaded files corrupted on windows ntfs - possible EOL conversion issue

25 views
Skip to first unread message

Siva Thiyagu

unread,
Feb 19, 2018, 2:08:22 AM2/19/18
to Google APIs Client Library for C++
The following c++ code is able to download google drive files 

      googleapis::StringPiece strpc = filePtr->get_download_url();
      googleapis::client::HttpTransport* ptrHttp = service_->transport();
      string filePath("C:/temp/"+filePtr->get_title().as_string());
      googleapis::client::DataWriter *dataPtr = googleapis::client::NewFileDataWriter(filePath);
      googleapis::client::HttpRequest *ptrHttpReq =
      ptrHttp->NewHttpRequest(googleapis::client::HttpRequest::GET);
      ptrHttpReq->set_url(strpc.as_string());
      ptrHttpReq->set_content_writer(dataPtr);
      ptrHttpReq->set_credential(&credential_);
      googleapis::util::Status rsp = ptrHttpReq->Execute();

But the resulting files on the NTFS windows local directory is corrupted (primarily images).  It looks like  the classic EOL issue CR LF conversion issue . A binary compare shows "0A" is replaced as "0D 0A" and the file sizes on the local folders are consistently 1kb larger than the size on the drive.

I have a screen shot of the hex diff attached here.

This problem does not happen when I directly download the same files from google drive via their browser client (right-click>download)

Here are a few files whose size differ consistently

Downloaded              file size 
bytes                    on disk (NTFS)
283374           --->    284427             delta of 1053
224201           --->    225025             delta of 824
248961           --->    249874             delta of 916
249438           --->    250413             delta of 975

Also buffer size reported by DataWriter when writing is the same reported in the browser client via file details information - so the received payload in the http response is the same size as on the drive. The problem is likely to be during file writing I guess.

Is there any binary stream post processing I need to do when the file is written? I am not able to tell at what layer this problem is creeping in.

A similar issue is reported here in stackoverflow, but I dont see any change trying to set file content type in the HttpRequest object


0D-0A-hex-diff.JPG
Reply all
Reply to author
Forward
0 new messages