SetTotalBytesLimit() help

[Xypher]

I got error message below.

I0428 19:22:43.862210 15440 net.cpp:219] Network initialization done.

I0428 19:22:43.862217 15440 net.cpp:220] Memory required for data: 1250445440

[libprotobuf ERROR google/protobuf/io/coded_stream.cc:171] A protocol message was rejected because it was too big (more than 0 bytes).  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.

init done 

opengl support available 

F0428 19:22:44.038230 15440 blob.cpp:215] Trying to copy blobs of different sizes.

*** Check failure stack trace: ***

So I modified the buffer size in src/caffe/util/io.cpp like this.

bool ReadProtoFromBinaryFile(const char* filename, Message* proto) {

  int fd = open(filename, O_RDONLY);

  CHECK_NE(fd, -1) << "File not found: " << filename;

  ZeroCopyInputStream* raw_input = new FileInputStream(fd);

  CodedInputStream* coded_input = new CodedInputStream(raw_input);

  // modified line

  //coded_input->SetTotalBytesLimit(1073741824, 536870912);

  coded_input->SetTotalBytesLimit(4294967296, 2147483648);

  bool success = proto->ParseFromCodedStream(coded_input);

  delete coded_input;

  delete raw_input;

  close(fd);

  return success;

}

But, still the error message isn't solved. I recompiled caffe, and the problem is same.

How can I solve this problem?

[Xypher]

I tried to change the default buffer in coded stream.h as below.

  // static const int kDefaultTotalBytesLimit = 64 << 20;  // 64MB

  static const int kDefaultTotalBytesLimit = 4000000000 << 20;  // 64MB

  // static const int kDefaultTotalBytesWarningThreshold = 32 << 20;  // 32MB

  static const int kDefaultTotalBytesWarningThreshold = 2000000000 << 20;  // 32MB

However, the error msg is same that the memory is limited..

Is there any other solution guys?

2015년 4월 28일 화요일 오후 7시 34분 31초 UTC+9, Xypher 님의 말: