Sorry for the delay in getting back to you ... I've been down with the flue for the last week and saw your mail just today ... not feeling that great today either, but I'll see what I can figure out ...
My first data point is that the difference betwee CentOs and Mac
appears to be simply related to how much TOC you've got allocated
on each machine ... after bumping up the TOC on the mac I get
exactly the same error ... this is for 3.4.0 right now ... I'll
give it a try on 3.3.x and see if I get the same error ... but
right now it certainly looks like a bug in the zip reader code...
Dale
--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Sorry for the delay in getting back to you ... I've been down with the flue for the last week and saw your mail just today ... not feeling that great today either, but I'll see what I can figure out ...
My first data point is that the difference betwee CentOs and Mac appears to be simply related to how much TOC you've got allocated on each machine ... after bumping up the TOC on the mac I get exactly the same error ... this is for 3.4.0 right now ... I'll give it a try on 3.3.x and see if I get the same error ... but right now it certainly looks like a bug in the zip reader code...
Dale
On 12/14/17 4:43 AM, Mariano Martinez Peck wrote:
Hi Dale,
I am trying to load HighchartsSt as a dependency with Metacello. This worked perfectly on Pharo but when I try on GemStone I am getting an error. The way to reproduce it for me is as simple as this:
MetacelloGemStonePlatform new downloadZipArchive: 'https://github.com/ba-st/HighchartsSt/zipball/highchart6-import' to: '/tmp/github-3574-bastHighchartsSthighchart6import.zip'
And this yields me the attached error.
I don't think Github is giving me broken zips. Furthermore, if I do a `wget` then `unzip` it does work correct. I am trying to check if this fails in older GemStone than 3.4 but I am failing to start my old stones too grrrrr...
--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Okay I get the same error when running with 3.3.6 and a large TOC .... so I will see what I can figure out where the bug is ...
Dale
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Metacello" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+unsubscribe@googlegroups.com.
Hmmm an off by 6 error .... 6 bytes short of the signatures
... 6 bytes in 70M is not a normal off-by error ... We're
storing the itsCollection as a String and the zipped contents
should actually be UTF8 so I am wondering if there is an 8 byte
character issue involved here somewhere???
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+...@googlegroups.com.
Hmmm .... replaced RWBinaryOrTextStream
with FileStreamPortable ... which BTW should do better memory
wise, since FileStreamPortable does not load the entire file
into memory, but instead depends up on the GsFile implementation
for buffering from disk ... but ended up with same error at the
same spot in the file so I don't think the error is related to
String/UTF8 errors ... sure enough when looking at it a bit closer,
we have yet to read a single member from the the zip file, so
the bookkeeping calculations must be allo wrong ... shouldn't be
too hard to find at this point ...
Dale
I just tried reading the zip file using the Pharo6 ZipArchive code and I get the same error as in GemStone ... so the error is in the (old) ZipArchive reader code in Pharo6 as well as GemStone ... Pharo6 is probably using something other than the ZipArchive reader for cracking zip files these days ...
Dale
outputFileName := '/tmp/test.zip'.
ZnClient new
url:
'https://github.com/ba-st/HighchartsSt/zipball/highchart6-import';
downloadTo: outputFileName.
^ ZipArchive new
readFrom: outputFileName asFileReference
So the bug _is_ in ZipArchive and seems to exist is Pharo6 and
GEmStone ... this is the Pharo6.0 I am running:
Pharo 6.0
Latest update: #60319
So perhaps the ZipARchive bug has been recently fixed or ???
Dale
Mariano,
The url that you used in your earlier email: 'https://github.com/ba-st/HighchartsSt/zipball/highchart6-import' is not the url that I get when I try to do to the following:
Metacello new
baseline: 'HighchartsSt';
repository: 'github://ba-st/HighchartsSt:master/repository';
get
Which btw worked for me when executing the expression for both GemStone and my version of Pharo6.0 .... on the mac ... and the size of the two zip files differs greatly:
successful downloaded by ZinClient for pharo ('github://ba-st/HighchartsSt:master/repository'):
-rw-r--r--@ 1 dhenrich wheel 9429194 Dec 18 16:26 github--bastHighchartsStmaster437317039036239984103512910491.zip
successful ownloaded by curl ('github://ba-st/HighchartsSt:master/repository'):
-rw-r--r-- 1 dhenrich wheel 9429194 Dec 18 16:28
github-34040-bastHighchartsStmaster.zip
Contrast that with the size of the file for the hichchart6-import url:
-rw-r--r-- 1 dhenrich wheel 82456389 Dec 18 12:14
github-3574-bastHighchartsSthighchart6import.zip
Which cannot be handled correctly by wither Pharo6 or GemStone ... and I've verified that the ZnClient/ZipArchive combo is being used by my version of Pharo6.0 and looks like the latest version that should be used by 6.0 from the github repo[1]...
So right now I'm inclined to think that we're comparing apples and oranges sizne the urls are very different and the size of the downloads are very different as well ... oh now I see that you are not using the master branch ... so I've wasted a bit of time going down this path ...
I'll retest everthing yet again using your branch and see what I find out ...
Dale
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+...@googlegroups.com.
Mariano,
The url that you used in your earlier email: 'https://github.com/ba-st/HighchartsSt/zipball/highchart6-import' is not the url that I get when I try to do to the following:
Metacello new
baseline: 'HighchartsSt';
repository: 'github://ba-st/HighchartsSt:master/repository';
getWhich btw worked for me when executing the expression for both GemStone and my version of Pharo6.0 .... on the mac ... and the size of the two zip files differs greatly:
successful downloaded by ZinClient for pharo ('github://ba-st/HighchartsSt:master/repository'):
-rw-r--r--@ 1 dhenrich wheel 9429194 Dec 18 16:26 github--bastHighchartsStmaster437317039036239984103512910491.zip
successful ownloaded by curl ('github://ba-st/HighchartsSt:master/repository'):
-rw-r--r-- 1 dhenrich wheel 9429194 Dec 18 16:28 github-34040-bastHighchartsStmaster.zip
Contrast that with the size of the file for the hichchart6-import url:
-rw-r--r-- 1 dhenrich wheel 82456389 Dec 18 12:14 github-3574-bastHighchartsSthighchart6import.zip
Which cannot be handled correctly by wither Pharo6 or GemStone ... and I've verified that the ZnClient/ZipArchive combo is being used by my version of Pharo6.0 and looks like the latest version that should be used by 6.0 from the github repo[1]...
So right now I'm inclined to think that we're comparing apples and oranges sizne the urls are very different and the size of the downloads are very different as well ... oh now I see that you are not using the master branch ... so I've wasted a bit of time going down this path ...
I'll retest everthing yet again using your branch and see what I find out ...
Dale
On 12/18/17 2:45 PM, Mariano Martinez Peck wrote:
Answering from phone... Yes it did work in GemStone but not the zip archive but the sender download to from Metacello platform . Which as you concluded itDoes something different.. I think it was using zinc streams.. I did not try the isolated zip stream code in Pharo as you just said.
So I agree it must be a zip stream bug in both dialects.
Thanks
but I am trying to understand why you are able to load it cleanly
in Pharo6.0 when my tests show that it should not load cleanly in
pharo 6.0 either ... the metacello get expression in Pharo6.0 ends
up calling the ZnClient/ZipArchive code that fails for me ... so I
think that either the ZipArchive bug was fixed in _your_ version
of Pharo 6.0 or ... we are comparing apples to oranges and your
claim that "it works in pharo" is not necessary correct ... if the
bug is fixed in ZipArchive then it can be fixed in GemStone if not
... then I assume that you will find that the github load will
fail in pharo as well --- as it does for me ...
Dale
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+...@googlegroups.com.
but I am trying to understand why you are able to load it cleanly in Pharo6.0 when my tests show that it should not load cleanly in pharo 6.0 either ... the metacello get expression in Pharo6.0 ends up calling the ZnClient/ZipArchive code that fails for me ... so I think that either the ZipArchive bug was fixed in _your_ version of Pharo 6.0 or ... we are comparing apples to oranges and your claim that "it works in pharo" is not necessary correct ... if the bug is fixed in ZipArchive then it can be fixed in GemStone if not ... then I assume that you will find that the github load will fail in pharo as well --- as it does for me ...
Dale
On 12/18/17 4:51 PM, Mariano Martinez Peck wrote:
Again from phone sorry....
Yes, we are using a special branch for highcharts 6 in which have a lot more of auto generated code than the the master. That's the difference in the zip files and kind of confirms my suspicious that zip file size is the key point of the bug.
After trying yet again, I am unable to get to work in the version
of Pharo6.0 that I'm using without getting exactly the same error
I getting for GemStone:
Metacello new
baseline: 'HighchartsSt';
repository:
'github://ba-st/HighchartsSt:highchart6-import/repository';
get
Geez ... pharo6.0 and pharo6.1 are very different AFAIK ...
iceberg is loaded in 6.1 while I assume that the code path for
downloading github zip files is same for pharo6.1, ... there is
the funky "enable metacello" stuff needed(?) when iceberg is being
used and I do not know what kind of monkey business is involved
there .... I just know that at ESUG the enable Metcello stuff
broke Metacello is several different ways that did not make sense
...
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+...@googlegroups.com.
Also pharo6.1 uses a different pharo vm ....
Also pharo6.1 uses a different pharo vm ....
On 12/18/17 5:29 PM, Dale Henrichs wrote:
Geez ... pharo6.0 and pharo6.1 are very different AFAIK ... iceberg is loaded in 6.1 while I assume that the code path for downloading github zip files is same for pharo6.1, ... there is the funky "enable metacello" stuff needed(?) when iceberg is being used and I do not know what kind of monkey business is involved there .... I just know that at ESUG the enable Metcello stuff broke Metacello is several different ways that did not make sense ...
Before I spend anymore time on this could you confirm that whether or not this expression works in your pharo image ... and share the exact version information about what you using:
outputFileName := '/tmp/test.zip'.
ZnClient new
url: 'https://github.com/ba-st/HighchartsSt/zipball/highchart6-import';
downloadTo: outputFileName.
^ ZipArchive new
readFrom: outputFileName asFileReference
Then set a halt in this method[1] in your image and run this metacello expression:
Metacello new
baseline: 'HighchartsSt';
repository: 'github://ba-st/HighchartsSt:highchart6-import/repository';
get
and see if you hit the halt ... if you do hit the halt, confirm that you get the same result as from the first expression with ZnClient/ZipArchive ...
if you don't hit the halt then figure our what code is being used in your image to download the file ...
Remember, if ZipArchive is being used and it works you image, then there's a chance that the ZipArchive patch (which is not present in Pharo6.0) can be applied to the GemStone code ... if not, then a patch will depend upon what you learn about the code path actually being used for Pharo6.1 ...
It's unfortunate that I didn't know you were using pharo6.1 earlier in the day:)
Dale
[1] https://github.com/Metacello/metacello/blob/master/repository/Metacello-Platform.pharo60.package/MetacelloPharoPlatform.class/instance/downloadZipArchive.to..stOn 12/18/17 5:13 PM, Mariano Martinez Peck wrote:
Uhhh good point. I am using 6.1 but I guess that shouldn't change much right?
To unsubscribe from this group and stop receiving emails from it, send an email to metacello+...@googlegroups.com.
Mariano,
I'm not quite sure why you are relying on zip downloads from
github for your production builds ... you should be able to clone
the HighChartsSt project to your local disk and rely on a
Metacello lock to ensure that the project is loaded from your
local clone ... you should do this for all of the github projects
that you are using ...
Using local clones and Metacello lock is definitely superior than
attempting to hack around with alternate schemes for dealing with
zip files .... the github project is git-based after all ...
Dale