Is sc2reader still up to date?

[Aldo Ercolani]

Hi I'm new here this is my first post.

I've been using sc2reader happily for few months in the first half of 2014.

I've already filed a issue on GitHub on a specific issue but now I'm writing here because I'm finding so many problems in trying to make it work again, that I'm wondering if it's completely broken or if it's a problem of mine, and I'd like some kind of general feedback on this.

So, to the point.

I am using a Mac, Yosemite OS.

Two days ago I started to use again my mid-2014-working installation of sc2reader (v0.6.5) running on python 2.7, and started to have three errors, after the third I stopped trying to fix things and come here.

I report them in order of appearance.

I am having these errors using the provided script sc2printer on the attached s2gs file.

I am NOT having these errors using sc2printer on the provided /test_s2gs/ files.

#1 - AttributeError: 'ByteDecoder' object has no attribute 'read_unit64'

The last two lines of the call stack trace are:

  File "build/bdist.macosx-10.9-intel/egg/sc2reader/decoders.py", line 371, in read_struct

AttributeError: 'ByteDecoder' object has no attribute 'read_unit64'

Actually the error is there, if you look at line 371 of decoders.py you'll see this statement:

            data = self._buffer.read_unit64()

 

The "unit64" part is a typo, it's obvious it should be "uint64" as stated in the documentation (http://sc2reader.readthedocs.org/en/latest/decoders.html?highlight=bytedecoder#sc2reader.decoders.ByteDecoder)

Looking through decoders.py history, I find this error is there from the very first version.

So, question #1: is it really possible nobody in the world ever had to parse a s2gs file with a 64-bit data since march 7th 2013? It looks so strange to me...

Anyway, that was easy to fix and fixed by myself changing to "uint64()".

Launched again sc2printer and got this new one:

#2 - TypeError: ord() expected a character, but string of length 0 found

Again, last two lines of the call stack point us to decoders.py:

  File "/Library/Python/2.7/site-packages/sc2reader-0.6.5-py2.7.egg/sc2reader/decoders.py", line 339, in read_struct

TypeError: ord() expected a character, but string of length 0 found

The guilty line seems to be:

        datatype = ord(self._buffer.read(1)) if datatype is None else datatype

That is not easy to understand for me.

It's not a typo, it has to do with inner working of all this stuff.

As far as I understand this, it looks like we're trying to read 1 byte while at EOF of our byte stream.

I tried to patch it up just to check for EOF before trying to decode to ord() and, if EOF, simply return a "NONE" datatype.

Hoping that calling code would handle such "None" datatype.

Of course that didn't happen and the error became "Unknown Data Structure: 'None'".

Again, I'm wondering I am the only person in the world having this strange issue?

BTW, I gave up on this and installed from github the latest "master" version (namely 0.7.0-pre).

But the errors (both) are still there.

Now, I'm stuck.

Did anyone have these errors? Or one of them?

Errors #1 and #2 looked really strange to me since it looks like the problem is in my replay, but it's an absolutely normal custom game.

I already filed an issue for #2.

Should I file an issue for #1 too?

Thanks

[Graylin Kim]

Hey Aldo, sorry to hear you are having problems.

I'll push a fix for the unit64 typo. It is quite likely that the line of code has never been run; normal replay files have no use for a 64 bit integer.

Re: your second issue. The code is written to fail as described. If you run out of bytes while decoding a data structure it is impossible to know how much of the previous data decoded is corrupt. The mistake could have happened a couple bytes ago or right near the beginning of the file. If parsing fails we must assume the whole result is corrupt.

You are seeing both of these issues because you are working with s2gs files. The only known s2gs user is http://ggtracker.com and I haven't looked at those files since initially doing the work years ago. It is quite possible that you will run into bugs that no one else is seeing. GGTracker hasn't reported any issues to me but it could be that they just throw the occasional bad s2gs file out.

Can you confirm if this second issue is frequent when you parse s2gs files or just this one?

~Graylin

--
You received this message because you are subscribed to the Google Groups "sc2reader" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sc2reader+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Graylin Kim]

Since most users aren't working with s2gs files, it'd be great if you could add your follow-ups on the issue you opened on github instead of here.

Thanks for following up on list though, sometimes sc2reader issues get lost in the flood of messages I get from Github.

~Graylin

[Aldo Ercolani]

Hey Graylin

thank you for your really quick reply.

That really goes far beyond my expectations.

Sure, I'm gonna follow up on github issue.

It's not an occasional error, I am getting this on 45 files on a total of 117.

I can provide them all if you need.

I'm cross posting this info on github too.

Btw you hit my curiosity when you said I am the only one to read s2gs files besides ggtracker.

What I need to do is to dump the final game summary scores for each player of a given match.

I could not even be one of those players.

Instead of downloading the whole replay files and parsing through them, I thought it was simpler to download the s2gs file (mouse-hovering over the wanted game in the player's match history will actually trigger such download) and parse this.

Is there another way to get the job done?

Thanks a lot.

Aldo.

[Graylin Kim]

That is pretty much the only way I know of to get s2gs files. I can't say that GGTracker is the only user, but they are the only s2gs user I've talked to. Most people find that the replay files can give them what they need.

~Graylin

[Aldo Ercolani]

I was wondering wether parsing the s2gs file is the only way to get the game summary scores, or there is another way.

[Graylin Kim]

You can likely generate the numbers you want by processing the tracker events (docs here). The numbers might not match 100% but you should be able to get close.

On Mon, Mar 16, 2015 at 10:59 AM, Aldo Ercolani <aldo.e...@gmail.com> wrote:

I was wondering wether parsing the s2gs file is the only way to get the game summary scores, or there is another way.

--