Handling unparsable replays

[sgco...@gmail.com]

Hi everybody! Love the package and am in the midst of developing an MMR system with my friends and me who love playing unbalanced team games. But I'm running into an issue...

I'm trying to import ~1000 replays with sc2reader.load_replays({file_path}, load_level=2) and I'm getting an error deep in the package:

  File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/factories/sc2factory.py", line 148, in load_all
    yield self._load(cls, resource, filename=filename, options=options)
  File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/factories/sc2factory.py", line 152, in _load
    obj = cls(resource, filename=filename, factory=self, **options)
  File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/resources.py", line 301, in __init__
    self._read_data(data_file, self._get_reader(data_file))
  File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/resources.py", line 893, in _read_data
    self.raw_data[data_file] = reader(data, self)
  File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/readers.py", line 274, in __call__
    for i in range(data.read_bits(5))
  File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/readers.py", line 274, in <listcomp>
    for i in range(data.read_bits(5))
  File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/readers.py", line 197, in <listcomp>
    for i in range(
  File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/decoders.py", line 210, in read_uint32
    data = self._buffer.read_uint32()
  File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/decoders.py", line 91, in read_uint32
    return self._unpack_int(self.read(4))[0]
struct.error: unpack requires a buffer of 4 bytes

I get the same error with load_level=1, but I don't see an error with load_level=0. I'm guessing that I've got some replays that sc2reader is trying to parse, but because of it being a mod, custom campaign, or something, has a different format that sc2reader doesn't like (I'm reading straight from the replay folder that SC2 generates for my account).

Instead of tracking down each unique case and writing a decoder for it, I'd rather have sc2reader gracefully pass over replays that it can't handle and move on to the next one. Is there functionality for this somewhere already? Happy to do some work myself to make this happen, but I want to be sure I've got the problem diagnosed correctly before embarking on any coding adventures.

Any help would be greatly appreciated!

glhf,

Steve Couch

[Kevin Leung]

Thanks for pointing out this issue. I am unaware of any graceful failure mode here, so there's no immediate recourse. 2 suggestions:

1. Can you share a small sample replay set to reproduce the issue?

2. You may have better luck putting this as an issue on GitHub since the (albeit small) community of sc2reader developers are more active there https://github.com/ggtracker/sc2reader/issues