Issue 30964 in chromium-os: genindex failing to generate chromeos-images index page

[chrom...@googlecode.com]

Status: Untriaged
Owner: ----
CC: ebl...@chromium.org, kr...@chromium.org, josa...@chromium.org,
or...@chromium.org
Labels: Type-Bug pri-1 Area-RelEng

New issue 30964 by d...@chromium.org: genindex failing to generate
chromeos-images index page
http://code.google.com/p/chromium-os/issues/detail?id=30964

Traceback (most recent call last):
File "/usr/local/google/crostools/genindex.py", line 648, in <module>
main()
File "/usr/local/google/crostools/genindex.py", line 639, in main
fastpath=options.cache))
File "/usr/local/google/crostools/genindex.py", line 489, in
ProcessListIntoVersionDict
board, entry, fastpath)
File "/usr/local/google/crostools/genindex.py", line 297, in AddLinkEntry
image_entry[version][channel]['date'] = GetDate(file_path)
File "/usr/local/google/crostools/genindex.py", line 152, in GetDate
date_cache[full_path] = GsDate(full_path).replace('GMT', '')
File "/usr/local/google/crostools/genindex.py", line 141, in GsDate
return match.group(1)
AttributeError: 'NoneType' object has no attribute 'group'


Nothing has changed in the code, so it looks like some new data format/type
is gumming up the works.

I've cleared .genindex_date.pickle and .milestone_cache.pickle but no
help. Needs further investigation.

[chrom...@googlecode.com]

Updates:
Status: Started
Owner: ebl...@chromium.org

Comment #1 on issue 30964 by ebl...@chromium.org: genindex failing to

generate chromeos-images index page
http://code.google.com/p/chromium-os/issues/detail?id=30964

looking at it now

[chrom...@googlecode.com]

Updates:
Status: Fixed

Comment #2 on issue 30964 by ebl...@chromium.org: genindex failing to

generate chromeos-images index page
http://code.google.com/p/chromium-os/issues/detail?id=30964

didn't actually do anything, genindex appears to be working again. Closing
for now.

[chrom...@googlecode.com]

Comment #3 on issue 30964 by ebl...@chromium.org: genindex failing to

generate chromeos-images index page
http://code.google.com/p/chromium-os/issues/detail?id=30964

Seeing this again today. The root cause seems to be when this
channel/version was being uploaded the google storage service was
unavailable.

from:
http://chromegw.corp.google.com/i/chromiumos.release/builders/x86-alex%20full%20release-R20-2268.B/builds/48/steps/Archive%20%5Bx86-alex%5D/logs/stdio
...
Copying
gs://chromeos-image-archive/x86-alex-release/R20-2268.43.0/image.zip...
Failure: BotoServerError: 503 Service Unavailable
.

$ /usr/local/google/crostools/genindex.py -p gs://chromeos-releases -w
/var/www/chromeos-official/index.html2 -v

...
05/31 21:20:35 DEBUG[genindex-137]: + gsutil ls -L
gs://chromeos-releases/dev-channel/x86-alex/2268.43.0/ChromeOS-R20-2268.43.0-x86-alex.zip

Traceback (most recent call last):
File "/usr/local/google/crostools/genindex.py", line 648, in <module>
main()
File "/usr/local/google/crostools/genindex.py", line 639, in main
fastpath=options.cache))
File "/usr/local/google/crostools/genindex.py", line 489, in
ProcessListIntoVersionDict
board, entry, fastpath)
File "/usr/local/google/crostools/genindex.py", line 297, in AddLinkEntry
image_entry[version][channel]['date'] = GetDate(file_path)
File "/usr/local/google/crostools/genindex.py", line 152, in GetDate
date_cache[full_path] = GsDate(full_path).replace('GMT', '')
File "/usr/local/google/crostools/genindex.py", line 141, in GsDate
return match.group(1)
AttributeError: 'NoneType' object has no attribute 'group'

Probably the best bet here is to just catch the error and ignore trying to
display those entries on the chromeos-images/ page. It is currently unclear
if we will be able to fix the perms on these files.

[chrom...@googlecode.com]

Comment #5 on issue 30964 by ebl...@chromium.org: genindex failing to

generate chromeos-images index page
http://code.google.com/p/chromium-os/issues/detail?id=30964

Just return an invalid date when we can't parse the date from google
storage. These errors are not critical in generating this informational
page.

https://gerrit-int.chromium.org/#change,18788