Re: [Bug 242060] Re: no downloadable source distribution weekly? nightly?
Terry Brown
which should have snapshots of the trunk sources for last night, the
night before, the night 5 days ago, and the night 10 days ago.
Currently the snapshots are separated by minutes, because the page is
only an hour old.
Cheers -Terry
On Tue, 01 Jul 2008 14:01:08 -0000
"Edward K. Ream" <edre...@gmail.com> wrote:
> On Mon, Jun 30, 2008 at 1:00 PM, ne1uno <ev...@mail.ru> wrote:
>
> >
> > the request for a nightly was a workaround entirely appropriate.
>
>
> I agree. Hope your bzr problems go away soon. When bzr works it
> seems to be the most convenient way.
>
> >
> >
> > thanks for 4.5b1
> > sans a few plugins that crash on load it's the best ever.
>
>
> Which plugins crash? I'd like to fix this for b2, due Friday.
>
> Edward
>
Edward K. Ream
I've made this page:
http://www.greygreen.org/leo/
which should have snapshots of the trunk sources for last night, the
night before, the night 5 days ago, and the night 10 days ago.
Cool. How did you make these snapshots?
Edward
Terry Brown
"Edward K. Ream" <edre...@gmail.com> wrote:
> Cool. How did you make these snapshots?
With this script which should run every night. Note that the directory
names like leo-bzr-snapshot-20080701 are embedded in the .zip file,
which means the checksums always change, even when the content doesn't.
# leomirror.py $Id$
# Author: Terry Brown
# Created: Tue Jul 01 2008
import os, glob, stat, hashlib, ftplib
os.system("""
TS=$(date +%Y%m%d%H%M)
FN=leo-bzr-snapshot-$TS
bzr branch lp:leo-editor $FN
rm -rf $FN/.bzr*
zip -r $FN.zip $FN
rm -rf $FN
""")
zips = glob.glob('*.zip')
zips.sort()
culls = zips[:-11]
post = []
for i in [1,2,5,10]: post.append(zips[-i])
html = file("index.html",'w')
html.write("""
<html><head><title>Leo bzr snapshots</title></head><body>
<h1>Leo bzr snapshots</h1>
<p>These are snapshots of Leo's bzr trunk branch. Go to the
<a href="http://webpages.charter.net/edreamleo/front.html">Leo
webpage</a> for links to the latest release, directions
for accessing the bzr repository directly, and other Leo
information.</p>
""")
for i in post:
s = os.stat(i)
s = s[stat.ST_SIZE]
m = hashlib.md5(file(i).read()).hexdigest()
s1 = hashlib.sha1(file(i).read()).hexdigest()
html.write("""<div><hr/><a href="%s">%s</a><br/>
Bytes: %d</br>
SHA1: %s</br>
MD5: %s</br>
""" % (i, i, s, s1, m))
html.write("""</body></head>""")
html.close()
ftp = ftplib.FTP('secret.net', 'secret', 'secret')
ftp.cwd('public_html/leo')
rzips = [i for i in ftp.nlst() if i.endswith('.zip')]
rzips.sort()
rculls = rzips[:-11]
for i in rculls:
ftp.delete(i)
for i in post:
if i not in rzips:
ftp.storbinary("STOR %s" % i, file(i))
ftp.storbinary("STOR index.html", file("index.html"))
ftp.quit()
if culls:
os.system("rm %s" % (' '.join(culls)))
Terry Brown
Terry Brown <terry_...@yahoo.com> wrote:
> > Cool. How did you make these snapshots?
>
> With this script which should run every night.
p.s. it uses a shared repository, so it's quite quick, it doesn't pull
everything from launchpad from scratch every time.
Cheers -Terry
TL
capability to restore an old version?
TL
Terry Brown
mess with bzr, I think it's reasonable there should be a simple .zip
download. If you use bzr, there's no need to use the snapshots.
Cheers -Terry
Edward K. Ream
On Tue, 1 Jul 2008 12:52:08 -0500
> Cool. How did you make these snapshots?With this script which should run every night.
I'm a bit lost. Is this a script that you plan to run every night?
Could you explain how it works?
Thanks.
Edward
Terry Brown
"Edward K. Ream" <edre...@gmail.com> wrote:
> I'm a bit lost. Is this a script that you plan to run every night?
Well, it's on an Ubuntu box that runs 24x7 (it has a lot of other
duties), so it's run at 2:53 am every day by the cron task. Hence the
timestamp on the newest snap at http://www.greygreen.org/leo/
So on the server there's a line in my crontab that looks like this:
53 2 * * * bash /home/tbrown/snapshot.repo/snap.sh
and snap.sh looks like this:
#!/bin/sh
cd /home/tbrown/snapshot.repo
python leomirror.py
> Could you explain how it works?
I guess I'll just comment it... although I suspect your question was
about the part described above. Anyway:
"""Mirror a remote bzr branch to a zip file on a remote web server.
Required: python executed from directory where .zip files are stored.
Helpful: if the directory above is a shared repository bzr will run faster.
"""
# leomirror.py $Id$
# Author: Terry Brown
# Created: Tue Jul 01 2008
import os, glob, stat, hashlib, ftplib
# use shell commands to create .zip
os.system("""
TS=$(date +%Y%m%d%H%M) # make a timestamp 200807010225
FN=leo-bzr-snapshot-$TS # directory name
bzr branch lp:leo-editor $FN # branch to that name
rm -rf $FN/.bzr* # remove bzr dirs
zip -r $FN.zip $FN # make .zip
rm -rf $FN # removed directory
""")
# list of zips in current dir sorted by time
zips = glob.glob('*.zip')
zips.sort()
# cull entries after the 10th, later
culls = zips[:-11]
# entries to post to site
post = []
for i in [1,2,5,10]:
if i < len(zips):
post.append(zips[-i])
else:
break
# write top of html file
html = file("index.html",'w')
html.write("""
<html><head><title>Leo bzr snapshots</title></head><body>
<h1>Leo bzr snapshots</h1>
<p>These are snapshots of Leo's bzr trunk branch. Go to the
<a href="http://webpages.charter.net/edreamleo/front.html">Leo
webpage</a> for links to the latest release, directions
for accessing the bzr repository directly, and other Leo
information.</p>
""")
# write blocks linking to each file
for i in post:
s = os.stat(i)
s = s[stat.ST_SIZE]
m = hashlib.md5(file(i).read()).hexdigest()
s1 = hashlib.sha1(file(i).read()).hexdigest()
html.write("""<div><hr/><a href="%s">%s</a><br/>
Bytes: %d</br>
SHA1: %s</br>
MD5: %s</br>
""" % (i, i, s, s1, m))
html.write("""</body></head>""")
html.close() # important
# ftp files to remote web server
ftp = ftplib.FTP('secret.com', 'secret', 'secret')
ftp.cwd('public_html/leo')
# list of zips already on remote server
rzips = [i for i in ftp.nlst() if i.endswith('.zip')]
rzips.sort()
# cull entries more than ten
rculls = rzips[:-11]
for i in rculls:
ftp.delete(i)
# upload any of the linked files not already present
for i in post:
if i not in rzips:
ftp.storbinary("STOR %s" % i, file(i))
# upload index
ftp.storbinary("STOR index.html", file("index.html"))
ftp.quit()
# remove local entries more than ten
Edward K. Ream
On Wed, 2 Jul 2008 06:07:03 -0500
> I'm a bit lost. Is this a script that you plan to run every night?Well, it's on an Ubuntu box that runs 24x7 (it has a lot of other
duties), so it's run at 2:53 am every day by the cron task. Hence the
timestamp on the newest snap at http://www.greygreen.org/leo/
So on the server there's a line in my crontab that looks like this:
Thanks for these details, and thanks for writing, and running, this script.
Edward
ne1uno
> > duties), so it's run at 2:53 am every day by the cron task. Hence the
> > So on the server there's a line in my crontab that looks like this:
>
> Thanks for these details, and thanks for writing, and running, this script.
>
with bundle-revisions --output=filename but I don't know the exact
syntax. there may be other pitfalls such as not including some
directories or too much history etc.
and would be complicated by needing to change to a more backward
compatible bzr format such as used in 014 or earlier.
just an FYI, not a suggestion or request! the ,bzr files/dirs would
just annoy anyone else and the zip probably can be imported fine.
the nightly/weekly zips will be appreciated, I know the request has
been made before bzr and svn back to cvs.
ne1
Terry Brown
ne1uno <eltr...@gmail.com> wrote:
> just an FYI, not a suggestion or request! the ,bzr files/dirs would
> just annoy anyone else and the zip probably can be imported fine.
I don't quite follow - the script deletes them, which it should because
they refer to a non-existent shared repository.
Unless you grabbed a snapshot where they weren't deleted, the current
version of the script deletes them and the latest snapshot does not
include them, but some of the earlier snapshots may, the first version
of the script didn't delete them.
Cheers -Terry
zpcspm
and the night 10 days ago" thingie to the page itself. The date in the
filenames is relevant, but it takes a couple of seconds to figure out
that the latest bzr snapshot is the first one.