difference in git versus tarball release

8 views
Skip to first unread message

Scott Moser

unread,
Nov 30, 2011, 4:58:32 PM11/30/11
to boto...@googlegroups.com
Hi,
I was looking at pulling in boto 2.1.1 to Ubuntu, and when importing
the new source realized that the git branch is significantly different
than the release tarball. The full list of changes, as well as how I got
them is below, but I wondered if this was known and by-design. I didn't
see any 'make release' or anything in git repo, so I'm not sure how the
release is made.

The changes in git->tarball I think can be sumarrized as:
- remove "*test*" all tests
(except for boto/manage/test_manage.py, boto/sdb/db/test_db.py)
- remove "*.cfg"
- remove "*.rst"
- keep only bin, boto, PKG-INFO, setup.py in top level directory
(which removes docs/, Changelog.rst, and README.markdown)
- remove bin/asadmin (probably a mistake)
- remove boto/file/README
- remove boto/pyami/installers

Ubuntu and likely other distros are going to prefer taking input from a
tarball release. If the tarball is significantly different than the
source branch then things are confusing at best. The notable issue is
that then ubuntu's packaged version wont have the tests in the source, and
thus its not easy to run unit tests from it.

Thoughts?

$ git clone g...@github.com:boto/boto.git
$ git clone boto boto-mine
$ ( cd boto-mine && git branch local-2.1.1 2.1.1 && git checkout local-2.1.1 )
$ ( cd boto-mine && find . -type f | grep -v ./.git/ ) | sort > git-2.1.1.list
$ tar xzf ../boto-2.1.1.tar.gz
$ ( cd boto-2.1.1 && find . -type f ) | sort > release-2.1.1.list
$ diff -u git-2.1.1.list release-2.1.1.list

--- git-2.1.1.list 2011-11-30 11:06:12.197543511 -0500
+++ release-2.1.1.list 2011-11-30 11:06:54.165922273 -0500
-./bin/asadmin
-./boto/emr/tests/test_emr_responses.py
-./boto/file/README
-./boto/fps/test/__init__.py
-./boto/fps/test/test_install_caller_instruction.py
-./boto/fps/test/test_verify_signature.py
-./boto/mturk/test/all_tests.py
-./boto/mturk/test/cleanup_tests.py
-./boto/mturk/test/common.py
-./boto/mturk/test/create_free_text_question_regex.doctest
-./boto/mturk/test/create_hit_binary.doctest
-./boto/mturk/test/create_hit.doctest
-./boto/mturk/test/create_hit_external.py
-./boto/mturk/test/create_hit_from_hit_type.doctest
-./boto/mturk/test/create_hit_test.py
-./boto/mturk/test/create_hit_with_qualifications.py
-./boto/mturk/test/.gitignore
-./boto/mturk/test/hit_persistence.py
-./boto/mturk/test/_init_environment.py
-./boto/mturk/test/__init__.py
-./boto/mturk/test/mocks.py
-./boto/mturk/test/reviewable_hits.doctest
-./boto/mturk/test/run-doctest.py
-./boto/mturk/test/search_hits.doctest
-./boto/mturk/test/selenium_support.py
-./boto/mturk/test/support.py
-./boto/mturk/test/test_disable_hit.py
-./boto/pyami/copybot.cfg
-./boto/pyami/installers/__init__.py
-./boto/pyami/installers/ubuntu/apache.py
-./boto/pyami/installers/ubuntu/ebs.py
-./boto/pyami/installers/ubuntu/__init__.py
-./boto/pyami/installers/ubuntu/installer.py
-./boto/pyami/installers/ubuntu/mysql.py
-./boto/pyami/installers/ubuntu/trac.py
-./boto/services/sonofmmm.cfg
-./Changelog.rst
-./docs/make.bat
-./docs/Makefile
-./docs/source/autoscale_tut.rst
-./docs/source/boto_theme/static/boto.css_t
-./docs/source/boto_theme/static/pygments.css
-./docs/source/boto_theme/theme.conf
-./docs/source/conf.py
-./docs/source/documentation.rst
-./docs/source/ec2_tut.rst
-./docs/source/elb_tut.rst
-./docs/source/emr_tut.rst
-./docs/source/index.rst
-./docs/source/ref/boto.rst
-./docs/source/ref/cloudfront.rst
-./docs/source/ref/contrib.rst
-./docs/source/ref/ec2.rst
-./docs/source/ref/ecs.rst
-./docs/source/ref/emr.rst
-./docs/source/ref/file.rst
-./docs/source/ref/fps.rst
-./docs/source/ref/gs.rst
-./docs/source/ref/iam.rst
-./docs/source/ref/index.rst
-./docs/source/ref/manage.rst
-./docs/source/ref/mashups.rst
-./docs/source/ref/mturk.rst
-./docs/source/ref/pyami.rst
-./docs/source/ref/rds.rst
-./docs/source/ref/route53.rst
-./docs/source/ref/s3.rst
-./docs/source/ref/sdb.rst
-./docs/source/ref/services.rst
-./docs/source/ref/ses.rst
-./docs/source/ref/sns.rst
-./docs/source/ref/sqs.rst
-./docs/source/ref/sts.rst
-./docs/source/ref/vpc.rst
-./docs/source/s3_tut.rst
-./docs/source/sqs_tut.rst
-./docs/source/_templates/layout.html
-./docs/source/vpc_tut.rst
-./.gitignore
-./MANIFEST.in
-./pylintrc
-./README.markdown
+./PKG-INFO
-./tests/autoscale/__init__.py
-./tests/autoscale/test_connection.py
-./tests/cloudfront/__init__.py
-./tests/cloudfront/test_signed_urls.py
-./tests/db/test_lists.py
-./tests/db/test_password.py
-./tests/db/test_query.py
-./tests/db/test_sequence.py
-./tests/devpay/__init__.py
-./tests/devpay/test_s3.py
-./tests/ec2/cloudwatch/__init__.py
-./tests/ec2/cloudwatch/test_connection.py
-./tests/ec2/elb/test_connection.py
-./tests/ec2/__init__.py
-./tests/ec2/test_connection.py
-./tests/__init__.py
-./tests/s3/cb_test_harnass.py
-./tests/s3/__init__.py
-./tests/s3/mock_storage_service.py
-./tests/s3/other_cacerts.txt
-./tests/s3/test_connection.py
-./tests/s3/test_encryption.py
-./tests/s3/test_gsconnection.py
-./tests/s3/test_https_cert_validation.py
-./tests/s3/test_pool.py
-./tests/s3/test_resumable_downloads.py
-./tests/s3/test_resumable_uploads.py
-./tests/s3/test_versioning.py
-./tests/sdb/__init__.py
-./tests/sdb/test_connection.py
-./tests/sqs/__init__.py
-./tests/sqs/test_connection.py
-./tests/test.py
-./tests/utils/test_password.py


Mitchell Garnaat

unread,
Nov 30, 2011, 5:07:20 PM11/30/11
to boto...@googlegroups.com
Hi Scott -

I need to look into this more but I'm wondering if the differences are a result of using "python setup.py sdist upload" to actually build the tarball that gets uploaded to PyPI (I assume that  is where you are getting the tarball?).  Although, that is the way I have built the release tarball since the beginning so I'm not sure why this would just show up now.

I need to do a bit more investigation.

Mitch

Scott Moser

unread,
Nov 30, 2011, 6:53:21 PM11/30/11
to boto...@googlegroups.com
On Wed, 30 Nov 2011, Mitchell Garnaat wrote:

> Hi Scott -
>
> I need to look into this more but I'm wondering if the differences are a
> result of using "python setup.py sdist upload" to actually build the
> tarball that gets uploaded to PyPI (I assume that is where you are getting
> the tarball?). Although, that is the way I have built the release tarball
> since the beginning so I'm not sure why this would just show up now.

Sorry for not including where I got the tarball.
http://code.google.com/p/boto/downloads/list
specifically "2.1.1.tar.gz" there:
http://boto.googlecode.com/files/boto-2.1.1.tar.gz

Mitchell Garnaat

unread,
Nov 30, 2011, 6:56:13 PM11/30/11
to boto...@googlegroups.com
Right.  That's the same tarball that is in PyPI.  I do think the issue is with distutils, or at least my use of it.  Maybe I should just use the github feature of automatically creating tarballs of tags and use those as the release tarballs.  Those are guaranteed to be the same.

Mitch 

Reply all
Reply to author
Forward
0 new messages