Private shared git repositories on the cheap

44 views
Skip to first unread message

Tom Allen

unread,
Oct 13, 2011, 7:45:45 PM10/13/11
to code-p...@googlegroups.com
I just discovered this rather neat trick, and figured I'd share it with the group. If you want to share a closed source git repository without paying for github, and without setting up your own server, just add Dropbox... You basically end up with both your local and remote repositories on your local machine (so yes, it's a waste of space), but the remote one is getting backed up to Dropbox and is accessible to other developers. (One extra tip - if you're doing this on a Mac, you might want to turn off Dropbox's Growl notifications!)

Git In A Dropbox [Source: http://altdevblogaday.com/2011/10/08/handy-git-tips/]

You can push your local repo to your Dropbox, and it automatically gets synced with all the PCs you are using Dropbox with. You can then share this Dropbox folder with other developers.

# go to your Dropbox and create your project directory
$ cd ~/Dropbox
$ mkdir my_project
$ cd my_project


# now initialize your git repo with
$ git --bare init


# As you have your remote-repo prepared, go to your local repository.
$ cd ~/dev/my_project


# First, you need to introduce the remote location to git
# this adds the specified path as the remote named 'origin'
# but you could as well name it 'Dropbox' or 'whatever'
$ git remote add origin file:///home/user/Dropbox/git/my_project


# git is set up, so push it to the remote ( 'origin' or whatever
# name you have used ).
$ git push origin master


Done, you know have your repo on your Dropbox. If you are on another PC and want to access it, just clone it from there, and you are set. You can use this like you would use any git-server.

Billy Huang

unread,
Oct 13, 2011, 8:13:48 PM10/13/11
to code-p...@googlegroups.com
pretty cool idea, probably look into this one too.

but bitbucket now offers free private git, unlimited size and repo, but i think its limited to certain amount of dev though

i'm using both git and mercurial up there. which is really easy and user friendly just like github. totally ditched the unfriendly interface of unfuddle..

Andrew Hill

unread,
Oct 13, 2011, 8:01:07 PM10/13/11
to code-p...@googlegroups.com
I smell trouble...

What would happen if you push/pull to/from the remote while DropBox is updating it?
Git repos are not a single file (there's one per commit I think, and a few database files), so DropBox will not sync the whole "remote" repository atomically.
You could be accessing some files from the new "version" of your "remote" and some files from the previous version, if you were to do a git push/pull while it was updating.

This is (I think) guaranteed to be 100% fine if you only use DropBox as backup (not for synchronising/hosting your shared repo).
But unless DropBox have some specific code to atomically update git repos or update them in the right order, it sounds dangerous to me for sharing/syncing the repo between computers.

Tom Allen

unread,
Oct 13, 2011, 8:53:39 PM10/13/11
to code-p...@googlegroups.com
Yes, having more of a think about it, this probably isn't so great... :-(

I'd figured it was OK, because you're pushing/pulling from your locally stored 'remote', and so although you could well find that it hadn't updated yet, it wouldn't ever get half-synced. However, since there's no delay on Dropbox pushing out changes, you could actually get half a modification in your local remote just as you try to git pull. There is a hash of the whole folder structure in the git database files so this would probably report an error rather than fail catastrophically, but overall it's probably not worth the hassle.

Bummer.

Matthew Fernandez

unread,
Oct 13, 2011, 11:29:48 PM10/13/11
to code-p...@googlegroups.com
I think this would actually be fine. Git is designed to cope with
things like your SSH connection unexpectedly hanging up in the middle
of a push. The final thing the push writes should be analogous to
committing a database transaction. So if Dropbox syncs part way
through, you should just get a partial commit that Git will notice and
automatically discard (on garbage collection). Of course, Mercurial is
a whole different kettle of fish.

Something I would be more concerned about is the following scenario:
1. Alice commits to her working directory and pushes (to the Dropbox remote).
2. Bob commits to his working directory and pushes (to the Dropbox remote).
3. Bob's Dropbox folder syncs.
4. Alice's Dropbox folder syncs *CHAOS?*
How does Dropbox resolve conflicts? I really, really don't want to
resolve a merge conflict inside my remote...

Andrew Hill

unread,
Oct 14, 2011, 12:16:23 AM10/14/11
to code-p...@googlegroups.com
Hanging up an SSH connection is quite different to files being present and accessible, but out of date — git will surely exit if a file isn't found, but if it's there but out of sync with the rest of the repo, it's much the same as your conflict example.

When DropBox has conflicts (e.g. you had a file open while it was deleted, or two people edited a file and tried to sync), it renames files, appending a reason for the renaming (e.g. "deleted" or "conflict" and a hash) so you could get
a) just Alice's files (if she synced first) + bob's put in there with funny names — at least shouldn't break it
b) just Bob's files (if he synced first) + alice's put in there with funny names — also shouldn't break it
c) a mix of the two! (under race conditions) — watch out for sharks

Matthew Fernandez

unread,
Oct 14, 2011, 12:40:46 AM10/14/11
to code-p...@googlegroups.com
On 14 October 2011 15:16, Andrew Hill <and...@thefrog.net> wrote:
> Hanging up an SSH connection is quite different to files being present and
> accessible, but out of date — git will surely exit if a file isn't found,
> but if it's there but out of sync with the rest of the repo, it's much the
> same as your conflict example.
What's the situation where there's an out of date file accessible? The
situation I was imagining was where the SSH connection drops halfway
through after git's already written some files. In fact it may even
have open file handles when the connection drops. In this case you'll
end up with part of a commit in your remote, but git should resolve
this.

> When DropBox has conflicts (e.g. you had a file open while it was deleted,
> or two people edited a file and tried to sync), it renames files, appending
> a reason for the renaming (e.g. "deleted" or "conflict" and a hash) so you
> could get
> a) just Alice's files (if she synced first) + bob's put in there with funny
> names — at least shouldn't break it
> b) just Bob's files (if he synced first) + alice's put in there with funny
> names — also shouldn't break it
> c) a mix of the two! (under race conditions) — watch out for sharks

I think a and b will still break. Git will think the commit succeeded
and dropbox will automatically resolve the conflicts this way. The
next time git is invoked on the remote it will probably see part of
one commit and part of the other in a way that it cannot resolve. Of
course you can probably force repair the remote and then push again,
but you may end up in the same situation.

The underlying problem here is that you never want dropbox to resolve
conflicts. Can you force a dropbox sync? In this case, add a git push
hook in your remote to sync with dropbox before and after pushing.
That way git will stop the push and force you to merge/rebase if there
are conflicts. There's still a race condition where the pre-hook can
see no changes and the post-hook can see changes. It's unlikely, but
"unlikely" in software development usually means a problem you never
hit until an hour before the release deadline.

David Wood

unread,
Oct 14, 2011, 6:59:18 AM10/14/11
to code-p...@googlegroups.com

As far as I've ever found there is no way to tell dropbox what to do or when to do it. The whole innovation behind dropbox is that it's a versioning repo for idiots, all the actual tech is hidden from the end user.

That said, if anyone does know a way of forcing a synch other than logging out/in please let me know, that'd be really handy.

-DKW

Matthew Fernandez

unread,
Oct 14, 2011, 8:08:40 AM10/14/11
to code-p...@googlegroups.com
On 14 October 2011 21:59, David Wood <d.w...@acfr.usyd.edu.au> wrote:
> As far as I've ever found there is no way to tell dropbox what to do or when
> to do it. The whole innovation behind dropbox is that it's a versioning repo
> for idiots, all the actual tech is hidden from the end user.
Dropbox provides versioning? I did not realise this. Quite cool, and
they even give you instructions for pulling your files out of the
cache! https://www.dropbox.com/help/11

Matthew Fernandez

unread,
Oct 26, 2011, 2:06:03 AM10/26/11
to code-p...@googlegroups.com
I wanted to revive this thread briefly to take a poll on what
solutions people are using for syncing. Like most of us, I have
several computers/devices that I use regularly and I like having my
settings, apps and some data available everywhere. I don't like using
a service like Dropbox because I don't want to rely on their
benevolence to store and not expose or censor my files. As a result I
have a collection of shell scripts in my crontab that do various
syncing operations with git/hg repos living on my own servers. What
are other people using for this? Dropbox, SparkleShare, something
homebrewed or something else?

Julian G

unread,
Oct 26, 2011, 2:42:37 AM10/26/11
to code-p...@googlegroups.com
Why not spin up a (free) EC2 instance and host your stuff on there? Use Git for your code, and S3 for everything that can't be versioned.

Also, Bitbucket has free private git hosting. I've moved over to them for my projects instead of my Gitorious server.

I use DropBox and Git. SugarSync is a rival to DropBox, and there are many other open source (and non cloud) alternatives to DropBox.

J

Andrew Hill

unread,
Oct 26, 2011, 9:25:58 AM10/26/11
to code-p...@googlegroups.com
Not that I use it (in this combined setup anyway), but I reckon encfs on top of Dropbox would be the best option for the paranoid.

Tbh I see no problem relying on benevolence for continued service, as it'll take all of an hour to switch to some other hosting service if one goes down or full retard. Even if you had to re-host once a year, it's almost certainly better uptime and less work than home brew.

Nothing against home brew, if you're keen to do it for the sake of home brewing, but it's rarely the best performance/reliability/persistent/etc in my experience. Also sharks.

Dropbox is just too easy, and gives excellent feedback about sync state.

And encfs is per file, so I'm pretty sure it'll cope with real-time syncing via Dropbox. Pretty sure... Report back with epic tales of data loss if it doesn't...
Reply all
Reply to author
Forward
0 new messages