[PATCH 3/4] Restrict and further document the -r/--remote syntax
0 views
Skip to first unread message
Rob Browning
Mar 2, 2026, 7:07:32 PM (2 days ago) Mar 2
to bup-...@googlegroups.com
For a long time, -r has supported some URLs in addition to the
host:port syntax. In preparation for more rigorous URL support in
other places, document and restrict -r so that we can more confidently
have the same URL syntax/semantics everywhere.
In order to avoid ambiguity (is file:x a host:port or a URL?), make
URLs have precedence, if and only if they have an authority, so that
any value with a syntactically valid URL prefix like x:// is taken to
be a URL, not a host:port.
Simplify the non-URL parsing by requiring a host (i.e. can't be
empty), which also means requiring a colon in every value because
previously, a value without a colon was taken as a path, not a host.
Change the "-r -" used for testing to "-r -:" to accommodate.
Drop support for the file: scheme, since it appears
unnecessary (redirects to ssh), and has potentially surprising
semantics (e.g. file://home/bup meant a host of home and a path of
/bup).
Since it looks like the URL support has never been publically
documented, the risks from any backward incompatibliy should be low.
Signed-off-by: Rob Browning <r...@defaultvalue.org>
Tested-by: Rob Browning <r...@defaultvalue.org>
---
Documentation/bup-get.1.md | 10 ++---
Documentation/bup-init.1.md | 14 +++----
Documentation/bup-join.1.md | 11 ++----
Documentation/bup-ls.1.md | 11 ++----
Documentation/bup-restore.1.md | 10 ++---
Documentation/bup-save.1.md | 10 ++---
Documentation/bup-split.1.md | 11 ++----
Documentation/bup.1.md | 24 ++++++++++++
lib/bup/client.py | 46 ++++++++++++++---------
note/main.md | 11 ++++++
test/ext/test-gc | 6 +--
test/ext/test-gc-removes-incomplete-trees | 2 +-
test/ext/test-index | 6 +--
test/ext/test-ls | 2 +-
test/ext/test-packsizelimit | 6 +--
test/ext/test-rewrite | 2 +-
test/ext/test-save-restore | 4 +-
test/ext/test-split-join | 8 ++--
test/int/test_client.py | 19 +++++-----
19 files changed, 115 insertions(+), 98 deletions(-)
diff --git a/Documentation/bup-get.1.md b/Documentation/bup-get.1.md
index 10b20e32..89c7e236 100644
--- a/Documentation/bup-get.1.md
+++ b/Documentation/bup-get.1.md
@@ -117,13 +117,9 @@ used to help test before/after results.)
-s, \--source=*path*
: use *path* as the source repository, instead of the default.
--r, \--remote=*host*:*path*
-: store the indicated items on the given remote server. If *path*
- is omitted, uses the default path on the remote server (you still
- need to include the ':'). The connection to the remote server is
- made with SSH. If you'd like to specify which port, user or
- private key to use for the SSH connection, we recommend you use
- the `~/.ssh/config` file.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: write the *ref*s to the specified remote repository, by default
+ via SSH. See bup(1) REMOTE OPTIONS for further information.
-c, \--print-commits
: for each updated branch, print the new git commit id.
diff --git a/Documentation/bup-init.1.md b/Documentation/bup-init.1.md
index cf4ef15d..2c33631e 100644
--- a/Documentation/bup-init.1.md
+++ b/Documentation/bup-init.1.md
@@ -19,14 +19,12 @@ set, or `~/.bup`.
# OPTIONS
--r, \--remote=*host*:*path*
-: Initialize not only the local repository, but also the
- remote repository given by the *host* and *path*. This is
- not necessary if you intend to back up to the default
- location on the server (ie. a blank *path*). The connection to the
- remote server is made with SSH. If you'd like to specify which port, user
- or private key to use for the SSH connection, we recommend you use the
- `~/.ssh/config` file.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: Initialize not only the local repository, but also the specified
+ remote repository. This is not necessary if you intend to use the
+ default location on the server (ie. with no *path*). By default
+ the connection to the remote server is made with SSH. See bup(1)
+ REMOTE OPTIONS for further information.
# EXAMPLES
bup init ~/archive
diff --git a/Documentation/bup-join.1.md b/Documentation/bup-join.1.md
index e1a708bf..86c7af44 100644
--- a/Documentation/bup-join.1.md
+++ b/Documentation/bup-join.1.md
@@ -25,14 +25,9 @@ join` reads them from stdin instead.
# OPTIONS
--r, \--remote=*host*:*path*
-: Retrieves objects from the given remote repository instead of the
- local one. *path* may be blank, in which case the default remote
- repository is used. The connection to the remote server is made
- with SSH. If you'd like to specify which port, user or private
- key to use for the SSH connection, we recommend you use the
- `~/.ssh/config` file. Even though the data source is remote, a
- local bup repository is still required.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: retrieve the data from the speciried remote repository, by default
+ via SSH. See bup(1) REMOTE OPTIONS for further information.
# EXAMPLES
# split and then rejoin a file using its tree id
diff --git a/Documentation/bup-ls.1.md b/Documentation/bup-ls.1.md
index 2e820d69..773fe57a 100644
--- a/Documentation/bup-ls.1.md
+++ b/Documentation/bup-ls.1.md
@@ -39,13 +39,10 @@ you can view its contents using `bup join` or `git show`.
# OPTIONS
--r, \--remote=*host*:[*path*]
-: list information for the repository at *path* on the indicated
- *host*. If *path* is omitted, uses the default path on the remote
- server (you still need to include the ':'). The connection to the
- remote server will be made by SSH. If you'd like to specify the
- port, user, or private key, we recommend you use the
- `~/.ssh/config` file (`ssh_config(5)`).
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: draw the information from the specified remote repository, by
+ default via SSH. See bup(1) REMOTE OPTIONS for further
+ information.
-s, \--hash
: show hash for each file/directory.
diff --git a/Documentation/bup-restore.1.md b/Documentation/bup-restore.1.md
index 8e211034..9111635c 100644
--- a/Documentation/bup-restore.1.md
+++ b/Documentation/bup-restore.1.md
@@ -98,13 +98,9 @@ See the EXAMPLES section for a demonstration.
# OPTIONS
--r, \--remote=*host*:*path*
-: restore the backup set from the given remote server. If
- *path* is omitted, uses the default path on the remote
- server (you still need to include the ':'). The connection to the
- remote server is made with SSH. If you'd like to specify which port, user
- or private key to use for the SSH connection, we recommend you use the
- `~/.ssh/config` file.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: restore from the specified remote repository, by default via SSH.
+ See bup(1) REMOTE OPTIONS for further information.
-C, \--outdir=*outdir*
: create and change to directory *outdir* before
diff --git a/Documentation/bup-save.1.md b/Documentation/bup-save.1.md
index a7ab5fa6..b123c84c 100644
--- a/Documentation/bup-save.1.md
+++ b/Documentation/bup-save.1.md
@@ -34,13 +34,9 @@ handling of metadata.
# OPTIONS
--r, \--remote=*host*:*path*
-: save the backup set to the given remote server. If
- *path* is omitted, uses the default path on the remote
- server (you still need to include the ':'). The connection to the
- remote server is made with SSH. If you'd like to specify which port, user
- or private key to use for the SSH connection, we recommend you use the
- `~/.ssh/config` file.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: save the data to the specified remote repository, by default via
+ SSH. See bup(1) REMOTE OPTIONS for further information.
-t, \--tree
: after creating the backup set, print out the git tree
diff --git a/Documentation/bup-split.1.md b/Documentation/bup-split.1.md
index 1af42c93..c901faff 100644
--- a/Documentation/bup-split.1.md
+++ b/Documentation/bup-split.1.md
@@ -91,14 +91,9 @@ being the most likely choice.
# OPTIONS
--r, \--remote=*host*:*path*
-: save the backup set to the given remote server. If *path* is
- omitted, uses the default path on the remote server (you still
- need to include the ':'). The connection to the remote server is
- made with SSH. If you'd like to specify which port, user or
- private key to use for the SSH connection, we recommend you use
- the `~/.ssh/config` file. Even though the destination is remote,
- a local bup repository is still required.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: split the data to the specified remote repository, by default via
+ SSH. See bup(1) REMOTE OPTIONS for further information.
-d, \--date=*seconds-since-epoch*
: specify the date inscribed in the commit (seconds since 1970-01-01).
diff --git a/Documentation/bup.1.md b/Documentation/bup.1.md
index 7fffbaa3..3690a788 100644
--- a/Documentation/bup.1.md
+++ b/Documentation/bup.1.md
@@ -155,6 +155,30 @@ Subcommands are described in separate man pages. For example
`bup-tick`(1)
: Wait for up to one second.
+# REMOTE OPTIONS
+
+Some options (currently just `--reverse`) allow the specification of a
+remote path as either a URL or a `[*user*@]*host*:[*path*]`.
+
+For either format, when there is no path, the default path on the
+server will be used, and SSH settings for the connection can be
+provided by a custom host to your `~/.ssh/config` file
+(`ssh_config(5)`).
+
+The argument is treated as a URL if it begins with a syntactically
+valid URL scheme prefix that contains an "authority" (meaning that it
+begins with `SCHEME://` as `ssh://...` does), and the scheme must be
+either `ssh` or `bup`; others will be rejected.
+
+For the `[*user*@]*host*:[*path*]` syntax the *host* must always be
+followed by a colon, and anything after the first colon is the *path*.
+
+For fully general purposes, prefer URLs to `[*user*@]*host*:[*path*]`,
+so that there is no potential ambiguity. For example, consider the
+(unlikely) case where `ssh://x/y` is generated for a host named `ssh`
+and path `//x/y` which would be interpreted as a URL with host `x` and
+path `/y`.
+
# ENVIRONMENT
`BUP_ASSUME_GIT_VERSION_IS_FINE`
diff --git a/lib/bup/client.py b/lib/bup/client.py
index 06373deb..ee005682 100644
--- a/lib/bup/client.py
+++ b/lib/bup/client.py
@@ -126,20 +126,30 @@ _url_rx = re.compile(br'%s(?:%s%s)?%s' % (_protocol_rs, _host_rs, _port_rs, _pat
def parse_remote(remote):
def parse_non_url(remote):
- rs = remote.split(b':', 1)
- if len(rs) == 1 or rs[0] in (b'', b'-'):
- return b'file', None, None, rs[-1]
- else:
- return b'ssh', rs[0], None, rs[1]
- assert remote is not None
- if remote and remote.startswith(b'bup-rev://'):
- parts = parse_non_url(remote[len(b'bup-rev://'):] + b':')
- return (b'bup-rev',) + parts[1:]
- url_match = _url_rx.match(remote)
- if url_match:
- if url_match.group(1) not in (b'ssh', b'bup', b'file'):
- raise ClientError('unexpected protocol: %s'
- % url_match.group(1).decode('ascii'))
+ if b':' not in remote:
+ raise ClientError(f'remote {pm(remote)} has no colon')
+ host, path = remote.split(b':', 1)
+ if host == b'-': # use a subprocess for testing
+ return b'ssh', None, None, path if path else None
+ if not host:
+ raise ClientError(f'remote {pm(remote)} has no host')
+ return b'ssh', host, None, path
+ if remote.startswith(b'file:'):
+ raise ClientError(f'unexpected file scheme for {pm(remote)}')
+ if remote.startswith(b'bup-rev://'):
+ # It should be a hostname, so just make the value the host for now
+ return b'bup-rev', remote[len(b'bup-rev://'):], None, None
+ m = re.match(br'([a-zA-Z][-+.a-zA-Z0-9]+):', remote) # has valid scheme
+ if m:
+ scheme = m.group(1)
+ if scheme not in (b'ssh', b'bup'):
+ raise ClientError(f'unexpected {scheme} scheme for {pm(remote)}')
+ if remote[3:6] != b'://':
+ raise ClientError(f'{scheme} URL {pm(remote)} has no host')
+ url_match = _url_rx.match(remote)
+ if not url_match:
+ raise ClientError(f'invalid URL {pm(remote)}')
+ assert url_match.group(1) == scheme
return url_match.group(1,3,4,5)
return parse_non_url(remote)
@@ -150,7 +160,7 @@ def _legacy_cache_id(remote, reversed=False):
# duplicate the index-cache. The index-cache for newer
# repositories is determined by the repo-id. Although remote will
# typically already have been vetted/restricted by parse_remote, a
- # reversed value normally just the BUP_SERVER_REVERSE value,
+ # reversed value is normally just the BUP_SERVER_REVERSE value,
# which should be a hostname, but could be anything.
def parse_non_url(remote):
rs = remote.split(b':', 1)
@@ -302,10 +312,12 @@ class Client:
self._busy = None
if self.protocol == b'bup-rev':
self._transport = Client.ViaBupRev()
- elif self.protocol in (b'ssh', b'file'):
+ elif self.protocol == b'ssh':
self._transport = Client.ViaSsh(self.host, self.port)
elif self.protocol == b'bup':
self._transport = Client.ViaBup(self.host, self.port)
+ else:
+ raise ClientError(f'unrecognized remote {pm(remote)}')
ctx.enter_context(self._transport)
self.conn = self._transport.conn
self._available_commands = self._get_available_commands()
@@ -319,7 +331,7 @@ class Client:
self.conn.write(b'set-dir %s\n' % self.dir)
self.check_ok()
if self.protocol == b'bup-rev':
- self.cachedir = self._prep_cache(remote[len(b'bup-rev://'):], True)
+ self.cachedir = self._prep_cache(self.host, True)
else:
self.cachedir = self._prep_cache(remote, False)
self.sync_indexes()
diff --git a/note/main.md b/note/main.md
index 6ec8f0c1..fb62948c 100644
--- a/note/main.md
+++ b/note/main.md
@@ -84,6 +84,17 @@ General
creation, or when run again on an existing repository. See
`bup-config`(5) for more information.
+* The `--remote` (`-r`) argument interpretation has changed. The
+ value is treated as a URL if it begins with a syntactically valid
+ URL scheme prefix that contains an "authority" (meaning that it
+ begins with `SCHEME://` as `ssh://...` does) and anything else is
+ interpreted as a `host:[path]` where the `host` is no longer
+ optional). `file:` URLs are no longer allowed; the semantics were
+ potentially surprising (e.g. `file://p` would ssh to host `p`). Use
+ `ssh:` URLs instead. The URL support, though long standing, was
+ previously undocumented. See REMOTE OPTIONS in `bup`(1) for further
+ information.
+
* The REMOTE directory name in the client index cache (typically
`~/.bup/index-cache/REMOTE`) is now the `bup.repo.id` when the
remote repository provides, one and existing directories will be
diff --git a/test/ext/test-gc b/test/ext/test-gc
index 2200b040..fd5ccff3 100755
--- a/test/ext/test-gc
+++ b/test/ext/test-gc
@@ -111,10 +111,10 @@ WVPASS rm -rf src-ab
WVPASS cp -pPR src-ab-clean src-ab
WVPASS bup index src-ab
-WVPASS bup save -r :bup-remote --strip -n src-ab src-ab
+WVPASS bup save -r -:bup-remote --strip -n src-ab src-ab
WVPASS bup index --clear
WVPASS bup index src-ab
-WVPASS bup save -r :bup-remote -vvv --strip -n a src-ab/a
+WVPASS bup save -r -:bup-remote -vvv --strip -n a src-ab/a
size_before=$(WVPASS data-size bup-remote) || exit $?
WVPASS rm bup-remote/refs/heads/src-ab
@@ -133,7 +133,7 @@ WVFAIL bup -d bup-remote restore -C "$tmpdir/restore" /src-ab/latest
# Make sure a post-gc index/save that includes gc-ed data works
WVPASS bup index src-ab
-WVPASS bup save -r :bup-remote --strip -n src-ab src-ab
+WVPASS bup save -r -:bup-remote --strip -n src-ab src-ab
WVPASS rm -r "$tmpdir/restore"
WVPASS bup -d bup-remote restore -C "$tmpdir/restore" /src-ab/latest
WVPASS compare-trees src-ab/ "$tmpdir/restore/latest/"
diff --git a/test/ext/test-gc-removes-incomplete-trees b/test/ext/test-gc-removes-incomplete-trees
index 76016c35..56abada0 100755
--- a/test/ext/test-gc-removes-incomplete-trees
+++ b/test/ext/test-gc-removes-incomplete-trees
@@ -57,7 +57,7 @@ WVPASS git show-index < bup/objects/pack/"$(basename ${packs[1]} .pack).idx" \
# Keep a safe copy of src in another repo
WVPASS bup -d bup-complete init
-WVPASS bup save -r :bup-complete --strip -n src src
+WVPASS bup save -r -:bup-complete --strip -n src src
# Promote hold to its own branch so we can drop everything else and gc
WVPASS bup get --append: src/latest/1-hold hold
diff --git a/test/ext/test-index b/test/ext/test-index
index 15c33565..d36639bd 100755
--- a/test/ext/test-index
+++ b/test/ext/test-index
@@ -121,9 +121,9 @@ WVPASSEQ "$(cd $D && bup index -m)" \
f
a
./"
-WVPASS bup save -r ":$BUP_DIR" -n r-test $D
-WVFAIL bup save -r ":$BUP_DIR/fake/path" -n r-test $D
-WVFAIL bup save -r ":$BUP_DIR" -n r-test $D/fake/path
+WVPASS bup save -r "-:$BUP_DIR" -n r-test $D
+WVFAIL bup save -r "-:$BUP_DIR/fake/path" -n r-test $D
+WVFAIL bup save -r "-:$BUP_DIR" -n r-test $D/fake/path
WVPASS cd "$top"
WVPASS rm -rf "$tmpdir"
diff --git a/test/ext/test-ls b/test/ext/test-ls
index b011015a..f34b808a 100755
--- a/test/ext/test-ls
+++ b/test/ext/test-ls
@@ -21,7 +21,7 @@ with-tty() { "$top/dev/with-tty" "$@"; }
bup-ls() {
if test "$BUP_TEST_REMOTE_REPO"; then
- "$top/bup" ls -r "$BUP_DIR" "$@"
+ "$top/bup" ls -r "-:$BUP_DIR" "$@"
else
"$top/bup" ls "$@"
fi
diff --git a/test/ext/test-packsizelimit b/test/ext/test-packsizelimit
index 1a84ae52..8ffb8024 100755
--- a/test/ext/test-packsizelimit
+++ b/test/ext/test-packsizelimit
@@ -40,7 +40,7 @@ WVPASS bup -d "$bup_dir2" init
WVPASS git --git-dir "$bup_dir2" config pack.packSizeLimit 10k
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASSEQ $(WVPASS find "$bup_dir2"/objects/pack -name "*.pack" | wc -l) 0
-WVPASS bup random 50k | WVPASS bup split -r "$bup_dir2" -n foo
+WVPASS bup random 50k | WVPASS bup split -r "-:$bup_dir2" -n foo
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASS test $(WVPASS find "$bup_dir2"/objects/pack/*.pack | wc -l) -gt 2
@@ -52,7 +52,7 @@ WVPASS bup -d "$bup_dir2" init
WVPASS git config pack.packSizeLimit 10k
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASSEQ $(WVPASS find "$bup_dir2"/objects/pack -name "*.pack" | wc -l) 0
-WVPASS bup random 50k | WVPASS bup split -r "$bup_dir2" -n foo
+WVPASS bup random 50k | WVPASS bup split -r "-:$bup_dir2" -n foo
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASS test $(WVPASS find "$bup_dir2"/objects/pack/*.pack | wc -l) -eq 1
@@ -65,7 +65,7 @@ WVPASS git config pack.packSizeLimit 1024k
WVPASS git --git-dir "$bup_dir2" config pack.packSizeLimit 10k
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASSEQ $(WVPASS find "$bup_dir2"/objects/pack -name "*.pack" | wc -l) 0
-WVPASS bup random 50k | WVPASS bup split -r "$bup_dir2" -n foo
+WVPASS bup random 50k | WVPASS bup split -r "-:$bup_dir2" -n foo
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASS test $(WVPASS find "$bup_dir2"/objects/pack/*.pack | wc -l) -gt 2
diff --git a/test/ext/test-rewrite b/test/ext/test-rewrite
index 1ad8a972..3fd3f3f9 100755
--- a/test/ext/test-rewrite
+++ b/test/ext/test-rewrite
@@ -64,7 +64,7 @@ WVPASS bup -d "$BUP_DIR4" get --rewrite -s "$BUP_DIR" --append save
WVPASS compare "$BUP_DIR" save "$BUP_DIR4" save
WVSTART "rewrite unchanged (to remote)"
-WVPASS bup get -r ":$BUP_DIR3" -s "$BUP_DIR" --append save
+WVPASS bup get -r "-:$BUP_DIR3" -s "$BUP_DIR" --append save
WVPASS compare "$BUP_DIR" save "$BUP_DIR3" save
WVPASSEQ "$(GIT_DIR="$BUP_DIR" WVPASS git log --pretty=format:%T -n1 save)" \
"$(GIT_DIR="$BUP_DIR3" WVPASS git log --pretty=format:%T -n1 save)"
diff --git a/test/ext/test-save-restore b/test/ext/test-save-restore
index 012ae688..3b781e76 100755
--- a/test/ext/test-save-restore
+++ b/test/ext/test-save-restore
@@ -18,7 +18,7 @@ validate-local-and-remote-restore()
WVPASS bup restore -C "$dest" "$src"
WVPASS "$top/dev/compare-trees" "$cmp_src" "$cmp_dest"
force-delete "$dest"
- WVPASS bup restore -r ":$BUP_DIR" -C "$dest" "$src"
+ WVPASS bup restore -r "-:$BUP_DIR" -C "$dest" "$src"
WVPASS "$top/dev/compare-trees" "$cmp_src" "$cmp_dest"
}
@@ -96,7 +96,7 @@ WVPASSEQ "$(cat buprestore.tmp/data)" ""
WVPASS "$top/dev/compare-trees" "$x/" "$tmp/restore/$(basename $x)"
done
WVPASS force-delete "$tmp/restore"
- WVPASS bup restore -r ":$BUP_DIR" -C "$tmp"/restore /foo/latest/x/
+ WVPASS bup restore -r "-:$BUP_DIR" -C "$tmp"/restore /foo/latest/x/
for x in "$tmp"/src/x/*; do
WVPASS "$top/dev/compare-trees" "$x/" "$tmp/restore/$(basename $x)"
done
diff --git a/test/ext/test-split-join b/test/ext/test-split-join
index 2ed18504..37aebe26 100755
--- a/test/ext/test-split-join
+++ b/test/ext/test-split-join
@@ -75,10 +75,9 @@ WVPASSEQ "$(echo "$BUP_DIR"/objects/pack/*.midx)" \
WVPASS bup margin
WVPASS bup split -t "$top/test/testfile2" >tags2t.tmp
WVPASS bup split -t "$top/test/testfile2" --fanout 3 >tags2tf.tmp
-WVPASS bup split -r "$BUP_DIR" -c "$top/test/testfile2" >tags2c.tmp
-WVPASS bup split -r ":$BUP_DIR" -c "$top/test/testfile2" >tags2c.tmp
+WVPASS bup split -r "-:$BUP_DIR" -c "$top/test/testfile2" >tags2c.tmp
WVPASS ls -lR \
- | WVPASS bup split -r ":$BUP_DIR" -c --fanout 3 --max-pack-objects 3 -n lslr \
+ | WVPASS bup split -r "-:$BUP_DIR" -c --fanout 3 --max-pack-objects 3 -n lslr \
|| exit $?
WVPASS bup ls
WVFAIL bup ls /does-not-exist
@@ -99,8 +98,7 @@ WVSTART "join"
WVPASS bup join $(cat tags1.tmp) >out1.tmp
WVPASS bup join <tags2.tmp >out2.tmp
WVPASS bup join <tags2t.tmp -o out2t.tmp
-WVPASS bup join -r "$BUP_DIR" <tags2c.tmp >out2c.tmp
-WVPASS bup join -r ":$BUP_DIR" <tags2c.tmp >out2c.tmp
+WVPASS bup join -r "-:$BUP_DIR" <tags2c.tmp >out2c.tmp
WVPASS diff -u "$top/test/testfile1" out1.tmp
WVPASS diff -u "$top/test/testfile2" out2.tmp
WVPASS diff -u "$top/test/testfile2" out2t.tmp
diff --git a/test/int/test_client.py b/test/int/test_client.py
index c37738cb..b48ebf2b 100644
--- a/test/int/test_client.py
+++ b/test/int/test_client.py
@@ -3,6 +3,7 @@ import os, time, random, subprocess, glob
import pytest
from bup import client, git, path
+#from bup.client import ClientError
from bup.compat import environ
from bup.config import ConfigError
from bup.repo import LocalRepo
@@ -29,7 +30,7 @@ def test_server_split_with_indexes(tmpdir):
git.init_repo(bupdir)
with local_writer() as lw:
lw.new_blob(s1)
- with client.Client(bupdir, create=True) as c, \
+ with client.Client(b'-:' + bupdir, create=True) as c, \
c.new_packwriter() as rw:
rw.new_blob(s2)
rw.breakpoint()
@@ -46,7 +47,7 @@ def test_multiple_suggestions(tmpdir):
lw.new_blob(s2)
assert len(glob.glob(git.repo(b'objects/pack'+IDX_PAT))) == 2
- with client.Client(bupdir, create=True) as c, \
+ with client.Client(b'-:' + bupdir, create=True) as c, \
c.new_packwriter() as rw:
assert len(glob.glob(c.cachedir+IDX_PAT)) == 0
@@ -123,7 +124,7 @@ def test_server_deduplicate_writes(deduplicate_mode, tmpdir):
with local_writer() as lw:
lw.new_blob(s1)
- with client.Client(bupdir, create=True) as c, \
+ with client.Client(b'-:' + bupdir, create=True) as c, \
c.new_packwriter() as rw:
assert len(glob.glob(c.cachedir+IDX_PAT)) == 1
rw.new_blob(s1)
@@ -135,7 +136,7 @@ def test_server_deduplicate_writes(deduplicate_mode, tmpdir):
def test_midx_refreshing(tmpdir):
environ[b'BUP_DIR'] = bupdir = tmpdir
git.init_repo(bupdir)
- with client.Client(bupdir, create=True) as c, \
+ with client.Client(b'-:' + bupdir, create=True) as c, \
c.new_packwriter() as rw:
rw.new_blob(s1)
p1base = rw.breakpoint()
@@ -169,8 +170,7 @@ def test_midx_refreshing(tmpdir):
def test_remote_parsing():
tests = (
- (b':/bup', (b'file', None, None, b'/bup')),
- (b'file:///bup', (b'file', None, None, b'/bup')),
+ (b'-:/bup', (b'ssh', None, None, b'/bup')),
(b'192.168.1.1:/bup', (b'ssh', b'192.168.1.1', None, b'/bup')),
(b'ssh://192.168.1.1:2222/bup', (b'ssh', b'192.168.1.1', b'2222', b'/bup')),
(b'ssh://[ff:fe::1]:2222/bup', (b'ssh', b'ff:fe::1', b'2222', b'/bup')),
@@ -178,8 +178,8 @@ def test_remote_parsing():
(b'bup://foo.com:1950/bup', (b'bup', b'foo.com', b'1950', b'/bup')),
(b'bup://[ff:fe::1]/bup', (b'bup', b'ff:fe::1', None, b'/bup')),
(b'bup://[ff:fe::1]/bup', (b'bup', b'ff:fe::1', None, b'/bup')),
- (b'bup-rev://', (b'bup-rev', None, None, b'')),
- (b'bup-rev://host/dir', (b'bup-rev', b'host/dir', None, b'')),
+ (b'bup-rev://', (b'bup-rev', b'', None, None)),
+ (b'bup-rev://host/dir', (b'bup-rev', b'host/dir', None, None)),
)
for remote, values in tests:
assert client.parse_remote(remote) == values
@@ -193,7 +193,6 @@ def test_legacy_cache_ids():
# repo-id to new repositories, this should only matter for legacy
# repositories. If we get this wrong (inadvertently change legacy
# id), then the client will create a duplicate index-cache.
-
def cid(reverse, remote):
if reverse: # see derive_repo_addr
assert not remote, remote
@@ -239,7 +238,7 @@ def test_config(tmpdir):
environ[b'BUP_DIR'] = bupdir = tmpdir
environ[b'GIT_DIR'] = bupdir = tmpdir
git.init_repo(bupdir)
- with client.Client(bupdir, create=True) as c:
+ with client.Client(b'-:' + bupdir, create=True) as c:
assert c.config_get(b'bup.split.trees') is None
assert c.config_get(b'bup.split.trees', opttype='int') is None
ex((b'git', b'config', b'bup.split.trees', b'0'))
--
2.47.3
host:port syntax. In preparation for more rigorous URL support in
other places, document and restrict -r so that we can more confidently
have the same URL syntax/semantics everywhere.
In order to avoid ambiguity (is file:x a host:port or a URL?), make
URLs have precedence, if and only if they have an authority, so that
any value with a syntactically valid URL prefix like x:// is taken to
be a URL, not a host:port.
Simplify the non-URL parsing by requiring a host (i.e. can't be
empty), which also means requiring a colon in every value because
previously, a value without a colon was taken as a path, not a host.
Change the "-r -" used for testing to "-r -:" to accommodate.
Drop support for the file: scheme, since it appears
unnecessary (redirects to ssh), and has potentially surprising
semantics (e.g. file://home/bup meant a host of home and a path of
/bup).
Since it looks like the URL support has never been publically
documented, the risks from any backward incompatibliy should be low.
Signed-off-by: Rob Browning <r...@defaultvalue.org>
Tested-by: Rob Browning <r...@defaultvalue.org>
---
Documentation/bup-get.1.md | 10 ++---
Documentation/bup-init.1.md | 14 +++----
Documentation/bup-join.1.md | 11 ++----
Documentation/bup-ls.1.md | 11 ++----
Documentation/bup-restore.1.md | 10 ++---
Documentation/bup-save.1.md | 10 ++---
Documentation/bup-split.1.md | 11 ++----
Documentation/bup.1.md | 24 ++++++++++++
lib/bup/client.py | 46 ++++++++++++++---------
note/main.md | 11 ++++++
test/ext/test-gc | 6 +--
test/ext/test-gc-removes-incomplete-trees | 2 +-
test/ext/test-index | 6 +--
test/ext/test-ls | 2 +-
test/ext/test-packsizelimit | 6 +--
test/ext/test-rewrite | 2 +-
test/ext/test-save-restore | 4 +-
test/ext/test-split-join | 8 ++--
test/int/test_client.py | 19 +++++-----
19 files changed, 115 insertions(+), 98 deletions(-)
diff --git a/Documentation/bup-get.1.md b/Documentation/bup-get.1.md
index 10b20e32..89c7e236 100644
--- a/Documentation/bup-get.1.md
+++ b/Documentation/bup-get.1.md
@@ -117,13 +117,9 @@ used to help test before/after results.)
-s, \--source=*path*
: use *path* as the source repository, instead of the default.
--r, \--remote=*host*:*path*
-: store the indicated items on the given remote server. If *path*
- is omitted, uses the default path on the remote server (you still
- need to include the ':'). The connection to the remote server is
- made with SSH. If you'd like to specify which port, user or
- private key to use for the SSH connection, we recommend you use
- the `~/.ssh/config` file.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: write the *ref*s to the specified remote repository, by default
+ via SSH. See bup(1) REMOTE OPTIONS for further information.
-c, \--print-commits
: for each updated branch, print the new git commit id.
diff --git a/Documentation/bup-init.1.md b/Documentation/bup-init.1.md
index cf4ef15d..2c33631e 100644
--- a/Documentation/bup-init.1.md
+++ b/Documentation/bup-init.1.md
@@ -19,14 +19,12 @@ set, or `~/.bup`.
# OPTIONS
--r, \--remote=*host*:*path*
-: Initialize not only the local repository, but also the
- remote repository given by the *host* and *path*. This is
- not necessary if you intend to back up to the default
- location on the server (ie. a blank *path*). The connection to the
- remote server is made with SSH. If you'd like to specify which port, user
- or private key to use for the SSH connection, we recommend you use the
- `~/.ssh/config` file.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: Initialize not only the local repository, but also the specified
+ remote repository. This is not necessary if you intend to use the
+ default location on the server (ie. with no *path*). By default
+ the connection to the remote server is made with SSH. See bup(1)
+ REMOTE OPTIONS for further information.
# EXAMPLES
bup init ~/archive
diff --git a/Documentation/bup-join.1.md b/Documentation/bup-join.1.md
index e1a708bf..86c7af44 100644
--- a/Documentation/bup-join.1.md
+++ b/Documentation/bup-join.1.md
@@ -25,14 +25,9 @@ join` reads them from stdin instead.
# OPTIONS
--r, \--remote=*host*:*path*
-: Retrieves objects from the given remote repository instead of the
- local one. *path* may be blank, in which case the default remote
- repository is used. The connection to the remote server is made
- with SSH. If you'd like to specify which port, user or private
- key to use for the SSH connection, we recommend you use the
- `~/.ssh/config` file. Even though the data source is remote, a
- local bup repository is still required.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: retrieve the data from the speciried remote repository, by default
+ via SSH. See bup(1) REMOTE OPTIONS for further information.
# EXAMPLES
# split and then rejoin a file using its tree id
diff --git a/Documentation/bup-ls.1.md b/Documentation/bup-ls.1.md
index 2e820d69..773fe57a 100644
--- a/Documentation/bup-ls.1.md
+++ b/Documentation/bup-ls.1.md
@@ -39,13 +39,10 @@ you can view its contents using `bup join` or `git show`.
# OPTIONS
--r, \--remote=*host*:[*path*]
-: list information for the repository at *path* on the indicated
- *host*. If *path* is omitted, uses the default path on the remote
- server (you still need to include the ':'). The connection to the
- remote server will be made by SSH. If you'd like to specify the
- port, user, or private key, we recommend you use the
- `~/.ssh/config` file (`ssh_config(5)`).
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: draw the information from the specified remote repository, by
+ default via SSH. See bup(1) REMOTE OPTIONS for further
+ information.
-s, \--hash
: show hash for each file/directory.
diff --git a/Documentation/bup-restore.1.md b/Documentation/bup-restore.1.md
index 8e211034..9111635c 100644
--- a/Documentation/bup-restore.1.md
+++ b/Documentation/bup-restore.1.md
@@ -98,13 +98,9 @@ See the EXAMPLES section for a demonstration.
# OPTIONS
--r, \--remote=*host*:*path*
-: restore the backup set from the given remote server. If
- *path* is omitted, uses the default path on the remote
- server (you still need to include the ':'). The connection to the
- remote server is made with SSH. If you'd like to specify which port, user
- or private key to use for the SSH connection, we recommend you use the
- `~/.ssh/config` file.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: restore from the specified remote repository, by default via SSH.
+ See bup(1) REMOTE OPTIONS for further information.
-C, \--outdir=*outdir*
: create and change to directory *outdir* before
diff --git a/Documentation/bup-save.1.md b/Documentation/bup-save.1.md
index a7ab5fa6..b123c84c 100644
--- a/Documentation/bup-save.1.md
+++ b/Documentation/bup-save.1.md
@@ -34,13 +34,9 @@ handling of metadata.
# OPTIONS
--r, \--remote=*host*:*path*
-: save the backup set to the given remote server. If
- *path* is omitted, uses the default path on the remote
- server (you still need to include the ':'). The connection to the
- remote server is made with SSH. If you'd like to specify which port, user
- or private key to use for the SSH connection, we recommend you use the
- `~/.ssh/config` file.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: save the data to the specified remote repository, by default via
+ SSH. See bup(1) REMOTE OPTIONS for further information.
-t, \--tree
: after creating the backup set, print out the git tree
diff --git a/Documentation/bup-split.1.md b/Documentation/bup-split.1.md
index 1af42c93..c901faff 100644
--- a/Documentation/bup-split.1.md
+++ b/Documentation/bup-split.1.md
@@ -91,14 +91,9 @@ being the most likely choice.
# OPTIONS
--r, \--remote=*host*:*path*
-: save the backup set to the given remote server. If *path* is
- omitted, uses the default path on the remote server (you still
- need to include the ':'). The connection to the remote server is
- made with SSH. If you'd like to specify which port, user or
- private key to use for the SSH connection, we recommend you use
- the `~/.ssh/config` file. Even though the destination is remote,
- a local bup repository is still required.
+-r, \--remote=[*user*@]*host*:[*path*], \--remote=URL
+: split the data to the specified remote repository, by default via
+ SSH. See bup(1) REMOTE OPTIONS for further information.
-d, \--date=*seconds-since-epoch*
: specify the date inscribed in the commit (seconds since 1970-01-01).
diff --git a/Documentation/bup.1.md b/Documentation/bup.1.md
index 7fffbaa3..3690a788 100644
--- a/Documentation/bup.1.md
+++ b/Documentation/bup.1.md
@@ -155,6 +155,30 @@ Subcommands are described in separate man pages. For example
`bup-tick`(1)
: Wait for up to one second.
+# REMOTE OPTIONS
+
+Some options (currently just `--reverse`) allow the specification of a
+remote path as either a URL or a `[*user*@]*host*:[*path*]`.
+
+For either format, when there is no path, the default path on the
+server will be used, and SSH settings for the connection can be
+provided by a custom host to your `~/.ssh/config` file
+(`ssh_config(5)`).
+
+The argument is treated as a URL if it begins with a syntactically
+valid URL scheme prefix that contains an "authority" (meaning that it
+begins with `SCHEME://` as `ssh://...` does), and the scheme must be
+either `ssh` or `bup`; others will be rejected.
+
+For the `[*user*@]*host*:[*path*]` syntax the *host* must always be
+followed by a colon, and anything after the first colon is the *path*.
+
+For fully general purposes, prefer URLs to `[*user*@]*host*:[*path*]`,
+so that there is no potential ambiguity. For example, consider the
+(unlikely) case where `ssh://x/y` is generated for a host named `ssh`
+and path `//x/y` which would be interpreted as a URL with host `x` and
+path `/y`.
+
# ENVIRONMENT
`BUP_ASSUME_GIT_VERSION_IS_FINE`
diff --git a/lib/bup/client.py b/lib/bup/client.py
index 06373deb..ee005682 100644
--- a/lib/bup/client.py
+++ b/lib/bup/client.py
@@ -126,20 +126,30 @@ _url_rx = re.compile(br'%s(?:%s%s)?%s' % (_protocol_rs, _host_rs, _port_rs, _pat
def parse_remote(remote):
def parse_non_url(remote):
- rs = remote.split(b':', 1)
- if len(rs) == 1 or rs[0] in (b'', b'-'):
- return b'file', None, None, rs[-1]
- else:
- return b'ssh', rs[0], None, rs[1]
- assert remote is not None
- if remote and remote.startswith(b'bup-rev://'):
- parts = parse_non_url(remote[len(b'bup-rev://'):] + b':')
- return (b'bup-rev',) + parts[1:]
- url_match = _url_rx.match(remote)
- if url_match:
- if url_match.group(1) not in (b'ssh', b'bup', b'file'):
- raise ClientError('unexpected protocol: %s'
- % url_match.group(1).decode('ascii'))
+ if b':' not in remote:
+ raise ClientError(f'remote {pm(remote)} has no colon')
+ host, path = remote.split(b':', 1)
+ if host == b'-': # use a subprocess for testing
+ return b'ssh', None, None, path if path else None
+ if not host:
+ raise ClientError(f'remote {pm(remote)} has no host')
+ return b'ssh', host, None, path
+ if remote.startswith(b'file:'):
+ raise ClientError(f'unexpected file scheme for {pm(remote)}')
+ if remote.startswith(b'bup-rev://'):
+ # It should be a hostname, so just make the value the host for now
+ return b'bup-rev', remote[len(b'bup-rev://'):], None, None
+ m = re.match(br'([a-zA-Z][-+.a-zA-Z0-9]+):', remote) # has valid scheme
+ if m:
+ scheme = m.group(1)
+ if scheme not in (b'ssh', b'bup'):
+ raise ClientError(f'unexpected {scheme} scheme for {pm(remote)}')
+ if remote[3:6] != b'://':
+ raise ClientError(f'{scheme} URL {pm(remote)} has no host')
+ url_match = _url_rx.match(remote)
+ if not url_match:
+ raise ClientError(f'invalid URL {pm(remote)}')
+ assert url_match.group(1) == scheme
return url_match.group(1,3,4,5)
return parse_non_url(remote)
@@ -150,7 +160,7 @@ def _legacy_cache_id(remote, reversed=False):
# duplicate the index-cache. The index-cache for newer
# repositories is determined by the repo-id. Although remote will
# typically already have been vetted/restricted by parse_remote, a
- # reversed value normally just the BUP_SERVER_REVERSE value,
+ # reversed value is normally just the BUP_SERVER_REVERSE value,
# which should be a hostname, but could be anything.
def parse_non_url(remote):
rs = remote.split(b':', 1)
@@ -302,10 +312,12 @@ class Client:
self._busy = None
if self.protocol == b'bup-rev':
self._transport = Client.ViaBupRev()
- elif self.protocol in (b'ssh', b'file'):
+ elif self.protocol == b'ssh':
self._transport = Client.ViaSsh(self.host, self.port)
elif self.protocol == b'bup':
self._transport = Client.ViaBup(self.host, self.port)
+ else:
+ raise ClientError(f'unrecognized remote {pm(remote)}')
ctx.enter_context(self._transport)
self.conn = self._transport.conn
self._available_commands = self._get_available_commands()
@@ -319,7 +331,7 @@ class Client:
self.conn.write(b'set-dir %s\n' % self.dir)
self.check_ok()
if self.protocol == b'bup-rev':
- self.cachedir = self._prep_cache(remote[len(b'bup-rev://'):], True)
+ self.cachedir = self._prep_cache(self.host, True)
else:
self.cachedir = self._prep_cache(remote, False)
self.sync_indexes()
diff --git a/note/main.md b/note/main.md
index 6ec8f0c1..fb62948c 100644
--- a/note/main.md
+++ b/note/main.md
@@ -84,6 +84,17 @@ General
creation, or when run again on an existing repository. See
`bup-config`(5) for more information.
+* The `--remote` (`-r`) argument interpretation has changed. The
+ value is treated as a URL if it begins with a syntactically valid
+ URL scheme prefix that contains an "authority" (meaning that it
+ begins with `SCHEME://` as `ssh://...` does) and anything else is
+ interpreted as a `host:[path]` where the `host` is no longer
+ optional). `file:` URLs are no longer allowed; the semantics were
+ potentially surprising (e.g. `file://p` would ssh to host `p`). Use
+ `ssh:` URLs instead. The URL support, though long standing, was
+ previously undocumented. See REMOTE OPTIONS in `bup`(1) for further
+ information.
+
* The REMOTE directory name in the client index cache (typically
`~/.bup/index-cache/REMOTE`) is now the `bup.repo.id` when the
remote repository provides, one and existing directories will be
diff --git a/test/ext/test-gc b/test/ext/test-gc
index 2200b040..fd5ccff3 100755
--- a/test/ext/test-gc
+++ b/test/ext/test-gc
@@ -111,10 +111,10 @@ WVPASS rm -rf src-ab
WVPASS cp -pPR src-ab-clean src-ab
WVPASS bup index src-ab
-WVPASS bup save -r :bup-remote --strip -n src-ab src-ab
+WVPASS bup save -r -:bup-remote --strip -n src-ab src-ab
WVPASS bup index --clear
WVPASS bup index src-ab
-WVPASS bup save -r :bup-remote -vvv --strip -n a src-ab/a
+WVPASS bup save -r -:bup-remote -vvv --strip -n a src-ab/a
size_before=$(WVPASS data-size bup-remote) || exit $?
WVPASS rm bup-remote/refs/heads/src-ab
@@ -133,7 +133,7 @@ WVFAIL bup -d bup-remote restore -C "$tmpdir/restore" /src-ab/latest
# Make sure a post-gc index/save that includes gc-ed data works
WVPASS bup index src-ab
-WVPASS bup save -r :bup-remote --strip -n src-ab src-ab
+WVPASS bup save -r -:bup-remote --strip -n src-ab src-ab
WVPASS rm -r "$tmpdir/restore"
WVPASS bup -d bup-remote restore -C "$tmpdir/restore" /src-ab/latest
WVPASS compare-trees src-ab/ "$tmpdir/restore/latest/"
diff --git a/test/ext/test-gc-removes-incomplete-trees b/test/ext/test-gc-removes-incomplete-trees
index 76016c35..56abada0 100755
--- a/test/ext/test-gc-removes-incomplete-trees
+++ b/test/ext/test-gc-removes-incomplete-trees
@@ -57,7 +57,7 @@ WVPASS git show-index < bup/objects/pack/"$(basename ${packs[1]} .pack).idx" \
# Keep a safe copy of src in another repo
WVPASS bup -d bup-complete init
-WVPASS bup save -r :bup-complete --strip -n src src
+WVPASS bup save -r -:bup-complete --strip -n src src
# Promote hold to its own branch so we can drop everything else and gc
WVPASS bup get --append: src/latest/1-hold hold
diff --git a/test/ext/test-index b/test/ext/test-index
index 15c33565..d36639bd 100755
--- a/test/ext/test-index
+++ b/test/ext/test-index
@@ -121,9 +121,9 @@ WVPASSEQ "$(cd $D && bup index -m)" \
f
a
./"
-WVPASS bup save -r ":$BUP_DIR" -n r-test $D
-WVFAIL bup save -r ":$BUP_DIR/fake/path" -n r-test $D
-WVFAIL bup save -r ":$BUP_DIR" -n r-test $D/fake/path
+WVPASS bup save -r "-:$BUP_DIR" -n r-test $D
+WVFAIL bup save -r "-:$BUP_DIR/fake/path" -n r-test $D
+WVFAIL bup save -r "-:$BUP_DIR" -n r-test $D/fake/path
WVPASS cd "$top"
WVPASS rm -rf "$tmpdir"
diff --git a/test/ext/test-ls b/test/ext/test-ls
index b011015a..f34b808a 100755
--- a/test/ext/test-ls
+++ b/test/ext/test-ls
@@ -21,7 +21,7 @@ with-tty() { "$top/dev/with-tty" "$@"; }
bup-ls() {
if test "$BUP_TEST_REMOTE_REPO"; then
- "$top/bup" ls -r "$BUP_DIR" "$@"
+ "$top/bup" ls -r "-:$BUP_DIR" "$@"
else
"$top/bup" ls "$@"
fi
diff --git a/test/ext/test-packsizelimit b/test/ext/test-packsizelimit
index 1a84ae52..8ffb8024 100755
--- a/test/ext/test-packsizelimit
+++ b/test/ext/test-packsizelimit
@@ -40,7 +40,7 @@ WVPASS bup -d "$bup_dir2" init
WVPASS git --git-dir "$bup_dir2" config pack.packSizeLimit 10k
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASSEQ $(WVPASS find "$bup_dir2"/objects/pack -name "*.pack" | wc -l) 0
-WVPASS bup random 50k | WVPASS bup split -r "$bup_dir2" -n foo
+WVPASS bup random 50k | WVPASS bup split -r "-:$bup_dir2" -n foo
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASS test $(WVPASS find "$bup_dir2"/objects/pack/*.pack | wc -l) -gt 2
@@ -52,7 +52,7 @@ WVPASS bup -d "$bup_dir2" init
WVPASS git config pack.packSizeLimit 10k
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASSEQ $(WVPASS find "$bup_dir2"/objects/pack -name "*.pack" | wc -l) 0
-WVPASS bup random 50k | WVPASS bup split -r "$bup_dir2" -n foo
+WVPASS bup random 50k | WVPASS bup split -r "-:$bup_dir2" -n foo
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASS test $(WVPASS find "$bup_dir2"/objects/pack/*.pack | wc -l) -eq 1
@@ -65,7 +65,7 @@ WVPASS git config pack.packSizeLimit 1024k
WVPASS git --git-dir "$bup_dir2" config pack.packSizeLimit 10k
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASSEQ $(WVPASS find "$bup_dir2"/objects/pack -name "*.pack" | wc -l) 0
-WVPASS bup random 50k | WVPASS bup split -r "$bup_dir2" -n foo
+WVPASS bup random 50k | WVPASS bup split -r "-:$bup_dir2" -n foo
WVPASSEQ $(WVPASS find "$BUP_DIR"/objects/pack -name "*.pack" | wc -l) 0
WVPASS test $(WVPASS find "$bup_dir2"/objects/pack/*.pack | wc -l) -gt 2
diff --git a/test/ext/test-rewrite b/test/ext/test-rewrite
index 1ad8a972..3fd3f3f9 100755
--- a/test/ext/test-rewrite
+++ b/test/ext/test-rewrite
@@ -64,7 +64,7 @@ WVPASS bup -d "$BUP_DIR4" get --rewrite -s "$BUP_DIR" --append save
WVPASS compare "$BUP_DIR" save "$BUP_DIR4" save
WVSTART "rewrite unchanged (to remote)"
-WVPASS bup get -r ":$BUP_DIR3" -s "$BUP_DIR" --append save
+WVPASS bup get -r "-:$BUP_DIR3" -s "$BUP_DIR" --append save
WVPASS compare "$BUP_DIR" save "$BUP_DIR3" save
WVPASSEQ "$(GIT_DIR="$BUP_DIR" WVPASS git log --pretty=format:%T -n1 save)" \
"$(GIT_DIR="$BUP_DIR3" WVPASS git log --pretty=format:%T -n1 save)"
diff --git a/test/ext/test-save-restore b/test/ext/test-save-restore
index 012ae688..3b781e76 100755
--- a/test/ext/test-save-restore
+++ b/test/ext/test-save-restore
@@ -18,7 +18,7 @@ validate-local-and-remote-restore()
WVPASS bup restore -C "$dest" "$src"
WVPASS "$top/dev/compare-trees" "$cmp_src" "$cmp_dest"
force-delete "$dest"
- WVPASS bup restore -r ":$BUP_DIR" -C "$dest" "$src"
+ WVPASS bup restore -r "-:$BUP_DIR" -C "$dest" "$src"
WVPASS "$top/dev/compare-trees" "$cmp_src" "$cmp_dest"
}
@@ -96,7 +96,7 @@ WVPASSEQ "$(cat buprestore.tmp/data)" ""
WVPASS "$top/dev/compare-trees" "$x/" "$tmp/restore/$(basename $x)"
done
WVPASS force-delete "$tmp/restore"
- WVPASS bup restore -r ":$BUP_DIR" -C "$tmp"/restore /foo/latest/x/
+ WVPASS bup restore -r "-:$BUP_DIR" -C "$tmp"/restore /foo/latest/x/
for x in "$tmp"/src/x/*; do
WVPASS "$top/dev/compare-trees" "$x/" "$tmp/restore/$(basename $x)"
done
diff --git a/test/ext/test-split-join b/test/ext/test-split-join
index 2ed18504..37aebe26 100755
--- a/test/ext/test-split-join
+++ b/test/ext/test-split-join
@@ -75,10 +75,9 @@ WVPASSEQ "$(echo "$BUP_DIR"/objects/pack/*.midx)" \
WVPASS bup margin
WVPASS bup split -t "$top/test/testfile2" >tags2t.tmp
WVPASS bup split -t "$top/test/testfile2" --fanout 3 >tags2tf.tmp
-WVPASS bup split -r "$BUP_DIR" -c "$top/test/testfile2" >tags2c.tmp
-WVPASS bup split -r ":$BUP_DIR" -c "$top/test/testfile2" >tags2c.tmp
+WVPASS bup split -r "-:$BUP_DIR" -c "$top/test/testfile2" >tags2c.tmp
WVPASS ls -lR \
- | WVPASS bup split -r ":$BUP_DIR" -c --fanout 3 --max-pack-objects 3 -n lslr \
+ | WVPASS bup split -r "-:$BUP_DIR" -c --fanout 3 --max-pack-objects 3 -n lslr \
|| exit $?
WVPASS bup ls
WVFAIL bup ls /does-not-exist
@@ -99,8 +98,7 @@ WVSTART "join"
WVPASS bup join $(cat tags1.tmp) >out1.tmp
WVPASS bup join <tags2.tmp >out2.tmp
WVPASS bup join <tags2t.tmp -o out2t.tmp
-WVPASS bup join -r "$BUP_DIR" <tags2c.tmp >out2c.tmp
-WVPASS bup join -r ":$BUP_DIR" <tags2c.tmp >out2c.tmp
+WVPASS bup join -r "-:$BUP_DIR" <tags2c.tmp >out2c.tmp
WVPASS diff -u "$top/test/testfile1" out1.tmp
WVPASS diff -u "$top/test/testfile2" out2.tmp
WVPASS diff -u "$top/test/testfile2" out2t.tmp
diff --git a/test/int/test_client.py b/test/int/test_client.py
index c37738cb..b48ebf2b 100644
--- a/test/int/test_client.py
+++ b/test/int/test_client.py
@@ -3,6 +3,7 @@ import os, time, random, subprocess, glob
import pytest
from bup import client, git, path
+#from bup.client import ClientError
from bup.compat import environ
from bup.config import ConfigError
from bup.repo import LocalRepo
@@ -29,7 +30,7 @@ def test_server_split_with_indexes(tmpdir):
git.init_repo(bupdir)
with local_writer() as lw:
lw.new_blob(s1)
- with client.Client(bupdir, create=True) as c, \
+ with client.Client(b'-:' + bupdir, create=True) as c, \
c.new_packwriter() as rw:
rw.new_blob(s2)
rw.breakpoint()
@@ -46,7 +47,7 @@ def test_multiple_suggestions(tmpdir):
lw.new_blob(s2)
assert len(glob.glob(git.repo(b'objects/pack'+IDX_PAT))) == 2
- with client.Client(bupdir, create=True) as c, \
+ with client.Client(b'-:' + bupdir, create=True) as c, \
c.new_packwriter() as rw:
assert len(glob.glob(c.cachedir+IDX_PAT)) == 0
@@ -123,7 +124,7 @@ def test_server_deduplicate_writes(deduplicate_mode, tmpdir):
with local_writer() as lw:
lw.new_blob(s1)
- with client.Client(bupdir, create=True) as c, \
+ with client.Client(b'-:' + bupdir, create=True) as c, \
c.new_packwriter() as rw:
assert len(glob.glob(c.cachedir+IDX_PAT)) == 1
rw.new_blob(s1)
@@ -135,7 +136,7 @@ def test_server_deduplicate_writes(deduplicate_mode, tmpdir):
def test_midx_refreshing(tmpdir):
environ[b'BUP_DIR'] = bupdir = tmpdir
git.init_repo(bupdir)
- with client.Client(bupdir, create=True) as c, \
+ with client.Client(b'-:' + bupdir, create=True) as c, \
c.new_packwriter() as rw:
rw.new_blob(s1)
p1base = rw.breakpoint()
@@ -169,8 +170,7 @@ def test_midx_refreshing(tmpdir):
def test_remote_parsing():
tests = (
- (b':/bup', (b'file', None, None, b'/bup')),
- (b'file:///bup', (b'file', None, None, b'/bup')),
+ (b'-:/bup', (b'ssh', None, None, b'/bup')),
(b'192.168.1.1:/bup', (b'ssh', b'192.168.1.1', None, b'/bup')),
(b'ssh://192.168.1.1:2222/bup', (b'ssh', b'192.168.1.1', b'2222', b'/bup')),
(b'ssh://[ff:fe::1]:2222/bup', (b'ssh', b'ff:fe::1', b'2222', b'/bup')),
@@ -178,8 +178,8 @@ def test_remote_parsing():
(b'bup://foo.com:1950/bup', (b'bup', b'foo.com', b'1950', b'/bup')),
(b'bup://[ff:fe::1]/bup', (b'bup', b'ff:fe::1', None, b'/bup')),
(b'bup://[ff:fe::1]/bup', (b'bup', b'ff:fe::1', None, b'/bup')),
- (b'bup-rev://', (b'bup-rev', None, None, b'')),
- (b'bup-rev://host/dir', (b'bup-rev', b'host/dir', None, b'')),
+ (b'bup-rev://', (b'bup-rev', b'', None, None)),
+ (b'bup-rev://host/dir', (b'bup-rev', b'host/dir', None, None)),
)
for remote, values in tests:
assert client.parse_remote(remote) == values
@@ -193,7 +193,6 @@ def test_legacy_cache_ids():
# repo-id to new repositories, this should only matter for legacy
# repositories. If we get this wrong (inadvertently change legacy
# id), then the client will create a duplicate index-cache.
-
def cid(reverse, remote):
if reverse: # see derive_repo_addr
assert not remote, remote
@@ -239,7 +238,7 @@ def test_config(tmpdir):
environ[b'BUP_DIR'] = bupdir = tmpdir
environ[b'GIT_DIR'] = bupdir = tmpdir
git.init_repo(bupdir)
- with client.Client(bupdir, create=True) as c:
+ with client.Client(b'-:' + bupdir, create=True) as c:
assert c.config_get(b'bup.split.trees') is None
assert c.config_get(b'bup.split.trees', opttype='int') is None
ex((b'git', b'config', b'bup.split.trees', b'0'))
--
2.47.3
Reply all
Reply to author
Forward
0 new messages