RBVExIm usage patterns

[Colin Howlett]

Hi,

I'm new around here so forgive my ignorance. I'm currently evaluating
RB and I'm exploring how best to use it with a source control system
(I'm looking at Perforce). RBVExIm looks very interesting, but I'm
worried that it forces me to use .rbvx files as the master in my source
repository, when the real benefit over simply having all my classes,
windows, modules etc as external items is that it allows me to do diffs
and merges.

So my question is - would it be a reasonable approach to store all my
classes, windows etc as binary external items in perforce, and simply
temporarily convert them to .rbvx for diffs and merges (i.e. just use
.rbvx files as a transient artifact during diffs and merges)? Can
RBVExIm create .rbvx files for individual classes, or does it only work
on complete projects?

Thanks for your help

Colin

[Mike Woodworth]

i've never used perforce, so this is all from the point of view of
cvs (YMMV):

unless you are sharing these external items between projects, you'll
have no advantage of keeping them external (and history has shown,
slightly more pain). in addition, you want diffs/merges to happen on
the same format you are storing, because those diffs are actually
written into the tree. if you use external xmls for instance, you
would be limited in how you could diff - you could only diff by
checking out two copies of your source, rbvexim-ing both (i declare
"rbvexim can be verbed"), and diffing those. i seem to remember
perforce is single user? desktop? so maybe this isnt a problem.
but for instance - i have a web tool that allows me to look at diffs/
history/ etc for my repository. obviously things like that require
that the source in your repository is the source you want to diff.

also, most versioning systems only work on textual info... so be
careful not to try to store binary externals ( yes yes you can store
them flagged as binaries, but at that point you might as well be
doing incremental backups instead of versioning - none of the systems
i know do actual diff storage on binary files)

mike
--
Mike Woodworth
mi...@divergentmedia.com

[Colin Howlett]

Mike,

Maybe I misunderstood, but I think we are talking slightly at cross
purposes.

I understand that version control systems tend (for textual items) to
store the most recent version in its entirety, and all previous
versions as automatically generated diffs - obtaining a reduction in
storage required - and then doing an automatic merge if an older
version is checked out. If I store binary items I lose this advantage.

That's not what I'm talking about when I talk about diffs and merges
however. I mean doing manual merges between branches.

For this, perforce (which _is_ actually client-server) has a standalone
(client-side) merge tool called p4merge. The user can also specify
their own merge tool.

So to compare files in the repository, I could write a script which:

1. obtains the binary versions to compare from the repository,
2. puts them into a temporary local folder,
3. calls RBVExIm to transform them into .rbvx format, and
4. compares those .rbvx files with p4merge.

To manually merge files:

obviously the target object would need to be checked out for editing.
then:

5. p4merge could create a new merged .rbvx file after I'd manually
specified the merges
6. RBVExIm would create a new binary file from this merged .rbvx file
7. The resulting merged binary would be checked back in.

I could specify that perforce use this script in place of a vanilla
p4merge for file comparison and manual merges ( the script may need to
pop up a dialog box or two of its own to manage both straightforward
comparison and actual merges)

IF I could do this it seems to me it would have a couple of advantages
over storing projects as .rbvx files in the repository:

1. I'm only relying on RBVExIm for helping manual merges

I'm sure RBVExIm is a great and reliable tool, but RS may take it into
their heads to change their formats, and Thomas may find something
better to do with his time than maintaining RBVExIm. If that happens I
may be in trouble if my entire repository consists of files understood
only by RBVExIm. Of course, I could maintain it myself (since it's open
source), but that may be hard, depending on where RS goes with their
format. So all in all, it seems safer to keep files that RB understands
natively in my repository. If the worst comes to the worst I can do a
manual merge by opening both binaries in RB and doing it by hand.

2. I don't have to mess around with .rbvault folders all the time

Of course what I don't fully understand - since I'm just evaluating RB
at the moment - is

1. Whether RBVExIm can even work this way?

2. What the risks and difficulties are of having all my classes,
windows, modules etc as external items?

If 2. also entails significant risk of data loss or daily pain, then I
may be better off using RBVExIm as currently documented.

Thanks

Colin

[Mike Woodworth]

On Feb 9, 2006, at 12:45 PM, Colin Howlett wrote:
>
> 1. Whether RBVExIm can even work this way?

currently it only works on binary projects. you could have a project
with everything external, and it will vacuum up all those externals,
so you should be fine. you'll have to re-checkout your whole
project... rbvexim that - then diff the parts you worried about. i
think you'll find rb changes externals even when nothing has been
done, ie you switched panes in the viewer. if you treating externals
as binary, this will trigger a new version in perforce (again, if its
at all similar to cvs). you'll get a lot of unnecessary commits.

>
> 2. What the risks and difficulties are of having all my classes,
> windows, modules etc as external items?

depends. i dont think there are any major problems now. in the past
there was a run of headaches during the switch to 2005. there are
some reports of problems getting externals to save sometimes (seems
to be a problem with things not being marked as dirty, most people
havent had any probs).

>
> If 2. also entails significant risk of data loss or daily pain, then I
> may be better off using RBVExIm as currently documented.

if you're going to do all your diffs client side, should be fine to
keep things in rb. i'd at least store the externals as xml. it'll
save some space, you can be sure perforce wont accidentally clobber
it, and in a bind you can read it.

[Colin Howlett]

Thanks Mike.

That's mostly what I figured. It's kind of a pain if RB changes
externals even if you don't do anything though.

I need to think this through a little more.

Colin

[Mike Woodworth]

well, if you make them xml, perforce should then do a diff and
realize they not actually different.

mike
--
Mike Woodworth
mi...@divergentmedia.com

[Thomas Tempelmann]

On 2/9/06, Colin Howlett <c.d.h...@gmail.com> wrote:
>
> So my question is - would it be a reasonable approach to store all my
> classes, windows etc as binary external items in perforce, and simply
> temporarily convert them to .rbvx for diffs and merges (i.e. just use
> .rbvx files as a transient artifact during diffs and merges)? Can
> RBVExIm create .rbvx files for individual classes, or does it only work
> on complete projects?

As the author of that rbvx tool I can tell you this:

the underlying class can operate on any kind of rb project file,
including external items, and can convert them back and forth to
binary form (xml writing is possible as well, only xml reading is not
implemented yet)

And I do actually use this functionality when I need to merge two
versions of a project manually: I export both projects to each their
own vaults and then use BBEdit to compare the files in the vault. I
can then edit those vault files and in the end I use the tool again to
recreate the original project from it.

You can do the same also just for single external items, or for the
entire project, without the vault, by using the GUI app ("RB Project
Tool") and drop the file onto "Export as Text", the check the created
rbvx file, and after editing drop this onto "Export as XML" to
recreate a xml file from it (what you'd want here is a "Export as
binary", but i kind of forgot this use and have not added this yet to
the gui tool - I probably need to redesign its UI soon).

And if you wanted to write your own tool for this functionality - look
at the source code and look into the RBPrjSupport class. That contains
easy-to-use methods to read a prj file into the internal tree and then
export it in a different format. Do this once for each direction and
you have your special solution. Or use RBVExIm, which basically does
the same.

So, yes, I believe you can do version control with the binary or even
xml files as long as you do not run the risk of conflicts (which can
happen when you make changes to two separate local versions and then
try to merge them in the repository) but instead only work alone on
the project. And when you then check out two separate changes and want
to compare them, just use my tool to create text files and then diff
those to see the changes.

[Colin Howlett]

Hmmm,

I feel like I'm going round in circles...

If I use xml in the repository, then I can't use RBVExIm at all right?
It only works on binary projects... RB's native XML export isn't pretty
to diff...

I'm still not sure I understand the problems with binary external items
though...

Let me walk through an example...

Suppose I have a new binary project with two binary external items -
say "windowA" and "classA" - , so I have three binary files in my local
work area and I check those into perforce, so they're all version 1.

Let's say that I want to add another class - "classB" with a single
method, a button on windowA to activate the method, and a textfield to
show the result.

So I check out the project and windowA (but not classA) so now my
workarea contains project v2, windowA v2 and classA v1, but currently
the v2s are identical to the v1s in the repository.

I open the project in RB, add classB and export it as a binary
external item, I make the changes to windowA (and implicitly to the
project) export windowA as a binary external item. I look at classA,
but don't change any of its source and save the project.

So now my work area contains project v2 (updated), windowA v2
(updated), classA v1 (hopefully unchanged?), and classB v1 (new). I
check project v2, windowA v2 amd classB v1 in.

Now I sync my work area with the repository (which, if RB naughtily
changed classA behind my back, will replace it with the old one), open
the project in RB and attempt to build it.

Is classA going to cause problems, because RB messed with it, and I
replaced the messed with version with the previous version?

I guess that could be a showstopper for using binary external items. If
not, then I think I can still do what I originally suggested.

Any suggestions welcome...

Thanks

Colin

[Colin Howlett]

Thomas,

Thanks for the quick and clear reply. I was hoping I could script the
transformation of individual binary external items into .rbvx files and
back again, but it seems I can't quite (or did I misunderstand you?).

I believe I would need RB Pro to compile command line applications, so
it could get expensive for me to create a custom process from your
class :(

I guess I could create a custom gui app (with nothing but menus) and
script the gui with applescript :)

[Mike Woodworth]

On Feb 9, 2006, at 5:03 PM, Colin Howlett wrote:

>
> Hmmm,
>
> I feel like I'm going round in circles...
>
> If I use xml in the repository, then I can't use RBVExIm at all right?
> It only works on binary projects... RB's native XML export isn't
> pretty
> to diff...
>
> I'm still not sure I understand the problems with binary external
> items
> though...
>
>
> Let me walk through an example...
>
> Suppose I have a new binary project with two binary external items -
> say "windowA" and "classA" - , so I have three binary files in my
> local
> work area and I check those into perforce, so they're all version 1.

right.

>
> Let's say that I want to add another class - "classB" with a single
> method, a button on windowA to activate the method, and a
> textfield to
> show the result.
>
> So I check out the project and windowA (but not classA) so now my
> workarea contains project v2, windowA v2 and classA v1, but currently
> the v2s are identical to the v1s in the repository.
>
> I open the project in RB, add classB and export it as a binary
> external item, I make the changes to windowA (and implicitly to the
> project) export windowA as a binary external item. I look at classA,
> but don't change any of its source and save the project.

right.

>
> So now my work area contains project v2 (updated), windowA v2
> (updated), classA v1 (hopefully unchanged?), and classB v1 (new). I
> check project v2, windowA v2 amd classB v1 in.

ok. if you manually check in items one at a time yes. i have some
500 items in my repository, so i rely on checking the whole thing in,
and letting cvs worry about finding altered items. in my case classA
v2 would be checked in. if you can manually keep track of what
you've altered, and only check in those items, you'll be fine. i
would worry about missing something (since i sync with a bug tracking
db, i checkin after each tkt resolved. if i missed a item in the
checkin, i wouldnt see it in later diffs over that fixed bug)

>
> Now I sync my work area with the repository (which, if RB naughtily
> changed classA behind my back, will replace it with the old one), open
> the project in RB and attempt to build it.
>
> Is classA going to cause problems, because RB messed with it, and I
> replaced the messed with version with the previous version?

you'll be fine, all that rb changed was where the cursor was or some
such drivel, when you replace with the old version, your cursor will
go back to the old location. no problem at all.

>
> I guess that could be a showstopper for using binary external
> items. If
> not, then I think I can still do what I originally suggested.
>
> Any suggestions welcome...

yeah, you should be fine (see caveat above). the more i talk to
people about versioning, the more i think i'm way out in left field
in how i use it.

[Colin Howlett]

Cool. Between your reply and Thomas's below, I think I'm (more or less)
in the clear! Thanks!

Colin

[Thomas Tempelmann]

On 2/9/06, Colin Howlett <c.d.h...@gmail.com> wrote:
>

> If I use xml in the repository, then I can't use RBVExIm at all right?
> It only works on binary projects... RB's native XML export isn't pretty
> to diff...

Well, once I add xml read support, you'll be fine. It's high on my
list (i.e. might happen by next week)

> I'm still not sure I understand the problems with binary external items
> though...

Without following what you write - did you read this:

http://www.rbwiki.org/doku.php/rbp/scm/rbvexim

At the end of that page I try to explain some of the possible issues
with externals.

> Now I sync my work area with the repository (which, if RB naughtily
> changed classA behind my back, will replace it with the old one), open
> the project in RB and attempt to build it.
>
> Is classA going to cause problems, because RB messed with it, and I
> replaced the messed with version with the previous version?

I can't see a problem here. Checking something into the repository
will only happen when it's newer, and nothing old from the repos will
wrote over your newer stuff - that's how any version control system
should work, AFAIK. Since RBVExIm does always collect all your project
items in a separate folder (the Vault), and you are supposed to sync
that Vault's contents with the repository, you always have it all
current in the Vault.

Now, there is one theoretical danger: You could fetch newer version
(created from somewhere else) into the vault and then try to recreate
your project from it (= Import from vault), overwriting your latest
work on your project. But that can't happen any more since version
1.1.2 of my tool because it will first make sure you have not changed
your local project before you overwrite it with the version from the
vault, by checking the modification dates of your project and storing
saving that information somewhere. If it finds that you have made
changes to your local project, it will not let you import a different
version from the vault, and you will be told about this conflict and
can fix it (i.e. first Export your project to the vault, _then_ use
the version control system to merge in the changes from the
repository).

Understood?

--
Thomas Tempelmann - exaggerating over a million times a day!
http://www.tempel.org/rb/ -- The primary source of outdated REALbasic
plugins and examples
Skype: tempel.org AIM: superTempel

[Colin Howlett]

Thanks Thomas.

I had read the wiki entry, but I'd only skimmed the section on
externals. Since I'm not planning to use a vault, but store the binary
externals in my repository, I'm not sure the caveats you note there are
relevant.

If I write a custom process based on your classes to convert individual
binary externals to .rbvx files and back, and use it only when doing a
merge between branches, I don't see how I can get into too much
trouble.

Thanks for all your help (and the hard work in producing RB project
tools and RBVExIm in the first place!)

regards

Colin

[Thomas Tempelmann]

On 2/10/06, Colin Howlett <c.d.h...@gmail.com> wrote:
>
> I had read the wiki entry, but I'd only skimmed the section on
> externals. Since I'm not planning to use a vault, but store the binary
> externals in my repository, I'm not sure the caveats you note there are
> relevant.

Oh yes, I forgot.

> If I write a custom process based on your classes to convert individual
> binary externals to .rbvx files and back, and use it only when doing a
> merge between branches, I don't see how I can get into too much
> trouble.

Right.