What is the correct approach when attempting to add multiple pre-commit diff files? (Multiple Parent Diffs)

[James Knight]

I have the following scenario, I have a remote Git repository (powered by GitLab) configured and working as expected with ReviewBoard (2.0.15). In my local repository (cloned), I have a series of ten (10) commits I'm about to push to the origin. Before I do this, I generate a (full indexed; unified) patch for the respective commits and I want to put them into ReviewBoard. Uploading the patches, I proceed as follows:

1. Make a new review. Select the first patch and upload to ReviewBoard. The review is created.
2. Make a second review. Select the second patch and attempt to upload to ReviewBoard. ReviewBoard complains the parent hash doesn't exist. I then upload my second patch with my first patch as a parent diff. The review is created.
3. Attempt to make a third review. Select the third patch to upload but find no way to upload since I cannot complete the parent chain of diffs. Full stop.

The exact error message is as follows:

The file "<file>" (revision <hash>) was not found in the repository

Is there a way I can append multiple parent diff's for a review? My attempts were to merge append patch 1 and 2 together, with no luck. Or, am I attempting to use ReviewBoard in an incorrect way?

The only work around I see is waiting until I commit the new patches into the remote repository before adding the other patches, for example:

1. Add patches 1 and 2 to respective reviews.
2. Reviews approved and patches committed.
3. Add patches 3 and 4 to respective reviews.
4. Reviews approved and patches committed.
5. ~keep repeating until final patch is committed~

Any help would be appreciated.

[Stephen Gallagher]

Try doing this:

rbt post <commit_id>

One at a time, from the oldest to the newest. Use *exactly* the commit ID as shown by 'git log'.

[James Knight]

So I wasn't using RBTools but I figured I'd try it first to see a working solution with rbt rather than using the web interface.

After installing and invoking `rbt`, the first commit of ten (10) created my first review for me. As soon as I try performing an `rbt posh <full_commit_index>` on the second or greater commits they fail. The specific error for my case was:

$ rbt post 2c6346ea50d25a974f4819a372f252d34d35d0da

ERROR: Error validating diff

<my_file>: The file was not found in the repository.

I assume this is a valid error message since the file is created in the first commit and ReviewBoard cannot interpret the parent state of the newer commit(s). After looking at the documentation, I don't see a way to provide a list of local parent commits which ReviewBoard can interpret the chain of changes. I assume the only work around I can do is actually push up the changes on the remote on a branch (something I wanted to avoid) and reference the branch in rbt's `--parent` parameter when generating a review for each pending commit.

[David Trowbridge]

How have you configured the repository on Review Board?

-David

--
Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups "reviewboard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[James Knight]

Repository options are configured as follows:

• Hosting Service: None - Custom Repository
• Repository Type: Git
• Path: git@myserver:mygroup/myproject.git
• Raw file URL mask: http://myserver/mygroup/myproject/raw/develop/<filename>

For our repository:

• GitLab (7.4.3)
• The project as a "develop" branch.
• The project has deploy keys setup for our ReviewBoard server.

I'm not sure how this information could help (unless I'm missing something). Just to make sure I've explained the scenario correctly:

• I have multiple commits in a local repository that have not been pushed up into the repository ReviewBoard is tracking.
• I desire to add reviews to ReviewBoard based on my local commits so they can be reviewed before I push it to the remote repository.

For example:

                             {ReviewBoard}

                     /\

                     ||

                     \/

     {Local}      {Remote}

     Commit 3

        |

     Commit 2

        |              Commit G

     Commit 1            /

        |               /

     Commit F      Commit F

        |             |

     Commit E      Commit E

In this example, I want to add Commit 1, 2 and 3 to ReviewBoard.

• I can add "Commit 1" by uploading the patch to ReviewBoard.
• I can add "Commit 2" by uploading the patch to ReviewBoard only when I also provide the parent patch for "Commit 1".
  
• ReviewBoard understands the base of "Commit 2" since the Git full index chain is completed (Commit F <- Commit 1 <- Commit 2)
• I cannot add "Commit 3" since by uploading the patch to ReviewBoard I cannot provided the parent patch of "Commit 1" and "Commit 2".
• ReviewBoard cannot interpret the chain of commits from "Commit F" to "Commit 3".
• If I upload "Commit 3" and parent "Commit 1", the missing index is: Commit F <- Commit 1 <- ? <- Commit 3
• If I upload "Commit 3" and parent "Commit 2", the missing index is: Commit F <- ? <- Commit 2 <- Commit 3

[Stephen Gallagher]

On Thu, Apr 23, 2015 at 11:49 AM James Knight <james.d...@live.com> wrote:

Repository options are configured as follows:

• Hosting Service: None - Custom Repository
• Repository Type: Git
• Path: git@myserver:mygroup/myproject.git
• Raw file URL mask: http://myserver/mygroup/myproject/raw/develop/<filename>

This isn't going to work if you don't have a way to represent the individual file hashes. That's why it can't find them to compare. What tool are you using to view the files via the web? Generally, cgit and gitweb work best.

[David Trowbridge]

Yeah, the raw file URL needs to have the revision in there somewhere. Since you're using GitLab, you should just choose "GitLab" instead of "None - Custom Repository"

-David

--

[James Knight]

First of all, thanks for the replies; I appreciate the help.

@Stephen Gallagher

I think this is were I am failing to communicate. I'm not trying to have my Git repository or web viewer to represent the file hashes as I haven't pushed anything to a remote Git repository. I'm hoping to avoid this by providing raw patches with full Git indexes to fill in the gap.

I'm going off the concept I see here.

A parent diff can be uploaded along with the main diff. A parent diff is a diff based on an existing commit in the repository, which will be applied before the main diff. The parent diff will not be included in the diff viewer. It’s useful when developing a change based on a branch that is not yet committed. In this case, a parent diff of the parent branch would be provided along with the diff of the new commit, and only the new commit will be shown.

I'm assuming that ReviewBoard just doesn't support multiple parent diffs. My current work around is to squash the series of parent commits for a given patch and add rebase my commits off their respective squashed parents. This will provide me with both my diff to review and a parent diff that I can submit to ReviewBoard. For example:

     {Local}

     Commit 4

        |

     Commit 3

        |

     Commit 2      Commit 3b           Commit 4b

        |             |                   |

     Commit 1   Squash Commit 1-2   Squash Commit 1-3

        |             |                   |

        |-------------|--------------------

        |

     Commit F

        |

     Commit E

• Make review 1 with `Commit 1` patch.
• Make review 2 with `Commit 2` patch with parent `Commit 1` patch.
• Make review 3 with `Commit 3b` patch with parent `Squash Commit 1-2` patch.
• Make review 4 with `Commit 4b` patch with parent `Squash Commit 1-3` patch.

Again, I know this isn't ideal but it works for now.

@David Trowbridge

I can't use the GitLab optional since it requires (to my knowledge) it's either for online GitLab hosting (which it not what I'm using; using a local GitLab CE installation) or requires an account setup on the GitLab server (which again, will not for us since we use LDAP and favor deployment keys). Unless there's another option I'm missing?

[Stephen Gallagher]

You're missing the point though. You still have to have an addressable hash from the repo in order to establish a baseline or else none of the parent diffs will have anything to compare against.

[James Knight]

Sorry, I don't understand.

My root patch has an addressable hash from the repository. When I initially showed the following diagram, the intent was to show that my local and ReviewBoard-watched remote repository are in sync.

                   {ReviewBoard}

                     /\

                     ||

                     \/

     {Local}      {Remote}

     

     Commit F      Commit F

        |             |

     Commit E      Commit E

If I then make a commit in my local repository, generate a patch, I can submit it successfully to ReviewBoard since all file indexes are addressable:

    /- (patch) ->  {ReviewBoard}

   /                 /\

  /                  ||

  |                  \/

   \ {Local}      {Remote}

    \  

     Commit 1

        |

     Commit F      Commit F

        |             |

     Commit E      Commit E

The patch changes to existing file indexes will all exist in Commit F (which ReviewBoard can handle) and any new files will have a zeroed-file index. If I introduce a new file in commit 1, I'll have an entry in my diff as follows:

diff --git a/newfile b/newfile

new file mode 100644

index 0000000000000000000000000000000000000000..E0C9035898DD52FC65C41454CEC9C4D2611BFB37

--- /dev/null

+++ b/newfile

Now, if I have a second commit which modifies the file (E0C9035898DD52FC65C41454CEC9C4D2611BFB37) I've introduced in Commit 1, ReviewBoard will not accept the patch since it cannot find the E0C9035898DD52FC65C41454CEC9C4D2611BFB37 object.

    /- (patch) ->  {ReviewBoard}    <-- Will fail.

   /                 /\

  /                  ||

  |                  \/

   \ {Local}      {Remote}

    \  

     Commit 2

        |

     Commit 1

        |

     Commit F      Commit F

        |             |

     Commit E      Commit E

Luckily, ReviewBoard supports uploading parent DIFF's so I can bridge the gap. By uploading my Commit 2 DIFF with a parent Commit 1 DIFF, ReviewBoard accepts the patch with no issues. I assume that this is the case since ReviewBoard has a list of all file indexes from Commit F and an overlay of file indexes from the parent DIFF provided. Therefore, if my Commit 2 has the following:

diff --git a/newfile b/newfile

new file mode 100644

index E0C9035898DD52FC65C41454CEC9C4D2611BFB37..7E240DE74FB1ED08FA08D38063F6A6A91462A815

+++ a/newfile

+++ b/newfile

ReviewBoard can establish following:

    0000000000000000000000000000000000000000 <- E0C9035898DD52FC65C41454CEC9C4D2611BFB37 <- 7E240DE74FB1ED08FA08D38063F6A6A91462A815

And the following will work:

    /--- (parent patch) \

   /                    \/

  / /- (patch) ->  {ReviewBoard}    <-- Will work.

 / /                 /\

/ /                  ||

| |                  \/

|  \ {Local}      {Remote}

 \  \  

  \  Commit 2

   \    |

     Commit 1

        |

     Commit F      Commit F

        |             |

     Commit E      Commit E

This leads back to the initial question. If I had a third commit (or more), how can I get ReviewBoard to interpret the changes? From what it looks like, it doesn't seem possible. For example, if Commit 3 had the following:

diff --git a/newfile b/newfile

new file mode 100644

index 7E240DE74FB1ED08FA08D38063F6A6A91462A815..70C881D4A26984DDCE795F6F71817C9CF4480E79

+++ a/newfile

+++ b/newfile

There is no means which I can provide all three (3) DIFFs to ReviewBoard to establish this chain:

    0000000000000000000000000000000000000000 <- E0C9035898DD52FC65C41454CEC9C4D2611BFB37 <- 7E240DE74FB1ED08FA08D38063F6A6A91462A815 <- 70C881D4A26984DDCE795F6F71817C9CF4480E79

I don't know how to explain it more than that. :(

[Christian Hammond]

Hey James,

The problem really has to do with the limitations we're under when talking to a Git repository. Let me go into that and then I'll go into how that relates to what you're dealing with.

The reason that raw file URL field exists is because, with Git, it's not possible to request a given file at a given SHA. You can clone the whole repository, but that's not something Review Board can sanely do for every change.

So, what we do instead is we build a URL, based off the mask provided, that can give us the raw contents of a file, given a file path and a blob SHA1.

We need this so that we have a source file to apply a patch on top of. Without that, we can't show a side-by-side diff.

Some services provide such a URL. GitWeb, for instance, has one. GitLab does not. For GitLab, we have to instead query their API to get the data we need. That requires such things as API tokens and other IDs, which Review Board has special code to deal with, but that only works when selecting GitLab as a hosting service.

Without either a selected (compatible) hosting service or a suitable Raw File URL, we just have no ability to get the data needed in order to render a change consistently.

If you have such a URL but without the field for the SHA, and you're only ever dealing with reviewing changes on top of the very latest revision in the repository, it will work, but that's going to fail the very moment someone puts something up for review that is based on an older commit (and this will happen in real usage all the time), or on top of a commit in a branch other than 'master'.

Now, as for the analysis you've done, and the need for a third file, the reason this is at all a problem is because you don't have the above setup, so you're having to play games with your repository in a way that just doesn't work.

If you did have such a setup, what you'd do is generate a diff between the latest upstream commit and the commit just before the one you want to post for review. That diff may cover a whole number of commits, but it doesn't matter. That's the "parent diff." Once we fetch the proper source file from the repository (using the hosting service or the raw file URL), we apply the parent diff, and treat that as the base for the diff being reviewed. Then, we apply the diff representing the commit(s) you want to actually review.

RBTools takes care of all this automatically, letting you just do:

    $ rbt post <mysha>

Christian

--
Christian Hammond - chi...@chipx86.com
Review Board - http://www.reviewboard.org

Beanbag, Inc. - http://www.beanbaginc.com

[James Knight]

Ah, completely understand now. Sorry for the trouble folks.

Thanks Christian (and of course David and Stephen as well).