Chromium development, Git, and you

[Chase Phillips]

Hello Chromium Developer,

Chrome PA's Infra team is working to migrate Chromium from a SVN project with a Git mirror into a Git project.

Why do this?

If you're interested in reasons behind this effort, you can read about that here.

What does this mean for me?

Today you fetch the code through Git and commit changes (when not using the CQ) through SVN.  Once the migration is complete, you will fetch code through Git and commit changes (when not using the CQ) by pushing the change to a Git repository.

Is there a plan somewhere?

Yes, you can read more here.

What can I do to prepare?

You should start using Git right now for development and Chromium mirroring.  The current instructions to get the code highlight how to get a properly configured git checkout of chromium.  If you have a checkout produced by the instructions in the link above, then it will continue to work once we migrate the repository to Git.

What exactly are you migrating to Git?

This project is focused exclusively on migrating svn://svn.chromium.org/chrome/trunk/src and the release branches of that repository to be read/write hosted in a Git repository.  Other parts of the SVN tree will not be migrated as part of this project.  We plan to come back to the other parts of the chrome/ SVN repository tree in the future after this is completed.

When will the switch happen?

The exact migration date is not yet set.  Google is organized around quarterly objectives, and so the people on the Infra team working on this project are aiming to have all of the prerequisites for the migration in place by the end of this quarter.  chromium-dev@ will have at least 3-4 weeks notice prior to the migration occurring.  We will pick a migration date in coordination with people involved in the release and trunk branching activity.  We absolutely want to find a date that is out of the way of any sensitive processes, so we'll all do our best to find a date that is the least impacting.

Will you mirror src.git Git repository content back into the SVN repository?

No.  When we migrate the source code to Git, the SVN repository parts that have been migrated will have write access disabled and they will stop being updated.  To fetch repository updates at this time, use a Git client to fetch the Chromium src.git repository.

Will the development process and 6-week release cycle for Chromium trunk and branch changes continue as it always had, just with a Git repository instead of a SVN repository?

Yes, no changes will happen to the 6-week release cycle and development process.

I can think of ways in which losing SVN revision numbers will interrupt my workflow.

That's good because we need your feedback for this to be a success.  Please fill out this form to give us more info about those cases.

Here is a link to this Chromium Git migration FAQ write-up, as well.

Thanks!

Chase

[Peter Kasting]

The last time I tried to work with one of the git-based subprojects of Chromium I discovered that there seemed to be no good equivalent to the SVN ViewVC tool we have online, which makes it easy to quickly trace back through the blame tree of a file.  There were a couple of different bare-bones git history viewing tools available, both of which were noticeably inferior in functionality and UI.

I scanned the FAQ and plan doc, but didn't immediately notice anything about this.  Is it considered "switchover-blocking" to ensure that the web revision history tooling for git repos is equivalent to the existing SVN tooling?  If not, can it be?

PK

[Chase Phillips]

On Mon, Mar 3, 2014 at 11:38 AM, Peter Kasting <pkas...@chromium.org> wrote:

The last time I tried to work with one of the git-based subprojects of Chromium I discovered that there seemed to be no good equivalent to the SVN ViewVC tool we have online, which makes it easy to quickly trace back through the blame tree of a file.  There were a couple of different bare-bones git history viewing tools available, both of which were noticeably inferior in functionality and UI.

I scanned the FAQ and plan doc, but didn't immediately notice anything about this.  Is it considered "switchover-blocking" to ensure that the web revision history tooling for git repos is equivalent to the existing SVN tooling?  If not, can it be?

The blame functionality you were missing has an equivalent available now.  See here for an example of blame on the root OWNERS file in chrome/trunk/src.  This capability has met the blame functionality from the POV of those we've spoken with so far.  Can you take a look at the example and let us know if it meets your view, too?

Chase

[Peter Kasting]

On Mon, Mar 3, 2014 at 12:54 PM, Chase Phillips <c...@google.com> wrote:

The blame functionality you were missing has an equivalent available now.  See here for an example of blame on the root OWNERS file in chrome/trunk/src.  This capability has met the blame functionality from the POV of those we've spoken with so far.  Can you take a look at the example and let us know if it meets your view, too?

There are a couple problems here:

(1) Because git hashes aren't sequential in nature and the view here displays a "rounded" last modified time, we have displays like:

12507e3 dpr...@chromium.org - 1 year, 4 months ago  ...

1f7d4b9 to...@chromium.org - 1 year, 4 months ago  ...

It's not possible to glance at this and tell which change came last.  The easiest fix I can think for this would be to change the "1 year, 4 months ago" string to a timestamp:

12507e3 dpr...@chromium.org - 12 Dec 2012 18:30 PST  ...

1f7d4b9 to...@chromium.org - 19 Dec 2012 4:27 PST  ...

(2) Clicking on one of these blame lines sends you to an annotated version of the file as of that commit.  ViewVC's behavior, which is vastly more useful, takes you to the diff from the selected commit against the previous file version (from which you can get to the full file as of either version).  With this UI, I don't see a trivial mechanism to compute that diff.

To me both of these would be seriously detrimental to my workflow.  It would be nice to address both before the switch.

PK

[Haixia Shi]

Would it still be possible to assign a sequential revision ID at the time of commit (to trunk) and append that to the commit message?

In addition to being sequential, the revision ID is also shorter and easier to memorize.

A short 7-char git hash is not guaranteed to be unique, for example

https://chromium.googlesource.com/chromium/src/+/5eed8577

https://chromium.googlesource.com/chromium/src/+/5eed857e

and

https://chromium.googlesource.com/chromium/src/+/e2cfc1fc6

https://chromium.googlesource.com/chromium/src/+/e2cfc1fc7

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Zelidrag Hornung]

+1 to what Haixia said - with sequential rev# you can also easily figure out if a given patch is part of a build or a branch

[John Abd-El-Malek]

On Mon, Mar 3, 2014 at 12:54 PM, Chase Phillips <c...@google.com> wrote:

Just saw this update, nice to see that work is going on this tool. To try it out, I picked a recent example that git users couldn't track down and which we were investigating because of a bug. Let's say I want to look at RenderViewImpl::OnMessageReceived, and the "GetContentClient()->SetActiveURL(main_frame->document().url());" line near the top and see the cl which added it. Currently using ViewVC my steps are:

1) go to src.chromium.org/viewvc/, find the file

2) 2s later http://src.chromium.org/viewvc/chrome/trunk/src/content/renderer/render_view_impl.cc?revision=254479 loads, click on "show annotations"

3) 20s later the page loads, I see that line with a link http://src.chromium.org/viewvc/chrome/trunk/src/content/renderer/render_view_impl.cc?r1=163060&r2=163061&. It's a cl from me moving code into a namespace. However the link is helpful in that it has a side-by-side diff and the left side has a link to the previous revision before this change, so I click on that link.

4) I go through the above line 4 more times because that line changed by a bunch of people. The total network load time is 35s. There's a bug in viewvc when the change is before a file rename, since it tries to generate a url with the old filename and that fails. However that bug is easy to workaround by correcting the filename in the url, which I know since it's what I started with. I finally get to the original cl that added this: http://src.chromium.org/viewvc/chrome?revision=11337&view=revision

This process was more convoluted than normal, because that line had 5-6 changes to it. However even then, it takes me about a minute or two to do this tracking.

I tried to do this with the git tool.

1) I click "blame" on https://chromium.googlesource.com/chromium/src/+/master/content/renderer/render_view_impl.cc

2) 108s later, I look for SetActiveURL and see that first change from me. It has a link on the left to https://chromium.googlesource.com/chromium/src/+blame/a80af12efcba198f4beda21c1d88c63f04c4dc41/content/renderer/render_view_impl.cc

3) 60s later, it takes me to https://chromium.googlesource.com/chromium/src/+blame/a80af12efcba198f4beda21c1d88c63f04c4dc41/content/renderer/render_view_impl.cc, i.e. the same link where I came from. At this point, I can't do anything.

So it seems that the tool needs more work to find changes as it could only go one step backwards. Having it match viewvc to go back multiple changes is needed as this information is often crucial in figuring out why a piece of code is the way it is when refactoring.

Hopefully these steps are useful to the team working on this tool in figuring out how to get this sort of investigation to work.

Chase

[Scott Violet]

Once we switch over will direct commits (not using the cq) be
automatically squashed?

-Scott

[Chris Mumford]

Post switch will we still have 250K+ branch merges, or can those be removed?

*   8dcc7ef (HEAD, origin/master, origin/HEAD, master) SVN changes up to revision 254595

|\

| *   890eb48 SVN changes up to revision 254593

| |\

| | *   3f189eb SVN changes up to revision 254591

| | |\

| | | *   961a421 SVN changes up to revision 254587

| | | |\

| | | | *   beadbd2 SVN changes up to revision 254584

| | | | |\

| | | | | *   80be447 SVN changes up to revision 254580

| | | | | |\

| | | | | | *   909b063 SVN changes up to revision 254578

| | | | | | |\

| | | | | | | *   8c83f060 SVN changes up to revision 254576

| | | | | | | |\

| | | | | | | | *   336e25c SVN changes up to revision 254575

| | | | | | | | |\

| | | | | | | | | *   566a15f SVN changes up to revision 254574

[Stefan Zager]

On Mon, Mar 3, 2014 at 1:59 PM, John Abd-El-Malek <j...@chromium.org> wrote:
>
> Just saw this update, nice to see that work is going on this tool. To try it

> out, I picked a recent example ...

It's true that the existing git blame functionality is somewhat
lacking; we are aware of it, and we plan to address it.

This mailing list is probably not the right place to discuss it, since
the primary developers of the googlesource.com web UI don't read it.
Here is the original bug requesting the blame feature:

https://code.google.com/p/gitiles/issues/detail?id=5

I'd advise you to cross-post there, to ensure that it gets to the right people.

[Stefan Zager]

On Mon, Mar 3, 2014 at 3:05 PM, Scott Violet <s...@chromium.org> wrote:
> Once we switch over will direct commits (not using the cq) be
> automatically squashed?

The repository will have a hard restriction that every push can only
upload a single new commit. So, yes, direct commits will have to be
squashed. I'm pretty sure that 'git cl push' will do this for you,
but I haven't verified that. I'll make sure it's part of the
developer docs (under construction).

Stefan

[Scott Violet]

Please make sure it automatically does this for us. Having to remember
to squash is extra work for every developer.

-Scott

[Chris Hamilton]

Having recently gone through this process for the Syzygy repo I can indeed
confirm that "git cl push" squashes things for you automatically.

One thing to watch out for is tracking: if your branch is not explicitly
configured to track origin/master it will simply push nothing, close your
Rietveld issue, and report success. This is contrary to the fact that "git
cl upload" seems to explicitly diff against master if no explicit tracking
branch is set. This is also different from the git svn workflow as no
tracking branch needs to be specified there either.

I made a note to look into why this difference exists (and potentially fix
it) but haven't gotten around to it.

[Robert Iannucci]

Yes, that stuff will go away. If you do 'git log origin/git-svn' today that is what 'origin/master' will look like after the switch.

--

[Robert Iannucci]

As part of the migration, we're documenting a recommended workflow which will ensure that you don't get into a weird state (like with branch tracking information). We'll share out a link to that documentation when it's ready (it's a rough draft right now).

We'll be testing (and adding unit tests) for the tools (like `git cl push`) to ensure that they behave in non-aggravating ways :). Pushing to nothing and then closing the issue is pretty tacky. Found the bug crbug.com/285408 to track this particular issue.

[Chase Phillips]

Thanks, I noted your comment in https://code.google.com/p/gitiles/issues/detail?id=5.

On Mon, Mar 3, 2014 at 1:02 PM, Peter Kasting <pkas...@chromium.org> wrote:

[Robert Iannucci]

Sequential revision numbers are tricky, though we can get something like SVN's numbers with the depot_tools program `git number`.

We think that the numbering produced by this tool will be sufficient to fill many of the current use cases of svn's revision numbers. It should cover mental checks of 'Did my commit end up in branch X?', and 'Does this build contain my commit?'.

Two of the big advantages of switching to git will be local branching workflows and fast multi-homed hosting services. Both of these make a repo-global revision number, i.e. a number that increments monotonically across all branches, difficult to implement efficiently (if it's even possible at all).

If you have other uses in mind, please take a moment to fill out the survey so we can gather feedback about how people rely on svn revisions in other ways :).

If there is enough interest, I would be happy to give a short tech-talk on the ins and outs of all we've learned so far on this topic :)

R

[Scott Hess]

Honestly, for checking if my revision is in a particular build, I'd far rather have all branches in my repo and be able to ask git directly whether my revision is in the build.  It could be easily wrapped up in a git-cl sub-command to encode our branching and build-tagging assumptions.

I may be grossly mis-understand something, but I have no idea why one's local workflow decisions or distributed repo hosting would have any bearing at all on global monotonic revision numbers for what Chrome is built out of.  While git itself allows very flexible relationships between different repos, Chrome releases will be built off of a specific repo which must have a well-specified commit chain.  If that wasn't true, Chrome builds wouldn't be reproducible, which is unacceptable.  So that repo has the capability to inject information into the commits without worrying that some other repo will deliver a conflicting view of the world.  If people want to rearrange things locally, fine, but they aren't talking about Chrome releases in that case.

-scott

[Robert Iannucci]

On Mon, Mar 3, 2014 at 11:36 PM, Scott Hess <sh...@chromium.org> wrote:

Honestly, for checking if my revision is in a particular build, I'd far rather have all branches in my repo and be able to ask git directly whether my revision is in the build.  It could be easily wrapped up in a git-cl sub-command to encode our branching and build-tagging assumptions.

Yes, directly invoking `git branch --contains <commit>` is the most accurate way to determine if, well, a branch contains your commit. Doing so requires a local git checkout though, and it's a difficult operation to perform in your head (unless you memorize the chromium commit chain).

 

I may be grossly mis-understand something, but I have no idea why one's local workflow decisions or distributed repo hosting would have any bearing at all on global monotonic revision numbers for what Chrome is built out of.  While git itself allows very flexible relationships between different repos, Chrome releases will be built off of a specific repo which must have a well-specified commit chain.  If that wasn't true, Chrome builds wouldn't be reproducible, which is unacceptable.  So that repo has the capability to inject information into the commits without worrying that some other repo will deliver a conflicting view of the world.  If people want to rearrange things locally, fine, but they aren't talking about Chrome releases in that case.

Maybe I'm misunderstanding something in that case... I didn't realize we were discussing official releases :). The numbering I was mentioning would not be used for any official branching/releasing process, just as a convenience method for humans without a local repo handy when looking at builds on the waterfall.

I was claiming that due to the flexible nature of git (specifically the local branching/distributed replication bits), having a svn-style revision number is difficult/impossible, but there are other, similar numbering schemes which can be useful (such as the one implemented by `git number`).

I'm not sure where the other repos come in though :)

R

[Zachary Turner]

Is there a reason why the timestamp is not sufficient for ordering commits in a fashion similar to what people are used to being able to do with an svn revision number?  Aside from the obvious of that it doesn't go up by 1 and it takes a slight amount of extra mental processing to compare two timestamps, it seems like (in theory), you could create a one-to-one mapping between timestamps and positive integers that preserves ordering, even if the tool didn't do that for us.  Does it really matter that some revision has the global number of 345789, or does it only matter that 345789 is less than 356820, for example?

[Robert Iannucci]

Unfortunately, there are a number of issues with timestamps:

  a) clocks are not globally synchronous

  b) timestamps of git commits are produced on the user's local machine

  c) time of git push != time of commit object creation

    1) b+c imply that even if commit A comes before commit B, it may actually have a later timestamp

  d) it gets kind-of tricky when changes land in rapid succession. You have to start looking at timestamps down to the level of seconds, which is easy to get wrong.

  e) git timestamps generally only have per-second accuracy, so it's quite possible for multiple commits to have the same timstamp

The git-number tool gives you numbers for commits which only observe their actual dependency relationship and can be used for basic 'less-than' comparisons (caveat: you can only compare revisions along the same line of history, i.e. not between branches.)

[Matt Giuca]

I just tried "git number" (didn't know about it) and it worked pretty well -- took about 5-10 seconds the first time, and subsequent times it was instant. I assume it's got a cache somewhere, of gitcommit -> revno, so if you query it again, it only has to trace back until it hits a gitcommit that's in the cache, and then just adds the revno to the existing count.

It reported 201191 for SVN 254424 -- presumably all of the non-trunk commits are not being counted here. If we do switch to such a system, we'll have to carefully communicate those new numbers, at least until they surpass the SVN number (because people are bound to be confused when the number suddenly drops by 50K).

I propose that we run "git number" during a Chrome build and put the result in chrome://version. If the cache works the way I assume it does, it will only add 5-10 seconds to a fresh build, and zero seconds to an incremental build.

I do think we want some good shared sequential number which we use to communicate about versions. I compare SVN revision numbers countless times per day and I would hate to lose the ability to quickly tell whether one build is older or newer than another (for example, to be able to look at chrome://version on Canary to see if my patch made it in, and dozens of other uses for a quick visual comparison). While solving this problem in the general case is hairy[1], assuming we plan to keep using a linear history for the main line development, it should not be too hard to use a scheme like git number. Tracking revisions on branches is trickier, but less necessary (since you have the build number for that).

[1] I say this a lot, but I'm quite sad that Bazaar lost to Git in the revision control wars, and one reason is that Bazaar did distributed revision numbering properly. When you are on trunk, every commit on the main branch of history (the git equivalent of log --first-parent) had a sequential integer, essentially the same as "git number". Commits that were merged in from other branches would be the triple X.Y.Z, where X is the trunk commit where it forked off, Y is usually 1 (but can be higher if multiple branches forked off at the same point), and Z is the revision number within the branch. Bazaar would cache these numbers so it didn't require a linear scan of the history to figure out what your number was. I suppose such a scheme could be built on top of Git (and this "git number" thing is basically a step in that direction).

On 4 March 2014 19:06, Zachary Turner <ztu...@chromium.org> wrote:

Is there a reason why the timestamp is not sufficient for ordering commits in a fashion similar to what people are used to being able to do with an svn revision number?  Aside from the obvious of that it doesn't go up by 1 and it takes a slight amount of extra mental processing to compare two timestamps, it seems like (in theory), you could create a one-to-one mapping between timestamps and positive integers that preserves ordering, even if the tool didn't do that for us.  Does it really matter that some revision has the global number of 345789, or does it only matter that 345789 is less than 356820, for example?

A few points:

1. Time stamps are technically not ordered. You can easily have a git commit with an older timestamp than its parent (it happens all the time when git rebase is involved). Now presubmit checks may enforce some level of sanity (you could make a check that refuses a commit that does not have a newer timestamp than its parent), but I'd be scared of edge cases.

2. "a one-to-one mapping between timestamps and positive integers" -- generally known as Unix timestamps. Even assuming seconds are fine grained enough (which they may not be), we're talking about 10-digit numbers (1393920797) that increment by 86400 every day. It's much harder to visually compare two Unix timestamps than 6-digit SVN numbers.

[Jiang Jiang]

Hi,

On Mon, Mar 3, 2014 at 8:32 PM, Chase Phillips <c...@google.com> wrote:
> Chrome PA's Infra team is working to migrate Chromium from a SVN project
> with a Git mirror into a Git project.

I couldn't find it from the docs or your email. Will the submodule
workflow [1] be completely phased out? Many of us still prefer it to
gclient/fetch and so far it seems to work well enough and much easier
to understand than gclient's magic.

- Jiang

[1] https://code.google.com/p/chromium/wiki/UsingGitSubmodules

[Peter Kasting]

On Tue, Mar 4, 2014 at 12:17 AM, Robert Iannucci <iann...@chromium.org> wrote:

Unfortunately, there are a number of issues with timestamps:

  a) clocks are not globally synchronous

  b) timestamps of git commits are produced on the user's local machine

  c) time of git push != time of commit object creation

    1) b+c imply that even if commit A comes before commit B, it may actually have a later timestamp

  d) it gets kind-of tricky when changes land in rapid succession. You have to start looking at timestamps down to the level of seconds, which is easy to get wrong.

  e) git timestamps generally only have per-second accuracy, so it's quite possible for multiple commits to have the same timstamp

That's fine, but none of these issues really disqualifies timestamps for the purpose I wanted them for, which was for the version control web interface to use them; in general, having visible timestamps, even with the inaccuracies noted here, would make e.g. the blame interface much more useful.

PK

[Robert Iannucci]

On Tue, Mar 4, 2014 at 12:20 AM, Matt Giuca <mgi...@chromium.org> wrote:

I just tried "git number" (didn't know about it) and it worked pretty well -- took about 5-10 seconds the first time, and subsequent times it was instant. I assume it's got a cache somewhere, of gitcommit -> revno, so if you query it again, it only has to trace back until it hits a gitcommit that's in the cache, and then just adds the revno to the existing count.

Yep. This cache is stored as a hash table locally in the git repo on an internal ref (refs/number/commits for those who care).

 

It reported 201191 for SVN 254424 -- presumably all of the non-trunk commits are not being counted here. If we do switch to such a system, we'll have to carefully communicate those new numbers, at least until they surpass the SVN number (because people are bound to be confused when the number suddenly drops by 50K).

Right. The precise algorithm is (pardon my pseudocode):

  number(commit, parents) = max(map(number, parents)) + 1

  number(commit, [])          = 0

 

I propose that we run "git number" during a Chrome build and put the result in chrome://version. If the cache works the way I assume it does, it will only add 5-10 seconds to a fresh build, and zero seconds to an incremental build. 

I do think we want some good shared sequential number which we use to communicate about versions. I compare SVN revision numbers countless times per day and I would hate to lose the ability to quickly tell whether one build is older or newer than another (for example, to be able to look at chrome://version on Canary to see if my patch made it in, and dozens of other uses for a quick visual comparison). While solving this problem in the general case is hairy[1], assuming we plan to keep using a linear history for the main line development, it should not be too hard to use a scheme like git number. Tracking revisions on branches is trickier, but less necessary (since you have the build number for that).

I think you're aware of this, but for those following along, the difference between git number and SVN revisions stems from the algorithm above. If you take a look, you'll see that if you branch at, say, a generation 100 commit, then the next commit on that branch as well as the next commit on master will both have a generation of 101.

The mapping of commit -> number is unique and deterministic. The reverse mapping is not. This makes it suitable for some commit ordering comparisons, but not for identifying the absolute positioning of commits in the commit graph.

No comment on bzr (I'm not very familiar with it :), but the basic difficulty in git (in the general case you mention) is that branch refs are made 'after the fact'. This is both good (it makes branching really easy!), and bad (refs don't have intrinsic history to them).

The claim is that since we can pre-determine the topology of the chromium project, we could devise a unique X.Y.Z -> hash numbering scheme which would work for chromium.

I'm on the fence about whether or not we should implement such a scheme (for official purposes), though :). Implementing this scheme would likely require a pretty large amount of tooling support, and it would be quite foreign to git, which means it would have an on-going maintenance cost (both in tooling and education). So... not impossible, but also not free.

/me mumbles something about 'great power -> responsibility'

[Robert Iannucci]

To clarify, you're wanting there to be a way to show absolute timestamps on e.g. https://chromium.googlesource.com/chromium/src instead of the relative timestamps?

[Stefan Zager]

On Tue, Mar 4, 2014 at 12:23 AM, Jiang Jiang <jia...@opera.com> wrote:
> Hi,
>
> On Mon, Mar 3, 2014 at 8:32 PM, Chase Phillips <c...@google.com> wrote:
>> Chrome PA's Infra team is working to migrate Chromium from a SVN project
>> with a Git mirror into a Git project.
>
> I couldn't find it from the docs or your email. Will the submodule
> workflow [1] be completely phased out? Many of us still prefer it to
> gclient/fetch and so far it seems to work well enough and much easier
> to understand than gclient's magic.

The submodule flow will most likely break when we do the final git
migration, and it's unclear whether or when we'd be able to re-enable
it. Personally, I'd like to keep supporting it, but it will require a
bit of technical work to do so, and that work will be on the back
burner while we drive to get the migration done.

Stefan

[Colin Blundell]

On Tue, Mar 4, 2014 at 8:50 AM, Robert Iannucci <iann...@chromium.org> wrote:

On Mon, Mar 3, 2014 at 11:36 PM, Scott Hess <sh...@chromium.org> wrote:

Honestly, for checking if my revision is in a particular build, I'd far rather have all branches in my repo and be able to ask git directly whether my revision is in the build.  It could be easily wrapped up in a git-cl sub-command to encode our branching and build-tagging assumptions.

Yes, directly invoking `git branch --contains <commit>` is the most accurate way to determine if, well, a branch contains your commit. Doing so requires a local git checkout though, and it's a difficult operation to perform in your head (unless you memorize the chromium commit chain).

 

I may be grossly mis-understand something, but I have no idea why one's local workflow decisions or distributed repo hosting would have any bearing at all on global monotonic revision numbers for what Chrome is built out of.  While git itself allows very flexible relationships between different repos, Chrome releases will be built off of a specific repo which must have a well-specified commit chain.  If that wasn't true, Chrome builds wouldn't be reproducible, which is unacceptable.  So that repo has the capability to inject information into the commits without worrying that some other repo will deliver a conflicting view of the world.  If people want to rearrange things locally, fine, but they aren't talking about Chrome releases in that case.

Maybe I'm misunderstanding something in that case... I didn't realize we were discussing official releases :). The numbering I was mentioning would not be used for any official branching/releasing process, just as a convenience method for humans without a local repo handy when looking at builds on the waterfall.

I was claiming that due to the flexible nature of git (specifically the local branching/distributed replication bits), having a svn-style revision number is difficult/impossible, but there are other, similar numbering schemes which can be useful (such as the one implemented by `git number`).

I'm not sure where the other repos come in though :)

FWIW, I also don't understand why local branching would have any effect on the ability to assign global monotonic revision numbers for patches that are committed to the Chromium source tree: the assignment of such a revision number would be done only when the CL is pushed to the remote I assume? I have less insight into how "distributed replication bits" might interact with this desire.

Thanks,

Colin

[Robert Iannucci]

The problem is that a globally incrementing number would mean that you would need to:

  1) distinguish between local and remote branches (git doesn't really do this)

  2) atomically

    a) capture the revision number of all branches in the repo 'right now'

    b) push a commit with max(revision) + 1 to the branch you care about

    c) hope no one else pushed to a different ref in the mean time

  3) Do this without the git server modifying your commit

1 is hard, but theoretically possible. 2 is just not a thing. 3 would be some other source control system that's not git.

If you only care about numbering on a single ref, then that's already done, and it's called `git number` :)

R

[Mike Frysinger]

On Tue, Mar 4, 2014 at 12:23 AM, Peter Kasting <pkas...@chromium.org> wrote:

they're also fully under control by the end user.  so which timestamp do you want to see ?  author date or commit date ?  generally the former is set once (at initial commit) while the latter can change many times (rebase/cherry pick/etc...).  git also provides options to automatically set the commit date to the author date when doing operations so that when you do rebase/etc..., the commit date stays the same as the author date.  i don't know how common it is, but i tend to use it myself.  it might not be too big an issue when using reitveld to do the commit+push, but when end users get direct push access, then all bets are off :).

-mike

[Greg Billock]

Could we git-tag revisions in the main build repo? We can do that sequentially. It would happen kinda out-of-band to git, but is within the use cases for tags and the repo is set up to preserve them faithfully. I think this would satisfy our tooling needs as well -- a commit isn't "in" until it's in the main repo, and that's when developers care what sequence number it has. There's some room for error with tags, but presubmits ought to be able to eliminate most of the easiest ones to make.

--

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[Daniel Bratell]

On Tue, 04 Mar 2014 18:39:22 +0100, Greg Billock <gbil...@google.com>
wrote:

> Could we git-tag revisions in the main build repo? We can do that
> sequentially. It would happen kinda >out-of-band to git, but is within
> the use cases for tags and the repo is set up to preserve them
> >faithfully. I think this would satisfy our tooling needs as well -- a
> commit isn't "in" until it's in >the main repo, and that's when
> developers care what sequence number it has. There's some room for
> >error with tags, but presubmits ought to be able to eliminate most of
> the easiest ones to make.

You don't want a tag for every commit. Tags pollute the global namespace
for all clients and there are some scaling issues when a git repository
has many (thousands to millions) references, be it tags or branches.

git has "notes" for when you want to write some metadata to a revision.
I've not used them myself but they are designed for things like what you
describe.

And talking about "describe", nobody has so far mentioned "git describe"
in this thread as far as I know. I think that is the gittiest way of
describing revisions so consider it mentioned.

/Daniel

[Aaron Gable]

On Tue, Mar 4, 2014 at 9:58 AM, Daniel Bratell <bra...@opera.com> wrote:

On Tue, 04 Mar 2014 18:39:22 +0100, Greg Billock <gbil...@google.com> wrote:

Could we git-tag revisions in the main build repo? We can do that sequentially. It would happen kinda >out-of-band to git, but is within the use cases for tags and the repo is set up to preserve them >faithfully. I think this would satisfy our tooling needs as well -- a commit isn't "in" until it's in >the main repo, and that's when developers care what sequence number it has. There's some room for >error with tags, but presubmits ought to be able to eliminate most of the easiest ones to make.

You don't want a tag for every commit. Tags pollute the global namespace for all clients and there are some scaling issues when a git repository has many (thousands to millions) references, be it tags or branches.

This is very true. Having thousands of named refs floating around is a recipe for poor performance in a lot of different domains.

 

git has "notes" for when you want to write some metadata to a revision. I've not used them myself but they are designed for things like what you describe.

And talking about "describe", nobody has so far mentioned "git describe" in this thread as far as I know. I think that is the gittiest way of describing revisions so consider it mentioned.

I urge everyone who is worried about monotonically increasing revision numbers to take a look at "git number". It's already live in depot_tools, so it should Just Work out of the box. "git number <ref>" will give you the generation number of that ref, i.e. the maximum number of commits between the root of the repository and the ref. You can also give "git number" a list of refs, in which case it will return the generation numbers for all of them. This is useful for sorting and comparing commits.

 

/Daniel

[Alexei Svitkine]

Is the plan to update all our tools that currently use the SVN number to use the git number? (e.g. omaha proxy, commit-bot, bugdroid, build waterfall, drover, chrome://version, etc) Is there a bug to track all this work? How will the transition be handled given that the numbers are different?

[Ryan Tseng]

On Tue, Mar 4, 2014 at 10:50 AM, Alexei Svitkine <asvi...@chromium.org> wrote:

Is the plan to update all our tools that currently use the SVN number to use the git number? (e.g. omaha proxy, commit-bot, bugdroid, build waterfall, drover, chrome://version, etc) Is there a bug to track all this work? How will the transition be handled given that the numbers are different?

Yes, the plan is to update our tool to use git tools, git-number or otherwise.

Most (all?) of the tools you mentioned above do not need to rely on the semantics a monotonically increasing revision number, and may already have native support for git hashes already.  We don't generally want to use the git numbers in our day to day operations (You can't "checkout" a git number, you can only use it to determine before/after semantics between two commits on the same branch)

Most of drover's functions can be done within native git.  The reason that drover is around is because it covered up deficiencies of SVN which is no longer required in git.

The meta tracking bug is here: https://code.google.com/p/chromium/issues/detail?id=341109 

On Tue, Mar 4, 2014 at 1:45 PM, Aaron Gable <aga...@chromium.org> wrote:

On Tue, Mar 4, 2014 at 9:58 AM, Daniel Bratell <bra...@opera.com> wrote:

On Tue, 04 Mar 2014 18:39:22 +0100, Greg Billock <gbil...@google.com> wrote:

Could we git-tag revisions in the main build repo? We can do that sequentially. It would happen kinda >out-of-band to git, but is within the use cases for tags and the repo is set up to preserve them >faithfully. I think this would satisfy our tooling needs as well -- a commit isn't "in" until it's in >the main repo, and that's when developers care what sequence number it has. There's some room for >error with tags, but presubmits ought to be able to eliminate most of the easiest ones to make.

You don't want a tag for every commit. Tags pollute the global namespace for all clients and there are some scaling issues when a git repository has many (thousands to millions) references, be it tags or branches.

This is very true. Having thousands of named refs floating around is a recipe for poor performance in a lot of different domains.

 

git has "notes" for when you want to write some metadata to a revision. I've not used them myself but they are designed for things like what you describe.

And talking about "describe", nobody has so far mentioned "git describe" in this thread as far as I know. I think that is the gittiest way of describing revisions so consider it mentioned.

I urge everyone who is worried about monotonically increasing revision numbers to take a look at "git number". It's already live in depot_tools, so it should Just Work out of the box. "git number <ref>" will give you the generation number of that ref, i.e. the maximum number of commits between the root of the repository and the ref. You can also give "git number" a list of refs, in which case it will return the generation numbers for all of them. This is useful for sorting and comparing commits.

 

/Daniel

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:   http://groups.google.com/a/chromium.org/group/chromium-dev

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[Peter Kasting]

On Tue, Mar 4, 2014 at 1:13 AM, Robert Iannucci <iann...@chromium.org> wrote:

To clarify, you're wanting there to be a way to show absolute timestamps on e.g. https://chromium.googlesource.com/chromium/src instead of the relative timestamps?

Yes; see my comments near the beginning of the thread.  The relative timestamps are increasingly imprecise ("1 year, 4 months ago") as the time delta increases, so for any changes not made very recently, it can be hard to tell the relative ordering between adjacent changes.

PK 

[Peter Kasting]

On Tue, Mar 4, 2014 at 8:23 AM, Mike Frysinger <vap...@chromium.org> wrote:

On Tue, Mar 4, 2014 at 12:23 AM, Peter Kasting <pkas...@chromium.org> wrote:

On Tue, Mar 4, 2014 at 12:17 AM, Robert Iannucci <iann...@chromium.org> wrote:

Unfortunately, there are a number of issues with timestamps:

  a) clocks are not globally synchronous

  b) timestamps of git commits are produced on the user's local machine

  c) time of git push != time of commit object creation

    1) b+c imply that even if commit A comes before commit B, it may actually have a later timestamp

  d) it gets kind-of tricky when changes land in rapid succession. You have to start looking at timestamps down to the level of seconds, which is easy to get wrong.

  e) git timestamps generally only have per-second accuracy, so it's quite possible for multiple commits to have the same timstamp

That's fine, but none of these issues really disqualifies timestamps for the purpose I wanted them for, which was for the version control web interface to use them; in general, having visible timestamps, even with the inaccuracies noted here, would make e.g. the blame interface much more useful.

they're also fully under control by the end user.  so which timestamp do you want to see ?

Whatever time value we use for the currently-displayed relative timestamps, I want displayed as absolute timestamps.  That's all.  No need to pick a new timestamp :)

PK 

[David Benjamin]

On Tue, Mar 4, 2014 at 1:45 PM, Aaron Gable <aga...@chromium.org> wrote:

On Tue, Mar 4, 2014 at 9:58 AM, Daniel Bratell <bra...@opera.com> wrote:

On Tue, 04 Mar 2014 18:39:22 +0100, Greg Billock <gbil...@google.com> wrote:

Could we git-tag revisions in the main build repo? We can do that sequentially. It would happen kinda >out-of-band to git, but is within the use cases for tags and the repo is set up to preserve them >faithfully. I think this would satisfy our tooling needs as well -- a commit isn't "in" until it's in >the main repo, and that's when developers care what sequence number it has. There's some room for >error with tags, but presubmits ought to be able to eliminate most of the easiest ones to make.

You don't want a tag for every commit. Tags pollute the global namespace for all clients and there are some scaling issues when a git repository has many (thousands to millions) references, be it tags or branches.

This is very true. Having thousands of named refs floating around is a recipe for poor performance in a lot of different domains.

What about a tag when BUILD and PATCH in chrome/VERSION bumps? That would make git describe --tags rather tidily answer "what builds do or don't have this commit?", which is when I most need to use revision numbers. Kind of the inverse to the Revision Lookup box on omahaproxy. Though perhaps those still bump frequently enough that git will be unhappy about having that many tags. Maybe an external tool instead is better for that, I dunno.

[Scott Hess]

On Tue, Mar 4, 2014 at 11:34 AM, David Benjamin <davi...@chromium.org> wrote:

On Tue, Mar 4, 2014 at 1:45 PM, Aaron Gable <aga...@chromium.org> wrote:

On Tue, Mar 4, 2014 at 9:58 AM, Daniel Bratell <bra...@opera.com> wrote:

On Tue, 04 Mar 2014 18:39:22 +0100, Greg Billock <gbil...@google.com> wrote:

Could we git-tag revisions in the main build repo? We can do that sequentially. It would happen kinda >out-of-band to git, but is within the use cases for tags and the repo is set up to preserve them >faithfully. I think this would satisfy our tooling needs as well -- a commit isn't "in" until it's in >the main repo, and that's when developers care what sequence number it has. There's some room for >error with tags, but presubmits ought to be able to eliminate most of the easiest ones to make.

You don't want a tag for every commit. Tags pollute the global namespace for all clients and there are some scaling issues when a git repository has many (thousands to millions) references, be it tags or branches.

This is very true. Having thousands of named refs floating around is a recipe for poor performance in a lot of different domains.

What about a tag when BUILD and PATCH in chrome/VERSION bumps? That would make git describe --tags rather tidily answer "what builds do or don't have this commit?", which is when I most need to use revision numbers. Kind of the inverse to the Revision Lookup box on omahaproxy. Though perhaps those still bump frequently enough that git will be unhappy about having that many tags. Maybe an external tool instead is better for that, I dunno.

Another option would be to have a git-<tool> to extra them from the check-in comments and cache them in .git/ somewhere.  Basically what git-svn does for findrev.

-scott

[Wojciech Dzierżanowski]

On Tue, 04 Mar 2014 07:48:45 +0100, Robert Iannucci
<iann...@chromium.org> wrote:

> Sequential revision numbers are tricky, though we can get something

> *like* SVN's

> numbers with the depot_tools program `git number`.
>
> We think that the numbering produced by this tool will be sufficient to
> fill many of the current use cases of svn's revision numbers. It should
> cover mental checks of 'Did my commit end up in branch X?', and 'Does
> this
> build contain my commit?'.
>
> Two of the big advantages of switching to git will be local branching
> workflows and fast multi-homed hosting services. Both of these make a
> repo-global revision number, i.e. a number that increments monotonically
> across all branches, difficult to implement efficiently (if it's even
> possible at all).
>
> If you have other uses in mind, please take a moment to fill out the

> survey<https://docs.google.com/a/chromium.org/forms/d/13pidx6fVeBkqN4zNjI1XpDVlmQiasIS5auhPTbmQvsY/viewform>

> so
> we can gather feedback about how people rely on svn revisions in other
> ways
> :).

I can't access the survey, so I'll post it here in case it's not there
already:

https://code.google.com/p/chromium/codesearch#chromium/src/sync/protocol/sync.proto&l=72

--
Wojtek

[Aaron Gable]

Woah, thanks for the heads-up. Luckily:

  // Note: The tag selection process is based on legacy versions of the
  // protocol which used protobuf extensions. We have kept the process
  // consistent as the old values cannot change.  The 5+ digit nature of the
  // tags also makes them recognizable (individually and collectively) from
  // noise in logs and debugging contexts, and creating a divergent subset of
  // tags would only make things a bit more confusing.

So really only the documentation needs to be updated -- the revision numbers chosen as tags aren't actually meaningful; that was just a useful way to make them large and unique.

Aaron

[Peter Kasting]

On Mon, Mar 3, 2014 at 1:02 PM, Peter Kasting <pkas...@chromium.org> wrote:

On Mon, Mar 3, 2014 at 12:54 PM, Chase Phillips <c...@google.com> wrote:

The blame functionality you were missing has an equivalent available now.  See here for an example of blame on the root OWNERS file in chrome/trunk/src.  This capability has met the blame functionality from the POV of those we've spoken with so far.  Can you take a look at the example and let us know if it meets your view, too?

There are a couple problems here:

Besides the problems I already mentioned, I've encountered another.  When trying to use gitiles' blame on the internal, pre-launch Chromium source tree, I found that the line height in the "blame" column is not identical to the line height in the "source" column, so that as you scroll down, the blame lines rapidly get out-of-sync with the source lines, and at the bottom of the page, you can see a discontinuity in the horizontal line that appears under the two columns.

This makes blame pretty much useless, because it's not clear where you should click to actually see the blame for a particular line of code.

PK

[Stefan Zager]

Can you please post this to the gitiles issue tracker:

https://code.google.com/p/gitiles/issues/detail?id=5

Thanks,

Stefan

[Peter Kasting]

On Wed, Mar 5, 2014 at 3:16 PM, Stefan Zager <sza...@google.com> wrote:

On Wed, Mar 5, 2014 at 3:14 PM, Peter Kasting <pkas...@chromium.org> wrote:> This makes blame pretty much useless, because it's not clear where you

> should click to actually see the blame for a particular line of code.

Can you please post this to the gitiles issue tracker:

https://code.google.com/p/gitiles/issues/detail?id=5

Done.

PK 

[Torne (Richard Coles)]

On 4 March 2014 18:50, Alexei Svitkine <asvi...@chromium.org> wrote:

Is the plan to update all our tools that currently use the SVN number to use the git number? (e.g. omaha proxy, commit-bot, bugdroid, build waterfall, drover, chrome://version, etc) Is there a bug to track all this work? How will the transition be handled given that the numbers are different?

Disclaimer: I'm not involved in the migration process, so this is all just my opinion as someone who appears to have reluctantly become a git expert:

I think we should absolutely not do this under any circumstances, and I'm dubious about "git number" even existing; I think it's an attractive nuisance that we would be better off without. I realise that sequential numbers are useful for all kinds of reasons, and like an earlier poster I too am sad that bzr lost the DVCS war for this reason, but all of git's design principles are just fundamentally opposed to the idea of having a sequential number that means anything useful and I think we have to just concede that this is tough. I've tried to retrofit sequential numbering in various different ways onto various projects and every time we have concluded that it's a counterproductive waste of time.

There are just too many ways for a human to make an incorrect assumption after examining numbers generated by this kind of simple calculation, because of the problems already mentioned by other people in this thread (git's dedication to treating all branches as equivalent whether local or remote, etc). I don't think the benefits outweigh the mistakes people will inevitably make.

 

On Tue, Mar 4, 2014 at 1:45 PM, Aaron Gable <aga...@chromium.org> wrote:

On Tue, Mar 4, 2014 at 9:58 AM, Daniel Bratell <bra...@opera.com> wrote:

On Tue, 04 Mar 2014 18:39:22 +0100, Greg Billock <gbil...@google.com> wrote:

Could we git-tag revisions in the main build repo? We can do that sequentially. It would happen kinda >out-of-band to git, but is within the use cases for tags and the repo is set up to preserve them >faithfully. I think this would satisfy our tooling needs as well -- a commit isn't "in" until it's in >the main repo, and that's when developers care what sequence number it has. There's some room for >error with tags, but presubmits ought to be able to eliminate most of the easiest ones to make.

You don't want a tag for every commit. Tags pollute the global namespace for all clients and there are some scaling issues when a git repository has many (thousands to millions) references, be it tags or branches.

This is very true. Having thousands of named refs floating around is a recipe for poor performance in a lot of different domains.

I've had discussions about this for the Android WebView with the maintainers of the git/gerrit instance on googlesource.com and we came to the conclusion that as long as you don't store these tags in the default refs/tags/* namespace there is unlikely to be a performance problem even with enormous numbers of named references. (in fact, *every* gerrit instance already has enormous numbers of named references under refs/changes, since every version of every CL uploaded for review is stored as a named reference forever, and so gerrit *has* to be able to deal with this).

Storing them under refs/buildversions/* or similar would mean that clients will not download them automatically, and so the mapping would not (by default) be available locally, but you could still trivially take a build version displayed on omahaproxy or chrome://version or wherever else and do "git ls-remote origin refs/buildversions/123456" to get the SHA for that build version. Since these labels would be arbitrary, we could in fact arrange for them to continue on logically from the SVN revision numbers, and avoid the issue where we'd have to jump backward.

Obviously if we wanted to go with that route the people here handling the migration need to discuss it with the googlesource owners and make sure that my understanding here is definitely correct, but if that's feasible, then I think this would be a much more sensible approach than the error-prone git-number script.

[Matt Giuca]

Sorry to resurface this somewhat stale thread.

On 6 March 2014 22:57, Torne (Richard Coles) <to...@chromium.org> wrote:

On 4 March 2014 18:50, Alexei Svitkine <asvi...@chromium.org> wrote:

Is the plan to update all our tools that currently use the SVN number to use the git number? (e.g. omaha proxy, commit-bot, bugdroid, build waterfall, drover, chrome://version, etc) Is there a bug to track all this work? How will the transition be handled given that the numbers are different?

Disclaimer: I'm not involved in the migration process, so this is all just my opinion as someone who appears to have reluctantly become a git expert:

Did you switch from Bzr too (judging by your next paragraph)? :( Pour one out for Bzr.

I think we should absolutely not do this under any circumstances, and I'm dubious about "git number" even existing; I think it's an attractive nuisance that we would be better off without. I realise that sequential numbers are useful for all kinds of reasons, and like an earlier poster I too am sad that bzr lost the DVCS war for this reason, but all of git's design principles are just fundamentally opposed to the idea of having a sequential number that means anything useful and I think we have to just concede that this is tough. I've tried to retrofit sequential numbering in various different ways onto various projects and every time we have concluded that it's a counterproductive waste of time.

The difference here is that, as far as I can tell, the plan for Chromium is to keep the trunk of the tree as a stick. Is this correct? This means that every commit will have a single parent (all of the local commits made during the construction of a CL will be squashed), possibly with the exception of the 200K SVN commits that each have 2 parents, but in a controlled manner (the height of both parents is always the same).

If my assumption is correct, then 'git number' will always have a sensible value as long as you are on the trunk of the tree. If we can somehow detect whether we are on the trunk (e.g., grep the most recent commit message for a fixed string like "Review URL: https://codereview.chromium.org"), then we can give a commit a revision number if and only if it is on trunk. We can report the "revision" like this:

• If it's on trunk (matches the "Review URL" string): "<git number> (<git hash>)"
• Otherwise: "<git number of most recent parent on trunk> + <n> branch commits (<git hash>)"

Another idea I've just had: have the CQ / presubmit script automatically put the revision number directly in the commit message. That way we could continue on from the SVN numbering without having to reset to a relatively smaller number. Every commit would include a string "Revision number: xxxxxx", and tools could find that string in the commit messages (for example, chrome://version would display the revision number of the most recent git commit). That would be preferable to polluting the tag namespace.

I've had discussions about this for the Android WebView with the maintainers of the git/gerrit instance on googlesource.com and we came to the conclusion that as long as you don't store these tags in the default refs/tags/* namespace there is unlikely to be a performance problem even with enormous numbers of named references. (in fact, *every* gerrit instance already has enormous numbers of named references under refs/changes, since every version of every CL uploaded for review is stored as a named reference forever, and so gerrit *has* to be able to deal with this).

Storing them under refs/buildversions/* or similar would mean that clients will not download them automatically, and so the mapping would not (by default) be available locally, but you could still trivially take a build version displayed on omahaproxy or chrome://version or wherever else and do "git ls-remote origin refs/buildversions/123456" to get the SHA for that build version. Since these labels would be arbitrary, we could in fact arrange for them to continue on logically from the SVN revision numbers, and avoid the issue where we'd have to jump backward.

The problem with tags, aside from performance, is the tag namespace. This will be up to each developer, but I personally use tags extensively on my local machine, to squirrel away branches I've committed or abandoned. (That way, I only have branches that I am actively working on.) I wouldn't want "git tag" to become filled with hundreds of thousands of tags that I didn't write.

[Robert Iannucci]

So, there's no official plan here, but (in linearized topological order ;):

  * `git number` does, in fact, exist :) (it's in depot tools. It works.)

  * `git number` is NOT unique across branches

  * Any full-numbering scheme involving `git number` would require fixing the topology of the chromium repo

  * We are not committing to fixing the topology of the repo at this time

  * But it will likely be very linear, so we could declare it fixed at some future point in time

  * The pros and cons of such a full-numbering scheme are hotly debated.

  * We are not planning to block the git migration on such a numbering scheme.

  * Huge numbers of tags aren't great for performance :)

R

[Torne (Richard Coles)]

On 10 March 2014 03:28, Matt Giuca <mgi...@chromium.org> wrote:

Sorry to resurface this somewhat stale thread.

On 6 March 2014 22:57, Torne (Richard Coles) <to...@chromium.org> wrote:

On 4 March 2014 18:50, Alexei Svitkine <asvi...@chromium.org> wrote:

Is the plan to update all our tools that currently use the SVN number to use the git number? (e.g. omaha proxy, commit-bot, bugdroid, build waterfall, drover, chrome://version, etc) Is there a bug to track all this work? How will the transition be handled given that the numbers are different?

Disclaimer: I'm not involved in the migration process, so this is all just my opinion as someone who appears to have reluctantly become a git expert:

Did you switch from Bzr too (judging by your next paragraph)? :( Pour one out for Bzr.

I've used just about every VCS that exists in some professional capacity, but bzr was one of my favourites :)

 

I think we should absolutely not do this under any circumstances, and I'm dubious about "git number" even existing; I think it's an attractive nuisance that we would be better off without. I realise that sequential numbers are useful for all kinds of reasons, and like an earlier poster I too am sad that bzr lost the DVCS war for this reason, but all of git's design principles are just fundamentally opposed to the idea of having a sequential number that means anything useful and I think we have to just concede that this is tough. I've tried to retrofit sequential numbering in various different ways onto various projects and every time we have concluded that it's a counterproductive waste of time.

The difference here is that, as far as I can tell, the plan for Chromium is to keep the trunk of the tree as a stick. Is this correct? This means that every commit will have a single parent (all of the local commits made during the construction of a CL will be squashed), possibly with the exception of the 200K SVN commits that each have 2 parents, but in a controlled manner (the height of both parents is always the same).

I don't think we want to commit to keeping the history linear forever, even if we do it during some transitional period.

 

If my assumption is correct, then 'git number' will always have a sensible value as long as you are on the trunk of the tree. If we can somehow detect whether we are on the trunk (e.g., grep the most recent commit message for a fixed string like "Review URL: https://codereview.chromium.org"), then we can give a commit a revision number if and only if it is on trunk. We can report the "revision" like this:

• If it's on trunk (matches the "Review URL" string): "<git number> (<git hash>)"
• Otherwise: "<git number of most recent parent on trunk> + <n> branch commits (<git hash>)"

This kind of heuristic is easy to fool by cherrypicking to release branches, or other fun. Having to come up with this kind of unreliable logic is exactly the reason I'm against the whole idea :)

I have tried to implement this before, and it's extremely hard to get it right even in the majority of cases, let alone when weird stuff comes up.

Another idea I've just had: have the CQ / presubmit script automatically put the revision number directly in the commit message. That way we could continue on from the SVN numbering without having to reset to a relatively smaller number. Every commit would include a string "Revision number: xxxxxx", and tools could find that string in the commit messages (for example, chrome://version would display the revision number of the most recent git commit). That would be preferable to polluting the tag namespace.

Again, this can only be done if you keep the history linear forever, which is bad.

 

I've had discussions about this for the Android WebView with the maintainers of the git/gerrit instance on googlesource.com and we came to the conclusion that as long as you don't store these tags in the default refs/tags/* namespace there is unlikely to be a performance problem even with enormous numbers of named references. (in fact, *every* gerrit instance already has enormous numbers of named references under refs/changes, since every version of every CL uploaded for review is stored as a named reference forever, and so gerrit *has* to be able to deal with this).

Storing them under refs/buildversions/* or similar would mean that clients will not download them automatically, and so the mapping would not (by default) be available locally, but you could still trivially take a build version displayed on omahaproxy or chrome://version or wherever else and do "git ls-remote origin refs/buildversions/123456" to get the SHA for that build version. Since these labels would be arbitrary, we could in fact arrange for them to continue on logically from the SVN revision numbers, and avoid the issue where we'd have to jump backward.

The problem with tags, aside from performance, is the tag namespace. This will be up to each developer, but I personally use tags extensively on my local machine, to squirrel away branches I've committed or abandoned. (That way, I only have branches that I am actively working on.) I wouldn't want "git tag" to become filled with hundreds of thousands of tags that I didn't write.

Anything not under refs/tags is not part of the tag namespace. "git tag" will not show them, and they are *not* subject to the git send-pack behaviour where any tag that points to a commit you download is automatically sent to you. So, you won't have these in your local repository at all unless you have manually reconfigured git to download them (which you probably won't want to because it will make git slow).

Technically, these would not be tags at all, since they would neither need to be actual tag objects (since they have no message or signature associated with them) or under the refs/tags/* namespace. They're just arbitrary refs, like the ones Gerrit creates under refs/changes for code reviews (if you're familiar with Gerrit from other projects).

[Scott Graham]

On Mon, Mar 3, 2014 at 1:59 PM, John Abd-El-Malek <j...@chromium.org> wrote:

On Mon, Mar 3, 2014 at 12:54 PM, Chase Phillips <c...@google.com> wrote:

On Mon, Mar 3, 2014 at 11:38 AM, Peter Kasting <pkas...@chromium.org> wrote:

The last time I tried to work with one of the git-based subprojects of Chromium I discovered that there seemed to be no good equivalent to the SVN ViewVC tool we have online, which makes it easy to quickly trace back through the blame tree of a file.  There were a couple of different bare-bones git history viewing tools available, both of which were noticeably inferior in functionality and UI.

I scanned the FAQ and plan doc, but didn't immediately notice anything about this.  Is it considered "switchover-blocking" to ensure that the web revision history tooling for git repos is equivalent to the existing SVN tooling?  If not, can it be?

The blame functionality you were missing has an equivalent available now.  See here for an example of blame on the root OWNERS file in chrome/trunk/src.  This capability has met the blame functionality from the POV of those we've spoken with so far.  Can you take a look at the example and let us know if it meets your view, too?

Just saw this update, nice to see that work is going on this tool. To try it out, I picked a recent example that git users couldn't track down and which we were investigating because of a bug. Let's say I want to look at RenderViewImpl::OnMessageReceived, and the "GetContentClient()->SetActiveURL(main_frame->document().url());" line near the top and see the cl which added it. Currently using ViewVC my steps are:

1) go to src.chromium.org/viewvc/, find the file

2) 2s later http://src.chromium.org/viewvc/chrome/trunk/src/content/renderer/render_view_impl.cc?revision=254479 loads, click on "show annotations"

3) 20s later the page loads, I see that line with a link http://src.chromium.org/viewvc/chrome/trunk/src/content/renderer/render_view_impl.cc?r1=163060&r2=163061&. It's a cl from me moving code into a namespace. However the link is helpful in that it has a side-by-side diff and the left side has a link to the previous revision before this change, so I click on that link.

4) I go through the above line 4 more times because that line changed by a bunch of people. The total network load time is 35s. There's a bug in viewvc when the change is before a file rename, since it tries to generate a url with the old filename and that fails. However that bug is easy to workaround by correcting the filename in the url, which I know since it's what I started with. I finally get to the original cl that added this: http://src.chromium.org/viewvc/chrome?revision=11337&view=revision

This process was more convoluted than normal, because that line had 5-6 changes to it. However even then, it takes me about a minute or two to do this tracking.

I tried to do this with the git tool.

1) I click "blame" on https://chromium.googlesource.com/chromium/src/+/master/content/renderer/render_view_impl.cc

2) 108s later, I look for SetActiveURL and see that first change from me. It has a link on the left to https://chromium.googlesource.com/chromium/src/+blame/a80af12efcba198f4beda21c1d88c63f04c4dc41/content/renderer/render_view_impl.cc

3) 60s later, it takes me to https://chromium.googlesource.com/chromium/src/+blame/a80af12efcba198f4beda21c1d88c63f04c4dc41/content/renderer/render_view_impl.cc, i.e. the same link where I came from. At this point, I can't do anything.

I've had the privilege of watching in wonder as John navigates his blame magic. :)

I've frequently wanted to do this with git too, but had previously just, uh, given up. I was actually about to start writing a tool to do this, when I finally discovered that it already exists in the standard tools. As it took me a while to find, I thought I'd mention it here in case other people aren't aware of it. For the example above:

Run:

1. git gui blame content/renderer/render_view_impl.cc

2. Type "/SetActiveURL"   (to find the thing we care about)

3. Click on the line you care about to see the CL description

4. And the Magic Hidden Functionality... Right click, and choose "Blame Parent Commit"

5. Repeat #4, eventually getting to f8b6b6fb... === http://src.chromium.org/viewvc/chrome?revision=11337&view=revision

Of course, it's not web-based so no links to diffs, and you need a local clone, etc, but it seems to work pretty well. It does a good job of keeping the relevant line "in focus" so you don't have to keep searching for it, or scrolling around.

It does sometimes take a while to load all the annotations, but you don't have to wait for it to finish, you can jump to the next parent back if you see it's not in the form you're looking for yet. Most hops were < 5s in this case, but one stalled for a long time, maybe ~90s.

 

So it seems that the tool needs more work to find changes as it could only go one step backwards. Having it match viewvc to go back multiple changes is needed as this information is often crucial in figuring out why a piece of code is the way it is when refactoring.

Hopefully these steps are useful to the team working on this tool in figuring out how to get this sort of investigation to work.

Chase

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Tomasz Jamroszczak]

On Wed, 12 Mar 2014 06:39:01 +0100, Scott Graham <sco...@chromium.org>

wrote:

> On Mon, Mar 3, 2014 at 1:59 PM, John Abd-El-Malek <j...@chromium.org>
> wrote:
>> I tried to do this with the git tool.
>> 1) I click "blame" on
>> https://chromium.googlesource.com/chromium/src/+/master/content/renderer/render_view_impl.cc
>> 2) 108s later, I look for SetActiveURL and see that first change from
>> me.
>> It has a link on the left to
>> https://chromium.googlesource.com/chromium/src/+blame/a80af12efcba198f4beda21c1d88c63f04c4dc41/content/renderer/render_view_impl.cc
>> 3) 60s later, it takes me to
>> https://chromium.googlesource.com/chromium/src/+blame/a80af12efcba198f4beda21c1d88c63f04c4dc41/content/renderer/render_view_impl.cc,
>> i.e. the same link where I came from. At this point, I can't do
>> anything.
>>
>>
> I've had the privilege of watching in wonder as John navigates his blame
> magic. :)
>

> Run:
>
> 1. git gui blame content/renderer/render_view_impl.cc
> 2. Type "/SetActiveURL" (to find the thing we care about)
> 3. Click on the line you care about to see the CL description
> 4. And the Magic Hidden Functionality... Right click, and choose "Blame
> Parent Commit"
> 5. Repeat #4, eventually getting to f8b6b6fb... ===
> http://src.chromium.org/viewvc/chrome?revision=11337&view=revision

FWIW, there's a vim plugin which speeds up git blaming:
https://github.com/tpope/vim-fugitive.

--
Best Regards
Tomasz Jamroszczak

[Bernhard Bauer]

FTR, appending a ^ after the revision in the blame URL seems to work: https://chromium.googlesource.com/chromium/src/+blame/a80af12efcba198f4beda21c1d88c63f04c4dc41%5E/content/renderer/render_view_impl.cc

Bernhard.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[Avi Drissman]

On Wed, Mar 12, 2014 at 4:47 AM, Tomasz Jamroszczak <tjamro...@opera.com> wrote:

1. git gui blame content/renderer/render_view_impl.cc

git gui isn't a standard tool.

People are missing the point. The SVN tools on the web are at least functional. And people are missing John's other key complaint:

On Mon, Mar 3, 2014 at 4:59 PM, John Abd-El-Malek <j...@chromium.org> wrote:

I tried to do this with the git tool.

1) I click "blame" on https://chromium.googlesource.com/chromium/src/+/master/content/renderer/render_view_impl.cc

2) 108s later, I look for SetActiveURL [...]

I repeated this, getting 79s.

When page load times aren't measured in milliseconds, aren't measured in seconds, but are measured in minutes, something is wrong.

Avi

[Michael Moss]

Agreed, but in fairness, some comparison timings on that file:

viewvc annotate: 90s

gitiles blame: 50s (probably faster due caching of your requests, but even taking the 108s worst case, it's in the same ballpark)

I've done similar tests on heavily changed files before, and gitiles has always been comparable if not faster. Another example is 13s for src/DEPS on gitiles vs. 40s for viewvc. And don't get me started on loading the history of a heavily changed file or dir in viewvc.

Avi