Bug in "Check for modifications" and "Commit": Files that were moved and modified are falsely shown as "normal (+)".

[Tobias Knauss]

ok, so here are a couple of bugs.

I am currently checking the result of a merge by checking the modifications. There are hundrets of modified files, dozens of added and deleted, and most are still "normal".

1) (critical!) I found that some files, that have been moved AND modified, are shown as "normal (+)". This is wrong, they must be shown as "modified (+)". I am checking all modified files, but if files are falsely shown as normal, then I don't check them.

Also, when I open them, the path of the file in the current working copy is given to both sides of the external comparison tool (Beyond Compare 4), which then says "no differences". This at least is consistent with the wrong status, but it means you need to fix both.

2) I did not know the meaning of "+", so I eventually looked it up and via stackoverflow I found https://tortoisesvn.net/docs/release/TortoiseSVN_en/tsvn-dug-wcstatus.html. This page says, " Purple: Added items. Items which have been added with history have a + sign in the Text status column, and a tooltip shows where the item was copied from."

First, + can also appear at normal.

Second, the description of that + is quite hidden. You could put it in a separate paragraph, and use the term "(+)", since that is easier to find and actually what is written in the status text.

Third, the mentioned tooltip is shown at wrong positions. Take the "check for modifications" dialog, select a file and then hold the mouse over the status of another file. The tooltip will be shown at the selected file, which is wrong, because it doesn't belong there. Also, the tooltip is shown over the line of the file, it should be shown underneath. If you scroll a bit, then the tooltip may even be shown outside the window, or outside the screen.

One last thing: You may want to add a link back to your TortoiseSVN product main page in the document pages.

I know that I did not buy the software (I will ask my boss to donate some money since we heavily depend on your software; I think he'll ask for an invoice). Therefore please apologize for my direct words, but that's just my language as a German Engineer. ;-)

So at least I have to say a big THANK YOU for all the work you do for us

[Tobias Knauss]

TortoiseSVN 1.14.99, Build 29032 - 64 Bit -dev, 2020/12/08 18:08:20
ipv6 enabled
Subversion 1.14.0, -release
apr 1.6.5
apr-util 1.6.1
serf 1.3.9
OpenSSL 1.1.1h  22 Sep 2020
zlib 1.2.11

SQLite 3.32.3

The latest nightly build. Downloaded and installed an hour ago.

[Tobias Knauss]

The status seems to be wrong also on added items.

Example:

In the branch, I have created a folder with some files.

After merging to the trunk, the folder is shown as "added (+)", but the files are shown as "normal (+)". The files did not exist in the trunk before, so how can their status be "normal"? It should be "added" as well.

[Stefan]

On Wednesday, December 9, 2020 at 12:32:39 PM UTC+1 Tobias Knauss wrote:

ok, so here are a couple of bugs.

I am currently checking the result of a merge by checking the modifications. There are hundrets of modified files, dozens of added and deleted, and most are still "normal".

1) (critical!) I found that some files, that have been moved AND modified, are shown as "normal (+)". This is wrong, they must be shown as "modified (+)". I am checking all modified files, but if files are falsely shown as normal, then I don't check them.

Also, when I open them, the path of the file in the current working copy is given to both sides of the external comparison tool (Beyond Compare 4), which then says "no differences". This at least is consistent with the wrong status, but it means you need to fix both.

if your compare-tool doesn't show any differences, then there really are no differences.

Also, the only way I can get a "normal (+)" status is by moving the parent folder and then merge that. And in that case, the files really haven't been modified.

Also: if you right-click on the list control header, you can select more columns, for example "text status" and "property status". Sometimes those can show more info.

Apart from that: I can't really show a "wrong" status here: if the svn lib reports the status as "normal", TSVN has to show exactly that. What I will do is change the color of an item with history, i.e. if the "(+)" is shown.

 

2) I did not know the meaning of "+", so I eventually looked it up and via stackoverflow I found https://tortoisesvn.net/docs/release/TortoiseSVN_en/tsvn-dug-wcstatus.html. This page says, " Purple: Added items. Items which have been added with history have a + sign in the Text status column, and a tooltip shows where the item was copied from."

First, + can also appear at normal.

Second, the description of that + is quite hidden. You could put it in a separate paragraph, and use the term "(+)", since that is easier to find and actually what is written in the status text.

I'll update the docs a little bit.

 

Third, the mentioned tooltip is shown at wrong positions. Take the "check for modifications" dialog, select a file and then hold the mouse over the status of another file. The tooltip will be shown at the selected file, which is wrong, because it doesn't belong there. Also, the tooltip is shown over the line of the file, it should be shown underneath. If you scroll a bit, then the tooltip may even be shown outside the window, or outside the screen.

the position of the tooltips is determined by the list control itself, so nothing I can do here, sorry.

 

One last thing: You may want to add a link back to your TortoiseSVN product main page in the document pages.

Will do.

 

I know that I did not buy the software (I will ask my boss to donate some money since we heavily depend on your software; I think he'll ask for an invoice). Therefore please apologize for my direct words, but that's just my language as a German Engineer. ;-)

So at least I have to say a big THANK YOU for all the work you do for us

you're welcome :)

 

[Tobias Knauss]

Stefan schrieb am Mittwoch, 9. Dezember 2020 um 19:02:52 UTC+1:

On Wednesday, December 9, 2020 at 12:32:39 PM UTC+1 Tobias Knauss wrote:

ok, so here are a couple of bugs.

I am currently checking the result of a merge by checking the modifications. There are hundrets of modified files, dozens of added and deleted, and most are still "normal".

1) (critical!) I found that some files, that have been moved AND modified, are shown as "normal (+)". This is wrong, they must be shown as "modified (+)". I am checking all modified files, but if files are falsely shown as normal, then I don't check them.

Also, when I open them, the path of the file in the current working copy is given to both sides of the external comparison tool (Beyond Compare 4), which then says "no differences". This at least is consistent with the wrong status, but it means you need to fix both.

if your compare-tool doesn't show any differences, then there really are no differences.

Also, the only way I can get a "normal (+)" status is by moving the parent folder and then merge that. And in that case, the files really haven't been modified.

Also: if you right-click on the list control header, you can select more columns, for example "text status" and "property status". Sometimes those can show more info.

Apart from that: I can't really show a "wrong" status here: if the svn lib reports the status as "normal", TSVN has to show exactly that. What I will do is change the color of an item with history, i.e. if the "(+)" is shown.

 

 

There ARE differences if I compare the file from the original file (the one to be deleted) and the added and modified file. I only don't see differences if I start the file compare by double-clicking on the line with the added file, because from this line, the same path and filename is sent by TSVN as left and right file in the comparison.

You should be able to reproduce it by changing the parent folder of a file AND modifying it (don't know whether the order of operations is relevant).

Since you say that you just forward the status, it looks like the failure is in SVN library then. This really is a problem when reviewing the merge. I had hoped for a quick solution until tomorrow, but that has become unlikely.

How can I give you more details about the issue? As suggested in the other issue, you may watch over teamviewer (contact me by email, knauss at schmoll-maschinen dot de).

Third, the mentioned tooltip is shown at wrong positions. Take the "check for modifications" dialog, select a file and then hold the mouse over the status of another file. The tooltip will be shown at the selected file, which is wrong, because it doesn't belong there. Also, the tooltip is shown over the line of the file, it should be shown underneath. If you scroll a bit, then the tooltip may even be shown outside the window, or outside the screen.

the position of the tooltips is determined by the list control itself, so nothing I can do here, sorry.

This is annoying. You might want to use another list control, or just evaluate the mouse-hover event on that list control and calculate the correct position yourself.

[Tobias Knauss]

Tobias Knauss schrieb am Mittwoch, 9. Dezember 2020 um 22:45:43 UTC+1:

Stefan schrieb am Mittwoch, 9. Dezember 2020 um 19:02:52 UTC+1:

On Wednesday, December 9, 2020 at 12:32:39 PM UTC+1 Tobias Knauss wrote:

ok, so here are a couple of bugs.

I am currently checking the result of a merge by checking the modifications. There are hundrets of modified files, dozens of added and deleted, and most are still "normal".

1) (critical!) I found that some files, that have been moved AND modified, are shown as "normal (+)". This is wrong, they must be shown as "modified (+)". I am checking all modified files, but if files are falsely shown as normal, then I don't check them.

Also, when I open them, the path of the file in the current working copy is given to both sides of the external comparison tool (Beyond Compare 4), which then says "no differences". This at least is consistent with the wrong status, but it means you need to fix both.

if your compare-tool doesn't show any differences, then there really are no differences.

Also, the only way I can get a "normal (+)" status is by moving the parent folder and then merge that. And in that case, the files really haven't been modified.

Also: if you right-click on the list control header, you can select more columns, for example "text status" and "property status". Sometimes those can show more info.

Apart from that: I can't really show a "wrong" status here: if the svn lib reports the status as "normal", TSVN has to show exactly that. What I will do is change the color of an item with history, i.e. if the "(+)" is shown.

 

 

There ARE differences if I compare the file from the original file (the one to be deleted) and the added and modified file. I only don't see differences if I start the file compare by double-clicking on the line with the added file, because from this line, the same path and filename is sent by TSVN as left and right file in the comparison.

You should be able to reproduce it by changing the parent folder of a file AND modifying it (don't know whether the order of operations is relevant).

Since you say that you just forward the status, it looks like the failure is in SVN library then. This really is a problem when reviewing the merge. I had hoped for a quick solution until tomorrow, but that has become unlikely.

How can I give you more details about the issue? As suggested in the other issue, you may watch over teamviewer (contact me by email, knauss at schmoll-maschinen dot de).

See this image for clarification:

Also, I was able to reproduce it in a test project. It's quite easy.

And, the situation is even worse: The log window does not show the modified file in the respective revision, even if I choose to open the log directly on that file.

Example:

- Top Left screenshot shows the current content of the text file. The timestamp at the end of line 4 was just merged from branch b01, and the file was moved in that branch to a subfolder.

- Top Right screenshot show the properties of that file.

- Screenshot in 2nd row shows the log window. The "doc1.txt is missing in rev.63, don't know why, since it was modified.

- Screenshot in 3rd row shows the file content in rev.63 (which is HEAD) when opened from repo browser. It equals with the file on disk, everything correct. However, the file revision is displayed as 62, which is wrong.

- Screenshot in last row shows the file content in rev.62 when opened from repo browser. You can see that line 4 is different. So it has changed in rev.63. However, the file revision is displayed as 60, which is also wrong.

I am beginning to think that the problem comes from the SVN server software.

I am using CollabNet Subversion Edge, version info from the web interface is:

Software version 5.2.4-4429.59
Subversion version 1.8.19-4429.59

I see that the subversion version is pretty old, and I actually wonder why, since I have updated to the latest SVN Edge version in February of this year. Also, 5.2.4 is the latest version of SVN Edge online available. Don't they update the internal subversion version?

I don't find any information about it. Their release notes stop at 5.0.

The server itself is running fine and reliable, and up to now was working without problems. But it looks like CollabNet is no longer interested in supporting SVN Edge.

[Tobias Knauss]

This failure even affects locking of files. I was going to lock the entire trunk of the respective repository to avoid that someone interrupts my series of partial commits.

A merged file that was modified and moved in the branch is now shown as "normal (+)", so TSVN tries to lock this file, but as it does not yet exist in the repo, the locking fails.

Other files, that are still in the repo, but moved / deleted in the working copy and not  yet committed, are not part of the locking list. I think they should be locked, too. And as a part of the commit, their locks must be released before they are moved / deleted.

[Tobias Knauss]

I was able to reproduce everything in a test repo that I created locally on my computer with TSVN.

This means, the server is not affected. The problem must either come from TSVN or the SVN library it uses.

The test repo is attached.

To reproduce: Merge branches/b01 into trunk, then try to commit or acquire a lock.

[Stefan]

with your test repo, I do the merge from branches/b01 into my wc from trunk.

Then committing works fine.

Getting a lock is obviously not possible because the only file is moved but not committed yet. So you can't get a lock without committing the move first.

And yes: in the commit dialog the file is shown as "normal (+)", but that's what the svn lib reports. You can try it yourself by running

svn st -v

and you'll get:

 M 4 2 toknauss .
D 4 2 toknauss doc01.txt
A + - 4 toknauss folder_a
+ - 4 toknauss folder_a\doc01.txt

[Tobias Knauss]

Which solution do you suggest for locking a repo that has such changes?

Should I use the repo browser and lock the unchanged head revision in it?

Would it make sense to automatically adapt the chosen files in the background (exclude local new files from the locking list, add local deleted ones), so that the locking operation achieves the expected result?

Okay, if the svn lib reports "normal (+)", then you can do nothing about it but forward it to the user.

But isn't this a bug in the svn lib then? In my opinion, "normal (+)" is wrong, it should be "added (+)" as mentioned above, also because your Commit dialog is not able to link the added and the deleted file and open a comparison with both.

What do you think about it?

[Stefan]

On Friday, December 11, 2020 at 5:32:55 PM UTC+1 Tobias Knauss wrote:

Which solution do you suggest for locking a repo that has such changes?

Should I use the repo browser and lock the unchanged head revision in it?

Would it make sense to automatically adapt the chosen files in the background (exclude local new files from the locking list, add local deleted ones), so that the locking operation achieves the expected result?

you don't have to lock that file:

the file is at a new location locally, so all other users don't have that file (yet). Only once you commit the merge and the moved folder/file, *then* you can lock the file because then it has a 'connection' to the repository.

 

Okay, if the svn lib reports "normal (+)", then you can do nothing about it but forward it to the user.

But isn't this a bug in the svn lib then? In my opinion, "normal (+)" is wrong, it should be "added (+)" as mentioned above, also because your Commit dialog is not able to link the added and the deleted file and open a comparison with both.

What do you think about it?

I'm working on a workaround the svn lib problem here when doing a diff-against-base. However the status will still be reported as normal.

And "added" would be wrong too: the file in relative to it's parent folder wasn't added at all but stayed the same.

 

[Stefan]

ok, just spent two hours trying to find a way to show a proper diff for such a "normal (+)" file, but then I realized it's simply not possible.

I try to explain:

Subversion has very weak support for renames/moves. While it *does* record a move, it does so only in the revision the move occurred. Now, when you do a merge, that move information is lost because a merge is basically applying a diff to a working copy. So when you merge, the moved folder is merged as "added" with the copy-from info set to the branch that's merged in, and the old folder is simply deleted.

Which means the information that the folder got renamed is lost in the merge.

so basically there's nothing I can do here.

Sorry.

[Tobias Knauss]

Thank you for your effort. I am sorry to hear that it didn't help.

> you don't have to lock that file:

> the file is at a new location locally, so all other users don't have that file (yet). Only once you commit the merge and the moved folder/file, *then* you can lock the file because then it has a 'connection' to the repository.

Yes, the added file is not yet in the repo, so of course it cannot be locked. But it is autotically maselected in the lock dialog, which should not be the case.

And the local deleted files are still part of the repo, but they are not locked, because they don't appear in the lock dialog:

> Other files, that are still in the repo, but moved / deleted in the working copy and not  yet committed, are not part of the locking list.   (me, 10.12.2020, 12:25:34)

This would create an incomplete lock on the folder.

> Subversion has very weak support for renames/moves. While it *does* record a move, it does so only in the revision the move occurred. Now, when you do a merge, that move information is lost because a merge is basically applying a diff to a working copy. So when you merge, the moved folder is merged as "added" with the copy-from info set to the branch that's merged in, and the old folder is simply deleted.

I think we're talking about files, not folders, but that should be the same.

And you said >the moved folder is merged as "added"<, but the moved file is not shown as "added", but as "normal", as shown in your test:

> M 4 2 toknauss .
> D 4 2 toknauss doc01.txt
> A + - 4 toknauss folder_a
> + - 4 toknauss folder_a\doc01.txt

Regarding the file, this combination of actions is just wrong in my opinion: The original file was deleted. Okay. But the file exists at a new location after the commit, so if it got *deleted* at the original place, it must be *added* at another place (with or without reference to the old one). Otherwise it does not explain that the file still exists. If it was *deleted* only, then the file should have been gone and not exist anymore.

> I'm working on a workaround the svn lib problem here when doing a diff-against-base. However the status will still be reported as normal.

> And "added" would be wrong too: the file in relative to it's parent folder wasn't added at all but stayed the same.

"Added" would not be wrong, because the file *was added* in the new parent folder.

If it had *not* been "added", then the consequence must be that there was no *deleted* file either. But there *is* a delete file, so there also must be an *added* file.

I am thinking about creating a bug report for the SVN lib, but I don't have any insight to the lib itself, since I don't use it.

> so basically there's nothing I can do here.

You can. I can contribute to a bug report, but I think you're the one who needs to create it. Please understand that it's not about offloading work to you, but if I was the one who starts a conversation with the developers of the SVN lib, then it basically is a ping-pong with 3 people, and I am in the middle, although I have the least participation in the whole process.

I will assist you in convincing the SVN lib developers if necessary.

> Subversion has very weak support for renames/moves

Maybe the developers need to improve that support in order to achieve a correct behaviour.

[Stefan]

On Saturday, December 12, 2020 at 10:02:37 AM UTC+1 Tobias Knauss wrote:

> you don't have to lock that file:

> the file is at a new location locally, so all other users don't have that file (yet). Only once you commit the merge and the moved folder/file, *then* you can lock the file because then it has a 'connection' to the repository.

Yes, the added file is not yet in the repo, so of course it cannot be locked. But it is autotically maselected in the lock dialog, which should not be the case.

the lock dialog does not contact the repository, so it's not possible for the lock dialog to know that the file isn't present (yet) in the repository.

 

> I'm working on a workaround the svn lib problem here when doing a diff-against-base. However the status will still be reported as normal.

> And "added" would be wrong too: the file in relative to it's parent folder wasn't added at all but stayed the same.

"Added" would not be wrong, because the file *was added* in the new parent folder.

If it had *not* been "added", then the consequence must be that there was no *deleted* file either. But there *is* a delete file, so there also must be an *added* file.

but since the move information is gone, how would I know that the added and the deleted item were the same before? They could be completely unrelated.

 

I am thinking about creating a bug report for the SVN lib, but I don't have any insight to the lib itself, since I don't use it.

> so basically there's nothing I can do here.

You can. I can contribute to a bug report, but I think you're the one who needs to create it. Please understand that it's not about offloading work to you, but if I was the one who starts a conversation with the developers of the SVN lib, then it basically is a ping-pong with 3 people, and I am in the middle, although I have the least participation in the whole process.

I will assist you in convincing the SVN lib developers if necessary.

good luck with that. You can search the svn dev mailing list for "move handling" and the like and you'll find tons of discussions. Handling renames is done the best way possible. Other implementations would lead to massive performance issues on the server side.

[Tobias Knauss]

Stefan schrieb am Samstag, 12. Dezember 2020 um 11:00:01 UTC+1:

On Saturday, December 12, 2020 at 10:02:37 AM UTC+1 Tobias Knauss wrote:

> you don't have to lock that file:

> the file is at a new location locally, so all other users don't have that file (yet). Only once you commit the merge and the moved folder/file, *then* you can lock the file because then it has a 'connection' to the repository.

Yes, the added file is not yet in the repo, so of course it cannot be locked. But it is autotically maselected in the lock dialog, which should not be the case.

the lock dialog does not contact the repository, so it's not possible for the lock dialog to know that the file isn't present (yet) in the repository.

 

Hm, yes, makes sense.

> I'm working on a workaround the svn lib problem here when doing a diff-against-base. However the status will still be reported as normal.

> And "added" would be wrong too: the file in relative to it's parent folder wasn't added at all but stayed the same.

"Added" would not be wrong, because the file *was added* in the new parent folder.

If it had *not* been "added", then the consequence must be that there was no *deleted* file either. But there *is* a delete file, so there also must be an *added* file.

but since the move information is gone, how would I know that the added and the deleted item were the same before? They could be completely unrelated.

 

I didn't mean that it was the fault of TSVN. It's the SVN lib again.

I am thinking about creating a bug report for the SVN lib, but I don't have any insight to the lib itself, since I don't use it.

> so basically there's nothing I can do here.

You can. I can contribute to a bug report, but I think you're the one who needs to create it. Please understand that it's not about offloading work to you, but if I was the one who starts a conversation with the developers of the SVN lib, then it basically is a ping-pong with 3 people, and I am in the middle, although I have the least participation in the whole process.

I will assist you in convincing the SVN lib developers if necessary.

good luck with that. You can search the svn dev mailing list for "move handling" and the like and you'll find tons of discussions. Handling renames is done the best way possible. Other implementations would lead to massive performance issues on the server side.

I wll have a look there. I think changing only the reported status should be possible; I can hardly imagine that it breaks the whole move handling.

Thank you for your support on this issue!

Best