Subversion assertion upon commit

[Don Newbold (GSC)]

I've received notice of an assertion upon committing some files.

The assertion was via a dialog box, which is shown in the attached
image. The text of the dialog box is as follows - the Ctrl-C



---------------------------
Subversion Exception!
---------------------------
Subversion encountered a serious problem.
Please take the time to report this on the Subversion mailing list
with as much information as possible about what
you were trying to do.
But please first search the mailing list archives for the error message
to avoid reporting the same problem repeatedly.
You can find the mailing list archives at
https://subversion.apache.org/mailing-lists.html

Subversion reported the following
(you can copy the content of this dialog
to the clipboard using Ctrl-C):

In file

'D:\Development\SVN\Releases\TortoiseSVN-1.14.0\ext\subversion\subversion\libsvn_subr\subst.c'
line 724: assertion failed (STRING_IS_EOL(newline_buf, newline_len))
---------------------------
OK
---------------------------



WHAT I WAS DOING:

I was committing several files spread across several directories. This
included .doc files, .docx files and a .txt files. The dialog box
appeared while the .txt file was being processed (based on commit
failure status text). I have many, many times over the years seen
similar commit request fail because of newline inconsistencies with the
.txt file. Upon using dos2unix on the .txt file, the commit normally
goes through to successful completion. I applied dos2unix this time and
the commit completed successfully.



CLIENT HOST:

Edition Windows 10 Pro (64-bit)
Version 21H2
Installed on ‎8/‎2/‎2021
OS build 19044.1889
Experience Windows Feature Experience Pack 120.2212.4180.0



CLIENT SOFTWARE

TortoiseSVN 1.14.0, Build 28885 - 64 Bit , 2020/05/24 13:32:45
ipv6 enabled
Subversion 1.14.0, -release
apr 1.6.5
apr-util 1.6.1
serf 1.3.9
OpenSSL 1.1.1g 21 Apr 2020
zlib 1.2.11
SQLite 3.29.0



SERVER HOST:

Edition Windows 10 Pro (64-bit)
Version 21H2
Installed on ‎8/16/‎2022
OS build 19044.2251
Experience Windows Feature Experience Pack 120.2212.4180.0



SERVER SOFTWARE:

VisualSVN Server Version 5.0.2
Subversion 1.14.2
Apache 2.4.53




--
Don Newbold
256-880-8787, x110
General Standards

[Nathan Hartman]

Hi Don,

Thank you for the detailed error report.

Since you've experienced this error before with .TXT files and the
issue appears somehow related to line endings, do you by chance have
any svn properties set on the file? In particular, does it have
"svn:eol-style", "svn:mime-type", or "svn:keywords" properties? If so,
what are these property values? Do you have any other properties set
which may seem significant to this issue?

To check for property values:

If you are using TortoiseSVN, you should be able to right-click the
file in Explorer to get the context menu and then TortoiseSVN ->
Properties.

Alternatively if you have the command line "svn" client, you could
check whether the file has properties with:

svn proplist FILENAME.TXT

and if that shows anything, show the property values with:

svn propget svn:eol-style FILENAME.TXT
svn propget svn:mime-type FILENAME.TXT

etc.

Cheers,
Nathan

[Don Newbold (GSC)]

Nathan,

To be clear, I have never seen any SVN related ASSERTION on any other
occasion.

The file in question is named "release.txt" and contains release notes
for a driver. I've got over 50 different such files and collectively
have made perhaps 500 different commits of these files. I see the commit
failure on the file's parent documentation subdirectory often enough
that I typically go to dos2unit on this file without checking to see
what the complaint is.

The property are ...

EOL: native
MIME: test/plain
Keywords: author, date, ID, revision, URL
needs lock: *
No other properties are set.

The file is virtually always edited under Windows using Visual Studio
2005. This inserts Windows style line endings. However, the file is used
under Linux and is run through dos2unix as part of the release
procedure. To clarify, the file is maintained with UNIX style line
endings, but gets Windows style line endings for newly added lines. The
SVN properties are set automatically based on it being a .txt file.
Every release includes one or more other .txt files, with identical SVN
properties. I'm sure I see this commit failure periodically with other
.txt files, but they aren't changed enough for me to have developed an
automatic response when a commit fails.

Feel free to contact me for any additional information.

Don

[Nathan Hartman]

Ah, I see. Thanks for the clarification.

There are two separate items here.

(1) Subversion complains when you try to commit a file that (a) has the
svn:eol-style property and (b) contains a mixture of newline styles.
This is actually *correct* behavior: Since Subversion is going to
modify the file as it passes from your working copy to the repository,
to normalize all line endings to LF style, it must first make sure that
the file does not contain a mixture, since that may indicate a user
error. Consider, for example, the possibility of accidentally setting
this property on a binary file. In such a file, the CR and LF
characters (decimal 13 and 10 respectively) could be not line endings
at all, but rather part of the binary data. Changing these during a
commit operation could irrecoverably corrupt the data. As a safety
measure, Subversion won't do it unless all lines have the same line
ending style.

(2) The assertion failure. This definitely indicates a bug somewhere.
The tricky part, of course, is to find the cause, which is made much
more difficult when there isn't a reproduction script that can expose
the issue for single-step debugging, or at least a stack trace. What I
do know is that the function in question, translate_newline(), which is
part of the logic that handles line ending and keyword translation and
de-translation, both of which the file in question has, is somehow
receiving 'newline_buf' contents that do not match one of the allowed
values "\n", "\r", or "\r\n". The data in this 2-character buffer is
coming from the file being (de-)translated, so it seems that somehow,
some sequence of '\n', '\r', and/or '$' (for keywords) that was present
in the file before you ran dos2unix confused things. It seems that
whatever happened is extremely rare, since this is the only report of
this assertion failure that I can find anywhere, except for one that
occurred in testing with an intermediate version of this logic during
its development a really long time ago [1]. I could go ahead and file
this as a bug in the issue tracker in case others encounter this
assertion failure. That could help to gather more data about it if
other users encounter it.

Meanwhile, a request: If you (or anyone) sees this again, please save a
copy of the file that exposes it and then talk to us so that with your
help we could figure out what is the offending sequence and how to
reproduce it for debugging and regression testing...

[1] https://lists.apache.org/thread/grcn9pvn1m1706j3s8znp4zqox5tlhfc

Thanks,
Nathan

[Daniel Sahlberg]

[...]

Meanwhile, a request: If you (or anyone) sees this again, please save a
copy of the file that exposes it and then talk to us so that with your
help we could figure out what is the offending sequence and how to
reproduce it for debugging and regression testing...

The mentioning of Windows and Linux above makes me wonder if you are using the same working copy from both Windows and Linux (WSL/Cygwin) environments. It was discussed on the TortoiseSVN mailing list a while ago [1] but I don't think anyone actually encountered an assertion and I'm not able to reproduce it either. (All my experiments ended with "E135000: Inconsistent line ending style"). The conclusion was to maintain separate working copies for the different environments.

If you use svn:eol-style native you should be able to checkout the file under Linux (including WSL/Cygwin) and automatically have \n line endings without any need for conversion. When you checkout the file under Windows (using TSVN or the svn command line tools) you will get \r\n line endings.

Kind regards,

Daniel

[1] https://groups.google.com/g/tortoisesvn/c/6g9nucDEtlA/m/no_3CEvbAAAJ

[Nathan Hartman]

On Thu, Dec 15, 2022 at 7:48 AM Don Newbold (GSC)
<d...@generalstandards.com> wrote:
>
> Nathan,
>
> Understood. If and when I see the assertion again I'll retain a copy of
> the file and send that along with the reporting email.
>
> Don

Hi Don,

Please note that submitting a file here would make it public, so
please take that into consideration!!

(If a file needs to remain proprietary, we can work with you to find a
way to get just the information needed without exposing the file's
contents...)

Thanks!
Nathan

[Don Newbold (GSC)]

Nathan,

Understood. If and when I see the assertion again I'll retain a copy of
the file and send that along with the reporting email.

Don

[Don Newbold (GSC)]

Daniel,

All of the subversion tools are used under Windows. No subversion tools
are even installed under any Linux installations.

The source code is used under Linux, but rarely edited under Linux. I
regularly use Cygwin tools (dos2unix.exe, for example), but rarely edit
files there.

You did make a minor assumption I'll correct. Files are rarely
explicitly locked. In 20 years I am the only individual here who has
ever touched these sources. Thus, I haven't locked any of the sources
before editing in over 10 years. Whatever locking that occurs, if any,
is done so magically by the tools.

Again, though, if I ever see an assertion again I'll retain the
offending file for submission with the error report.

Don

On 12/15/2022 2:05 AM, Daniel Sahlberg wrote:
> Den tors 15 dec. 2022 kl 06:04 skrev Nathan Hartman

> <hartman...@gmail.com <mailto:hartman...@gmail.com>>:

> <https://groups.google.com/g/tortoisesvn/c/6g9nucDEtlA/m/no_3CEvbAAAJ>

[Don Newbold (GSC)]

Noted.