Re: Mixed line endings problem

[Steve deRosier]

This isn't a problem with BBEdit, it's a problem with your process.
Fix the process. You should be using consistent line endings.

Frankly I'm surprised that the various editors ignore the line
endings. DOS endings are \r\n, UNIX is \n. BBEdit, if it doesn't
autodetect the DOS endings, it should only show a newline with the \r
set as an invisible at the end of the line, not a second newline
(IIRC, I could be wrong, I usually don't have to work with mixed line
ends). More likely: inconsistant line ends and improper editor and RCS
settings are causing duplicated new-lines and so on in your files. If
this is happening, it will propagate and get even worse.

Fix the process before it gets worse:
1. Choose a line-ending that everyone will use. Document it. Train new
guys on it.
2. Configure your editors for that ending by default. If you have any
editors in the mix that can't be configured, drop that one: it's not
worth using.
3. Configure your RCS to default to the ending. Bonus if you set it up
to detect and warn or reject commits that are screwed up.
4. Reformat all and commit all in one commit. There's about 1000
different automated scripts, programs (astyle), or awk/sed one-liners
and such that can do this all in one shot. Google it.
5. Smile because you _fixed_ the problem.

- Steve

On Mon, Dec 10, 2012 at 5:51 PM, Troy Gillette <633...@gmail.com> wrote:
> I work on a large cross-platform project for Mac and Windows. Every editor
> used on the Windows team treats UNIX and Windows style line endings as if it
> is simply a native carriage return. So if any mixed line endings end up in a
> file, it is completely transparent.
>
> Xcode, TextMate and even TextEdit on the Mac do the same thing.
>
> BBEdit on the other hand, looks at the first line of the file and assumes
> ALL other line endings in the file match that one. So, if you open a file
> with a UNIX line ending on the first line, any lines with Windows EOLs will
> end up with a second empty line after them and if the first line is Windows
> style, any UNIX lines will get mushed together into a single line.
>
> The only way I see to fix this is to make the entire file conform to one or
> the other. This can be a real pain, especially if only the first few lines
> are different because it will mess up everything after that (I see no way to
> force bbedit to load one way or the other, it always chooses for me). It
> also results in excessive/unnecessary changes in revision control.
>
> I understand there are times when you would want to be able to know there
> are differing line endings, but the current system makes life difficult for
> those of us who must cooperate with developers outside of the Mac world.
>
> Is there a chance that we could have a preferences option to make BBEdit
> behave like every other developers editor on the market?
> Is there already such an option that I just can't seem to find?
>
> Thank you in advance for any suggestions.
>
> --
> --
> You received this message because you are subscribed to the
> "BBEdit Talk" discussion group on Google Groups.
> To post to this group, send email to bbe...@googlegroups.com
> To unsubscribe from this group, send email to
> bbedit+un...@googlegroups.com
> For more options, visit this group at
> <http://groups.google.com/group/bbedit?hl=en>
> If you have a feature request or would like to report a problem,
> please email "sup...@barebones.com" rather than posting to the group.
> Follow @bbedit on Twitter: <http://www.twitter.com/bbedit>
>
>
>

[Doug McNutt]

At 17:51 -0800 12/10/12, Troy Gillette wrote:
>I work on a large cross-platform project for Mac and Windows. Every editor used on the Windows team treats UNIX and Windows style line endings as if it is simply a native carriage return. So if any mixed line endings end up in a file, it is completely transparent.
>
>Xcode, TextMate and even TextEdit on the Mac do the same thing.
>
>BBEdit on the other hand, looks at the first line of the file and assumes ALL other line endings in the file match that one. So, if you open a file with a UNIX line ending on the first line, any lines with Windows EOLs will end up with a second empty line after them and if the first line is Windows style, any UNIX lines will get mushed together into a single line.
>
>The only way I see to fix this is to make the entire file conform to one or the other. This can be a real pain, especially if only the first few lines are different because it will mess up everything after that (I see no way to force bbedit to load one way or the other, it always chooses for me). It also results in excessive/unnecessary changes in revision control.
>
>I understand there are times when you would want to be able to know there are differing line endings, but the current system makes life difficult for those of us who must cooperate with developers outside of the Mac world.
>
>Is there a chance that we could have a preferences option to make BBEdit behave like every other developers editor on the market?
>Is there already such an option that I just can't seem to find?

I feel your pain.

BBEdit's editing procedure demands that line ends all be the same while displayed in an editing window. They are converted as desired to the output format on a save or when a copy is put onto the system clipboard. Linux' gedit has pretty much the same problems. Nisus for OS 9 is nice but unsupported and the neXt version is too different. Simpletext is a solution of sorts but only on classic machines. VI... . . Naah.

Have a close look at the format of a BBEdit worksheet. It's actually an Apple *.plist file and the guts of the content is a single binary entry that's buried in a cell of the underlying XML. The XML uses 0A line ends and the "binary" data is the worksheet. with its 0D endings.

You cannot edit a bbedit worksheet using the bbedit editor.

I work with strange electronic devices like oscilloscopes and digitally controlled power supplies and other machinery. Some of it actually requires different kinds of line ends but the ends are simply never the same so there is no way to prepare a report in BBEdit that needs to contain samples of code or measured data.

<ftp://macnauchtan.com/Software/LineEnds/> contains some AppleScripts that I think will still work to "repair" files with divers ends. I'm stuck, though, at 10.3.9 because Tiger refused to honor my SE/30 file server and newer Macs won't talk RS232 to my machinery.

God help the world when the two new UTF-16 line ends, soft and hard, that now exist, become popular.

And long live Apple's MPW which can edit its own worksheets and then execute them by name.

And I also want tab stops that work like my Olympia typewriter or an 026 keypunch.
--

Fe++
// \
Fe++ Fe++
| ||
Fe++ Fe++
\\ /
Fe++

[Gregory Shenaut]

It seems to me that a file with mixed line endings should be interpreted such that \r sends the carriage back to first column but doesn't advance to the next line, with \n serving to move down one in the same column: in other words, a file intended for a printer. In that world, only DOS-style \r\n will work as a line-end per se. Or, to generalize this a little, in a file containing both \r and \n characters, those characters should assume their ASCII significance, with \r\n counting as a line ending. In files with \r but not \n, use \r as line-end; in files with \n but not \r, use \n as line-end.

Cheers,
Greg

[François Schiettecatte]

Ok, obviously an issue for you, have you filed a report with Bare Bones Software Technical Support @ sup...@barebones.com ?

François

On Dec 10, 2012, at 11:33 PM, Troy Gillette <633...@gmail.com> wrote:

> Modern text editors should simply ignore CR characters. Most scripting languages do. Mac, Unix and Windows compilers do.
>
> We have several million lines of code from the last ten years. C, C++, Objective-C, Ruby, Python, XML, numerous others... Nothing in our process cares whether anything ends in LF or CR+LF, but BBEdit does.
>
> I understand needing the ability to recognize CR characters. Some people may have code from the last century that uses them as line endings, in that case they should be updated. In all other cases, they should be ignored!
>
> You shouldn't have to maintain a rigid enforcement of line endings across teams, platforms and years of development when there are so few tools that can't deal with it.
>
> Come on BBEdit, let's enter the 21st century.

[Doug McNutt]

At 21:00 -0800 12/10/12, Gregory Shenaut wrote:
>It seems to me that a file with mixed line endings should be interpreted such that \r sends the carriage back to first column but doesn't advance to the next line, with \n serving to move down one in the same column: in other words, a file intended for a printer. In that world, only DOS-style \r\n will work as a line-end per se. Or, to generalize this a little, in a file containing both \r and \n characters, those characters should assume their ASCII significance, with \r\n counting as a line ending. In files with \r but not \n, use \r as line-end; in files with \n but not \r, use \n as line-end.

You're showing your youth!

I can remember a time when those line ends did exactly what you say. The machine is called an ASR33 teletype or, later, a Flexowriter (TM) where you could personalize letters to lots of people..

One could send a CR and retype the same line. That allowed for a version of bold emphasis or a way to make a phi out of an O by adding a / character on the second pass. You could also underline stuff.

An interesting feature was that one needed to send returns and line feeds in the right order for the destination machine and some nulls were needed to allow for the required motions to occur before the next character to be printed.

Other control characters, pretty much ignored today, were used to shift to upper case or to numerics with 6, or even 5, bits per byte.

What was really fun was sending lines to a Control Data line printer where you could have a whole line reprinted as many times as you want before sending a line feed. I still have a calendar for something like 1965 that includes the Playboy centerfolds done that way. The code depended on carefully worked tables of the density of overprinted pairs of characters.

My machines expect RS232 line feeds and returns and do things like that today. BBEdit just cannot be used. Perl is the way to go but don't even think about debugging code with BBEdit as the viewer for the output as well as for the source code being edited..

And. . . Do you know that, in the Apple world \r is supposed to be equivalent to 0A and \n is a 0D? That was observed in the day when MPW was the way to go but it made it into some IETF standards and hasn't been removed even though OS neXt with it's NEXT, Inc origins doesn't follow that standard.

--

--> Halloween == Oct 31 == Dec 25 == Christmas <--

[Gregory Shenaut]

On Dec 11, 2012, at 17:14 , Doug McNutt <doug...@macnauchtan.com> wrote:

> At 21:00 -0800 12/10/12, Gregory Shenaut wrote:
>> It seems to me that a file with mixed line endings should be interpreted such that \r sends the carriage back to first column but doesn't advance to the next line, with \n serving to move down one in the same column: in other words, a file intended for a printer. In that world, only DOS-style \r\n will work as a line-end per se. Or, to generalize this a little, in a file containing both \r and \n characters, those characters should assume their ASCII significance, with \r\n counting as a line ending. In files with \r but not \n, use \r as line-end; in files with \n but not \r, use \n as line-end.
>
> You're showing your youth!

Heh. I thought I was showing my age (I remember those days very well and wrote drivers for plenty of printers back then that used those codes and other control codes).

Greg

[Doug McNutt]

At 09:40 -0500 12/11/12, Fran�ois Schiettecatte wrote:
>Ok, obviously an issue for you, have you filed a report with Bare Bones Software Technical Support @ sup...@barebones.com ?

Perhaps not, but I sure have. It started pretty much with the introduction of worksheets which I really hoped would be a replacement for MPW which Apple killed with OS 10.

BBEdit 6.5 on this Mac 8500 works pretty well. The newer stuff I see as a violation of the bare-bones concept and I expect to be declared persona non grata on this list any time now.

--
--> Marriage and kilo are troubled words. Turmoil results when centuries-old usage is altered in specialized jargon <--.

[G. T. Stresen-Reuter]

On Dec 12, 2012, at 1:20 AM, Doug McNutt wrote:

> The newer stuff I see as a violation of the bare-bones concept and I expect to be declared persona non grata on this list any time now.

I'm hesitant to label anything but I understand your lament of changing things that seemed to work perfectly well for changes that seem unmotivated (or motivated by forces I'm unable to perceive). That said, change seems inevitable so I chose to focus on the positive and look for ways to keep moving forward. Best of luck!

Ted S-R

[John Delacour]

On 11/12/2012 04:33, Troy Gillette wrote:

> We have several million lines of code from the last ten years. C, C++,
> Objective-C, Ruby, Python, XML, numerous others... Nothing in our
> process cares whether anything ends in LF or CR+LF, but BBEdit does.

I think, in contrast to some others, that you have a good point. And the
point is that BBEdit, having determined, by a test of the first line or
so, that the line-endings are one byte then proceeds to convert each
individual \r or \n into the default line-ending. In my case that is \n
and a mixed document such as you are talking of, will end up with \n\n
where there was \r\n and there is no way to distinguish in BBEdit
between what was a DOS line end and what was a blank line.

So the only way to resolve the problem is to regularize the line endings
before BBEdit gets its hands on the doc. The best way to do this, it
strikes me, is to run all the files through a UNIX or Perl filter in one
go, converting every line-end byte or pair to \n and get rid of the
problem at source once and for all. This is very easy to do.

On the other hand I do think it is something that BBEdit should do
automatically, because it makes no sense to convert \r\n to \n\n, as now
can happen, under any circumstances.

JD

[Steve deRosier]

On Fri, Dec 14, 2012 at 5:35 AM, John Delacour <johnde...@gmail.com> wrote:
>
> On the other hand I do think it is something that BBEdit should do
> automatically, because it makes no sense to convert \r\n to \n\n, as now can
> happen, under any circumstances.
>

And this is something it shouldn't do. If it's going to convert the
endings, it _must_ do it correctly. Really, that it silently converts
line endings on opening is a problem. There needs to be some
notification, perhaps a yes/no dialog. That it does it when you switch
the line-end mode in the bottom drop-down is fine; that's the result
of an user-requested action. But actually mucking with the contents of
the file when it opens it (yes I know you still need to save it), in a
way that is silent and non-obvious is bad.

All that said, the real problem is the OP has inconsistant
line-endings in his source files. That's a problem that needs to be
fixed. BBEdit's handling of them is simply a side-show here.

- Steve