Huge paste not pasting all the characters

[hkroger]

Hello,

I have a problem with the basic copy/paste functionality of iTerm2. I
have the beta2 package installed (the version reported is Build
0.20.20110529 though).

Anyway, my problem is that when I copy&paste a huge block of text.
Then I hear the bell sound a lot and the copy paste is missing many
characters.

I wanted to copy a bigger block of awstats.conf like this on a remote
shell:

cat <<EOF >> myfile.txt
# Enter here your log format (Must match your web server config. See
setup
# instructions in documentation to know how to configure your web
server to
# have the required log format).
# Possible values: 1,2,3,4 or "your_own_personalized_log_format"
# 1 - Apache or Lotus Notes/Domino native combined log format (NCSA
combined/XLF/ELF log format)
# 2 - Old IIS log format (IIS W3C log format). See FAQ for IIS 6.x.
# 3 - Webstar native log format.
# 4 - Apache or Squid native common log format (NCSA common/CLF log
format)
# With LogFormat=4, some features (browsers, os, keywords...)
can't work.
# "your_own_personalized_log_format" = If your log is ftp, mail or
other format,
# you must use following keys to define the log format string (See
FAQ
# for ftp, mail or exotic web log format examples):
# %host Host client name or IP address
# %lognamequot Authenticated login/user with format: "john"
# %logname Authenticated login/user with format: john
# %time1 Date and time with format: [dd/mon/yyyy:hh:mm:ss
+0000] or [dd/mon/yyyy:hh:mm:ss]
# %time2 Date and time with format: yyyy-mm-dd hh:mm:ss
# %time3 Date and time with format: Mon dd hh:mm:ss or
Mon dd hh:mm:ss yyyy
# %time4 Date and time with unix timestamp format:
dddddddddd
# %methodurl Method and URL with format: "GET /index.html
HTTP/x.x"
# %methodurlnoprot Method and URL with format: "GET /index.html"
# %method Method with format: GET
# %url URL only with format: /index.html
# %query Query string (used by URLWithQuery option)
# %code Return code status (with format for web log:
999)
# %bytesd Size of document in bytes
# %refererquot Referer page with format: "http://from.com/
from.htm"
# %referer Referer page with format: http://from.com/from.htm
# %uaquot User agent with format: "Mozilla/4.0
(compatible, ...)"
# %ua User agent with format: Mozilla/
4.0_(compatible...)
# %gzipin mod_gzip compression input bytes: In:XXX
# %gzipout mod_gzip compression output bytes & ratio:
Out:YYY:ZZpct.
# %gzipratio mod_gzip compression ratio: ZZpct.
# %deflateratio mod_deflate compression ratio with format: (ZZ)
# %email EMail sender (for mail log)
# %email_r EMail receiver (for mail log)
# %virtualname Web sever virtual hostname. Use this tag when
same log
# contains data of several virtual web servers.
AWStats
# will discard records not in SiteDomain nor
HostAliases
# %cluster If log file is provided from several computers
(merged by
# logresolvemerge.pl), use this to define cluster
id field.
# %extraX Another field that you plan to use for building
a
# personalized report with ExtraSection feature
(See later).
# If your log format has some fields not included in this list, use:
# %other Means another not used field
# %otherquot Means another not used double quoted field
#
# Examples for Apache combined logs (following two examples are
equivalent):
# LogFormat = 1
# LogFormat = "%host %other %logname %time1 %methodurl %code %bytesd
%refererquot %uaquot"
#
LogFormat="%virtualname %host %other %logname %time1 %methodurl %code
%bytesd %refererquot %uaquot"
.
.
.
.
EOF

and I saw many characters missing in the myfile.txt. You can test by
making a huge copy paste block by repeating the awstats configuration
in question on a texteditor and then copying and pasting it to iTerm2.

Out of curiosity I tested the same with the Terminal.app and it works
just fine.

Is there a bug or is it just me?

Hannu

[George Nachman]

I haven't seen this happen before. Can you create a text file with the
stuff you were trying to paste and send it as an attachment? I think
the text above got mangled by going through email. I'll try to
reproduce with that.

Thanks,
G

[Hannu Kröger]

Hello and thanks for a quick response

I have attached a text file which I used for testing. If I copy all of
text from the file and paste in iTerm, it crashes. However, my test
case earlier was that I had an ssh connection to a server which wasn't
local. And there when I pasted, I got those bell sounds and parts of
the text was missing in the file which the copy paste was supposed to
create.

If it means anything, I copied the text from a text editor called
"TextWrangler" which is for free in Mac App Store. The ssh server I
connected to was a Centos 5.6 server and the connection was about
10mbit.

Let me know if you have some further questions.

Cheers,
Hannu

2011/6/6 George Nachman <gnac...@llamas.org>:

[George Nachman]

Odd, it seems to work fine for me. I see that you've reported several
crashes--were they all caused while doing this? I believe it could
happen while ssh'ed because there's something funny on the remote
host, so I want to concentrate on the non-ssh case first.

Can you reproduce the crash on more than one Mac?

[Hannu Kröger]

I can make it crash on the localhost (no ssh) so that I set scrollback
buffer to 5000. The crashing stops if I change it to 10. If it doesn't
crash, then the end of the file becomes garbled. I was able to make my
gf's macbook iTerm produce garble if I switch "Use Transparency" off.
Which is weird. Maybe it was just a coincidence.

Both computers are a bit older laptops, a 15" Macbook Pro from 2007 &
13,3" Macbook from 2008 with the latest OS X.

I couldn't make it crash anymore after changing some configuration. I
attached a screen shot which shows the attempts of Pasting in two
windows. You can see that the content of those windows is not exactly
the same. Not even close.

Hannu

2011/6/7 George Nachman <gnac...@llamas.org>:

[George Nachman]

Does it crash when you use the file you attached? Wil running that
once cause the crash/corruption? How often can you reproduce it?

Please send me your ~/Library/Preferences/com.googlecode.iterm2.plist
file in case there's something configuration-specific that's causing
this. I'm really interested in reproducing it, but I've been catting
that file for a while with no effect :(

[Hannu Kröger]

Ok, I have attached the file you asked here. I just tested it again
and it crashed again several times.

To be sure that we do it the same way I have the steps here:
1) Open that itermbug.txt in TextEdit or some other text editor.
2) Copy all the text
3) Paste that in a iTerm2 terminal
4) It will create a file mytestfile.txt and it should contain some
errors and should be different almost every time you paste the text in
the terminal.

Is there any other information I could provide?

Hannu

2011/6/8 George Nachman <gnac...@llamas.org>:

[Tom Feist]

I've just managed to reproduce it on my (freshly compiled) version, both
with default config and my usual one.

Weirdly, it only seems to crash if I paste it directly into the shell;
pasting into vim, nano both seem to handle it fine (for at least 10
pastes in a row). Pasting into the shell crashes it every time.

See below for the crash report (UKCrashReporter seems to be
misbehaving, and
I'm on the wrong branch for fixing it just now :/)

--Tom

On 8 Jun 2011, at 07:04, George Nachman wrote:

> Does it crash when you use the file you attached? Wil running that
> once cause the crash/corruption? How often can you reproduce it?
>
> Please send me your ~/Library/Preferences/com.googlecode.iterm2.plist
> file in case there's something configuration-specific that's causing
> this. I'm really interested in reproducing it, but I've been catting
> that file for a while with no effect :(

Process: iTerm [34344]
Path: /Applications/iTerm.app/Contents/MacOS/iTerm
Identifier: com.googlecode.iterm2
Version: 0.20.20110607 (0.20.20110607)
Code Type: X86-64 (Native)
Parent Process: launchd [160]

Interval Since Last Report: 310 sec
Crashes Since Last Report: 1
Per-App Interval Since Last Report: 248 sec
Per-App Crashes Since Last Report: 1

Date/Time: 2011-06-08 11:05:26.457 +0100
OS Version: Mac OS X 10.5.8 (9L31a)
Report Version: 6
Anonymous UUID: 0DD14239-1E0C-4B6E-9651-E784CA9CE13E

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000118990fff
Crashed Thread: 2

Thread 0:
0 libSystem.B.dylib 0x00007fff83973dd6 mach_msg_trap + 10
1 libSystem.B.dylib 0x00007fff8397b413 mach_msg + 59
2 com.apple.CoreFoundation 0x00007fff82e1e9cf
CFRunLoopRunSpecific + 1631
3 com.apple.HIToolbox 0x00007fff80da4d0e
RunCurrentEventLoopInMode + 278
4 com.apple.HIToolbox 0x00007fff80da4b44
ReceiveNextEventCommon + 322
5 com.apple.HIToolbox 0x00007fff80da49ef
BlockUntilNextEventMatchingListInMode + 79
6 com.apple.AppKit 0x00007fff8405ee70 _DPSNextEvent +
603
7 com.apple.AppKit 0x00007fff8405e7b1 -[NSApplication
nextEventMatchingMask:untilDate:inMode:dequeue:] + 136
8 com.apple.AppKit 0x00007fff84058523 -[NSApplication
run] + 434
9 com.apple.AppKit 0x00007fff840252f0
NSApplicationMain + 373
10 com.googlecode.iterm2 0x0000000100001094 start + 52

Thread 1:
0 libSystem.B.dylib 0x00007fff8397ada2 __semwait_signal
+ 10
1 com.apple.Foundation 0x00007fff8080ec4f +[NSThread
sleepForTimeInterval:] + 63
2 com.googlecode.iterm2 0x0000000100039ba4 -[ProcessCache
_run] + 116 (ProcessCache.m:253)
3 com.apple.Foundation 0x00007fff807cad35
__NSThread__main__ + 1157
4 libSystem.B.dylib 0x00007fff839a1e8f _pthread_start +
316
5 libSystem.B.dylib 0x00007fff839a1d51 thread_start + 13

Thread 2 Crashed:
0 libSystem.B.dylib 0x00007fffffe00ea7 __memcpy + 1799
(cpu_capabilities.h:246)
1 com.googlecode.iterm2 0x000000010007df6c -[PTYTask
processWrite] + 220 (PTYTask.m:616)
2 com.googlecode.iterm2 0x000000010007ee90 -[TaskNotifier
run] + 1440 (PTYTask.m:306)
3 com.apple.Foundation 0x00007fff807cad35
__NSThread__main__ + 1157
4 libSystem.B.dylib 0x00007fff839a1e8f _pthread_start +
316
5 libSystem.B.dylib 0x00007fff839a1d51 thread_start + 13

Thread 2 crashed with X86 Thread State (64-bit):
rax: 0x74697720796c6e6f rbx: 0x0000000000023125 rcx:
0x0000000000000001 rdx: 0x0000000000000020
rdi: 0x0000000118991008 rsi: 0x0000000118990fff rbp:
0x0000000118c5f950 rsp: 0x0000000118c5f950
r8: 0x0000000117f0f678 r9: 0x0000000000000000 r10:
0x0000000102980200 r11: 0x0000000118991000
r12: 0x0000000117f0f510 r13: 0xffffffffffffffff r14:
0x0000000118991000 r15: 0x0000000117fe2d80
rip: 0x00007fffffe00ea7 rfl: 0x0000000000010216 cr2:
0x0000000118990fff

Binary Images:
0x100000000 - 0x10011cfef +com.googlecode.iterm2
0.20.20110607 (0.20.20110607) <eb27c3da51f8ddc4594939cb7e922137> /
Applications/iTerm.app/Contents/MacOS/iTerm
0x10020e000 - 0x100217fff +iTerm ??? (???)
<433d9e3780c33d01a263a347ab3872f5> /Applications/iTerm.app/Contents/
Frameworks/iTerm.framework/Versions/A/iTerm

[George Nachman]

In case you're curious and so I don't forget while I'm out of town this weekend:

The cause of the crash is that we mishandle a return value of -1 from write(). But fixing that doesn't fix the problem of the file being mangled. Strangely, if we ever try to write() more than the shell's file descriptor can accept, this bug will be triggered, but not where the short write occurred. I verified that we are actually writing correctly, but the shell doesn't seem to be reading it correctly. Writing slowly definitely fixes it. I need to see if there's a way to tell how much buffer space a file descriptor has available to write to it and always write less than that, which I suspect might fix it.

Doubly bizarrely, running the shell inside /usr/bin/script and doing a paste actually causes script to hang, hard. I haven't succeeded in building my own script binary. It may be worthwhile to build a debug bash binary to see what's going on internally.

[George Nachman]

This looks like it's a bug in Readline, of all things. I doubt it'll be fixed any time soon. I changed iTerm2 to paste a little bit slower so bash has a chance to read its input. If it's not slow enough, there's a hidden setting to tweak it. This is a lame solution, but it's all I got!

[Edward Marczak]

Despite the presence of /usr/lib/libreadline.dylib, Mac OS X doesn't
have readline.

The readline library is GPL, and Apple is generally allergic to that
license. They use NetBSD's "editline" library (libedit.dylib).

I mention this because libedit is FOSS, so you can look at the source.
Last I looked, editline was reasonably compatible with readline,
however, readline still has features that libedit doesn't, so it is
plausible that any app depending on those features might misbehave in
the absence of the _real_ readline. Nevertheless, the compatibility is
sufficient that Apple simply symlinks libreadline -> libedit.

--
Edward Marczak
w: http://www.radiotope.com/writing
Managed Prefs Book: http://goo.gl/8iVmo
MacTech Conference: http://www.mactech.com/conference

[George Nachman]

Hard to say what /bin/bash uses since it's not dynamically linked against either libedit or readline. I can confirm that readline has this problem because I built bash from source (and readline is included with bash). I wouldn't be surprised if libedit has the same problem, though.

[Hannu Kröger]

It seems that the latest beta fixes those problems. Thanks!

Hannu

2011/6/15 George Nachman <gnac...@llamas.org>: