DrScheme 6.2.1 crash when running a web crawler

[Brian Adkins]

I wrote a simple web crawler in Racket. It ran fine via "racket crawler.rkt" from the command line, but I had a number of crashes in DrScheme 6.2.1. I haven't been able to narrow it down to a nice, small, reproducible test case, but I thought I'd provide some of the info I have in case it may be of use. Here is a gist that has two crash logs and the source code:

https://gist.github.com/lojic/685a71594bb3f0cf1faa

I've modified the code since the crashes, and I'm not sure if it's still crashing in the version in the gist - I ran the latest briefly w/o a crash. The crash may have happened when I was getting an http connection and using http-conn-sendrecv! instead of getting a fresh connection each time, but that's a guess.

If the crash reports aren't sufficient in themselves, I can try and produce a more suitable test case.

As an aside, the web crawler was used to gather programming language popularity stats for this blog post:

http://blog.lojic.com/2015/10/06/programming-language-popularity-part-nine/

I'm hoping to help increase Racket's rank in that list as I gain some competency :)

Thanks,

Brian

[Matthew Flatt]

From the stack trace, it looks like this is an

about to suspend in atomic mode

error that was hit in the process of trying to report another error by
writing that error to DrRacket's window.

I don't see what has put the VM in atomic mode, but it looks like the
original error was a "port is closed" error via `read-line`. Maybe the
`read-line` call was reached through `http-conn-sendrecv!`, since I see
one in the `net/http-client` library.

Does a "port is closed" error sound familiar, especially during the
time that you used `http-conn-sendrecv!`? Can you reconstruct the
version that used `http-conn-sendrecv!`?

Thanks for the report,
Matthew

> --
> You received this message because you are subscribed to the Google Groups
> "Racket Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-dev+...@googlegroups.com.
> To post to this group, send email to racke...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-dev/2e35260e-7916-4e3d-9c3a-464a9bd26a
> 3a%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Brian Adkins]

Yes, I think a "port is closed" error is likely. Although the crashes both happened immediately after running the program w/in DrScheme, I later added the sleep to avoid getting blocked, and then I got an error regarding the port being closed, so I read up on http-conn-sendrecv! and realized that, according to the docs, the server could request the port to be closed.

Since I was going to be sleeping anywhere between 10 and 50 seconds, I thought it likely the port could be closed, so I switched to the version in the gist to get a fresh connection each time and close it automatically.

I should be able to go back to the http-conn-sendrecv! version and get a small reproducible test case. It crashed three out of three times before I gave up and switched to running on the command line, so I think it shouldn't be difficult to reproduce. I'll try and do that on Wed.

Thanks for looking into it.

[Brian Adkins]

Good news and bad news - I've been unable to reproduce the error. I also retract my statement about the likelihood of the port being closed because when I tried manually closing the port just now, I received the contract error regarding the port not being open, and I remember receiving that during testing before vs. a crash.

I modified the code to get a connection and call http-con-sendrecv!, and tried a handful of changes that I thought might cause the crash, but it's working fine w/in Dr Scheme now :(

Lesson learned though - next time I'll make a snapshot of the code after a crash.

I do use time machine on my Mac, so there's a very slight chance one of the hourly backups grabbed the code at its worst, but I kind of doubt that. I'll report back if I can reproduce it.

Brian