[BUG] Segmentation fault

[Julien Genestoux]

Hey,

I have now been using Feedzirra extensively and I am confronted to Segmentation fault errors from time to time.
The vast majority of them (if not all) happens within  self.decode_content(c) of feed.rb (on xml = gz.read to be even more precise). I have no idea how to solve that.
However, I am having a lot of problems to reproduce. With the same inputs it might run once and bug the other time. Has anyone observed that as well?

Thanks for you help and sorry for being to imprecise.





--
Julien Genestoux, Notifixio.us
http://twitter.com/julien51
http://www.ouvre-boite.com
http://blog.notifixio.us

+1 (415) 254 7340
+33 (0)9 70 44 76 29

[Paul Dix]

Hi Julien,
That definitely sounds like a problem with the Zlib::GzipReader. I
don't really have experience with that piece. I wonder if there's
something better that Feedzirra should be using?

Paul

[Bobby Wilson]

Hey Guys,

I just wanted to pop in and say that I too experience the seg fault from time to time and have an interest in helping resolve the bug. I don't know C libraries very well but will be happy to help in any way possible.

[Julien Genestoux]

Hum,

I have investigated and didn't really found anything that seem to work...
I am also very "limited" in my C knowledge... but I'll try to work on that as well if anyone has "clues" or "tips" on where to start?

It might not be Zlib::GzipReader though but rather curb? (if curb returns some "wong" data that would explain why the GzipReader fails.. I am suggesting this since Zlib seems to be part of Ruby, and I expect Ruby to be pretty well tested, while Curb seems to be a pretty recent lib so maybe not as reliable as Zlib...

What do you guys think?

Julien

--
Julien Genestoux, Notifixio.us
http://twitter.com/julien51
http://www.ouvre-boite.com
http://blog.notifixio.us

+1 (415) 254 7340
+33 (0)9 70 44 76 29

[Jesse Clark]

I am working on a feed parsing application using the latest version of
Feedzirra and I am consistently getting segmentation faults on two of
the feeds I am trying to parse.

First here is my environment info:
RUBY VERSION: 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0]
Rails 2.3.2
pauldix-feedzirra (0.0.12)
pauldix-sax-machine (0.0.12)
mdalessio-dryopteris (0.1.1)

In an effort to locate the cause of the segfaults I simplified my
parsing code down to this class method which is called from a rake
task:

class Feed < ActiveRecord::Base
def self.parse(id)
ar_feed = Feed.find id
Rails.logger.info "Starting feed #{ar_feed.title} #
{ar_feed.feed_config.url}"
fz_feed = Feedzirra::Feed.fetch_and_parse
( ar_feed.feed_config.url )
Rails.logger.info "Parsed feed #{ar_feed.title} #
{ar_feed.feed_config.url}"
fz_feed.sanitize_entries!
end
end

When I execute this code I get this from the console:

Starting feed Countdown http://rss.msnbc.msn.com/id/18424715/device/rss/rss.xml
Parsed feed Countdown http://rss.msnbc.msn.com/id/18424715/device/rss/rss.xml
/usr/bin/rake:19: [BUG] Segmentation fault
ruby 1.8.6 (2008-03-03) [universal-darwin9.0]

Abort trap

And from the other feed I get:

Starting feed Red Tape Chronicles http://redtape.msnbc.com/index.rdf
Parsed feed Red Tape Chronicles http://redtape.msnbc.com/index.rdf
[path_to_my_app]/vendor/gems/mdalessio-dryopteris-0.1.1/lib/dryopteris/
sanitize.rb:45: [BUG] Segmentation fault
ruby 1.8.6 (2008-03-03) [universal-darwin9.0]

Abort trap

If I comment out the call to 'fz_feed.sanitize_entries!' the feeds
parse fine with no segfaults. Also just out of curiosity, I saved on
of the feeds to disk and fired up the console and executed:

>> file = File.new "[path_to_app]/tmp/redtape_feed.xml"
>> 1000.times { |i| Dryopteris.sanitize( file.read ) }
=> 1000

So, it seems that sanitizing in the context of parsing the feed with
Feedzirra causes the segfault but sanitizing the same content from the
console is fine...

I would appreciate any insight or assistance in resolving this issue.

Thanks,
-Jesse

On Mar 26, 9:53 pm, Julien Genestoux <julien.genest...@gmail.com>
wrote:

> Hum,
>
> I have investigated and didn't really found anything that seem to work...
> I am also very "limited" in my C knowledge... but I'll try to work on that
> as well if anyone has "clues" or "tips" on where to start?
>
> It might not be Zlib::GzipReader though but rather curb? (if curb returns
> some "wong" data that would explain why the GzipReader fails.. I am
> suggesting this since Zlib seems to be part of Ruby, and I expect Ruby to be
> pretty well tested, while Curb seems to be a pretty recent lib so maybe not
> as reliable as Zlib...
>
> What do you guys think?
>
> Julien
>
> --

> Julien Genestoux, Notifixio.ushttp://twitter.com/julien51http://www.ouvre-boite.comhttp://blog.notifixio.us

>
> +1 (415) 254 7340
> +33 (0)9 70 44 76 29
>

> On Wed, Mar 25, 2009 at 12:06 PM, Bobby Wilson <bobbywils...@gmail.com>wrote:
>
> > Hey Guys,
> > I just wanted to pop in and say that I too experience the seg fault from
> > time to time and have an interest in helping resolve the bug. I don't know C
> > libraries very well but will be happy to help in any way possible.
>

[Paul Dix]

Did you try doing Feed.parse on the file read from disk? I'm wondering
if there's some weird character encoding thing that changed when you
saved the feed to disk. What did you do to download it?

Cheers,
Paul

[Jesse Clark]

I just tried Feed.parse passing it the file contents read from disk
and it parsed successfully.

If I then call 'feed.sanitize_entries!' I get the segfault.

Looping over feed.entries and calling entry.content.sanitize! also
causes a segfault.

To download it I just viewed source of the feed in FF, copied it into
a new Textmate file, and saved it.

Thanks,
-Jesse

[Paul Dix]

This must be a bug in a version of dryopteris that you're running. I
just tried both those feeds and can't reproduce the problem. Can you
update your dryopteris and try again? Also, it might help to find out
which entry it dies on and if it dies on sanitizing the title or the
content.

Paul

[Jesse Clark]

I am using: mdalessio-dryopteris (0.1.1) and rubygems tells me this is
the most recent version.

The error seems to be happening when sanitizing content.

I created a simple script that should allow you to reproduce the error.
Script: http://pastie.org/464015

I have attached the sample xml file to use with the script because it
was too large for pastie.

The actual error messages I get from running the script are different
for what I was getting in the rails console. Running the script gives
this instead of a Segmentation fault exception:

ruby(17578) malloc: *** error for object 0x93b20006: Non-aligned
pointer being freed
*** set a breakpoint in malloc_error_break to debug

I wonder if this has anything to do with me being on Ruby 1.8.6...

Thanks,
-Jesse

[Paul Dix]

Hi Jesse,
I just tried and couldn't repro it. I'm running 1.8.7 with the same
version of Dryopteris. Have you made sure your libxml2 is up to date?
I'm not sure if the macports version is current.

Thanks,
Paul

>> --~--~---------~--~----~------------~-------~--~----~
>> You received this message because you are subscribed to the Google
>> Groups "feedzirra" group.
>> To post to this group, send email to feed...@googlegroups.com
>> To unsubscribe from this group, send email to feedzirra+...@googlegroups.com
>> For more options, visit this group at http://groups.google.com/group/feedzirra?hl=en
>> -~----------~----~----~----~------~----~------~--~---
>>
>
>
>

[Jesse Clark]

I just upgraded to libxml2-2.7.3 and still get the malloc errors.

Unfortunately, I am stuck using ruby 1.8.6 for the time being so if
that is the issue I may be SOL...

-Jesse

[Mike Dalessio]

On Apr 30, 2:00 pm, Jesse Clark <je...@jesseclark.com> wrote:
> I just upgraded to libxml2-2.7.3 and still get the malloc errors.

It's possible that, despite haing 2.7.3 installed, that you're still
dynamically loading 2.6.16 at runtime. There are definitely memory
bugs in that version of libxml2.

Can you please dump out the values of these two constants inside your
script?

* Nokogiri::LIBXML_VERSION
* Nokogiri::LIBXML_PARSER_VERSION

A quick word of explanation: LIBXML_VERSION is a C constant that is
compiled at gem-install time. This actually doesn't matter very much.
LIBXML_PARSER_VERSION is actually the version of the dynamically-
loaded libxml2 library.

If you're LIBXML_PARSER_VERSION should be "20703" if you're truly
loading 2.7.3 at runtime. I suspect that for you it will be "20616",
meaning that you're running on a buggy version of libxml2.

Let's verify which library you're running with, and then proceed from
there?

> >>>>>> Parsed feed Countdownhttp://rss.msnbc.msn.com/id/18424715/device/rss/rss.xml

> >>>>>> /usr/bin/rake:19: [BUG] Segmentation fault
> >>>>>> ruby 1.8.6 (2008-03-03) [universal-darwin9.0]
>
> >>>>>> Abort trap
>
> >>>>>> And from the other feed I get:
>

> >>>>>> Starting feed Red Tape Chronicleshttp://redtape.msnbc.com/
> >>>>>> index.rdf
> >>>>>> Parsed feed Red Tape Chronicleshttp://redtape.msnbc.com/

> >>>>>>> Julien Genestoux, Notifixio.ushttp://twitter.com/julien51http://www.ouvre-boite.comhttp://blog.noti...

[Jesse Clark]

You were correct.

Nokogiri::LIBXML_VERSION: 2.6.16
Nokogiri::LIBXML_PARSER_VERSION: 20616

I re-installed nokogiri and it built against the newer version of
libxml and the errors went away.

Thanks for your help,
-Jesse