Nokogiri 1.6.0 slow build times
3,154 views
Skip to first unread message
Adam Cohen
Jun 13, 2013, 10:59:33 PM6/13/13
to nokogi...@googlegroups.com
Just tried upgrading our rails app from Nokogiri 1.5.9 to 1.6.0 and I've noticed that the build time when doing a bundle install has increased from about 5 minutes to over 30 minutes. I've tried exporting NOKOGIRI_USE_SYSTEM_LIBRARIES=true to prevent nokogiri from compiling libxml2 and libxslt, but this hasn't decreased the build time, it's only made the final gem smaller. Anyone have any suggestions on what else I can try to speed up the build? Thanks,
Adam
Adam
Jonathan Rochkind
Jun 23, 2013, 11:47:40 AM6/23/13
to nokogi...@googlegroups.com
Yeah, I understand and appreciate the motivation for the embedded build-from-source libxml in nokogiri 1.6.0 --but it makes the install time a LOT longer on my OSX box. Like CRAZY
long.
I am for the moment locking to "~> 1.5.0" on my own app, although that's certainly not a sustainable solution.
I am curious if the nokogiri devs have any response to this; if it's known how wide this problem is (everyone? Everyone on osx? just a few of us with machines in certain ways to trigger this? Etc.); etc.
From: nokogi...@googlegroups.com [nokogi...@googlegroups.com] on behalf of Adam Cohen [adambenj...@gmail.com]
Sent: Thursday, June 13, 2013 10:59 PM
To: nokogi...@googlegroups.com
Subject: [nokogiri-talk] Nokogiri 1.6.0 slow build times
Sent: Thursday, June 13, 2013 10:59 PM
To: nokogi...@googlegroups.com
Subject: [nokogiri-talk] Nokogiri 1.6.0 slow build times
Just tried upgrading our rails app from Nokogiri 1.5.9 to 1.6.0 and I've noticed that the build time when doing a bundle install has increased from about 5 minutes to over 30 minutes. I've tried exporting NOKOGIRI_USE_SYSTEM_LIBRARIES=true to prevent
nokogiri from compiling libxml2 and libxslt, but this hasn't decreased the build time, it's only made the final gem smaller. Anyone have any suggestions on what else I can try to speed up the build? Thanks,
Adam
Adam
--
You received this message because you are subscribed to the Google Groups "nokogiri-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nokogiri-tal...@googlegroups.com.
To post to this group, send email to nokogi...@googlegroups.com.
Visit this group at http://groups.google.com/group/nokogiri-talk.
For more options, visit https://groups.google.com/groups/opt_out.
You received this message because you are subscribed to the Google Groups "nokogiri-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nokogiri-tal...@googlegroups.com.
To post to this group, send email to nokogi...@googlegroups.com.
Visit this group at http://groups.google.com/group/nokogiri-talk.
For more options, visit https://groups.google.com/groups/opt_out.
Mike Dalessio
Jul 9, 2013, 2:28:49 PM7/9/13
to nokogiri-talk
Hi all,
Can you be more specific than "crazy long", and include how you're measuring crazy and what commands you're running?
Sorry for the delayed reply.
On my machine I can turn off libxml2/libxslt compilation at gem install time:
~ $ time env NOKOGIRI_USE_SYSTEM_LIBRARIES=true gem install nokogiriBuilding native extensions. This could take a while...Successfully installed nokogiri-1.6.01 gem installedreal 0m11.139suser 0m6.576ssys 0m1.468s~ $ time env gem install nokogiriBuilding native extensions. This could take a while...Successfully installed nokogiri-1.6.01 gem installedreal 1m32.177suser 1m4.608ssys 0m10.461s
Jonathan Rochkind
Jul 9, 2013, 3:13:36 PM7/9/13
to nokogi...@googlegroups.com, Mike Dalessio
On my brand new iMac, I'm getting about 1.5 minutes for gem install.
(Had to install gnu grep to get it to install at all, see
https://github.com/sparklemotion/nokogiri/issues/935)
On my slower (but ordinarily still serviceable) linux staging box, I'm
getting 2.5 minutes for gem install.
So, yeah, not entirely dissimilar to what you are getting. I think I was
getting even longer times before, on the order of 10-20 minutes -- but
maybe I'm misremembering, or maybe my machine was just bogged down at
that point.
It is obviously still quite a bit longer than without compile of
nokogiri. But maybe not entirely ridiculous. Mike, do you consider 1-3
minute install times on modern hardware to be within what you'd consider
acceptable parameters?
On 7/9/13 2:28 PM, Mike Dalessio wrote:
> Hi all,
>
> Sorry for the delayed reply.
>
> On my machine I can turn off libxml2/libxslt compilation at gem install
> time:
>
> ~ $ time env NOKOGIRI_USE_SYSTEM_LIBRARIES=true gem install nokogiri
> Building native extensions. This could take a while...
> Successfully installed nokogiri-1.6.0
> 1 gem installed
>
> *real0m11.139s*
> user0m6.576s
> sys0m1.468s
> user1m4.608s
> sys0m10.461s
> *From:* nokogi...@googlegroups.com
> <mailto:nokogi...@googlegroups.com>
> [nokogi...@googlegroups.com
> <mailto:nokogi...@googlegroups.com>] on behalf of Adam Cohen
> [adambenj...@gmail.com <mailto:adambenj...@gmail.com>]
> *Sent:* Thursday, June 13, 2013 10:59 PM
> *To:* nokogi...@googlegroups.com
> <mailto:nokogi...@googlegroups.com>
> *Subject:* [nokogiri-talk] Nokogiri 1.6.0 slow build times
(Had to install gnu grep to get it to install at all, see
https://github.com/sparklemotion/nokogiri/issues/935)
On my slower (but ordinarily still serviceable) linux staging box, I'm
getting 2.5 minutes for gem install.
So, yeah, not entirely dissimilar to what you are getting. I think I was
getting even longer times before, on the order of 10-20 minutes -- but
maybe I'm misremembering, or maybe my machine was just bogged down at
that point.
It is obviously still quite a bit longer than without compile of
nokogiri. But maybe not entirely ridiculous. Mike, do you consider 1-3
minute install times on modern hardware to be within what you'd consider
acceptable parameters?
On 7/9/13 2:28 PM, Mike Dalessio wrote:
> Hi all,
>
> Sorry for the delayed reply.
>
> On my machine I can turn off libxml2/libxslt compilation at gem install
> time:
>
> ~ $ time env NOKOGIRI_USE_SYSTEM_LIBRARIES=true gem install nokogiri
> Building native extensions. This could take a while...
> Successfully installed nokogiri-1.6.0
> 1 gem installed
>
> user0m6.576s
> sys0m1.468s
>
> ~ $ time env gem install nokogiri
> Building native extensions. This could take a while...
> Successfully installed nokogiri-1.6.0
> 1 gem installed
>
> *real1m32.177s*
> ~ $ time env gem install nokogiri
> Building native extensions. This could take a while...
> Successfully installed nokogiri-1.6.0
> 1 gem installed
>
> user1m4.608s
> sys0m10.461s
>
>
> Can you be more specific than "crazy long", and include how you're
> measuring crazy and what commands you're running?
>
>
>
>
> On Sun, Jun 23, 2013 at 11:47 AM, Jonathan Rochkind <roch...@jhu.edu
> <mailto:roch...@jhu.edu>> wrote:
>
> Yeah, I understand and appreciate the motivation for the embedded
> build-from-source libxml in nokogiri 1.6.0 --but it makes the
> install time a LOT longer on my OSX box. Like CRAZY long.
>
> I am for the moment locking to "~> 1.5.0" on my own app, although
> that's certainly not a sustainable solution.
>
> I am curious if the nokogiri devs have any response to this; if it's
> known how wide this problem is (everyone? Everyone on osx? just a
> few of us with machines in certain ways to trigger this? Etc.); etc.
>
> ------------------------------------------------------------------------
>
> Can you be more specific than "crazy long", and include how you're
> measuring crazy and what commands you're running?
>
>
>
>
> On Sun, Jun 23, 2013 at 11:47 AM, Jonathan Rochkind <roch...@jhu.edu
> <mailto:roch...@jhu.edu>> wrote:
>
> Yeah, I understand and appreciate the motivation for the embedded
> build-from-source libxml in nokogiri 1.6.0 --but it makes the
> install time a LOT longer on my OSX box. Like CRAZY long.
>
> I am for the moment locking to "~> 1.5.0" on my own app, although
> that's certainly not a sustainable solution.
>
> I am curious if the nokogiri devs have any response to this; if it's
> known how wide this problem is (everyone? Everyone on osx? just a
> few of us with machines in certain ways to trigger this? Etc.); etc.
>
> *From:* nokogi...@googlegroups.com
> <mailto:nokogi...@googlegroups.com>
> [nokogi...@googlegroups.com
> <mailto:nokogi...@googlegroups.com>] on behalf of Adam Cohen
> [adambenj...@gmail.com <mailto:adambenj...@gmail.com>]
> *Sent:* Thursday, June 13, 2013 10:59 PM
> *To:* nokogi...@googlegroups.com
> <mailto:nokogi...@googlegroups.com>
> *Subject:* [nokogiri-talk] Nokogiri 1.6.0 slow build times
>
> Just tried upgrading our rails app from Nokogiri 1.5.9 to 1.6.0 and
> I've noticed that the build time when doing a bundle install has
> increased from about 5 minutes to over 30 minutes. I've tried
> exporting NOKOGIRI_USE_SYSTEM_LIBRARIES=true to prevent nokogiri
> from compiling libxml2 and libxslt, but this hasn't decreased the
> build time, it's only made the final gem smaller. Anyone have any
> suggestions on what else I can try to speed up the build? Thanks,
>
> Adam
>
> --
> You received this message because you are subscribed to the Google
> Groups "nokogiri-talk" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to nokogiri-tal...@googlegroups.com
> <mailto:nokogiri-talk%2Bunsu...@googlegroups.com>.
> Just tried upgrading our rails app from Nokogiri 1.5.9 to 1.6.0 and
> I've noticed that the build time when doing a bundle install has
> increased from about 5 minutes to over 30 minutes. I've tried
> exporting NOKOGIRI_USE_SYSTEM_LIBRARIES=true to prevent nokogiri
> from compiling libxml2 and libxslt, but this hasn't decreased the
> build time, it's only made the final gem smaller. Anyone have any
> suggestions on what else I can try to speed up the build? Thanks,
>
> Adam
>
> --
> You received this message because you are subscribed to the Google
> Groups "nokogiri-talk" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to nokogiri-tal...@googlegroups.com
> To post to this group, send email to nokogi...@googlegroups.com
> <mailto:nokogi...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/nokogiri-talk.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "nokogiri-talk" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to nokogiri-tal...@googlegroups.com
> <mailto:nokogiri-talk%2Bunsu...@googlegroups.com>.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "nokogiri-talk" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to nokogiri-tal...@googlegroups.com
> To post to this group, send email to nokogi...@googlegroups.com
> <mailto:nokogi...@googlegroups.com>.
Walter Lee Davis
Jul 26, 2013, 4:31:20 PM7/26/13
to nokogi...@googlegroups.com
I'm currently watching the Nokogiri 1.6 install on AWS Ubuntu 12, and while this worked just fin on my Lion Mac, it's just sitting there at the "building native extensions" stage of gem install nokogiri and not reporting any progress at all after ten minutes.
As I'm typing this, it finally finished. But serious discrepancy between a Mid-2010 MBP and an AWS instance.
Walter
As I'm typing this, it finally finished. But serious discrepancy between a Mid-2010 MBP and an AWS instance.
Walter
Max Kaplan
Oct 14, 2013, 9:17:35 PM10/14/13
to nokogi...@googlegroups.com
I'm also running AWS with CentOS and it takes about 20-30 minutes to install nokogiri 1.6.0.
Jonathan Rochkind
Oct 15, 2013, 3:10:05 PM10/15/13
to nokogi...@googlegroups.com, Max Kaplan
I wonder if the nokogiri build process (which I think is different on
different platforms? using the platform package manager? I'm not sure)
builds dependencies, such that the first time you run the build it's
going to be longer (order of magnitudes longer?) then subsequent times?
That could explain the order of magnitude differences in build times
people are reporting. As well as my own recollection that I thought it
took 20+ minutes, but could only reproduce 1-2 minutes. Maybe it takes
20+ minutes the first time?
Which could be problematic in 'cloud' deploy scenarios where there are a
lot of 'first times' on newly provisioned machines.
Just brainstorming.
On 10/14/13 9:17 PM, Max Kaplan wrote:
> I'm also running AWS with CentOS and it takes about 20-30 minutes to
> install nokogiri 1.6.0.
>
> On Friday, July 26, 2013 1:31:20 PM UTC-7, Walter Lee Davis wrote:
>
> I'm currently watching the Nokogiri 1.6 install on AWS Ubuntu 12,
> and while this worked just fin on my Lion Mac, it's just sitting
> there at the "building native extensions" stage of gem install
> nokogiri and not reporting any progress at all after ten minutes.
>
> As I'm typing this, it finally finished. But serious discrepancy
> between a Mid-2010 MBP and an AWS instance.
>
> Walter
>
> On Jul 9, 2013, at 3:13 PM, Jonathan Rochkind wrote:
>
> > On my brand new iMac, I'm getting about 1.5 minutes for gem
> install. (Had to install gnu grep to get it to install at all, see
> https://github.com/sparklemotion/nokogiri/issues/935
> <https://github.com/sparklemotion/nokogiri/issues/935>)
> <roch...@jhu.edu <javascript:>
> >> <mailto:nokogi...@googlegroups.com <javascript:>>
> >> [nokogi...@googlegroups.com <javascript:>
> >> <mailto:nokogi...@googlegroups.com <javascript:>>] on behalf
> of Adam Cohen
> >> [adambenj...@gmail.com <javascript:>
> <mailto:adambenj...@gmail.com <javascript:>>]
> >> <mailto:nokogi...@googlegroups.com <javascript:>>
> >> <mailto:nokogiri-talk%2Bunsu...@googlegroups.com
> <javascript:>>.
> >> <mailto:nokogi...@googlegroups.com <javascript:>>.
> <http://groups.google.com/group/nokogiri-talk>.
> <https://groups.google.com/groups/opt_out>.
> >> <mailto:nokogiri-talk%2Bunsu...@googlegroups.com
> <javascript:>>.
> >> <mailto:nokogi...@googlegroups.com <javascript:>>.
> <http://groups.google.com/group/nokogiri-talk>.
> <https://groups.google.com/groups/opt_out>.
different platforms? using the platform package manager? I'm not sure)
builds dependencies, such that the first time you run the build it's
going to be longer (order of magnitudes longer?) then subsequent times?
That could explain the order of magnitude differences in build times
people are reporting. As well as my own recollection that I thought it
took 20+ minutes, but could only reproduce 1-2 minutes. Maybe it takes
20+ minutes the first time?
Which could be problematic in 'cloud' deploy scenarios where there are a
lot of 'first times' on newly provisioned machines.
Just brainstorming.
On 10/14/13 9:17 PM, Max Kaplan wrote:
> I'm also running AWS with CentOS and it takes about 20-30 minutes to
> install nokogiri 1.6.0.
>
> On Friday, July 26, 2013 1:31:20 PM UTC-7, Walter Lee Davis wrote:
>
> I'm currently watching the Nokogiri 1.6 install on AWS Ubuntu 12,
> and while this worked just fin on my Lion Mac, it's just sitting
> there at the "building native extensions" stage of gem install
> nokogiri and not reporting any progress at all after ten minutes.
>
> As I'm typing this, it finally finished. But serious discrepancy
> between a Mid-2010 MBP and an AWS instance.
>
> Walter
>
> On Jul 9, 2013, at 3:13 PM, Jonathan Rochkind wrote:
>
> > On my brand new iMac, I'm getting about 1.5 minutes for gem
> install. (Had to install gnu grep to get it to install at all, see
> https://github.com/sparklemotion/nokogiri/issues/935
> >> <mailto:roch...@jhu.edu <javascript:>>> wrote:
> >>
> >> Yeah, I understand and appreciate the motivation for the
> embedded
> >> build-from-source libxml in nokogiri 1.6.0 --but it makes the
> >> install time a LOT longer on my OSX box. Like CRAZY long.
> >>
> >> I am for the moment locking to "~> 1.5.0" on my own app,
> although
> >> that's certainly not a sustainable solution.
> >>
> >> I am curious if the nokogiri devs have any response to this;
> if it's
> >> known how wide this problem is (everyone? Everyone on osx?
> just a
> >> few of us with machines in certain ways to trigger this?
> Etc.); etc.
> >>
> >>
> ------------------------------------------------------------------------
> >> *From:* nokogi...@googlegroups.com <javascript:>
> >>
> >> Yeah, I understand and appreciate the motivation for the
> embedded
> >> build-from-source libxml in nokogiri 1.6.0 --but it makes the
> >> install time a LOT longer on my OSX box. Like CRAZY long.
> >>
> >> I am for the moment locking to "~> 1.5.0" on my own app,
> although
> >> that's certainly not a sustainable solution.
> >>
> >> I am curious if the nokogiri devs have any response to this;
> if it's
> >> known how wide this problem is (everyone? Everyone on osx?
> just a
> >> few of us with machines in certain ways to trigger this?
> Etc.); etc.
> >>
> >>
> ------------------------------------------------------------------------
> >> <mailto:nokogi...@googlegroups.com <javascript:>>
> >> [nokogi...@googlegroups.com <javascript:>
> >> <mailto:nokogi...@googlegroups.com <javascript:>>] on behalf
> of Adam Cohen
> >> [adambenj...@gmail.com <javascript:>
> <mailto:adambenj...@gmail.com <javascript:>>]
> >> *Sent:* Thursday, June 13, 2013 10:59 PM
> >> *To:* nokogi...@googlegroups.com <javascript:>
> >> <mailto:nokogi...@googlegroups.com <javascript:>>
> >> *Subject:* [nokogiri-talk] Nokogiri 1.6.0 slow build times
> >>
> >> Just tried upgrading our rails app from Nokogiri 1.5.9 to
> 1.6.0 and
> >> I've noticed that the build time when doing a bundle install has
> >> increased from about 5 minutes to over 30 minutes. I've tried
> >> exporting NOKOGIRI_USE_SYSTEM_LIBRARIES=true to prevent nokogiri
> >> from compiling libxml2 and libxslt, but this hasn't decreased
> the
> >> build time, it's only made the final gem smaller. Anyone
> have any
> >> suggestions on what else I can try to speed up the build?
> Thanks,
> >>
> >> Adam
> >>
> >> --
> >> You received this message because you are subscribed to the
> Google
> >> Groups "nokogiri-talk" group.
> >> To unsubscribe from this group and stop receiving emails from
> it,
> >> send an email to nokogiri-tal...@googlegroups.com <javascript:>
> >>
> >> Just tried upgrading our rails app from Nokogiri 1.5.9 to
> 1.6.0 and
> >> I've noticed that the build time when doing a bundle install has
> >> increased from about 5 minutes to over 30 minutes. I've tried
> >> exporting NOKOGIRI_USE_SYSTEM_LIBRARIES=true to prevent nokogiri
> >> from compiling libxml2 and libxslt, but this hasn't decreased
> the
> >> build time, it's only made the final gem smaller. Anyone
> have any
> >> suggestions on what else I can try to speed up the build?
> Thanks,
> >>
> >> Adam
> >>
> >> --
> >> You received this message because you are subscribed to the
> >> Groups "nokogiri-talk" group.
> >> To unsubscribe from this group and stop receiving emails from
> it,
> >> <mailto:nokogiri-talk%2Bunsu...@googlegroups.com
> <javascript:>>.
> >> To post to this group, send email to
> nokogi...@googlegroups.com <javascript:>
> >> <mailto:nokogi...@googlegroups.com <javascript:>>.
> <http://groups.google.com/group/nokogiri-talk>.
> <https://groups.google.com/groups/opt_out>.
> >>
> >>
> >> --
> >> You received this message because you are subscribed to the
> Google
> >> Groups "nokogiri-talk" group.
> >> To unsubscribe from this group and stop receiving emails from
> it,
> >> send an email to nokogiri-tal...@googlegroups.com <javascript:>
> >>
> >> --
> >> You received this message because you are subscribed to the
> >> Groups "nokogiri-talk" group.
> >> To unsubscribe from this group and stop receiving emails from
> it,
> >> <mailto:nokogiri-talk%2Bunsu...@googlegroups.com
> <javascript:>>.
> >> To post to this group, send email to
> nokogi...@googlegroups.com <javascript:>
> >> <mailto:nokogi...@googlegroups.com <javascript:>>.
> <http://groups.google.com/group/nokogiri-talk>.
> <https://groups.google.com/groups/opt_out>.
> >>
> >>
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> >> Groups "nokogiri-talk" group.
> >> To unsubscribe from this group and stop receiving emails from
> it, send
> >> an email to nokogiri-tal...@googlegroups.com <javascript:>.
> >>
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> >> Groups "nokogiri-talk" group.
> >> To unsubscribe from this group and stop receiving emails from
> it, send
> >> To post to this group, send email to nokogi...@googlegroups.com
> <javascript:>.
> >> Visit this group at http://groups.google.com/group/nokogiri-talk
> <http://groups.google.com/group/nokogiri-talk>.
> >> For more options, visit https://groups.google.com/groups/opt_out
> <https://groups.google.com/groups/opt_out>.
> >>
> >>
> >
> > --
> > You received this message because you are subscribed to the
> Google Groups "nokogiri-talk" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send an email to nokogiri-tal...@googlegroups.com <javascript:>.
> >>
> >
> > --
> > You received this message because you are subscribed to the
> Google Groups "nokogiri-talk" group.
> > To unsubscribe from this group and stop receiving emails from it,
> > To post to this group, send email to nokogi...@googlegroups.com
> <javascript:>.
> > Visit this group at http://groups.google.com/group/nokogiri-talk
> <http://groups.google.com/group/nokogiri-talk>.
> > For more options, visit https://groups.google.com/groups/opt_out
> <https://groups.google.com/groups/opt_out>.
Fernando Tobar
Feb 24, 2015, 6:13:17 PM2/24/15
to nokogi...@googlegroups.com
Similar for me. Take several minutes in MAC OS,
but works faster using:
gem install nokogiri -- --use-system-libraries with previous dependencies installed
Best,
Fernando
Fernando
Reply all
Reply to author
Forward
0 new messages