[ruby-core:67712] [ruby-trunk - Bug #10761] [Open] Marshal.dump 100% slower in 2.2.0 vs 2.1.5

4 views
Skip to first unread message

janmayjuldec...@yahoo.com

unread,
Jan 21, 2015, 12:14:21 AM1/21/15
to ruby...@ruby-lang.org
Issue #10761 has been reported by David Jenkins.

----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761

* Author: David Jenkins
* Status: Open
* Priority: High
* Assignee:
* ruby -v: 2.2.0
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
Calling Marshal.dump on a complex (nested) object 100 times shows the following:

2.1.5
user system total real
Marshal Dump 24.350000 0.250000 24.600000 ( 29.330294)

2.2.0
user system total real
Marshal Dump 50.450000 0.140000 50.590000 ( 52.268773)

To reproduce, put marshal_slow.rb and nested_ds.txt in same directory and run marshal_slow.rb.

I found that Marshal.dump only shows this kind of discrepancy on deeply nested objects. The nested_ds.txt file is a dump of an object that contained a collection of sub objects of the same type.

---Files--------------------------------
nested_ds.txt (2.1 MB)
marshal_slow.rb (212 Bytes)


--
https://bugs.ruby-lang.org/

janmayjuldec...@yahoo.com

unread,
Jan 21, 2015, 9:49:17 AM1/21/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by David Jenkins.

File dataset_dump.txt added
File better_slow_marshal.rb added

I have an even better repro (attached) which yielded the following results:

2.1.5
user system total real
Marshal Dump 4.250000 0.080000 4.330000 ( 4.682933)
Marshal Dump 4.440000 0.040000 4.480000 ( 4.613776)
Marshal Dump 4.500000 0.050000 4.550000 ( 4.717084)

2.2.0
user system total real
Marshal Dump 13.390000 0.070000 13.460000 ( 13.742177)
Marshal Dump 14.540000 0.090000 14.630000 ( 14.884479)
Marshal Dump 13.580000 0.060000 13.640000 ( 13.968422)


----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51151

* Author: David Jenkins
* Status: Open
* Priority: High
* Assignee:
* ruby -v: 2.2.0
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
Calling Marshal.dump on a complex (nested) object 100 times shows the following:

2.1.5
user system total real
Marshal Dump 24.350000 0.250000 24.600000 ( 29.330294)

2.2.0
user system total real
Marshal Dump 50.450000 0.140000 50.590000 ( 52.268773)

To reproduce, put marshal_slow.rb and nested_ds.txt in same directory and run marshal_slow.rb.

I found that Marshal.dump only shows this kind of discrepancy on deeply nested objects. The nested_ds.txt file is a dump of an object that contained a collection of sub objects of the same type.

---Files--------------------------------
nested_ds.txt (2.1 MB)
marshal_slow.rb (212 Bytes)
dataset_dump.txt (362 KB)
better_slow_marshal.rb (239 Bytes)


--
https://bugs.ruby-lang.org/

janmayjuldec...@yahoo.com

unread,
Jan 21, 2015, 11:15:26 AM1/21/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by David Jenkins.


Final repro, no data files, all code:

~~~
require 'benchmark'

class ToBeMarshaled

def initialize n
@a = []
n.times do |i|
@a << i.to_f
end
end

end

tbm = ToBeMarshaled.new(10000)

n = 100

Benchmark.bm do |x|
x.report("Marshal Dump") {for i in 1..n; Marshal.dump(tbm); end}
end

~~~

results (ran benchmark 3 times for each Ruby version):

2.1.5
user system total real
Marshal Dump 0.640000 0.010000 0.650000 ( 0.744080)
Marshal Dump 0.670000 0.000000 0.670000 ( 0.758597)
Marshal Dump 0.650000 0.020000 0.670000 ( 0.747583)

2.2.0
user system total real
Marshal Dump 25.070000 0.220000 25.290000 ( 27.980023)
Marshal Dump 24.100000 0.160000 24.260000 ( 26.633049)
Marshal Dump 24.440000 0.230000 24.670000 ( 27.540826)

~35 times slower

----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51158

Eric Wong

unread,
Jan 21, 2015, 3:37:03 PM1/21/15
to Ruby developers
janmayjuldec...@yahoo.com wrote:
> Final repro, no data files, all code:

Thanks, looks like my fault in r45384 [Feature #9425]
Looking for a fix which doesn't involve reverting...

normal...@yhbt.net

unread,
Jan 21, 2015, 3:38:50 PM1/21/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by Eric Wong.
----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51160

normal...@yhbt.net

unread,
Jan 21, 2015, 5:21:05 PM1/21/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by Eric Wong.

File flonum-hash-f...@r49365.patch added
Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN to 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: REQUIRED

Using Marshal.load on untrusted data is too risky for me,
so I only tested your last test case:

$ ./patched/ruby --disable=gems /tmp/marshal.rb
real 0m0.311s
user 0m0.308s
sys 0m0.002s

$ ~/ruby-2.1/bin/ruby --disable=gems /tmp/marshal.rb
real 0m0.354s
user 0m0.351s
sys 0m0.002s

patch (also attached):
http://80x24.org/spew/m/flonum-hash-f...@r49365.txt

benchmark suite results (with new benchmarks for flonum keys):
http://80x24.org/spew/m/8d1ad66566f9e76b07e000e8dd335b9ebcb06b01.txt

I'll commit soon unless there's problems.


----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51161

* Author: David Jenkins
* Status: Open
* Priority: High
* Assignee:
* ruby -v: 2.2.0
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: REQUIRED
----------------------------------------
Calling Marshal.dump on a complex (nested) object 100 times shows the following:

2.1.5
user system total real
Marshal Dump 24.350000 0.250000 24.600000 ( 29.330294)

2.2.0
user system total real
Marshal Dump 50.450000 0.140000 50.590000 ( 52.268773)

To reproduce, put marshal_slow.rb and nested_ds.txt in same directory and run marshal_slow.rb.

I found that Marshal.dump only shows this kind of discrepancy on deeply nested objects. The nested_ds.txt file is a dump of an object that contained a collection of sub objects of the same type.

---Files--------------------------------
nested_ds.txt (2.1 MB)
marshal_slow.rb (212 Bytes)
dataset_dump.txt (362 KB)
better_slow_marshal.rb (239 Bytes)
flonum-hash-f...@r49365.patch (2.21 KB)


--
https://bugs.ruby-lang.org/

Eric Wong

unread,
Jan 21, 2015, 7:08:43 PM1/21/15
to ruby...@ruby-lang.org
NAK. "gem install <anything>" seems to infinite loop with this.

normal...@yhbt.net

unread,
Jan 21, 2015, 7:09:25 PM1/21/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by Eric Wong.


----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51163

Eric Wong

unread,
Jan 21, 2015, 7:52:19 PM1/21/15
to ruby...@ruby-lang.org
Testing this now on top of the previous commit:
http://80x24.org/spew/m/be1385d4f7cea47dae5aebc7577f3d9c94c18d21.txt

normal...@yhbt.net

unread,
Jan 21, 2015, 7:58:58 PM1/21/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by Eric Wong.


----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51164

Eric Wong

unread,
Jan 22, 2015, 2:57:06 AM1/22/15
to Ruby developers
Committed as r49376. Scary regression, sorry for the breakage :x

normal...@yhbt.net

unread,
Jan 22, 2015, 2:59:02 AM1/22/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by Eric Wong.


Committed as r49376. Scary regression, sorry for the breakage :x

----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51176

* Author: David Jenkins
* Status: Closed

nar...@airemix.jp

unread,
Feb 5, 2015, 11:19:36 AM2/5/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by Yui NARUSE.

Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: REQUIRED to 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: DONE

ruby_2_2 r49513 merged revision(s) 49376,49387,49389.

----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51412

* Author: David Jenkins
* Status: Closed
* Priority: High
* Assignee:
* ruby -v: 2.2.0
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: DONE

no...@ruby-lang.org

unread,
Feb 5, 2015, 4:40:03 PM2/5/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by Nobuyoshi Nakada.

Status changed from Closed to Open
Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: DONE to 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: REQUIRED

r49386 is missing.

----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51419

* Author: David Jenkins
* Status: Open
* Priority: High
* Assignee:
* ruby -v: 2.2.0
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: REQUIRED

u...@garbagecollect.jp

unread,
Feb 5, 2015, 9:41:02 PM2/5/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by Usaku NAKAMURA.

Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: REQUIRED to 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: REQUIRED

----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51422

* Author: David Jenkins
* Status: Open
* Priority: High
* Assignee:
* ruby -v: 2.2.0
* Backport: 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: REQUIRED

nar...@airemix.jp

unread,
Feb 13, 2015, 10:55:58 PM2/13/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by Yui NARUSE.

Backport changed from 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: REQUIRED to 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: DONE

ruby_2_2 r49513 merged revision(s) 49376,49387,49389.

----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51501

* Author: David Jenkins
* Status: Open
* Priority: High
* Assignee:
* ruby -v: 2.2.0
* Backport: 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: DONE

u...@garbagecollect.jp

unread,
Feb 16, 2015, 9:27:29 AM2/16/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by Usaku NAKAMURA.

Status changed from Open to Closed

----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51512

* Author: David Jenkins
* Status: Closed

s.wa...@gmail.com

unread,
Feb 17, 2015, 8:57:20 AM2/17/15
to ruby...@ruby-lang.org
Issue #10761 has been updated by _ wanabe.

Backport changed from 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: DONE to 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: REQUIRED

r49386 is not backported yet in ruby_2_2.
In ruby_2_2@49627, there is '#ifdef USE_FLONUM' in st_numhash() of st.c.
But in trunk, the '#ifdef' has been deleted at r49386.

----------------------------------------
Bug #10761: Marshal.dump 100% slower in 2.2.0 vs 2.1.5
https://bugs.ruby-lang.org/issues/10761#change-51528

* Author: David Jenkins
* Status: Closed
* Priority: High
* Assignee:
* ruby -v: 2.2.0
* Backport: 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: REQUIRED
Reply all
Reply to author
Forward
0 new messages