Winston one thousand time too slow to be usable currently

801 views
Skip to first unread message

Perrin Meyer

unread,
Jul 23, 2013, 7:25:10 PM7/23/13
to julia...@googlegroups.com
winston is currently too slow to be usable on linux with even "moderate" datasets....

is there a path towards usable interactive plotting for "large" datasets? 

is there a unit test for plotting speed i could track? 

thanks!

perrin


julia>  using Winston
julia> z = rand(40000,1) ; tic() ; plot(z) ; toc()
elapsed time: 29.739992104 seconds
29.739992104

MATLAB
>> z = rand(40000,1) ; tic() ; plot(z) ; toc()
Elapsed time is 0.071495 seconds.


Isaiah Norton

unread,
Jul 24, 2013, 12:46:45 AM7/24/13
to julia...@googlegroups.com
plot() has to be JIT'd like everything else, so the first timing is not representative. That said, it shouldn't take that long. On my ubuntu 13.04/virtualbox I get 3.8s for the first run, and .88s for the second run. Not great, but more reasonable.

Tim Holy

unread,
Jul 24, 2013, 1:10:27 AM7/24/13
to julia...@googlegroups.com
Mine is somewhere between yours and Isaiah's in timing.

This observation was one of the main things that motivated me to write Profile.
The somewhat discouraging answer turns out to be that all the time seems to be
in libcairo. I just tried turning off antialiasing, but that didn't really
help.

Curiously, it only seems to happen for rand() and friends. When I try
x = linspace(0, 3pi, 10000);
y = cos(x);
it's reasonably fast (0.2 seconds). I'm guessing that line_to() does less work
when the jumps are small, but I'm just speculating.

What versions of libcairo is everyone using? I'm at 1.10.2 (Ubuntu 12.04).

More importantly, does anyone have any ideas here? This is a pretty big
problem.

--Tim

On Tuesday, July 23, 2013 04:25:10 PM Perrin Meyer wrote:
> winston is currently too slow to be usable on linux with even "moderate"
> datasets....
>
> is there a path towards usable interactive plotting for "large" datasets?
>
> is there a unit test for plotting speed i could track?
>
> thanks!
>
> perrin
>
>
> julia> using Winston
> julia> z = rand(40000,1) ; tic() ; plot(z) ; toc()
> elapsed time: *29.739992104 seconds*
> 29.739992104
>
> MATLAB
>
> >> z = rand(40000,1) ; tic() ; plot(z) ; toc()
>
> Elapsed time is* 0.071495 seconds.*

Andreas Lobinger

unread,
Jul 24, 2013, 2:59:46 AM7/24/13
to julia...@googlegroups.com
Hello,

this might not be helpful, but ... one thing that clearly annoys me in matlab is the slow plotting speed...

I'm writing a graphical application (in python with gtk & cairo) and the cairo speed was not a problem up to now (read as: python and gtk speeds are).
The drawing speed itself is good (for a proper definition of goodness), what can take time is providing the drawing list, so the path definition. There are some hints in the cairo doc to e.g. create paths economically and my own observation was that putting the coordinates into cairo already takes time.

There is some tooling available starting with cairo-trace, that should record something like a profile for cairo usage. Unfortunately (at least at my box) this does not work with julia (reports Warning: Possible conflict in library symbol cairo_paint etc).

I put the above example into a draw_test.jl and my tic/toc time returns 48s, BUT the window shows up on the screen ~7s, the drawing shows up ~30s, and the toc reports the 48s. So from my perspective there seems to be some time spend after calling cairo.

If today is rainy, i might look into this (still learning julia).

Wishing a happy day,
 Andreas  

Tim Holy

unread,
Jul 24, 2013, 6:36:55 AM7/24/13
to julia...@googlegroups.com
If you find anything, that would be awesome. FYI the line that appears to
consume almost all the time is the call to stroke (which ccalls cairo_stroke)
in Winston's renderer.jl: curve() (line 240). Also, since you're new to Julia
and mentioned the difficulty of using cairo-trace: the Profile package
(https://github.com/timholy/Profile.jl) allows you to profile down into C code.
To be useful, one would probably need a debug build of libcairo.

My experience with ImageView shows that Tk & Cairo can indeed yield much
faster graphics than Matlab, so I suspect there must be a solution here.

Best,
--Tim

Patrick O'Leary

unread,
Jul 24, 2013, 9:06:03 AM7/24/13
to julia...@googlegroups.com
On Tuesday, July 23, 2013 6:25:10 PM UTC-5, Perrin Meyer wrote:
MATLAB
>> z = rand(40000,1) ; tic() ; plot(z) ; toc()
Elapsed time is 0.071495 seconds.

Out of curiosity, which renderer is MATLAB using?

>> get(gcf, 'Renderer')

Tim Holy

unread,
Jul 24, 2013, 11:55:49 AM7/24/13
to julia...@googlegroups.com
Out of curiosity, I decided to do a quick comparison benchmark of drawing
lines in pure-Julia:

(A is the "canvas", true for any pixel that has a line cross through it)

function draw_in_julia!(A::Array{Bool}, n)
fill!(A, false)
x = rand(1:size(A,2), n)
y = rand(1:size(A,1), n)
# Activate pixels on the line between (x[i], y[i]) and (x[i+1], y[i+1])
@time for i = 2:n
xstart = x[i-1]
ystart = y[i-1]
dx = x[i] - xstart
dy = y[i] - ystart
m = max(abs(dx),abs(dy))
A[ystart,xstart] = true
xf = float64(xstart)
yf = float64(ystart)
dxf = dx/m
dyf = dy/m
for j = 1:m
xf += dxf
yf += dyf
A[itrunc(yf+0.5), itrunc(xf+0.5)] = true
end
end
A, x, y
end

julia> A = Array(Bool, 512, 512);

julia> A, x, y = draw_in_julia!(A, 40000);
elapsed time: 0.027561252 seconds (0 bytes allocated)

By comparison, for me Matlab takes about 0.1 seconds. That's for a real plot
(figure window already open and on top), but it's clearly handing the actual
display off to a separate thread, so this may actually be a pretty relevant
comparison.

So there is something deeply wrong in Cairo, or in our usage of it. Clearly we
could plot lines in this way if we had to, but I suspect we'd rather exploit
Cairo. So we need to fix whatever is wrong.

@pao, for me Matlab is using 'painters'.

--Tim


On Tuesday, July 23, 2013 11:59:46 PM Andreas Lobinger wrote:

Mike Nolta

unread,
Jul 24, 2013, 12:29:42 PM7/24/13
to julia...@googlegroups.com
Perrin, could you tell us more about your system?

On my centos 6.3 desktop, your example takes 2.3s the first time and
0.049s the second.

-Mike

Perrin Meyer

unread,
Jul 24, 2013, 1:00:14 PM7/24/13
to julia...@googlegroups.com
whats strange is that my julia installation appears to not be doing JIT????? (note it takes 29 seconds each time????).  I compile from GIT??? 

Its a new unix box from microway (whisperstation) -- dual 12core AMD running opensuse so it should be fast. its my first openSUSE system so i'm not too familiar yet.  

thanks

perrin


julia> a = rand(10000,1) ; tic() ; plot(a) ; toc()
elapsed time: 1.237124124 seconds
1.237124124

julia> a = rand(40000,1) ; tic() ; plot(a) ; toc()
elapsed time: 28.832082498 seconds
28.832082498

julia> a = rand(40000,1) ; tic() ; plot(a) ; toc()
elapsed time: 29.60638276 seconds
29.60638276

julia> a = rand(40000,1) ; tic() ; plot(a) ; toc()
elapsed time: 29.304323715 seconds
29.304323715

MATLAB
>> a  = rand(40000,1); tic() ; plot(a) ; toc()
Elapsed time is 0.017461 seconds.
>> a  = rand(40000,1); tic() ; plot(a) ; toc()
Elapsed time is 0.011723 seconds.
>> a  = rand(40000,1); tic() ; plot(a) ; toc()
Elapsed time is 0.004921 seconds.
>> ver
--------------------------------------------------------------------------------------------------------
MATLAB Version: 8.1.0.604 (R2013a)
MATLAB License Number: 271558
Operating System: Linux 3.4.47-2.38-desktop #1 SMP PREEMPT Fri May 31 20:17:40 UTC 2013 (3961086) x86_64
Java Version: Java 1.6.0_17-b04 with Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM mixed mode
--------------------------------------------------------------------------------------------------------
MATLAB                                                Version 8.1        (R2013a)
Curve Fitting Toolbox                                 Version 3.3.1      (R2013a)
DSP System Toolbox                                    Version 8.4        (R2013a)
Filter Design HDL Coder                               Version 2.9.3      (R2013a)
Fixed-Point Designer                                  Version 4.0        (R2013a)
HDL Coder                                             Version 3.2        (R2013a)
HDL Verifier                                          Version 4.2        (R2013a)
Phased Array System Toolbox                           Version 2.0        (R2013a)
Signal Processing Toolbox                             Version 6.19       (R2013a)
>> get(gcf,'Render')

ans =

painters


>> get(gcf,'Render')

ans =

painters

MY UNIX SYSTEM: 
wks-perrinm:/home/perrin # hwinfo --short
cpu:                                                            
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
                       AMD Opteron(tm) Processor 6344                 , 1400 MHz
keyboard:
  /dev/input/event3    ATEN International Hermon USB hidmouse Device
  /dev/input/event0    AT Translated Set 2 keyboard
mouse:
  /dev/input/mice      Kensington USB/PS2 Wheel Mouse
  /dev/input/mice      ATEN International Hermon USB hidmouse Device
graphics card:
                       Matrox MGA G200eW WPCM450
                       nVidia VGA compatible controller
sound:
                       nVidia Audio device
                       ASUSTeK Virtuoso 100 (Xonar DX)
storage:
                       ATI SB700/SB800 SATA Controller [IDE mode]
                       ATI SB700/SB800 IDE Controller
network:
  eth0                 Intel 82576 Gigabit Network Connection
  eth1                 Intel 82576 Gigabit Network Connection
network interface:
  lo                   Loopback network interface
  eth1                 Ethernet network interface
  eth0                 Ethernet network interface
disk:
  /dev/sda             INTEL SSDSC2CW24
  /dev/sdb             ST2000NM0011
partition:
  /dev/sda1            Partition
  /dev/sda4            Partition
  /dev/sda5            Partition
  /dev/sda6            Partition
  /dev/sda7            Partition
  /dev/sda8            Partition
  /dev/sda9            Partition
  /dev/sdb1            Partition
cdrom:
  /dev/sr0             ATAPI iHAS124   D
usb controller:
                       ATI SB700/SB800 USB OHCI0 Controller
                       ATI SB700 USB OHCI1 Controller
                       ATI SB700/SB800 USB EHCI Controller
                       ATI SB700/SB800 USB OHCI0 Controller
                       ATI SB700 USB OHCI1 Controller
                       ATI SB700/SB800 USB EHCI Controller
                       ATI SB700/SB800 USB OHCI2 Controller
bios:
                       BIOS
bridge:
                       ATI RD890 Northbridge only dual slot (2x16) PCI-e GFX Hydra part
                       ATI RD890 PCI to PCI bridge (PCI express gpp port B)
                       ATI SB700/SB800 LPC host controller
                       ATI SBx00 PCI to PCI Bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       ATI RD890 PCI to PCI bridge (external gfx1 port B)
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       AMD Host bridge
                       ATI RD890 Northbridge only dual slot (2x16) PCI-e GFX Hydra part
                       ATI RD890 PCI to PCI bridge (PCI express gpp port B)
                       PLX PEX8112 x1 Lane PCI Express-to-PCI Bridge
hub:
                       Linux 3.4.47-2.38-desktop ohci_hcd OHCI Host Controller
                       Linux 3.4.47-2.38-desktop ohci_hcd OHCI Host Controller
                       Linux 3.4.47-2.38-desktop ohci_hcd OHCI Host Controller
                       Linux 3.4.47-2.38-desktop ohci_hcd OHCI Host Controller
                       Linux 3.4.47-2.38-desktop ohci_hcd OHCI Host Controller
                       Linux 3.4.47-2.38-desktop ehci_hcd EHCI Host Controller
                       Linux 3.4.47-2.38-desktop ehci_hcd EHCI Host Controller
memory:
                       Main Memory
unknown:
                       FPU
                       DMA controller
                       PIC
                       Timer
                       Keyboard controller
                       PS/2 Controller
                       ATI SBx00 SMBus Controller
  /dev/ttyS0           16550A
  /dev/ttyS1           16550A
  /dev/ttyS2           16550A
wks-perrinm:/home/perrin # 

Tim Holy

unread,
Jul 24, 2013, 1:04:01 PM7/24/13
to julia...@googlegroups.com
On Wednesday, July 24, 2013 12:29:42 PM Mike Nolta wrote:
> Perrin, could you tell us more about your system?
>
> On my centos 6.3 desktop, your example takes 2.3s the first time and
> 0.049s the second.

Whoa! I'll jump in here too, since mine agrees with Perrin's.

For 40k points, it's consistently 10s of seconds. If the window gets "damaged"
and needs to be redrawn, it's 10s of seconds more. And the profiler says it's
all happening inside of libcairo.

libcairo from Kubuntu 12.04, installed packages: libcairo-gobject2, libcairo-
script-interpreter2, libcairo2, libcairo2-dev, libcairo2-doc,
libcairomm-1.0.-1. Aside from the last, all are version 1.10.2-6.1ubuntu3.

--Tim

Tim Holy

unread,
Jul 24, 2013, 1:26:18 PM7/24/13
to julia...@googlegroups.com
Perrin, what's your libcairo version?

--Tim
> ---------------------------- MATLAB Version: 8.1.0.604 (R2013a)
> MATLAB License Number: 271558
> Operating System: Linux 3.4.47-2.38-desktop #1 SMP PREEMPT Fri May 31
> 20:17:40 UTC 2013 (3961086) x86_64
> Java Version: Java 1.6.0_17-b04 with Sun Microsystems Inc. Java HotSpot(TM)
> 64-Bit Server VM mixed mode
> ----------------------------------------------------------------------------
> ---------------------------- MATLAB
> > <tim....@gmail.com<javascript:>>

Stefan Karpinski

unread,
Jul 24, 2013, 1:51:13 PM7/24/13
to Julia Users
On Wed, Jul 24, 2013 at 1:00 PM, Perrin Meyer <perri...@gmail.com> wrote:
whats strange is that my julia installation appears to not be doing JIT????? (note it takes 29 seconds each time????).  I compile from GIT???

This is definitely not the case. Julia only works by JITing, so you wouldn't be seeing any results if JIT wasn't working. This is something else entirely. I kind of suspect that Matlab and other systems are probably avoiding drawing some of those 40k points at all since a lot of them are bound to be redundant.

Perrin Meyer

unread,
Jul 24, 2013, 2:03:09 PM7/24/13
to julia...@googlegroups.com
/usr/lib64/libcairo.so.2.11200.2

(do you happen to know the magic openSUSE incantation to get installed library versions?)

perrin

Perrin Meyer

unread,
Jul 24, 2013, 2:04:59 PM7/24/13
to julia...@googlegroups.com
sorry, grasping at straws, first time plotting is 62 seconds and then 28 seconds and 28 seconds.... 

perrin

               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.2.0-2785.r900478c44
 _/ |\__'_|_|_|\__'_|  |  Commit 900478c44 2013-07-24 11:11:57
|__/                   |  x86_64-suse-linux

julia> Pkg.update()
Branch devel set up to track remote branch devel from origin.
Already up-to-date.
MESSAGE: Upgrading BinDeps: v0.2.0 => v0.2.1
MESSAGE: Installing URIParser v0.0.0
Cloning into 'URIParser'...
remote: Counting objects: 37, done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 37 (delta 14), reused 31 (delta 8)
Receiving objects: 100% (37/37), 8.11 KiB, done.
Resolving deltas: 100% (14/14), done.

julia> using Winston

julia> a = rand(40000,1); tic() ; plot(a) ; toc()
elapsed time: 62.881795747 seconds
62.881795747

julia> a = rand(40000,1); tic() ; plot(a) ; toc()
elapsed time: 28.99441395 seconds
28.99441395

julia> a = rand(40000,1); tic() ; plot(a) ; toc()
elapsed time: 29.64797414 seconds
29.64797414

julia> 

Mike Nolta

unread,
Jul 24, 2013, 2:05:21 PM7/24/13
to julia...@googlegroups.com
Perrin, are you running KDE?

-Mike

Perrin Meyer

unread,
Jul 24, 2013, 2:13:35 PM7/24/13
to julia...@googlegroups.com
yes, its KDE, but i really don't pay much attention to GUI desktops (i have an all black background and i run emacs and matlab from the command line/terminal....) I have a good sys admin and he set this machine up since all i need is a terminal, gcc, emacs, chrome browser, and matlab... 

so its whatever is "stock" on an openSUSE setup.  it had all sorts of window animations and i have managed to turn most everything off... maybe i turned off too much? but matlab is "fast" so probably not. 

i'll run any command or i could do screenshots if helpful. 

thanks

perrin

Tim Holy

unread,
Jul 24, 2013, 2:46:35 PM7/24/13
to julia...@googlegroups.com
On Wednesday, July 24, 2013 01:51:13 PM Stefan Karpinski wrote:
> I kind of suspect that Matlab and other systems are probably
> avoiding drawing some of those 40k points at all since a lot of them are
> bound to be redundant.

My Julia example shows it's trivial to get this kind of performance without
skulduggery. More to the point, Winston on @nolta's system is 1000x faster
than Winston on my laptop, yet it's the same code. Something's fishy.

Does it work fast for you?

--Tim

Keno Fischer

unread,
Jul 24, 2013, 2:58:31 PM7/24/13
to julia...@googlegroups.com
Is there some sort of GPU acceleration at play here?

Tim Holy

unread,
Jul 24, 2013, 2:59:30 PM7/24/13
to julia...@googlegroups.com
We have a culprit: I'm running KDE too (as I'm sure you guessed from the
Kubuntu part). I also typically disable desktop effects. I just tried turning
them on, and it doesn't seem to help.

I'll try another few machines and see what happens.

--Tim

On Wednesday, July 24, 2013 11:13:35 AM Perrin Meyer wrote:
> yes, its KDE, but i really don't pay much attention to GUI desktops (i have
> an all black background and i run emacs and matlab from the command
> line/terminal....) I have a good sys admin and he set this machine up since
> all i need is a terminal, gcc, emacs, chrome browser, and matlab...
>
> so its whatever is "stock" on an openSUSE setup. it had all sorts of
> window animations and i have managed to turn most everything off... maybe i
> turned off too much? but matlab is "fast" so probably not.
>
> i'll run any command or i could do screenshots if helpful.
>
> thanks
>
> perrin
>
> On Wednesday, July 24, 2013 2:05:21 PM UTC-4, Mike Nolta wrote:
> > Perrin, are you running KDE?
> >
> > -Mike
> >
> > On Wed, Jul 24, 2013 at 2:04 PM, Perrin Meyer
> > <perri...@gmail.com<javascript:>>>

Tim Holy

unread,
Jul 24, 2013, 3:01:25 PM7/24/13
to julia...@googlegroups.com
On Wednesday, July 24, 2013 02:58:31 PM Keno Fischer wrote:
> Is there some sort of GPU acceleration at play here?

I wondered that, but I think the opengl backend of cairo requires that the
window manager be opengl-capable. Tk by itself isn't, AFAICT, although there
is Togl. I don't think we're using it, but I could be entirely wrong.

--Tim

Andreas Lobinger

unread,
Jul 24, 2013, 3:19:58 PM7/24/13
to julia...@googlegroups.com

Hello,

a few things collected here:

@Stefan: I'm pretty sure matlab draws all the lines (40k). I'm using quite often a plot(z) to get an overview of the data and if subsampling would apply, i wouldn't recognize outliers or peaks or similar.

@Tim: I'm pretty sure libcairo does a few things more than simply marking a single pixel line. Clipping and coordinate transformation to start with. But i agree with you, if it's possible to replace calling libcairo with local julia code to paint on a surface and use cairo only to map on the screen, it would be a success story.

@All: As my quest to Profile in julia continues, i went off the path of the righteous man and did a small python + cairo example and used cairo-trace and cairo-perf-trace to do the following numbers:

a 10000 line example:

lobi@maroon:~/semademo$ cairo-perf-trace python.4255.trace
[ # ]  backend                         test   min(s) median(s) stddev. count
[  0]      xcb                       python    1.101    1.106   0.98%    6/6
[  0]     xlib                       python    1.108    1.129   1.20%    6/6
[ # ]    image: pixman 0.24.4
[  0]    image                       python    0.520    0.523   0.63%    6/6
[ # ]  image16: pixman 0.24.4
[  0]  image16                       python    0.521    0.521   0.46%    6/6

a 40000 line example:

lobi@maroon:~/semademo$ cairo-perf-trace python.4293.trace
[ # ]  backend                         test   min(s) median(s) stddev. count
[  0]      xcb                       python   21.365   21.976   1.94%    6/6
[  0]     xlib                       python   21.640   21.730   0.69%    5/6
[ # ]    image: pixman 0.24.4
[  0]    image                       python    5.080    5.134   0.61%    6/6
[ # ]  image16: pixman 0.24.4
[  0]  image16                       python    5.125    5.203   0.73%    6/6

(The example itself is simply building a linear x and a random y for a given number of steps, And just adds line_to(x[k],y[k]) for a single stroke).

Libcairo seems to have a problem with exceedingly long paths...

@All: Note: libcairo itself has a few backends, so the actual painting on my box seems to be done by pixmap which seemed to have some dependency on the system architecture (optimized code).

Andreas Lobinger

unread,
Jul 24, 2013, 3:24:37 PM7/24/13
to julia...@googlegroups.com
 i forgot:

[cairo] Does Cairo Use Hardware Acceleration?

http://lists.cairographics.org/archives/cairo/2012-October/023601.html

Tim Holy

unread,
Jul 24, 2013, 3:25:44 PM7/24/13
to julia...@googlegroups.com
On Wednesday, July 24, 2013 12:19:58 PM Andreas Lobinger wrote:
> @Tim: I'm pretty sure libcairo does a few things more than simply marking a
> single pixel line. Clipping and coordinate transformation to start with.

True indeed. But clipping and coord transforms should be on a lower order: you
have to do one of those per point, but you have to set many pixels per line.
It's hard to imagine what could be a 1000-fold more expensive.

> But i agree with you, if it's possible to replace calling libcairo with
> local julia code to paint on a surface and use cairo only to map on the
> screen, it would be a success story.

To be clear, I don't think we should do this, I just wanted a sense for how
large of a computational challenge this is. It seems not very large.

Your profiling investigations into Cairo's performance were very informative
but somewhat disheartening. @nolta, do you see any evidence of such nonlinear
behavior on your system?

--Tim

James Cloos

unread,
Jul 24, 2013, 3:57:31 PM7/24/13
to julia...@googlegroups.com, Tim Holy
>>>>> "TH" == Tim Holy <tim....@gmail.com> writes:

TH> For 40k points, it's consistently 10s of seconds. If the window gets "damaged"
TH> and needs to be redrawn, it's 10s of seconds more. And the profiler says it's
TH> all happening inside of libcairo.

Same here; gentoo box; no kde; tk/tcl are 8.6. Cairo is git master.

Perhaps, then, it is dependent on the version of tk/tcl?

If, instead, I run:

using PyCall
@pylab as plt
z = rand(40000,1) ; tic() ; plt.plot(z) ; toc()

it reports about 0.7 seconds for the first run, and 0.0333 for
subsequent runs of that last line.

Cairo shouldn't need that long to plot the lines. A test for that would
be to create the plot as a pdf and view it in evince.

This snippet:

using Winston
z=rand(40000,1);tic();p=FramedPlot();Winston.compose_plot(p,z);file(p,"rand40k.pdf");toc()

takes 3.286784993 seconds here.

mupdf needs about 24 s, evince about 7 seconds to view that pdf.

Just to confirm, I used mupdfclean -d to decompress the pdf and:

:; tr \ \\n <rand40k.pdfc|egrep -c '^l$'

reported 40109 lineto calls, so testing Winston’s pdf output against its
tk output is valid.

Winston.display() should not require any more time than that.

I tried specifying gtk in Winston.ini, but that fails with:

julia> using Winston
WARNING: start_timer now expects arguments in units of seconds. you may need to update your code
in start_timer at deprecated.jl:231

julia> z = rand(40000,1) ; tic() ; plot(z) ; toc()
ERROR: type Canvas has no field draw
in display at /home/cloos/.julia/Winston/src/gtk.jl:42
in gtk at /home/cloos/.julia/Winston/src/gtk.jl:29
in _plot at /home/cloos/.julia/Winston/src/plot.jl:99
in plot at /home/cloos/.julia/Winston/src/plot.jl:20

-JimC
--
James Cloos <cl...@jhcloos.com> OpenPGP: 1024D/ED7DAEA6

Tim Holy

unread,
Jul 24, 2013, 4:07:57 PM7/24/13
to julia...@googlegroups.com
I tested two other machines with a Julia installation.

CentOS 6.4.4, libcairo 1.8.8: fast (!)
Another Kubuntu 12.04: slow (not quite as bad as my laptop, but it is beefier)

--Tim

Stefan Karpinski

unread,
Jul 24, 2013, 5:39:13 PM7/24/13
to Julia Users
On Wed, Jul 24, 2013 at 3:19 PM, Andreas Lobinger <lobi...@gmail.com> wrote:
@Stefan: I'm pretty sure matlab draws all the lines (40k). I'm using quite often a plot(z) to get an overview of the data and if subsampling would apply, i wouldn't recognize outliers or peaks or similar.

I wan't talking about sampling. You can look at points before drawing them and tell if they're close enough that you might as well only draw one of them and then only send sufficiently distinct points to the rendering engine. 

Elliot Saba

unread,
Jul 24, 2013, 5:42:49 PM7/24/13
to julia...@googlegroups.com
I was just coming on here to say this.  I don't know if it's what MATLAB does, but it's certainly a neat technology.  Here's a demo of something closely related (they're doing map projections though)
-E

Jameson Nash

unread,
Jul 24, 2013, 6:12:20 PM7/24/13
to julia...@googlegroups.com
If you do Pkg.update(), Gtk should work now. I had forgot to put my updates into METADATA.

j verzani

unread,
Jul 24, 2013, 10:10:43 PM7/24/13
to julia...@googlegroups.com
Not sure this is any help, but on my Mac it starts to take about 4 times as long to plot twice the points after about 1000 points. This doesn't seem consistent with the issue being just with move to, as I would guess that issue would scale with the number of points:

```
julia> using Winston
julia> out = [begin tic(); plot(rand(2^n)); toc() end for n in 1:15]
julia> [2.^(1:15) out [1, out[2:15]./out[1:14]]]
15x3 Any Array:
     2   0.0139679  1       
     4   0.0172905  1.23787 
     8   0.0135485  0.783577
    16   0.0142008  1.04815 
    32   0.0177034  1.24665 
    64   0.0160067  0.90416 
   128   0.0176779  1.1044  
   256   0.0246461  1.39418 
   512   0.0308524  1.25182 
  1024   0.0976547  3.16522 
  2048   0.421769   4.31898 
  4096   1.87874    4.45444 
  8192   7.15302    3.80734 
 16384  20.4316     2.85636 
 32768  80.7785     3.95361
```

On Tuesday, July 23, 2013 7:25:10 PM UTC-4, Perrin Meyer wrote:
winston is currently too slow to be usable on linux with even "moderate" datasets....

is there a path towards usable interactive plotting for "large" datasets? 

is there a unit test for plotting speed i could track? 

thanks!

perrin


julia>  using Winston
julia> z = rand(40000,1) ; tic() ; plot(z) ; toc()
elapsed time: 29.739992104 seconds
29.739992104

MATLAB
>> z = rand(40000,1) ; tic() ; plot(z) ; toc()

Viral Shah

unread,
Jul 25, 2013, 12:04:18 AM7/25/13
to julia...@googlegroups.com
I have seen this on mac too, where as you give more and more points, the plots become quite slow. I had noticed this a few days back, and had forgotten to file a Winston issue.

The specific example in this issue just gave me the spinning beachball of death.

-viral

Chris Foster

unread,
Jul 25, 2013, 1:53:54 AM7/25/13
to julia...@googlegroups.com
On Thu, Jul 25, 2013 at 12:10 PM, j verzani <jver...@gmail.com> wrote:
> Not sure this is any help, but on my Mac it starts to take about 4 times as
> long to plot twice the points after about 1000 points. This doesn't seem
> consistent with the issue being just with move to, as I would guess that
> issue would scale with the number of points:
>
> ```
> julia> using Winston
> julia> out = [begin tic(); plot(rand(2^n)); toc() end for n in 1:15]
> julia> [2.^(1:15) out [1, out[2:15]./out[1:14]]]
> 15x3 Any Array:
> 2 0.0139679 1
> 4 0.0172905 1.23787
> 8 0.0135485 0.783577
> 16 0.0142008 1.04815
> 32 0.0177034 1.24665
> 64 0.0160067 0.90416
> 128 0.0176779 1.1044
> 256 0.0246461 1.39418
> 512 0.0308524 1.25182
> 1024 0.0976547 3.16522
> 2048 0.421769 4.31898
> 4096 1.87874 4.45444
> 8192 7.15302 3.80734
> 16384 20.4316 2.85636
> 32768 80.7785 3.95361

Plotting these timings on a loglog scale, there's a very suspicious quadratic
scaling above about 500 line segments. It's like libcairo is being told to
draw n^2 paths for some reason. Or perhaps after each path is added there's
an event generated to update the screen, which causes the current set of m
paths to be drawn for m in 1:n?

~Chris

Andreas Lobinger

unread,
Jul 25, 2013, 1:59:17 AM7/25/13
to julia...@googlegroups.com

Actually cairo does not draw lines, but fills polygons.

Elliot Saba

unread,
Jul 25, 2013, 3:09:22 PM7/25/13
to julia...@googlegroups.com
Perhaps this is a memory management issue?  Is there a way we can track if cairo is allocating larger and larger buffers to hold things or something?
-E

Tim Holy

unread,
Jul 25, 2013, 5:48:44 PM7/25/13
to julia...@googlegroups.com
On Thursday, July 25, 2013 03:53:54 PM Chris Foster wrote:
> Plotting these timings on a loglog scale, there's a very suspicious
> quadratic scaling above about 500 line segments. It's like libcairo is
> being told to draw n^2 paths for some reason. Or perhaps after each path
> is added there's an event generated to update the screen, which causes the
> current set of m paths to be drawn for m in 1:n?

I was excited because this has an element of plausibility to it, especially
when I noticed that the transition to quadratic behavior occurs near 50ms,
which happens to be our polling interval for Tk.

However, I just tried changing the polling interval to 5s, but sadly it that
didn't help. Not evidence that this theory is wrong, but one easy possibility
didn't work out.

--Tim

Tim Holy

unread,
Jul 25, 2013, 6:13:18 PM7/25/13
to julia...@googlegroups.com
I tried this again, and with your latest push it works (sometimes). When it
produces a plot, it pops up quickly on my system (<0.1s). So it does seem to
be something about the interaction of Cairo & Tk.

Just FYI Gtk.jl is not yet robust as a Winston back-end on my machine;
sometimes it works, sometimes it shows me an empty window, sometimes it pauses
a long time and then segfaults if I hit Ctrl-C.

--Tim

Jameson Nash

unread,
Jul 25, 2013, 6:34:04 PM7/25/13
to julia...@googlegroups.com
Of 5066 samples from @sprofile, while plotting 1/10 of that data, it appears that almost all of the time is spent in the cairo renderer:
5066 ...son/Documents/no-backup/julia/./julia; main; line: 89
...
                                      4882 ...inston/src/renderer.jl; curve; line: 240
                                       4882 ...ia/Cairo/src/Cairo.jl; stroke; line: 353
                                        4882 .../lib/libcairo.2.dylib; cairo_stroke; line: 29
                                         4882 .../lib/libcairo.2.dylib; _cairo_default_context_stroke; line: 28
                                          4882 ...lib/libcairo.2.dylib; _cairo_gstate_stroke; line: 421
                                           4882 ...lib/libcairo.2.dylib; _cairo_surface_stroke; line: 299
                                            4882 ...lib/libcairo.2.dylib; _cairo_quartz_surface_stroke; line: 71
                                             4882 ...ib/libcairo.2.dylib; _cairo_compositor_stroke; line: 248
                                              1    ...ib/libcairo.2.dylib; _cairo_quartz_cg_stroke; line: 77
                                               1 ...ib/libcairo.2.dylib; _cairo_quartz_setup_state; line: 128
                                                1 ...ib/libcairo.2.dylib; _cairo_surface_clipper_set_clip; line: 228
                                                 1 ...ib/libcairo.2.dylib; ...ce_clipper_intersect_clip_path; line: 131
                                                  1 ...ions/A/CoreGraphics; CGContextRestoreGState; line: 32
                                                   1 ...ons/A/CoreGraphics; CGGStackRestore; line: 60
                                                    1 ...ons/A/CoreGraphics; CGGStateRelease; line: 39
                                                     1 ...ns/A/CoreGraphics; CGClipStackRelease; line: 52
                                                      1 ...ns/A/CoreGraphics; CGClipRelease; line: 17
                                              4881 ...ib/libcairo.2.dylib; _cairo_quartz_cg_stroke; line: 892
                                               4881 ...ons/A/CoreGraphics; CGContextDrawPath; line: 172
                                                4    ...ces/libRIP.A.dylib; ripc_DrawPath; line: 585
                                                 2 ...rces/libRIP.A.dylib; ripr_Path; line: 154
                                                  2 ...ions/A/CoreGraphics; CGPathApply; line: 46
                                                   2 ...ons/A/CoreGraphics; ...7CGPathElementTypePK7CGPointE; line: 41
                                                    2 ...ons/A/CoreGraphics; ...CGPathElementTypePK7CGPointE; line: 41
                                                     2 ...ns/A/CoreGraphics; ...CGPathElementTypePK7CGPointE; line: 56
                                                      2 ...ns/A/CoreGraphics; ...CGPathElementTypePK7CGPoint; line: 171
                                                       2 ...s/libRIP.A.dylib; ripr_path_stroke; line: 249
                                                        2 .../A/CoreGraphics; ...K7CGPointP13path_iterator; line: 1178
                                                         1 .../A/CoreGraphics; ...mentEP13path_iteratordddd; line: 551
                                                         1 .../A/CoreGraphics; ...mentEP13path_iteratordddd; line: 597
                                                 2 ...rces/libRIP.A.dylib; ripr_Path; line: 162
                                                  2 ...ions/A/CoreGraphics; ...h_stroke_endPvP13path_iterator; line: 36
                                                   2 ...ons/A/CoreGraphics; ..._stroke_dataP13path_iterator; line: 164
                                                    2 ...ons/A/CoreGraphics; path_iterator_unrollpath; line: 93
                                                     1 ...ns/A/CoreGraphics; aa_lineto; line: 519
                                                      1 ...ns/A/CoreGraphics; aa_add_edges; line: 11
                                                     1 ...ns/A/CoreGraphics; aa_lineto; line: 563
                                                4877 ...ces/libRIP.A.dylib; ripc_DrawPath; line: 606
                                                 4877 ...es/libRIP.A.dylib; ripc_Render; line: 445
                                                  4877 ...es/libRIP.A.dylib; ripr_Coverage; line: 1812


@TimHoly I haven't seen those issues in my usage of gtk+2 (from MacPorts, with quartz backend). If you have a test case or operating system that I should try, please open an issue.

Tim Holy

unread,
Jul 25, 2013, 6:40:52 PM7/25/13
to julia...@googlegroups.com
I see the same thing. Can you email me that profile as a text file? Syntax:
sprofile_tree(stream, true, true). Also, what version of the cairo library was
this? I will go check the source.

> @TimHoly I haven't seen those issues in my usage of gtk+2 (from MacPorts,
> with quartz backend). If you have a test case or operating system that I
> should try, please open an issue.

Will do. I didn't play with it enough to find out anything systematic, but I'll
experiment more.

--Tim

Andreas Lobinger

unread,
Jul 26, 2013, 4:17:03 AM7/26/13
to julia...@googlegroups.com
Hello,

i somehow cannot follow the discussion anymore (gtk2+ ?) and (having problems with Profile) cannot make substantial claims, how and where julia + cairo looses/takes time, a few remarks about what we should expect.

- performance of libcairo is system dependent (on some systems xlib+xRender is the fastest, on some systems pixman is the fastest).
- libcairo (for sure) wasn't designed for path construction and stroking of >10k lines.
-> work around: split the drawing in plot in segments of shorter length (i did that for my python+cairo example and went for 100k lines from 100s to 0.4s)
- if profile says, most of the time is spent in rendering, that's good news, because all other code is efficient in comparison.
- a test set would be nice: 1k/10k/30k/100k lines, in different colors, 1k/10k/30k//100k markers (Points) etc.

Tim Holy

unread,
Jul 26, 2013, 9:41:36 AM7/26/13
to julia...@googlegroups.com
I seem to have a workaround. I found this actually intending to debug the
problem, and so built Cairo from source. When I did so, rendering was fast.

I did this manually, which was quite easy. I then tried building with the
build.jl script, which turned out not to be easy (it tries to compile all
possible dependencies, and one was missing; I finally figured out what it was
and added it. The corrected version has been pushed to the repository).
However, this resulted in a _slow_ Cairo. So for me, on Kubuntu 12.04, manual
is the way to go.

Here are the steps:
Inside ~/.julia/Cairo/deps:
mkdir src
<Now download http://www.cairographics.org/releases/cairo-1.2.14.xz into this
directory>
cd src
tar xfJ cairo-1.2.14.xz
cd cairo-1.2.14
./configure
make
cd ../..
mkdir usr
mkdir usr/lib
cd usr/lib
ln -s ../../src/cairo-1.12.14/src/.libs/libcairo.so ./


--Tim

Mike Nolta

unread,
Jul 26, 2013, 11:24:53 AM7/26/13
to julia...@googlegroups.com
On Fri, Jul 26, 2013 at 9:41 AM, Tim Holy <tim....@gmail.com> wrote:
> I seem to have a workaround. I found this actually intending to debug the
> problem, and so built Cairo from source. When I did so, rendering was fast.
>
> I did this manually, which was quite easy. I then tried building with the
> build.jl script, which turned out not to be easy (it tries to compile all
> possible dependencies, and one was missing; I finally figured out what it was
> and added it. The corrected version has been pushed to the repository).
> However, this resulted in a _slow_ Cairo. So for me, on Kubuntu 12.04, manual
> is the way to go.
>
> Here are the steps:
> Inside ~/.julia/Cairo/deps:
> mkdir src
> <Now download http://www.cairographics.org/releases/cairo-1.2.14.xz into this
> directory>
> cd src
> tar xfJ cairo-1.2.14.xz
> cd cairo-1.2.14
> ./configure
> make
> cd ../..
> mkdir usr
> mkdir usr/lib
> cd usr/lib
> ln -s ../../src/cairo-1.12.14/src/.libs/libcairo.so ./
>

Fascinating. On linux, the only options build.jl passes to cairo/configure are:

"LDFLAGS=-Lusr/lib"
"CPPFLAGS=-Iusr/include -D_SSIZE_T_DEFINED=1"

AFAICT, the _SSIZE_T_DEFINED option only affects the windows build.
Which would seem to imply that the problem isn't in cairo, but one of
the dependencies?

-Mike

PS: is CPPFLAGS missing quotes?

Tim Holy

unread,
Jul 26, 2013, 12:25:42 PM7/26/13
to julia...@googlegroups.com
On Friday, July 26, 2013 11:24:53 AM Mike Nolta wrote:
> Fascinating. On linux, the only options build.jl passes to cairo/configure
> are:
>
> "LDFLAGS=-Lusr/lib"
> "CPPFLAGS=-Iusr/include -D_SSIZE_T_DEFINED=1"
>
> AFAICT, the _SSIZE_T_DEFINED option only affects the windows build.
> Which would seem to imply that the problem isn't in cairo, but one of
> the dependencies?

That's my guess too. Speculating wildly: when I build just the cairo library,
it looks to see what I have installed in my system path, and decides to turn
some option off. Turning it off speeds up the rendering.

> PS: is CPPFLAGS missing quotes?

I think so too. I have a partial fix queued up here:
https://github.com/JuliaLang/Cairo.jl/pull/21

My change from this morning means we'll have to fix another merge conflict.

--Tim

Reply all
Reply to author
Forward
0 new messages