Is that a problem? Processes aren't physical objects; they're just memory.
Russ
It wouldn't be able to take down the main IDE with any form of IPC,
although this is probably indeed the best way to do it (provided you
want them to be separate processes).
> still not great for tight integration (realtime pixel editing and the like)
If you're on linux, you're using IPC anyway via X. If you make a
decent protocol (and with netchan, that's pretty easy), things should
work smoothly.
--
Scott Lawrence
--
Scott Lawrence
--
Regards,
-Clark
--
Scott Lawrence
On 7/31/10, RogerV <roger....@gmail.com> wrote:Only for you. The computer doesn't really care.
> Agreed, a process per plugin would get unwieldy for something like
> Eclipse. Hence why was proposing to just statically link the plugins
> into the core program image of the host app, so would all run as one
> happy process.
Sounds like using this model would make writing closed-source software
On 7/31/10, Cory Mainwaring <olr...@gmail.com> wrote:
> Sounds like writing closed-source software would be difficult in this model,
> if not impossible.
difficult, if not impossible. Pity. (Like reverse 'tivoization'?)
</tongue-in-cheek> Well then, closed-source authors will just have to
use an alternate model until/unless go supports/decides-to-support
dynamic loading.
This has the disadvantage that catching and recovering from errors
On 7/31/10, RogerV <roger....@gmail.com> wrote:
> Hence why I proposed to just take the approach of statically linking
> the so-called plugin modules to the core image of the host app - run
> everything in just one process and use goroutines for concurrency.
-could be very difficult- is occasionally downright impossible.
Goroutines share memory. You can't expect your plugins to be
intelligent. If a plugin wanted to use cgo for some feature (an
entirely reasonable assumption, and something any decent
statically-linking plugin system should support), a small bug could
crash everything, despite the fact that the vast majority of the code
is written in a language without pointer arithmetic.
--
Scott Lawrence
Well, every process represents a thread of execution that has to be
scheduled by the OS. Certainly more heavy weight that goroutines.
Hence why I proposed to just take the approach of statically linking
the so-called plugin modules to the core image of the host app - run
everything in just one process and use goroutines for concurrency.
Surely there's no problem in including the Go static linker with an
application's distribution...
Plugins could be object files, yes?
Relinking doesn't mandate recompilation of the plugins, or most
of the core.
Chris
--
Chris "allusive" Dollin
Ship 8(6)(whatever)l.
> For something like Eclipse, the only possible solution I see is to
> have one Go process
> per plugin, making use of process IPC to communicate.
Why? If we're linking a binary, we're done.
> Depending on the OS, this would be quite costly, or not.
>
> If we use Eclipse as example, my current installation has 327 plugins.
> Should we then have so many processes running?
I don't know abouit "should". "Could", perhaps. And "no need to."
Doing plugins by spawning processes is a different choice than
doing them by static linking.
Chris
[And relinking time only needs to compete with Eclipse startup
time, which is not exactly brief. Perhaps an incremental linker
so that some of the linking could be done in advance?]
Only for you. The computer doesn't really care.
On 7/31/10, Cory Mainwaring <olr...@gmail.com> wrote:
> Sounds like writing closed-source software would be difficult in this model,
> if not impossible.
Sounds like using this model would make writing closed-source software
difficult, if not impossible. Pity. (Like reverse 'tivoization'?)
</tongue-in-cheek> Well then, closed-source authors will just have to
use an alternate model until/unless go supports/decides-to-support
dynamic loading.
On 7/31/10, RogerV <roger....@gmail.com> wrote:
> Hence why I proposed to just take the approach of statically linking
> the so-called plugin modules to the core image of the host app - run
> everything in just one process and use goroutines for concurrency.
This has the disadvantage that catching and recovering from errors
For what it's worth, the approach you describe is what xmonad
(http://xmonad.org) uses for plugins. Xmonad itself is essentially a
library, and the configuration file can be a Haskell file, and there's
a little driver that will recompile the configuration file if needed,
or generate a default one if it's not present. Extending that to
automagically load plugins wouldn't be difficult.
You can also dynamically load plugins if you can serialize your
program state, which is what xmonad does.
David
I'd debate the "successfully" there. (I have encountered plugins that
have caused problems.)
--
Scott Lawrence
Note that the total memory used by all these processes isn't
necessarily the naive sum of each process' consumption. The latter
tends to double-count shared resources.
http://sites.google.com/a/chromium.org/dev/memory-usage-backgrounder
has some more detail.
I doubt it's the number of processes that's causing the trouble but
rather the amount of memory used. On my computer, the chrome
processes have resident set sizes ranging from 12m to 246m, whereas
the "top" process I'm using to view memory use only uses 1.4m. Of
course, that's still reasonably bloated, and if you wrote your plugins
efficiently, they should take way under a megabyte of memory, and
running hundreds of them shouldn't be a problem.
--
David Roundy
Given that Hello World in Go uses about 700KB of RAM, I'm a little
dismayed that someone would claim that each process should use "way
under" a megabyte. Static linking makes a "many small processes"
approach somewhat more expensive, and this needs to be taken into
account.
Such an approach probably would take hundreds of megabytes on top of
the memory used elsewhere for the described scenario, and I wouldn't
precisely consider that "not a problem"; indeed, I find that it's been
written off as a concern so quickly rather alarming.
I'm open to solutions other than dynamic loading for writing modular,
live-upgradable software, and I think some others that have been
suggested could be okay, but if this idea was the best Go had to offer
for a "several hundred modules" scenario, I would personally write it in
C.
Yikes! I hadn't paid attention to memory use of small go programs, and
that is pretty crazy! I knew that the binaries were a bit large, but
imagined that the entire executable wouldn't need to be loaded
immediately into memory.
--
David Roundy
It's bigger than we'd like it to be, but it's competitive with C and C++.
If you run a hello world under /usr/bin/time on Linux, it prints
the number of page faults in the statistics. That's the most
reliable number I've found for measuring how much memory
the program actually used. On my Ubuntu 10.04 box, I get:
110 ( 451K) Go println("hello, world")
140 ( 573K) Go os.Stdout.WriteString("hello, world\n")
145 ( 594K) C printf("hello, world\n");
229 ( 938K) Go fmt.Println("hello, world")
313 (1282K) C++ cout << "hello, world\n";
I think the difference with fmt is that fmt imports unicode (indirectly),
and there are some initialized maps in that package that get
created at runtime. If the compiler could lay them out statically
in the data section, I think it would cut the memory usage down.
In fact if fmt.Println can't actually refer to them it would cut them
out of the binary entirely, which would be even better.
There's definitely room for improvement, but we're not doing too bad.
And if you pass in a nil pointer to fmt.Println you get a much
more useful failure than when you do that in C or C++.
Russ
> the Go runtime seems to be around 400k, with fmt adding 300k. So half
> a meg, instead of 3/4ths. Not much, but a big difference at scale.
Working without fmt would mean lacking fmt.Fprintf() and Sprintf() and
the rest while working with IPC; if you're communicating over a
socket or pipe, that seems rather a pain. I thought fmt was actually a
nice example of a module you would be likely to need.
You would also need all other packages needed for whatever your
approach to IPC was, and all packages related to what the module was
actually doing, and internal packages of your program for its types and
functions (likely including an IPC wrapper?), any or all of which could
be significant in size.
All in all, I don't think the plugins being the minimum possible size
for a Go program is likely, even if fmt could be avoided.
You really wanna do IPC with fmt? I thought we were talking rpc package.
The rpc package imports gob, which imports fmt.