share dictionary between processes

blumenkraft

unread,

Dec 18, 2009, 2:48:11 AM12/18/09

to

Hi,

I want to share dictionary between two distinct processes.

Something like this:

first.py
import magic_share_module

def create_dictionary():
return {"a": 1}

magic_share_module.share("shared_dictionary",
creator.create_dictionary)
while True:
pass

second.py
import magic_share_module
d = magic_share_module.get_shared("shared_dictionary")
print d["a"]

And then run in command line:
python first.py &
sleep 1
python second.py

I have looked at POSH, but it requires master process that will fork
childs. I want read-only sharing between completely unrelated
processes.
Is it possible?

garabik-ne...@kassiopeia.juls.savba.sk

unread,

Dec 18, 2009, 3:07:57 AM12/18/09

to

blumenkraft <voh...@gmail.com> wrote:
> Hi,
>
> I want to share dictionary between two distinct processes.
>

...

> I have looked at POSH, but it requires master process that will fork
> childs. I want read-only sharing between completely unrelated
> processes.
> Is it possible?

Depends on your exact needs - dbm or shelve might be enough, especially for
read only access.

--
-----------------------------------------------------------
| Radovan Garabík http://kassiopeia.juls.savba.sk/~garabik/ |
| __..--^^^--..__ garabik @ kassiopeia.juls.savba.sk |
-----------------------------------------------------------
Antivirus alert: file .signature infected by signature virus.
Hi! I'm a signature virus! Copy me into your signature file to help me spread!

Michele Simionato

unread,

Dec 18, 2009, 3:10:01 AM12/18/09

to

Yes, see http://docs.python.org/library/multiprocessing.html

News123

unread,

Dec 18, 2009, 3:35:11 AM12/18/09

to

Hi Michael,

I'm new to the module multiprocessing, but at a first glance
it seems, that multiprocessing.Value can only be shared if
you create the second process from the first one.

Id like to start the first process from the command line and much later
the second process from the command line.

Is this possible?

thanks in advance and bye

N

Daniel Molina Wegener

unread,

Dec 18, 2009, 5:32:04 AM12/18/09

to

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On Vie 18 Dic 2009 04:48,
blumenkraft wrote:

Hello,

Did you tried the mmap extension, plus the cPickle
extension?

Also you need a kind of semaphore or mutex to restrict
the memory access and create the proper concurrent access
to shared memory.

iQIcBAEBCgAGBQJLK1okAAoJEHxqfq6Y4O5N+tIQAK0N9l6ymNbWAaStFSP5bV7p
6NYJaPQFUjpGJok1Ip9t8gyVrNRlpy5Z5+JzFGu2G23lJwlv3z6GO/jp2IkzmLSP
tr4kXzLbwq48WeTOsGeBjJ/316B1RAN3H73J+sHikWzPcOSLb8VE+6CUjpMUg6Gn
GIvRoK1BZ41xc7DdUdmNH4G4ycks6eZKo+U8AFxTWQTGkiJ30hoUAJ/x6eXvipyi
H79fTLCGLs1s0uFL/vpQ53LbLlbpM6SaemzTrYGP520QZGouJzLDpkD7f8KHmDdW
tJMD/Wo54dQsjMCwD4wm2H5bIDceauR+0HaGTpKdgOQ0utuvwXATlUqfieFSVgwD
D8v1jDGeRE4EwT/n49S8oJFImjvyshcy/RaaxeoI+s3zcxeUU4ktrKUimJOi5Dgw
ca/R9CSrZpsb7/2iADkvasif6ZbvjmUNB5CTynJ432QTpChr0js9nelGvqzmsbtp
Z7y4aKIt74XgbEcVQKF4YpurzeeSdZSgop2zyAd/O3A2sJR2pO6ZBNiVLM8KQDmT
RL/bgrG2q6r8D+/g0k3RONGgM3n2LvCchPUDQX/OCj5QcfavbPZotElXifR6Ziq1
+EYLadbhDBvIScThixmNisWO26Kh1Clo+MVZxr+XsrbubDCBNyWJYgW80s8xcntb
A42+GNQEk5FT6QPWPdnR
=TkTW
-----END PGP SIGNATURE-----

r0g

unread,

Dec 18, 2009, 8:27:30 AM12/18/09

to

Depends if you need to have access to the object itself or merely look
up values in it. If it's the latter you could start a thread in first.py
that offers dictionary lookups via a specific UDP socket. Any number of
subsequent progs and processes could then pitch up and lookup whatever
values they like whenever they like. ~10 lines of code in first.py,
maybe 5 in second.py

Roger.

Wolodja Wentland

unread,

Dec 18, 2009, 9:20:42 AM12/18/09

to pytho...@python.org

On Thu, Dec 17, 2009 at 23:48 -0800, blumenkraft wrote:
> I want to share dictionary between two distinct processes.
> Something like this:
>
> first.py
> import magic_share_module
> def create_dictionary():
> return {"a": 1}
>
> magic_share_module.share("shared_dictionary",
> creator.create_dictionary)
> while True:
> pass

Have a look at multiprocessing.Manager() it provides (among other
things) proxies for dictionaries that can be used in different threads.
These are even accessible on different hosts if configures correctly.
--
.''`. Wolodja Wentland <went...@cl.uni-heidelberg.de>
: :' :
`. `'` 4096R/CAF14EFC
`- 081C B7CD FF04 2BA9 94EA 36B2 8B7F 7D30 CAF1 4EFC

signature.asc

Steve Holden

unread,

Dec 18, 2009, 12:27:55 PM12/18/09

to pytho...@python.org, pytho...@python.org

Take a look at pyro, though it may be overkill for your needs.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS: http://holdenweb.eventbrite.com/

Steve Holden

unread,

Dec 18, 2009, 12:27:55 PM12/18/09

to blumenkraft, pytho...@python.org

Take a look at pyro, though it may be overkill for your needs.