BatchGenerator using filterStore

[Cyril Schmidiger]

Dear Simpy community
I'm not sure if this is described elsewhere and therefore I would kindly ask for your help:
I'm trying to create a BatchGenerator process which "gets" a specific number of objects from a filterStore. Let's assume that the filterStore contains over a 100 different objectsTypes like A, B, C, D, etc. and that each batch generated may only contain objects of the same type.

I know that filterStore has some interesting features like:
result = yield store.get(lambda store: store.objectType == "D")

But in my case I'm actually trying to do something like this:
"Get 10 objects from the filterStore, no matter which objectType it is. By the way, it is not known which objectTypes to look for e.g. A, B, D, etc. (C might be missing)..

Here is some pseudo code:
batchSize = 10
filterStore capacity = infinite
objectType = A, B, C, D, E, F, ...

t = 1
filterStore contains 4x A, 8x D
--> Unfortunately, no batch can be generated since batchSize of 10 is not reached for any of the available objectTypes

t=2
filterStore cointains 4x A, 12x D
--> Now, the first batch of 10x D can be generated since there is more than 10 objects of type D in the filterStore

Anyone has some ideas? I'm very happy to get some input on this one.
Best regards
C

[Stefan Scherfke]

Hi Cyril,

an interesting problem, that you have there. :-)

The functions itertools.groupby() may help you. Here is a rough scetch:

----
import functools
import itertools

def filterfunc(store, min_count, item):
# keyfunc = lambda i: i.type # for complex types
keyfunc = lambda i: i # for simple types, e.g. strings

sorted_items = sorted(store.items, key=keyfunc)
key_found = None
for k, g in itertools.groupby(sorted_items, keyfunc):
if len(list(g)) >= min_count:
key_found = k
break

if key_found is None:
return False
return keyfunc(item) == key_found

store = FilterStore(...)
filterfunc = functools.partial(filterfunc, store, 10)

def proc(...)
result = yield store.get(filterfunc)

----

Note that this approach has O(n^2) because the the filterfunc is applied to
every object in the store (until one satisfies the filterfunc) and the
filterfunc itself iterates over all objects in the store.

If it gets to slow you may need to implement a custom FilterStore or FilterStoreGet event where you only run the groupby-stuff once before you
iterate all items in the store and apply the filter function.

Cheers,
Stefan

> --
> You received this message because you are subscribed to the Google Groups "python-simpy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to python-simpy...@googlegroups.com.
> To post to this group, send email to python...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/python-simpy/b74a0509-72b4-4cc3-93ad-52a96c30ca3d%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.