On Sun, Mar 18, 2012 at 12:03, Christoph Hochreiner
<ch.hoc...@gmail.com> wrote:
> the first prototype of the similarity infrastructure is finished and i'd
> like some comments/reviews.
>
> The EDB accesses the similarity service with triggers (former hooks, but
> renamed due to the fact that they actually do not change the EDB-Objects).
> They can be found in [1].
OK, I can life with the argumentation that they change nothing
(although I would still prefer the name hook). Nevertheless in that
case listener might be better than triggers?
> According to Felix there should not be that many
> problems with implementing them into the current EDB infrastructure. The
> basic assumption is that the triggers present in the OSGI registry are not
> existing (--> no overhead). If there are triggers, they are used.
> The triggers [2] themselves are only adapter to the similarity index, that
> call the correct implementation based on service properties.
Do I get it right that there is no implementation right now?
> The current similarity index implementation contains an lucene
> implementation, but it could be replaced with other (almost certainly faster
> services) in the future. Currently there are 4 sample implementations for
> different index implementations. There was an idea to put the configuration
> into config files an load them on the fly (similar to the
> ConnectorDeployerService), but imho there are some downsides/unsolved
> problems.
> - the index creation could use more advanced java code --> problematic to
> store them in config files
> - if i use hotdeploy for the index, there has to be some configuration in
> the trigger (no real hotdeploy possible; you could use some wiring
> techniques, but i don't know if that would be too much overhead)
>
> The pros would be:
> - hotdeploy
> - no configuration code in the similarity service
>
> So overall: WDYT?
> There is a features.xml[4].
Basically you code looks OK, but I don't get it where exactly you
would like to have our feedback. What do you plan to do with those
hooks? Do we like to provide them as part of the engsb? *confused*
kind regards,
Andreas
HeyOn Sun, Mar 18, 2012 at 12:03, Christoph Hochreiner
<ch.hoc...@gmail.com> wrote:
> the first prototype of the similarity infrastructure is finished and i'd
> like some comments/reviews.
>
> The EDB accesses the similarity service with triggers (former hooks, but
> renamed due to the fact that they actually do not change the EDB-Objects).
> They can be found in [1].OK, I can life with the argumentation that they change nothing
(although I would still prefer the name hook). Nevertheless in that
case listener might be better than triggers?
> According to Felix there should not be that many
> problems with implementing them into the current EDB infrastructure. The
> basic assumption is that the triggers present in the OSGI registry are not
> existing (--> no overhead). If there are triggers, they are used.
> The triggers [2] themselves are only adapter to the similarity index, that
> call the correct implementation based on service properties.Do I get it right that there is no implementation right now?
> The current similarity index implementation contains an lucene
> implementation, but it could be replaced with other (almost certainly faster
> services) in the future. Currently there are 4 sample implementations for
> different index implementations. There was an idea to put the configuration
> into config files an load them on the fly (similar to the
> ConnectorDeployerService), but imho there are some downsides/unsolved
> problems.
> - the index creation could use more advanced java code --> problematic to
> store them in config files
> - if i use hotdeploy for the index, there has to be some configuration in
> the trigger (no real hotdeploy possible; you could use some wiring
> techniques, but i don't know if that would be too much overhead)
>
> The pros would be:
> - hotdeploy
> - no configuration code in the similarity service
>
> So overall: WDYT?
> There is a features.xml[4].
On Sun, Mar 18, 2012 at 20:42, Christoph Hochreiner
<ch.hoc...@gmail.com> wrote:
> On Sunday, March 18, 2012 7:46:31 PM UTC+1, Andreas Pieber wrote:
</SKIP>
> So currently there are three different names:
> # hook
> + the concept behind are basically hooks
> - possible collision of names/concepts with weaver hooks
>
> # trigger
> + based in the database name space
> - with this hooks, the elements do not get altered like in pl/sql; there are
> only external effects
>
> # listener
> + describes the functionality of this hooks
>
> WDYT?
I still think Hook is the best name (since the EDB is kind of an SCM
and hook is the name typically used for the scripts interacting with
the process there. Nevertheless my second favor would be listener,
since I've other situations with a trigger... @ChristophGr what do you
think about this? IIRC the Trigger name was invented by you.
OK, after giving the code a second look and reading your detailed
descriptions I'm getting an idea of what you're planing the to (at
least I think so :-)). The question is: is there a way to provide an
option to configure the similarity index in a way that you can adapt
it without changing the code.
Yes: Great, include it into the OpenEngSB per se
No: Include it only in documentation and provide only a library (in a
seperate labs-XXX proejct) and document how this library could be used
in hooks.
I think you answered those questions partly in previous posts;
nevertheless can you summarize them here again to make the decision
process clear for all?
Thank you very much for your effort and kind regards,
Andreas
On 03/19/2012 03:59 AM, Andreas Pieber wrote:
>
> I still think Hook is the best name (since the EDB is kind of an SCM
> and hook is the name typically used for the scripts interacting with
> the process there. Nevertheless my second favor would be listener,
> since I've other situations with a trigger... @ChristophGr what do you
> think about this? IIRC the Trigger name was invented by you.
Listeners is fine with me.
SCM-hooks can also change the content of e.g. the commit-message. Also
they can be executed _before_ an action.
>
>
> OK, after giving the code a second look and reading your detailed
> descriptions I'm getting an idea of what you're planing the to (at
> least I think so :-)). The question is: is there a way to provide an
> option to configure the similarity index in a way that you can adapt
> it without changing the code.
>
> Yes: Great, include it into the OpenEngSB per se
> No: Include it only in documentation and provide only a library (in a
> seperate labs-XXX proejct) and document how this library could be used
> in hooks.
Problem is, that the code relies on the EDB notifying the listeners
which requires the interfaces to be added to the API and some minor
changes to the EDB.
kind regards,
christoph
---------------------------------------------------------------
On Mon, Mar 19, 2012 at 09:45, Christoph Gritschenberger
<christoph.gr...@gmail.com> wrote:
> Hi,
>
> On 03/19/2012 03:59 AM, Andreas Pieber wrote:
>>
>> I still think Hook is the best name (since the EDB is kind of an SCM
>> and hook is the name typically used for the scripts interacting with
>> the process there. Nevertheless my second favor would be listener,
>> since I've other situations with a trigger... @ChristophGr what do you
>> think about this? IIRC the Trigger name was invented by you.
>
> Listeners is fine with me.
> SCM-hooks can also change the content of e.g. the commit-message. Also
> they can be executed _before_ an action.
Well we can also define before and after hooks; and some of them might
also be able to change values. TBH I don't see a problem with that, do
you?
>> OK, after giving the code a second look and reading your detailed
>> descriptions I'm getting an idea of what you're planing the to (at
>> least I think so :-)). The question is: is there a way to provide an
>> option to configure the similarity index in a way that you can adapt
>> it without changing the code.
>>
>> Yes: Great, include it into the OpenEngSB per se
>> No: Include it only in documentation and provide only a library (in a
>> seperate labs-XXX proejct) and document how this library could be used
>> in hooks.
>
> Problem is, that the code relies on the EDB notifying the listeners
> which requires the interfaces to be added to the API and some minor
> changes to the EDB.
Sorry for not being clear on that point. Of course the API needs to be
included; I'm rather talking about the other components (similarity
checker, ...)
Kind regards,
Andreas
kind regards,
christoph
Maybe not all hooks, but definitely some
Kind regards
Send from my mobile. Please excuse the brevity and/or possible auto correction errors.
On Mon, Mar 19, 2012 at 13:26, Christoph Hochreiner
<ch.hoc...@gmail.com> wrote:
> so some news on the codebase:
> - there is a new EDBHook interface that covers the 3 old interfaces [1]
> the reason for this was to reduce the configuration overhead for the new
> EDBHook Implementations
> --> should go into the openengsb codebase; implementation for EDB comes
> from felix m. in 2-3 weeks
+1, but we're not settled on the methods right now tbh. (1) I really
preferred to keep them separated. You should be able to register a
single concern and should not be forced to register to all at once and
simple ignored the call. In addition I don't get who calls the recover
method one. I'm also not sure who and when calls the collision method.
> - the EDBHook represents an industrial usecase implementation for the Hook
> [2]
> --> should be documented how to write this and/or be part of a labs
> project
> - similarity service [3]
Great, we can document them independently in the labs project. IMHO we
should keep the documentation in the the github wiki there (as for all
other lab projects) and also keep the issues there for the lab
project; issues affecting the openengsb should still be created at
issues.openengsb.org
> the Abstract index is not really abstract anymore; there is a default
> implementation (index all fields and search all fields with levenstein
> similarity 80%
> this abstract service should be extended as you can see in the EDBHook -->
> no prject specific configuration is here
Not quite sure this should happen; can you explain please?
> --> similarity service should become an openengsb project like the
> connectors (git, jira, ...)
In that case we need to create a connector/domain. Not sure how this
makes sense. IMHO we should not include such extensions too far into
the core but rather focus on the connector/domain/workflow things and
keep such extensions in labs for ppl interested in those more exotic
use cases.
> IMHO there are some things to do/discuss:
> - renaming index to a less taken name:
> suggestions:
> # ModelIndex
> # SimilarityIndex
I like both and will follow others here. Felix? ChristophGr? Any suggestions?
> - adopt similarity project code according to quality guidelines of openengsb
+1; best to do this at pull requests. IMHO we should start with the
API. Once this is fixed (and cleaned up during the pull-request phase)
I'll create the additional lab project and we'll do the same there in
the pull-request.
WDYT?
Kind regards,
Andreas
>
> best regards
> Christoph
>
>
>
>
> [1] https://github.com/chhochreiner/openengsb-framework/tree/edbtrigger
> [2] https://github.com/chhochreiner/EDBHook
> [3] https://github.com/chhochreiner/similarityIndexPrototype
>
>
Am 2012-03-19 16:42, schrieb Andreas Pieber:
> Hey,
>
> On Mon, Mar 19, 2012 at 13:26, Christoph Hochreiner
> <ch.hoc...@gmail.com> wrote:
>> so some news on the codebase:
>> - there is a new EDBHook interface that covers the 3 old interfaces [1]
>> the reason for this was to reduce the configuration overhead for the new
>> EDBHook Implementations
>> --> should go into the openengsb codebase; implementation for EDB comes
>> from felix m. in 2-3 weeks
> +1, but we're not settled on the methods right now tbh. (1) I really
> preferred to keep them separated. You should be able to register a
> single concern and should not be forced to register to all at once and
> simple ignored the call. In addition I don't get who calls the recover
> method one. I'm also not sure who and when calls the collision method.
Maybe the separation is not necessary. One thing I could imagine is that
we create an abstract class EDBHook, which implements this interface and
simply do nothing in each method. If the user wants to add an EDBHook,
he just create a bundle and extend the methods he want to include in
this hook. The EDB then would then call at specific code positions all
EDBHooks which are in the running OSGi environment.
About the collision method: what collision method? I thought we just
call an EDBHook method which gives us, if at least one EDBHook is
running, similar objects back in the event that some models can't be
updated. When this Hook method shall be called is the work of the EDB.
>> - the EDBHook represents an industrial usecase implementation for the Hook
>> [2]
>> --> should be documented how to write this and/or be part of a labs
>> project
>> - similarity service [3]
> Great, we can document them independently in the labs project. IMHO we
> should keep the documentation in the the github wiki there (as for all
> other lab projects) and also keep the issues there for the lab
> project; issues affecting the openengsb should still be created at
> issues.openengsb.org
>
>> the Abstract index is not really abstract anymore; there is a default
>> implementation (index all fields and search all fields with levenstein
>> similarity 80%
>> this abstract service should be extended as you can see in the EDBHook -->
>> no prject specific configuration is here
> Not quite sure this should happen; can you explain please?
>
>> --> similarity service should become an openengsb project like the
>> connectors (git, jira, ...)
> In that case we need to create a connector/domain. Not sure how this
> makes sense. IMHO we should not include such extensions too far into
> the core but rather focus on the connector/domain/workflow things and
> keep such extensions in labs for ppl interested in those more exotic
> use cases.
I wouldn't do that on the base of connector/domain, but rather handle an
EDBHook implementation simply as a bundle which exports a service for
that (like already written above). Then the user can simply add such a
service himself and the only thing to do is to provide such a service.
In that way it wouldn't go too much into the core, just the api things.
>> IMHO there are some things to do/discuss:
>> - renaming index to a less taken name:
>> suggestions:
>> # ModelIndex
>> # SimilarityIndex
> I like both and will follow others here. Felix? ChristophGr? Any suggestions?
I think ModelIndex is better, since the EDB works with models and the
service does a search for similar models. But I don't know in which way
this feature could be differently used in future.
Kind regards,
Felix
>>> <christoph.gritschenberger@gmail.com> wrote:
On 03/19/2012 08:17 PM, Felix Mayerhuber wrote:
> Hey,
>
> Maybe the separation is not necessary. One thing I could imagine is that
> we create an abstract class EDBHook, which implements this interface and
> simply do nothing in each method. If the user wants to add an EDBHook,
> he just create a bundle and extend the methods he want to include in
> this hook. The EDB then would then call at specific code positions all
> EDBHooks which are in the running OSGi environment.
Well, I'm not so sure about this. This should only be done if there are
a lot of methods to implement. Drools for example handles
AgendaListeners that way. The "DefaultAgendaListener" has about 20
Methods (beforeRuleActivated, afterNodeFinished, ...). And there is a
great chance the user may want to implement several of these methods.
Providing an interface for each method would be quite tedious.
In our case there are only 3(?) methods and mostly you only want to
implement one. In that case I favor an interface for each method.
kind regards,
christoph
Kind regards,
Andreas
On Tue, Mar 20, 2012 at 08:01, Christoph Gritschenberger
>>>>> <christoph.gritschenberger@gmail.com> wrote:
Kind regards,
Andreas
>> >>>>> <christoph.gr...@gmail.com> wrote:
>> >>>>> <christoph.gritschenberger@gmail.com> wrote:
Now I'm confused. What should the PreCommitHook now do? At least now we
havn't a usecase where we need that? And the similarity check isn't
something that needs to be done in the precommit. Only if there is a
problem when you want to update a model. And if that happens, I need the
list of OID's which are similar to that models which have an error.
We need good names since they are crucial for the understandability of
that mechanism.
Kind regards
Felix
Kind regards,
Andreas
Kind regards,
Felix
Kind regards,
Andreas
I would name the hook-interfaces after where they are executed.
The implementations are then named after what they do.
So for every stage there may exist 0..* hook-services registered. They
are executed in order following the standard-osgi ranking-rules (as in
spec).
The ExceptionHandling can be similar to WeavingHooks as specified in
OSGi Core 4.3.
kind regards,
christoph
I would name the hook-interfaces after where they are executed.
The implementations are then named after what they do.
So for every stage there may exist 0..* hook-services registered. They
are executed in order following the standard-osgi ranking-rules (as in
spec).
The ExceptionHandling can be similar to WeavingHooks as specified in
OSGi Core 4.3.
kind regards,
christoph
On 03/20/2012 10:56 AM, Andreas Pieber wrote:
commit(EKBCommit commit) throws EDBException{
try {
check(commit);
} catch (Exception e) {
similar = invokeAllOsgiServices(FindSimilar.class, a list of all
models where the check has failed because of specific errors);
throw new EDBException("Incompatibles models which shall be
updated {models}. Maybe you meant {similar}.");
}
// rest of the commit part
}
Kind regards
Felix
E.g.
invokeAllOsgiServices(PreCommitHook.class,
allValuesToCommitAsReferenceListClientsCanModify) throws
AbortException, LogAndContinueException,
...otherControllingException...
Kind regards,
Andreas
Kind regards
Felix
Kind regards,
Andreas
But from my point of view I don't know exactly what the problem with the
approach is with my previous thing:
commit(EKBCommit commit) throws EDBException{
try {
check(commit);
} catch (Exception e) {
similar = invokeAllOsgiServices(FindSimilar.class, a list of all models where the check has failed because of specific errors);
throw new EDBException("Incompatibles models which shall be updated {models}. Maybe you meant {similar}.");
}
// rest of the commit part
}
In that way it would be possible that:
- if such a service is present in the OSGi environment, the error would
also return similar EDB objects.
- if such a service isn't present, the error would not append similar
EDB objects.
That would have the fealing of a plug-in for me. The only thing to
discuss here would be if there are other feature requests where this
approach would also be useful.
Kind regards,
Felix
>>>>>>>>>>>>>>>>>> <christoph.gritschenberger@gmail.com> wrote:
----------------------------------------------------------
invokeAllOsgiServices(PreCommit.class, edbCommit)
try{
check(edbCommit)
} catch (Exception e){
for(ErrorHook eh : allErrorHookServices){
if(eh.onError(edbCommit, edbCommit)){
// check again
}
}
throw e;
}
// commit
invokeAllOsgiServices(PostCommit.class, edbCommit)
-----------------------------------------------------------
WDYT?
kind regards,
christoph
> > <felix.ma...@gmail.com <mailto:felix.ma...@gmail.com>>
> >>> <felix.ma...@gmail.com <mailto:felix.ma...@gmail.com>>
> <mailto:felix.ma...@gmail.com>> wrote:
> >>>>>>> We could introduce an ErrorHook "needSimilarModels", but I
> don't know
> >>>>>>> how to
> >>>>>>> name that properly. Also something like that is very
> specific. Maybe
> >>>>>>> we
> >>>>>>> should take a step back think about what our usecases are,
> what we
> >>>>>>> really
> >>>>>>> need and what could be interessting in the future.
> >>>>>>>
> >>>>>>> Kind regards,
> >>>>>>> Felix
> >>>>>>>
> >>>>>>> Am 2012-03-20 10:32, schrieb Andreas Pieber:
> >>>>>>>
> >>>>>>>> what about an error hook?
> >>>>>>>>
> >>>>>>>> Kind regards,
> >>>>>>>> Andreas
> >>>>>>>>
> >>>>>>>> On Tue, Mar 20, 2012 at 10:29, Felix Mayerhuber
> >>>>>>>> <felix.ma...@gmail.com
> >>>>>>>>>>>>>>>> issues.openengsb.org <http://issues.openengsb.org>
> <https://github.com/chhochreiner/EDBHook>
> >>>>>>>>>>>>>>>>> [3] https://github.com/chhochreiner/
> similarityIndexPrototype
> <https://github.com/chhochreiner/similarityIndexPrototype>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Monday, March 19, 2012 10:22:22 AM UTC+1,
> Andreas Pieber
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>> Hey
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Mar 19, 2012 at 09:45, Christoph
> Gritschenberger
> >>>>>>>>>>>>>>>>>> <christoph.gritschenberger@gmail.com
> <https://github.com/chhochreiner/openengsb-framework/tree/edbtrigger>
> >>>>>>>>>>>>>>>>>>>>>>> [2] https://github.com/chhochreiner/EDBHook
> <https://github.com/chhochreiner/EDBHook>
> >>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> https://github.com/chhochreiner/
> similarityIndexPrototype
> <https://github.com/chhochreiner/similarityIndexPrototype>
> >>>>>>>>>>>>>>>>>>>>>>> [4]
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://github.com/chhochreiner/
> similarityIndexPrototype/blob/master/src/main/filtered-
> ressources/features.xml
> <https://github.com/chhochreiner/similarityIndexPrototype/blob/master/src/main/filtered-ressources/features.xml>
> >>>>>>>>>>>>>>>>>>>>>>>
>
sounds good, for the pre and post commit stuff. But for the usecase, for
which the whole thing was intended, I'm not that sure about it. Problem
is: Errors could be of any type and an ErrorHook can handle any type of
error. In the way you posted it, the ErrorHookServices would try to
repair the error and then the commit would be checked again. But the
thing we need for the usecase is: if an error of that specific type
occurs and a similarity service is present, then enrich the exception
that is thrown back to the user with similar objects. If such a service
is not present, return the exception back unenriched. It's quite a
specific thing. But we could of course do it like that and introduce a
new service which only does this specific thing and catch both types and
handle each of them separately. Another way would be to let the called
errorhook append stuff to the exception that will be thrown in the EDB.
But we should think about this ErrorHooks anyway: How many errors in the
EDB could be managed by a service autonomously? The only exceptions that
can occur in the check-phase (at least now) are:
- deletion of a non existing model
- insertion of an already existing model
- update of a model that is not existing
- version conflict
IMHO there is no need and usecase for ErrorHooks, at least I don't see
it. But we need that specific usecase (even though it's most certainly
only a marketing thing which no company would use).
Kind regards
Felix
The Error-hook may return either:
* a new edbcommit-object --> recheck
* a new exception --> throw that one
* null --> nothing...
WDYT?
kind regards,
christoph
Kind regards,
Felix
kind regards,
christoph
Kind regards,
Felix
>>>>>> <felix.ma...@gmail.com<mailto:felix.mayerhuber@gmail.com>>
>>>>>> <felix.ma...@gmail.com<mailto:felix.mayerhuber@gmail.com>>
>>>>>> <mailto:felix.mayerhuber@gmail.com>> wrote:
>>>>>> >>>>>>> We could introduce an ErrorHook
>>>>>> "needSimilarModels", but I
>>>>>> don't know
>>>>>> >>>>>>> how to
>>>>>> >>>>>>> name that properly. Also something like that is very
>>>>>> specific. Maybe
>>>>>> >>>>>>> we
>>>>>> >>>>>>> should take a step back think about what our
>>>>>> usecases are,
>>>>>> what we
>>>>>> >>>>>>> really
>>>>>> >>>>>>> need and what could be interessting in the future.
>>>>>> >>>>>>>
>>>>>> >>>>>>> Kind regards,
>>>>>> >>>>>>> Felix
>>>>>> >>>>>>>
>>>>>> >>>>>>> Am 2012-03-20 10:32, schrieb Andreas Pieber:
>>>>>> >>>>>>>
>>>>>> >>>>>>>> what about an error hook?
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> Kind regards,
>>>>>> >>>>>>>> Andreas
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> On Tue, Mar 20, 2012 at 10:29, Felix Mayerhuber
>>>>>> >>>>>>>> <felix.ma...@gmail.com
>>>>>> <mailto:felix.mayerhuber@gmail.com>> wrote:
>>>>>> <mailto:ch.hochreiner@gmail.com>> wrote:
>>>>>> <mailto:christoph.gritsch...@gmail.com>> wrote:
Yep, imho the error hook should get everything as detailed as we have it. The security implementation can root out the danger of such an approach at another place.
Kind regards,
Andreas
...