emulate a logistic regression model using PSL?

[pli...@gmail.com]

I'm currently trying to build a model for distilling knowledge graphs from text reports (from the cyber-security domain), and I have a quite naive question that I don't quite manage to answer: is it possible to emulate a simple logistic regression model using Probabilistic Soft Logic?

Part of my model involves relying on a number of linguistic cues to determine whether a relation exists between two entities. In other words, I have a number of features that I wish to use to determine the presence/absence of a relation.  I was thinking of using rules such as:

w_1: Feature_1(e_i, e_j) -> HasRelation(e_i, e_j)

w_2: Feature_2(e_i, e_j) -> HasRelation(e_i, e_j)

Would that be equivalent to a logistic regression model? Or would you suggest to another way to express such relations?

(I realise that there are most efficient ways to fit a classification model than through PSL of course.  But this classification task is part of a larger relational structure for which PSL is particularly well-suited).

[Connor Pryor]

Hey,

No, you cannot directly make a Logistic Regression classifier inside PSL. By only using these feature based rules you aren't benefiting from using an SRL framework. Under certain circumstances it will work, though in most situations it is better to just use Logistic Regression.

You can, however, get the benefit of both worlds by incorporating a Logistic Regression model into PSL by directly passing the output into a predicate. Here is an example of the rule:

W1: LogisticRegression(e1, e2) -> HasRelation(e1, e2) ^2

As a quick note, it may be beneficial to use the probabilities outputted from Logistic Regression rather than the binary predictions, since PSL natively handles continuous values.

One direct expansion you might already see, is to treat PSL as a Fusion model combining data from different sources. For example, suppose we were in a recommender setting, in which we are recommending products to users. Two iid methods could be used: matrix factorization for item similarity, and logistic regression for user similarity. Both of these classifiers can be incorporated into the same PSL model:

W1: SimilarUsers(User1, User2) & Recommend(User1, Item) -> Recommend(User2, Item) ^2

W2: SimilarItems(Item1, Item2) & Recommend(User1, Item1) -> Recommend(User1, Item2) ^2

bests,

Connor

[pli...@gmail.com]

OK, thanks! I was hoping to implement the logistic regression as for instance in Markov Logic (https://alchemy.cs.washington.edu/tutorial/4Logistic_Regression.html), but you're right, the easiest is probably to implement the classification outside the PSL model.

[pli...@gmail.com]

(but the main limitation of course is that using an external logistic regression model makes it impossible to learn both the coefficients of the regression model simultaneously with the weights of the PSL model)

[Eriq Augustine]

Hey,

Efficiently and simultaneously learning an external model's parameters and weights in PSL is something that we are currently looking into.

However, we have not seen any successful models that try to add in these independent feature rules (which would constitute a LR classifier) into a PSL/MLN model.

I think that trying to learn a LR model like this would be pretty inefficient.

Normally LR is learned as an optimization problem, but the weight learning methods in PSL and MLN would have much slower iterations.

I think the complexity of the relational model would inhibit the learning of the LR model (when the LR model is created using these feature functions).

Given infinite time, a better LR model will probably be learned.

So we believe that training it externally is currently a better option.

If you end up trying to do this, let us know.

-eriq

On Sat, Apr 4, 2020 at 7:40 PM <pli...@gmail.com> wrote:

(but the main limitation of course is that using an external logistic regression model makes it impossible to learn both the coefficients of the regression model simultaneously with the weights of the PSL model)

--
You received this message because you are subscribed to the Google Groups "PSL Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to psl-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/psl-users/5f67a06d-42ab-4c28-a0a8-2bc067a2d53a%40googlegroups.com.