26 views

Skip to first unread message

Dec 6, 2022, 4:33:03 PM12/6/22

to TensorFlow Probability

Hi, I'm working on a project for implementing Bayesian networks on financial data

2 True False True 729 0.931034

3 True False False 54 0.068966

4 False True True 62 0.089985

5 False True False 627 0.910015

6 False False True 620 0.145677

7 False False False 3636 0.854323

My goals is to create a framework where you can construct a directed graph, being the Bayesian network, and fit it to historical data, in the form of events. Think of it as cloudy/sprinkler/wetgrass. Rather than defining probabilities, it would use it's last posteriori as prior like MCMC, but with real observations rather than generated samples, hopefully converging if given informative conditions.

I came across TFP and I think implementing it in it would be perfect right?

I'm not sure what the best way is to model this. My POC uses an event/node class, which contains a CPT for itself and its parents. Each observation would add a count in the correct row, and update probabiliy based on that:

Here is a POC of what I want, and want to convert to TFP:

a truth-table of 2^(nparents+1) to denote all possible states of the parents and itself.

... par1..par2..self...count..prob

0 True True True 5028 0.900591

1 True True False 555 0.0994092 True False True 729 0.931034

3 True False False 54 0.068966

4 False True True 62 0.089985

5 False True False 627 0.910015

6 False False True 620 0.145677

7 False False False 3636 0.854323

....

Can someone give me some guidance to achieving this within TFP?

Should I create joint distributions for each node? or a batch of Bernoulli's?

My goal was to be able to cut down on the size of parameters (opposed to a joint dist of all features, or having a fully-connected NN), by giving constraints to the network in the form of dependencies in the enviroment.

I hope I was clear, thanks in advance.

If TF team is somewhat interested, I'd like to stay in contact.

Let me know here or at

Thanks!

Dec 6, 2022, 4:37:50 PM12/6/22

to TensorFlow Probability, JQ Chen

To make it more clear;

Given a dataset of boleaan/categorical features; and given a list of edges (feature_i -> feature_j, .... , ...),

How to construct a Bayesian network, with each event having distributions for all possible states of itself and its parents?

must be able to run 'live'.

Then following, can I update these distributions on the go efficiently? I want to start with no observations, and train through the dataset, similar to a NN; rather than using all past data in one go. This is also because the BN must be able to run 'live'.

Dec 12, 2022, 7:36:30 AM12/12/22

to TensorFlow Probability, holy...@gmail.com

Hello.

Recently I made a neural network for three categories of discrete
distributions.

I don't know if that's what you want, but I think it can
help you.

First you will need to know how many categories (C) and how
many parameters (P) you want to calculate.

Create a normal NN.

Create a layer with C*P units
and a RESHAPE layer to transform the data into a C by P matrix.

In the last layer for probability distribution use the following code fragment:

EVENT_SHAPE is the number of categories and 3 is the number of paramenters in my final distribution

...

concat = layers.Dense(3*EVENT_SHAPE) (concat)

concat = layers.Activation('relu')(concat)

concat = layers.Reshape((EVENT_SHAPE, 3))(concat)

output = tfpl.DistributionLambda(

make_distribution_fn=lambda t: tfd.ZeroInflatedNegativeBinomial(

total_count=tf.abs( 1e-5 + t[..., 0:1]),

logits=t[..., 1:2],

inflated_loc_logits=t[..., 2:3],

require_integer_total_count=False,

validate_args=False,

allow_nan_stats=False),

convert_to_tensor_fn=tfd.Distribution.mean)(concat)

concat = layers.Activation('relu')(concat)

concat = layers.Reshape((EVENT_SHAPE, 3))(concat)

output = tfpl.DistributionLambda(

make_distribution_fn=lambda t: tfd.ZeroInflatedNegativeBinomial(

total_count=tf.abs( 1e-5 + t[..., 0:1]),

logits=t[..., 1:2],

inflated_loc_logits=t[..., 2:3],

require_integer_total_count=False,

validate_args=False,

allow_nan_stats=False),

convert_to_tensor_fn=tfd.Distribution.mean)(concat)

Another complex thing is how to prepare the label data. The shape must be (NUMBER_OF_SAMPLES, NUMBER_OF_CATEGORIES, 1).

I
strongly recommend to you that you do not start creating the
probabilistic neural network.

Start with a common network and understand
what is the best architecture for your problem.

After that, start the
studies to turn it into a probabilistic neural network.

Remember that this is a data-driven science, not the other way around.

Reply all

Reply to author

Forward

Message has been deleted

0 new messages

Search

Clear search

Close search

Google apps

Main menu