extracting distribution from data

[JQ Chen]

 Hi, I'm working on a project for implementing Bayesian networks on financial data

My goals is to create a framework where you can construct a directed graph, being the Bayesian network, and fit it to historical data, in the form of events. Think of it as cloudy/sprinkler/wetgrass. Rather than defining probabilities, it would use it's last posteriori as prior like MCMC, but with real observations rather than generated samples, hopefully converging if given informative conditions.

I came across TFP and I think implementing it in it would be perfect right?

I'm not sure what the best way is to model this. My POC uses an event/node class, which contains a CPT for itself and its parents. Each observation would add a count in the correct row, and update probabiliy based on that: 

Here is a POC of what I want, and want to convert to TFP:

a truth-table of 2^(nparents+1) to denote all possible states of the parents and itself.

... par1..par2..self...count..prob

0 True True True 5028 0.900591

1 True True False 555 0.099409
2 True False True 729 0.931034
3 True False False 54 0.068966
4 False True True 62 0.089985
5 False True False 627 0.910015
6 False False True 620 0.145677
7 False False False 3636 0.854323

....

Can someone give me some guidance to achieving this within TFP?

Should I create joint distributions for each node? or a batch of Bernoulli's? 

My goal was to be able to cut down on the size of parameters (opposed to a joint dist of all features, or having a fully-connected NN), by giving constraints to the network in the form of dependencies in the enviroment.

I hope I was clear, thanks in advance.

If TF team is somewhat interested, I'd like to stay in contact. 

Let me know here or at

j...@hegde.ml

Thanks!

[JQ Chen]

To make it more clear; 

Given a dataset of boleaan/categorical features; and given a list of edges (feature_i -> feature_j, .... , ...),

How to construct a Bayesian network, with each event having distributions for all possible states of itself and its parents?

must be able to run 'live'. 

Then following, can I update these distributions on the go efficiently? I want to start with no observations, and train through the dataset, similar to a NN; rather than using all past data in one go. This is also because the BN must be able to run 'live'. 

[Guilherme Namen Pimenta]

Hello.

Recently I made a neural network for three categories of discrete distributions.

I don't know if that's what you want, but I think it can help you.

First you will need to know how many categories (C) and how many parameters (P) you want to calculate.

Create a normal NN.

Create a layer with C*P units and a RESHAPE layer to transform the data into a C by P matrix.

In the last layer for probability distribution use the following code fragment:

EVENT_SHAPE is the number of categories and 3 is the number of paramenters in my final distribution

...

concat = layers.Dense(3*EVENT_SHAPE) (concat)
concat = layers.Activation('relu')(concat)
concat = layers.Reshape((EVENT_SHAPE, 3))(concat)

output = tfpl.DistributionLambda(
        make_distribution_fn=lambda t:  tfd.ZeroInflatedNegativeBinomial(
            total_count=tf.abs( 1e-5 + t[..., 0:1]),
            logits=t[..., 1:2],
            inflated_loc_logits=t[..., 2:3],
            require_integer_total_count=False,
            validate_args=False,
            allow_nan_stats=False),
        convert_to_tensor_fn=tfd.Distribution.mean)(concat)

Another complex thing is how to prepare the label data. The shape must be (NUMBER_OF_SAMPLES, NUMBER_OF_CATEGORIES, 1).

I strongly recommend to you that you do not start creating the probabilistic neural network.

Start with a common network and understand what is the best architecture for your problem.

After that, start the studies to turn it into a probabilistic neural network.

Remember that this is a data-driven science, not the other way around.