Architectural considerations - time series plugin

[Martin Cerny]

Hi,

I realized that what large part of what I am building for my transcriptional network inference app is actually a processing and visualisation tool for time series, so I decided to split the time series handling code into separate app - might be useful for someone later. I would like to have the data model to align nicely with the data model of the core apps, so I would welcome any feedback on the general architecture of the app. Sorry for the long mail, hope someone reads it :-)

The premise:

My primary application is working with time series of gene expression data in a network of genes.

To generalize a bit, I want to handle numerical data measured at multiple times for a set of elements in the network. Time series may be associated with both nodes and edges. I want a serie to have the same time points (the times when the measurements were taken, not necessarily uniform steps) across all nodes/edges in the network. I also want to be able to query what time series are available and store application specific additional attributes for the time series. I explicitly want to avoid repeating definitions for each node/edge (as is done for example when displaying a series using the EnhancedGraphics app).

My current implementation approach:
Create a new table for the network where each row corresponds to a definition of a single time series - it contains the time points and references to column(s) that store the data in the node/edge table. The actual data may be either stored in a set of columns in the orignal table or as a single list column.

There is a service that creates and destroys TimeSeries objects. The objects themselves are just wrappers around a corresponding CyRow inte time series table (all getters and setters translate directly to calling get/set on the row).

The service also lets you to extract the actual values for a given node/edge.

GUI:

The GUI will let you display JFreeChart visualisation of selected time series either for a selected node/edge in the data panel or directly within the node, letting you map a set of available time series to a custom graphics attribute. There will be a method to directly import CSV as time series providing some small perks over importing the normal way and then creating a time series (e.g. importing a row in the CSV into a list column, guessing time points from column names).

 

Other apps in a similar domain:

I have looked at DynNetwork, which has some goals in common with my approach, but the interval based definition of data seems very cumbersome for the simple use cases I need and it is also not compatible with a lot of Cytoscapes basic functionalities - I want the network to be fully amenable to any processing in Cytoscape.

ANIMO app has a time series component to it, but the source is not available (AFAIK).

EnhancedGraphics handles some of the visualisation tasks I want to be able to handle, but it requires having a relatively tricky string definition separately for each row. Also you can visualise the series only inside the node which is not always practical (the nodes then take a lot of space).

If you have read until here, you are great and thank you a lot!

Martin Cerny

[Scooter Morris]

Hi Martin,
    Very interesting approach.  I would be interested in talking about this in more detail off-list.  I'm particularly interested in combining efforts so that I can begin the implementation of time-series cluster routines.  I was thinking that it might also be nice to think about how to automate animations of time series data (using CyAnimator).

-- scooter

--
You received this message because you are subscribed to the Google Groups "cytoscape-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cytoscape-disc...@googlegroups.com.
To post to this group, send email to cytoscap...@googlegroups.com.
Visit this group at https://groups.google.com/group/cytoscape-discuss.
For more options, visit https://groups.google.com/d/optout.

[Martin Cerny]

Hi,

so I tried to create a prototype for the time series plugin but it seems that Cytoscape currently does not support user-defined tables associated with a network.

To be specific:

After creating the table, I tried to register the table with the network and my custom TimeSeries class (extending CyIdentifiable):

        CyTable timeSeriesTable = cyTableFactory.createTable("Time Series Definitions", "SUID" /*Key column*/, Long.class /*Key type*/, true /*public*/, true/*isMutable*/);
        cyTableManager.addTable(timeSeriesTable);           
        cyNetworkTableManager.setTable(network, TimeSeries.class, TimeSeriesManager.TIME_SERIES_TABLE_CONTEXT /*String constant for context*/, timeSeriesTable);

This mostly works while the session is running - when querying the network for the table, it works:
        network.getTable(TimeSeries.class, TimeSeriesManager.TIME_SERIES_TABLE_CONTEXT); //returns the table

However, from the moment the table is created, it is displayed in "Unassigned tables"
Further, the table is always saved as a global (unassigned) table, so when I save and load the session, the table is no longer associated with the network and the getTable call above returns null.
I also checked the .cys file and table is indeed stored under tables/global.       

The reason for this is that any table whose type is not included in CyTableManagerImpl.COMPATIBLE_TYPES is treated as a global table, regardless of its association with a network - CyTableManagerImpl.getGlobalTables explicitly enumarates all tables and returns those with types not in CyTableManagerImpl.COMPATIBLE_TYPES as global. Currently, CyTableManagerImpl.COMPATIBLE_TYPES includes only CyNode, CyEdge and CyNetwork.

This behavior is reproducible in both 3.3.0 and 3.4.0

Is this the intended behavior? If so, what is the correct way to associate custom data with networks?

If anyone is interested, I can send a minimal test case for this.

Also sorry, if this does not belong to the list and should be filed in RedMine instead - not sure if this is a bug though...

Thanks a lot

Martin

[Matthias König]

Hi Martin and Scooter,
could you include me in the private discussion.

I am also very interested in this functionality. I have mainly kinetic timecourse simulations with the variables/rates being nodes in the Cytoscape networks. It would be nice if there could be some common functionality to deal with timecourse/time data series in Cytoscape on which other plugins could build.
I.e. perhaps we could have some common timecourse/timedata app which handles the storage (Timecourse CyTables) and associated functionality of timecourse data (time course correlations, interpolations, ...) on which other apps are build, i.e. Martin timecourse transcription specific functionality, me ode-based simulation functionality.

@Martin Where is the repository for your code?
The best Matthias