skyrails language

[Yose Widjaja]

Hey everyone,

Currently there's a plan to write out the documentations of the skyrails language, and also to produce a paper along the way for the language. This will probably involve changing/adding to the language, so..

Any comments about the language? It's called skyline/runescript (two randomly chosen names), and if anyone has any wishlist for the language, please let me know in this thread. Any requests, ideas, etc will be awesome, and I'll probably work on any ideas that are achievable.

Cheers

--
-- -- -- -- -- -- -- -- -- -- -- -- -- --
Yose Widjaja
Supreme Emperor @ Jormy Games
www.jormy.com
yo...@jormy.com
-- -- -- -- -- -- -- -- -- -- -- -- -- --

[RileyWilliamMurphy]

The ability to easily connect to database sources via ODBC would be incredibly useful and freaking awesome...then you should specify the sql somewhere in the script. I'm not sure how the language works exactly, so I'd imagine this could get tricky...as you'd have to have another way of categorizing the data that is returned (or before it is returned based on column name) from the SQL query...it would be interesting to discuss this possibility.

 

I'd be curious if anyone else in the group has figured this out, or has the same potential need for connecting to a DB (MySQL, Oracle, MS SQL Server, etc)...

[Yose Widjaja]

Is there a library for that, in particular cross-platform libraries?

I've never worked with ODBC, so I don't know. But visualizing SQL queries, that sounds interesting..

[RileyWilliamMurphy]

I'm not sure about how this functionality would be achieved, all I know is that it would be

magnificent.

 

I would imagine that you would pass a particular 'connection string' in the script that references the ODBC connection. The ODBC connection would be named, and set up manually by the user on their computer - and would pass the proper credentials to the database to gain access. Depending on different databases, different ODBC drivers are used.

 

Once you have that ODBC connection set up, - you could reference it with a connection string in the skyrails code/script -  then you could pass SQL queries to it.

 

But I'm sure some extra work would be needed to define the relationships in the resulting data set.

[Andrew Geiger]

To be able to connect to a database would be wildly useful. The
potential for applying this to industrial process data staggers my
mind (at least the kind I work with). That table linking could
possibly be automatically interpreted into useful sorting/
visualizations practically makes me catatonic.

Hyperbole aside, my ignorance of how the backend works overshadows my
enthusiasm for such a handy feature. I'll try to dig up as much on
the topic as I can. Likely, a hack will be needed, but it might not
be too bad to implement. I dearly hope!

On Aug 14, 10:33 am, RileyWilliamMurphy <riley.mur...@gmail.com>
wrote:

> I'm not sure about how this functionality would be achieved, all I know is
> that it would be
> magnificent.
>
> I would imagine that you would pass a particular 'connection string' in the
> script that references the ODBC connection. The ODBC connection would be
> named, and set up manually by the user on their computer - and would pass
> the proper credentials to the database to gain access. Depending on
> different databases, different ODBC drivers are used.
>
> Once you have that ODBC connection set up, - you could reference it with a
> connection string in the skyrails code/script -  then you could pass SQL
> queries to it.
>
> But I'm sure some extra work would be needed to define the relationships in
> the resulting data set.
>

> On Fri, Aug 14, 2009 at 8:22 AM, Yose Widjaja <ywid...@gmail.com> wrote:
> > Is there a library for that, in particular cross-platform libraries?
>
> > I've never worked with ODBC, so I don't know. But visualizing SQL queries,
> > that sounds interesting..
>
> > On Sat, Aug 15, 2009 at 12:55 AM, RileyWilliamMurphy <
> > riley.mur...@gmail.com> wrote:
>
> >> The ability to easily connect to database sources via ODBC would be
> >> incredibly useful and freaking awesome...then you should specify the sql
> >> somewhere in the script. I'm not sure how the language works exactly, so I'd
> >> imagine this could get tricky...as you'd have to have another way of
> >> categorizing the data that is returned (or before it is returned based on
> >> column name) from the SQL query...it would be interesting to discuss this
> >> possibility.
>
> >> I'd be curious if anyone else in the group has figured this out, or has
> >> the same potential need for connecting to a DB (MySQL, Oracle, MS SQL
> >> Server, etc)...
>

> >>   On Fri, Aug 14, 2009 at 5:43 AM, Yose Widjaja <ywid...@gmail.com>wrote:
>
> >>>  Hey everyone,
>
> >>> Currently there's a plan to write out the documentations of the skyrails
> >>> language, and also to produce a paper along the way for the language. This
> >>> will probably involve changing/adding to the language, so..
>
> >>> Any comments about the language? It's called skyline/runescript (two
> >>> randomly chosen names), and if anyone has any wishlist for the language,
> >>> please let me know in this thread. Any requests, ideas, etc will be awesome,
> >>> and I'll probably work on any ideas that are achievable.
>
> >>> Cheers
>
> >>> --
> >>> -- -- -- -- -- -- -- -- -- -- -- -- -- --
> >>> Yose Widjaja
> >>> Supreme Emperor @ Jormy Games
> >>>www.jormy.com

> >>> y...@jormy.com

> >>> -- -- -- -- -- -- -- -- -- -- -- -- -- --
>
> > --
> > -- -- -- -- -- -- -- -- -- -- -- -- -- --
> > Yose Widjaja
> > Supreme Emperor @ Jormy Games
> >www.jormy.com

> > y...@jormy.com

[Yose Widjaja]

Hmm, if it sounds that great, then I guess I really will take a look into it. Was originally going to just write up the language and finalize the design, but this sounds too interesting to pass up.

Cheers

yo...@jormy.com

[Yose Widjaja]

I was looking at unixODBC, iODBC and the windows ODBC. I don't know much about this stuff, does anyone have any recommendations?

Right now, I'm thinking this is how it'll work. Users will specify the visualization spec for the database, using a text-based settings file (not a script file). The settings file will have things such as which record's attributes should be recognized as an edge in the graph, or which records should be recognized as an edge etc..

User then connects to the database, using the settings described in the settings file. He can then do select queries from the skyrails console (or something similar), and the results of the queries will come up as a graph.

User then can clear the result, clear individual nodes, subselect, etc. Maybe delete/modify records directly, but not too sure on that yet.

But yeah, it doesn't seem too complicated atm. What is needed is a method to link to a good ODBC library that is cross-platform (and not GPL'ed etc). And then a further few weeks of work should be good enough..

Any thoughts?

Cheers

[Remy]

Excellent project Yose. I am looking forward to the next release and
database implementation. Most importantly though, I can't wait to get
my hands on some docs :)

Keep it up

On Aug 14, 4:05 pm, Yose Widjaja <ywid...@gmail.com> wrote:
> I was looking at unixODBC, iODBC and the windows ODBC. I don't know much
> about this stuff, does anyone have any recommendations?
>
> Right now, I'm thinking this is how it'll work. Users will specify the
> visualization spec for the database, using a text-based settings file (not a
> script file). The settings file will have things such as which record's
> attributes should be recognized as an edge in the graph, or which records
> should be recognized as an edge etc..
>
> User then connects to the database, using the settings described in the
> settings file. He can then do select queries from the skyrails console (or
> something similar), and the results of the queries will come up as a graph.
>
> User then can clear the result, clear individual nodes, subselect, etc.
> Maybe delete/modify records directly, but not too sure on that yet.
>
> But yeah, it doesn't seem too complicated atm. What is needed is a method to
> link to a good ODBC library that is cross-platform (and not GPL'ed etc). And
> then a further few weeks of work should be good enough..
>
> Any thoughts?
>
> Cheers
>
>
>
> On Sat, Aug 15, 2009 at 11:37 AM, Yose Widjaja <ywid...@gmail.com> wrote:
> > Hmm, if it sounds that great, then I guess I really will take a look into
> > it. Was originally going to just write up the language and finalize the
> > design, but this sounds too interesting to pass up.
>
> > Cheers
>

[Yose Widjaja]

Thanks. I am still trying to decide on the best database library to use though..

I found some that costs $$$ to use. If anyone has any recommendations, please tell me.

Cheers

yo...@jormy.com

[Remy]

DataDirect and OpenLink are two vendors of ODBC drivers

[shadowbq]

I still think you should look at implementing a ruby based DSL. Ruby
like other newer languages (python ,etc) have great capabilities built
in. Ruby is cross platform and already has all the backend
requirements to do just about anything. (You want ODBC.. great.. use
ruby odbc based gems.. ) whatever.. The DSLs allow you to use the same
exisiting mark up that you currently use but allow for much more
flexiblity in configurations.

The odbc connections in the dsl would query the database on the
configuration side and not on the back process side thou..

Take a look at this..
http://www.artima.com/rubycs/articles/ruby_as_dsl.html

[Yose Widjaja]

I actually found a C++ (template) library that seems to handle ODBC stuff, is cross platform, and has a very permissive license : DTL.
http://dtemplatelib.sourceforge.net/

Is implementy a ruby DSL time consuming?

yo...@jormy.com

[shadowbq]

well take a look at the ruby language.. It would be time consuming to
rewrite your all your c code and your current language into ruby
or.....

What I am trying to say is to write a bit of ruby code that can create
the configuration file.

The ruby code /could be a dsl of your existing configuration language/
which would allow a huge set of flexiblity to added to your program
with little effort.

basically you can write this..

in the cleanest idea this would parse in your configuration

(Your language.. )

with all edges do (
itemcolor rgb:20 20 20;
) end;

I would be awesome to be able to leverage a real language in
configuration files.

your language could be mapped so that creates your same
configurations..

(ruby)

edges.each { |edge|
edge.itemcolor("rgb:20 20 20")
}

Learning new languages (like your proprietary scripting language) just
for the sake of one program from the user perspective is silly.


The point is your code here :

with all nodes as x do (
if(#pdbids != null) then (
refindex: "/stru".(getName(@)) ::> (
add(@ c:x); // add the group
describe(@ (c:x::#pdbids), (c:x::#realcolor));
addAction(@ "view", "focus on this node", rgb:0 0 50, (> ./
gesture_select [(focus <- this)] ) );
);
) end;
) end;

is not intuitive enough to understand. sorry :( The at's and the
multitude of colons and <- is not ok.

It took a lot to learn just some of your language...

FYI: You should look at the cytoscape project, it has extensions that
built in capabilities now for ruby, perl, and python bindings..

On Aug 25, 9:29 pm, Yose Widjaja <ywid...@gmail.com> wrote:
> I actually found a C++ (template) library that seems to handle ODBC stuff,

> is cross platform, and has a very permissive license : DTL.http://dtemplatelib.sourceforge.net/

[shadowbq]

( ... I might do it.. but you have not open sourced your code.. )

> ...
>
> read more »

[Yose Widjaja]

Hmm, it might have been a good idea, but what license repercussions would including ruby/python etc have on Skyrails?

Originally, I had to implement a scripting language fast (undergraduate thesis deadline : not much time), and since I have 0 knowledge on ruby, python etc, making my own language was my only choice. And since I only had less than 2 weeks to implement the whole scripting language, it worked really well in the end.

Now, about the most suitable language for implementing a DTL for. Can ruby (or any language), do set arithmetic? For example

for set:proteins.* + set:membranes.* do (
   itemcolor rgb:50 50 50;
);

(which means, every neighbour of nodes in the set proteins, and every neighbour in the set membranes)

You're right that it's not intuitive. Hell, I made some of the operators in a whim, even. But I want to fix up this mess of a language eventually, and this is the best time for it. I'm actually looking at Ruby on Rails for now, because a few friends got me convinced to work in a startup and we've decided to use it.

Anyways, yeah. I'm not against implementing Ruby on Skyrails or anything. I actually had thought of implementing a haskell hookup into the language, and maybe at the same time make Ruby and Python work at the same time, but that was just an undergrad's naive idea.

But by now, I've started to get the feeling that it's best to keep as much logic off the script, and into skyrails. This way, the scripts can move into describing "what" as opposed to "how". This means the scripts will be much less complicated and closer to being a configuration file, rather than a script.

But yeah, I'm going to at least tidy up the set arithmetic bits.

Cheers

p.s Skyrails will probably never be open-sourced

-- -- -- -- -- -- -- -- -- -- -- -- -- --
Yose Widjaja
Supreme Emperor @ Jormy Games
www.jormy.com

yo...@jormy.com
-- -- -- -- -- -- -- -- -- -- -- -- -- --

[Christian Glahn]

Hi Yose and all,

On Fri, Aug 28, 2009 at 3:39 AM, Yose Widjaja <ywi...@gmail.com> wrote:

Hmm, it might have been a good idea, but what license repercussions would including ruby/python etc have on Skyrails?

It depends on what you like to do. Embedding the OSS scripting languages is usually not a big deal as long you follow the rules of LGPL (though this depends on the scripting language you choose to support).
 

Originally, I had to implement a scripting language fast (undergraduate thesis deadline : not much time), and since I have 0 knowledge on ruby, python etc, making my own language was my only choice. And since I only had less than 2 weeks to implement the whole scripting language, it worked really well in the end.

There is nothing wrong about this, as embedding scripting languages correctly into a project has its own pitfalls that can slow the development down. ;-)

Now, about the most suitable language for implementing a DTL for. Can ruby (or any language), do set arithmetic? For example

for set:proteins.* + set:membranes.* do (
   itemcolor rgb:50 50 50;
);

(which means, every neighbour of nodes in the set proteins, and every neighbour in the set membranes)

I think you are talking about syntax here. The narratives may differ a bit. In Perl your code could look like

foreach $edge ( $nodeset->subset(qw(proteins membranes))->edges) {
     $edge->itemcolor_rgb(50,50,50);
}

or for all edges (similar to ruby)

foreach $edge ($nodeset->edges) {
    $edge->itemcolor_rgb(50,50,50);
}

How the code will actually look for a scripting language depends much on what the underlying API of skyrails provides.

You're right that it's not intuitive. Hell, I made some of the operators in a whim, even. But I want to fix up this mess of a language eventually, and this is the best time for it. I'm actually looking at Ruby on Rails for now, because a few friends got me convinced to work in a startup and we've decided to use it.

Anyways, yeah. I'm not against implementing Ruby on Skyrails or anything. I actually had thought of implementing a haskell hookup into the language, and maybe at the same time make Ruby and Python work at the same time, but that was just an undergrad's naive idea.

This makes sense, but this approach would be slightly different to just embedding *one* language. In this case you may consider to have some kind of interaction layer, which bindings for different scripting languages can use to hook into skyrails.
 

But by now, I've started to get the feeling that it's best to keep as much logic off the script, and into skyrails. This way, the scripts can move into describing "what" as opposed to "how". This means the scripts will be much less complicated and closer to being a configuration file, rather than a script.

I agree with that. The scripting language should do only the things that Skyrails cannot do itself. for example to interact with databases or web-services for retrieving or updating datasets. But again, this depends on what the API offers.

Actually, I would prefer an optimized scripting API that allows to provide different plugins for skyrails rather than a bloated application that reimplements well tested solutions.

But yeah, I'm going to at least tidy up the set arithmetic bits.

Cheers

p.s Skyrails will probably never be open-sourced

The open source topic pops up  every now and then in the discussion - and from this thread it seems that there are some people who are willing to help you with improving and porting Skyrails (and are even willing to dive into undocumented code). Since our discussion more than two years ago, your point towards not releasing skyrails as open source has been hardened - although it is difficult to understand why, as you apparently have no real interest in commercializing skyrails.

Best
Christian

[shadowbq]

FYI ruby has a huge community

BIO-RUBY
http://bioruby.open-bio.org/

About Code Syntax.. its just syntax. Note the OO code in ruby
though.. :)

Your Example Code:

for set:proteins.* + set:membranes.* do (
itemcolor rgb:50 50 50;
);

Perl

foreach $edge ( $nodeset->subset(qw(proteins membranes))->edges) {
$edge->itemcolor_rgb(50,50,50);
}

Ruby
%w(proteins membranes).each {|node|
node.edge(:itemcolor_rgb => [50, 50, 50])

This I also agree with..

> I agree with that. The scripting language should do only the things that
> Skyrails cannot do itself. for example to interact with databases or
> web-services for retrieving or updating datasets. But again, this depends on
> what the API offers.

> Actually, I would prefer an optimized scripting API that allows to provide
> different plugins for skyrails rather than a bloated application that
> reimplements well tested solutions.

On this matter of open source.

>> p.s Skyrails will probably never be open-sourced

> The open source topic pops up every now and then in the discussion - and
> from this thread it seems that there are some people who are willing to help
> you with improving and porting Skyrails (and are even willing to dive into
> undocumented code). Since our discussion more than two years ago, your point
> towards not releasing skyrails as open source has been hardened - although
> it is difficult to understand why, as you apparently have no real interest
> in commercializing skyrails.

I would like at least clarification of the current license. I can not
use this (period) with out some sort of license agreement. As I said
before, I (my company) would be interested / willing to pay for a user/
developer license of skyrails but there is nothing that I can find. I
would like to try to implement this software on my companies purchased
"touch-table"(http://www.touchtable.com/) but can not due to lack of
licensing. (FYI: I do not work for TouchTable, Inc.)..

So sell it or license it somehow (opensource [gpl v2, bsd, apache,
mit, gplv3, cc, corporate EULA, software as lease, no commercial
use, .. whatever..) so that the use of this software is clarified.

[shadowbq]

I finally found this... (in the release notes of the interactorium
release..)

You can use Skyrails / the Interactorium for any non-commercial
research purposes (ie.
academic purposes). Please remember to cite the Interactorium paper.
Commercial use of Skyrails or the Interactorium is prohibited. If you
need to use it for such
purposes, please contact yo...@jormy.com

[Yose Widjaja]

Actually, a while ago I did have interest in commercializing it, except there were some legal mumbo-jumbo that I'd had to go through due to the University's IP policy. I had talks about it with various people but nothing ever worked out.

I also think that open-sourcing it will just put too much strain on me. 50k lines worth of c++ code (that's source lines of code), all undocumented and sometimes having strange class names, will just make it hard for others to do anything on it. They're not duplicate codes or anything too, as I think the system structure itself is quite modular (if I do say so myself). There's code in there that manages networking, language parsing, graphics, SDL input handling (including joypads/joysticks).. etc.

At least in the next few years I don't think Skyrails will be open-source or at least licensable.

My plan for growing skyrails is through me finishing my PhD (the ODBC and writing up the language thing is part of this), founding an analytics company, and slowly writing documentation for it. Then maybe, it can be open-sourced.

But right now, open-sourcing it is just out of the question.

Cheers

p.s I'm thinking of allowing for an API to the system. Not sure how to do this yet, what's the best way for a c++ program (say, in windows, compiled with g++) to let other programs call c++ functions in it?

p.p.s thanks for the examples on iterators for ruby etc, but what I was looking for is set operators. Things like
(membranes + proteins).*.* // this means second neighbors of this set
I think for the scripting language at the least, a graph-specific language would be the best. If there's a good, graph processing language that is elegant to use, I'd like to take a look.

[Yose Widjaja]

Oops, forgot about the commercial usage thing.

What kind of usage are you after? I can just let you use it for a particular thing if you really need it, as long as Skyrails is credited for it. I put that "prohibited for commercial" notification because I haven't thought of things like that thoroughly. Maybe mail me privately for that.