Re: meet with Scott Morrison?

[Scott Morrison]

ps -- a while ago we made a mailing list at
categor...@googlegroups.com, and I'd vote for trying to cc it on
all technical discussions, for archival purposes.

best,
Scott

On Thu, Feb 21, 2013 at 12:15 PM, Scott Morrison <sc...@tqft.net> wrote:
> Okay, I will try to learn how to write a grammar for a JSON format,
> and produce one!
>
> Metaphor doesn't output CSV at all, so we haven't thought about
> skolems, etc. I've so far been thinking about CSV data as always
> implicitly living on a star-shaped schema, with the first row of data
> in a file giving the names of the boxes.
>
> scott
>
> On Thu, Feb 21, 2013 at 12:09 PM, Ryan Wisnesky <ry...@cs.harvard.edu> wrote:
>> I spent some time today trying to reverse engineer the metaphor text formats
>> by looking at the examples on categoricaldata.net. The formats are
>> intuitive, but I think to actually use them I am going to need more
>> documentation. A formal grammar for the JSON format would be rather useful.
>> I'm happy to provide a formal grammar for FQL if that would be useful (we
>> could put it in the appendix of our paper too, I guess).
>>
>> I also had a few questions about the CSV format:
>>
>> - it seems like CSV is a possible choice for inputting instances. It looks
>> a lot like "categorical normal form", but there are no table names. Are the
>> table names the first column names? Also, does metaphor output instances in
>> CSV as well?
>>
>> - In the JSON format, it appears that schema definitions are contained
>> within instance definitions; can one specify schemas using the CSV format?
>>
>> - As we saw with the FQL implementation, computed instances require
>> constructing values that do not appear in input instances, like the "tags"
>> from a disjoint union or additional IDs during Pi. How are these handled
>> with the CSV format?
>>
>> Perhaps CSV may not be as good as JSON for our purposes…?
>>
>>
>>
>> On Feb 20, 2013, at 9:55 AM, Ryan Wisnesky <ry...@cs.harvard.edu> wrote:
>>
>> Quite possibly. In the interim, if you all have any documentation on the
>> JSON formats themselves that would be useful.
>>
>> On Feb 20, 2013, at 9:49 AM, David Spivak <dsp...@gmail.com> wrote:
>>
>> Thanks for the update. Scott and I have a meeting planned on Tuesday at 7pm
>> EST. Maybe it would be useful if you joined?
>>
>> David
>>
>> On Wed, Feb 20, 2013 at 9:34 AM, Ryan Wisnesky <ry...@cs.harvard.edu> wrote:
>>>
>>> Just a heads up that over the next few weeks I'll be working on converting
>>> FQL to/from the metaphor/JSON text formats.
>>>
>>> On Jan 28, 2013, at 11:24 PM, Ryan Wisnesky <ry...@cs.harvard.edu> wrote:
>>>
>>> > I agree that it should be straightforward to convert FQL schemas,
>>> > instances, and mappings to metaphor/JSON, and to test them in the manner you
>>> > suggest. FQL is implemented in java, so it should be easier to interoperate
>>> > with Scala. I can start working on converters once FQL implements all the
>>> > required functionality from our paper.
>>> >
>>> > On Jan 28, 2013, at 6:10 PM, Scott Morrison <sc...@tqft.net> wrote:
>>> >
>>> >> Hi Ryan, David,
>>> >>
>>> >> sorry to have missed the phone call this morning. I'm down at the
>>> >> beach with some collaborators and completely forgot that I might be
>>> >> interacting with the outside world today.
>>> >>
>>> >> It sounds like you managed okay without me anyway! Here's a rough
>>> >> proposal for a division of labour:
>>> >>
>>> >> 1) Scott makes "find an isomorphism between these two datasets"
>>> >> accessible via the metaphor JSON API.
>>> >> 2) Ryan writes converters between the FQL and metaphor/JSON text
>>> >> formats.
>>> >> 2b) Scott, if considered necessary, writes a formal description of the
>>> >> metaphor/JSON formats.
>>> >> 3) Ryan(?) writes some tests for various commuting squares (i.e.
>>> >> compute something in FQL, vs. convert to JSON, compute in metaphor,
>>> >> and convert back).
>>> >>
>>> >> ...
>>> >>
>>> >> 4) If the converters are easily accessible (e.g. in the JVM, or even
>>> >> just running a process), I could try to write another implementation
>>> >> of my JSON API that calls FQL as the backend, rather than the metaphor
>>> >> backend. (It would be sort of cool to be able to construct URLs which
>>> >> output a dataset that comes from a composition of pushforwards,
>>> >> computed on different computers by different backends.)
>>> >>
>>> >> I know that writing these converters is not particularly glorious, but
>>> >> I think it will be good to gain confidence in both programs. It's also
>>> >> a start towards an "ecosystem" of tools for manipulating categorical
>>> >> data.
>>> >>
>>> >> best,
>>> >> Scott
>>> >>
>>> >> On Thu, Jan 24, 2013 at 12:27 PM, Ryan Wisnesky <ry...@cs.harvard.edu>
>>> >> wrote:
>>> >>> Sure, that works for me.
>>> >>>
>>> >>> On Jan 23, 2013, at 8:01 PM, David Spivak <dsp...@gmail.com> wrote:
>>> >>>
>>> >>> Hi Ryan,
>>> >>>
>>> >>> Tonight I had a conversation with Scott Morrison, a mathematician and
>>> >>> colleague of mine, who wrote a Scala program to perform data migration
>>> >>> queries, similar to yours. That work can be found at
>>> >>> http://www.categoricaldata.net or, say, here. (It's all open-source.)
>>> >>> In
>>> >>> particular he has a JSON interface that was built explicitly to be
>>> >>> independent of the category-theoretic back-end.
>>> >>>
>>> >>> In our conversation tonight, Scott suggested that we might make a
>>> >>> translator
>>> >>> between your textual stuff and his, and then connect your system up to
>>> >>> his
>>> >>> JSON interface and see if the two programs really do produce
>>> >>> isomorphic
>>> >>> output. Doing this would be a good exercise and a useful step in
>>> >>> making
>>> >>> these ideas usable by a larger audience.
>>> >>>
>>> >>> Of course, making this is something I'd be happy to pay you to do
>>> >>> (once that
>>> >>> gets set up). But before we do any of that, he and I thought it would
>>> >>> be
>>> >>> nice if the three of us met up on a phone call. Scott is in Australia,
>>> >>> so
>>> >>> such a meeting has to be later in the day.
>>> >>>
>>> >>> Would you be available for a google-voice or hangout on Monday 1/28 at
>>> >>> 5pm?
>>> >>> Scott and I are both free at that time. I'll also send out a calendar
>>> >>> invitation to this effect.
>>> >>>
>>> >>> Best,
>>> >>> David
>>> >>>
>>> >>>
>>> >
>>>
>>
>>
>>