Updating OpenSocial Compliance Tests...

[Mark W.]

OK... Jonathan Beri has volunteered to get us organized and update the
OpenSocial compliance pages. Jonathan will need some help, so this is
your call to action!

Here are the links:
http://code.google.com/p/opensocial-resources/wiki/ComplianceTests
http://code.google.com/p/opensocial-resources/wiki/ComplianceTestsDirectory

Thanks in advance to Jonathan for jumping in on this!

-Mark W.

[Jonathan Beri]

Here's a breakdown of the exisiting docs:

Test Directory: http://code.google.com/p/opensocial-resources/wiki/ComplianceTestsDirectory

Compliance Matrix: http://opensocial-compliance.appspot.com/

Compliance Test overview: http://code.google.com/p/opensocial-resources/wiki/ComplianceTests

How to use the framework: http://code.google.com/p/opensocial-resources/wiki/OpenSocialTestFramework

Codeing style: http://code.google.com/p/opensocial-resources/wiki/ContributingToTestFramework

Main contributor is Lane & more recently Chris from MySpace. Can
either of you give an update on where the tests are?

On Oct 6, 10:45 am, "Mark W." <weitzelm....@gmail.com> wrote:
> OK... Jonathan Beri has volunteered to get us organized and update the
> OpenSocial compliance pages. Jonathan will need some help, so this is
> your call to action!
>

> Here are the links:http://code.google.com/p/opensocial-resources/wiki/ComplianceTestshttp://code.google.com/p/opensocial-resources/wiki/ComplianceTestsDir...

[Mark W.]

Jonathan,

Thanks again for starting to pull this together.

I'd like to reach out to the Jive team as well. They just did a big
announce for their app marketplace and I'm wondering if helping rev
this kind of compliance suite to the 1.0 and 1.1 levels would help
them. I've copied Matt T., I don't know what they are doing for
compliance now, but I'd imagine they've got something...

Going through this, it turns out there's also code in the SVN of the
OpenSocial-Resources project. If we are going to spend time revamping
this, I'm wondering if it makes sense to simply consolidate the
testing in one place and move this code to Shindig. We might also be
able to make it part of the shindig tests. I've copied Paul L. on this
note for his thoughts.


-Mark W.

On Oct 7, 1:53 pm, Jonathan Beri <jonathan.b...@magento.com> wrote:
> Here's a breakdown of the exisiting docs:
>

> Test Directory:http://code.google.com/p/opensocial-resources/wiki/ComplianceTestsDir...

>
> Compliance Matrix:http://opensocial-compliance.appspot.com/
>
> Compliance Test overview:http://code.google.com/p/opensocial-resources/wiki/ComplianceTests
>

> How to use the framework:http://code.google.com/p/opensocial-resources/wiki/OpenSocialTestFram...
>
> Codeing style:http://code.google.com/p/opensocial-resources/wiki/ContributingToTest...

>
> Main contributor is Lane  & more recently Chris from MySpace. Can
> either of you give an update on where the tests are?
>
> On Oct 6, 10:45 am, "Mark W." <weitzelm....@gmail.com> wrote:
>
>
>
> > OK... Jonathan Beri has volunteered to get us organized and update the
> > OpenSocial compliance pages. Jonathan will need some help, so this is
> > your call to action!
>

> > Here are the links:http://code.google.com/p/opensocial-resources/wiki/ComplianceTestshtt......

[Paul Lindner]

I'm sure one could migrate the test gadgets to the shindig repository.  Apache is a bit pickier about getting commit access though, so it might prove to be more of a hassle if we want to have many collaborators.

If we could associate the gadgets with the opensocial.org domain that would be best option. 

--
Paul Lindner -- lin...@inuus.com -- linkedin.com/in/plindner

[Jonathan Beri]

Is the google code project no longer used?

> Paul Lindner -- lind...@inuus.com -- linkedin.com/in/plindner

[Mark W.]

The project at google code, opensocial-resources, is used for spec
development. If you look at the SVN, you'll see the xml documents that
make up the spec. There is other stuff in the svn as well. This is
where the current set of tests are, some examples, etc... We will
continue to use this for the spec, in fact, if you look at the issues
that have been opened, this is where we are tracking all the changes
to the spec. [Of course, you knew this already because you, I'm sure,
are in the process of reviewing the spec, the open items, and voting
on them accordingly. ;-) ]

So we have a choice... We can update the compliance tests in the
opensocial-resources project, or we can consolidate things to Shindig.

It's my hope that we can start to simplify and clean up OpenSocial's
web presence. That is, we've got OpenSocial.org, wiki.opensocial.org,
opensocial-resources, shindig, not to mention all the google groups
like this one.

If we can get all code, tests, required spec prototypes, etc... moved
to Shindig, and have any new artifacts created there as well, then
this starts to draw some clear lines for the community:
* Code artifacts at Shindig
* Spec artifacts at opensocial-resources
* Design documents & documentation in general at wiki.opensocial.org
* Community landing page - opensocial.org (although this needs some
serious work as of now)

Part of the motivation for this is because we want to make sure we
have clean IP for the specification. It's much easier to track commits
that are just for the spec vs. spec, compliance test, examples, etc..
(Granted, it's not impossible, just easier.) Also, I'd like to do
everything to grow the Shindig community.

The opensocial-resources project will continue to be important,
especially since once we close down 1.1, we want to immediately start
working on an implementor's draft of the next version.

That's the net of my current thinking and why I'd like to get
everything in Shindig. I'm up for anything, what do you think?

-Mark W.

[Jonathan Beri]

I think its fine to move to Shindig - more and more Shindig is tightly
coupled to OpenSocial.

One idea I had was to simplify maintaining test by including metadata
into the spec itself & utilizing a generation tool like JSDoc Toolkit.

In the XML, we would need to add a custom tag for MUST, MAY, etc. Then
we would need extend JSDoc to output stub functions.

This would help with documenting the JS apis as well. We actually
started a similar project while I was @ MySpace but the project didn't
go very far.

Thoughts? Other tools to consider?

Thanks.

[Paul Lindner]

Seems fine to me.  Let me know how I can help the effort.

--
You received this message because you are subscribed to the Google Groups "OpenSocial and Gadgets Specification Discussion" group.
To post to this group, send email to opensocial-an...@googlegroups.com.
To unsubscribe from this group, send email to opensocial-and-gadg...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/opensocial-and-gadgets-spec?hl=en.

[Matthew Marum]

Jonathan Beri and I would like to setup a conference call sometime next week to kick off the effort to update the OpenSocial Compliance Tests.

If you are interested in helping or otherwise being involved in this effort, could you please let us know?  That way we can make sure you are included.

Matt Marum

[Mark W.]

+1
I'd also like to see if I can get some other folks to participate as well so some lead time would be good.

[Matthew Marum]

We have a meeting scheduled for Tuesday, February 15th at 3pm EST to kick off this effort.

If you want to be included, please contact me so that I can add you (and share teleconference info with you).

[goosemanjack]

Please include me. Perhaps you could post a conference bridge and
time on this thread.

As far as the compliance suite we submitted, that is specifically
targeting OSML features and EL processing. It is server-side only and
does not exercise any client-side APIs. I believe we have an
extensive suite of javascript container unit tests as well (built on
YUI unit test framework I'm pretty sure). I'll do a little research
and see what we can do.
--
clc

[Matthew Marum]

Tuesday, February 15, 2011 at 3:00:00 PM EST (20:00 GMT)

USA Toll-Free:  888-426-6840

International Access Numbers:
https://www.teleconference.att.com/servlet/glbAccess?process=1&accessCode=9011916&accessNumber=2158616239

Participant Code:  9011916

[Matthew Marum]

Meeting is still on for today, but we had to move it up an hour.

So 2pm EST or 19:00 GMT.

[Matthew Marum]

OpenSocial Compliance Tests Kick-off
Attendees:  Jonathan Beri, Mark Weitzel, Daniel Miles, and Matt Marum

Here are the actions we've agreed to take...  (other attendees feel free to revise if I've missed something or got something wrong)

1) Mine the existing Javascript compliance tests to create QUnit tests as part of the new compliance test framework.
2) Mine the existing REST compliance tests to create tests using a tool like oacurl or RESTClient as part of the new compliance test framework.
3) New compliance test framework will live in Apache Shindig.  However, these tests will not have any dependency on Shindig.
4) Each compliance test is to be linked back to the corresponding part of the OpenSocial specification.  The goal is to require submitting updated compliance tests for acceptance of OpenSocial specification changes.
5) Metadata will be added to OpenSocial specification like new XML tags or attributes to mark assertions that need compliance tests.  An XSLT will be created to generate tests based upon these assertions in the spec.

Action Items:
Matt to start migrating old tests to QUnit
Jon to start investigating adding metadata to spec for generating tests

We will have a 30 minute checkpoint meeting next week.  Details will be posted later.

[Matthew Marum]

I've setup a checkpoint meeting for Wednesday of next week.  All are welcome.

February 23rd, 2-2:30pm EST

Google Calendar Event link
Using this link, you should be able to add yourself to the event so you can get updates, etc.  Let me know if it doesn't work.

[Mark W.]

I just wanted to say THANKS! to Matt & Jonathan for setting this up and working together to drive this. The compliance tests are sorely out of date and this an important piece of work. I hope everyone will find a way to contribute just a little bit.

[Matthew Marum]

By the way, the meeting got rescheduled to same time on Friday because of conflicts.  The calendar entry is up to date.

I've been making some progress. I've got QUnit running within the the gadget container and executing tests much like the old test suite. I've abandoned trying to port any part of the existing test framework. The existing tests are very tightly coupled to the custom framework so I've setted on rewriting each test essentially from scratch. Couldn't do a simple find & replace. I'm not sure how far I want to get on this before we start using the generated framework.

If there's any reference to how QUnit is used by Magento or Jive that I could reference, that would probably be helpful to me (and others) as we work on this.

[goosemanjack]

Settling on QUnit for the client container unit tests is awesome. In
tomorrow's meeting we should also consider how to deal with server-
side (OSML) unit tests. On our unit tests we render the output,
normalize it by stripping white space, then compare the results to an
expected output. The difficulty of this is rendering often drops
hidden codes and JS that will differ across containers. Our solution
was to add an ability to instruct the renderer to not output any
additional markup.

A few things to consider having:

* A defined Mock data set to utilize for unit tests
* Compliance rendering endpoint that would allow POSTing a test gadget

I'll do my best to make tomorrow's meeting.
--
clc

[rbaxter85]

I am going to try to joint the call as well, hope thats OK.

[Matthew Marum]

Of course, all are welcome to join the meeting.  The more people contributing the better.

I've created a Shindig issue to track the tests I've been working on.  Criticism is welcome.  Let's get it right.

https://issues.apache.org/jira/browse/SHINDIG-1510

Matt Marum

[Matthew Marum]

Attendees: Matt Marum, Ryan Baxter, Chris Cole, Andy Smith, Jon Beri

Action Items:
Jon-
Working on gathering some QUnit extension tests from his end.
Investigating OAuth curl as a tool for writing scriptable REST compliance test.
Come up with a framework & transform for linking compliance tests to the assertions in the OpenSocial specification.

Matt-
Continue working on migrating existing tests to QUnit
Investigate Shindig automated tests to see if there's any that could be used as compliance tests

Chris -
To generate/find a useful compliance datastore and attach to Shindig issue.

Meeting again next Friday, same time.  All are welcome.  Follow link to add meeting entry to your calendars.
OpenSocial Compliance Tests 3/4/11 Checkpoint

[goosemanjack]

Created a Shindig issue to attach sample data to.  I'll work on generating the data shortly.

https://issues.apache.org/jira/browse/SHINDIG-1514

[Matthew Marum]

Attendees: Matt Marum, Chris Cole, Ryan Baxter

Jon Beri wasn't able to make it, but did let me know that he will be uploaded his proposed XML markup soon.
Chris will have post sample data to SHINDIG-1514 soon.

I've made progress in porting existing People, AppData, and Activities tests to 1.1 running QUnit.  The patch attached to SHINDIG-1510 is up to date with latest changes.  The patch has not yet been included into shindig.

I'll schedule another checkpoint call for next week.

[Matthew Marum]

FYI.. QUnit based javascript OS 1.1 compliance tests have been checked in to Shindig at shindig-project\content\gadgets\compliance\javascript-tests\1.1

Still a work in progress but people should be able to run them.  Some tests are reliant on the data found in the Shindig's JSON db.

Another checkpoint meeting has been scheduled for this Friday from 2:30-3pm.

Meeting Calendar Entry

[Matthew Marum]

I've created a wiki page to start documenting the new OpenSocial compliance tests.  It has instructions for executing and modifying the test suite (JavaScript tests only right now) within Apache Shindig.

http://docs.opensocial.org/display/OSD/OpenSocial+Compliance+Tests

[Mark W.]

Matt,

This is great start!!!!

A HUGE +1

[Matthew Marum]

I want to spark a discussion on how to write the REST test cases.

From what I can tell, there is nothing in the Server specs that require the use of OAuth.  So it should be possible to write a complete set of REST compliance tests while ignoring OAuth.  However, I know that in production scenarios some containers require OAuth.  So first question is, do we really need to support OAuth out of the gate?

I've investigated scripting the "HTTP work" to a command line tool like oacurl / curl / etc.  They work great but you get back a response that you still need to parse and validate in some way.  So if you're writing a bunch of code to validate responses, why add the level of indirection of calling out to a separate tool?  I feel like I've come full circle on writing a standalone suite (say in JUnit) to run the REST compliance tests.  Thoughts?

[rbaxter85]

Matt OAuth is part of the OpenSocial spec so we should have some tests
for this as well. However I don't think need REST API needs to be
tested using OAuth.

[Bess Ho]

Is it possible to provide OAuth playground similar to Google API Playground?

--
You received this message because you are subscribed to the Google Groups "OpenSocial and Gadgets Specification Discussion" group.
To post to this group, send email to opensocial-an...@googlegroups.com.
To unsubscribe from this group, send email to opensocial-and-gadg...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/opensocial-and-gadgets-spec?hl=en.

--
Bess Ho
UI Architect / Developer / Designer
iPhone Developer
Silicon Valley Web Builder (SVWB) Founder

The information transmitted is intended only for the person or entity to which it is addressed and may contain CONFIDENTIAL material. If you receive this material/information in error, please contact the sender and delete or destroy the material/information.

[Jonathan Beri]

We can easily create a playground using Apigee To-Go. We'll need to set up an instance of Shindig first, but then it takes minutes to set up a console with Apigee.

[Bess Ho]

Would you be able to setup for Open Social API?

On Thu, Mar 24, 2011 at 12:59 PM, Jonathan Beri <jonath...@magento.com> wrote:

We can easily create a playground using Apigee To-Go. We'll need to set up an instance of Shindig first, but then it takes minutes to set up a console with Apigee.

[Jonathan Beri]

Yes, but we need a hosted instance of Shindig to make it happen. I remember us talking about trying to get it up and running on App Engine - has anyone had success with that?

[Paul Lindner]

Regarding a playground on appengine -- Shindig won't run on appengine without serious de-threading.

I can host an instance on my home server.

As far as REST/RPC tests go we should test the auth layer separate from the command/response layer.  The only place where this makes a different really is @me/@self expansion, in which case we may end up needed both.

Also 2LO is often used for makeRequest/osapi.http, it would be good to have a stable endpoint that one can use to test that..

On Fri, Mar 25, 2011 at 8:08 AM, Jonathan Beri <jonath...@magento.com> wrote:

Yes, but we need a hosted instance of Shindig to make it happen. I remember us talking about trying to get it up and running on App Engine - has anyone had success with that?

--

You received this message because you are subscribed to the Google Groups "OpenSocial and Gadgets Specification Discussion" group.
To post to this group, send email to opensocial-an...@googlegroups.com.
To unsubscribe from this group, send email to opensocial-and-gadg...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/opensocial-and-gadgets-spec?hl=en.

--
Paul Lindner -- lin...@inuus.com -- linkedin.com/in/plindner