Combat API

[Andrew Reutter]

Hi folks,

I've been threatening this for weeks, and am finally getting off my
lazy butt to do something about it. Time to talk about the combat
API. There are now multiple third party products that integrate with
iplaye to load character and even campaign data. They're stunningly
cool - but we can do better.

The basic idea is for iplay4e to be the master source for
character-related combat data, with an arbitrary number of other
products able to simultaneously read from and write to that data in
real-time. I should be able to run a combat using inCombat while
displaying a map using Masterplan, while one of my players is viewing
their character on iplay4e with their iPhone, another is seeing their
sheet embedded on Epic Words, and another is using Google Wave to view
all the characters in the campaign. Furthermore, when a change is
made to a character using any of those mechanisms, it should be
visible in all the products without undue delay.

To pull this off, we'll need a set of APIs, some of which already
exist and are documented, some of which need docs, and some of which
remain to be specified.

These already exist, complete with docs on the main group page:

- Authentication API enabling products to retain iplay4e sessions
using Google Account credentials.
- Search API enabling the listing of campaigns and characters for a
user, the listing of the characters in a campaign, and ad-hoc queries.

These also exist currently but lack documentation (volunteers? ;-):

- Data API describing how to consume character and campaign XML. This
is for static values like max HP, defenses, and so forth. The docs
are currently in the XML itself; I would love for it to move to the
group for consolidation and data size reasons.
- Initialization API describing how to retrieve and consume starting
character state such as current hit points and conditions. Not
documented, but reverse-engineerable because iplay4e itself uses it.
- Update API enabling products to push changes made to characters.
Also reverse engineerable.

Finally, this API is missing:

- Polling API enabling efficient retrieval of state updates for
multiple characters simultaneously.

Before we dive into the specifics of any individual API, I'd like to
nail down this high level list. Am I missing anything?

Cheers,
Andrew

[Christopher Brind]

Hi Andrew,

So far I haven't used iPlay4e in a campaign, but it is definitely on the cards.  

Right now I presume in a campaign only the DM and the Player can update the characters?  

Is it possible to have a scenario where the DM can make it so that he has to approve changes made by a player, or disable certain changes all together?

Examples:

- a player uses a healing surge inappropriately - the DM might deny that change

- the DM disables the player's ability to update their equipment / money

I don't know if I'm inadvertently jumping in to the detail of the polling API, or possibly in to client implementation territory, but is the above supported and if not, do you think it's worth doing?

Thanks,

Chris

--
To unsubscribe, reply using "remove me" as the subject.

[jholmes]

Andrew,

When you go to implement your polling, consider checking out Tornado
(http://www.tornadoweb.org/) for the heavy lifting. You can create a
web server in python in like 5 lines code, but more to point its
designed to handle thousands of incoming connects. SceneGrinder uses
long polling (I know... yuck) but we're ready for html5 websockets
when that matures a tad more. We discovered apache just couldn't
handle it. I don't know how google app engine fairs, though.

Basically, a url gets called to the message server. if there are
messages available, it returns them immediately... they get
processed... and then it goes back and asks again. If there are no
messages pending, the web server just holds on to the connection. We
have it time out in 30 seconds which causes the web client to just try
again. As soon as a message is available, it releases... the web
client processes... and then it tries again.

Long polling is usually a BAD idea... it'll bring a normal web server
to its knees, but Tornado was designed to handle it. I know Google
Wave uses long polling, too, in some cases, so maybe app engine covers
it already.

Jerry

[Christopher Brind]

I'm hoping that they'll add WebSocket support to GAE eventually, but that's not even on the road map at the moment:

http://code.google.com/appengine/docs/roadmap.html

However, comet support is... but who knows what timescales they are working to?  =/

I'm inclined to agree about long polling.  That said, Twitter use it so clients can get streaming/real time updates:

http://apiwiki.twitter.com/Streaming-API-Documentation

But on GAE, since requests are explicitly restricted to a maximum of around 30 seconds on GAE before being interrupted, have to ask what value there is holding a socket open for such a short amount of time anyway.

Another option is that clients register as 'listeners' and then iPlay4e uses URL fetch to send data to registered clients at a specified web address - this API can be designed such that on any other response than success the client stops receiving messages and would have to re-register.  But URLFetch is a whole bunch of different quotas and limits and I'm sure you can see the limitation immediately - the client has to be an accessible web server.

Most likely the polling API will be just that - regular polls by interested clients.  I imagine Andrew will be interested in "what" the API should do rather than "how", since I think there's actually very options for the "how" anyway (it seems to me).

Tornadoweb looks interesting though! :)

Cheers,

Chris

[Jerry Holmes]

Yeah... but here's what we learned the hard way... 30 seconds is an eternity to keep a socket open on a web server. It'll take a hundred or so users to get the dreaded "exceeded connection limit." That's why we use tornado for messaging. It doesn't care.

The reason you want to do it all, rather than just poll at intervals, is that you're holding the socket open you can return and release it as soon as messages are available.  it gives an illusion of immediacy. Otherwise, if you poll to often, you clog your pipes... if  you don't poll often enough, it fells "laggy."

Now... it may that the application can deal with the fixed lag. Users simply may not care.

[Christopher Brind]

Well Twitter uses Jetty on the server for it's streaming API ... so it's obviously possible as they handle hundreds of thousands of concurrent long polling connections (I imagine).  It just comes down to your server implementation.  I'm 100% not surprised that Apache HTTP can't handle it very well - it's very good for static content and can be made to be good for quick request/response CGI type activities, but I dare say it's threading model isn't as sophisticated as it could be given it's age.

One of the servers (Jetty or Tomcat, can't remember) were doing some interesting things with their threads behind the scenes to support exactly this.  I wouldn't be surprised if Twitter had some part in the implementation (in a similar way to their contribution to the likes of Cassandra).

Totally agree with your assessment of the reasons for doing it.  I just don't know how practical it's going to be with GAE.

A couple of other things/options occurred to me:

- GAE supports XMPP - very easy for clients to hook in to this and very fast messaging http://code.google.com/appengine/articles/using_xmpp.html

- Could setup an external service like Tornadoweb and push messages in to it using URL Fetch.  Then clients can long poll against TornadoWeb to receive those messages as they are pushed in. Downside here is having to resource outside of GAE.

It's definitely an interesting problem! :)

Cheers,

Chris

[OverloadUT]

I don't have much insight in to the specifics of how to do this, but I
just want to say that I am extremely excited that you are doing this!

I was going to request exactly this because it's in line with exactly
what my encounter manager tool is going to do. Having iPlay4e handle
the networked bits with the players would be a huge help!

The biggest thing I am interested in which you did not mention, is for
the api to communicate dice rolls from player sheets. I don't know if
this is within the scope of what you're talking about, but that's
definitely my #1 desire.

Thanks,
Greg

[Andrew Reutter]

Boy did I unleash the beast on this one! Thanks, everyone, for your
feedback. I'll be breaking this out into separate threads for the
sake of my own sanity; keep your eyes peeled for the issues of
interest to you personally.

Cheers,
Andrew