Greetings All,
I've like to revive this thread with a description of a protocol, architecture, and implementation details for a smart agent framework based on Linked Data, which addresses many of Melvin's earlier points. The work draws heavily on ideas from
DARQ, but applies them to web services in general, not just SPARQL endpoints. I would be grateful if you could provide feedback on my design choices, and/or help in developing it into a general recommendation for smart agent deployment.
RDF Universally Distributed Index (RUDI) - is a protocol for web services message routing and response federation. It is essentially an intelligent API Gateway that plugs RESTful services into a hyperdata access layer (HAL).
Actors:
- Client
- Router
- Service
- Sponsor
Architectural Constraints:
1) Stateless - the router is unaware of transaction states
2) Infinite Scrolling - the Client browses Service responses in "scrollable" groups so as not to overwhelm Services, and deep fetching should be discouraged
3) Messaging Queue - the Client and Service interact through a message queue, never directly
4) Planning Support - the Service should provide metadata to support Service call planning and optimization, e.g. number of triples and supported classes, properties and namespaces for SPARQL endpoints
5) Built-in Economy - native support for incentivizing Services through (optional) royalties payments paid for by non-intrusive Sponsor messages, or by clients, or both
6) Distributed Indexing - channels in the message queue should be syndicated to subscribing Services, the Service owner's involvement should not be required in the subscription process
7) Trust - all Actors are identified by Public Key, and the queue should expose a web of trust rating for each
8) Federated Results - Clients should be able to subscribe to a virtual channel corresponding to a request and along which responses from various Services are asynchronously transmitted by the Router
I achieve these goals with Adaptors (6), which are RDF documents that describe how to:
1b) Extract patterns from Client input
2b) Use extracted data to fill placeholders in Service call templates
3b) Transform Service output into Linked Data
4b) How, and who, to charge for adapted responses
A pattern (1b) can be regex with captured groups, a type of RDF class, a type of RDF property, or one of many parsed data types supplied by the system (e.g. dateTime, address, geo, etc). JSON, and other non-XML Service output, is converted to XML automatically by the system. Translations (3b) are provided by XSLT documents whose URLs are contained in the Service call templates. Credentials for the Services are obtained and managed by the Routers, and are accessible to Adaptor authors via reserved placeholders. Adaptors are signed by their authors and submitted to the queue via the Router. Untrusted Adaptors are not accepted. Service trust is based on its SSL Certificate. Adaptors involving Services new to the Router and which require credentials must be manually credentialed by the Router's maintainers. The spec gives no guidance on how such cases are to be caught.
To suppress spam, Client UIs can filter the content in response channels based on trust. Special Services called Sponsors can subscribe to request channels and attach a message to the response in exchange for a micropayment to the Service and Adaptor author (adaptors sign their adaptors and Services prove they own the URL). These payments are based on Public Keys and are managed by the blockchain (e.g. Ethereum). Sponsor channels are SPARQL queries that return trusted Sponsor messages maintained in the queue. Like all channels, the Sponsor query is continually polled and results are placed a non-intrusive inbox (the Sponsor channel).

There are several potential business opportunities in this model. A company could provide secure Routers in exchange for a small portion of the Sponsor revenue. Sponsors in turn would subscribe to Routers based on various preferences (Router fees, API coverage, etc). Adaptor authors could earn revenue by adapting existing APIs on the web. Services owners could write Adaptors to plug their own web services into this economy. Since the blockchain manages the economy, users could even attach a promised payment to their requests. Many types of Clients could be built that allow human interaction with Routers. These actors should partner together to build out the architecture and provide open source implementations.
I've been working on a reference Router and some simple Adaptors, which I plan to share sometime next week.
Cheers,
-sherman