Align with JSON-RPC? #4

tlrobinson · 2012-12-02T09:19:23Z

I think Matt mentioned this early on: what we have is fairly close to JSON-RPC. Maybe it's worth making it compatible?

JSON-RPC 1.0 uses an array of arguments, but 2.0 supports named parameters, so it would be almost identical to what we have (rename inputs to params, outputs to result).

One nice thing about using JSON-RPC is we could potentially expose WebPipes over more efficient transports like TCP sockets/WebSockets/ZeroMQ. When using HTTP we could drop the id and method field (since the "method" is in the URL). When using a socket, the method would just be the url or path, and id would be used to pair the response with the request.

The text was updated successfully, but these errors were encountered:

progrium · 2012-12-02T17:18:37Z

Not a bad idea.

On Sun, Dec 2, 2012 at 1:19 AM, Tom Robinson [email protected]:

I think Matt mentioned this early on: what we have is fairly close to
JSON-RPC. Maybe it's worth making it compatible?

JSON-RPC 1.0 uses an array of arguments, but 2.0 supports named
parameters, so it would be almost identical to what we have (rename inputsto
params, outputs to result).

One nice thing about using JSON-RPC is we could potentially expose
WebPipes over more efficient transports like TCP sockets/WebSockets/ZeroMQ.
When using HTTP we could drop the id and method field (since the "method"
is in the URL). When using a socket, the method would just be the url or
path, and id would be used to pair the response with the request.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4.

Jeff Lindsay
http://progrium.com

matthewhudson · 2012-12-02T17:42:25Z

I approve. It's also a bit easier to explain to developers as "Federated RPC".

The id field is nice since it essentially behaves as a token/key which is useful in cases where a Block needs to handle state.

tlrobinson · 2012-12-02T17:45:35Z

I don't think the JSON-RPC id field is useful for tracking state since it's
unique per request. I think we'll need another way to identify the pipeline.

On Dec 2, 2012, at 9:42 AM, Matthew Hudson [email protected] wrote:

I approve. It's also a bit easier to explain to developers as "Federated
RPC".

The id field is nice since it essentially behaves as a token/key which is
useful in cases where a Block needs to handle state.

—
Reply to this email directly or view it on
GitHubhttps://github.com//issues/4#issuecomment-10932422.

progrium · 2012-12-02T20:18:32Z

Headers. Either HTTP or if new JSON-RPC supports some kind of headers.

On Sun, Dec 2, 2012 at 9:45 AM, Tom Robinson [email protected]:

I don't think the JSON-RPC id field is useful for tracking state since
it's
unique per request. I think we'll need another way to identify the
pipeline.

On Dec 2, 2012, at 9:42 AM, Matthew Hudson [email protected]
wrote:

I approve. It's also a bit easier to explain to developers as "Federated
RPC".

The id field is nice since it essentially behaves as a token/key which is
useful in cases where a Block needs to handle state.

—
Reply to this email directly or view it on
GitHubhttps://github.com//issues/4#issuecomment-10932422.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4#issuecomment-10932448.

Jeff Lindsay
http://progrium.com

matthewhudson · 2012-12-09T03:08:56Z

A database Block perfectly illustrates the value of a "methods" key in the Block Definition.

{
    "name": "database",
    "description": "Store and fetch persistent data.",
    "url": "/block-endpoint",
    "methods": {
        "get": {
            "inputs": [],
            "outputs": []
        },
        "set": {
            "inputs": [],
            "outputs": []
        }
    }
}

To accomodate this new Block Definition format, a HTTP request would change in the following ways:

The "inputs" wrapper would take the name of the method invoked.
As of WebPipe Spec v0.2 we no longer use the 'inputs' wrapper array.

curl -i -X POST -d '{"methodName":{"inputName":"inputValue", ... }}' -H "Content-Type: application/json" http://block-endpoint

Thoughts?

matthewhudson · 2012-12-09T03:15:40Z

Or, if we went full JSON-RPC:

curl -i -X POST -d '{ "method": "get", "params": ["database_key"], "id": null }' -H "Content-Type: application/json" http://block-endpoint

Using the database Block Definition I mentioned in my last post, this example would return a record for a given database key.

tlrobinson · 2012-12-09T19:44:30Z

Including a method name in the request body is very un-RESTful, but then again what we have so far isn't exactly RESTful either (and JSON-RPC definitely isn't either).

It would be nice if we could just use GET methods to enable caching, etc, but GET requests don't have a request body for us to include the inputs. Inputs could be mapped to the query string (or even include the JSON in the URL).

The HTTP methods could be mapped to the method field for other JSON-RPC transports. If more than one block were handled by a single JSON-RPC server the method could include the url. GET:/my-database or something.

Also we definitely shouldn't use JSON-RPC 1.0 positional arguments. JSON-RPC 2.0 supports named arguments.

matthewhudson · 2012-12-10T04:05:34Z

I'm sorry, I wasn't as clear as I could have been with my database example. We aren't passing a HTTP Method as an argument, we're passing the name of the "remote procedure" we want to run. In JSON-RPC, the "method" is the remote procedure you want to invoke.

In my example, the database block exposed two methods/functions/remote procedures - 'get' and 'set'. 'Get' would return a database record, and 'set' would save a database record.

As for REST, we're pretty much eschewing HTTP all together if we decide to go multi-protocol. That makes the following things irrelevant:

HTTP Subscriptions
CORS
HTTP Methods (OPTIONS, POST)
HTTP Response Body
HTTP Status Codes signaling success/failure

progrium · 2012-12-10T04:15:09Z

I feel like maybe aligning with JSON-RPC is getting a little out of hand.
We did agree these are WebPipes and so it makes sense to focus on web
technologies. Using other protocols is going to have very little benefit to
the goals of the project.

On Sun, Dec 9, 2012 at 8:05 PM, Matthew Hudson [email protected]:

I'm sorry, I wasn't as clear as I could have been with my database
example. We aren't passing a HTTP Method as an argument, we're passing the
name of the "remote procedure" we want to run. In JSON-RPC, the "method" is
the remote procedure you want to invoke.

In my example, the database block exposed two methods/functions/remote
procedures - 'get' and 'set'. 'Get' would return a database record, and
'set' would save a database record.

As for REST, we're pretty much eschewing HTTP all together if we decide to
go multi-protocol. That makes the following things irrelevant:

HTTP Subscriptions

CORS

HTTP Methods (OPTIONS, POST)

HTTP Response Body

HTTP Status Codes signaling success/failure

—
Reply to this email directly or view it on GitHubhttps://github.com/Align with JSON-RPC? #4#issuecomment-11181297.

Jeff Lindsay
http://progrium.com

matthewhudson · 2012-12-10T04:34:57Z

I have to agree with Jeff. I think we ought to leverage JSON-RPC only for inspiration, not total adoption. Like Jeff said, we agreed to focus on HTTP. I think the two biggest inspirations we gain from JSON-RPC are:

The 'error' key in output. In addition to HTTP Status Codes, this allows Blocks to provide a precise explanation for failure.
The 'method' key in the input. This affords Blocks the ability to expose multiple functions which is crucial for something like a database block.

progrium · 2012-12-10T04:46:51Z

Why can't a database block be split into different blocks?

On Sun, Dec 9, 2012 at 8:34 PM, Matthew Hudson [email protected]:

I have to agree with Jeff. I think we ought to leverage JSON-RPC only for
inspiration, not total adoption. Like Jeff said, we agreed to focus on
HTTP. I think the two biggest inspirations we gain from JSON-RPC are:

The 'error' key in output. In addition to HTTP Status Codes, this
allows Blocks to provide a precise explanation for failure.

The 'method' key in the input. This affords Blocks the ability to
expose multiple functions which is crucial for something like a database
block.

—
Reply to this email directly or view it on GitHubhttps://github.com/Align with JSON-RPC? #4#issuecomment-11181698.

Jeff Lindsay
http://progrium.com

matthewhudson · 2012-12-10T04:56:26Z

Wouldn't that be confusing? How would a block specify a relationship with one or more other blocks?

progrium · 2012-12-10T04:57:45Z

I don't know what you mean. Give me an example.

On Sun, Dec 9, 2012 at 8:56 PM, Matthew Hudson [email protected]:

Wouldn't that be confusing? How would a block specify a relationship with
one or more other blocks?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4#issuecomment-11181979.

Jeff Lindsay
http://progrium.com

matthewhudson · 2012-12-10T05:09:00Z

So, I'm proposing we adopt the "methods" key from JSON-RPC so that a single block can expose multiple functions. This new "methods" key would map method names to our inputs/outputs.

Here's an example:

{
    "name": "database",
    "description": "Store and fetch persistent data.",
    "url": "/block-endpoint",
    "methods": {
        "get": {
            "inputs": [],
            "outputs": []
        },
        "set": {
            "inputs": [],
            "outputs": []
        }
    }
}

You seem to be proposing that we don't need to allow blocks to expose multiple methods and that my example database block should instead be two separate blocks, like this:

Save Record:

{
    "name": "database-set-record",
    "description": "Save persistent data.",
    "url": "/block-endpoint",
    "inputs": [],
    "outputs": []

}

Retrieve Record:

{
    "name": "database-get-record",
    "description": "Fetch persistent data.",
    "url": "/block-endpoint",
    "inputs": [],
    "outputs": []
}

So my question to you is this: how does "database-set-record" describe its relationship with "database-get-record" and vice-versa? How does an enduser who knows about one of these block know about the other?

progrium · 2012-12-10T05:19:09Z

Naming convention? Meta-data? Maybe that's a registry/index problem?

There was an operating system called Genera. It was 100% LISP. Think about
the complexity of an operating system. Now think about there being a single
namespace for every operation. The operating system was amazing in that
every function was available to the user to explore and everything in it
was inspectable. It was truly an open operating system. If you found a get
function and wondered if there was a set function, you'd just look it up.
There were indexes and things to help you find functions. But you wouldn't
find one function from another function.

In fact, think of Unix today. There are lots of system calls with related
functions. It's in the man pages that you find the related functions.

Anyway, I'm getting too far from the point. I think the point is that the
reason you'd want a block to serve multiple functions is understandable,
but it's not necessary. The big reason why I'm against it is that every
graphical programming environment has maintained a simple relationship
between their equivalent of a block and a function. And I think it's for a
good reason.

What do you call a block with multiple functions? An aggregate block? Wait
wait, we already have that concept. So let's introduce another name that's
inevitably going to be similar? This is something I'd only want to do as a
last resort.

-jeff

On Sun, Dec 9, 2012 at 9:09 PM, Matthew Hudson [email protected]:

So, I'm proposing we adopt the "methods" key from JSON-RPC so that a
single block can expose multiple functions. This new "methods" key would
map method names to our inputs/outputs.

Here's an example:

{
"name": "database",
"description": "Store and fetch persistent data.",
"url": "/block-endpoint",
"methods": {
"get": {
"inputs": [],
"outputs": []
},
"set": {
"inputs": [],
"outputs": []
}
}
}

You seem to be proposing that we don't need to allow blocks to expose
multiple methods and that my example database block should instead be two
separate blocks, like this:

Save Record:

{
"name": "database-set-record",
"description": "Save persistent data.",
"url": "/block-endpoint",
"inputs": [],
"outputs": []

}

Retrieve Record:

{
"name": "database-get-record",
"description": "Fetch persistent data.",
"url": "/block-endpoint",
"inputs": [],
"outputs": []
}

So my question to you is this: how does "database-set-record" describe its
relationship with "database-get-record" and vice-versa? How does an enduser
who knows about one of these block know about the other?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4#issuecomment-11182159.

Jeff Lindsay
http://progrium.com

tlrobinson · 2012-12-11T22:47:03Z

I agree the JSON-RPC thing might be too complicated, but I think we should commit to something that's RPC-like or REST-like, but not halfway in between. Personally I favor REST.

If we go REST it might make sense to have the "methods" key in the OPTIONs response map to HTTP methods, so we can do GET, POST, PUT, DELETE on the same endpoint. But I don't like having arbitrary methods like "set" in there.

This is sort of why I was interested in working on Skylib concurrently with WebPipes. If we had a generic way to document REST APIs in a machine readable way then WebPipes APIs would just be a slightly constrained subset of Skylib APIs (a "flat" map of inputs in the request, either as a JSON object or maybe query string parameters, or even XML, and a flat output object, or array of objects. Or maybe they don't even need to be flat objects, but that complicates things).

matthewhudson · 2012-12-11T23:38:44Z

I was never suggesting implementing a new HTTP Method called "set", or mapping HTTP Methods to an OPTIONS key called 'methods'.

I was proposing we nest our input/output arguments inside something like the JSON-RPC request object member called 'method'.

Here's an example of how the Block Definition would change:

{
    "name": "database",
    "description": "Store and fetch persistent data.",
    "url": "/block-endpoint",
    "methods": {
        "get": {
            "inputs": [],
            "outputs": []
        },
        "set": {
            "inputs": [],
            "outputs": []
        }
    }
}

Jeff argued against that idea and I'm slowly coming around to his way of thinking.

tlrobinson · 2012-12-11T23:50:22Z

Yeah, I understood that completely and didn't mean to suggest otherwise, but including any kind of method name in either the request body (as you're suggesting) or in the URL (even as a separate endpoint for the same "resource", as Jeff is suggesting) is more "RPCful" than RESTful.

If we want to be RESTful I am proposing mapping different HTTP method names in the OPTIONs request.

If we want to be RPCful then why not just use JSON-RPC over HTTP?

But I'd rather be RESTful.

tlrobinson · 2012-12-11T23:58:06Z

BTW, WADL looks a lot like what I'm proposing, but it's XML :(

Here's an example: http://www.w3.org/Submission/wadl/#x3-40001.3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Align with JSON-RPC? #4

Align with JSON-RPC? #4

tlrobinson commented Dec 2, 2012

progrium commented Dec 2, 2012

matthewhudson commented Dec 2, 2012

tlrobinson commented Dec 2, 2012

progrium commented Dec 2, 2012

matthewhudson commented Dec 9, 2012

matthewhudson commented Dec 9, 2012

tlrobinson commented Dec 9, 2012

matthewhudson commented Dec 10, 2012

progrium commented Dec 10, 2012

matthewhudson commented Dec 10, 2012

progrium commented Dec 10, 2012

matthewhudson commented Dec 10, 2012

progrium commented Dec 10, 2012

matthewhudson commented Dec 10, 2012

progrium commented Dec 10, 2012

tlrobinson commented Dec 11, 2012

matthewhudson commented Dec 11, 2012

tlrobinson commented Dec 11, 2012

tlrobinson commented Dec 11, 2012

Align with JSON-RPC? #4

Align with JSON-RPC? #4

Comments

tlrobinson commented Dec 2, 2012

progrium commented Dec 2, 2012

matthewhudson commented Dec 2, 2012

tlrobinson commented Dec 2, 2012

progrium commented Dec 2, 2012

matthewhudson commented Dec 9, 2012

matthewhudson commented Dec 9, 2012

tlrobinson commented Dec 9, 2012

matthewhudson commented Dec 10, 2012

progrium commented Dec 10, 2012

matthewhudson commented Dec 10, 2012

progrium commented Dec 10, 2012

matthewhudson commented Dec 10, 2012

progrium commented Dec 10, 2012

matthewhudson commented Dec 10, 2012

progrium commented Dec 10, 2012

tlrobinson commented Dec 11, 2012

matthewhudson commented Dec 11, 2012

tlrobinson commented Dec 11, 2012

tlrobinson commented Dec 11, 2012