Discussion:
REST for web service
Alex Hudson
2011-03-18 12:25:48 UTC
Permalink
This is more really a set of notes given a discussion between Lance,
Steve, SEJeff, and some others on IRC.

The original Dragonfly schema was REST-inspired, but in many ways not
RESTful:

http://bongo-project.org/Guides/Dragonfly_URL_Schemes

The main problem I have with this schema is that it ties the web server
implementation very much to the UI implementation on the front end:
everything is defined more or less as the UI displays it, which is a
clear red flag to me.

I was going to propose something which is actually closer to how things
work in the Store - so that the same interface (I'll call it an API)
applies no matter whether you're accessing mails, contacts, or some
other form of content (this was raised when talking about Tomboy notes
really).

I think versioning the API and stuff is sensible, so as a starting point
I'd proffer the base URI of http://server/bongo/api-1.0/ - everything
below would hang off that.

The way the store works right now is that you have these top-level
groups of data - called user stores - so that would be the first set of
resources that you'd access. Under that store, you have collections, and
under collections you have documents.

I propose that we wouldn't access anything by name/path within the
store, but simply by ID, so all resource paths would look something
like:

/store-name/collection_or_documentID

So, for example, we know that ID 0x1 is always the 'root' collection, ID
0x6 is always /mail/INBOX. Not all collections have static IDs, of
course, but those key first IDs give you discovery.

As an example:

GET http://server/bongo/api-1.0/_system/1

might return

{
meta-data: {
id: '0000000000000001',
type: 4096,
ctime: 1220527135,
filename: '/'
},
data: [
{
id: '0000000000000002',
type: 4096,
ctime: 1220527135,
filename: '/addressbook'
},
....
]
}

.. which represents a directory listing of /. I'm not totally sure about
separating data: and meta-data: that's not very RESTful. Ideally, we'd
put meta-data in the HTTP headers as X-Bongo-Id: etc., but to be honest
I'm not absolutely sure that would work for all clients.

Special stuff for collections.

When we're doing lists, we probably want to be able to filter somehow,
and we certainly want to be able to paginate (maybe by default). I think
these things ought to be GET parameters, because if we stuck some of
this stuff in the URI it would again break REST (harming caching, for
one).

Special stuff for documents.

It's almost certain we would want some documents in different
representations. Whereas contacts and things are stored internally as
JSON, we may also need vCard versions or whatever. The resource URI
wouldn't change, again you'd have some kind of parameter specifying what
MIME type of data you're actually looking for.

I think also we'd want to have some way of opening up MIME-encoded
documents so that you could address the content inside of them. I'm
proposing to hand-wave over that for now: there are caching issues
again, there are also other issues like supporting HTTP byte ranges,
download resumption, and other cool stuff. I'd like to pretend that
problem doesn't exist right now.

Doing it.

The cool thing about the above is that we already have a basic PHP
implementation that could be hooked up to do something like this quite
quickly; Lance is also looking at a Ruby based system to present a more
cohesive web UI which could do things a PHP implementation couldn't
(websockets spring to mind).

Does anyone have any comments about the above?

Ta

Alex.


--
This message was scanned by Better Hosted and is believed to be clean.
http://www.betterhosted.com
Lance Haig
2011-03-18 12:44:30 UTC
Permalink
I am resending this so that "others" might see it now after subscribing
to the list :-)

Lance
Post by Alex Hudson
This is more really a set of notes given a discussion between Lance,
Steve, SEJeff, and some others on IRC.
The original Dragonfly schema was REST-inspired, but in many ways not
http://bongo-project.org/Guides/Dragonfly_URL_Schemes
The main problem I have with this schema is that it ties the web server
everything is defined more or less as the UI displays it, which is a
clear red flag to me.
I was going to propose something which is actually closer to how things
work in the Store - so that the same interface (I'll call it an API)
applies no matter whether you're accessing mails, contacts, or some
other form of content (this was raised when talking about Tomboy notes
really).
I think versioning the API and stuff is sensible, so as a starting point
I'd proffer the base URI of http://server/bongo/api-1.0/ - everything
below would hang off that.
The way the store works right now is that you have these top-level
groups of data - called user stores - so that would be the first set of
resources that you'd access. Under that store, you have collections, and
under collections you have documents.
I propose that we wouldn't access anything by name/path within the
store, but simply by ID, so all resource paths would look something
/store-name/collection_or_documentID
So, for example, we know that ID 0x1 is always the 'root' collection, ID
0x6 is always /mail/INBOX. Not all collections have static IDs, of
course, but those key first IDs give you discovery.
GET http://server/bongo/api-1.0/_system/1
might return
{
meta-data: {
id: '0000000000000001',
type: 4096,
ctime: 1220527135,
filename: '/'
},
data: [
{
id: '0000000000000002',
type: 4096,
ctime: 1220527135,
filename: '/addressbook'
},
....
]
}
.. which represents a directory listing of /. I'm not totally sure about
separating data: and meta-data: that's not very RESTful. Ideally, we'd
put meta-data in the HTTP headers as X-Bongo-Id: etc., but to be honest
I'm not absolutely sure that would work for all clients.
Special stuff for collections.
When we're doing lists, we probably want to be able to filter somehow,
and we certainly want to be able to paginate (maybe by default). I think
these things ought to be GET parameters, because if we stuck some of
this stuff in the URI it would again break REST (harming caching, for
one).
Special stuff for documents.
It's almost certain we would want some documents in different
representations. Whereas contacts and things are stored internally as
JSON, we may also need vCard versions or whatever. The resource URI
wouldn't change, again you'd have some kind of parameter specifying what
MIME type of data you're actually looking for.
I think also we'd want to have some way of opening up MIME-encoded
documents so that you could address the content inside of them. I'm
proposing to hand-wave over that for now: there are caching issues
again, there are also other issues like supporting HTTP byte ranges,
download resumption, and other cool stuff. I'd like to pretend that
problem doesn't exist right now.
Doing it.
The cool thing about the above is that we already have a basic PHP
implementation that could be hooked up to do something like this quite
quickly; Lance is also looking at a Ruby based system to present a more
cohesive web UI which could do things a PHP implementation couldn't
(websockets spring to mind).
Does anyone have any comments about the above?
Ta
Alex.
--
This message was scanned by Better Hosted and is believed to be clean.
http://www.betterhosted.com
_______________________________________________
Bongo-devel mailing list
https://mail.gna.org/listinfo/bongo-devel
--
This message was scanned by Better Hosted and is believed to be clean.
http://www.betterhosted.com
Baris Balic
2011-03-18 16:08:57 UTC
Permalink
Thanks Lance!

After reading over the messages I would say that I agree with Alex for the
most part, I haven't looked over the Dragonfly schema but the things he
highlights indicate it could be neater. We certainly shouldn't be passing
parameters in the URL (aside from ids).

I'm a little curious however, are we talking about writing a system that
wraps the protocol and exposes a RESTful API or are we talking about
building the RESTful API into bongo? The latter option will obviously be
slower overall but it does keep it isolate the rest interface from the
protocol itself, and there will be numerous ways of speeding it up if we
find it's too slow.

I think addressing resources down to the document makes perfect sense, but
addressing parts of a document directly sounds a little overkill, especially
given the variety of types that may be available. It would be better to
hook up a full-text indexing engine like Lucene and have it crawl documents
rather than write code paths for all types of documents.
Addressing should definitely be based on the resources and not the views, a
view could easily be concerned with more than one resource at a time.

Versioning I also agree with but I would suggest we start at 0.1 and we only
reach 1.0 when we have something comprehensive that we are prepared to
support into the next major version.

I'll try to be on channel more this week so that we can discuss in realtime
if there's anything you strongly agree/disagree with.

Cheers,
Baris
I am resending this so that "others" might see it now after subscribing to
the list :-)
Lance
Post by Alex Hudson
This is more really a set of notes given a discussion between Lance,
Steve, SEJeff, and some others on IRC.
The original Dragonfly schema was REST-inspired, but in many ways not
http://bongo-project.org/Guides/Dragonfly_URL_Schemes
The main problem I have with this schema is that it ties the web server
everything is defined more or less as the UI displays it, which is a
clear red flag to me.
I was going to propose something which is actually closer to how things
work in the Store - so that the same interface (I'll call it an API)
applies no matter whether you're accessing mails, contacts, or some
other form of content (this was raised when talking about Tomboy notes
really).
I think versioning the API and stuff is sensible, so as a starting point
I'd proffer the base URI of http://server/bongo/api-1.0/ - everything
below would hang off that.
The way the store works right now is that you have these top-level
groups of data - called user stores - so that would be the first set of
resources that you'd access. Under that store, you have collections, and
under collections you have documents.
I propose that we wouldn't access anything by name/path within the
store, but simply by ID, so all resource paths would look something
/store-name/collection_or_documentID
So, for example, we know that ID 0x1 is always the 'root' collection, ID
0x6 is always /mail/INBOX. Not all collections have static IDs, of
course, but those key first IDs give you discovery.
GET http://server/bongo/api-1.0/_system/1
might return
{
meta-data: {
id: '0000000000000001',
type: 4096,
ctime: 1220527135,
filename: '/'
},
data: [
{
id: '0000000000000002',
type: 4096,
ctime: 1220527135,
filename: '/addressbook'
},
....
]
}
.. which represents a directory listing of /. I'm not totally sure about
separating data: and meta-data: that's not very RESTful. Ideally, we'd
put meta-data in the HTTP headers as X-Bongo-Id: etc., but to be honest
I'm not absolutely sure that would work for all clients.
Special stuff for collections.
When we're doing lists, we probably want to be able to filter somehow,
and we certainly want to be able to paginate (maybe by default). I think
these things ought to be GET parameters, because if we stuck some of
this stuff in the URI it would again break REST (harming caching, for
one).
Special stuff for documents.
It's almost certain we would want some documents in different
representations. Whereas contacts and things are stored internally as
JSON, we may also need vCard versions or whatever. The resource URI
wouldn't change, again you'd have some kind of parameter specifying what
MIME type of data you're actually looking for.
I think also we'd want to have some way of opening up MIME-encoded
documents so that you could address the content inside of them. I'm
proposing to hand-wave over that for now: there are caching issues
again, there are also other issues like supporting HTTP byte ranges,
download resumption, and other cool stuff. I'd like to pretend that
problem doesn't exist right now.
Doing it.
The cool thing about the above is that we already have a basic PHP
implementation that could be hooked up to do something like this quite
quickly; Lance is also looking at a Ruby based system to present a more
cohesive web UI which could do things a PHP implementation couldn't
(websockets spring to mind).
Does anyone have any comments about the above?
Ta
Alex.
--
This message was scanned by Better Hosted and is believed to be clean.
http://www.betterhosted.com
_______________________________________________
Bongo-devel mailing list
https://mail.gna.org/listinfo/bongo-devel
--
This message was scanned by Better Hosted and is believed to be clean.
http://www.betterhosted.com
_______________________________________________
Bongo-devel mailing list
https://mail.gna.org/listinfo/bongo-devel
Alex Hudson
2011-03-18 16:23:44 UTC
Permalink
Hey Baris,
Post by Baris Balic
I'm a little curious however, are we talking about writing a system
that wraps the protocol and exposes a RESTful API or are we talking
about building the RESTful API into bongo?
I'm not sure exactly what difference you see between the two, so what I
would say is it's not something we'd build into the Store itself.

In a way, it would be an agent like any other in the system: something
which converts from one protocol (whether it's REST over HTTP, or SMTP,
or something else) to another.

Potentially, this could be something that wouldn't necessarily have to
have a Bongo server behind it (if someone wanted to write that
interface).
Post by Baris Balic
Versioning I also agree with but I would suggest we start at 0.1 and
we only reach 1.0 when we have something comprehensive that we are
prepared to support into the next major version.
Yeah, I think that makes sense, although I wouldn't want to keep bumping
it because every time we bump it we'd need to alter the base URI in the
clients using it.

Maybe we just say 0.9 is the development version, it's 1.0 when it's
released, and that if you're using 0.9 then it may break because it's in
development?

Or alternatively, we start at 0.1 and bump it only when we make
incompatible changes. So, if we just build up the API over time we don't
bump the number, because adding interfaces isn't incompatible per se...?

Cheers

Alex.





--
This message was scanned by Better Hosted and is believed to be clean.
http://www.betterhosted.com

Lance Haig
2011-03-18 13:07:05 UTC
Permalink
As I see it we would use this "API Server" as the means that apps
connect to our environemtn.

This will allow for oAuth etc as well

Lance
Post by Alex Hudson
This is more really a set of notes given a discussion between Lance,
Steve, SEJeff, and some others on IRC.
The original Dragonfly schema was REST-inspired, but in many ways not
http://bongo-project.org/Guides/Dragonfly_URL_Schemes
The main problem I have with this schema is that it ties the web server
everything is defined more or less as the UI displays it, which is a
clear red flag to me.
I was going to propose something which is actually closer to how things
work in the Store - so that the same interface (I'll call it an API)
applies no matter whether you're accessing mails, contacts, or some
other form of content (this was raised when talking about Tomboy notes
really).
I think versioning the API and stuff is sensible, so as a starting point
I'd proffer the base URI of http://server/bongo/api-1.0/ - everything
below would hang off that.
The way the store works right now is that you have these top-level
groups of data - called user stores - so that would be the first set of
resources that you'd access. Under that store, you have collections, and
under collections you have documents.
I propose that we wouldn't access anything by name/path within the
store, but simply by ID, so all resource paths would look something
/store-name/collection_or_documentID
So, for example, we know that ID 0x1 is always the 'root' collection, ID
0x6 is always /mail/INBOX. Not all collections have static IDs, of
course, but those key first IDs give you discovery.
GET http://server/bongo/api-1.0/_system/1
might return
{
meta-data: {
id: '0000000000000001',
type: 4096,
ctime: 1220527135,
filename: '/'
},
data: [
{
id: '0000000000000002',
type: 4096,
ctime: 1220527135,
filename: '/addressbook'
},
....
]
}
.. which represents a directory listing of /. I'm not totally sure about
separating data: and meta-data: that's not very RESTful. Ideally, we'd
put meta-data in the HTTP headers as X-Bongo-Id: etc., but to be honest
I'm not absolutely sure that would work for all clients.
Special stuff for collections.
When we're doing lists, we probably want to be able to filter somehow,
and we certainly want to be able to paginate (maybe by default). I think
these things ought to be GET parameters, because if we stuck some of
this stuff in the URI it would again break REST (harming caching, for
one).
Special stuff for documents.
It's almost certain we would want some documents in different
representations. Whereas contacts and things are stored internally as
JSON, we may also need vCard versions or whatever. The resource URI
wouldn't change, again you'd have some kind of parameter specifying what
MIME type of data you're actually looking for.
I think also we'd want to have some way of opening up MIME-encoded
documents so that you could address the content inside of them. I'm
proposing to hand-wave over that for now: there are caching issues
again, there are also other issues like supporting HTTP byte ranges,
download resumption, and other cool stuff. I'd like to pretend that
problem doesn't exist right now.
Doing it.
The cool thing about the above is that we already have a basic PHP
implementation that could be hooked up to do something like this quite
quickly; Lance is also looking at a Ruby based system to present a more
cohesive web UI which could do things a PHP implementation couldn't
(websockets spring to mind).
Does anyone have any comments about the above?
Ta
Alex.
--
This message was scanned by Better Hosted and is believed to be clean.
http://www.betterhosted.com
_______________________________________________
Bongo-devel mailing list
https://mail.gna.org/listinfo/bongo-devel
--
This message was scanned by Better Hosted and is believed to be clean.
http://www.betterhosted.com
Alex Hudson
2011-03-18 14:07:23 UTC
Permalink
Post by Lance Haig
As I see it we would use this "API Server" as the means that apps
connect to our environemtn.
This will allow for oAuth etc as well
Yeah, I should have made something else clear as well.

I don't particularly see any value in trying to have a single Bongo HTTP
API. So, for example, to access the store URIs you'd need to have some
kind of authentication - but yeah, something like oAuth makes a lot of
sense.

I wouldn't want to try to shoe-horn the oAuth system into the store REST
API, so I would see that as a separate API.

Equally, if we supported server-side Tomboy sync, that would be another
API as well.

Another set of API might be user/admin management: for example, if you
wanted to reset your password, we maybe would have an admin user API as
well.

Cheers

Alex.


--
This message was scanned by Better Hosted and is believed to be clean.
http://www.betterhosted.com
Loading...