protocols

[jacobhinkle]

I just wanted to state what I have in mind for protocols. Of course we'll have a TCP interface which will probably be ws:// websockets. Whatever that is let's not pigeonhole ourselves: we want to be sure we can easily support it on multiple types of client (besides webbrowsers) and that we can wrap authentication and SSL around it without much fuss.

As for local clients, for obvious performance and space reasons it would be super cool to have a shmem interface, perhaps communicating through unix sockets for the metadata, etc. I have exactly zero idea how to do this, and as long as we're not retards i think we could always solder this back on if we just start with the websockets interface first.

Thoughts?

[cscheid]

we can have an HTTP interface, as simple as you can imagine. GETting

http://imp.sci.utah.edu:8900/jacob/is/a/poopy/head

would return exactly what you'd expect it to.

If we allow people to post binary blobs (like images) this would be a brain-dead way to share results as they come out of a simulation run, with no additional processing of any kind required. That would be super convenient.

There's other HTTP verbs which we could in theory support, which would turn imp into a real web service (just teach it to respond to HEAD, GET, PUT, and DELETE, and that covers a large portion of our use cases). Not for the first demo, but something to keep in mind.

The shmem interface would be great, but I think we'd get a good chunk of the performance simply by detecting a local connection and switching to Unix sockets. loopback sockets are likely not as fast as shmem, but if I understand them correctly, sockets on both ends share the buffer, so there's one fewer copy. The main thing is that I don't see how to make the metadata work without some kind of socket, and so shmem is just extra complication.

[jacobhinkle]

This is sort of what I was thinking, but as I have no websockets experience wasn't sure what would allow us a kept-alive connection. I agree that this format is really convenient since imp is sort of situated as a web-based data server. I also love the idea that some day we'd serve a basic directory listing or more complicated querying interface as well.

Re sockets and shmem I defer to you onwhat sounds like good news. Do you agree that we should start with the http interface? It seems logical and possibly easier anyway. BTW do you know if there's some existing project that would ease creating HTTP-based protocols? Seems like a common task...

[cscheid]

To get something up and running as quickly as possible, you won't find anything simpler than python's HTTP server:

http://docs.python.org/library/basehttpserver.html (huh, *.python.org appears to be offline. for now, type this:

Python 2.7.2 (default, Jul 24 2011, 14:23:56)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import BaseHTTPServer
help(BaseHTTPServer)

I've implemented a handful of web servers using BaseHTTPServer and it's absolutely trivial.

The right way to do it is to write an apache module, but I don't know anything about that.

[jacobhinkle]

I had kind of expected we'd want to use python. I am not into distributing imp as an apache module. That'd be a big turnoff to people. I'm down with the python route. I'm creating a milestone for us to shoot for a first prototype. Just a basic server operation, handshakes and hangs up. Then another milestone where we store some shit and read it back out, etc.

[jacobhinkle]

ah, took me a while to figure out this is now called http.server in python3. looks like a plan.

[cscheid]

What is this python3 you speak of?

[jacobhinkle]

don't get me started... let's just use haskell

[jacobhinkle]

FYI, this discussion on the Julia list

https://groups.google.com/d/topic/julia-dev/ezXlnBSrvDI/discussion

[cscheid]

That's pretty cool. It gave me an idea for a nice demo to showcase Facet. Thanks! :)

-carlos

On Mar 23, 2012, at 12:27 AM, Jacob Hinkle wrote:

FYI, this discussion on the Julia list

https://groups.google.com/d/topic/julia-dev/ezXlnBSrvDI/discussion

---

Reply to this email directly or view it on GitHub:
#5 (comment)

[cscheid]

http://cscheid.github.com/facet/demos/graphing_calculator/graphing_calculator.html

Nevermind the tacky wavy thing. The point is the expressions you type in the fields get translated into GPU code for you.. this is something that would be fairly painful to do correctly directly in Javascript, but with Facet it's trivial.

-carlos

On Mar 23, 2012, at 12:37 AM, Carlos Scheidegger wrote:

That's pretty cool. It gave me an idea for a nice demo to showcase Facet. Thanks! :)

-carlos

On Mar 23, 2012, at 12:27 AM, Jacob Hinkle wrote:

FYI, this discussion on the Julia list

https://groups.google.com/d/topic/julia-dev/ezXlnBSrvDI/discussion

---

Reply to this email directly or view it on GitHub:
#5 (comment)

[waxlamp]

Here's a buzzword for the HTTP verbs idea Carlos mentioned: http://en.wikipedia.org/wiki/Representational_state_transfer. We've been making use of this RESTful API business in some of my work at Kitware.