Archive for the ‘Development’ Category

Encoding and XML

Tuesday, January 6th, 2009

In speccing a project last month, we discussed the best way to attach PDFs to the XML documents we send between very distributed systems. We quickly decided it would avoid a whole host of atomicity, reliability, and redesign issues to simply update our XML schema to include Base64 encoded documents inside the body of the XML document itself.

For my half of our application ecosystem I researched .NET’s Base64 encoding/decoding support, got curious about character set encoding, and set out to write a universal encoder to make it simple, easy, and guaranteed safe to insert any kind of binary or text data into an XML document.

But first, a quick nano-refresher: Character encoding specifies how a string of bytes should be mapped to specific text characters. In the simplest case, ASCII, one byte maps to one of 256 possible characters. 65=A, 66=B, …. 90=Z, etc. In Unicode, two (or even four) bytes map to thousands or even (theoretically) billions of characters. So when my program reads four bytes from a text file, I need to know if it represents four 1-byte characters, two 2-byte characters, or one 4-byte character. It’s actually even more complicated than that, but the basic problem is making sure I don’t accidentally turn a 400-byte ASCII text file into 200 Japanese characters. Or vice versa. Or garbage.

Fortunately, .NET has robust support for character encoding, so all I have to do is load the correct encoding class and ask it to take care of this for me. If I know I will only ever need to deal with Unicode, that class is Encoding.Unicode, but for maximum flexibility I can call Encoding.GetEncoding(encodingName) and get any encoding by name. Like so:

public string GetStringFromFile(string myFilename, string encodingName)
{
byte[] fileBytes = System.IO.File.ReadAllBytes(myFilename);
System.Text.Encoding myEncoding = System.Text.Encoding.GetEncoding(encodingName);
return myEncoding.GetString(fileBytes);
}

After I’m done modifying that string I can easily convert it back into a byte array and save it, preserving the original character encoding:

public void SaveStringToFile(string myFilename, string encodingName, string myString)
{
System.Text.Encoding myEncoding = System.Text.Encoding.GetEncoding(encodingName);
byte[] fileBytes = myEncoding.GetBytes(myString);
System.IO.File.WriteAllBytes(myFilename, fileBytes);
}

But now let’s get back to my actual business problem, storing a PDF or other binary data in my XML document. Because the bytes I encounter are not supposed to represent character data, attempting to map them to characters may result in nonsense. For example, whether I decode the byte sequence 00 00 00 00 as ASCII, Unicode, or UTF-32, I get either one, two, or four null characters that will screw up string processing. Note that use of CDATA sections doesn’t help. Only being very lucky about which byte sequences I encounter would avert disaster, and I don’t like writing lucky code.

Enter Base64, which is designed for exactly this purpose: encoding arbitrary binary data into a string guaranteed to consist of only “safe” ASCII characters and decoding that string back to bytes with 100% fidelity later. Microsoft places Base64 functionality under the System.Convert class, not System.Text.Encoding because it’s more of a conversion and translation process, not a direct byte-to-character encoding like those described above.

To read a file into a Base64 string:

public string GetBase64StringFromFile(string myFilename)
{
byte[] fileBytes = System.IO.File.ReadAllBytes(myFilename);
return Convert.ToBase64String(fileBytes, Base64FormattingOptions.InsertLineBreaks);
}

And to decode it and save it back to the filesystem:

public void SaveBase64StringToFile(string myFilename, string myString)
{
byte[] fileBytes = Convert.FromBase64String(myString);
System.IO.File.WriteAllBytes(myFilename, fileBytes);
}

I combined both “real” character encodings and Base64 encoding in my XmlFileEncoder class (attached) to provide unified access to both encodings when working with XML documents. If you know you’re dealing with Unicode, simply call XmlFileEncoder.InsertFileIntoXmlDocument with the encoding UTF-16 and the file will safely be inserted as text. If you don’t always know the file format, or you are dealing with binary files, simply call the same method with the encoding Base64. In either case, a new node will be added to contain your file data and the encoding attribute will record the encoding method so XmlFileEncoder.ExtractFileFromXmlDocument will use the correct character/Base64 decoding automatically.

The demo WinForms app starts up displaying an XML document with some UTF-16 data already encoded into it. Use the controls along the bottom to experiment with inserting differently encoded files (provided, or use your own) using different application character encodings. Some files will clearly look wrong in the XML when you select the wrong encoding. Others may look correct, or almost correct, but when you press the SaveEncodedFile to File button it will report Copy accuracy FAILED when it verifies against the source file. However, the files encoded and decoded using Base64 will always copy accurately. The only downside is that Base64 encoded data is always 1/3 bigger and much less human readable than the source.

You can find the source code here: XmlEncodingDemo.zip

Have a happy, healthy, and correctly encoded 2009!

Configuring Restlet 1.1 with Spring

Monday, September 15th, 2008

Back in May I wrote a post about using Spring with Restlet 1.0.x and promised another about doing the same with Restlet 1.1.x, which provides better Spring support. Well, the time has come, so here goes!

In Restlet 1.1, which is currently at Release Candidate 1, you can now configure an application completely within the Spring application context. This means that the only Java classes you will need to create are one for starting the application, and one for each of the resources in your class. Although the Restlet authors prefer specifying the configuration in code, I like specifying it in an external, non-compiled configuration file, and this provides a great way to do that.

Continue reading»

Custom HTTP Response Headers with Restlet

Monday, September 15th, 2008

Our Java apps use the Restlet framework, and we sometimes need to send custom headers in our HTTP responses. This is infrequent enough that we forget how to do so, and need to look it up, but frequent enough that we should really have this snippet close at hand.

Continue reading»

Arc90's first summer internship

Friday, August 29th, 2008

So today is the last day of our first summer Internship Program. All-in-all, a huge success, both for Arc and for our first two interns–Alex and Ben. These two guys shattered any preconceived notions of ‘intern’ that we had going into this summer, which makes sense, as they’ve each been programming for over a decade (it just so happens that that represents more than half of their lives…). Seriously though, these two guys are brilliant–the bar for next year is high.

We learned a lot this summer about how to structure this Program going forward, particularly the size and scope of the efforts we should embark on, and how to structure their work within the company. I wish I could go back and do some things differently, but when I voiced that concern, both Alex and Ben were quick to let me know that they wouldn’t have had this summer any other way. Oftentimes, you learn more from challenging and ambitious efforts, than from safe work.

Anyway, don’t take my word for it, read Alex’s and Ben’s recaps, in their own words.

Thanks guys–good luck in school this year.

An Intern's Experience; or How Code Learns

Friday, August 29th, 2008

Until recently my conception of a software company consisted of two guys crammed into a former closet, surrounded by ashtrays stacked on ashtrays stacked on servers and other assorted hardware, both functioning and non. Being asthmatic, this never appealed to me. My only other imagining is Mike Judge’s, and as charming as the Initech boys were the corporate atmosphere is as strangling–if not more strangling–than the smoke-filled closet. Where, then, do I fit in?

Stereotypes aside,when all the news is about either two-man startups or huge, huge firms where does an intelligent, but relatively normal person fit it? I’m passionate about technology but not to the point where I’d have to give up my family, in this case my dog, to sing the startup song all day, every day. Likewise I’m not savvy enough to deal with more than six people at once, so the meetings in a big organization wouldn’t do.

With arc90 I found a good balance. Sure, we still had meetings but never with more than six people, so I was within my capacity. And all the while I was surrounded with extremely bright people who really care about what they are doing. These guys (and gals) care. There’s a tangible energy around, so much so that I managed to do the impossible: get excited about insurance software. Yes, I didn’t think it was within the realm of reason either, but here I present to you a verifiable instance of this mythical affliction.

So after bringing out the boxing gloves over specification, Ben (the other intern) and I were let loose. Code was created. Diet coke was consumed. Yelling, well, happened. And yet, through all this programming in good times and bad, I kept wondering again: where do I fit in? I don’t mean in the organization, they had welcomed me with open arms and the accompanying intern-hazing, but rather as a developer in the grander scheme of things.

This is something I’ve always wrestled with. The normal analogies try to say we are scientists because of the degree to which we value quantification. Or that we are artists because we strive to create beauty in simple elegance. Or that we are architects because we scheme to create this beauty of elegance within a greater system, which in itself is beautifully elegant.

I’ve always thought maybe we were vanilla engineers, building the infrastructure and cool tools for the success of other disciplines. Or clowns, playing part in the grand comedy that’s life. This might be just me, but software makes me laugh pretty frequently, especially when I’ve written it and it sucks.

Kidding aside, architect used to be my favorite of the often-offered ideas. This was only magnified by my initial experience with our project. See, the specifications took a while to hash out for a variety of reasons, so before we had them in their entirety, Ben and I figured we’d create a gorgeous abstract system in which any of a variety of RESTful java applications could reside. It was tall, it was glassy. Maybe there was stainless steel, though I’ve always preferred aluminum.

Then as the specifications rolled in we just fit the pieces into this beautiful structure. We need a xeriscape garden on every floor. Check. Marble lobby. Check. Heated toilet-seats. Check. Our masterpiece was done. At the risk of running the metaphor dead, I’ll continue. Curiously, after we finished what we thought were the finishing touches, more contractors started showing up. The elevator guy wanted to install an elevator. “It’s not elegant, it’s not beautiful”, Ben and I implored, “It would cut up the xeriscape gardens in all the wrong places! What about their flow of energy?”.

There was a flash of realization. We had created something beautiful, sure. The framework we wrote was absolutely lovely, mostly due to Ben’s work; but when done it proved itself unsuitable in a lot of ways to the software we were actually building.

Around the time this was happening I found Stewart Brand’s series How Buildings Learn online. The first of six parts takes thirty minutes and talks about all the ways in which beautiful architecture has failed the people who actually end up using the building. He talks about how unfit MIT’s media lab (an I.M. Pei) is for it’s purpose– the exchange and cultivation of ideas. From this sort of soft failure he talks about more concrete failures such as the un-tinted, large windows at the French National Library ruins books with heat, and on the other hand how the tinted windows in the BBC building make it impossible to get sunlight considering London’s climate. Through all of this there was, in the back of my mind, the ghostly image of what Ben and I had built.

It was heartbreaking. Overall, I think we had done a great job, but there were key design decisions revolving around the use of Hibernate that just kept getting in the way. It was absurd to the point where when given additional specifications or even upon realizing the existence of preexisting specifications rather than simply implement them, we instead fought hard to bodge that functionality into the beautiful but unfit framework. On reflection I wish that we had the spot-courage to rip down what needed to be ripped down. The 100ft tall statue of Perilous, the great greek god of abstraction, in the lobby where elevators should have been, comes to mind.

In the end, we did pull it all together, but I’m still nagged. I’m a decent programmer, Ben’s a better one and it was still painful to build this system. Upon reflection I think it comes down to something not really talked about in the programming world: courage.

It’s not a crime to build beautiful systems, Elegance is a merit– not a flaw, and abstraction is a key tool to both; but one needs to realize that these are not total goals unto themselves. The creation of functional software fits in as well. If the world were perfect, and thus specifications perfect and upfront this would not be an issue, keen eyes and minds could combine programmatic aesthetics and functionality ahead of time. Shall we invoke the idea of an intelligent designer? However, specifications are incomplete and fluctuating; thus the systems we build to meet those specifications must be able to do so as well. Shall we invoke the idea of evolution? The issue is that software does not fluctuate. It can be built to meet a variety of needs, it can be flexible, but the flexibility is not absolute. That’s what we ran into. We couldn’t stick around without creating anything and wait for complete specification before doing what we do, so we made a call and wrote the software we thought would be flexible enough to meet any foreseen requirements in our domain.

Ultimately, we thought wrong, but instead of saying, “alright, this isn’t right.”, we said, “how can we force this unforeseen specification upon our lovely and elegant code-base”. When this process was over we had frankenstein. It was all backwards and jumbled, all buggy and unmaintainable. What we didn’t realize is that our framework wasn’t holy, didn’t have feelings, and wouldn’t press charges. We were afraid to hurt it. Instead, we should have ripped it apart. It wasn’t a black box, we wrote the damn thing, so why were we afraid to break it down to make it better? We simply didn’t have the courage.

It takes courage to destroy what you’ve created. Everyone’s got a bit of a god complex and wants what they’ve made to be perfect. Listen: even god wasn’t so lucky. You figure, though, that writers write and re-write to create the what best reflects their aim (I don’t, but I’m not such a great writer), why should programmers not do the same? If we had realized this and ripped the guts out of our prior work at an earlier time the whole experience would have been easier and more enjoyable. To a programmer, programming should be enjoyable, fighting with your own codebase gone-bad isn’t enjoyable. If you’re at that point make the codebase right. Then keep developing, not fighting. Have some courage.

When Brand asked one brick-and-mortar architect how he learned from past mistakes, he replied, “Oh, you never go back, it’s too discouraging”. Discouraging it is, but I’ve learned a lot.

Fixing the Can't move '.svn/tmp/entries' to '.svn/entries' error

Wednesday, July 2nd, 2008

I’d been having trouble deploying the latest version of a Flex Module due to an SVN error I had never come across before. I was trying to update the svn:externals property with a new URL but once I saved my change I would get the following error in Terminal:

Can’t move ‘.svn/tmp/entries’ to ‘.svn/entries’: Operation not permitted

I tried running an svn cleanup command but no luck. After digging through Google I came across an excellent post (it’s in Spanish) with a command to run that prevents the above error. Basically on the directory I was trying to run the svn propedit svn:externals . command, all I had to was run the following command first: chflags -R nouchg ./

After that I could update the svn:externals property without any errors. The blog post suggests that the issue could be with how Windows changes certain file attributes with later on updating on Mac OSX. Makes sense for me since I develop on Mac while the developer who originally made changes to the working copy is on Windows.

Also I want to note that I’ve noticed others were getting the same error but instead of it saying “Operation not permitted” it would say “Permission denied” which I myself have not come across. I’m not sure if the same command will solve that problem.

Building RESTful Web Apps with Groovy and Restlet, Part 2: Resources

Friday, June 13th, 2008

In Part 1 of this series, we set up a development environment for our apps, and created and tested a simple “Hello, world” example. In this installment, we’ll make our app more realistic and useful.

If we revisit the current makeup of restfulapp.groovy, we see that there are two elements: the class RequestHandler, and the line which creates and starts an HTTP server. RequestHandler has only one method: handle(), which is called for every request. handle() checks whether the request method is GET, which is the only method supported by the app at this stage, and responds with “Hello, world!” if it is, and with an error response if it isn’t.

It works, but there’s definitely some key elements missing from the picture at this point – namely resources and representations. While our RequestHandler is indeed responding to requests and is therefore doing actual work, it’s not particularly RESTful. There’s no clear resource specified, and no representations of the state of that resource transferred back and forth. So that’s what we’ll add to the app now.

Let’s update RequestHandler so it’s no longer a generic “request handler” class, but instead models a resource. I’m not going to go into REST theory here to discuss what a resource is; if you could use a primer see your favorite REST tutorial or reference, or refer to the relevant section of The Dissertation.

So what resource should our class model? For the purposes of this article, we could choose almost anything. But we’ll want something that’ll allow us to demonstrate some important RESTful concepts, and to build something that resembles real-world resources in transactional software systems.

How about a mailbox? It’s a good candidate because it has state of its own, which would be retrieved with GET, might support both PUT to update its own state, and POST to accept entities for processing, and maybe even DELETE. It also might have “child resources“: messages.

Let’s try changing the name of RequestHandler to MailboxResource and see what it looks like:

class MailboxResource extends Restlet
{
void handle(Request request, Response response)
{
if (request.method == Method.GET)
{
response.setEntity("Hello, world!", MediaType.TEXT_PLAIN)
}
else
{
// The request method is not GET, so set an error response status
response.setStatus(Status.CLIENT_ERROR_METHOD_NOT_ALLOWED)
response.setAllowedMethods([Method.GET] as Set)
}
}
}

Next, let’s change the handling of GET to return a meaningful representation of the current state of the mailbox:

if (request.method == Method.GET)
{
response.setEntity("access: closed; messages: 0;", MediaType.TEXT_PLAIN)
}

OK, it’s primitive. But it’s a plausible representation of the current state of the mailbox – and that’s all we need at this point.

Next, let’s add support for PUT, which a client might use to update the open/closed state of the Mailbox, and POST which a client might use to put a message into the Mailbox.

A first attempt to do so might look like this:

switch (request.method)
{
case Method.GET:
response.setEntity("access: closed; messages: 0;", MediaType.TEXT_PLAIN)
break
case Method.PUT:
if (request.attributes.access)
{
this.access = request.attributes.access
}
else
{
response.setStatus(Status.CLIENT_ERROR_BAD_REQUEST, "The request representation must include the parameter 'access'.")
}
break
case Method.POST:
if (request.attributes.messageContent)
{
createMessage(request.attributes.messageContent)
}
else
{
response.setStatus(Status.CLIENT_ERROR_BAD_REQUEST, "The request representation must include the parameter 'messageContent'.")
}
break
default:
// The request method is not allowed; set an error status
response.setStatus(Status.CLIENT_ERROR_METHOD_NOT_ALLOWED)
response.setAllowedMethods([Method.GET, Method.PUT, Method.POST] as Set)
}

This code would work, as long as we implemented the field this.access set on line 10, and the method createMessage(), called from line 21. But as you can see, our once-svelte method is getting fairly unwieldy, and at this point we’re still only doing very basic processing for our requests. You can imagine that once we need some more complex processing, our method could become impossible to work with.

OK, no problem, we all know what to do in situations like this: create class methods to handle each of our allowed HTTP methods. After which, MailboxResource might look something like this:

class MailboxResource extends Restlet
{
void handle(Request request, Response response)
{
switch (request.method)
{
case Method.GET:
handleGet(request, response)
break
case Method.PUT:
handlePut(request, response)
break
case Method.POST:
handlePost(request, response)
break
default:
// The request method is not allowed; set an error status
response.setStatus(Status.CLIENT_ERROR_METHOD_NOT_ALLOWED)
response.setAllowedMethods([Method.GET, Method.PUT, Method.POST] as Set)
}
}
void handleGet(request, response)
{
response.setEntity("access: closed; messages: 0;", MediaType.TEXT_PLAIN)
}
void handlePut(request, response)
{
if (request.attributes.access)
{
this.access = request.attributes.access
}
else
{
response.setStatus(Status.CLIENT_ERROR_BAD_REQUEST, "The request representation must include the parameter 'access'.")
}
}
void handlePost(request, response)
{
if (request.attributes.messageContent)
{
createMessage(request.attributes.messageContent)
}
else
{
response.setStatus(Status.CLIENT_ERROR_BAD_REQUEST, "The request representation must include the parameter 'messageContent'.")
}
}
}

Definitely easier to read, but still fairly verbose. That switch statement is looking kinda dumb at this point; it’s not doing much.

So let’s use some dynamic Groovy magic to get rid of it:

void handle(Request request, Response response)
{
def allowedMethods = [Method.GET, Method.PUT, Method.POST] as Set
if (request.method in allowedMethods)
{
def handleMethodName = "handle" + request.method.name.substring(0, 1).toUpperCase() + request.method.name.substring(1).toLowerCase()
this."$handleMethodName"(request, response)
}
else
{
// The request method is not allowed; set an error status
response.setStatus(Status.CLIENT_ERROR_METHOD_NOT_ALLOWED)
response.setAllowedMethods(allowedMethods)
}
}

OK, now we’re getting somewhere! Easy dynamic method invocation – pretty cool.

Thing is, our handle() method is at this point barely doing anything – just figuring out whether the requested HTTP method is allowed, and calling the appropriate class method if it is, or returning an error if it isn’t. We shouldn’t have to implement this standard logic in every single one of our resources, that’d be pretty redundant.

Thankfully, the developers of Restlet realized this, and they created a very useful class which takes care of much of this sort of overheard for you. It’s named, simply, Resource.

Let’s convert MailboxResource to extend Resource. We’ll do this in a few steps, to illustrate some of the work that Resource does for us.

Let’s start MailboxResource from scratch:

#!/usr/bin/env groovy -classpath org.restlet.jar:com.noelios.restlet.jar
import org.restlet.*
import org.restlet.data.*
import org.restlet.resource.*
class MailboxResource extends Resource
{
static String access = 'closed'
static int messageCount = 0
def MailboxResource(Context context, Request request, Response response)
{
super(context, request, response)
variants.add(new Variant(MediaType.APPLICATION_WWW_FORM))
modifiable = true
}
}
// Create a Finder which will create new instances of MailboxResource as needed
finder = new Finder(new Context(), MailboxResource.class)
/* Create a new HTTP Server on port 3000, pass it the Finder,
to which it will pass all incoming Requests, and start it. */
new Server(Protocol.HTTP, 3000, finder).start()

As you can see, there’s a new import on line 5, because Resource is in a Restlet sub-package. Now’s also a good time to add some static fields to store the state of the resource, on lines 9 and 10. Normally we wouldn’t store state this way; this is just more convenient for now than using a real persistent data store.

Next, you’ll see that MailboxResource now has a constructor. This may seem like annoying overhead, but trust me, it’ll be worth it. For example: the framework will pass the request and response to the constructor, which passes them to the super constructor, which stores them in class fields. This means there’s no more need to pass the request and response around to every method – we can just refer to this.request and this.response.

There’s some other good things going on in the constructor. On line 15 we add a Variant to the variants collection; this is used for HTTP content negotiation – which Resource does for you. That’s right – one of the trickiest aspects of implementing a sophisticated RESTful web app, and you get it for free just by using Resource. Good deal! I’ll demonstrate content negotiation in a later article in this series.

The last line of the constructor sets the resource as modifiable; by default instances of Resource only allow GET; setting modifiable to true allows PUT, POST, and DELETE.

We don’t want to allow DELETE for now, so let’s add this method:

def boolean allowDelete()
{
return false
}

It’s time to add our request handling logic back into the application. Let’s add this method:

def Representation represent(Variant variant)
{
def form = new Form()
form.add("access", access)
form.add("messages", messageCount as String)
return form.webRepresentation
}

Notice that the new method isn’t named “handle” or “handleGet”; instead it’s represent(). There are many good and useful reasons for this, but I’m not going to go into all of them at the moment. I’ll just point out one for now: after processing a PUT or POST request, a common behaviour is to return a representation of the new, altered, state of the resource. So it’s not only when responding to GET that we need to construct a representation of the resource; it’s also frequently needed for other methods. Therefore, a specific method for this need makes a lot of sense.

All we need to do now is implement PUT and POST. Here’s PUT:

def void storeRepresentation(Representation representation)
{
def form = new Form(representation)
if (form.getFirst("access"))
{
access = form.getFirstValue("access")
response.entity = represent()
}
else
{
response.setStatus(Status.CLIENT_ERROR_BAD_REQUEST, "The request representation must include the parameter 'access'.")
}
}

And as you can see, the HTTP method PUT is handled by a method named storeRepresentation(). This is because Restlet attempts to model REST semantics in its API. The framework passes the request representation into the method, because it is the main input of the operation.

On line 3 we see another useful Restlet class: Form. As you might guess, this is convenient for working with web forms.

Finally, let’s add support for POST:

def void acceptRepresentation(Representation representation)
{
if (access != 'open')
{
response.setStatus(Status.CLIENT_ERROR_BAD_REQUEST, "A message can only be put into the mailbox if it is open.")
return
}
def form = new Form(representation)
if (form.getFirst("message_content"))
{
messageCount++
response.entity = represent()
}
else
{
response.setStatus(Status.CLIENT_ERROR_BAD_REQUEST, "The request representation must include the parameter 'message_content'.")
}
}

Normally, an operation like this would create a sub-resource, and the URL of that resource would be returned as a response header, along with the response status 201 Created. This will be illustrated in a later article in this series.

And that’s it, we’re done with our first Resource! We just need to make one last change to our script to make it all work; we need to change the last line to:

// Create a Finder which will create new instances of MailboxResource as needed
finder = new Finder(new Context(), MailboxResource.class)
/* Create a new HTTP Server on port 3000, pass it the Finder,
to which it will pass all incoming Requests, and start it. */
new Server(Protocol.HTTP, 3000, finder).start()

I’ll explain the Finder class in a later article.

As the comments indicate, a new instance of MailboxResource will be created for every single HTTP request. At first this might seem inefficient, but it means that every Resource class is inherently threadsafe – no small value there. And it’s still plenty fast: my ab script from Part 1 now yields 688 requests per second.

Tests!

Let’s update testfulapp.groovy to test our new resource:

#!/usr/bin/env groovy -classpath org.restlet.jar:com.noelios.restlet.jar
import org.restlet.*
import org.restlet.data.*
client = new Client(Protocol.HTTP)
mailboxResourceUrl = "http://localhost:3000/mailbox"
/*** Test 1: at first, access to the mailbox should be closed ***/
response = client.get(mailboxResourceUrl)
assert response.status.code == 200
assert response.entity.mediaType.equals(MediaType.APPLICATION_WWW_FORM, true)
assert response.entityAsForm instanceof Form
assert response.entityAsForm.getFirstValue("access") == "closed"
/*** Test 2: we shouldn't be able to post messages when the mailbox is closed ***/
form = new Form()
form.add("message_content", "whatever")
response = client.post(mailboxResourceUrl, form.webRepresentation)
assert response.status.code == 400
assert response.isEntityAvailable() == false
/*** Test 3: change the mailbox access to "open" ***/
form = new Form()
form.add("access", "open")
response = client.put(mailboxResourceUrl, form.webRepresentation)
assert response.status.code == 200
assert response.isEntityAvailable() == true
assert response.entityAsForm instanceof Form
assert response.entityAsForm.getFirstValue("access") == "open"
messageCount = response.entityAsForm.getFirstValue("messages") as Integer
/*** Test 4: we should be able to post a message now that the mailbox is open ***/
form = new Form()
form.add("message_content", "whatever")
response = client.post(mailboxResourceUrl, form.webRepresentation)
assert response.status.code == 200
assert response.isEntityAvailable() == true
assert response.entityAsForm instanceof Form
assert response.entityAsForm.getFirstValue("access") == "open"
assert response.entityAsForm.getFirstValue("messages") as Integer == messageCount + 1
/*** Test 5: change the mailbox access to "closed" ***/
form = new Form()
form.add("access", "closed")
response = client.put(mailboxResourceUrl, form.webRepresentation)
assert response.status.code == 200
assert response.isEntityAvailable() == true
assert response.entityAsForm instanceof Form
assert response.entityAsForm.getFirstValue("access") == "closed"
println "\nAll tests passed successfully!\n"

That’s all, folks!

And that’s it! We’ve now used Restlet to implement a truly RESTful resource. And we used some Groovy goodness as well: switch, Groovy Truth, dynamic method invocation, casting, and more.

I hope this second installment in the series has been interesting and valuable for you, and I look forward to your feedback, comments, critiques, and constructive criticism.

Coming up in Part 3: Routing. And it’ll be shorter than this monstrosity, I promise.

Full source of restfulapp.groovy

#!/usr/bin/env groovy -classpath org.restlet.jar:com.noelios.restlet.jar
import org.restlet.*
import org.restlet.data.*
import org.restlet.resource.*
class MailboxResource extends Resource
{
static def access = 'closed'
static def messageCount = 0
def MailboxResource(Context context, Request request, Response response)
{
super(context, request, response)
variants.add(new Variant(MediaType.APPLICATION_WWW_FORM))
modifiable = true
}
def boolean allowDelete()
{
return false
}
def Representation represent(Variant variant)
{
def form = new Form()
form.add("access", access)
form.add("messages", messageCount as String)
return form.webRepresentation
}
def void storeRepresentation(Representation representation)
{
def form = new Form(representation)
if (form.getFirst("access"))
{
access = form.getFirstValue("access")
response.entity = represent()
}
else
{
response.setStatus(Status.CLIENT_ERROR_BAD_REQUEST, "The request representation must include the parameter 'access'.")
}
}
def void acceptRepresentation(Representation representation)
{
if (access != 'open')
{
response.setStatus(Status.CLIENT_ERROR_BAD_REQUEST, "A message can only be put into the mailbox if it is open.")
return
}
def form = new Form(representation)
if (form.getFirst("message_content"))
{
messageCount++
response.entity = represent()
}
else
{
response.setStatus(Status.CLIENT_ERROR_BAD_REQUEST, "The request representation must include the parameter 'message_content'.")
}
}
}
// Create a Finder which will create new instances of MailboxResource as needed
finder = new Finder(new Context(), MailboxResource.class)
/* Create a new HTTP Server on port 3000, pass it the Finder,
to which it will pass all incoming Requests, and start it. */
new Server(Protocol.HTTP, 3000, finder).start()

Building RESTful Web Apps with Groovy and Restlet, Part 1: Up and Running

Wednesday, June 4th, 2008

Here at Arc90, we love REST. We’ve implemented RESTful web apps using ColdFusion, Java, .NET/C#, and PHP. We’re big fans of Restlet, a Java REST framework. I’ve also recently started experimenting with Groovy, “an agile dynamic language for the Java Platform,” and so far I’m liking it.

So it’s a no-brainer for us to put Groovy and Restlet together and see what happens. So far, I like what I see. That said, it’ll be tough to convince some of my PHP-loving and Pythonistic colleagues to even consider Groovy, given its Java roots. Some of them fairly shudder when they hear those two syllables “ja-va”. Ooh, scary.

This blog series will be my attempt to present how Groovy and Restlet together create a compelling platform for rapid RESTful web application development.

So let’s get started with the classic Hello world!

Prerequisites

Before we get started, two prerequisites:

  1. Java 1.5 or newer – download (I used 1.5.0_13)
  2. Groovy 1.5 or newer – download (I used 1.5.6)

Setup

  1. Download the latest version of Restlet 1.1 – link – and extract it anywhere
  2. Create a folder named restfulapp
  3. Create a file named restfulapp.groovy
  4. Copy two files from the Restlet distribution’s lib folder to restfulapp:
    • org.restlet.jar
    • com.noelios.restlet.jar

I’m specifying Restlet 1.1, which hasn’t yet hit final release, as opposed to the stable 1.0 branch, because it has a built-in HTTP server and client, which makes it really easy to just dive right into development and have something up and running quickly. These built in HTTP components aren’t production-class, but they’re very convenient for our purposes here.

The Code

Before we get into the code, please note: this code does not demonstrate best practices, and is not production ready; at this point I just want to establish basic concepts.

restfulapp.groovy

Copy the following code into restfulapp.groovy:

#!/usr/bin/env groovy -classpath org.restlet.jar:com.noelios.restlet.jar
import org.restlet.*
import org.restlet.data.*
// RequestHandler will handle all of our requests
class RequestHandler extends Restlet
{
// handle() is called by the framework whenever there's a HTTP request
def void handle(Request request, Response response)
{
/* The only method that's allowed is GET, so check it -- if it's GET,
return a representation, and the default status will be returned,
which is 200 OK. If not, set the response status to an error. */
if (request.method == Method.GET)
{
/* handle() returns void, for reasons that will become clear
later. The way to output content is modify the passed-in response
by setting the Entity of the response. Restlet refers to the
Representation as an "entity"; I'm actually not 100% sure why. */
response.setEntity("Hello, world!", MediaType.TEXT_PLAIN)
}
else
{
// The request method is not GET, so set an error response status
response.setStatus(Status.CLIENT_ERROR_METHOD_NOT_ALLOWED)
response.setAllowedMethods([Method.GET] as Set)
}
}
}
/* Create a new HTTP Server on port 3000, pass it a new instance of RequestHandler,
to which it will pass all incoming Requests, and start it. */
new Server(Protocol.HTTP, 3000, new RequestHandler()).start()

I’m hoping this code illustrates some of the great aspects of Restlet and Groovy. Restlet’s class model is clear and intuitive because it was designed specifically to model REST concepts, so a server is a Server, a method is a Method, a Request is a request, etc. And because this app is a Groovy script, we can use inline class definitions, and the syntax is nice and clean and easy.

I particularly like that Groovy maps the == operator to “value equality” as opposed to “identity equality” as in Java. The Java equivalent of line 16 would be:

if (request.getMethod().equals(Method.GET))

This also illustrates Groovy’s automatic getter referencing: to retrieve the method of the request, in Groovy I can use the syntax request.method, and Groovy automatically converts it into request.getMethod(). Nice!

Up and Running

Unices

$ chmod +x restfulapp.groovy
$ ./restfulapp.groovy

Windows

Launch a command prompt window, navigate to restfulapp, and run the command:

groovy -classpath org.restlet.jar;com.noelios.restlet.jar restfulapp.groovy

Test!

If you have cURL installed, you can run a quick ad-hoc test with the following command:

curl -v http://localhost:3000

But Groovy can make unit testing really easy:

Create a new file named testfulapp.groovy and copy the following code into it:

#!/usr/bin/env groovy -classpath org.restlet.jar:com.noelios.restlet.jar
import org.restlet.*
import org.restlet.data.*
client = new Client(Protocol.HTTP)
response = client.get("http://localhost:3000/")
assert response.status.code == 200
assert response.entity.mediaType.equals(MediaType.TEXT_PLAIN, true)
assert response.entity.size == 13
assert response.entity.text == "Hello, world!"
println "\nAll tests passed successfully!\n"

Open a new command prompt window, alongside the one already running restfulapp.groovy, and run testfulapp.groovy using the same steps outlined above for restfulapp.groovy for your OS. If all goes well, you should see “All tests passed successfully!” as the result!

Performance

Just for fun, if you’ve got a copy of ab installed:

ab -c 2 -t 10 http://localhost:3000/

Any numbers you see from ab can only be compared to your own, because everyone’s hardware and software varies, but it’s still fun to try pounding on your code and see what happens. I got 713 requests per second on my 2.2 GHz Santa Rosa Core 2 Duo MacBook, on OS X 10.5.3 and Java 1.5.0_13 – not bad!

That’s all, folks!

And that’s all for now. I hope this has been interesting and/or informative. I’d love to hear some feedback from readers: has this article made you interested in learning more about Groovy and/or Restlet?

Coming soon: Part 2: Resources.

Update, June 13th: Just published Part 2: Resources.

WizTools.org RESTClient 2.1 Released, Lauds Yours Truly

Thursday, March 13th, 2008

RESTClient is an excellent GUI app for RESTful usage of HTTP; kind of like a GUI cURL. It’s very useful for testing RESTful web applications, particularly for testers who aren’t used to cURL and don’t want to learn its syntax for only occasional usage.
Start using!