Latest ESXX Release


Sunday, February 22, 2009

JavaScript web applications/browser services, the ESXX way

In ESXX, the difference between a web service (a program that produces XML or JSON intended for other programs) and a web application or browser service (a program that produces HTML documents intended to be viewed by a human using a web browser) is minimal. Basically, it's only the data format (XML/JSON vs HTML) that differs.

It's a good idea to remember this when you build a web application. The response from your request handlers should be raw XML objects that includes all information that is unique for this particular web page or form, but nothing more. Layout, common headers and footers, static sidebars … everything that is common to all pages should be added by the XSLT stylesheet and/or external CSS stylesheets.

(Similarly, if you use filters to plug in a JavaScript template engine instead of using ESXX' built-in XSLT engine, your handlers should return plain JavaScript objects, containing the information that is unique for the page in question.)

ESXX provides two tools that are useful for building web applications: stylesheet handlers and request filters. Both can be used when building web services as well. Especially filters are useful for both kind of services.

Stylesheet handlers

As mentioned before, stylesheet handlers can be triggered by both HTTP and SOAP handler responses. When such a handler returns XML, the registered stylesheet handlers are searched based on the response's content type and the part of the URI that follows the .esxx file.

The handler specifies an XSLT 2.0 stylesheet that will, just like the application itself, be compiled and cached in memory until it times out or the file is modified. The stylesheet will then be applied to the response on every request. The params property of the ESXX.Response object can be used to set stylesheet parameters (response.params.mode refers to the <xsl:param name="'mode'/"> XSLT parameter).

If you're used to thinking in Model/View/Controller terms, the matched stylesheet is a collection of views and the name of the root element in the data returned by the request handler (the controller) is the name of the view to apply.

XSLT 2.0 is much more advanced and useful than XSLT 1.0, which is currently implemented by the browsers, but should the need arise, it's possible to call any JavaScript function from an XPath context using javascript: URIs.

For instance, the following JavaScript function, defined by your web application
function MyClass() {}

function MyClass.protoype.getCurrentDate() {
let now = new Date();

return <currentDate>
<day>{now.getDate()}</day>
<month>{now.getMonth() + 1}</month>

<year>{now.getFullYear()}</year>
</currentDate>;
}

var myObject = new MyClass();
can be used by the XSLT stylesheet like this:
The current year is <xsl:value-of my="javascript:myObject" select="my:getCurrentDate()/year">.
Global functions can be called by leaving out the object from the URI:
Today is <xsl:value-of my="javascript:" select="my:Date()">.
It's also possible to call class or instance Java methods:
The current year is <xsl:value-of my="java:java.util.Date" select="my:getYear(my:new()) + 1900">.
Request filters

Request filters differ from request handlers in that more than one filter may be invoked for a single request. They are defined as follows:
<esxx xmlns="http://esxx.org/1.0/">
<handlers>

</handlers>

<filters>

<filter method={http-method}
uri={path-info}
handler={object-and-method} />
<filter … />

</filters>
</esxx>
For each request, a list of matching filters are built and the first filter in the list is invoked with two parameters: the request object and a function that calls the next filter in the list, or, if the current filter is the last, a function that calls the request handler.

Each filter is supposed invoke the next function and return its value. This way, the request handler will eventually be invoked and its response will propagate back to the client via the filter handlers. By simply not calling the next function, a filter can abort the request and return its own response.

(The next function optionally takes a single argument, which will be passed as the req parameter to the next filter or handler. If unspecified, the current request object is passed. This way, it's possible to completely replace the request object.)

Here are a few filter handler examples:
function noOpFilter(req, next) {
return next();
}

function forbiddenFilter(req, next) {
// Abort request by not calling next() if cookie not set
if (req.cookies.secret != "Open Sesame") {
return [ESXX.Response.FORBIDDEN, {}, "Access denied"];
}
else {
return next();
}
}

function postFilter(req, next) {
let res = next();

// Set XSLT params
res.params.mode="silver";

// Add HTTP response header
res.headers["Cache-Control"] = "max-age=1800, public";

return res;
}

Saturday, February 21, 2009

JavaScript request filters

It's never to late to change your mind, and I'm happy to let you know that I checked in support for request filters in the trunk yesterday.

Filters differ from handlers in that more than one filter may be executed for a given request. Before a request is serviced, a matching filter chain is built and all all matching filters are then executed in turn as part of the request handling.

Each filter may modify (or even replace) both the request and response object or simply abort the request by not calling the next filter in the filter chain.

In ESXX, a filter is called with two parameters: the request object and a function that calls the next filter in the chain. The filter is supposed to return a response that will be sent to the client (optionally passing through the XSLT engine). That means that a filter may:
  • Abort the request by not calling the next() function at all.
  • Tweak the request by modifying the request object before calling the next() function.
  • Tweak the response by modifying the result of the next() function.
Filters are great for password-protecting parts of a site, setting XSLT stylesheet parameters or modifying responses (like converting XML to JSON or processing views in JavaScript instead of using ESXX' built-in XSLT engine).

Do have a look at an example to see how ESXX filters work.

Sunday, February 15, 2009

JavaScript servlets

Today, I checked in support for running ESXX as a inside servlet a Java EE application server. Why, someone may ask, run an application server within another application server?

Well, for starters, it allows you to use ESXX with your existing infrastructure and tool set. If you're already using an app server such as Glassfish or Tomcat, you can add modules powered by ESXX by rebuilding the war archive. Simply add your JavaScript code files, change the http-root parameter in web.xml, rebuild and deploy the generated esxx.war archive on your app server.

More work is required, but if you're interested to try it, just check out trunk from the Subversion repository at http://svn.berlios.de/svnroot/repos/esxx/trunk/ and type ant jee-war to build your own esxx.war archive. By default, it serves the ESXX example applications.

Sunday, February 8, 2009

JavaScript web services, the ESXX way

By "web service", we mean a service that is intended to be used by a program, not directly by a human surfing the web with a web browser.

To use ESXX as a web service engine, you first need to define what request handlers should be available. ESXX currently provides six kinds of handlers: HTTP, SOAP, stylesheet, timer, exit and error handlers. Handlers are defined using a section of the main document within the http://esxx.org/1.0/ namespace:
<esxx xmlns="http://esxx.org/1.0/">
<handlers>
<http method={http-method} uri={path-info} handler={object-and-method} />
<soap action={soap-action} uri={path-info} object={object} />

<error handler={object-and-method} />
<stylesheet media-type={content-type} uri={path-info} href={xslt-file} type="text/xsl" />
<timer delay={delay-in-seconds} period={period-in-seconds} handler={object-and-method} />
<exit handler={object-and-method} />
</handlers>

</esxx>
HTTP handlers
HTTP handlers are JavaScript functions that are triggered by a normal HTTP request. What handler to call is defined by the request method and the part of the URI that follows the .esxx file, which defines the web application.
SOAP handlers
SOAP handlers are JavaScript objects that are triggered by SOAP request, which means a POST request with the SOAPAction HTTP header set (SOAP 1.2 is not yet supported but will be in the future). The object to act upon is defined by the SOAP action and the part of the URI that follows the .esxx file, which defines the web application. What method to call on the object is determined from the SOAP request's Body element.
Error handlers
Error handlers are JavaScript functions that will be called whenever a HTTP or SOAP handler throws an exception and gives the application a chance to recover or to produce a more good-looking error message than ESXX's default error message.
Stylesheet handlers
Stylesheet handlers are XSLT stylesheets that are triggered by XML response from the HTTP, SOAP and error handlers. What stylesheet to execute is defined by the content type and the part of the URI that follows the .esxx file, which defines the web application.
Timer handlers
Timer handlers are JavaScript functions that will be triggered once the application has been loaded and the periodically for the lifetime of the app.
Exit handlers
Exit handlers are JavaScript functions will be executed by ESXX when an application is unloaded. Applications are unloaded if they have been idle for a while, if any of the source code files they were compiled from change or as a direct command via the JMX interface.
Methods, SOAP actions, Media types and URIs
The attributes method, action, media-type and uri are all interpreted as Java (not JavaScript!) regular expressions, with one important addition: they allow named groups using the (?{name}...) syntax. More on that in the next section.

Multiple handlers of a given type are matched in the order they appear in the document. When an application is first loaded, all handler rules of a given type are merged and compiled into a single NFA, which is then used for each request to determine what handler to call.

(Normal Java regex recommendations apply: try to use non-greedy multipliers and use expression like [^a]*a instead of .*a wherever possible, in order to limit the amount of backtracking required.)

Here are a few examples:
...
<http method="GET|HEAD" handler="global.getHanlder" />

<http method="POST" uri="/upload" handler="myUploadHandler" />
<soap uri="/soap" object="soapObject" />

<soap action="urn:xmethods-delayed-quotes" uri="/" object="stocks" />
<stylesheet media-type="application/xml" uri="(?!soap$).*" href="anything-but-soap.xslt" />

...
Handlers, objects, hrefs and Request objects
The handler, object and href attribute specify JavaScript functions, JavaScript object and XSLT stylesheets, respectively. Named parameters from the attributes method, action, media-type and uri attributes may be inserted by using the {name} syntax. For instance, you the following HTTP handler invokes one of the following methods on object: handleDELETE() handleGET(), handleHEAD(), handlePUT():
<http method="(?{verb}GET|HEAD|PUT|DELETE)" handler="object.handle{verb}" />
Additionally, all parameters are available in the args property of the Request object that is passed to the matched JavaScript handler. Not only does this allow full JavaScript control of how to handle a request, it also means that it's easy to pass parameters in the URI:
<http method="GET" uri="/products/(?{category}\w+/(?{id}\d+)" handler="shop.getProductInfo" />
When the method shop.getProductInfo(req) is triggered by an HTTP GET request to .../app.esxx/products/games/10, req.args.category will refer to a word indicating the category ("games") and req.args.id will be a sequence of decimal digits indicating the item ID (10).

Regular expressions and named groups are very powerful, but can be error prone and hard to read. For large web applications with lots of defined handlers, XML entities can be used to create aliases for common URI parameters that are used in many handlers:
<?xml version="1.0" ?>

<!DOCTYPE esxx [
<!ENTITY user_id "(?{user_id}\d+)">
<!ENTITY email "(?{email}[a-zA-Z][\p{Alnum}_\-.+]*@([\p{Alnum}_\-]+\.)+[a-zA-Z]{2,6})">
]>

...

<esxx>
<handlers>
<http method="POST" uri="/users/&user_id;/sendmail/&email;" handler="sendMail" />

...
</handlers>
</esxx>
HTTP handlers
HTTP handlers handle plain HTTP requests based on the request method and the URI. The handler specifies a JavaScript function (like myHandler) or object and method (like object.myMethod), which will be called with an initialized object as single parameter.

As noted above, any groups in the regular expressions that caused the handler to be called are available in the args property of the ESXX.Request object. The contentType property will contain the basic content type (without parameters) and message property will contain the parsed request body.

The handler should either return an ESXX.Response object, an Array (whose elements will be used as constructor parameters to initialize a new ESXX.Response object), a number (which indicates the HTTP status code and an empty body) or any other supported object, which will be automatically wrapped in an ESXX.Response object with HTTP status 200 and no extra headers (the content-type will be guessed based on the object's type).
  • If the payload of the ESXX.Response object is an E4X or DOM node, a stylesheet handler will be looked up and if found, the XSLT stylesheet will be used to transform the node.
  • Otherwise, the ESXX.Response payload will simply be serialized and sent to the client.
  • If the handler throws an exception, the error handler, if defined, will be invoked.
The most common form of return values are XML nodes ("application/xml") or plain JavaScript objects ("application/json"), but it's also possible to return Java InputStreams or RenderedImage objects.

SOAP handlers
SOAP handlers handle SOAP 1.1 requests based on the SOAPAction HTTP header and the URI. The handler specifies a JavaScript object and the local-name of the first child of the Body element is used to determine what method to call on the object. The method will be called with an initialized ESXX.Request object as first parameter and the first child of the Body element (as an E4X node) as the second parameter.

As noted above, any groups in the regular expressions that caused the handler to be called are available in the args property of the ESXX.Request object.

The soapAction property will contain the value of the SOAPAction HTTP header and the message propery will contain a Java javax.xml.soap.SOAPMessage object.
  • The return value is handled just like for HTTP handlers and the payload should be an XML node. Unlike HTTP handler responses, however, if the XML node's local-name is not "Envelope", a SOAP envelope will automatically be added.
  • If the handler throws an exception, the error handler, if defined, will be invoked.
The implementation is very simple, but should be quite usable for basic SOAP requests. If a real SOAP service is required, you should use normal HTTP handlers and call the appropriate Java SOAP framework from there. (Or, you could just reconsider and use HTTP the way it was intended to be used.)

Error handlers
Whenever an HTTP or SOAP handler throws an exception, the error handler, if defined is invoked. The handler specifies a JavaScript function (like myHandler) or object and method (like object.myMethod), which will be called with the original ESXX.Request object as first parameter and the exception object as the second parameter.

The handler may return null or undefined to act as if there were no handler installed in the first place, or return a new response in the same way HTTP handlers return responses.

This handler should not throw exceptions.

Stylesheet handlers
Stylesheet handlers handle responses from HTTP or SOAP handlers based on the content type and the URI. The handler specifies an XSLT stylesheet that will be executed if the reponse was an XML node.

Please stay tuned for more information about the stylesheet handlers. I'll post more information about them later.

Timer handlers
Periodic or one-shot timers may be installed by specifying timer handlers. The delay and period is specified in seconds (decimal numbers are accepted). The handler specifies a JavaScript function (like myHandler) or object and method (like object.myMethod), which will be called with a single Date parameter, which indicates the time when the handler was scheduled to be executed.

A timer handler is never executed more than once at a given time. If a timer handler takes too long to complete, subsequent events will be discarded until the handler returns.

No return value is expected. This handler should not throw exceptions.

Exit handlers
When an ESXX application is about to be unloaded from memory, an exit handler may be called to perfrom final cleanup. The handler specifies a JavaScript function (like myHandler) or object and method (like object.myMethod), which will be called with no arguments.

No return value is expected. This handler should not throw exceptions.

Friday, February 6, 2009

F-Script vs. JavaScript


I read about F-Script on Ars Technica today and couldn't help wondering how the example might have looked like if written as an ESXX command line application. Something like this, perhaps?
#!/usr/bin/env esxx-js

with (JavaImporter(java.awt, javax.swing)) {
function main() {
// Create the window
let window = new JFrame("TimeSetter");
window.setLayout(null);
window.setPreferredSize(new Dimension(383, 175));
window.setLocation(125, 513);
window.defaultCloseOperation = WindowConstants.EXIT_ON_CLOSE;

// Add a button
let button = new JButton("Push Me");
button.setBounds(100, 50, 150, 50);
window.add(button);

// Set up the two scripts -- one beeps, the other updates the window title
function beepScript() { Toolkit.defaultToolkit.beep(); }
function titleScript() { window.title = new Date(); }

// When the button is clicked, do something
button.addActionListener(titleScript);
window.rootPane.defaultButton = button;

window.pack();
window.visible = true;
esxx.wait(this);
}
}

Thursday, February 5, 2009

Server-side XML processing, the ESXX way

I've continued to add more documentation to the wiki. Let's talk about server-side XML processing via XSL this time.

With all due respect to command line applications, ESXX was designed for web applications.

An ESXX web application is defined by an XML file and zero or more JavaScript handlers or XSLT stylesheets. The XML file may be any XML file, and it will be available as an E4X node in the esxx.document variable. There are a few special constructs that may be present in it:
  • Other XML documents may be included using XInclude.
  • An <?esxx-stylesheet?> processing instruction may be specified to instruct ESXX to transform the document using the specified XSLT 2.0 stylesheet.
  • One or more <?esxx-include?> processing instructions may be used to load JavaScript code into memory.
  • One or more <?esxx?> processing instructions can also be used to specify JavaScript code in-line.
  • Various handlers may be installed by including elements from the http://esxx.org/1.0/ namespace in the document.

The most common situation is simply an http://esxx.org/1.0/ describing the handlers, plus <?esxx-stylesheet?> and <?esxx?> to set up the JavaScript environment, but let's look at a few examples first.

Plain XML
ESXX can serve static XML files. The file will be parsed, stored in esxx.document and serialized on every request. Hardly useful, but possible.

<?xml version="1.0"?>
<xml>Hello from a totally useless ESXX application!</xml>

XSLT transformation of static XML documents
Perhaps a bit more realistic, ESXX can be instructed to transform an XML document on each request. For instance, assume you have a DocBook document that you'd like to serve as HTML on the web. Instead of converting the document by hand every time you modify it, let ESXX serve and transform it using the XSLT 2.0 stylesheet specified by an <?esxx-stylesheet?> processing instruction in the document.

ESXX will load and parse the DocBook document and the stylesheet, and the compiled stylesheet will be executed on every request. If either of the DocBook document or the XSLT stylesheet is modified, the application will automatically be reloaded, which actually makes this a nice way to preview DocBook documents you're working on.

Here's a simple (non-DocBook) example:

<?xml version="1.0"?>
<?esxx-stylesheet type="text/xsl" href="useful.xslt"?>
<doc>
<para>Hello from a somewhat useful ESXX application!</para>
</doc>
And the stylesheet:
<?xml version="1.0"?>
<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns="http://www.w3.org/1999/xhtml">
<xsl:output
doctype-public="-//W3C//DTD XHTML 1.1//EN"
doctype-system="http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"
method="xhtml" version="1.0" media-type="text/html" />
<xsl:template match="@*|node()">
<xsl:copy>
<xsl:apply-templates select="@*|node()"/>
</xsl:copy>
</xsl:template>

<xsl:template match="/doc">
<html>
<body>
<xsl:apply-templates/>
</body>

</html>
</xsl:template>

<xsl:template match="para">
<p>
<xsl:apply-templates/>

</p>
</xsl:template>
</xsl:stylesheet>
JavaScript and XSLT handlers

Stay tuned ...

Wednesday, February 4, 2009

JavaScript command line applications, the ESXX way

Today, I added some documentation to the wiki about ESXX command line applications, and I though I'd post it here as well.

An ESXX command line application is one or more JavaScript files that define a main() function in the global scope. It's common to also begin the main JavaScript with a shebang, and set the executable flag, so it can easily be executed from the Unix command line. /usr/bin/env is usually used as a helper method, so it won't matter where the esxx-js program is located, as long as it's in the command search path.

#!/usr/bin/env esxx-js

... code follows ...

(Note that if the esxx-binfmt service is started, any executable JavaScript file may be executed directly from the command line shell, even if it lacks a shebang. Unfortunately, this currently only works in Linux.)

The command line application is loaded and executed like this:

  1. First, a new global scope is set up, with all the Core JavaScript, Rhino, ESXX and LiveConnect host objects present.
  2. The JavaScript file is loaded, compiled and executed with this set to the global scope.
    1. During the execution, one or more functions are defined in the global scope.
    2. esxx.include() may be called to bring in other JavaScript files into the global scope, either from the current directory or from the <esxx-root>/share directory. These files may call esxx.include() in turn; however, a single file will only be loaded and executed once — attempts to load a file that has already been loaded will be silently ignored.
  3. As a final step, ESXX invokes the main() function, with the main script file name as first argument followed by the command line arguments.

Below is an example of a very simple ESXX command line application that loads data from an URI, using the URI's default parser, and prints the result to the console.
#!/usr/bin/env esxx-js

var err = java.lang.System.err;
var out = java.lang.System.out;

function main(prg, location) {
if (!location) {
err.println("Usage: " + prg + " <location URI>");
return 10;
}

var uri = new URI(location);
var data = uri.load();

out.println(data);
return 0;
}

The script can be used for many purposes. Assuming the script is called load.js, it may be used to load a web page and convert it into XML:
[martin@elsa ~]$ ./load.js http://example.com
<html>
<head>
<title>Example Web Page</title>
</head>
<body>

...
</body>
</html>

But it can also be used as an DNS resolver, thanks to ESXX's dns: URI protocol implementation:
[martin@elsa ~]$ ./load.js dns:/example.com
<result>
<entry uri="dns:/example.com??base">

<a>208.77.188.166</a>
<ns>b.iana-servers.net.</ns>
<ns>a.iana-servers.net.</ns>
</entry>

</result>

ESXX command line applications can also be GUI applications by using Java's Swing GUI toolkit. Here's a small app that displays today's XKCD strip in a window:
#!/usr/bin/env esxx-js

with (JavaImporter(javax.swing)) {
function main() {
let atom = new Namespace("http://www.w3.org/2005/Atom");

let entry = new URI("http://xkcd.com/atom.xml").load().atom::entry[0];
let img_tag = new XML(entry.atom::summary.toString());
let image = new URI(img_tag.@src).load();

let frame = new JFrame(entry.atom::title);

frame.defaultCloseOperation = WindowConstants.EXIT_ON_CLOSE;
frame.add(new JLabel(new ImageIcon(image)));
frame.pack();
frame.locationRelativeTo = null;
frame.visible = true;

esxx.wait(this); // Wait forever
}
}

Exciting, isn't it?

Monday, February 2, 2009

What does "alpha" mean?

You might wonder what the fact that ESXX is still in "alpha" stage means. Is it a constant crashing piece of crap?

Far from it! ESXX has been used at a company (which shall remain unnamed for now) for almost six months, serving tens of thousands of user without problems.

No, "alpha" simply means that the API's are not yet completely stable and might change slightly. For example, in the last release, the Logger, Request and Response classes were moved inside the ESXX namespace in order to pollute the global namespace even less than before. Some existing code were affected, but not very much since you normally don't instansiate objects of any of these classes manually (with the exception of Response, perhaps).

With the next major release, I hope to finally freeze the API and move on to a beta period.

Sunday, February 1, 2009

Final alpha release?

It's been quite a while, but a new release is finally available for download. This time, there are proper RPM and DEB packages, as well as Opensolaris IPS and Mac OSX PackageMaker packeges. And a Windows installer, as usual of course.

Some of the highlights of the release include:
  • Proper RPM (Fedora/RHEL), DEB (Debian/Ubuntu), PackageMaker (OSX) and IPS (Opensolaris) packages.
  • dns, https URI handler.
  • multipart/form-data parser.
  • Better SOAP service support.
  • Improved stylesheet matching rules.
  • Timers.
  • Rhino 1.7 release 2 RC3, H2 Database 1.1.107.
  • Partial ECMAscript 3.1 compatibility provided by the "esxx/harmony.js" include file.
  • New JS class, LRUCache, which is used for thread-local and program-local data.
  • Improved JMX interface.
  • And lots more ...
If all goes well, this will be the final alpha release, meaning there will hopefully not be any more API changes that are not backwards-compatible.