Latest ESXX Release


Thursday, November 19, 2009

90% of all web apps are broken

A couple of weeks ago, ยต ran an article that stated that nice out of ten web applications are broken from a security standpoint. Half of the volunabilities were SQL injections and Cross-site scriping problems.

A few days later, hackers break in to a Brazilian power grid operator, using, you guessed it, an SQL injection attack.

Seriously, isn't is about time we stop accepting these kinds of failures? I think it is, and thats one of the reasons I wrote ESXX in the first place:

  • ESXX apps never build XML or HTML output as strings, they always operate on real XML nodes which ensures that all text nodes or attributes are properly quoted when output or transformed by the style-sheets. Meaning no cross-site scripting attacks, without effort!
  • In ESXX, it's easier to use prepared SQL statements than it is to build SQL queries by hand, which means that all your SQL parameters will always be properly encoded. And without encoding errors, there can be no SQL injection attacks.
For more information on how ESXX helps you write secure web applications, please read this wiki page.

Sunday, June 14, 2009

Using Apache's HttpClient on Google App Engine

If you, like me, have tried to use Google's URL Fetch Java API on the Google App Engine, you've probably been disappointed. Sure, it's a small, clean API, but it's totally feature-less. The most advanced thing it supports seems to be ... well, it can follow redirects automatically. Wow. Cookies? Authentication? Forget it.

In ESXX, I use Apache's HttpClient 4, and it works really well. Wouldn't it be nice if you could use HttpClient on the App Engine? Well, now you can. All it takes is a custom connection manager that converts the final requests and feed them into the URL Fetch service, and then feeds the responses back into HttpClient.

You can have a look at the implementation here. It's just two classes, one ClientConnectionManager and one ManagedClientConnection class.

PS. ESXX now runs really well on GAE. Timers, the http, https, mailto, jdbc (in-memory H2) and data URI protocols and HTML parsing, yep, works! Only the dns and ldap are non-functional (they will probably never work). Check out master from Git to try it yourself. Build using ant gae-war.

Updated 2009-12-11: The URIs were double-encoded and query parameters did not work at all. Thanks for pointing this out, Thibaut!

Updated 2010-08-08: The URIs lacked a colon before the optional port number. Thanks for pointing this out, Nello! Also, I changed the license for the two files to LGPLv3.

Updated 2014-11-18: Updated links to Git repo (moved to Github from Berlios).

Saturday, June 6, 2009

ESXX on Google App Engine

I got most of ESXX running on Google App Engine today! How cool is that?

Most of the code is in the subversion repository already. I'll try to finish the port as soon as possible and also add seamless support for Google's HTTP client APIs. Once fully checked in, you too can deploy ESXX + your custom JavaScript apps on GAE.

In the meantime, have a look at http://esxx-demo.appspot.com/.

The friendliest server-side JavaScript, now also on Google's servers. Life's sweet.

ThreadPoolExecutor deadlocks

I'm currently trying to get ESXX running on Google's App Engine. One of the problems are that GAE won't let you create background threads or timers, something which most applications, including ESXX, often do.

My initial plan was to switch from a plain ThreadPoolExecutor and Timers to a ScheduledExecutorService. Once done, I would write a GAE-specific, single-threaded version of that class tries to mimic the intended behaviour as good as possible.

The problem is that the ScheduledExecutorService class is a fixed-size thread pool using an unbounded work queue, which just doesn't work in ESXX. Consider this simple example:

import java.util.concurrent.*;

class STPEDeadlock {
public static void main(String[] args) {
final ExecutorService exe =
new ScheduledThreadPoolExecutor(1, new ThreadPoolExecutor.CallerRunsPolicy());

System.out.println(Thread.currentThread() + ": Submitting job 1");
Future j1 = exe.submit(new Runnable() {
@Override public void run() {
System.out.println(Thread.currentThread() + ": Submitting job 2");
Future j2 = exe.submit(new Runnable() {
@Override public void run() {
System.out.println(Thread.currentThread() + ": Running job 2");
}
});

try {
System.out.println(Thread.currentThread() + ": Waiting for job 2");
j2.get();
}
catch (Throwable t) {
t.printStackTrace();
}
}
});

try {
System.out.println(Thread.currentThread() + ": Waiting for job 1");
j1.get();
}
catch (Throwable t) {
t.printStackTrace();
}

System.out.println(Thread.currentThread() + ": All done");
exe.shutdown();
}
}
It creates a fixed-size thread pool, and adds and waits for a job. The job, in turn, adds and waits for another job. The problem is that there are no worker threads left, so the second job is just queued and never executed, resulting in a deadlock.

If you replace the executor with
final ExecutorService exe = new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new SynchronousQueue(),
new ThreadPoolExecutor.CallerRunsPolicy());
then all is fine again. This is the fixed-size executor configuration I use in ESXX. Unfortunately, there is no way to configure a ScheduledExecutorService to match this behavior.

What were the Sun developers thinking??

Sunday, May 24, 2009

Scaling up with GoGrid

My small GoGrid experiment the other day made me curios. Assume the blog I put online became really popular. How would my deployment cope, and how would I be able to increase capacity?

So I figured I'd script a small benchmark. Assume all visitors land on the Blog's front page, and that half of the visitors click on the first post to read it comments. Finally, assume a third of those add a comment of their own. Translated into a bash script that uses Apache ab, this scenario might look something like this:

#/bin/bash

AB="ab"
BASE=http://216.121.78.66/blog/index.esxx

function bench {
start=$(date +%s)

${AB} > /dev/null -n 600 -c 60 -e apa ${BASE} &
pid1=$!

${AB} > /dev/null -n 300 -c 30 -e apa ${BASE}/posts/1.html &
pid2=$!

${AB} > /dev/null -n 100 -c 10 -T application/x-www-form-urlencoded -p ab-blog.post ${BASE}/posts/1.html
pid3=$!

while [ -n "$(ps | grep -E "^ *(${pid1}|${pid2}|${pid3})")" ]; do
sleep 0.1
done

end=$(date +%s)

echo "Total time: $(expr ${end} - ${start})"
}

bench
bench
bench
bench
bench

As you can see, I assume 100 concurrent requests.

On my particular configuration (a 1 core, 0.5 GB RAM virtual GoGrid server in San Francisco and an OpenSolaris laptop in Sweden acting as the client), it turns out that the benchmark takes about 19 seconds per iteration, or 19 ms/request.

It's not too bad, but lets see if we can improve. First of all, we should switch from an embedded to a standalone SQL database. By watching 'top' on our single server, it appears that the ESXX Java process is using up all the CPU; however, since the DB is embedded, we don't know what is the limiting factor; it could be the XSLT transformation, the JavaScript code or the SQL queries.

Since the DB can't easily be scaled horizontally, we deploy a 3 core, 4 GB RAM virtual server and start H2 in server mode on that server, using the following command line:

java -cp h2/bin/h2-1.1.107.jar -Dh2.bindAddress=10.102.228.78 org.h2.tools.Server -tcp -tcpAllowOthers

We then modify the following line in /var/www/ajax-blog/blog.js, to make it connect to the external H2 server instead of using H2 in embedded mode:

var blog = new Blog("jdbc:h2:tcp://10.102.228.78/~/Blog;AUTO_RECONNECT=TRUE", "admin", "secret");

Re-running the benchmark reveals that the SQL database sits mostly idle. That's hardly surprising. After all, how much CPU can a few SELECTs from a 100-something row table use? Knowing this, I could have just started the H2 database as a process on the initial web node, but it's always easy to be wise after the event.

Anyway, now we know how to improve the performance. Let's deploy another web front-end, add a load balancer and rerun the benchmark. The result? Almost twice the performance, or 10 ms/request!


And all you need is a web browser, an SSH client and a credit card. The Internet is truly an amazing place.

PS. With four front-ends, we're no longer CPU limited and probably need to tweak the Apache configuration (it's forking like crazy) or ESXX's thread pool settings to obtain further performance improvements.

Friday, May 22, 2009

From localhost to live in 60 minutes using GoGrid

I made an experiment today. The question I wanted to answer was this:
Given a locally developed ESXX application, running on my laptop, how long would it take to go live, assuming you own no servers or Internet connection suitable for such deployment?

For this, I turned to my favourite grid/cloud service, GoGrid. GoGrid is pretty amazing. With just a few clicks in their admin UI, you can create all kinds of servers in their data center, and they will be available online within minutes. Pretty cool stuff, and a perfect match for ESXX, considering the "friendliness" factor!

The app I tested was the Ajax Blog tutorial, which is available as one of the examples in the ESXX distribution.

So I logged in to my GoGrid account and created a "Web/App Server" called "ajax-blog" using the 64-bit CentOS 5.1/Apache 2.2 image. About 15 minutes later, the server was online and I could log in as the root user. After a "yum update" and a reboot, the system was updated to CentOS 5.3 and ready to be configured according to my requirements.

All in all, setting up the "hardware" and the base OS took about two minutes of work and forty minutes of waiting.

First up: software installation. We need to add the ESXX RPM repository, install Java and (unfortunately) we also need to compile mod_fastcgi ourselves. No big deal.

[root@17914_1_20449_109821 ~]# cat > /etc/yum.repos.d/esxx.repo
[esxx]
name=ESXX
baseurl=http://esxx.org/repos/rpm/
enabled=1
gpgcheck=0

[root@17914_1_20449_109821 ~]# yum install esxx java-1.6.0-openjdk httpd-devel.x86_64
...
Complete!
[root@17914_1_20449_109821 ~]# service esxx start
Starting esxx: [ OK ]
[root@17914_1_20449_109821 ~]# wget http://www.fastcgi.com/dist/mod_fastcgi-2.4.6.tar.gz
...
[root@17914_1_20449_109821 ~]# tar xfz mod_fastcgi-2.4.6.tar.gz
[root@17914_1_20449_109821 ~]# cd mod_fastcgi-2.4.6
[root@17914_1_20449_109821 mod_fastcgi-2.4.6]# make -f Makefile.AP2 top_dir=/usr/lib64/httpd/ local-install
...
[root@17914_1_20449_109821 mod_fastcgi-2.4.6]#

Time to install our ESXX app. We'll create a copy of the Ajax Blog tutorial in /var/www/ajax-blog and tweak the settings a bit:

[root@17914_1_20449_109821 ~]# cp -r /usr/share/doc/esxx/examples/ajax-blog /var/www/
[root@17914_1_20449_109821 ~]# chgrp apache /var/www/ajax-blog/
[root@17914_1_20449_109821 ~]# chmod g+w /var/www/ajax-blog/

Edit /var/www/ajax-blog/blog.js and change the line that creates the 'blog' object to include the full path of the SQL database and a new admin password:
esxx.include("src/Blog.js");

XML.ignoreWhitespace = false;
XML.prettyPrinting = false;

var blog = new Blog("jdbc:h2:/var/www/ajax-blog/Blog", "admin", "secret");

Next, create the file /etc/httpd/conf.d/ajax-blog.conf as follows:
LoadModule fastcgi_module modules/mod_fastcgi.so

FastCGIExternalServer /usr/sbin/esxx -host localhost:7654 -pass-header Authorization

# Install handler for all '*.esxx' files
AddType text/x-esxx .esxx
Action text/x-esxx /cgi-bin-esxx
ScriptAlias /cgi-bin-esxx /usr/sbin/esxx

# The Ajax Blog
Alias /blog /var/www/ajax-blog/public
RedirectMatch ^/blog/?$ /blog/index.esxx

Restart httpd, and yeah, that's it. The app is now live and you should be able to visit it at http://YOUR-SERVER-IP/blog/. How's that for friendly deployment?

Thursday, May 21, 2009

ESXX advances into beta

Whoo, long time, no see ... It's been a while since the last blog post, but today, we celebrate the fact that ESXX is no longer considered alpha quality with an all-new look of esxx.org.

I've already blogged about some of the new features in this release, but I just want to mention that there are now two tutorials available in the Wiki. The first one is very basic, while the second one demonstrates how easy it is to create a fully functional, database-backed blog using ESXX, complete with an advanced AJAX administration UI.

It's definately worth a read!

Sunday, February 22, 2009

JavaScript web applications/browser services, the ESXX way

In ESXX, the difference between a web service (a program that produces XML or JSON intended for other programs) and a web application or browser service (a program that produces HTML documents intended to be viewed by a human using a web browser) is minimal. Basically, it's only the data format (XML/JSON vs HTML) that differs.

It's a good idea to remember this when you build a web application. The response from your request handlers should be raw XML objects that includes all information that is unique for this particular web page or form, but nothing more. Layout, common headers and footers, static sidebars … everything that is common to all pages should be added by the XSLT stylesheet and/or external CSS stylesheets.

(Similarly, if you use filters to plug in a JavaScript template engine instead of using ESXX' built-in XSLT engine, your handlers should return plain JavaScript objects, containing the information that is unique for the page in question.)

ESXX provides two tools that are useful for building web applications: stylesheet handlers and request filters. Both can be used when building web services as well. Especially filters are useful for both kind of services.

Stylesheet handlers

As mentioned before, stylesheet handlers can be triggered by both HTTP and SOAP handler responses. When such a handler returns XML, the registered stylesheet handlers are searched based on the response's content type and the part of the URI that follows the .esxx file.

The handler specifies an XSLT 2.0 stylesheet that will, just like the application itself, be compiled and cached in memory until it times out or the file is modified. The stylesheet will then be applied to the response on every request. The params property of the ESXX.Response object can be used to set stylesheet parameters (response.params.mode refers to the <xsl:param name="'mode'/"> XSLT parameter).

If you're used to thinking in Model/View/Controller terms, the matched stylesheet is a collection of views and the name of the root element in the data returned by the request handler (the controller) is the name of the view to apply.

XSLT 2.0 is much more advanced and useful than XSLT 1.0, which is currently implemented by the browsers, but should the need arise, it's possible to call any JavaScript function from an XPath context using javascript: URIs.

For instance, the following JavaScript function, defined by your web application
function MyClass() {}

function MyClass.protoype.getCurrentDate() {
let now = new Date();

return <currentDate>
<day>{now.getDate()}</day>
<month>{now.getMonth() + 1}</month>

<year>{now.getFullYear()}</year>
</currentDate>;
}

var myObject = new MyClass();
can be used by the XSLT stylesheet like this:
The current year is <xsl:value-of my="javascript:myObject" select="my:getCurrentDate()/year">.
Global functions can be called by leaving out the object from the URI:
Today is <xsl:value-of my="javascript:" select="my:Date()">.
It's also possible to call class or instance Java methods:
The current year is <xsl:value-of my="java:java.util.Date" select="my:getYear(my:new()) + 1900">.
Request filters

Request filters differ from request handlers in that more than one filter may be invoked for a single request. They are defined as follows:
<esxx xmlns="http://esxx.org/1.0/">
<handlers>

</handlers>

<filters>

<filter method={http-method}
uri={path-info}
handler={object-and-method} />
<filter … />

</filters>
</esxx>
For each request, a list of matching filters are built and the first filter in the list is invoked with two parameters: the request object and a function that calls the next filter in the list, or, if the current filter is the last, a function that calls the request handler.

Each filter is supposed invoke the next function and return its value. This way, the request handler will eventually be invoked and its response will propagate back to the client via the filter handlers. By simply not calling the next function, a filter can abort the request and return its own response.

(The next function optionally takes a single argument, which will be passed as the req parameter to the next filter or handler. If unspecified, the current request object is passed. This way, it's possible to completely replace the request object.)

Here are a few filter handler examples:
function noOpFilter(req, next) {
return next();
}

function forbiddenFilter(req, next) {
// Abort request by not calling next() if cookie not set
if (req.cookies.secret != "Open Sesame") {
return [ESXX.Response.FORBIDDEN, {}, "Access denied"];
}
else {
return next();
}
}

function postFilter(req, next) {
let res = next();

// Set XSLT params
res.params.mode="silver";

// Add HTTP response header
res.headers["Cache-Control"] = "max-age=1800, public";

return res;
}

Saturday, February 21, 2009

JavaScript request filters

It's never to late to change your mind, and I'm happy to let you know that I checked in support for request filters in the trunk yesterday.

Filters differ from handlers in that more than one filter may be executed for a given request. Before a request is serviced, a matching filter chain is built and all all matching filters are then executed in turn as part of the request handling.

Each filter may modify (or even replace) both the request and response object or simply abort the request by not calling the next filter in the filter chain.

In ESXX, a filter is called with two parameters: the request object and a function that calls the next filter in the chain. The filter is supposed to return a response that will be sent to the client (optionally passing through the XSLT engine). That means that a filter may:
  • Abort the request by not calling the next() function at all.
  • Tweak the request by modifying the request object before calling the next() function.
  • Tweak the response by modifying the result of the next() function.
Filters are great for password-protecting parts of a site, setting XSLT stylesheet parameters or modifying responses (like converting XML to JSON or processing views in JavaScript instead of using ESXX' built-in XSLT engine).

Do have a look at an example to see how ESXX filters work.

Sunday, February 15, 2009

JavaScript servlets

Today, I checked in support for running ESXX as a inside servlet a Java EE application server. Why, someone may ask, run an application server within another application server?

Well, for starters, it allows you to use ESXX with your existing infrastructure and tool set. If you're already using an app server such as Glassfish or Tomcat, you can add modules powered by ESXX by rebuilding the war archive. Simply add your JavaScript code files, change the http-root parameter in web.xml, rebuild and deploy the generated esxx.war archive on your app server.

More work is required, but if you're interested to try it, just check out trunk from the Subversion repository at http://svn.berlios.de/svnroot/repos/esxx/trunk/ and type ant jee-war to build your own esxx.war archive. By default, it serves the ESXX example applications.

Sunday, February 8, 2009

JavaScript web services, the ESXX way

By "web service", we mean a service that is intended to be used by a program, not directly by a human surfing the web with a web browser.

To use ESXX as a web service engine, you first need to define what request handlers should be available. ESXX currently provides six kinds of handlers: HTTP, SOAP, stylesheet, timer, exit and error handlers. Handlers are defined using a section of the main document within the http://esxx.org/1.0/ namespace:
<esxx xmlns="http://esxx.org/1.0/">
<handlers>
<http method={http-method} uri={path-info} handler={object-and-method} />
<soap action={soap-action} uri={path-info} object={object} />

<error handler={object-and-method} />
<stylesheet media-type={content-type} uri={path-info} href={xslt-file} type="text/xsl" />
<timer delay={delay-in-seconds} period={period-in-seconds} handler={object-and-method} />
<exit handler={object-and-method} />
</handlers>

</esxx>
HTTP handlers
HTTP handlers are JavaScript functions that are triggered by a normal HTTP request. What handler to call is defined by the request method and the part of the URI that follows the .esxx file, which defines the web application.
SOAP handlers
SOAP handlers are JavaScript objects that are triggered by SOAP request, which means a POST request with the SOAPAction HTTP header set (SOAP 1.2 is not yet supported but will be in the future). The object to act upon is defined by the SOAP action and the part of the URI that follows the .esxx file, which defines the web application. What method to call on the object is determined from the SOAP request's Body element.
Error handlers
Error handlers are JavaScript functions that will be called whenever a HTTP or SOAP handler throws an exception and gives the application a chance to recover or to produce a more good-looking error message than ESXX's default error message.
Stylesheet handlers
Stylesheet handlers are XSLT stylesheets that are triggered by XML response from the HTTP, SOAP and error handlers. What stylesheet to execute is defined by the content type and the part of the URI that follows the .esxx file, which defines the web application.
Timer handlers
Timer handlers are JavaScript functions that will be triggered once the application has been loaded and the periodically for the lifetime of the app.
Exit handlers
Exit handlers are JavaScript functions will be executed by ESXX when an application is unloaded. Applications are unloaded if they have been idle for a while, if any of the source code files they were compiled from change or as a direct command via the JMX interface.
Methods, SOAP actions, Media types and URIs
The attributes method, action, media-type and uri are all interpreted as Java (not JavaScript!) regular expressions, with one important addition: they allow named groups using the (?{name}...) syntax. More on that in the next section.

Multiple handlers of a given type are matched in the order they appear in the document. When an application is first loaded, all handler rules of a given type are merged and compiled into a single NFA, which is then used for each request to determine what handler to call.

(Normal Java regex recommendations apply: try to use non-greedy multipliers and use expression like [^a]*a instead of .*a wherever possible, in order to limit the amount of backtracking required.)

Here are a few examples:
...
<http method="GET|HEAD" handler="global.getHanlder" />

<http method="POST" uri="/upload" handler="myUploadHandler" />
<soap uri="/soap" object="soapObject" />

<soap action="urn:xmethods-delayed-quotes" uri="/" object="stocks" />
<stylesheet media-type="application/xml" uri="(?!soap$).*" href="anything-but-soap.xslt" />

...
Handlers, objects, hrefs and Request objects
The handler, object and href attribute specify JavaScript functions, JavaScript object and XSLT stylesheets, respectively. Named parameters from the attributes method, action, media-type and uri attributes may be inserted by using the {name} syntax. For instance, you the following HTTP handler invokes one of the following methods on object: handleDELETE() handleGET(), handleHEAD(), handlePUT():
<http method="(?{verb}GET|HEAD|PUT|DELETE)" handler="object.handle{verb}" />
Additionally, all parameters are available in the args property of the Request object that is passed to the matched JavaScript handler. Not only does this allow full JavaScript control of how to handle a request, it also means that it's easy to pass parameters in the URI:
<http method="GET" uri="/products/(?{category}\w+/(?{id}\d+)" handler="shop.getProductInfo" />
When the method shop.getProductInfo(req) is triggered by an HTTP GET request to .../app.esxx/products/games/10, req.args.category will refer to a word indicating the category ("games") and req.args.id will be a sequence of decimal digits indicating the item ID (10).

Regular expressions and named groups are very powerful, but can be error prone and hard to read. For large web applications with lots of defined handlers, XML entities can be used to create aliases for common URI parameters that are used in many handlers:
<?xml version="1.0" ?>

<!DOCTYPE esxx [
<!ENTITY user_id "(?{user_id}\d+)">
<!ENTITY email "(?{email}[a-zA-Z][\p{Alnum}_\-.+]*@([\p{Alnum}_\-]+\.)+[a-zA-Z]{2,6})">
]>

...

<esxx>
<handlers>
<http method="POST" uri="/users/&user_id;/sendmail/&email;" handler="sendMail" />

...
</handlers>
</esxx>
HTTP handlers
HTTP handlers handle plain HTTP requests based on the request method and the URI. The handler specifies a JavaScript function (like myHandler) or object and method (like object.myMethod), which will be called with an initialized object as single parameter.

As noted above, any groups in the regular expressions that caused the handler to be called are available in the args property of the ESXX.Request object. The contentType property will contain the basic content type (without parameters) and message property will contain the parsed request body.

The handler should either return an ESXX.Response object, an Array (whose elements will be used as constructor parameters to initialize a new ESXX.Response object), a number (which indicates the HTTP status code and an empty body) or any other supported object, which will be automatically wrapped in an ESXX.Response object with HTTP status 200 and no extra headers (the content-type will be guessed based on the object's type).
  • If the payload of the ESXX.Response object is an E4X or DOM node, a stylesheet handler will be looked up and if found, the XSLT stylesheet will be used to transform the node.
  • Otherwise, the ESXX.Response payload will simply be serialized and sent to the client.
  • If the handler throws an exception, the error handler, if defined, will be invoked.
The most common form of return values are XML nodes ("application/xml") or plain JavaScript objects ("application/json"), but it's also possible to return Java InputStreams or RenderedImage objects.

SOAP handlers
SOAP handlers handle SOAP 1.1 requests based on the SOAPAction HTTP header and the URI. The handler specifies a JavaScript object and the local-name of the first child of the Body element is used to determine what method to call on the object. The method will be called with an initialized ESXX.Request object as first parameter and the first child of the Body element (as an E4X node) as the second parameter.

As noted above, any groups in the regular expressions that caused the handler to be called are available in the args property of the ESXX.Request object.

The soapAction property will contain the value of the SOAPAction HTTP header and the message propery will contain a Java javax.xml.soap.SOAPMessage object.
  • The return value is handled just like for HTTP handlers and the payload should be an XML node. Unlike HTTP handler responses, however, if the XML node's local-name is not "Envelope", a SOAP envelope will automatically be added.
  • If the handler throws an exception, the error handler, if defined, will be invoked.
The implementation is very simple, but should be quite usable for basic SOAP requests. If a real SOAP service is required, you should use normal HTTP handlers and call the appropriate Java SOAP framework from there. (Or, you could just reconsider and use HTTP the way it was intended to be used.)

Error handlers
Whenever an HTTP or SOAP handler throws an exception, the error handler, if defined is invoked. The handler specifies a JavaScript function (like myHandler) or object and method (like object.myMethod), which will be called with the original ESXX.Request object as first parameter and the exception object as the second parameter.

The handler may return null or undefined to act as if there were no handler installed in the first place, or return a new response in the same way HTTP handlers return responses.

This handler should not throw exceptions.

Stylesheet handlers
Stylesheet handlers handle responses from HTTP or SOAP handlers based on the content type and the URI. The handler specifies an XSLT stylesheet that will be executed if the reponse was an XML node.

Please stay tuned for more information about the stylesheet handlers. I'll post more information about them later.

Timer handlers
Periodic or one-shot timers may be installed by specifying timer handlers. The delay and period is specified in seconds (decimal numbers are accepted). The handler specifies a JavaScript function (like myHandler) or object and method (like object.myMethod), which will be called with a single Date parameter, which indicates the time when the handler was scheduled to be executed.

A timer handler is never executed more than once at a given time. If a timer handler takes too long to complete, subsequent events will be discarded until the handler returns.

No return value is expected. This handler should not throw exceptions.

Exit handlers
When an ESXX application is about to be unloaded from memory, an exit handler may be called to perfrom final cleanup. The handler specifies a JavaScript function (like myHandler) or object and method (like object.myMethod), which will be called with no arguments.

No return value is expected. This handler should not throw exceptions.

Friday, February 6, 2009

F-Script vs. JavaScript


I read about F-Script on Ars Technica today and couldn't help wondering how the example might have looked like if written as an ESXX command line application. Something like this, perhaps?
#!/usr/bin/env esxx-js

with (JavaImporter(java.awt, javax.swing)) {
function main() {
// Create the window
let window = new JFrame("TimeSetter");
window.setLayout(null);
window.setPreferredSize(new Dimension(383, 175));
window.setLocation(125, 513);
window.defaultCloseOperation = WindowConstants.EXIT_ON_CLOSE;

// Add a button
let button = new JButton("Push Me");
button.setBounds(100, 50, 150, 50);
window.add(button);

// Set up the two scripts -- one beeps, the other updates the window title
function beepScript() { Toolkit.defaultToolkit.beep(); }
function titleScript() { window.title = new Date(); }

// When the button is clicked, do something
button.addActionListener(titleScript);
window.rootPane.defaultButton = button;

window.pack();
window.visible = true;
esxx.wait(this);
}
}

Thursday, February 5, 2009

Server-side XML processing, the ESXX way

I've continued to add more documentation to the wiki. Let's talk about server-side XML processing via XSL this time.

With all due respect to command line applications, ESXX was designed for web applications.

An ESXX web application is defined by an XML file and zero or more JavaScript handlers or XSLT stylesheets. The XML file may be any XML file, and it will be available as an E4X node in the esxx.document variable. There are a few special constructs that may be present in it:
  • Other XML documents may be included using XInclude.
  • An <?esxx-stylesheet?> processing instruction may be specified to instruct ESXX to transform the document using the specified XSLT 2.0 stylesheet.
  • One or more <?esxx-include?> processing instructions may be used to load JavaScript code into memory.
  • One or more <?esxx?> processing instructions can also be used to specify JavaScript code in-line.
  • Various handlers may be installed by including elements from the http://esxx.org/1.0/ namespace in the document.

The most common situation is simply an http://esxx.org/1.0/ describing the handlers, plus <?esxx-stylesheet?> and <?esxx?> to set up the JavaScript environment, but let's look at a few examples first.

Plain XML
ESXX can serve static XML files. The file will be parsed, stored in esxx.document and serialized on every request. Hardly useful, but possible.

<?xml version="1.0"?>
<xml>Hello from a totally useless ESXX application!</xml>

XSLT transformation of static XML documents
Perhaps a bit more realistic, ESXX can be instructed to transform an XML document on each request. For instance, assume you have a DocBook document that you'd like to serve as HTML on the web. Instead of converting the document by hand every time you modify it, let ESXX serve and transform it using the XSLT 2.0 stylesheet specified by an <?esxx-stylesheet?> processing instruction in the document.

ESXX will load and parse the DocBook document and the stylesheet, and the compiled stylesheet will be executed on every request. If either of the DocBook document or the XSLT stylesheet is modified, the application will automatically be reloaded, which actually makes this a nice way to preview DocBook documents you're working on.

Here's a simple (non-DocBook) example:

<?xml version="1.0"?>
<?esxx-stylesheet type="text/xsl" href="useful.xslt"?>
<doc>
<para>Hello from a somewhat useful ESXX application!</para>
</doc>
And the stylesheet:
<?xml version="1.0"?>
<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns="http://www.w3.org/1999/xhtml">
<xsl:output
doctype-public="-//W3C//DTD XHTML 1.1//EN"
doctype-system="http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"
method="xhtml" version="1.0" media-type="text/html" />
<xsl:template match="@*|node()">
<xsl:copy>
<xsl:apply-templates select="@*|node()"/>
</xsl:copy>
</xsl:template>

<xsl:template match="/doc">
<html>
<body>
<xsl:apply-templates/>
</body>

</html>
</xsl:template>

<xsl:template match="para">
<p>
<xsl:apply-templates/>

</p>
</xsl:template>
</xsl:stylesheet>
JavaScript and XSLT handlers

Stay tuned ...

Wednesday, February 4, 2009

JavaScript command line applications, the ESXX way

Today, I added some documentation to the wiki about ESXX command line applications, and I though I'd post it here as well.

An ESXX command line application is one or more JavaScript files that define a main() function in the global scope. It's common to also begin the main JavaScript with a shebang, and set the executable flag, so it can easily be executed from the Unix command line. /usr/bin/env is usually used as a helper method, so it won't matter where the esxx-js program is located, as long as it's in the command search path.

#!/usr/bin/env esxx-js

... code follows ...

(Note that if the esxx-binfmt service is started, any executable JavaScript file may be executed directly from the command line shell, even if it lacks a shebang. Unfortunately, this currently only works in Linux.)

The command line application is loaded and executed like this:

  1. First, a new global scope is set up, with all the Core JavaScript, Rhino, ESXX and LiveConnect host objects present.
  2. The JavaScript file is loaded, compiled and executed with this set to the global scope.
    1. During the execution, one or more functions are defined in the global scope.
    2. esxx.include() may be called to bring in other JavaScript files into the global scope, either from the current directory or from the <esxx-root>/share directory. These files may call esxx.include() in turn; however, a single file will only be loaded and executed once — attempts to load a file that has already been loaded will be silently ignored.
  3. As a final step, ESXX invokes the main() function, with the main script file name as first argument followed by the command line arguments.

Below is an example of a very simple ESXX command line application that loads data from an URI, using the URI's default parser, and prints the result to the console.
#!/usr/bin/env esxx-js

var err = java.lang.System.err;
var out = java.lang.System.out;

function main(prg, location) {
if (!location) {
err.println("Usage: " + prg + " <location URI>");
return 10;
}

var uri = new URI(location);
var data = uri.load();

out.println(data);
return 0;
}

The script can be used for many purposes. Assuming the script is called load.js, it may be used to load a web page and convert it into XML:
[martin@elsa ~]$ ./load.js http://example.com
<html>
<head>
<title>Example Web Page</title>
</head>
<body>

...
</body>
</html>

But it can also be used as an DNS resolver, thanks to ESXX's dns: URI protocol implementation:
[martin@elsa ~]$ ./load.js dns:/example.com
<result>
<entry uri="dns:/example.com??base">

<a>208.77.188.166</a>
<ns>b.iana-servers.net.</ns>
<ns>a.iana-servers.net.</ns>
</entry>

</result>

ESXX command line applications can also be GUI applications by using Java's Swing GUI toolkit. Here's a small app that displays today's XKCD strip in a window:
#!/usr/bin/env esxx-js

with (JavaImporter(javax.swing)) {
function main() {
let atom = new Namespace("http://www.w3.org/2005/Atom");

let entry = new URI("http://xkcd.com/atom.xml").load().atom::entry[0];
let img_tag = new XML(entry.atom::summary.toString());
let image = new URI(img_tag.@src).load();

let frame = new JFrame(entry.atom::title);

frame.defaultCloseOperation = WindowConstants.EXIT_ON_CLOSE;
frame.add(new JLabel(new ImageIcon(image)));
frame.pack();
frame.locationRelativeTo = null;
frame.visible = true;

esxx.wait(this); // Wait forever
}
}

Exciting, isn't it?

Monday, February 2, 2009

What does "alpha" mean?

You might wonder what the fact that ESXX is still in "alpha" stage means. Is it a constant crashing piece of crap?

Far from it! ESXX has been used at a company (which shall remain unnamed for now) for almost six months, serving tens of thousands of user without problems.

No, "alpha" simply means that the API's are not yet completely stable and might change slightly. For example, in the last release, the Logger, Request and Response classes were moved inside the ESXX namespace in order to pollute the global namespace even less than before. Some existing code were affected, but not very much since you normally don't instansiate objects of any of these classes manually (with the exception of Response, perhaps).

With the next major release, I hope to finally freeze the API and move on to a beta period.

Sunday, February 1, 2009

Final alpha release?

It's been quite a while, but a new release is finally available for download. This time, there are proper RPM and DEB packages, as well as Opensolaris IPS and Mac OSX PackageMaker packeges. And a Windows installer, as usual of course.

Some of the highlights of the release include:
  • Proper RPM (Fedora/RHEL), DEB (Debian/Ubuntu), PackageMaker (OSX) and IPS (Opensolaris) packages.
  • dns, https URI handler.
  • multipart/form-data parser.
  • Better SOAP service support.
  • Improved stylesheet matching rules.
  • Timers.
  • Rhino 1.7 release 2 RC3, H2 Database 1.1.107.
  • Partial ECMAscript 3.1 compatibility provided by the "esxx/harmony.js" include file.
  • New JS class, LRUCache, which is used for thread-local and program-local data.
  • Improved JMX interface.
  • And lots more ...
If all goes well, this will be the final alpha release, meaning there will hopefully not be any more API changes that are not backwards-compatible.