Joost

This page contains information about the world's first prototypic Java implementation of the Streaming Transformations for XML (STX) language. It is intended as a reference implementation which demonstrates the functionality of STX.

Check out StDB, a debugger for STX based on Joost

The latest version was created 2005-05-21 and may be downloaded here. Note: though the code runs hopefully without major crashes, it should still be regarded as an alpha version.

Joost is released under the Mozilla Public License 1.1. Have a look at the project page for further information. Any questions concerning Joost may be asked on the joost-help mailing list.

In order to run Joost you need additionally

a SAX2 Parser, for example Crimson (already included in Java 1.4 or downloadable separately on Sun's Java XML Pack page), Xerces from Apache XML, or Ælfred2 from the GNU JAXP Project
optionally (see below) the Logging Component from Apache Jakarta Commons
and of course a Java platform, at least version 1.4

Run Joost with
java net.sf.joost.Main [opts] xml-source stx-sheet [pars] {stx-sheet [pars]}
or simply
java -jar joost.jar [opts] xml-source stx-sheet [pars] {stx-sheet [pars]}

The dash character "-" for the xml-source parameter denotes the standard input. The specification of multiple stx-sheets creates a transformation chain. Global parameters for each STX transformation sheet may be specified by adding command line arguments (pars) of the form parameter=value (on Windows enclosed in additional quotes: "parameter=value") directly after each stx-sheet. value is interpreted as a string, not as an expression.

The list of possible options includes

	`-help`		print a help message
	`-version`		print the version information and exit
	`-o <filename>`		write the result to the file <filename>
	`-m <classname>`		use a <classname> object for `stx:message` output (unless the `stx:message` element has a `logger` attribute and logging is available, see below) this class must implement the interface `net.sf.joost.emitter.StxEmitter`
	`-nodecl`		don't include an XML declaration in the result
	`-time`		print timing information on standard error output
	`-pdf`		pass the result to FOP for PDF generation (requires -o), see below

If Commons Logging is available, Joost will log possible runtime errors and messages produced with stx:message/@logger using the appropriate logger objects. You can check whether logging is available or not by running Joost with the -help or -version command line options.

Joost is able to pass its output to FOP for producing PDF output from a stream of XSLFO SAX events. To use this function you have to add fop.jar, batik.jar and avalon.jar (FOP 0.20.5 comes with avalon-framework-cvs-20020806.jar) to your classpath. It is not possible to change FOP's verbosity on the command line at the moment (Joost sets it to warn).

Joost implements the TrAX interfaces from JAXP 1.1. In order to create a Joost transformer you have to set the Java property javax.xml.transform.TransformerFactory to the value net.sf.joost.trax.TransformerFactoryImpl.

^*) Joost [pronounced yohst] meant once Joost is Oli's Original Streaming Transformer and was inspired by the given name a good fiend's little son.

Features

Supported stx elements (and their attributes) in the namespace http://stx.sourceforge.net/2002/ns

stx:transform (version, stxpath-default-namespace?, output-method?, output-encoding?, pass-through?, strip-space?, recognize-cdata?, exclude-result-prefixes?)
stx:group (name?, pass-through?, strip-space?, recognize-cdata?)
stx:include (href)
stx:namespace-alias (sheet-prefix, result-prefix) – (uses the sheet-prefix in the result)
stx:template (match, priority?, public?, visibility?, new-scope?)
stx:procedure (name, public?, visibility?, new-scope?)
stx:call-procedure (name, group?)
stx:copy (attributes?)
stx:process-children (group?, filter-method?, filter-src?)
stx:process-self (group?, filter-method?, filter-src?)
stx:process-siblings (group?, while?, until?)
stx:process-attributes (group?)
stx:process-document (href, base?, group?, filter-method?, filter-src?)
stx:result-document (href, output-method?, output-encoding?)
stx:buffer (name)
stx:result-buffer (name, clear?)
stx:process-buffer (name, group?, filter-method?, filter-src?)
stx:value-of (select, separator?)
stx:if (test)
stx:else
stx:choose
stx:when (test)
stx:otherwise
stx:for-each-item (name, select)
stx:while (test)
stx:text
stx:cdata
stx:element (name, namespace?)
stx:start-element ({name}, {namespace}?)
stx:end-element ({name}, {namespace}?)
stx:attribute ({name}, {namespace}?, select)
stx:comment (select?)
stx:processing-instruction ({name}, select?)
stx:variable (name, select?, keep-value?)
stx:assign (name, select)
stx:param (name, select?, required?)
stx:with-param (name, select?)
stx:message (select?, {terminate}?, logger?, level?) – (prints its content to standard error output or uses the specified logger with the given level)
The terminate attribute behaves as in XSLT.
The logger and level attributes can be used to produce log messages that will be send to a special Log object. The name of this object must be specified in the logger attribute. The level attribute must be one of the values trace, debug, info, warn, error, or fatal. Joost will use the Commons Logging framework if it can be found on the classpath. Otherwise the attributes logger and level will simply be ignored, i.e. the stx:message instruction behaves as if these attributes weren't present. Since Commons Logging is just a wrapper for other logging facilities, Joost doesn't provide any mechanisms to configure the underlying logger.

Supported STXPath functions

string(object?)
number(object?)
boolean(object)
position()
name(node?)
local-name(node?)
namespace-uri(node?)
get-namespace-uri-for-prefix(string, node)
get-in-scope-prefixes(node)
node-kind(node)
not(boolean)
true()
false()
floor(number)
ceiling(number)
round(number)
concat(string, string+)
string-join(sequence, string)
string-length(string?)
normalize-space(string?)
contains(string, string)
starts-with(string, string)
ends-with(string, string)
substring(string, string, string?)
substring-before(string, string)
substring-after(string, string)
translate(string, string, string)
string-pad(string, number)
escape-uri(string, boolean)
has-child-nodes()
empty(sequence)
exists(sequence)
item-at(sequence, number)
index-of(sequence, object)
subsequence(sequence, number, number?)
insert-before(sequence, number, sequence)
remove(sequence, number)
count(sequence)
sum(sequence)
min(sequence)
max(sequence)
avg(sequence)
filter-available(string)

Built-in extension functions
A built-in extension function must be used with a fully-qualified name and the prefix bound to the namespace http://joost.sf.net/extension

sequence(obj)
converts a Java array or a java.util.List object to a sequence, see Extensions below

Limitations

no full Unicode support (surrogate pairs not implemented, missing check for well-formedness of created XML names)
the STXPath implemented in this release obeys still the productions of the grammar in the former STX draft (5 May 2003)
all STX elements and functions not explicitely mentioned on this page wait for an implementation
...

Extensions

HTML output

Joost provides an experimental HTML 4.01 output method that can be invoked by specifying joost:html in the output-method attribute. The prefix joost must be bound to Joost's extension namespace http://joost.sf.net/extension. CDATA boundaries will not be printed in HTML. This means the beloved disable-output-escaping feature of XSLT can be simulated by using the stx:cdata instruction.

Invoking Java methods

Joost allows the use of extension functions written in Java. An extension function is invoked using the notation prefix:localname(params?). The prefix must be bound to a namespace indicating the Java class to which the function belongs. This namespace must use the URN notation "java:" followed by the full-qualified class name, for example "java:java.util.Random". The referenced class must be found in the current classpath.

The rules for invoking Java extension functions are similar to those used by Saxon or Xalan:

A static method can be invoked directly. The local name of the function must match the name of a public static method in this class. The names match if they contain the same characters, excluding hyphens and forcing any character that follows a hyphen to upper-case. For example the STXPath function call prefix:to-string() matches the Java method toString(); but if you prefer, the function call can also be written as prefix:toString().

The number of the arguments supplied to the function call must match exactly the number of the parameters of the Java method. If there are more than one candidate, Joost attempts to find the one that is the best fit to the types of the supplied arguments (see Conversion rules below). Joost will report an error if no matching method at all can be found, or if several methods may be used but none is preferable to the others.

Example:
<stx:value-of select="math:sqrt($arg)" xmlns:math="java:java.lang.Math"/>
A constructor is invoked by using a function named prefix:new. If there are several constructors, then again Joost tries to find the one that is the best fit, according to the types of the supplied arguments. The result of invoking new() is an STXPath value of type Java Object; the only things that can be done with a Java Object are to assign it to a variable, to pass it to an extension function, and to convert it to a string, number, or boolean, using the rules given below.
An instance-level method is invoked by supplying an extra first argument of type Java Object which is the object on which the method is to be invoked. Matching of method names is done as for static methods.

Example:
<stx:value-of select="date:to-string(date:new())" xmlns:date="java:java.util.Date" />

It is recommended not to invoke an extension function having side-effects within the predicates of a match pattern, because the order of these function calls and whether such a function will be used at all is undefined (implementation dependent).
(For example in match="a[ex:foo()]//b[ex:bar()]/c[ex:baz()]")

Conversion rules
Depending on the type of the argument values supplied to the function call, Joost tries to find a method that has parameters of the following types (Java types with a higher conversion preference appear first in the list; if one parameter is of a primitve type and the argument in the function call is a non-empty sequence, only the first item will be considered):

supplied STXPath type	required Java type
empty sequence	`java.util.List`, `java.lang.Object` (or a derived type, i.e. a reference type)
`boolean`	`boolean`, `byte`, `short`, `int`, `long`, `char`, `java.lang.String`, `float`, `double`, `java.util.List`, `java.lang.Object`
`number`	`double`, `float`, `long`, `int`, `short`, `byte`, `java.lang.String`, `char`, `boolean`, `java.util.List`, `java.lang.Object`
`node`, `string`	`java.lang.String`, `char`, `double`, `float`, `int`, `long`, `short`, `byte`, `boolean`, `java.util.List`, `java.lang.Object`
wrapped Java Object	same type, base type, `java.lang.Object`, `java.util.List`, `java.lang.String`

If the target type is java.lang.Object then a conversion according to the following table will be performed. If the target type is java.util.List then a java.util.ArrayList will be constructed containing a list of java.lang.Objects (see table) for each of the items. An empty sequence will be represented as an empty list (not as a list with a null member).

supplied STXPath type	target type `java.lang.Object`
empty sequence	`null`
`boolean`	`java.lang.Boolean` containing the boolean value
`number`	`java.lang.Double` containing the number
`string`	`java.lang.String`
`node`	`java.lang.String` containing the node value
wrapped Java Object	the object itself

The result of a function call will be converted as follows:

return type	STXPath type / value
`void`	empty sequence
`java.lang.String`, `char`	`string`
`boolean`	`boolean`
`double`, `float`, `byte`, `short`, `int`, `long`	`number`
anything else, i.e. any reference type, including the value `null`	wrapped Java Object

A wrapped Java Object may be converted to another data type as follows.

required type	conversion rule
`node`	not possible
`string`	`null` will be converted to the empty string, any other object will be converted by invoking its `toString()` method
`boolean`	`null` will be converted to `false`, any other object will be converted via a `string` conversion and then applying the STXPath function `boolean()`
`number`	via a `string` conversion and then applying the STXPath function `number()`

A wrapped Java Object of type java.lang.Object[] or java.util.List may be converted to a sequence using the Joost extension function sequence. If the supplied parameter is neither an array nor a java.util.List object, this function returns its parameter.

Using external SAX2 filters

Joost implements the usage of external SAX2 filters as described in the WD of the STX specification section 4.19 (CVS version Jul 23 2003). Joost uses the TrAX API to instantiate an XSLT transformer object. The actual XSLT implementation (i.e. the javax.xml.transform.TransformerFactory) to be used may be controlled by setting the property net.sf.joost.trax.TrAXConstants.KEY_XSLT_FACTORY, either as a system property or using TransformerFactory.setAttribute.

Besides the XSLT (http://www.w3.org/1999/XSL/Transform) and STX (http://stx.sourceforge.net/2002/ns) filter methods, Joost provides built-in implementations for the following two additional values of the filter-method attribute:

http://xml.org/sax: Provides a SAX parser. All character data will be fed into the parser, all other events will be ignored. The events generated by the parser constitute the result of this filter.
http://www.ietf.org/rfc/rfc2616.txt#POST: Sends the filter input data per HTTP-POST request to the URL specified in the required target parameter (i.e. <stx:param name="target" ... >). The data of the HTTP response are the result of this filter. The HTTP-POST filter can be used to access web services from STX.

A custom filter implementation must support the javax.xml.transform.sax.TransformerHandler interface. Such a filter can be invoked by implementing a net.sf.joost.TransformerHandlerResolver object which returns the filter object for a requested filter URI. This resolver must be registered using the property net.sf.joost.trax.TrAXConstants.KEY_TH_RESOLVER in the TransformerFactory.setAttribute method, or by invoking setTransformerHandlerResolver on a net.sf.joost.stx.Processor object. There is an example implementation of a filter and an appropriate resolver in the src/samples directory (shipped with the source release only).

Examples

The current release contains an example directory with a set of selected STX examples. In addition you may browse Joost's CVS test directory as well as the examples directory from the STX CVS tree.

Resources

This release implements (partly) the STX Working Draft, 1 July 2004
The functions taken from the W3C spec are defined in XQuery 1.0 and XPath 2.0 Functions and Operators, W3C Working Draft 12 November 2003

An Introduction to Streaming Transformations for XML, Article on xml.com, 26 February 2003
Transforming XML on the Fly, Presentation at XML Europe 2003, London, 5-8 May 2003
Extended SAX Filter Processing with STX, Presentation at Extreme Markup Languages, Montréal, 4-8 August 2003

Contributors

Joost is supported by the YourKit Java Profiler

Have fun and tell me what you think!

ob / 2005-05-21