» Publishers, Monetize your RSS feeds with FeedShow: More infos (Show/Hide Ads)
Introduction
For many years, we have been using Java Threads for developing numerous Client-Server based applications. While it's always desirable to have a highly responsive Client application, handling a high volume of client requests has always been a prime goal of any Server application. However, designing a highly-scalable server requires lots of analysis. Without careful design and adherence to a well thought out underlying policy, a thread based system can fail to produce the desired outcome.
In this article, we'll discuss certain design principles that should be followed when the objective is to build an efficient, thread-safe application.
Thread Safety lies in Domain
As a design principle of thread safety, the vital point to look into is "Domain First." After all, the whole structure of an application lies on the Domain itself. If you get your domain model right, most of the problems are automatically taken care of. A model represents certain aspects of business logic consisting of various constraints and invariants. Understanding the pre and post conditions of a model are much needed in order to make the realized application thread-safe.
Let's imagine an travel agency that arranges budgeted cars for their traveler guests. They need an interactive car rental application that allows them to book multiple cars at different tourist locations on different dates within a specified price range. As they keep making multiple rental requests, at the same time if the system fails to find a car at particular location within price range, they want to be notified with suggested locations on the same screen.
We can think of this as a booking Requester taking the renal requests and putting them in a thread safe collection CarBookings that would be read
concurrently by an actual CarFinder process to provide suggested locations. If CarFinder succeeds, it forwards this via the booking engine to the booking rental location. If CarFinder does not succeed, it tries to find the cars based on the price range and nearest location. Once found, it adds the info to another thread safe collection
CarBookingAdvices.
There are many thread safe data structures available, but if you're sure about the Thread Safety policy that you want to adhere to, it's best to create a domain model on your own. For example:
class CarBookings{
private final Set bookings = new HashSet();
public synchronized void putCar(CarBooking booking){
bookings.add(booking);
}
public synchronized Set getBookings(){
return Collections.unmodifiableSet(bookings);
}
}
Since the methods putCar() and getBookings() provide a secure way of accessing the non-thread-safe instance variable bookings, the CarBookings can be used
safely by both the BookingRequester and CarFinder process running in different threads. This meets our simple thread safety requirement.
//Defining BookingAdvice
class BookingAdvice{
private final String bookingId="";
private final BigDecimal rate=new BigDecimal("");
private final String location="";
//Getter and Setter ommited
}
class BookingAdvices{
private final Set<BookingAdvice> advices = new HashSet<BookingAdvice>();
public synchronized void putAdvice(BookingAdvice advice){
advices.add(advice);
}
public synchronized Set<BookingAdvice> getAdvices(){
return Collections.unmodifiableSet(advices);
}
}
//Defining Requester
class BookingRequester{
private CarBookings bookings;
private BookingAdvices advices;
public void requestBooking(CarBooking booking){
bookings.putCar(booking); //Line 1
displayAdvice(advices.getAdvices()); //Line 2
}
private void displayAdvices(Set<BookingAdvice> advices){
//Displays the advices on the same screen
}
}
//Defining Finder
class CarFinder{
private BookingAdvices advices;
private CarBookings bookings;
public void processAdvice(){
List<BookingAdvice> adviceList = getGeneratedAdvices();
for(BookingAdvice advice: adviceList)
advices.putAdvice(advice); //Line 3
findAndProcessBookings(bookings.getBookings()); //Line 4
}
private List<BookingAdvice> getGeneratedAdvices(){
//Generate advice based on requests
}
private void findAndProcessBookings(Set<CarBooking> bookings){
//Finds advice for the booking if required
}
}
Hiding Model State
But, there is a caveat: if the CarBooking class is mutable, then there is a possibility of modifying each CarBooking instance in the set returned by getBookings(). Does it make our model brittle? Well, as long as we can make sure that it won't be modified later on, we'll be fine with that. But, sometimes it's easier get thread safety by hiding the model state and blocking the access to it thereafter. Here, we make CarBooking immutable and add a copy constructor:
class CarBooking{
private final String bookingId;
private final String location;
private final BigDecimal rate;
//Copy constructor
public CarBooking(CarBooking booking){
this.bookingId= booking.getBookingId();
this.location= booking.getLocation();
this.rate= booking.getRate();
}
//public Getter methods below
}
class CarBookings{
//Modified version
public synchronized Set getBookings(){
Set newBookings = new HashSet();
for(CarBooking b : bookings){
CarBooking newBooking = new CarBooking(b);
newBookings.add(newBooking);
}
return Collections.unmodifiableSet(newBookings);
}
}
So, when getBookings() returns, it returns a copy of the CarBooking, thus not allowing modification of the original state--which makes it purely thread safe.
Do Thread Delegation if you can
Sometimes you might find yourself being overburdened with handling the thread safety in your domain model; in that case you can simply delegate it to a range
of Java built-in data structures that are already proven and tested to be thread safe. Like in our example, if we had to use bookings as HashMap<String, CarBooking>, where String could represent bookingId, we could have easily replaced that with ConcurrentHashMap--which is a thread safe HashMap. Likewise there are many other data structures available (under the java.util.concurrent package) like CopyOnWriteArrayList and ConcurrentLinkedQueue, to name a few.
Figuring Single Thread Confinement
To coordinate multiple threads in a lock-based application, the OS and JVM have to burn fair amount of CPU cycles for context switching and thread scheduling--making the system less responsive sometimes.
This can be avoided by allowing only one thread to gain access to work on a unit model (either CarBooking or BookingAdvice) at a time, and also making sure that same unit model should not be re-worked by the same thread. To make this happen, we can employ bounded BlockingQueues (BookingQ and AdviceQ) that would act
as a mediator between Requester and CarFinder.
So, for a rental request, the Requester would just create a CarBooking and put it in the BookingQ queue, and on the other hand, CarFinder would take that
off of the queue. The same thing will be done by the CarFinder to return a BookingAdvice in the AdviceQ queue. This way, multiple Requesters and CarFinders can work
asynchronously, making the system highly responsive.
Use synchronized lock wisely
While synchronized lock provides an easy way to work with multiple threads, improper usage of the lock may cause unpredictable results. This becomes obvious when multiple locks are acquired to perform an atomic operation where the first lock is held until last lock operation is performed. Let's consider our BookingRequester and CarFinder that are using simple java Sets instead of CarBookings and BookingAdvices. Here two operations: putting request and getting
advices for Requester (vice versa for CarFinder, lines 5,6,7 and 8) become implicitly atomic under the same lock:
class BookingRequester{
private final Set bookings = new HashSet();
private final CarFinder finder;
public BookingRequester(CarFinder cf){
finder = cf;
}
public synchronized void requestBooking(CarBooking booking){
bookings.add(booking); //Line 5
displayAdvice(finder.getAdvices()); //Line 6
}
public synchronized Set getBookings(){
return bookings;
}
}
class CarFinder{
private final Set advices = new HashSet();
private BookingRequester bookings;
public CarFinder(BookingRequester br){
bookings = br;
}
public synchronized void processAdvice(){
List adviceList = getGeneratedAdvices();
for(Advice advice: adviceList
advices.add(advice); //Line 7
findAndProcessBookings(bookings.getBookings()); //Line 8
}
public synchronized Set getAdvices(){
return advices;
}
}
Now, a problem starts to creep in when two threads T1 and T2 try to call Requester requestBooking() and CarFinder processAdvice() respectively at the
same time. During requestBooking, before T1 calls getAdvices() on CarFinder (Line 6), it has to wait for T2 to complete processAdvice. But to complete processAdvice, T2 needs to call getBookings() on Requester (Line 8), which won't happen until T1 completes requestBooking()--leading to a deadlock situation.
In our previous implementation of BookingRequester and CarFinder, we have already delegated the two operations to be handled independently under different locks (one for CarBookings and another for BookingAdvices, lines 1, 2, 3 and 4) to avoid any deadlock.
But, here we can simply resolve it by moving synchronized from the entire method to only the time required to put the booking (Line 5) and advice (line 7).
Another important point regarding deadlocks would be the order in which the resources are accessed in an atomic operation. If two threads work on an operation with two resource locks being held in a different order, it can lead to a deadlock situation too. So, make sure all threads calling an atomic operation get all resource locks in the same order.
Conclusion
Designing and improvising a thread based application is a challenge. But by following certain design principles and guidance, this can be easily overcome. At the same time, clear understanding of thread safety policy is also essential as it helps you simplify the design. There are many other techniques available that we couldn't cover here. However, the principles presented here will always assist you in overcoming some of the thread safety related hurdles you might be facing as you develop thread-safe applications intended for operation on modern multicore and multiprocessor computers.
Part of the Java Standard Edition since the release of the Java Development Kit 1.1, the Java Database Connectivity (JDBC) API has become the industry standard for providing standards-based data access from Java. It allows you to have a single API into a database connectivity driver--whether it be Oracle, SQL server, or DB2--and to write your application code in such a way that you needn't concern yourself with the underlying data source.
Evolution of JDBC drivers
The JDBC API specification and the drivers it enables have evolved over time, from the original (Type1) drivers--JDBC-ODBC bridges that are dependent upon ODBC drivers and native database libraries on the client side and that are comparatively lacking in features--to native-protocol (Type 4) drivers, also known as Direct to Database Pure Java Drivers, which are entirely written in Java, are platform-independent, and are run inside the client's Java Virtual Machine (JVM) requiring no associative software (such as libraries) to work. Some type 4 drivers have also come to offer numerous features--single sign-on, Kerberos security, and NTLM authentication, for example--largely addressing the need to securely integrate JDBC drivers with complex application servers and database features targeting the enterprise market.
As much as JDBC driver architecture has evolved, however, evolution within the enterprise Java ecosystem has left it with some troublesome shortcomings. Ironically, one trend that has helped to make Java-based, data-driven applications so pervasive in global IT organizations has also taken JDBC off the radar for many developers, who remain unaware of these shortcomings and/or how to address them. That trend is the movement away from a data access model where Java developers program directly to JDBC to a model where they instead use a framework-based object-relational mapping (ORM) technology--such as JPA, Hibernate, or Spring--or an application server such as JBoss that sits on top of JDBC. With these newer, more accessible data models that permit no access to underlying JDBC calls, developers almost never think about JDBC or what JDBC driver to use as part of determining a data access strategy.
However, the JDBC drivers used in contemporary enterprise application design, deployment, and runtime scenarios are well worth thinking about. As things have stood, evaluating a JDBC driver has meant simply selecting a JDBC driver with the best architecture--that is, a Type 4 JDBC driver. Until very recently, however, JDBC architecture types have remained constant in the face of rapidly changing IT conditions within the enterprise. Aside from the widespread adoption of ORM-based data access models discussed above, some of the more sweeping and significant developments include virtualization, advancements and new features in database technologies, and rapid adoption of business intelligence and data warehousing initiatives:
- The proliferation of virtualizing both hardware and software resources has made it possible for IT organizations to deliver massive scalability on an affordable growth curve, placing a much higher value than ever before on efficiency and optimum performance throughout the entire application stack.
- The increasingly more complex features and functionality of relational database offerings, while in high customer demand, frequently involve complicated and proprietary implementations that have made them all but inaccessible to most applications.
- And enterprise organizations standardizing on a single RDBMS platform as their data warehouse must move vast amounts of data from diverse systems and are discovering that time-consuming data loads using batch mechanisms are impeding the production of timely business intelligence reports.
Limitations of Type 4 JDBC drivers
While superior to other JDBC driver architecture types, most Type 4 drivers come with glaring limitations that make them impractical for today's Java-based enterprise application environments. Most, for instance, require changes in an application's JDBC code in order to be tuned for performance. Doing this for each unique application deployment scenario is unmanageable and impractical. When you throw an ORM on top, if you must have a vendor-specific statement method, you'll be unable to do that casting without modifying the code of the ORM. So unless you want to be modifying, let's say, a Hibernate implementation, you need to make sure that those JDBC drivers are clean--that is, that they adhere to the standard while yet executing things in a flexible manner.
Suppose you've been tasked with tracking down the memory usage of an application you've written with Hibernate. You've traced and tuned, and think you should be able to squeeze more out of your Oracle database drivers. For this specific example, we'll say that we need to tune parameter bindings.
In order to more accurately control the amount of memory the driver allocates for each parameter in the PreparedStatement using Oracle's Thin driver, you must use the OraclePremparedStatement.defineColumnType() method (which is not part of the JDBC specification).
Listing 1.
PreparedStatement pstmt = con.prepareStatement(
"insert into perftest (id, code, descr, insert_user, insert_date)
values (?,?,?,?,?)");
((OraclePreparedStatement)pstmt).defineColumnType(1, Types.INTEGER);
((OraclePreparedStatement)pstmt).defineColumnType(2, Types.VARCHAR, 50);
((OraclePreparedStatement)pstmt).defineColumnType(3, Types.VARCHAR, 100);
((OraclePreparedStatement)pstmt).defineColumnType(4, Types.VARCHAR, 100);
((OraclePreparedStatement)pstmt).defineColumnType(5, Types.TIMESTAMP);
pstmt.setInt(1, count);
pstmt.setString(2, Integer.toString(count));
pstmt.setString(3, "123456789012345678901234567890");
pstmt.setString(4, "TONGUC");
pstmt.setDate(5, new java.sql.Date(System.currentTimeMillis()));
pstmt.execute();
However, if on a subsequent execute you should re-bind your parameters using larger data sizes, then the driver will be silently truncated to the size specified--not a good thing. Also, due to the fact that Hibernate abstracts the actual JDBC calls themselves, you lose the power to customize these calls unless you modify the Hibernate code itself, thus casting the PreparedStatement to an OraclePreparedStatement in the process. This is generally unacceptable, as it is costly to modify the Hibernate code and you must duplicate the code changes every time you upgrade the version of Hibernate you're using.
A much less costly option would be to tune memory at the JDBC driver level, where connect options could be specified to set the initial size to be tried for each parameter. This simple solution would save you from having to change Hibernate code in order to tune the application, and could also--by having the driver auto-adjust for the parameter's size--remove the limitation of truncating the data on subsequent executes and binds, making it unnecessary for you to spend time analyzing every parameter binding.
One of the promises on which the typical Type 4 driver architecture has failed to deliver is simple and uncomplicated deployment. It is anything but. Multiple JAR files are required to support deployment across different JVMs or hardware, as well as to access all supported versions of a particular database. This limitation can be overcome by packaging the JDBC driver as a single JAR file regardless of the IT environment, having no dependencies on external DLLs or shared libraries or native database components outside the JVM.
Listing 2. Code sample illustrating all-in-one (clean) implementation.
con = DriverManager.getConnection("jdbc:datadirect:oracle://nc-pgm1;
databaseName=orcl; user=scott; password=tiger;
initialColumnBufferSize=128");
PreparedStatement pstmt = con.prepareStatement(
"insert into perftest (id, code, descr, insert_user, insert_date)
values (?,?,?,?,?)");
pstmt.setInt(1, count);
pstmt.setString(2, Integer.toString(count));
pstmt.setString(3, "123456789012345678901234567890");
pstmt.setString(4, "TONGUC");
pstmt.setDate(5, new java.sql.Date(System.currentTimeMillis()));
pstmt.execute();
The support of typical Type 4 drivers for mission-critical database features targeting the enterprise is nearly always limited to a bare minimum of functionality. Many such features require proprietary code and the use of external DLLs or shared libraries--security features, bulk data loading, high availability, and XA features ordinarily do, for instance. With each data source that an application must support, the amount of data-source-specific code that must be maintained increases. In most cases, the driver simply exposes whatever support for these features is implemented by the database layer. Consequences of this strategy include inefficiency due to performance overhead, use of proprietary code, and requirements to license expensive database clustering technologies. These can be addressed with 100 percent JDBC-compliant driver-layer-only implementations.
Bulk load and JDBC drivers
Bulk load, a type of feature that addresses the limitations of batch data loads mentioned earlier, deserves a closer look in this context--not least because it has real potential to be a truly game-changing concept for organizations implementing or planning to implement data warehousing for reporting, decision support, and data mining purposes--in other words, just about any enterprise-scale business organization today.
Traditionally, moving very large amounts of data required the use of something called a bulk-moving tool or bulk-loading tool. An example would be a SQL loader, or a Sybase SQL Server BCP, or the DB2 Load command. Available for decades, these are tools that people use and build within their infrastructure on the back end. The demand for data in the enterprise makes such tools necessary; however, in light of the trend to standardize data on a single warehouse platform, the challenge becomes whether or not these tools can be used in a standard way. You've got SQL loader and BCP and DB2 Load, and they're all different. To this point, the only way to do this in any API has been through a batch mechanism such as JDBC batches. But batch methods are prohibitively slow for most data warehousing initiatives.
With JDBC driver technology that communicates directly in the native "wire" protocols of target databases, bulk load technology that's long been locked in C code can be applied to Java for bulk loading even non-relational data from a mainframe platform into a relational data warehouse. Say you have a key application in your organization that stores its data in a VSAM file on the mainframe. But all your BI dashboards and reporting analysis run against Oracle. How do you unlock that crucial data and make it available to your analysis staff so they can operate on the data using their accustomed tools?
You might deploy an event-driven scenario to trigger a bulk load into Oracle from the VSAM file whenever a given set of conditions is met. It would perform a single Select statement from the VSAM file, pass the ResultSet to the load method on a bulk load object for the Oracle database, and insert the data. If you imagine that it would take an awful lot of coding to accomplish this, you'd be mistaken. This graphic gives a real-life example of the code used to implement one such scenario.

Figure 1. A look at the code
In this example, a mere five lines of code were used to pull data out of VSAM and bulk load it into an Oracle database!
Conclusion
JDBC Type 4 drivers offer the best architecture and are adequate for many data-driven Java-based applications and scenarios. Nevertheless, the numerous solutions required to address serious shortcomings within demanding, complex, and sophisticated enterprise environments warrant looking beyond what a bare-bones Type 4 architecture typically offers--that is, client-side, single-tier, 100 percent Java architecture. A JDBC driver thus designed but that also embraces a comprehensive suite of such solutions might even, perhaps, be better thought of as a "Type 5" JDBC driver. To sum up what a "Type 5" driver should offer:
- Unrestricted performance: Data throughput is maximized regardless of the runtime environment or data access model.
- Codeless enhancement: Features and functionality can be added, configured, or tuned for any application without changing application code, regardless of runtime or data access model.
- Resource efficiency: Use of application runtime CPU and memory resources is minimized and can be tuned as needed to fit specific runtime environment parameters or limits.
- All-in-one deployment: A single driver JAR file that maximizes the data access simplicity for any Java environment or application.
- Streamlined standard: No proprietary extensions to the JDBC specification are required for any supported data source--a "clean" spec implementation.
Such "Type 5" JDBC drivers would truly enable modern data-driven Java applications to take advantage of years of innovation in database features, data access models, and virtualization technologies--in many cases without requiring code changes. This in turn would save organizations considerable time and money enhancing their modern data-driven Java applications by expanding feature sets and improving performance and reliability without having to make major changes to those applications.
| Attachment | Size |
|---|---|
| Graphic1.png | 13.04 KB |
image/png ( 13 ko)Contents
Prerequisites
The following background is prerequisite for complete understanding of this article:
- basic knowledge about web service simulation
- understanding of basic servlet technology concepts
- basic understanding of XML
See the Resources section for references on these technologies.
Introduction
Web service simulation is a growing need for every web service based assignment. Many of the assignments do not have the ready-to-use web service available during development stage, so the developers try to write their own mock implementation. A typical way of simulating a web service is:
- Using a bottom up approach, generate the artifacts from WSDL.
- Write the logic to choose the alternate response.
- Deploy to some server.
This is a common mechanism for simulating each service, and this effort is repeated for each simulation. Suppose you have 1000 web services: you have to repeat these three steps 1000 times. However, these can be avoided by writing one servlet which accepts all soap input messages and generates the response file name using the message content. This way, there is no need to create a web service for each WSDL, resulting in lot of time and money savings. Let's look at the some common cons of the standard web service simulation approach.
[Note: This article only provides conceptual knowledge. You need to do your own implementation.]
Drawbacks of the common web service simulation approach
As discussed in the introduction section, the standard three steps are typically repeated for each web service simulation. Buth this method has to the following drawbacks:
- Effort Repetition/Time consumed for the three steps
- Effort for defect resolution in simulation itself
- Deployment to servers (Generally, large projects have different environments, for example, Development, QA, etc). It also constitutes some effort
Of course, these are not all the drawbacks. However, some common issues can be mitigated with clever design and simulation. In the next section, I will discuss one of the web service simulation design practices that can mitigate these issues.
Web service simulation - designing using servlets
Let's go into the basics of web services and soap envelopes. The intended message resides in the soap body and is based on the message/content of the body; the response from the file system is picked up and can be returned to the caller.
This task simply can be achieved with servlets using following steps.
- Extract the soap envelop from input stream.
- Retrieve the first node of soap body.
- Apply the algorithm to generate the response file name using input message. (I'll discuss one of the algorithm in following sections).
- Read the response from file system.
- Prepare the soap envelop and send it to the caller
Let's discuss the algorithm to generate file name using input message.
Response File Name Generation algorithm
Let's discuss the algorithm using one example: the input SOAP message as shown in Listing 1.
Listing 1
<soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope"
soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding"
xmlns="http://abc.com">
<soap:body>
<m:findPlace xmlns:m="PlaceFinder">
<placeName>redlands</placeName>
<placeFinderOptions>
<dataSource>Oracle</dataSource>
<filterType>G</filterType>
<searchType>Random</searchType>
<fileterCountry>IN<fileterCountry>
<startIndex>0<startIndex>
<count>5<count>
</placeFinderOptions>
<token>TheToekn</token>
</m:findPlace>
</soap:body>
</soap:Envelope>
Note 1 - Expected response file to be picked from file system is findPlace_redlands_G.xml
Note 2 - Careful observations of the soap envelope suggest that the expected response file name convention is findPlace_ placeName_ filterType.xml
Let's understand this convention:
findPlaceis the first local node name of the soap bodyplaceNameis the child offindPlace(its xpath is //m:findPlace/placeName)filterTypeis the child of the nodeplaceFinderOptions(its xpath is //m:findPlace/placeFinderOptions/filterType)
Concatenating these three xpaths yields the expected response file name findPlace_redlands_G.xml. So, the crux of the algorithm is to use xpath to retrieve the node values and concatenate them. In the following section, I'll discuss the high level design & implementation to achieve this.
Designing the fully configurable algorithm
The heart of the idea is now how to achieve the above mentioned algorithm in a configurable manner so that there should not be any need to write any java code -- in other words 100% configurable and no deployment required for any change.
To achieve this, let us define XSD/XML of the form as in Listing 2. It is just a sample XML and you can define your own XML structure. Of course, you will have to write your one time xml parser/processor to process this XML.
Listing 2
<cfg:entry servicename="PlaceHunt">
<cfg:method methodName="findPlace">
<cfg:component>
<cfg:constant>findPlace</cfg:constant>
<cfg:xpath dataType="string">//m:findPlace/placeName </cfg:xpath>
</cfg:component>
<cfg:component>
<cfg:xpath dataType="string">//m:findPlace/placeFinderOptions/filterType</cfg:xpath>
</cfg:xpath>
</cfg:component>
</cfg:method>
</cfg:entry>
Here is the basic defination of XML element and attribute used in Listing 2:
- servicename - It is an attribute representing the name of the web service
- methodName - It is an attribute representing the name of the method inside the webservice
- cfg:constant - It specifies the constant value to be used for file name generation
- cfg:xpath - Specify the Xpath to node whose value will be used as a part of file name
Let's apply the algorithm using the Listing 2 XML.
- Web service client sends a request to simulation servlet (Say
WebServiceSimulatorServlet) - Web service simulator servlet extracts the first node from received soap body. This node is the actual web service message.
WebServiceSimulatorServletcalls the FileNameGenerator component to generate the response file name (It is the component which actually implements the file name generation algorithm. You need to write your own component)FileNameGeneratoruses the Listing 2 XML to generate the file name. It reads the XML, apply the xpath/xsl/etc on the input message received from servlet and concatenates the all values to generate the file name.FileNameGeneratorreturns the file name to theWebServiceSimulatorServlet.WebServiceSimulatorServletreads the response from file systemWebServiceSimulatorServletprepares the soap envelop and returns it to the caller.
Summary
The web service simulation has been achieved using a simple mechanism. During the journey key findings are:
- Servlet to simulate web service.
- Alter response file name generation logic can be done using simple XML logic.
- You need to write your own XML structure to achieve this. You can customize my XSD to accomplish your task.
- This logic needs to be developed once and afterwards there is no java coding is required. It's fully configurable which saves lot of effort, time and money.
Let's follow some good design and save the effort, time and money. You can find a sample XSD design to make a configurable web service simulator.
Resources
- Sample Simulation XSD. It can be used for starting up the design for configurable algorithm.
- Web service smiulation overview
- Servlet Tutorial
- XML Tutorial
Contents
If you use Maven, or even if you just use Maven repositories for your dependency management, you should be using a Maven Repository Manager. It's like using a kayak without paddles: you'll get there eventually without them, but your life will be much easier if you are properly equipped.
In this article, we'll be looking at some of the things a repository manager can do for you. A correctly-configured repository manager can speed up your builds, save bandwidth, help you share artifacts within your organization, and give you better control as to what dependencies are used in your projects and where they are coming from. It can also play a key role in your development infrastructure, helping you set up a fully-blown automated build and deployment pipeline.
There are currently three main Maven repository managers on the market. Archiva is a light-weight repository manager in the Apache fold. Artifactory is a powerful and quite innovative product from JFrog. And Nexus is a powerful, flexible, and superbly documented repository manager, this time from Sonatype, the company behind Maven. All are free and open source, though both Nexus and Artifactory have professional versions with extra features aimed at enterprise requirements. Most of the examples in this article will be using Nexus and it's commercial cousin Nexus Pro, though many of the techniques and general approaches are also applicable to the other products.
So read on, and learn how a repository manager can simplify your life as a software developer! And if you are already using a repository manager, this article might give you some tips as to how to get the most out of your repository.
Why use a Repository Manager
Probably the most fundamental role of a repository manager is to act as a proxy/cache between you and the internet. One of the most important features of Maven is the notion of declarative dependency management. Indeed, in a Maven project, you don't store the JAR files your project needs in a lib directory within your project - rather you list the dependencies you need directly in your build script. Ant users can also use declarative dependency management, either by using the Maven ant libraries or by using Ivy.
By default, Maven will attempt to download any dependencies it needs from public repositories on the internet, such as the Maven Central repository or the Codehaus repository. Any given jar file is only downloaded once, and cached on your workstation, no matter how many projects need it. And that's just fine if you are working alone. However, software development is a collaborative game, and most of us work in teams. So every developer will have to download the dependencies from the internet separately. In addition, Maven itself also downloads its own dependencies, so in a team of any size the quantity of downloaded files, and the time taken to download them, can mount up very quickly.
The first role of any Maven repository manager is to optimize this process (see Figure 1). It sits between your developer workstation and the internet repositories. Rather than going directly to the internet to download the required dependencies, Maven goes to the repository manager. Repository managers cache the files they download, so if the repository manager has already downloaded the dependency before, it can serve it out directly. If not, it goes to the internet to get it. In terms of overall build performance and consumed bandwidth, the gains are enormous.

Figure 1. A Repository Manager acts as a proxy server between you and the internet
Finer control on your downloads
But the benefits don't stop at economizing bandwidth. Once the repository manager becomes the central point of access for Maven dependencies within your organization, you can also use it to control where these dependencies come from, in much the same way as an internet proxy controls which web sites users can view. Indeed, many organizations like to have a clear visibility and control over what public repositories developers are using, and a repository manager can provide just that. Developers can no longer add arbitrary public repositories into their pom files - all dependencies must come from the repository manager, and all public repositories are configured and managed inside the repository manager. This also has the pleasant side effect of simplifying the repository configuration on the developer machines: you no longer need to maintain a long list of public repositories in each developer's settings.xml file.
This is typically done using the notion of Groups (in Nexus) or Virtual Repositories (in Artifactory). Repositories are organized into groups, so that a single URL (such as 'http://myserver:8081/nexus/content/groups/public/') provides access to an arbitrary list of real repositories (see Figure 2). From the point of view of the Maven user, all dependencies come from this single URL. However, behind the scenes, you can associate (and manage) many physical repositories. This gives organizations much better control over which repositories they want to authorize.

Figure 2. Repository Managers hide several public and internal repository URLs behind a single point of access.
In some organizations, such as in Defense or in the medical sector, requirements are even stricter. Developers cannot add arbitrary new versions of Spring or Hibernate, for example: new libraries must first be approved by an Enterprise Architect or Security Specialist before they can be deployed to production. Some organizations even require vetting of new libraries before they can be used by development teams. Another common requirement is to be able to vet artifacts provided by external vendors before using them internally.
A repository manager can help to rationalize and organize this process. In organizations like this, developers may only be allowed to use a repository containing approved dependencies, for example. Using Artifactory, you could achieve this by copying or moving artifacts between repositories by hand. The Nexus Procurement Suite, available in Nexus Pro, provides more sophisticated support for several procurement scenarios, including both approving individual JAR files and verifying entire configurations before they go into production.
Figure 3. Using a Repository Manager to allow only selected JAR files to be used for development.
This centralized management also opens the way to other forms of build optimization. For example, Nexus also has the notion of Routes, a very flexible feature which lets you give Nexus hints about where certain artifacts can be found. The latest version of Artifactory has a similar concept in the form of Include/Exclude patterns for each remote proxy. In my own repository, for example, internal artifacts (in the 'com.wakaleo' domain) are deployed to the local Snapshots and Releases repositories. Likewise, no 'org.apache' artifacts are ever deployed to these repositories - it would be a waste of time to look for them there. So, as shown in Figure 4 I have configured Nexus to only look for the internal artifacts in the Snapshots and Releases repositories, and never to look for the Apache artifacts in these repositories. In a similar vein, Nexus also lets you set up mirrors for your repositories, so that requests to a particular repository will go to a closer or faster mirror instead. This sort of fine-tuning reduces the number of places Nexus needs to look for a given artifact, and speeds up performance accordingly.
Figure 4. Setting up Routes in Nexus to optimize downloads by giving it hints about where to find certain artifacts.
The Repository Manager as part of the build lifecycle
But that's not all a repository manager can do for you. The second fundamental function of a repository manager is to make it easier to share build artifacts (JAR files, WAR files, and so forth) within an organization. You do this by setting up hosted repositories, internal repositories that are designed to store your own internal JAR files as well as proprietary JAR files not available from the public repositories.
Internal artifacts are typically released and deployed following a well-defined build lifecycle. For example, while work is in progress on a particular version, releases will only be made available for team members. Once the product is ready for further testing, it might be released to a test or UAT (User Acceptance Testing) platform, and then to production. Depending on your infrastructure setup, your repository manager can play a key role in this process.
In Maven development, for instance, it is good practice to distinguish between SNAPSHOT versions, which are under active development and are not considered stable or finalized, and RELEASE versions, which are tested and officially made public with a unique version number. A common best practice which is well supported by all the repository managers involves deploying SNAPSHOT versions and RELEASE versions to different hosted repositories within your repository manager.
Security considerations are also important. Larger organizations may want to limit deployment rights for projects, so that only developers from a given project team are allowed to deploy snapshot or release artifacts for that project to the repository. One way to do this would be to set up a separate repository for each project, but this is not a very scalable solution. A better approach is to define rules defining who is allowed to deploy to different parts of the repository. Nexus in particular provides a particularly fine-grained security model that lets you specify rules like this using a combination of regular expressions and pre-defined 'privileges'. The Apache repository is a good example of this - all of the release artifacts are stored in a single hosted Nexus repository, but developers for each of the many hosted projects can only deploy artifacts for their own projects.
For larger organizations, it is often important to be able to work with an existing LDAP repository when configuring what users are allowed to do with the repository. Artifactory supports basic authentication LDAP. As of its latest version (1.4.2), Nexus comes with sophisticated LDAP support, including both authentication and support for more advanced features such as mapping LDAP groups to Nexus roles.
The repository manager can also play a more active role in the build lifecycle. Typically, once a version has been released, it needs to be tested and validated by QA before it can be deployed to production. You can do this by using different repositories for different build promotion stages. Smaller organizations can often get away with a simple architecture consisting of just a snapshots repository (for work in progress) and a release repository (for release candidates and official releases). However, larger organizations will usually need a more sophisticated build promotion strategy.
Figure 5 illustrates one such possible build promotion strategy. In this architecture, snapshot builds are automatically deployed to the Enterprise Snapshots repository, to be used by developers within the project team itself. Release Candidate builds are prepared using the Maven Release plugin, and deployed to the Release Candidates repository. Builds in this repository can be deployed to System and UAT (User Acceptance Testing) testing environments. Once a version has been approved in the UAT environment, it can be promoted and deployed to the official Enterprise Releases repository, from where it can be deployed to production. This staged approach ensures that only properly validated versions can be deployed to production environments.
Figure 5. An example of a build promotion architecture.
In practice, implementing this sort of build promotion architecture requires a little thought and often some manual scripting. Indeed, the build promotion process itself can be tricky. You can rebuild and deploy from the source code for each platform, or redeploy manually to another repository; some repository managers like Artifactory even let you manually copy or move artifacts between repositories, though duplicating the same artifact between repositories is not considered to be good practice.
Nexus Pro, the commercial version of Nexus, comes with the Staging Suite which provides more sophisticated support for the build promotion process. The Staging Suite works by intercepting certain deployed artifacts and placing them in a special staging repository, dynamically-created by Nexus. For example, you might set up a staging configuration to intercept all artifacts from the com.mycompany.killerapp project. Whenever a developer deploys an artifact for this project, Nexus creates a special staging repository and places the newly deployed artifacts here (see Figure 6).
Figure 6. Staged artifacts are deployed to a special repository created by Nexus.
Once all the artifacts are deployed, the administrator can lock down the staging repository and make it available for further testing (for example, via an automated deployment to a QA platform). One approach is to make the staging repository available in a special Release Candidate repository group, from which it can be deployed to the various UAT and other testing environments (see Figure 7). At this point, other interested users will be notified that the artifacts have been staged.
Figure 7. Adding the staged repository to the Release Candidate repository group
If the testing is conclusive, the administrator can promote the build and deploy it to it's final repository (see Figure 8). If not, the administrator simply drops the staging repository and the developer redeploys a fixed version later on.
Figure 8. Promoting an artifact in Nexus.
There is also a Maven plugin for the Nexus Staging plugin, allowing this process to be integrated into Hudson or another CI tool.
This whole process makes generating releases from Maven projects a relatively smooth and painless operation. It is also a good example of how a repository manager is not just a directory of JAR files, but can underpin your whole deployment lifecycle as well.
Keeping your repository clean
A repository manager can also help you keep your repository clean. For example, automated builds can take up a lot of disk space, especially if you are generating regular snapshot builds. A repository manager will help you avoid wasting space with excessive numbers of snapshots. Nexus, for example, will regularly delete older snapshots, with the option of deleting all the snapshots for a particular artifact once that artifact is released.
Conclusion
A Maven repository manager is much more than a file server. In addition to the obvious performance benefits, using a repository manager also gives you more control over what artifacts are being downloaded, and from where. More advanced features such as Procurement can give an even finer control over the dependencies being used in development and/or being deployed to production.
A repository manager also can and should play a key role in your build promotion strategy. For smaller organizations, a simple snapshot/release repository setup may be enough, whereas larger organizations will often want to use the repository manager to underpin the entire build promotion and staging process.
Resources
JavaFX 1.2 introduced many interesting APIs, including APIs for reading RSS and Atom newsfeeds. My recent article, Learn about JavaFX's APIs for Reading RSS and Atom Newsfeeds, introduces you to these APIs, and includes
behind-the-scenes material on how the FeedTask class polls newsfeeds.
This article presents FeedRead, a newsfeed reader that demonstrates how the RSS and Atom APIs simplify integrating newsfeed-reading code into a JavaFX application. You first explore this example's code, and then examine some oddities that arise when running the example.
Discovering FeedRead
I started to play with JavaFX's RSS and Atom APIs after encountering Mark Macumber's inspiring JavaFX and RSS blog post, and created an application for reading RSS and Atom newsfeeds: FeedRead. Figure 1 shows FeedRead's user interface.

Figure 1. FeedRead's colorful user interface lets you navigate through RSS and Atom newsfeeds.
FeedRead's user interface consists of a textbox for entering a newsfeed URL, a Go button for obtaining the feed, a feed item panel for displaying individual feed items, and a set of four navigation buttons. Each of the five buttons enables/disables itself as necessary.
The panel presents the item's title and a link to its Web page. When you run FeedRead as an applet, and you click the link, browsers such as Firefox reveal a new tab that presents this item. (Clicking the link achieves nothing when you run FeedRead as a standalone application.)
I used NetBeans IDE 6.5.1 with JavaFX 1.2 to create and test FeedRead. The same-named project consists of two source files
(Main.fx and FeedItemPanel.fx) and two PNG-based images (feedicon.png and
feedread.png). Check out Main.fx:
/*
* Main.fx
*/
package feedread;
import java.lang.Exception;
import javafx.data.feed.atom.AtomTask;
import javafx.data.feed.atom.Entry;
import javafx.data.feed.atom.Feed;
import javafx.data.feed.rss.Channel;
import javafx.data.feed.rss.Item;
import javafx.data.feed.rss.RssTask;
import javafx.scene.Group;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.Label;
import javafx.scene.control.TextBox;
import javafx.scene.effect.DropShadow;
import javafx.scene.effect.Reflection;
import javafx.scene.image.Image;
import javafx.scene.image.ImageView;
import javafx.scene.layout.Flow;
import javafx.scene.paint.Color;
import javafx.scene.paint.LinearGradient;
import javafx.scene.paint.Stop;
import javafx.scene.text.Font;
import javafx.scene.text.Text;
import javafx.stage.Alert;
import javafx.stage.Stage;
class FeedItem
{
var title: String;
var link: String
}
var items: FeedItem [];
var index: Integer = 0;
var feedImageURL: String;
var buttonGoRef: Button;
var textBoxURLRef: TextBox;
function go (url: String): Void
{
delete items;
index = 0;
buttonGoRef.disable = true;
buttonGoRef.focusTraversable = false;
RssTask
{
interval: 60s
location: url
onChannel: function (c: Channel)
{
feedImageURL = c.image.url
}
onItem: function (i: Item)
{
insert FeedItem
{
title: i.title
link: i.link
}
into items
}
onException: function (e: Exception)
{
def msg = e.getMessage ();
if (msg == "must use AtomTask for Atom feeds")
goAtom (url)
else
Alert.inform ("Error", e.getMessage ());
buttonGoRef.disable = false;
buttonGoRef.focusTraversable = true
}
onDone: function ()
{
buttonGoRef.disable = false;
buttonGoRef.focusTraversable = true;
}
}.update ()
}
function goAtom (url: String): Void
{
AtomTask
{
interval: 60s
location: url
onFeed: function (f: Feed)
{
feedImageURL = f.icon.uri
}
onEntry: function (e: Entry)
{
var title = e.title.text;
if (title.length () > 100)
title = "{title.substring (0, 100)}...";
insert FeedItem
{
title: title
link: e.links [0].href
}
into items
}
onException: function (e: Exception)
{
Alert.inform ("Error", e.getMessage ());
buttonGoRef.disable = false;
buttonGoRef.focusTraversable = true
}
onDone: function ()
{
buttonGoRef.disable = false;
buttonGoRef.focusTraversable = true
}
}.update ()
}
Stage
{
title: "FeedRead"
var sceneRef: Scene
scene: sceneRef = Scene
{
width: 550
height: 350
fill: LinearGradient
{
startX: 0.0
startY: 0.0
endX: 0.0
endY: 1.0
stops:
[
Stop { offset: 0.0 color: Color.NAVY },
Stop { offset: 0.5 color: Color.BLUEVIOLET },
Stop { offset: 1.0 color: Color.NAVY }
]
}
var flowRef: Flow
content: flowRef = Flow
{
vertical: true
vgap: 20
layoutX: bind (sceneRef.width-flowRef.layoutBounds.width)/2-
flowRef.layoutBounds.minX
layoutY: bind (sceneRef.height-flowRef.layoutBounds.height)/2-
flowRef.layoutBounds.minY
content:
[
ImageView
{
image: Image
{
url: "{__DIR__}res/feedread.png"
}
}
Flow
{
hgap: 20
var textBoxURLRef: TextBox
content:
[
Label
{
graphic: Text
{
content: "URL"
fill: Color.GOLD
font: Font
{
name: "Arial BOLD"
size: 14
}
effect: DropShadow
{
spread: 0.5
}
}
}
textBoxURLRef = TextBox
{
columns: 40
}
buttonGoRef = Button
{
text: "Go"
action: function (): Void
{
go (textBoxURLRef.text)
}
}
]
}
Group
{
content: FeedItemPanel
{
itemTitle: bind items [index].title
itemLink: bind items [index].link
feedImageURL: bind feedImageURL
effect: Reflection { fraction: 0.6 }
}
}
Flow
{
hgap: 20
content:
[
Button
{
text: "|<"
disable: bind if ((sizeof items == 0) or
(index == 0))
then true else false
focusTraversable: bind if ((sizeof items == 0) or
(index == 0))
then false else true
action: function (): Void
{
index = 0
}
}
Button
{
text: "<"
disable: bind if ((sizeof items == 0) or
(index == 0))
then true else false
focusTraversable: bind if ((sizeof items == 0) or
(index == 0))
then false else true
action: function (): Void
{
if (index != 0)
index--
}
}
Button
{
text: ">"
disable: bind if ((sizeof items == 0) or
(index == sizeof items-1))
then true else false
focusTraversable: bind if ((sizeof items == 0) or
(index == sizeof items-1))
then false else true
action: function (): Void
{
if (index != sizeof items-1)
index++
}
}
Button
{
text: ">|"
disable: bind if ((sizeof items == 0) or
(index == sizeof items-1))
then true else false
focusTraversable: bind if ((sizeof items == 0) or
(index == sizeof items-1))
then false else true
action: function (): Void
{
index = sizeof items-1
}
}
]
}
]
}
}
}
This code models a newsfeed as an items sequence of FeedItem instances (identifying each feed item's
title and link), an index into this sequence (identifying the current
FeedItem), and feedImageURL (a URL to the newsfeed's logo).
The model specification is followed by a go() function that's invoked when the user presses the
Go button. This function performs the following tasks:
-
Remove the previous newsfeed from the model:
delete items; index = 0; -
Disable the Go button so that it cannot be clicked:
buttonGoRef.disable = true; -
Prevent the user from shifting focus to this button via the keyboard:
buttonGoRef.focusTraversable = false; -
Assume that the newsfeed is in RSS format by instantiating
RssTask. -
Invoke
update()on theRssTaskinstance so that theonDone()function is invoked (if the newsfeed is in RSS format) -- it's important to re-enable the Go button when the newsfeed's items have been successfully obtained. -
From within
RssTask'sonException()function, invoke thegoAtom()function to instantiateAtomTaskand invoke itsupdate()function ifonException()'seargument'sgetMessage()method returns"must use AtomTask for Atom feeds". Although this test is brittle and could break in a future JavaFX version (if the message changes), it's easier than working withHttpRequestto obtain the newsfeed XML, and working withPullParserto parse enough of the feed to determine if it's Atom or RSS. WhenRssTask's orAtomTask'sonException()function is invoked, it's important to alert the user to the problem. This task is accomplished by invokingjavafx.stage.Alert'spublic static void inform(String title, String message)method to present a dialog displayinggetMessage()'s value. The Go button is then re-enabled.
The model's items sequence is populated in RssTask's onItem() and
AtomTask's onEntry() functions. Although I restrict the lengths of Atom feed item titles (which can
become quite lengthy), I haven't yet needed to do the same with RSS feed item titles.
Also, the model's feedImageURL variable is populated in RssTask's onChannel() function
via feedImageURL = c.image.url;, and in AtomTask's onFeed() function via
feedImageURL = f.icon.uri;.
At this point, the application's stage is created and its scene is laid out. The scene is organized within a
javafx.scene.layout.Flow instance, whose layoutX and layoutY variables are initialized
such that the scene is centered on the stage.
Perhaps you're wondering why I subtract flowRef.layoutBounds.minX from the rest of the expression in
layoutX: bind (sceneRef.width-flowRef.layoutBounds.width)/2-flowRef.layoutBounds.minX
and do something similar with layoutY.
Amy Fowler provides the rationale in her JavaFX1.2: Layout blog post. She first states that "to
establish a node's stable layout position, set layoutX/layoutY." Amy then goes on to state the
following:
Be aware that these variables define a translation on the node's coordinate space to adjust it from its current
layoutBounds.minX/minYlocation and are not final position values. This means you should use the following formula for positioning a node at x,y:node.layoutX = x - node.layoutBounds.minX node.layoutY = y - node.layoutBounds.minYOr, in the case of object literals, don't forget the
bind:def p = Polygon { layoutX: bind x - p.layoutBounds.minX layoutY: bind y - p.layoutBounds.minY ... }
The remaining scene code is fairly straightforward, but you might wonder why I wrap a FeedItemPanel instance in
a javafx.scene.Group instance. I do this to include this component's reflection in the group's layout bounds, so
the navigation buttons appear below the reflection.
The FeedItemPanel component is responsible for presenting the current FeedItem's (represented via
items [index]) title, and associating it with a link. This class's source code is presented below.
/*
* FeedItemPanel.fx
*/
package feedread;
import javafx.scene.CustomNode;
import javafx.scene.Group;
import javafx.scene.Node;
import javafx.scene.control.Hyperlink;
import javafx.scene.image.Image;
import javafx.scene.image.ImageView;
import javafx.scene.layout.Panel;
import javafx.scene.paint.Color;
import javafx.scene.shape.Rectangle;
import javafx.scene.text.Font;
import javafx.scene.text.Text;
import javafx.scene.text.TextOrigin;
import javafx.stage.AppletStageExtension;
public class FeedItemPanel extends CustomNode
{
public var itemTitle: String;
public var itemLink: String;
public var feedImageURL: String on replace
{
if (feedImageURL == null or feedImageURL == "")
imageRef = Image { url: "{__DIR__}res/feedicon.png" }
else
imageRef = Image { url: feedImageURL }
}
var imageRef: Image;
override function create (): Node
{
Group
{
var hyperlinkRef: Hyperlink
var imageViewRef: ImageView
var rectangleRef: Rectangle
var titleTextRef: Text
content:
[
rectangleRef = Rectangle
{
width: 400
height: 80
arcWidth: 15
arcHeight: 15
stroke: Color.GOLD
strokeWidth: 3.0
fill: Color.WHITE
}
imageViewRef = ImageView
{
layoutX: bind (rectangleRef.layoutBounds.width-
imageViewRef.layoutBounds.width)/2-
imageViewRef.layoutBounds.minX
layoutY: bind (rectangleRef.layoutBounds.height-
imageViewRef.layoutBounds.height)/2-
imageViewRef.layoutBounds.minY
opacity: 0.3
fitWidth: bind if (imageRef.width > 380) then 380
else imageRef.width
fitHeight: bind if (imageRef.height > 60) then 60
else imageRef.height
preserveRatio: true
image: bind imageRef
}
titleTextRef = Text
{
layoutX: bind (rectangleRef.layoutBounds.width-
titleTextRef.layoutBounds.width)/2-
titleTextRef.layoutBounds.minX
layoutY: bind if (itemLink != null)
10-titleTextRef.layoutBounds.minY
else
(rectangleRef.layoutBounds.height-
titleTextRef.layoutBounds.height)/2-
titleTextRef.layoutBounds.minY
content: bind itemTitle
wrappingWidth: bind rectangleRef.width-10
textOrigin: TextOrigin.TOP
font: Font
{
name: "Arial"
size: 16
}
}
Panel
{
content: hyperlinkRef = Hyperlink
{
layoutX: bind (rectangleRef.layoutBounds.width-
hyperlinkRef.layoutBounds.width)/2-
hyperlinkRef.layoutBounds.minX
layoutY: bind (rectangleRef.layoutBounds.height-
hyperlinkRef.layoutBounds.height-10)-
hyperlinkRef.layoutBounds.minY
text: bind if (itemLink == null) then ""
else ">>> Check it out! <<<"
focusTraversable: false
action: function (): Void
{
AppletStageExtension.showDocument (itemLink,
"_blank")
}
}
}
]
}
}
}
FeedItemPanel subclasses javafx.scene.CustomNode, provides itemTitle and
itemLink variables for storing the current feed item's title and link, and provides a feedImageURL
variable for storing the URL of the newsfeed's logo image.
The feedImageURL variable is associated with a replace trigger that instantiates
javafx.scene.image.Image for the default logo, or for the newsfeed's logo whenever this variable is modified.
The Image reference is stored in imageRef.
Initially, I attached the trigger's code with a bind to the javafx.scene.image.ImageView instance's
image variable. However, I discovered that the newly-created Image's reference wasn't always
assigned to imageRef. Go figure!
CustomNode's create() function returns a Group instance that specifies this
component's user interface via instances of the following four classes:
-
javafx.scene.shape.Rectangleis used to present a background rectangle with a golden border and rounded corners. -
ImageViewis used to present the newsfeed logo. I found that some logos can get very large, and overflow theFeedItemPanel's display area. Rather than resort to explicit clipping, I take advantage offitWidthandfitHeightvariables to constrain the size of the displayed image, andpreserveRatioto ensure that the image looks good. -
javafx.scene.text.Textis used to present the current item's title. I originally intended to have theFeedItemPanelcomponent also present exception messages, and to center these messages within the panel. This is the reason for theif-elseexpression to whichText'slayoutYvariable is bound. -
javafx.scene.control.Hyperlinkis used to present generic link text and associate the current item's link with this text. I assignfalsetoHyperlink'sfocusTraversablevariable to prevent the hyperlink from receiving focus -- when this happens, the focus rectangle disappears, which can be disconcerting to the user.
The AppletStageExtension.showDocument (itemLink, "_blank") method call within the function that's assigned to
Hyperlink's action attribute ensures that the Web page at the associated itemLink
value is presented in a _blank browser window.
Finally, you might be wondering why I wrap the javafx.scene.control.Hyperlink instance within a
javafx.scene.layout.Panel instance. The brief answer is that I want FeedItemPanel to display
>>> Check it out! <<<, which won't happen without Panel.
The "When Resizables Are Not Managed by Containers" section of Amy Fowler's JavaFX1.2:
Layout blog post explains why Panel is required. Because Amy does an excellent job of explaining JavaFX, I
recommend that you read that section to discover the answer.
Testing FeedRead
FeedRead can be created to run as an application or an applet. However, you'll probably want to run it as an applet so that you can click on an item's link and view the item in its own Web page window. Just make sure to sign the applet's JAR file before deploying FeedRead.
I tested the FeedRead applet with JavaFX 1.2 and Mozilla Firefox 3.5.5 on a Windows XP SP3 platform. During my tests, I encountered a few oddities, which I discuss in this section. Most of these oddities probably result from JavaFX runtime bugs.
The first oddity involves the textbox not always receiving focus when FeedRead runs as an applet. You need to reload the applet or switch from the browser window to another window and then back to the browser window, to observe a focused textbox.
This oddity has been documented in the JIRA issue tracker for JavaFX as issue number JFXC-3431, "Signed javafx-applet does not get focus before html page area is clicked."
The next oddity deals with a button occasionally looking disabled when it's actually enabled. This sometimes happens with the Go button, but it can also happen with a navigation button, as revealed in Figure 2.
< button appears to be disabled when it's actually enabled." src="http://download.java.net/general/jn_images/008/figure2.jpg">
Figure 2. The < button appears to be disabled when it's actually enabled.
The third oddity deals with my not providing code to unescape an Atom feeditem's title -- notice the escaped
in Figure 3's title text. I leave it as an exercise for you to provide code that unescapes an Atom
feed's title.

Figure 3. Notice the escaped in the title text; also, the Go button appears to be disabled when it's actually enabled.
The previous oddities aren't as severe as specifying a URL such as http://www.x.com and selecting
Go, which results in Go remaining disabled. You must reload the
applet because onDone()/onException() is never invoked to re-enable this button.
One oddity that bugs me is receiving an empty string from getMessage() (from within onException()).
Because it's disconcerting to see an error dialog without any text, modify FeedRead to test for this possibility, and present
a generic error message if "" is returned.
Entering nothing into the textbox and selecting Go causes RssTask or
AtomTask to throw IllegalArgumentException, preventing Go from being
re-enabled. It's too bad that onException() isn't invoked.
Finally, pasting HTML into the textbox and clicking Go can cause the textbox to lockup. You'll
probably notice thrown IllegalArgumentExceptions with the message TextHitInfo is out of range. It
would be nice to disable copy-and-paste.
Conclusion
Now that you've explored FeedRead, you might want to extend this example. If you need some ideas, check out the RSS newsreaders shown in Rakesh Menon's RSS Viewer - JavaFX to JavaScript Communication sample article and the RSS Reader in JavaFX YouTube video.
Resources
- Sample code for this article
- Amy Fowler's JavaFX1.2: Layout blog post
- Atom specification
- Learn about JavaFX's APIs for Reading RSS and Atom Newsfeeds
- JavaFX 1.2.1 API Documentation
- JavaFX issues tracker at JIRA
- JavaFX's official Website
- JFXC-3431: Signed javafx-applet does not get focus before html page area is clicked
- Mark Macumber's JavaFX and RSS
- Rakesh Menon's RSS Viewer - JavaFX to JavaScript Communication sample article
- RSS specification
- Wikipedia's Atom (standard) entry
- Wikipedia's RSS entry
- Youtube's RSS Reader in JavaFX video
- Contents
- JNA Development First Steps
- A Proxy for the DLL
- Linkage: What's in a Name?
- Parameter and Return Types
- C structs in Java
- Pointers and Strings
- Converting from JNI to JNA
- Detecting a Struct Memory Alignment Problem
- Reproducing the Struct Alignment Error
- Preventing Struct Alignment Errors
- Running the Sample Code
- Resources
This article describes the Java Native Access (JNA) approach to integrating native libraries with Java programs. It shows how JNA enables Java code to call native functions without requiring glue code in another language. The examples illustrate usage patterns, common pitfalls, and troubleshooting techniques. The article also enables a comparison of JNA and JNI (Java Native Interface) by describing the conversion of sample JNI code from an earlier java.net article to JNA.
It is useful to know JNA because the Java APIs, with their architecture-neutral emphasis, will never support platform-specific functionality. So, for example, if that killer app you've just invented needs to play the Windows "Critical Stop" sound you'll be stuck as the Windows MessageBeep() function can't be called via the standard APIs.
Though Java itself is architecture-neutral, the example code used in this article is, perforce, platform-specific. The code has been developed and tested on a Laptop PC running 32-bit Microsoft Windows XP and Sun JRE 1.6.0 update 16. However, the code is quite generic and should run on a range of Windows and JVM versions. Features new in Java 1.6, Windows 2008, and Windows Vista have not been used.
JNA Development First Steps
Here are a few things you have to take care of when starting a JNA project:
Download jna.jar from the JNA project site and add it to your project's build path. This file is the only JNA resource you need. Remember that jna.jar must also be included in the run-time classpath.
Find the names of the DLLs that your Java code will access. The DLL names are required to initialize JNA's linkage mechanisms.
Create Java interfaces to represent the DLLs your application will access. The sample code accompanying this article contains example interfaces for three DLLs: kernel32.dll, user32.dll, and Twain_32.dll.
Test linkage of your Java code to the native functions. The first example below, "Linkage: What's in a Name?", describes the exceptions to expect when JNA can't find a DLL or a function in a DLL.
If your project is large or complex, it may be a good idea to complete these steps in an early phase. If a proof of concept (POC) is required, consider including a significant portion of JNA interface code in the POC. This helps to validate assumptions about JNA's suitability for the job, and reduces overall project risk.
A Proxy for the DLL
JNA uses the proxy pattern to hide the complexity of native code integration. It provides a factory method that Java programs use to obtain a proxy object for a DLL. The programs can then invoke the DLL's functions by calling corresponding methods of the proxy object. The sequence diagram in Figure 1 below depicts the creation and use of a proxy object.

Figure 1. Creation of a Java proxy object for a DLL
JNA takes care of all run-time aspects, but it requires your help to
create the proxy's Java class.
So the first piece of code you need to create is a Java interface with
method definitions that match the DLL's
C functions.
To play with JNA's run-time correctly, the interface must
extend com.sun.jna.Library.
The code below shows an abbreviated view of a proxy interface for the
Windows
user32 DLL.
Note that there should be one such Java interface for each DLL.
package libs;
import com.sun.jna.win32.Library;
public interface User32 extends Library {
... (lines deleted for clarity) ...
boolean LockWorkStation();
boolean MessageBeep(int uType);
... (lines deleted for clarity) ...
}
Many DLLs, such as those in the Windows API, host a large number of functions. But the proxy interface need only contain declarations for the methods your application actually uses.
Linkage: What's in a Name?
Our first example (LockWorkStation.java) is extremely simple, and locks the
workstation when it is run (same effect as pressing the
Windows logo + L
keys together). It uses the User32 interface shown above to
create a proxy for the Windows
user32 DLL.
It then calls the proxy's LockWorkStation() method -- which
in turn invokes the DLL's
LockWorkStation() function. The run-time mapping
of the proxy method to the DLL function is handled transparently by JNA --
the user just has to ensure that the method name matches the function name exactly.
import com.sun.jna.Native; // JNA infrastructure
import libs.User32; // Proxy interface for user32.dll
public class LockWorkStation {
public static void main(String[] args) {
// Create a proxy for user32.dll ...
User32 user32 = (User32) Native.loadLibrary("user32", User32.class);
// Invoke "LockWorkStation()" via the proxy ...
user32.LockWorkStation();
}
}
To compile and run this program follow the instructions at "Running the Sample Code" below.
The absence of parameters and a return value in the LockWorkStation()
call eliminates the possibility of any programming errors. But there are still
two things that can go wrong with code as simple as this:
loadLibrary()throws ajava.lang.UnsatisfiedLinkErrorwith the message "Unable to load library ... The specified module could not be found." If this error occurs check the spelling of the DLL name, and verify that the DLL is in one of the searched directories (check JNA documentation).A proxy method (such as
LockWorkStation()) throws ajava.lang.UnsatisfiedLinkErrorwith the message "Error looking up function ... The specified procedure could not be found." If this error occurs check the spelling of the function name, and verify that the "function" is not actually a macro. The Windows API DLLs contain quite a few such macros (e.g. GetMessage(), defined in Winuser.h), so read the DLL's documentation (and the associated header files) carefully. Macro names must be translated manually.
You shouldn't get either of these exceptions when running LockWorkStation.java. But you can simulate these errors just by changing the name of a DLL or a function and recompiling the code. JNA does, in fact, have mechanisms to allow you to use a method name (in the proxy interface) that is different from the function name (in the DLL). More information on this feature can be found in the JNA documentation.
Parameter and Return Types
Our next example, BeepMorse.java shown below, uses the Windows Beep() function to literally beep "Hello world" in Morse code.
import com.sun.jna.Native; // JNA infrastructure
import libs.Kernel32; // Proxy interface for kernel32.dll
public class BeepMorse {
private static Kernel32 kernel32 = (Kernel32)
Native.loadLibrary("kernel32", Kernel32.class);
private static void toMorseCode(String letter) throws Exception {
for (byte b : letter.getBytes()) {
kernel32.Beep(1200, ((b == '.') ? 50 : 150));
Thread.sleep(50);
}
}
public static void main(String[] args) throws Exception {
String helloWorld[][] = {
{"....", ".", ".-..", ".-..", "---"}, // HELLO
{".--", "---", ".-.", ".-..", "-.."} // WORLD
};
for (String word[] : helloWorld) {
for (String letter : word) {
toMorseCode(letter);
Thread.sleep(150);
}
Thread.sleep(350);
}
}
}
Beep() takes two arguments, frequency and duration,
both of type DWORD which is
defined
as unsigned long. Since an unsigned long occupies
32 bits in all current flavors of Windows, we use a Java int for
both arguments in the proxy interface definition shown below:
package libs;
import com.sun.jna.Library;
public interface Kernel32 extends Library {
// ... (lines deleted for clarity) ...
boolean Beep(int frequency, int duration);
int GetLogicalDrives();
// ... (lines deleted for clarity) ...
}
It is important to deduce the argument types correctly as you can verify
by changing the type of Beep()'s arguments. Changing the definition to
Beep(long, long) or Beep(float, float)
does not cause any run-time error, but you will hear no sound at all.
The JNA web-site has
some information on translating
Windows types to Java types. More details can be found at wikibooks'
Windows Programming/Handles and Data Types page and Microsoft's
Windows Data Types page.
To compile and run this program follow the instructions at "Running the Sample Code" below, but remember to turn the volume down first!
Beep() returns a boolean value, although it is ignored in this example.
But if the value returned by a function has to be used, the return type must be
mapped to a suitable Java type using the same guidelines as for parameter types.
The code below (GetLogicalDrives.java) illustrates the use of the
int value returned by
GetLogicalDrives() in the kernel32 DLL.
import com.sun.jna.Native;
import libs.Kernel32;
public class GetLogicalDrives {
public static void main(String[] args) {
Kernel32 kernel32 = (Kernel32)
Native.loadLibrary("kernel32", Kernel32.class);
int drives = kernel32.GetLogicalDrives();
for (int i = 0; i < 32; ++i) {
int bit = (1 << i);
if ((drives & bit) == 0)
continue;
System.out.printf("%c:\\%n", (char) ((int) 'A' + i));
}
}
}
Note, however, that in practice a Java program should never have to call
GetLogicalDrives() using JNA as java.io.File.listRoots()
provides the same information.
To compile and run this program follow the instructions at "Running the Sample Code" below.
The article's introduction mentioned the use of Windows standard sounds for
indicating specific events. These sounds can be produced by calling the
MessageBeep(int type) function. Example code showing the use of
MessageBeep() can be found in the file MessageBeep.java.
C structs in Java
C Functions often use
structs as arguments. But since Java does not have structs,
JNA uses classes instead. Classes are
closely related
to structs, so the associated Java code looks intuitive, and works well.
The following code extracted from Kernel32.java, the proxy interface for
kernel32.dll, illustrates the conversion of
struct SYSTEMTIME
into a Java class to support the
GetSystemTime()
function.
import com.sun.jna.Library;
import com.sun.jna.Structure;
public interface Kernel32 extends Library {
// ... (other members deleted) ...
public static class SYSTEMTIME extends Structure {
public short wYear;
public short wMonth;
public short wDayOfWeek;
public short wDay;
public short wHour;
public short wMinute;
public short wSecond;
public short wMilliseconds;
}
void GetSystemTime(SYSTEMTIME st);
// ... (other members deleted) ...
}
Note that Java classes that substitute C structs must extend JNA's
com.sun.jna.Structure base class.
Embedding these classes inside the proxy interface helps to keep everything
neatly organized in a single file. This is particularly effective when the struct is
only used by functions in the same proxy interface. These classes can, however, also be
defined as standalone public classes (outside the proxy interface) if that is
required or preferred. The JNA web site has more information on these aspects.
The code shown below, GetSystemTime.java in the sample code, illustrates
the use of structs. In this example the called function uses the struct to pass
information "out", but structs can be used to pass information "in" (as in the
Windows
SetSystemTime() function) or "in and out" as well.
import libs.Kernel32;
import libs.Kernel32.SYSTEMTIME;
import com.sun.jna.Native;
public class GetSystemTime {
public static void main(String[] args) {
Kernel32 kernel32 = (Kernel32)
Native.loadLibrary("kernel32", Kernel32.class);
SYSTEMTIME st = new SYSTEMTIME();
kernel32.GetSystemTime(st);
System.out.printf("Year: %d%n", st.wYear);
System.out.printf("Month: %d%n", st.wMonth);
System.out.printf("Day: %d%n", st.wDay);
System.out.printf("Hour: %d%n", st.wHour);
System.out.printf("Minute: %d%n", st.wMinute);
System.out.printf("Second: %d%n", st.wSecond);
}
}
To compile and run this program follow the instructions at "Running the Sample Code" below.
It is important to deduce the type of each member of a converted struct correctly.
Erring here usually has catastrophic consequences that you can sample by changing the
types in SYSTEMTIME. There are other JNA tweaks that can
be applied to specify whether a struct should be
passed by reference
(the default) or
by value, and also
how a struct embedded within another should be stored.
The JNA web-site has much guidance on these aspects.
The section titled "Converting from JNI to JNA" below has several examples of
the conversion of C structs to Java classes.
No discussion of struct portability across languages is complete without also considering memory alignment requirements. Since this part of the article is dedicated to JNA basics we defer discussion of alignment requirements to a later section "Converting from JNI to JNA".
Pointers and Strings
Using pointers is a perfectly natural thing to do in C, C++, and certain other languages. But the use of pointers also proliferated certain errors and programming malpractices that Java's inventors wanted to prevent. So, although Java programs have an uncanny resemblance to C++ code, Java has no pointers. But pointers of one kind or another are commonly used as parameters in native functions, so JNA programs must be creative in working around this limitation.
The following example (GetVolumeInformation.java) exploits a language feature from Java's C heritage: an array reference is a pointer to the array's first element.
import libs.Kernel32;
import com.sun.jna.Native;
public class GetVolumeInformation {
private static String b2s(byte b[]) {
// Converts C string to Java String
int len = 0;
while (b[len] != 0)
++len;
return new String(b, 0, len);
}
public static void main(String[] args) {
Kernel32 kernel32 = (Kernel32) Native.loadLibrary(
"kernel32", Kernel32.class);
int drives = kernel32.GetLogicalDrives();
for (int i = 0; i < 32; ++i) {
if ((drives & (1 << i)) == 0)
continue;
String path = String.format("%c:\\", (char) ((int) 'A' + i));
byte volName[] = new byte[256], fsName[] = new byte[256];
int volSerNbr[] = new int[1], maxCompLen[] = new int[1], fileSysFlags[] = new int[1];
boolean ok = kernel32.GetVolumeInformationA(path, volName,
256, volSerNbr, maxCompLen, fileSysFlags, fsName, 256);
if (ok)
System.out.printf("%s %08X '%s' %s %08X%n", path, volSerNbr[0],
b2s(volName), b2s(fsName), fileSysFlags[0]);
else
System.out.printf("%s (Offline)%n", path);
}
}
}
GetVolumeInformation()'s
specification states that its
4th thru 6th arguments (highlighted above) are of type
LPDWORD which translates to "pointer to int".
We circumvent Java's lack of pointers by using int arrays for
these arguments instead. So in the proxy's method declaration these arguments
are defined to be of type int[], and at run-time (see
code above) we pass int arrays of one element. The values
returned by GetVolumeInformation() are left in the single
int that populates each array.
The output from this program is shown below. On my computer D: is a CD-ROM drive that was not loaded at the time this output was captured. The device at G: was a USB flash drive.
C:\ 609260D7 'My-C-Drive' NTFS 000700FF
D:\ (Offline)
E:\ C8BCF084 'My-E-Drive' NTFS 000700FF
G:\ 634BE81B 'SDG-4GB-DRV' FAT32 00000006
To compile and run this program follow the instructions at "Running the Sample Code" below.
Another thing to notice in the above code is the way strings are passed to and
from native code. Java Strings can be passed "in" to the native code
without special effort (check variable path in code above). But
null-terminated strings
passed "out" to Java require careful handling.
Check the use of the variables volName
and fsName, and the method b2s(byte b[]) in the code above.
Finally, note that
GetVolumeInformation() is a macro whose "real" name
is GetVolumeInformationA(). Read the function's
specification for all the details.
Another approach to pointers in Java is based on the classes in the
package com.sun.jna.ptr and the class com.sun.jna.Pointer.
Examples of the use of these classes can be found in the code discussed under
"Converting from JNI to JNA" below.
Converting from JNI to JNA
Having covered the basics, it's now time to pit your wits against something more substantial. The rest of this article describes issues faced in converting an existing application (based on JNI) to JNA, Reviewing the converted code (included with the sample code) should provide greater insight into how JNA can be used to handle the complexities of a "real" application. The JNI code used comes from the article "Java Tech: Acquire Images with TWAIN and SANE, Part 1", which describes how the TWAIN library is used to obtain images from scanners webcams, and other imaging devices.
To run the TWAIN code you should ideally have a TWAIN device (scanner, webcam, etc.) connected to your computer. But if your computer does not have a TWAIN device, you should download and install the TWAIN Developer Toolkit which contains a program that simulates an image source. To understand the code you should also have the TWAIN header file available.
To run the TWAIN demo program execute JTwainDemo.bat as described at "Running the Sample Code" below. To understand the overall flow of the program, follow the instructions starting at Let There Be TWAIN in the original JNI article.
Figure 2 below depicts the changes that have been made to the sample code from the JNI article.

Figure 2. Changes to "code.zip" from the JNI article
jtwain.cpp and twain.h have been deleted as
they contained only JNI-specific code. Philos.java has been deleted as
it was unrelated to TWAIN or JNI. JTwain.java has been modified to
contain a JNA implementation of the TWAIN functionality instead of the original
JNA code.
The package libs is new. Its three files (Kernel32.java,
User32.java, and Win32Twain.java) are the proxy
interfaces discussed in this article.
The remaining 3 files stay unchanged. Observe that the package democode,
containing the simple example programs described above, is not shown in Figure 2.
Converting the TWAIN code to JNA provides the usual learning experiences of any non-trivial project. But it also throws up a rare and elusive type of bug -- struct memory alignment error -- that is unique to Java projects using native-code. Since memory alignment errors are difficult to detect, and they may also be new to many Java users, the following sections provide a detailed guide to handling these errors.
Detecting a Struct Memory Alignment Problem
The devil is in the details here, so there can no simple, non-intrusive, way of
concluding that a particular bug is caused by a memory alignment mismatch.
But the following, necessarily tedious, discussion describes one approach.
The affected code (shown below) is in JTwain.java,
and can be located by searching for "Memory Alignment Problem Demo".
The code creates an instance of TW_IDENTITY (a Java substitute
for a struct), and passes it to the TWAIN run-time. TWAIN then interacts with the
user to select a source device. The TW_IDENTITY instance is
uninitialized when passed from Java to TWAIN, but is returned populated with
information about the selected device. The printf()s, and the
dump() at the end of the code display parts of the struct
to help in detecting the problem.
// BEGIN: Memory Alignment Problem Demo
TW_IDENTITY srcID = new TW_IDENTITY(Structure.ALIGN_DEFAULT);
stat = SelectSource(g_AppID, srcID);
if (stat != TWRC_SUCCESS) {
//... (lines deleted for clarity) ...
}
System.out.printf("ProtocolMajor: %02x%n", srcID.ProtocolMajor);
System.out.printf("ProtocolMinor: %02x%n", srcID.ProtocolMinor);
System.out.printf("SupportedGroups: %04x%n", srcID.SupportedGroups);
System.out.printf("Manufacturer: %s%n", new String(srcID.Manufacturer, 0, 34));
dump(srcID);
// END: Memory Alignment Problem Demo
The output from the printf() statements shown below give
a strong hint that a memory alignment problem exists:
ProtocolMajor: 01
ProtocolMinor: 09
SupportedGroups: 694d0000
Manufacturer: crosoft Tw
The first two values, ProtocolMajor and ProtocolMinor, are
correct but the next two are certainly corrupted. In a previous TWAIN call, the Java code
negotiated the value 0x0003 for SupportedGroups, so that same value
should have been returned. Also the value of Manufacturer certainly
looks like "Microsoft" with the first two characters lopped off.
Now let's look at the output from dump() shown below.
The "dump" displays the contents of the struct
as received from the native code, before separation by JNA into individual
member values.
000: 11 04 00 00 01 00 00 00 0d 00 01 00 32 36 20 4a
000: . . . . . . . . . . . . 2 6 J
016: 75 6e 65 20 32 30 30 30 00 00 00 00 00 00 00 00
016: u n e 2 0 0 0 . . . . . . . .
032: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00
032: . . . . . . . . . . . . . . . .
048: 09 00 03 00 00 00 4d 69 63 72 6f 73 6f 66 74 00
048: . . . . . . M i c r o s o f t .
064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
064: . . . . . . . . . . . . . . . .
080: 00 00 00 00 00 00 00 00 54 77 61 69 6e 20 44 61
080: . . . . . . . . T w a i n D a
... (other lines deleted for clarity) ...
Each line of the dump displays the values of 16 bytes of "raw" memory.
The number at the beginning of each line (before the colon) is the line's
offset from the start of the struct. Each set of 16 bytes is printed twice
-- first as hexadecimal integers, and then as
ASCII characters.
The colors (assigned manually) serve to delimit adjacent members of the struct,
and the underlined part is the struct TW_VERSION embedded within
struct TW_IDENTITY (see Win32Twain.java).
The location and extent of each member in the struct's memory space
is determined from the declaration order and size of each member.
Looking at the dump above, it is obvious that the information returned by the native code
is correct (remember that the PC's Intel processor is
little endian). Specifically,
the values of SupportedGroups and Manufacturer are also correct
in the dump:
ProtocolMajor(the 2 magenta bytes at offset 46) = 0x01ProtocolMinor(the 2 cyan bytes at offset 48) = 0x09SupportedGroups(the 4 magenta bytes at offset 50) = 0x0003Manufacturer(the 34 cyan bytes at offset 54) = "Microsoft" (padded out to 34 bytes with 0-valued bytes)
Comparison of the values from the printf() statements and those
in the dump shows that JNA's sense of struct-member location has, mysteriously,
slipped by 2 bytes starting at SupportedGroups. This is the classic
symptom of a memory alignment issue.
The alignment error occurs because the native code strings together the values of
the struct's members without any intervening gaps, whereas the JNA code
expects to find them at memory offsets that are multiples of the member's length.
Thus, the native code places SupportedGroups at offset 50,
but JNA looks for it at offset 52 (a multiple of 4, the size of
SupportedGroups). The struct members following
SupportedGroups also get pushed back by 2 bytes, leading to the
corruption of Manufacturer's value shown above. You should now also
be able to explain how the "Tw" creeps in at the end of Manufacturer's value.
Finally, a short digression on another aspect of pointers: the code of
dump() shows how Structure.getPointer() can be used to
get a pointer to the beginning of a struct. The com.sun.jna.Pointer
object returned by getPointer() can be used to access
the struct as an array of bytes (a C programmer's void*).
Reproducing the Struct Alignment Error
The file JTwain.java actually contains the code with the memory alignment error so that readers may explore this further if they wish. But the TWAIN demo program still works correctly as it does not use the values in the struct.
To reproduce the memory alignment error compile the program as described at
"Running the Sample Code" below, then execute JTwainDemo.bat.
You should see the window titled "JTwain Demo" in Figure 3 below. At the menu bar
select "File" -> "Select Source..." as shown in the figure. The window titled
"Select Source" will pop up with a list of the installed TWAIN devices.
Choose any TWAIN device, and click the button labelled "Select".
This executes the code with the alignment error, and displays the
contents of the struct TW_VERSION in the command window.

Figure 3. Running JTwainDemo
Note that the struct TW_VERSION contents you see will likely differ
from the example values shown above (unless you have the same TWAIN device installed).
But you should be able to see the same kind of evidence of a memory alignment problem.
If the pop-up window titled "Select Source" displays no TWAIN devices, you should download and install the TWAIN developer toolkit. The toolkit simulates an image source (the first entry in the "Select Source" window in Figure 3 above) that returns an image of the TWAIN logo.
Preventing Struct Alignment Errors
Native libraries come in various memory alignment flavors (because of differences between compilers and compiler options). So, since JNA is typically used in situations where re-compiling the native code is not an option, it has facilities for setting the alignment strategy used.
The alignment strategy for members of a Java class that extend Structure
can be set by invoking Structure.setAlignType(int alignType) method. There are
four options for alignment type as described in the table below.
| Alignment Specification | JNA Description |
ALIGN_DEFAULT | Use the platform default alignment. |
ALIGN_GNUC | validated for 32-bit x86 linux/gcc; align field size, max 4 bytes |
ALIGN_MSVC | validated for w32/msvc; align on field size |
ALIGN_NONE | No alignment, place all fields on nearest 1-byte boundary |
The output from dump() shown above makes it clear that the TWAIN native code
uses no particular alignment strategy (ALIGN_NONE in the table
above). But since this is not also JNA's default setting, all of the Java classes
that substitute C structs have a default constructor that sets alignment type to
ALIGN_NONE (see Win32Twain.java). The following code is an
abbreviated view of the Java class for struct TW_IDENTITY with the
default constructor.
public class TW_IDENTITY extends Structure {
public TW_IDENTITY() {
setAlignType(Structure.ALIGN_NONE);
}
public int Id;
public TW_VERSION Version = new TW_VERSION();
public short ProtocolMajor;
public short ProtocolMinor;
. . .
}
In general, there is no way of knowing the alignment strategy used by any particular native library. So, if a DLL's documentation does not specify this information some experimentation will be required to determine the correct alignment setting to use.
Running the Sample Code
To run the sample code described in this article proceed as follows:
- Download the zip containing the sample code, and extract it into a directory (say, samples)
- Open a command window, and use the "CD" command to navigate to the samples\code directory.
- Execute the batch file build.bat. This compiles all of the code (and is required to be run just once). The class files are located in a directory called samples\bin.
- To run a program execute the batch file with the same name (e.g. LockWorkStation.bat, BeepMorse.bat, GetLogicalDrives.bat, GetSystemTime.bat, GetVolumeInformation.bat, or JTwainDemo.bat)
The samples zip contains jna.jar, so you don't have to download anything else. The batch files listed above also have the classpath specified, so you don't have to change anything to compile and run the sample code.
Resources
- Sample code for this article
- JNA project site
- Windows API Reference
- Windows Data Types
- No More Pointers
- Wikipedia Pointers Page
- Wikipedia "Windows Programming/Handles and Data Types"
- C Strings
- Java Tech: Acquire Images with TWAIN and SANE, Part 1
- TWAIN Developer's Toolkit
- TWAIN Header File
Excerpts from the JavaOne 2009 MIDP 3.0 In Depth: Tutorials and Demonstrations session with Roger Riggs, Lakshmi Dontamsetti and Stan Kao.
Right-click or Control-click to download this MP3 file. You can also subscribe to the Java Mobility Podcast Feed to get the latest podcast automatically. If you use iTunes you can open iTunes and subscribe with this link: Java Mobility Podcast in iTunes.
View all Java Mobility Podcasts.
I recently interviewed Java Champion Adam Bien, asking him about Java EE 6 and some other currently prominent issues in Java technology. Here's our discussion.
Editor: The Java EE 6 platform has been approved. What are the most significant features in the newly approved platform for Java developers?
Adam: Java EE 5 was the revolution - Java EE 6 is the evolution. JSF 2.0 is a significant step in the right direction. Introduction of annotations, easy creation of components, and integration with facelets are huge news. You can create a JSF 2.0 application in minutes without having sophisticated tools.
EJB 3.1 / REST synergy is very interesting and the Context and Dependency Injection JSR-299 / JSR-330 marriage greatly extends the DI capabilities of the platform. Now even a spec led by the head of Spring (Rod Johnson) is a part of the Java EE 6 spec.
Editor: What do you make of the final vote, where several JCP committee members voted against the approval (Apache) or abstained, due to licensing concerns? Is this significant for Java developers? For companies that develop Java EE solutions?
Adam: Apache votes every spec with "no" and the same comment. This is nothing new. SAP seems to be concerned about the full licensing terms for the TCK. The question is whether Sun didn't manage to provide the licensing terms, or didn't intend to do so. The majority of the appserver vendors voted with "yes" and SAP had no objection about the technology or programming model. From my perspective the future of Java EE 6 is very bright.
Java EE 5 is supported already by 14 vendors - a huge adoption story. Java EE 5/6 are the only vendor-neutral component model. You are not only able to run your application on different application servers without any modification, but you can only pick and choose the right licensing and support model. Java EE 5/6 are very portable and the "ancient" J2EE 1.X apps can be also very easily be migrated to Java EE 6.
Editor: Along with the Java EE 6 platform approval, several related component technology specifications were also approved. Which of these do you consider most significant?
Adam: CDI - together with JSR-330. Now Java EE 6 is a very capable Dependency Injection platform.
Editor: At DEVOXX, it was announced that closures will be included in Java 7. What's your view of this?
Adam: All Java SE APIs were developed without closures in mind already. It is actually too late. However: closures are huge news for application developers. They really can stream line the application code. I'm really looking forward to hack some closure code with JDK 1.7.
Editor: Do you agree that adding closures to Java is needed to enable Java to meet "the Multicore Challenge"?
Adam: I'm not sure about that. You can run perfectly scalable code right now with plain Java. Closures could make it more convenient - but it isn't impossible to write multicore code without them.
Editor: Since EJBs are now lightweight, can't EJBs, an established, proven, rock-solid, technology be effectively applied to desktop applications that run on multicore processors to deliver maximal performance? Thus, meeting the "Multicore Challenge" without changing the Java language?
Adam: EJBs are lightweight since 2006. They were always perfectly scalable on multicore systems because of their procedural nature. I actual never had any scalability problems with EJBs and was always surprised by their good performance.
Even more important: EJB 3.0/3.1 do follow the Convention Over Configuration / Configuration By Exception principle, what makes them the simplest possible choice for building distributed systems.
With the availability of the embeddable container in EJB 3.1 you could even run them on a desktop, or at least in a JUnit test. Glassfish EJB 3.1 container is about 1 MB, openEJB and JBoss are also very lightweight - it could really work.
Editor: Adam, thanks for providing your insight to the java.net community!
Adam: Thanks for the interview!
FeedTask class's newsfeed-polling
implementation. JavaFX 1.2 introduced many interesting APIs, including APIs for reading RSS and Atom newsfeeds. If you haven't worked with these APIs, you'll discover that they greatly simplify the task of integrating a newsfeed reader into a JavaFX application.
This article introduces you to the RSS and Atom APIs. You first explore their common foundation, and then tour each API's key
classes. Finally, you gain insight into how these APIs work by exploring the FeedTask class's newsfeed-polling
implementation.
Common Foundation
The RSS and Atom APIs are offshoots of a common foundation that's rooted in the abstract javafx.async.Task
class. This class makes it possible to start, stop, and track an activity (task) that runs on a background thread.
Task provides onStart and onDone variables that identify functions to be invoked at
the start/end of the task, and other variables that report task progress and disposition (success or failure). This class
also provides abstract start(): Void and stop(): Void functions to initiate and terminate task
execution.
The abstract javafx.data.feed.FeedTask class extends Task. In addition to inheriting
Task's variables, and overriding its start() and stop() functions,
FeedTask provides the following functions and variables:
-
poll(): Void: Poll the newsfeedlocationfor updated content, which is fetched, parsed, and delivered to the application. -
update(): Void: Poll the newsfeedlocation. All content is fetched, parsed, and delivered to the application. -
headers(of typejavafx.io.http.HttpHeader[]) identifies a sequence of HTTP request headers that are to be sent tolocationeach time this newfeed is polled. This variable defaults tonull. -
interval(of typejavafx.lang.Duration) specifies the amount of time that must elapse before the newsfeed is once more polled for updates. You must specify a positive value for this variable, which defaults to0.0. (I wonder if it wouldn't be better to choose a positive value, such as60s, to be the polling default, and perhaps allow0.0to indicate that polling isn't desired.) -
location(of typeString) specifies the newsfeed's address. This variable defaults to the empty string (""). -
onException(of typefunction(:Exception):Void) identifies a function that's invoked when an exception occurs during the current poll. This variable defaults tonull. -
onForeignEvent(of typefunction(:javafx.data.pull.Event):Void) identifies a function that's invoked to handle extension elements, which are newsfeed elements whose namespace URI is not Atom or RSS. For example, given an Atom newsfeed whose feed element's start tag is specified as<feed xmlns="http://www.w3.org/2005/Atom" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">, parsing a subsequent<opensearch:totalResults>1911</opensearch:totalResults>element results in three foreign events (for the start tag, text, and end tag) because the namespace fortotalResultsishttp://a9.com/-/spec/opensearch/1.1/(as specified by theopensearch:prefix) instead ofhttp://www.w3.org/2005/Atom. This variable defaults tonull.
The common foundation is also rooted in the abstract javafx.data.feed.Base class, which is the base class for
RSS and Atom classes that describe various newsfeed elements. RSS's RSS and Atom's Feed top-level
element classes are examples of Base subclasses.
Base provides a namespaces variable (of type javafx.data.Pair[]) that contains the
namespace definitions in effect for the element. The name member of each Pair specifies the
namespace prefix; the value member specifies the namespace URI.
Base also provides a parent variable (of type Base) that identifies the parent
(enclosing) element. For example, the parent variable of Atom's Entry element class refers to its
containing Feed instance. If there's no parent (as is the case with Feed), this variable contains
null.
Finally, Base provides several functions that are useful when you need to create a custom feed parser. Because
this task is beyond the scope of this article, I refer you to Rakesh Menon's Custom Feed Parsers
blog post for more information and an example.
RSS API Overview
The RSS (Resource Description Framework Site Summary, Really Simple Syndication, Rich Site Summary)
API consists of 10 classes that are located in the javafx.data.feed.rss package. Central to this package is the
RssTask class.
| RSS versions supported by the API |
|---|
| The RSS API handles newsfeeds that conform to versions 0.91 (with non-optional item elements) through 2.0.11 (the most recent version at time of writing) of the RSS specification. |
The RssTask entry-point class extends FeedTask, and provides the following variables for installing
a custom factory, for reporting the newsfeed's channel element's non-item content, and for
reporting the content of each of the channel element's item elements:
-
factory(of typeFactory) identifies the factory that's used to create objects that represent newsfeed elements. You only need to install your own factory when creating a custom feed parser. -
onChannel(of typefunction(:Channel):Void) identifies a function that's invoked to report the channel element's non-item elements -- the RSS channel element contains item and non-item elements, and is itself contained within the top-level rss element. This variable defaults tonull. -
onItem(of typefunction(:Item):Void) identifies a function that's invoked to report the current item element. This variable defaults tonull.
The Channel class extends the abstract RSS class, which represents the top-level
rss element, and which provides members for accessing the factory that's creating objects, for accessing the
task that's parsing the newsfeed, and more. In turn, RSS extends Base.
Channel also provides the following variables for accessing channel-oriented
(non-item-specific) content:
-
categories(of typeCategory[]) identifies the categories (in terms of domains and text values) to which this channel belongs. -
copyright(of typeString) specifies a copyright notice for channel content. -
description(of typeString) presents a phrase or sentence that describes this channel. -
docs(of typeString) specifies a URL that points to documentation for the format used in the RSS file. This might simply be a pointer to a Web page, and is useful for letting people, who encounter this RSS file in the future, understand the file's purpose (much like code comments). -
generator(of typeString) identifies the program that was used to generate this channel. -
image(of typeImage) identifies an image (in terms of description, height, link, title, URL, and width) that can be displayed with the channel content. -
language(of typeString) identifies the language in which the channel was written. -
lastBuildDate(of typejavafx.date.DateTime) specifies when this channel's content was last changed. -
link(of typeString) provides the URL to the Website that corresponds to this channel. -
pubDate(of typeDateTime) identifies the date when this channel was published. -
title(of typeString) provides this channel's title. -
ttl(of typeDuration) provides the number of minutes in which the news-reader can cache this channel before it must poll the newsfeed to refresh channel content.
| Unsupported channel elements |
|---|
For whatever reason, the RSS API doesn't support the channel element's cloud,
textInput, skipHours, and skipDays elements. These elements are not
represented by javafx.data.xml.QName constants in the RSS class, and they are not represented by
variables in the Channel class.
|
As with Channel, the Item class, which describes one of the channel's
item elements, extends RSS. It provides the following variables:
-
author(of typeString) provides the email address of this item's author. -
categories(of typeCategory[]) identifies the categories to which this item belongs. -
comments(of typeString) specifies the URL of a Web page containing comments about this item. -
description(of typeString) provides a description of this item. -
enclosure(of typeEnclosure) describes a media object (in terms of length, MIME type, and URL) that's attached to this item. -
guid(of typeGuid) specifies, for this item, a globally unique identifier (in terms of text and an indicator of whether or not this text permanently points to the full item described by this item). -
link(of typeString) provides this item's URL. -
pubDate(of typeDateTime) identifies the date when this item was published. -
source(of typeSource) identifies the originating channel (in terms of the name of the channel and an XMLization of that channel) for this item. -
title(of typeString) provides this item's title.
I've created a NetBeans RSSDemo project whose Main.fx source code demonstrates RssTask
in terms of its interval, location, onStart, onChannel,
onItem, onException, onForeignEvent, and onDone variables.
/*
* Main.fx
*/
package rssdemo;
import java.lang.Exception;
import javafx.data.feed.rss.Channel;
import javafx.data.feed.rss.Item;
import javafx.data.feed.rss.RssTask;
import javafx.data.pull.Event;
def MAX_POLLS = 3;
var counter = 0;
def task:RssTask = RssTask
{
interval: 15s
// The following location demonstrates a basic RSS newsfeed.
location: "http://javajeff.mb.ca/rss/javajeff.xml"
// The following location demonstrates onException().
// location: "http://developers.sun.com/rss/sdn_features.xml"
// The following location demonstrates onForeignEvent().
// location: "http://feeds.dzone.com/javalobby/frontpage?format=xml"
// The following location demonstrates IllegalArgumentException (must use
// AtomTask for Atom feeds).
// location: "http://feeds.sophos.com/en/atom1_0-sophos-company-news.xml"
onStart: function (): Void
{
println ("Task is starting");
if (++counter > MAX_POLLS)
{
task.stop ();
FX.exit ()
}
}
onChannel: function (c: Channel): Void
{
println ("Channel: {c}")
}
onItem: function (i: Item): Void
{
println ("Item: {i}")
}
onException: function (e: Exception): Void
{
println ("Exception: {e}");
task.stop ();
FX.exit ()
}
onForeignEvent: function (e: Event): Void
{
println ("Event: {e}")
}
onDone: function (): Void
{
println ("Completed poll #{counter}")
}
}
task.start ()
The source code introduces a constant that specifies the maximum number of times to poll the newsfeed, and a variable that counts the number of polls that have been made so far. The idea is to limit the number of times the newsfeed is polled so that the application won't run indefinitely.
After invoking the RssTask instance's start() function, which starts the newsfeed-polling
operation, the newsfeed located at the address assigned to location is polled every 15 seconds. The
onStart() callback is invoked at the start of each poll.
This callback tests to see if the counter has exceeded the maximum number of polls. If so, stop() is invoked to
stop the polling, and FX.exit() is invoked to kill the background thread that's associated with the
RssTask instance, allowing the application to exit.
Perhaps you're wondering why I placed if (++counter > MAX_POLLS) in onStart(), as opposed to
onDone's callback. I did this because onDone() isn't always called at the end of each poll. (You'll
discover why this happens later in the article.)
It's possible that an exception might be thrown as a result of the newsfeed being read or parsed. If this happens, the
onException() callback invokes stop() to stop the polling task, and then invokes
FX.exit() to kill the background thread and terminate the application.
This simple framework serves as a starting point for exploring the RSS API. As an exercise, expand onChannel()
and onItem() to output the values of their Channel and Item arguments' various
variables.
Atom API Overview
In contrast to RSS, the Atom API consists of 12 classes that are located in the
javafx.data.feed.atom package. Central to this package is the AtomTask class.
| Atom versions supported by the API |
|---|
| The Atom API handles newsfeeds that conform to version 1.0 (the most recent version at time of writing) of the Atom specification. |
The AtomTask entry-point class extends FeedTask, and provides the following variables for
installing a custom factory, for reporting the newsfeed's feed element's non-entry content,
and for reporting the content of each of the feed element's entry elements:
-
factory(of typeFactory) identifies the factory that's used to create objects that represent newsfeed elements. You only need to install your own factory when creating a custom feed parser. -
onFeed(of typefunction(:Feed):Void) identifies a function that's invoked to report the feed element's non-entry elements -- the Atom feed element contains entry and non-entry elements, and is itself the top-level element. This variable defaults tonull. -
onEntry(of typefunction(:Entry):Void) identifies a function that's invoked to report the current entry element. This variable defaults tonull.
The Feed class extends the abstract Atom class (inheriting members for accessing the newsfeed's
base URI, for accessing the factory that's creating objects, and more), which extends Base.
Feed also provides the following variables for accessing feed-oriented
(non-entry-specific) content:
-
authors(of typePerson[]) identifies the authors (in terms of email address, name, additional person-specific text, and the Internationalized Resource Identifier (IRI) associated with the person) of this feed. -
categories(of typeCategory[]) identifies the categories (in terms of a human-readable label, category name, and categorization scheme IRI) to which this feed belongs. -
contributors(of typePerson[]) identifies the persons who have contributed to this feed. -
generator(of typeGenerator) identifies the program (in terms of human-readable name, program URI, and program version number) that was used to generate this feed. This information can be used to debug an Atom newsfeed. -
icon(of typeId) identifies this feed's iconic image (in terms of a URI to the image). -
id(of typeId) specifies a universally unique and a permanent identifier (in terms of a URI) for this feed. -
links(of typeLink[]) specifies links (in terms of href, hreflang, length, rel, title, and type XML attributes, and text associated with the link) from this feed to Web resources. -
logo(of typeId) identifies this feed's non-iconic image. -
rights(of typeContent) specifies the rights (in terms of src, text, and type XML attributes) held in and over this feed. -
subtitle(of typeContent) provides this feed's subtitle. -
title(of typeContent) provides this feed's title. -
updated(of typeDate) specifies when this feed's content was last changed.
As with Feed, the Entry class, which describes one of the feed's
entry elements, extends Atom. In addition to sharing most of the same variables as
Feed, Entry provides the following unique variables:
-
content(of typeContent) specifies this entry's content. -
published(of typeDate) specifies when this entry was published. -
source(of typeFeed) identifies this entry's feed source. -
summary(of typeContent) specifies a short summary, abstract, or excerpt for this entry.
I've created an AtomDemo NetBeans project for demonstrating AtomTask. This project's
Main.fx source code is very similar to RSSDemo's Main.fx source code.
/*
* Main.fx
*/
package atomdemo;
import java.lang.Exception;
import javafx.data.feed.atom.AtomTask;
import javafx.data.feed.atom.Entry;
import javafx.data.feed.atom.Feed;
import javafx.data.pull.Event;
def MAX_POLLS = 3;
var counter = 0;
def task:AtomTask = AtomTask
{
interval: 15s
// The following location demonstrates a basic Atom newsfeed.
location: "http://photos.dailycamera.com/hack/feed.mg?Type=gallery&Data;=9573834_9ysrR&format;=atom10"
// The following location demonstrates onForeignEvent().
// location: "http://blogsearch.google.com/blogsearch/feeds?bc_lang=en&hl;=en&output;=atom"
// The following location demonstrates IllegalArgumentException (must use
// RssTask for RSS feeds).
// location: "http://javajeff.mb.ca/rss/javajeff.xml"
onStart: function (): Void
{
println ("Task is starting");
if (++counter > MAX_POLLS)
{
task.stop ();
FX.exit ()
}
}
onFeed: function (f: Feed): Void
{
println ("Feed: {f}")
}
onEntry: function (e: Entry): Void
{
println ("Entry: {e}")
}
onException: function (e: Exception): Void
{
println ("Exception: {e}");
task.stop ();
FX.exit ()
}
onForeignEvent: function (e: Event): Void
{
println ("Event: {e}")
}
onDone: function (): Void
{
println ("Completed poll #{counter}")
}
}
task.start ()
This simple framework serves as a starting point for exploring the Atom API. Consider expanding onFeed() and
onEntry() to output the values of their Feed and Entry arguments' various variables.
Behind the Scenes with FeedTask
The important task of polling an RSS or Atom newsfeed occurs in FeedTask and a related class. I recently
decompiled these classes to explore how newsfeeds are polled, and share my findings in this section to deepen your
understanding of RssTask and AtomTask.
FeedTask creates an instance of the java.util.Timer class in its static initializer. This instance
starts a background thread and works with an instance of FeedTask's nested SubscriptionTask class
(a java.util.TimerTask subclass) to support newsfeed-polling.
FeedTask's overridden start() function schedules the SubscriptionTask instance for
execution by invoking Timer's public void schedule(TimerTask task, long delay, long period) method
with the following arguments:
-
The
SubscriptionTaskinstance is passed totask. -
The long integer
0Lis passed todelay. -
The value of
FeedTask'sintervalvariable is passed toperiod.
Approximately every period milliseconds, the SubscriptionTask instance's public void
run() method is invoked. This method invokes the SubscriptionTask-specific doPoll() method
with a true argument.
The doPoll() method first clears FeedTask's inherited started, stopped,
failed, and done Boolean variables to false. It also nulls out the inherited
causeOfFailure variable, and assigns -1 to the inherited progress and
maxProgress variables.
doPoll() next instantiates the javafx.io.http.HttpRequest class, which is the vehicle used to
obtain newsfeed content, and initializes the following HttpRequest variables prior to executing this task:
-
location: The value ofFeedTask'slocationvariable is assigned to this variable. -
onStarted: A function is assigned to this variable, and is invoked when the request starts to execute. The function is responsible for invokingonStart(). -
onResponseHeaders: A function is assigned to this variable to retrieve and save the values of the HTTP ETag and Last-Modified response headers. These values are needed to ensure that only changed newsfeed content will be returned in the next poll request. -
onToRead: A function is assigned to this variable to obtain the total number of bytes to read, which is assigned tomaxProgress. -
onRead: A function is assigned to this variable to obtain the number of bytes read so far, which is assigned toprogress. -
onInput: A function is assigned to this variable to parse the request content via an internalparse(is)method call (whereisisonInput()'sjava.io.InputStreamargument). If parsing results in a thrown exception, the exception object is assigned tocauseOfFailure,trueis assigned tofailed, andonException()is invoked. Finally,trueis assigned todone, andonDone()is invoked. (TheonInput()function isn't invoked, and henceonDone()isn't invoked, when only changed content is requested but that content isn't available.) -
onException: A function is assigned to this variable to report a problem with the request itself (and not parsing). If the request fails, the exception object is assigned tocauseOfFailure,trueis assigned tofailed, andonException()is invoked.
Continuing, doPoll() ensures that only updated newsfeed content is returned by setting the request's
If-Modified-Since and If-None-Match headers to the previously saved
Last-Modified and ETag values, respectively.
| Obtaining a newsfeed's updated versus entire content |
|---|
When true is passed to doPoll(), which happens when this method is called from
SubscriptionTask's run() method or FeedTask's poll() function,
doPoll() sets If-Modified-Since and If-None-Match so that only updated content
is returned. In contrast, when you invoke FeedTask's update() function, which invokes
doPoll() with a false argument, those request headers will not be set, and the entire content will
be returned.
|
doPoll() now iterates over FeedTask's headers variable, assigning each stored
HttpHeader instance to the HttpRequest instance by invoking the latter instance's
setHeader() function.
Finally, doPoll() invokes the HttpRequest instance's start() function to execute this
task, resulting in retrieved and parsed content. doPoll() then returns to the run() method. If it
throws an exception, run() invokes onException().
| A parsing tidbit |
|---|
For brevity, I don't discuss parsing beyond the parse(is) method call. However, if you decide to explore the
parsing implementation, here's a tidbit to save you some head-scratching: The parse(InputStream) method
initializes the javafx.data.pull.PullParser instance's impl_skippedElements variable to the
qualified names of Atom's summary, content, rights,
title, and subtitle elements, and RSS's description,
title, and copyright elements, to ensure that the parser treats any HTML or other markup
that's embedded in these elements as literal text.
|
At some point, you'll probably invoke FeedTask's overridden stop() function. This function invokes
the SubscriptionTask instance's inherited public boolean cancel() method to cancel the
newsfeed-polling task (but not kill the Timer instance's background thread).
Conclusion
Enough theory! Now that you've gained knowledge of JavaFX's RSS and Atom APIs, you might want to create your own newsfeed reader. To help you with this task, I present a practical example that handles RSS and Atom newsfeeds in my forthcoming companion to this article.
Resources
- Sample code for this article
- Atom specification
- JavaFX 1.2.1 API Documentation
- JavaFX's official Website
- Rakesh Menon's Custom Feed Parsers blog post
- RSS specification
- Wikipedia's Atom (standard) entry
- Wikipedia's RSS entry
Excerpts from the JavaOne 2009 MIDP 3.0 session with Angus Huang, Roger Riggs, and Paul Su.
Right-click or Control-click to download this MP3 file. You can also subscribe to the Java Mobility Podcast Feed to get the latest podcast automatically. If you use iTunes you can open iTunes and subscribe with this link: Java Mobility Podcast in iTunes.
View all Java Mobility Podcasts.
One of the key concepts in the software industry is that of separation of concerns: the only way to reliably build any type of complex system is to break it down into small, simple, and focused components that each perform a specific function. Performing only a specific function makes each component easier to understand, develop, unit test, reuse, and maintain. You see strict adherence of separation of concerns within the standard three tier Java web application design that has come to dominate Java software development. It is common practice to use a web model-view-controller (MVC) framework in the presentation/web tier (such as Spring Web MVC or Struts2), a middleware framework in the middle tier (such as Spring), and an object-relational mapping (ORM) tool in the data tier (such as Hibernate or any JPA implementation).
The whole point of using frameworks and tools is to free yourself from the complex technical challenges and boiler plate code you would otherwise be forced to write on your own and allow you to be more focused and productive while developing your application. But because frameworks and tools handle the hard parts (and are also reused in a wide range of situations), they tend to be more technically complex than the application you are using them to create. The result is that frameworks and tools typically demand much more emphasis on architecture and design, and developers working on frameworks and tools tend to be much more diligent in incorporating design best practices into their software. Once again, this frees the application developer to enjoy the benefits of having to "just" design and build an application that takes maximum advantage of all of the frameworks and tools and ensure that they all integrate and interact well not only with each other but with the application's custom business logic.
And therein lies the design gap: while the creators of frameworks and tools must invest a great deal of attention and focus on design, application developers can afford to be more lax. A classic example of the design gap is what I call controller bloat. In the case of web applications, controller bloat occurs when application developers violate separation of concerns and put non-web service code directly into their web MVC framework's controllers.
When an object receives a request, the object can choose to either perform the request itself or to delegate the request to a second object which will do the work. The industry naming of this second object varies, but I refer to it as the service delegate. This term reinforces this second object's role as a delegate, and also underscores the fact that it contains service level business logic.
This article discusses using a service delegate in combination with your MVC framework for a design that avoids controller bloat. A background with the MVC pattern is assumed, and although the examples use Java and Spring Web MVC to apply the design to a web application, the concepts apply to any language as well as to any MVC framework. The terms service, service code, service layer, and service delegate are used interchangeably to refer to your custom business logic.
The problem: MVC Controller Bloat
Using Spring Web MVC as an example, let's look at a sample of a typical web MVC controller.
public class TypicalParameterizableViewController extends ParameterizableViewController
{
private UserService userService;
protected ModelAndView handleRequestInternal(HttpServletRequest request,
HttpServletResponse response) throws Exception
{
Map model = new HashMap();
String userId = request.getParameter("id");
User user = userService.findById(userId);
user.setLastAccessTime(new Date());
userService.persist(user);
if (user.isAccountExpirationWithin90Days())
{
userService.sendAccountExpirationWarningEmail();
}
// Dispatch to the view
String viewName = getViewName();
ModelAndView modelAndView = new ModelAndView(viewName, model);
return modelAndView;
}
}
While the above code sample is functional, it does exhibit a major design problem: service code (i.e., your business logic) is unnecessarily located within the controller itself. The majority of the code within the handleRequestInternal method has nothing to do with the web tier and therefore violates separation of concerns. This leads to a handful of cascading disadvantages. Because the code lives in a class directly depending on the web tier, it can't be easily reused within non-web-based applications. Next, just as the code is difficult to reuse outside of a web-based application, it is difficult to unit test. While there is a plethora of options such as using mock objects or sophisticated strategies such as hot deploying code to running application servers and automated remote unit tests, that fact that your service coded is coupled to your presentation tier complicates both its testing and its reuse.
All of these problems stem from the same root cause: there is no reason to have the service code located in the controller. In my opinion, a controller should be as "thin" as possible, and only contain code that directly relates to handling the incoming request, delegating all non-request/response processing to a service delegate that will generate the model objects for use in the view, and then create and return the outgoing response. Because MVC frameworks handle the first and third responsibilities for you, by relocating service code (the second responsibility) to a service delegate, controller bloat can be completely avoided. While most architects and developers agree with this design philosophy, in practice few actually incorporate it into their software.
A Solution: The ViewService
Knowing that we want our controllers to delegate processing to a service delegate, the first step in moving away from or preventing controller bloat is to create an interface to act as a behavioral contract for the interaction between the controller and the delegate.
public interface ViewService
{
public String REQUEST_PARAMETERS_KEY = "request-parameters";
public String REQUEST_QUERY_STRING_KEY = "request-query-string";
public void generateViewModel(Map model);
}
While it's easy to be initially underwhelmed, the ViewService's power and flexibility are derived from its simplicity. Because it does not contain any implementation details of either the presentation or service tier, it's able to act as an independent intermediary without allowing one layer to bleed into the other. As you'll see shortly, from the controller perspective, all the controller knows is that it will invoke this method with a Map as an argument, and that once the invocation is complete the results will be available in the Map. From the service perspective, all the service class knows is that this method will be invoked with a Map of input objects, that it should use these input objects to generate whatever data the implementing class is responsible for, and then return the results through the same Map. By agreeing to pass only a Map between layers, each side is fully encapsulated and has no knowledge of how the other side works.
This one, simple, single line interface leads to many tangible benefits.
The use of the ViewService interface completely decouples your code from your selection of which MVC framework you use. Your MVC selection no longer needs to be a strategic or organization-wide decision because switching, or even using a separate MVC framework per application, is now trivial. Each application you create now has the freedom and flexibility to use the best framework for the given requirements and technical circumstances.
Going even further, note that a class implementing the ViewService can now be reused in any type of application: web (servlet or portal based), batch, fat client, EJB, non-EJB, and remoting applications such as web service or JMS based software are now all possibilities. This level of reuse will allow you to follow the Don't Repeat Yourself (DRY) principal, and because you are programming to interfaces, an industry best practice, your presentation and service tiers can now be built in parallel by multiple developers. Unit testing of your service code is significantly less complicated because it can be exercised through straightforward unit tests interacting directly with your service layer. This greatly simplifies testing, makes it easier to isolate issues, and saves you time.
It doesn't scream for your attention, but another subtle benefit is that this design makes it easier for new team members or less experienced developers to join and contribute to your project. For a multitude of reasons, not all developers can jump head first into an existing system and make substantial contributions right away. By having your layers and your complexity cleanly separated, you can have a particular developer work on an individual area where they are strongest or most comfortable, while they gain experience with or knowledge of a different part of the system.
With the ViewService in place, the next step is to incorporate it into your application. The following code sample shows a Spring Web MVC Controller that, through the use of a support class, delegates to an underlying ViewService implementation.
public class ViewServiceParameterizableViewController extends ParameterizableViewController
{
private ViewService viewService;
protected ModelAndView handleRequestInternal(HttpServletRequest request,
HttpServletResponse response) throws Exception
{
Map model = HttpViewServiceSupport.processRequest(request, response, viewService);
String viewName = getViewName();
ModelAndView modelAndView = new ModelAndView(viewName, model);
return modelAndView;
}
...
}
As with the ViewService, it's easy to be underwhelmed by the controller's simplicity. But once again, the controller's power and flexibility is derived from its simplicity. Holding true to the intent and vision of the design, the controller is as thin as practically possible. It's so thin, as you can see, that creating controllers for any other MVC framework is trivial and quick work.
Now that we've created an ultra thin controller, let's take a look at the HttpViewServiceSupport class that contains the core functionality.
public class HttpViewServiceSupport
{
public static Map processRequest(HttpServletRequest request,
HttpServletResponse response,
ViewService viewService)
{
Map model = new HashMap();
buildModelFromRequest(request, model);
viewService.generateViewModel(model);
return model;
}
public static void buildModelFromRequest(HttpServletRequest request, Map model)
{
String queryString = request.getQueryString();
model.put(ViewService.REQUEST_QUERY_STRING_KEY, queryString);
processParameters(request, model);
processAttributes(request, model);
processCookies(request, model);
processHeaders(request, model);
}
public static void processParameters(HttpServletRequest request, Map model)
{
Enumeration parameterNames = request.getParameterNames();
if (parameterNames.hasMoreElements())
{
Map parameters = new HashMap();
while (parameterNames.hasMoreElements())
{
String parameterName = (String) parameterNames.nextElement();
String values[] = request.getParameterValues(parameterName);
parameters.put(parameterName, values);
}
model.put(ViewService.REQUEST_PARAMETERS_KEY, parameters);
}
}
...
}
For any incoming request, processRequest()'s execution flow is composed of four steps. First, an empty Map is instantiated. As you know from earlier in the discussion, the central concept of the entire design is to pass only a Map between the various layers of the system (in our case the controller and the ViewService), and the very fact that this Map is just a plan old Java object (POJO) means that no layer is tightly coupled to any other. Second, the incoming request is transformed from it's current presentation-tier-specific format (in this case, an HttpServletRequest) into a more generic form. This is accomplished in buildModelFromRequest(), which extracts the interesting or useful information from the incoming request and places it into the previously instantiated Map. Third, the ViewService is invoked, and the underlying implementation performs its processing. Once processing is complete, the ViewService will store its results into the Map and thereby make the results available to any portion of the system from which the Map is accessible. Finally, with the ViewService's work complete,
the populated Map is simply returned to the invoking method. From there, it is ultimately returned to your MVC framework and, as you would expect, the normal processing flow of your MVC framework takes place.
Having seen an ultra thin controller as well as a MVC independent support class used to process the incoming request and invoke an underlying ViewService, let's next look at an example implementation of the ViewService.
public class UserLoginService implements ViewService
{
private UserService userService;
public static final String USER_KEY = "user";
public static final String USER_ID_KEY = "user-id";
public void generateViewModel(Map model)
{
String userId = (String) model.get(USER_ID_KEY);
User user = userService.findById(userId);
model.put(USER_KEY, user);
user.setLastAccessTime(new Date());
userService.persist(user);
if (user.isAccountExpirationWithin90Days())
{
userService.sendAccountExpirationWarningEmail();
}
Map parameterMap = (Map) model.get(ViewService.REQUEST_PARAMETERS_KEY);
String day[] = (String[]) parameterMap.get("day");
if (day != null)
{
model.put("day", day[0]);
}
}
...
}
You'll hopefully recognize most of the above code from our original and flawed example. What's new is the use of String constants as well known key names to store objects within the Map. Finally, here is a small JSP page to put it all together.
<%@ taglib prefix="c" uri="http://www.hdiv.org/jsp/jstl/core" %>
Welcome ${user.firstname}!
Your last login was ${user.lastAccessTime}.
<c:if test="${not empty day}">
Today is ${day}.
</c:if>
Using the same well known key names as provided by the ViewService implementation, JSTL was used to pull the model objects from the JSP context and dynamically populate the JSP page.
Conclusion
Separation of concerns is a central concept in the software industry. Controller bloat, such when a MVC controller violates separation of concerns and unnecessarily includes service code, leads to many significant problems. This article presented a reusable design in which a service delegate was used in combination with a MVC framework that allowed all of these problems to be avoided. While sample code demonstrating the design used Spring Web MVC for a Java based web application, the concepts presented universally apply to all types of applications as well as software created in any language for any platform. Because any time you emphasis good design, the end result is that your code will be easier to understand, develop, test, reuse, and maintain.
Resources
When overlaying lightweight Swing widgets on top of heavyweight AWT components, it is convenient to group the Swing widgets on a transparent (non-opaque) JPanel so that one can use Swing layout managers and/or Swing utilities on a group of related Swing widgets. For example, all widgets added to the transparent JPanel can be made visible or invisible simultaneously with one setVisible() call on the JPanel. Unfortunately, a lightweight JPanel overlaid on top of a heavyweight AWT component cannot be transparent.
To overcome this problem, one can implement a TransparentPanel as shown below.
public class TransparentPanel extends JPanel {
/**
* Default constructor. Sets the layout manager to JPanel's
* default layout manager: FlowLayout.
*/
public TransparentPanel() {
this(new FlowLayout());
}
/**
* Construct panel with input layout manager.
*
* @param mgr The layout manager for the extended JPanel.
*/
public TransparentPanel(LayoutManager mgr) {
// set layout manager, if any
super(mgr);
// set opaque false so that an isOpaque() == false
super.setOpaque(false);
// set mixing cutout -- see AWTUtilitesClass below
AWTUtilitiesClass.setMixingCutoutShape(this,
new Rectangle());
}
/**
* Override setOpaque so that the user cannot change the
* opacity of this panel, since it is after all suppose to
* be transparent.
*
* @param isOpaque Is this panel opaque?
*/
@Override
public void setOpaque(boolean isOpaque) {
// do not allow this to become opaque because it is
// transparent after all
}
}
Note that the constructor of TransparentPanel invokes AWTUtilitiesClass.setMixingCutoutShape(). Invoking AWTUtilitiesClass.setMixingCutoutShape() is necessary to avoid non-rendering of our TransparentPanel which would result in a solid gray (and not transparent!) TransparentPanel.
The AWTUtilitiesClass "mixing cutout" technique is described in Mixing Heavyweight and Lightweight Components, October 2009 by Sharon Zakhour and Anthony Petrov, whose code has been excerpted below:
public class AWTUtilitiesClass {
private static Method mSetComponentMixing;
static try {
Class awtUtilitiesClass =
Class.forName("com.sun.awt.AWTUtilities");
Method mSetComponentMixing =
awtUtilitiesClass.getMethod(
"setComponentMixingCutoutShape",
Component.class, Shape.class);
mSetComponentMixing.invoke(null, component, shape);
} catch (Execption ex) {
ex.printStackTrace();
}
public static void setMixingCutoutShape(Component c, Shape s)
{
if (mSetComponentMixing != null) {
try {
mSetComponentMixing.invoke( null, c, s );
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
}
Real World Example: Mixing Swing and Java WorldWind - Software vs Hardware Acceleration
When developing Java WorldWind applications, it is typical for the developer to overlay a JPanel (which contains Java Swing GUI components) above a WorldWind Java container (which contains the rendered world). The WorldWind Java container is usually (1) a GLJPanel which is a subclass of the lightweight Swing JPanel, or (2) a GLCanvas which is a subclass of the heavyweight AWT Canvas.
If Option 1, GLJPanel, is utilized, then the Java WorldWind developer (a) maintains the ability to overlay transparent Swing components above the world (that is, the GLJPanel). However, when attempting to turn on software acceleration, the developer will often find that doing so (b) creates run time exceptions that are difficult if not impossible to overcome, and (c) the performance gained is inferior to performance gains of hardware acceleration.
If Option 2, GLCanvas, is utilized, then the Java WorldWind developer (a) loses the ability to overlay transparent Swing components above the world (that is, the GLCanvas). However (b) runtime exceptions are not experienced, and (c) the performance gained is superior to software acceleration. Additionally, the TransparentPanel introduced in this article coupled with the "mixing cutout" from A. Petrov's article restores the ability for the Java WorldWind developer to maintain inter-component transparency, thus partially alleviating the problems of (a) and more readily allowing the Java WorldWind developer to combine lightweight Swing overlays (containing "grouped" Swing widgets) with heavyweight hardware acceleration.
Notes:
During development, when using Java Swing components rendered in an overlay above a Java WorldWind layer, a predictable gray flash (flicker) was detected. To alleviate this problem, one can utilize the flag:
-Dsun.awt.noerasebackground=true
If one wishes to override the paint() method of a TransparentPanel subclass such that the subclass uses Java2D to draw a Shape onto the superclass TransparentPanel, then the "mixing cutout" described above can cause the TransparentPanel and anything drawn on it to vanish. To resolve this, instead of passing the empty Shape via new Rectangle() to AWTUtilitiesClass.setMixingCutoutShape(), one should pass the shape of the desired drawing. See Mixing Heavyweight and Lightweight Components, October 2009.
If one wishes to overlay a non-rectangular lightweight component above a heavyweight component, one must bear in mind that Java Shapes (and thus "mixing cutouts") cannot be anti-aliased, so for non-rectangular Swing components, one must smooth the non-aliased "mixing cutouts" using hues or blurring.
Resources
- Mixing Heavyweight and Lightweight Components, October 2009 by Sharon Zakhour and Anthony Petrov, Sun Developer Network
- World Wind Java SDK, U.S. National Aeronautics and Space Administration








