Saturday, July 31, 2010

GlassFish Security

Pact Publishing was kind enough to send me a copy of the book "GlassFish Security" that was released very recently.


It's tough to find the time to read a book cover to cover. In fact, it's been a while since I've read a book from beginning to end. Typically I'm only interested in a few chapters which I then read. Later, when the need arises, I may get back to other chapter. It's like treating a book like an encyclopedia or a dictionary. I bet that most people read technical books that way nowadays. So it's important that a book lends itself to be read that way.

Security is a very broad field with many many different topics, e.g. user authentication and authorization in web applications, integration with an external security server, web services security, and so on. Fortunately, very few people will have to deal with all these aspects at the same time. GlassFish Security covers many if not all of those aspects, and that's another reason why it should be possible to read this book like an encyclopedia.

I went back in time and tried to remember all the times that I have had anything to do with security in GlassFish and checked what the book had to say about it. For instance, to start simple, there's the issue of configuring realms and users. There's the time I tried to get JSPWiki to work on GlassFish using declarative authentication and authorization, something that goes through server.policy. More advanced, there's the time we tried to integrate GlassFish with AccessManager. On all these topics, the book delivers. There are many other topics (e.g. integration with OpenSSO, securing JMX, etc), all described in detail without wasting much space on repetition or endless code or XML listings. The book would have saved me a lot of time had I had it at the time when I needed it.

If you're using GlassFish, this book belongs on your bookshelf!

Wednesday, April 28, 2010

Google TV Ads

On Tuesday I went to the Google office to talk about what I might be working on. It's likely going to be Google TV Ads. What is that? Here is a 3 minute video that explains it.

Monday, April 26, 2010

Working with tips and tricks for project owners

In the past two years I've managed the OpenESB open source project. This project is hosted on In this post I outline a number of tips and tricks I've learned while managing this project on This is useful information for my successor, but it also may be useful to other people who are managing projects on

Web presence

An important factor in the success of an open source project is its web presence: it will be the first point of contact between a potential new user and the project.

New visitors will want to see answers to these four items immediately:

  • What is it?
  • License?
  • Getting started
  • Downloads

It is important that the links to these pages are static, i.e. that they do not change. That is because a new visitor may enter the site through a link on a different site or through a search engine. For this very reason, it is especially important to keep a single downloads page rather than a page per version, so that users do end up immediately on the latest version, rather than a previous (old) version.

Looking at the front page of OpenESB, these four items are immediately apparent, and the links are permanent. Two more items are important: a visitor will judge a project for it being

  • Alive
    • Active mailing list
    • Recent/frequent announcements
  • Professional
    • Well organized, thought through
    • Documentation easily found and comprehensive
  • Inviting
    • New users should feel welcome to not just use the product, but also to participate in the community.

You be the judge how well these goals have been met. Let's look at the mechanics of now.

openesbsite project web site

How processes HTML files

The HTML files for a project's web site are stored in the project's VCS system in a directory called www. There's a big twist here: although the HTML files are ordinary HTML files, i.e. with an <HTML>, <HEAD> and <BODY> tag, the web server reads the HTML files and does a number of substitutions on the file before it writes the resulting file to the client.

Conceptually, there is a template HTML file which has its own <HEAD> element, its own <BODY> element with static content, and some placeholders in these two elements for:

  • some elements of <HEAD> element of the user defined HTML file, e.g. <TITLE>
  • the contents of the project_tools.html file
  • the contents of the <BODY> element from the user defined HTML file

As such, the project owner has to perform special tricks to have the page displayed in other ways than the standard way.

In OpenESB we have several projects. All of them should have the same look and feel, i.e. the OpenESB "brand" and a common navigation. The same goes for the OpenESB wiki.

How OpenESB organizes HTML files

Each project has the same project_tools.html file. This file takes care of the following:

  • It includes custom CSS files. These files override the styles.
  • It hides the standard navigation bar. For admins, the navbar display can be turned on: the state is maintained in a cookie. The value in the cookie can be toggled by loading the admin.html file in OpenESB:
  • Loads the menu. The menu is defined in a separate JavaScript file.
  • Displays the search box
  • Sets an event handler that is invoked when the page's loading is complete. The event handler manipulates the layout of the page and invokes Google Analytics.

As a result, the other HTML files in OpenESB or any of its sub projects don't have to bother with the common look and feel, the menu, etc. All of that is taken care of by project_tools.html, which as mentioned should be duplicated to all the OpenESB projects.

Some or if you will, many, HTML files in OpenESB have another common style element: a right hand side bar that displays common advertising for downloading GlassFish ESB, the wealth of components, and the a partner highlight. This right side bar is taken care of a separate JavaScript file. Files that should display this right hand side bar should explicitly be formatted to do so: it should contain a table in which the right column loads the JavaScript of the right hand side bar.

Previewing HTML

The fact that the look and feel of an HTML file is now defined in project_tools.html makes it difficult to see what a file will actually look like when deployed (i.e. checked in into the project's VCS) on There is a workaround: I created a directory that when deployed on a web server, emulates the environment. To edit a file with the ability to preview it without checking in the file, follow this procedure:

  1. Load the file in your browser through's web server, e.g.
  2. Save this file to the emulation directory, e.g. Tomcat/webapps/ROOT/Downloads.html
  3. Now load the file into your browser through your local web server, e.g. http://localhost:9080/Downloads.html. This should look identical to what was loaded from
  4. Locate the text in the file to be edited. Change the text and save the file. Reload the file in your browser. Repeat this step until the file looks OK.
  5. Do a diff using a tool like Araxis Merge, or Beyond Compare between the local file and the file that was checked in into VCS, e.g. diff Tomcat/webapps/ROOT/Downloads.html and open-esb/www/Downloads.html.
  6. Merge the changes into the VCS file, i.e. open-esb/www/Downloads.html. Save and check in into VCS.

Here is the directory for Tomcat. If you use it for projects other than OpenESB, you can remove many of the files. Make sure to leave branding and css directories in tact. If you intend to reuse project_tools.html, don't forget to change the Google Analytics account ID.


Unfortunately, uses SSL for all its files. This is unfortunate for two reasons:

  • Files do not get cached in the browser: every time a user opens his browser and goes to OpenESB, all the images, style sheets, etc are reloaded again.
  • To avoid security warnings in Internet Explorer, files that are included in an HTML file also should come through SSL. This rule is violated on the OpenESB main page (of all places!) where it loads a dynamic list of news item from a non-https server.

There's no workaround for this. Collabnet promised a long time ago that SSL would be removed but this still has not happened.

Changing the menu

The menu is defined in a json like format in a file called menu.js. This file is duplicated in the wiki. See below. Make sure to use absolute URLs because the file is used from different projects, i.e. different roots.

Wiki (not on

OpenESB is not using the wiki infrastructure. I don't recall why this decision was made. New projects should probably evaluate the wiki infrastructure before considering setting up a wiki elsewhere.

The OpenESB wiki is hosted on a machine hosted at Sun/Oracle, not at Collab net. It is collocated with other wikis, e.g. GlassFish, UpdateCenter, etc. The wiki engine is JSP wiki. The templates have been adapted to look and feel like the OpenESB web site.

Since has / used to have a lot of downtime, and to avoid having this downtime impact the wiki, the menu.js file is duplicated on the wiki. That uses ssl is another reason to duplicate the file. For the same reason, the icons displayed in the menu are also duplicated on the wiki server.

Users are managed separately from User management / restrictions are necessary to avoid spam. Spam has been a been big problem on the OpenESB wiki and the other wikis collocated with it. Cleaning up spam can be done by directly manipulating the text files that make up the wiki page.

JSPWiki security is configured to use JAAS. This fact is important if the wiki needs to be setup on a different GlassFish server. In that case make sure to update the server.policy file of GlassFish. The policy file specifies that only "validated" users can edit pages. Users are stored in a user database and a group database. The latter specifies group membership, of which the "validated" group is important. Both files are in the etc directory of the wiki directory.

To add a user, the group database needs to be updated. This can be done through the web interface, or can be done by editing the group file directly.

The raw data files and user management files of the wiki can be accessed through SSH.

Since the user management of the wiki is separate from, users can choose a different userid on the wiki than they do on Allowing this is a mistake in retrospect: editing the wiki requires a SCA (Sun Contributor Agreement) or employment in Sun/Oracle, and having to manage/audit two sets of users instead of one is a waste of time. Again, in retrospect.

openesbwiki user management

Anybody on can request developer privileges. When that happens, an email is sent by to the project owner. An automated process is setup that replies automatically to the user list asking the applicant for his motivation, background, etc. This reply is specified in role-approval.policy in the www directory.

For some reason, many people ask for developer privileges. They are not known, have not contributed or communicated before, and are never heard from again.

Consequently, this automated process sounds nice, but is useless in practice. A better process is that someone first communicates on the mailing list, proves he/she is serious and able to contribute, after which the project owner grants privileges. Before privileges are granted, a signed SCA needs to be on file. This process is documented on the OpenESB web site.

The link to the membership management page is displayed in the nav bar. Recall that the nav bar is hidden. To toggle display of the nav bar, load Alternatively, jump to the page directly:

When a user is granted commit privileges to a project, he/she also automatically gets commit privileges to the code repositories of the sub projects. E.g. granting commit privileges to open-esb also gives commit privileges to open-jbi-components, a sub project of open-esb. This is only the case if the VCS of the child project is of the same type as the parent project. Since open-jbi-components and open-esb both use CVS, open-esb committers have access to open-jbi-components. However, if the VCS system is different, e.g. openesb-builds uses SVN, separate access to this project needs to be granted next to open-esb. email management

Email lists are managed by an email list manager running in This email list manager is only partially integrated in Users can click on a button on the site to subscribe to an email list. What effectively happens is that it will register the email address that is associated with that user at that moment in the list server. Similarly, a user can unsubscribe through a button, which will remove the email list associated with that user at that moment. Where this knowledge comes in handy? There is confusion if the user changes his email address associated with his id between subscribing and unsubscribing. Sometimes users get so confused that they'll need help unsubscribing from the list. After all the last thing you want is repeated posts of "how do I unsubscribe from this list?".

Here is another task with respect to email: spam. Protecting the list to spam is very important. Therefore lists should be setup as "discuss" or "moderated". A setting of "discuss" means that only "subscribed" or "allowed" posters can post. If someone posts a message who is not in the "subscribed" or "allowed" list, the message will be forwarded to the list owner(s) for review. Replying to that message will allow the message to be posted. To avoid getting too many of these "review messages", frequent posters can be added to the "allowed" list. The "allowed" list is accessible from the email management page.

The link to the email list management page is displayed in the nav bar. Recall that the nav bar is hidden. To toggle display of the nav bar, load Alternatively, jump to the page directly:

Nabble is a view on the mailing list. OpenESB has integrated it on the web site. Note that posters are identified by their "from" email address only, but posters on Nabble somehow do not need to be in the "allowed" of "subscribed" list. This is fortunate, but puzzling from a technical point of view.


This brings me to one more email list related tasks: once in a while, someone posts an email that he/she wants to revoke. For example, someone could accidentally send a private email to the list, or include confidential (customer information for example) information in an email.

Of course there's no such thing as undoing the sending of an email. Subscribed users will have received the email by the time that the sender notices the mistake. However, there is an option to remove the email from Nabble and from the mail browser. Note that there are two places where the email needs to be removed: both Nabble and Removing the mail there at least makes the mistake less visible and will make it harder for the mistaken email to be picked up by search engines. automation

Some of the management tasks on can be automated. Kohsuke has developed an extensive set of tools that provide a Java api to web interface:

I've written a few tools to make community management easier. One such tool is used to make an inventory of all the users in the community with commit privileges. Everybody in this list should either be a Sun/Oracle employee, or should have signed an SCA. The tool loads and parses the list of signatories (from and compares this list with the one extracted from Discrepancies are people who are currently or have been Sun/Oracle employees. Since it is difficult to track who is no longer an employee (especially with the layoffs that used to happen every now and then), the tool can email all "discrepancies" and ask them to either submit an SCA or to verify employment by clicking a link pointing to an internal server. These tools are available on

Wednesday, April 21, 2010

OpenESB under Oracle

It's been a few months since Oracle acquired Sun. At the time there were a lot of questions about what Oracle would do with OpenESB. On Feb 15, I posted a plan to the users mailing list. What was the plan? And how is the plan coming together? How is OpenESB doing a few months after the acquisition? What will the future hold for OpenESB?

First, let's try to understand Oracle's perspective. Oracle already has an integration product: the SOA Suite. Through the BEA acquisition it got another one, and through the Sun acquisition it got yet two more: CAPS and OpenESB. Of course there's no sense in keeping high levels of investments in all these products; it makes much more sense to focus on one product. What is this strategic product? Oracle is very clear about that: Oracle's strategic SOA middleware platform is the Oracle SOA Suite. Consequently, Oracle has reduced the level of investment in OpenESB.

That was the bad news. Now for the good news. Oracle could have just pulled out from OpenESB and let it fall into a black hole. Or worse, it could have taken the site down, removed the downloads, or put other obstacles in place. It did not. It did quite the contrary.

Although Sun was the main sponsor behind OpenESB, Oracle recognized that a lot of  people made investments into OpenESB, either in the community or into their own company by using OpenESB, and with a feeling of responsibility and fairness, Oracle decided to help the community to find a way to stand on its own, so that all these investments would bear fruit for as long a time as possible.

Did you invest in OpenESB? Let's look at what will change for you depending on the kind of investment you made.

What will change for you

What will change for you if you bought GlassFish ESB support? Nothing will change: you are still fully supported. The same Sun SOA Support department is still available to help on issues. Should patches be created to address issues, the full Sun SOA Sustaining department is still there to create patches.

Are you using GlassFish ESB but did not buy support? Unlike Sun, Oracle is no longer trying to sell GlassFish ESB licenses to new customers. Instead, you may rely on community support, or commercial support provided by one of the OpenESB community partners. The user mailing list is nowadays less frequented by Sun/Oracle engineers, but other community members have stepped up and are keeping the mailing list responsive. More about that below.

Are you investing in OpenESB by contributing code or other artifacts, or are considering to do so? You will find that it has become easier to contribute. I've put together a new governance document that gives greater freedom to community contributors. Oracle can still exert influence, but that is intended to keep the peace in the community should that be necessary. Overall, you will find it easier to propose and implement new changes, to commit code, and last but not least, to influence the roadmap. This brings us to the future of OpenESB.

The future of OpenESB

The future of OpenESB revolves around OpenESB becoming an open source community that can stand on its own, i.e. without Sun or Oracle as the single major sponsor. That transformation will of course not happen overnight, and Oracle is committed to help with this transformation. For instance, Oracle will do periodic builds and post these on the downloads site. Another commitment is that Oracle will merge patches that it makes for customers, back into the open source repository.

What did Oracle do so far? Next to the governance document I already mentioned, the sources of GlassFish ESB v2.2 can now be found in the open source repository. The HL7 BC and the WLM SE are also back in the open source repository, and the binaries can be downloaded from the downloads page. The process of an automatic periodic build has incurred some delays due to technical issues, but is well on its way.

OpenESB today

How is OpenESB doing today? Let's look at the users mailing list. From the Users mailing list on Markmail, it can be seen that the list activity has declined a bit, but is not much below the activity of that of a year ago.


What we also can learn from Markmail is that community members who are not on the Oracle payroll are stepping up. For example, here are the most active posters for March:


Another metric that we can look at is the number of users. In GlassFish ESB v2.1 we introduced a feature in which NetBeans checks for updates upon startup. By looking at the traffic to the updates-server, we can estimate how many users there are of GlassFish ESB. I defined the number of users as the number of NetBeans installations that ping the server at least three times in a time window greater than five days. As can be seen, the number of users is going up.


In terms of a roadmap there are no concrete proposals from the community yet, but several members have expressed interest in continuing with Fuji.

More good news: a few new committers have joined the project and are contributing code.

Moving on… next stop: Google

I've decided it's time to move on. Yesterday I put in my resignation at Oracle. I'll be starting at Google in two weeks.

Why? Was Oracle such a bad place to work? Was it such a culture shock for a Sun employee that it just could not work? Could I not stand my manager? Could I not get along with my new co-workers?

If you're expecting a bitching session about big-bad-Oracle, I have to disappoint you. In the months that I've been at Oracle, I've come to the conclusion that it is a pretty decent company to work for. I like my manager, my co-workers are intelligent and pleasant, and the culture at Oracle is very similar as it was at Sun. So, honestly, no complaints there.

The reason for my leaving is that I've been doing SOA for 7+ years now. I've been involved in the complete lifecycle of two products: CAPS and OpenESB. Now, with my transition into Oracle, I was about to commit to a third SOA product. Before I dove into that, I did some soul searching, and realized that I ready for something new, something in a different field.

As I was looking for a change, I looked at what interesting companies there are here in Orange County. Google and Amazon came to mind. Google is said to be the engineering Valhalla, so I submitted my resume there. And surprise, surprise, I got an offer.

It's a big jump. It's starting from the bottom again… jumping into something brand new. Will I like the change? I'm eager to find out!


Thursday, February 25, 2010

I got a patent

Today I got a pleasant surprise when I got into the office: I found a big box on my desk. It turns out it was a nice engraved plaque of my patent on "Protected Concurrent FIFO" message processing.


The funny thing is that I didn't even know that the patent was awarded. After the patent application in 2004 or 2005, I don't even remember when it was, I lost track of its status. Emails to the patent lawyer went unanswered.

I googled for the text of the actual patent (US7644197 Queue management by multiple processors) and found a PDF which I copied to Mediacast.

A thank-you to Jerry Waldorf (my boss during the SeeBeyond days and co-applicant of the patent) for asking me to work on this subject, and a thank-you to Sun for the plaque!

By the way, I have often said that almost all software patents are nonsense and inhibit progress rather than promote it. I still feel this way. Does that make me a hypocrite? Not necessarily: companies are forced to submit patents applications so that they can defend themselves against patent litigation from competitors. In that sense I'm happy that Sun got another patent. And that I'm happy that my name is on it: well, yes, I guess that's an ego-thing.

Friday, January 15, 2010

How to enlist and delist XA resources: trial and error

Once in a while a problem pops up with how a middleware stack supports XA. The problem I'm discussing here is the one of how to exactly enlist and delist XAResource-s with isSameRM()returning true. The XA spec is vague on this very topic, so this warrants some exploration.

First of all, what do I mean with "enlist and delist XAResource-s with isSameRM()returning true?"

Enlisting and delisting

When using an XA capable system, the connection needs to be enlisted in the transaction before it is used. A typical sequence on the XAResource, let's call it r1, is:

   1: r1.start(xid, TMNOFLAGS);
   2: ... // use (r1)
   3: r1.end(xid, TMSUCCESS);
   4: r1.commit(xid, true);

With a second resource in the mix, the call sequence may become:

   1: r1.start(xid1, TMNOFLAGS);
   2: ... // use r1
   3: r2.start(xid2, TMNOFLAGS);
   4: ... // use r2
   5: r2.end(xid2, TMSUCCESS);
   6: r1.end(xid1, TMSUCCESS);
   7: r1.prepare(xid1);
   8: r2.prepare(xid2);
   9: p1.commit(xid1, false);
  10: p2.commit(xid2, false);

The transaction is now escalated to a full two-phase transaction. But if r1 and r2 represent the same resource manager, a shortcut can be taken: one resource can "piggy back ride" on the other resource. The isSameRM() method can be used to check if this should be attempted. If it returns true, the sequence may become:

   2: r1.start(xid, TMNOFLAGS);
   3: r1.isSameRM(r2); // returns true
   4: r2.start(xid, TMJOIN);
   5: r2.end(xid, TMSUCCESS);
   6: r1.end(xid, TMSUCCESS);
   7: r1.commit(xid, true);

The transaction now again is a single-phase transaction, and the performance difference with the two-phase commit case is usually very significant.

This sequence doesn't work for a number of popular systems, e.g. MQSeries and Oracle. Unfortunately the XA specification, but that specification is not quite clear at all about what resource managers should be able to do, so there is to some extent some trial and error involved.

Trial and error

How should isSameRM() be used then? By testing a number of different systems, it turns out that for some systems, there can be only one enlisted XAResource active in the transaction. If a second XAResource joins the transaction, the first one should be deactivated. Here is an example:

   1: // SINGLE ACTIVE 1
   2: r1.start(xid, TMNOFLAGS);
   3: r1.isSameRM(r2); // should return true
   4: ... // use r1
   5: r1.end(xid, TMSUSPEND);
   7: r2.start(xid, TMJOIN);
   8: ... // use r2
   9: r2.end(xid, TMSUCCESS);
  11: r1.start(xid, TMRESUME);
  12: ... // use r1 again
  13: r1.end(xid, TMSUCESS);
  15: r1.commit(xid, true);
A variation of this is calling TMSUCCESS instead of TMSUSPEND. In that case TMJOIN should be called instead of TMRESUME:
   1: // SINGLE ACTIVE 2
   2: r1.start(xid, TMNOFLAGS);
   3: r1.isSameRM(r2); // should return true
   4: ... // use r1
   5: r1.end(xid, TMSUCCESS);
   7: r2.start(xid, TMJOIN);
   8: ... // use r2
   9: r2.end(xid, TMSUCCESS);
  11: r1.start(xid, TMJOIN);
  12: ... // use r1 again
  13: r1.end(xid, TMSUCESS);
  15: r1.commit(xid, true);
I ran these tests on a number of systems, and here is what I found:
  isSameRM() Test: multiple active Test: single active 1 Test: single active 2
STCMS yes yes yes yes
JMQ as of 4.4 yes no: throws on line 7 yes
WebSphereMQ 6 yes no: blocks on line 4 no: blocks on line 7 yes
Derby yes no: blocks on line 4 yes yes
Oracle yes no: blocks on line 4 yes yes
MySQL 5.1 yes no: throws on line 4 no: throws on line 5 no: throws on line 7

As can be seen, the SINGLE ACTIVE 2 code sequence works best.

As a side note, MySQL is showing surprising behavior: TMJOIN and TMSUSPEND are not supported (as documented in the MySQL documentation), so why does it bother to return true on isSameRM()? Behavior like this makes it difficult to write portable code: a container now has to provide a wrapper around the DataSource that corrects for this. It would have been much better if it simply had returned false on isSameRM().

How does this relate to what application code can do in for instance a Java EE container? That's the topic of a different blog post.

Tuesday, January 12, 2010

Regex editor now on

By popular demand, the sources of the Regular Expression Editor I discussed in a previous post, is now available on

Book "SOA with Java": Rough cut now online


A few months ago I got a request from Satadru Roy, one of the authors of the book "SOA with Java", to write a section on Open Source ESBs for this book.

This was a good opportunity to advertise OpenESB, so I eagerly said yes. Satadru suggested that a practical example should be the core of the chapter, so I sat down with two of my colleagues, Murali Pottlapelli and Sujit Biswas, to discuss how the example would look like.

The result is a 17 page chapter that gives an overview of OpenESB. The book is now available for review on Safari. This link will probably be invalidated when the review period ends, but perhaps the link to the book itself is more permanent (the book is also on Amazon).