john-boyer

As the world wide web transforms from a vehicle of information dissemination and e-commerce transactions
into a writable nexus of human collaboration, the Web 2.0 technologies at the forefront of the tranformation
may be seen as special cases of a more general shift in the conceptual application model of the web.

The classic 3-tier model has the web browser and OS on a thin client tier, the traditional server tier
offerings for data persistence and workflow or business process managment, and a fat middle tier
of custom JEE code. The custom middle tier code has two responsibilities. It has to invoke all
the server tier API calls necessary to perform business transactions, persist data and the like.
But frankly, the server tier has received work for decades, so its scalable, robust and has well-defined
easy to use APIs. So, the bulk of the work in today's web applications consists of back-filling for the
underpowered thin client tier. To achieve a single business transaction, we have to do a complicated
juggling act of the multiple pages, some only conditionally presented, each collecting some small amount
of information that has to be validated, and the results aggregated until the transaction is ready.

The Web 2.0 application model is essentially a 2-tier model. It was a little hard to recognize that this
transition was happening in the early days when Web 2.0 just meant blogs and wikis. At that time, Web 2.0
just meant content democratization, which is the idea that non-technical users could create and
share content on the web without needing to involve web application programmers. This was quite
rightly viewed as a revolutionary breakthrough because it did for the web what the word processor did for
the personal computer. But if you remember that it takes two points to make a line, and then look at
at the history of the personal computer, it's trajectory as a revolution did not become evident
until the second killer application of computing, the spreadsheet. This application created what we call
process democratization. This is the idea that non-technical users, or semi-programmers,
could define how computing power would be dispensed to solve problems without needing to involve high-end
application programmers. This was revolutionary because the non-technical users who were enabled to define
their own processes were closer to the business problems and also far more numerous. The same thing is
now happening on the web, and it is the next phase of Web 2.0.

The Web 2.0 server tier is smarter than it used to be, exposing its capabilities directly to the
Web 2.0 client tier with web services, REST services, or feeds and ATOM publishing.
Similarly, the Web 2.0 client tier can connect directly to these services without needing the JEE
web developer in the middle. Through AJAX presentational libraries like Dojo
and AJAX data interaction libraries like Ubiquity XForms,
the client is smart enough to drive not just simple user interfaces, but data validation, conditional user interface presentation,
and access to web services throughout the end-user fill experience. It is even smart enough to offer webtop design and
deployment experiences for these applications.

In this regard, I would like to draw your attention to the most exciting product addition in the
new Lotus Forms 3.5 release. The new product,
Lotus Forms Turbo, provides non-technical users with a web experience for the
design and deployment of data collection forms and reporting functions.
To get an idea of how the non-technical user is enabled, see the new
YouTube videos on Lotus Forms Turbo.
If a picture is worth a thousand words, these videos are priceless, so I'll stop writing because when you see, you'll know!

For over 15 years now, the solutions built with the software products from
the IBM Lotus Forms team have been based on a simplifying system architecture
depicted below. This architecture uses a sophisticated, intelligent XML document
as the unit of information that flows among collaborators in a business process.

To support such high-value intelligent interaction, the document has a number
of layers as depicted below. These layers have different responsibilities.
Foundationally, the document format is an XML format, which means it is based on the
standard that allows introspection of the document to inject or extract information.
This also means the full power of XML signatures can be brought to bear to meet
archival and security requirements, particularly non-repudiation of the transaction
effected by the business process based on the document. The XForms layer allows
representation of a rich interaction layer for collection of structured data content.

Our work on Lotus Forms has been focused on the collection of structured content
needed to drive a transaction through a business process, and this has included
defining the XML vocabulary called XFDL to allow application developers to build the
high precision user interface needed to drive interaction with end-users.

But consider for a moment the great number of situational applications that are
constantly being created and deployed around the globe that are based on the
requirement to have a mix of structured data and unstructured content. In other words, consider the case of
office documents with high business value. We're talking purchase contracts, supplier
agreements, insurance polices, RFP responses, financial arrangements, and the like.
These are documents that can make effective use of multipage free-flowing unstructured
content and graphics interspersed with fill-in-the-blanks content for names,
dates, addresses, monetary values, milestones and so forth.

These documents support an important class of live, interactive applications in
which multiple collaborators work together to solve a semi-structured content problem.
The OASIS Technical Committee for the Open Document Format, in their wisdom, incorporated
the XForms model into ODF, and this delightful feat of standards reuse allows us to
reap the benefits of ODF's unstructured content creation features together with
the interactive structured data collection and both the data and document submission
facilities of XForms 1.1 at the same time. The layered document picture looks
pretty much the same, except as you can see below the on-the-glass presentation
layer is ODF, and otherwise the overall business process system architecture is
the same as depicted above.

The result of these considerations is a focus on
Interactive Office Documents
as a strategic means of going Beyond Office, i.e. going beyond the traditional usage patterns
for office documents that have become entrenched due to the office document limitations
of the past and instead solving the real business problems that cause people to select the use
of office documents in the first place. Using XForms 1.1, office documents based on ODF can be
first class participants in collaborative business processes.

I'm just leaving Sao Paulo after attending the ACM Symposium on Document Engineering. Great conference (people and technical content). I'd like to say more about a number of the other papers, but I'll focus here on the one that won this year's Best Paper Award: Enabling Adaptive Time-based Web Applications with SMIL State by Jack Jansen and Dick Bulterman.

The paper presents a very interesting mix of SMIL, XForms and other technologies to enable end-users to customize their experience of content where time dictates the major structure. One example given is of a video bike tour of Amsterdam created by the first author. Fragments of the video content can be identified and tagged with keywords. During run-time, an XForm is used to allow an end-user to select from the available keywords those which they find interesting, and the corresponding fragments of video are played.

This is the type of thinking that will clearly help make video content a first class citizen on the web. There is significant potential for the use of this technology for stream-lining educative experiences as well. Imagine, for example, a 2-hour audiovisual presentation that provides a thorough introduction to a topic, say CSS. But suppose the end-user really needs to know only about setting up borders on tables. By selecting the content, the user can find out what they need to know in 10 minutes. What's more, it is easy to imagine how the technology reported by Jansen and Bulterman could be refined to attenuate the video content download to the selected fragments.

A second example reported in [1] pertains to ad selection within video content. When you download free video for your favorite entertainment, you're going to get ads and you won't have the ability to skip them because otherwise how do people manage to make a living providing you with this free entertainment?? So, sprinkled through the video will be ad slots with default advertisements lined up. But, in addition there will be hotspots on the video where various product icons may appears from time to time, and if the user clicks the icon, then the default ad is replaced with an ad corresponding to the clicked icon. Thus, you could find out what's new at the electronics shop in lieu of an ad for laundry soap or some such.

It's a great example on the technical side of the need for interactive video, but this is because there's huge market potential for this idea. Not only does the user get a more pleasant ad experience with their free content, but the content provider gets to find out what their audience is most interested in, and the advertisers get to find out what content their customers are most interested in. Very Web 2.0.

I encourage you to download and read the paper.

At the specification level, the W3C Forms working group is working on modularization of XForms as one means of promoting incremental adoption. This is following the simple business axiom that "embrace and extend" is economically preferable to "rip and replace". If you have existing web applications, and you want to start incorporating selected features of XForms to get at certain benefits, it helps to be able to get at just those features without having to change your entire application over to XForms in one development cycle. Modularization supports agile development.

The Forms working group is now also working on XForms 1.2, even as XForms 1.1 continues toward full W3C Recommendation status. In XForms 1.2, the second approach the Forms working group is using to promote incremental adoption is to stream-line web application authoring through emphasis on attribute decoration and exploitation of UI element nesting as a means of expressing the simpler classes of MVC applications. The attributes are attached directly to UI elements, but they imply a proper XForms MVC architecture. In other words, the XForms processor can read the attributes and auto-generate a proper model for you. To get an idea of what the stream-lined vocabulary might look like, and what canonical XForms it might correspond to, look here.

The approach of providing stream-lined syntax is simpatico with modularization for two reasons. First, it means we can provide the stream-lined vocabulary and processing model as a module separately from modules that provide the components of the core XForms engine (e.g. the instance data module, the submission module, and so forth). Second, the stream-lined syntax approach means that authors can start with attribute decoration and UI nesting, and then scale up in more complex functional areas as the need arises by adopting the core XForms modules appropriate to the problem at hand.

The stream-lined syntax approach is also interesting because techniques such as attribute decoration, UI nesting, and progressive elaboration of a document are techniques familiar to AJAX developers and made popular by AJAX libraries such as Dojo. In fact, I am really coming to the conclusion that AJAX libraries are a viable alternative to browser-maker adoption as a means of delivering the most web-centric components of the W3C technology stack. It should not be necessary for Microsoft, Mozilla, Apple, Opera and other browser makers to build a W3C technology into their browsers before it becomes widely adopted and deployed. More importantly, this means that the W3C, not the browser makers, defines the web and its standards.

The XForms team finished our face-to-face meeting in Amsterdam last week. A major focus of the work on XForms 1.2 is called modularization.

The rationale for this work is the observation that the set of XML data processing problems which may have first arisen in the electronic form space are really more generally applicable to the XML data processing needs of RIAs and web applications. On the other hand, those who hear the word "form" may think they do not have a form problem because they still think of a "form" as a simple or static application like those for ordering pizza or flowers. However, XForms has solved numerous problems that keep coming up again and again throughout the W3C standards stack as well as the web application stack. By modularizing the components of XForms, we believe we can increase adoption of the components in other technologies which may not have need of all aspects of XForms.

The current view of the XForms 1.2 modularization can be viewed here. As an example of the rationale above, consider an application that may want to use the submission capability from XForms in a regular web application. The application would import the instance data module, but it might have an application-specific way of populating a data instance with data. The submission module would reference a data instance for the upload data and another instance for the submission results. An application-specific method would then be invoked to consume the submission result into the application, but the means of invoking that method could be an event handler for the xforms-submit-done event. The key issue here is that submission could be consumed by a non-XForms application without needing to incorporate the XForms recalculation engine, user interface controls and so forth.

The full elaboration of this modularization will allow applications to consume pieces of XForms incrementally, including the notion of data validation, data relevance, declarative data calculations, event-drive action scripting, repeats, switches, groups, basic user interface controls, and of course submissions.

Within IBM, there is an "Extreme Blue for Employees" program called Biztech, which promotes innovative development within IBM by funding special projects developed by teams that can come from all across IBM. Today, the 6th Biztech Showcase is being held at the IBM T.J. Watson Research Lab, Yorktown. It consists of 12 projects including ours.

Our project, which will be presented this afternoon, is called Dual Forms. To understand the basic business problem, suppose you have a complex office document such as a contract, a patent filing or a new supplier agreement. It will have many sections to fill out, some with precise information like names, supply codes, dates, milestones, deadlines, and payment amounts. These are things that need to be validated in order to prevent errors, or which may need to match with data stored elsewhere. However, other parts of the document will contain free flowing text, such as special terms and conditions. The document may also contain pie charts, bar graphs, or other rich content that serves as visual aids. The key here is that the document is complicated and hard to complete, which frustrates those who must collaborate to complete and results in costly delays in completion.

Now imagine you had a technology that could augment the office document completion experience with:

• wizard behaviors that take the user step-by-step through the fill process and help correct errors immediately
• automatically invoke web services of an SOA to obtain data during the fill experience, rather than making the user responsible to type in matching data.
• digital signature security so that the legally binding agreement can be made electronically rather than having to print, sign, fax, scan and store document images.
• automatic submission of completed documents to the back-end workflow system or the repository of executed legal agreements.

This is the domain of Dual Forms. The Lotus Forms technology provides the above benefits as well as the ability to enclose any other files within a Lotus Form. One such enclosure can be an ODF office document. Lotus Symphony is an implementation that runs under the Lotus Notes application platform as well as the underlying Lotus Expeditor application platform. In this project, we've created a Lotus Notes composite application that allows the Lotus Forms Viewer to run under Notes, including read/write access to the running form. The composite application also hosts the Lotus Symphony ODF editors at the same time.

Using this project, the end-user can get access within a single application to the capabilities of office documents AND the capabilities of Lotus Forms (wizards, help, web services, signatures, document submission to the business process). The underlying technology that allows the two rich document formats to interoperate is... XForms.

Today at the IMPACT 2008 conference, IBM made an announcement covered by the press about its new Web 2.0 Portal Software capabilities. That announcement included the following content from the Lotus Forms team:

"With diverse data sources and inefficient, time consuming processes needed to turn data into useful information, more companies are turning to web-based forms. In response, IBM announced today a new lightweight entry-level product code-named TotalForms which the company plans to ship in beta in June."

"TotalForms is an easy-to-use version of IBM's Lotus Forms software that will enable nontechnical people to quickly create, publish and route electronic forms submitted via the web. It can be used for a variety of tasks including customer satisfaction surveys, job applications and product orders. Based on an open and scalable nonproprietary software platform and Web 2.0 technology, TotalForms will integrate with IBM WebSphere Portal to provide one familiar interface for customers."

So you know, Total Forms, will offer not just web forms based on XForms, but also a web-based design experience for forms. More importantly, though, the value of Total Forms is that it provides a server-side repository for completed form data as well as listing and reporting functions. This means that users will have a point-and-click gestural design experience and run-time capability for the full, end-to-end Web 2.0 application. This puts the computing power into the hands of the people who have the problems that computing power can solve.

I'd like to elaborate further, and in concrete terms, a blog post from late last year on Talking to C-Level Execs about XForms. This will also make more sense of the XForms is a killer app of Web 2.0 messaging you got from me in this post.

For a long time now, web applications have had to use middle tier coding to backfill for the underpowered client tier and its inability to talk directly to server tier applications like DB2 that provide the persistent storage for the web. The Web 2.0 movement generally is about defining web applications that factor out the middle tier, allowing users of client applications to create and collaborate on content that Web 2.0 server applications natively understand how to store and serve.

Applications like XForms and DB2's PureXML form an instance of this movement. The server tier (DB2) moves itself closer to consumability by the Web 2.0 client tier by providing the ability to expose web services that give direct access to database table operations. Meanwhile, the conceptual client tier (XForms) now contains a comprehensive interactivity layer to drive a rich front-end user experience as well as a communications layer that allows the client to speak directly to web services. The net result is that a large class of important applications are enabled without the need for middle tier programmers.

To learn more about this union of technologies, including reference customer information, have a look at the new Tax Solution Education Kit.

Check out this cool video that will help you understand how and why your business processes need to lose the paper and get on board with dynamic, interactive electronic forms... powered by the XForms standard of course!

The Lotus Forms team will be out in force next week at Lotusphere 2008. To me, the most exciting presentations will be on a new product in development currently codenamed Total Forms. You can expect it to be shown in the Portal vignette during the OGS (opening general session), but there is also a focused session at 11:15 on the 23rd in SW Pelican. I don't want to spoil any surprises, so I won't say too much more, except it's a revolutionary new way to build XForms-based applications that really does put the computing power into the hands of the people. So go see the next killer app of Web 2.0.

Turns out the 23rd is a busy day for the forms team. At 10, there's a session on server-side mashups. Basically, we're talking new tooling that makes the Web 2.0 server accessible to non-coders. Again, putting computing power into the hands of the people, it's for people who want the next step up from Total Forms in application creation capability.

On the 23rd again, at 4:15, treat yourself to the latest in composite application development based on Lotus Forms under Expeditor and Portal. This talk will tell you, for example, how to get your portal and expeditor applications to push data into and extract data from live running Lotus Forms. This capability is brought to you buy the new Javascript APIs in our 3.0 release, which were added for tighter integration between Filenet applications and Lotus Forms.

OK, so now I should take a step back because I started out talking about the 23rd because of Total Forms, but we actually also have a fine session coming on the 22nd at 4:15pm to tell you about all the cool new stuff in our latest 3.0 release.

And on the 24th at 8:30 sharp, there's a session devoted to the integration of Lotus Forms, Portal and Process Server. This session gives you what it takes for the full meal deal of forms applications, those involving human centered workflows that allow people to collaborate on the business processes they are involved in every day.

And rounding out the Lotus Forms development team's showing at Lotusphere is the "Meet the Developer" pedestal, where you can talk one on one with some of our best software architects, development team leads, and even a red-headed senior manager whose technical gifts are the very antithesis of Dilbert's pointy-haired boss!

When I read the material on Yahoo! Blueprint, I was pretty pleased and shared the links with you in the last entry. However, I'd like to draw your attention to the first paragraph of their roadmap (emphasis is mine):

Declarative vs. Imperative

Much of Blueprint's philosophy and syntax comes from XForms. We opted for
a full declarative language because it was the only way we could
effectively run on the wide range of devices out there, some of which have
no scripting at all. By using declarative syntax, we can encapsulate and
hide the scripting specifics. In some cases, the code could run on the
phone, in other case, such as XHTML, we can put the logic on our servers.
It's the perfect way to deal with the various environments and their capabilities.

In a nutshell, this is the multimodality story of XForms.

I should clarify, though, that this is not saying that XForms has no imperative scripting of its own. The difference, though, is that the XForms script commands are data-centric, which means that they essentially declare what has to happen to the data in response to a particular event (like pressing a button to add a row to a purchase order). The meaning of each script command in XForms is tailored by the declarative constructs that bind to the data. For example, when you add a node of data containing the subtree of nodes needed to represent a purchase order item, the declarative bindings of XForms automatically create the UI controls and the line calculation formula needed to drive that new data. XForms processors, including Blueprint, are able to decide where these behavioral updates go depending on what devices are at play.

This issue of declarative constructs tailoring the meaning of imperative script commands is extremely powerful. Using the same one line of code to insert or delete a node, the rest of the application may respond by creating or destroying an arbitrary amount of user interface, including nested tables, as well as creating, destroying or updating any formulae that may be needed for the data being created or destroyed. The same one command that removes a row of a purchase order could also remove the entire list of delinquent payments for a customer who has just paid up.

In conclusion, it's not so much "declarative versus imperative". Rather, it's a matter of being much more effective with a hybrid model in which the imperative is data-centric and augmented by the declarative.

How much more effective, you ask? In The Mythical Man Month, Brookes states empirical results showing a relationship between complexity and code length of N^1.5, where N is the number of lines of code. To put that in concrete terms, 10 times less code is not just 10 times easier to maintain, it's about 30 times easier. This is the kind of difference that turns years into months or months into days. For the services company this means lower RFP bids, which translates into winning more deals. And it means significantly lower TCO for the enterprise IT shop building that web-app in-house.

After the Candidate Recommendation announcement for XForms 1.1 at the end of last year, the public implementation report period now begins in earnest. To celebrate the candidate recommendation, today you can now access a 25-minute podcast (MP3) I made to explain to you What's new in XForms 1.1 and why you should care.

And now for a very interesting news item. You know how good it feels to see a prediction you made come true? Well, that's me right now, and oh is it sweet. For some time now, I've been telling you about the new wave Web 2.0 client tier based on technologies like XForms. We deliver that vision to today's to web browsers using either a server that boils things down to the "assembly language of the web" or a client plugin. Either way the web is not solely defined by web browser vendors. Well, at the XML conference in December 2006, I saw it clear as day that the search companies were going to play an important role in demonstrating this. What'd'ya know, the first thing this year and Yahoo! has announced its strategy to foster a mobile ecosystem and deliver the best mobile internet experience to billions of consumers. The underlying technology for delivering on this strategy is the Yahoo! Mobile Developer Platform, which includes XForms. Google can't be far behind!

It was a great year for XForms at XML 2007. There were quite a number of presentations that featured XForms as a component, though in this blog I want to focus on the big event for XForms, which of course was the "XForms Everywhere" special session held on Monday evening.

It is no exaggeration to say it was an unqualified success. This was a 2-hour event consisting of six 15 minute presentations by some of the XForms community leaders, including Mark Birbeck, John Boyer (yours truly), Erik Bruchez, Dan McCreary, Keith Wells, and Charles Wiecha. This was followed by a 30 minute keynote from Elliotte Rusty Harold on "How XForms can Win". You can find program details here.

Naturally, we were a bit concerned that the evening slot (7:30-9:30) would test the stamina of the even most eager conference-goers, but the room was filled to capacity (about 60 chairs) and then there wasn't even standing room due to about 20 people standing or sitting at the back of the room.

And then the content was exquisite. The format of the session turned out to be a great idea. With only 15 minutes, each of us was required to refine our content to the cream of the cream. The order of speakers was defined by Leigh Klotz (Xerox) based on the most natural flow of thought about forms design, run-time case studies, general application architectures, and futures. Since I was talking about design, I went first. This created a happy coincidence because it made the most sense to start by announcing the most recent accomplishment the Forms Working Group, which is the transition to Candidate Recommendation of XForms 1.1. Not only did my own design-time presentation depend upon XForms 1.1, but also most of the other presenters used XForms 1.1 features as well.

With such a huge turnout, it occurred to me that quite a large fraction of people might not know much of what XForms is capable of. I started thinking back to the Compound Documents workshop of 2004 where Bert Bos, the chair of the CSS working group, said "Forms, I know what forms are. Name, address, pepperoni, extra cheese. I want to talk about more sophisticated web applications. The kind that can play games. Not necessarily Doom, but you know what I mean." When my turn came to talk at that conference, I started by launching my BlackJack form, and I played a few hands. Lady Luck was with me that day, and she was with me again on the XForms evening at XML 2007. I showed that form for effect, but then I showed a much more sophisticated "Mortgage Pre-approval Form" in both English and Chinese. Next I showed how much of the complexity of that form could be seen as simple aggregation of the concepts one can find in a reasonably small purchase order form. And at last, I launched into a 7 minute demonstration the XForms design experience for that purchase order form. You can see a video clip of that design experience here.

I enjoyed the presentations by Dan McCreary and Keith Wells because they demonstrated sophisticated applications of XForms that would be an order of magnitude harder to build with existing web application technology. During the breaks between speakers, I took the time to reiterate to the audience the business value here in terms of being able to win deals based on lower RFP bids.

I also enjoyed the two more forward-looking presentations by Mark Birbeck and Charlie Wiecha. Mark demonstrated and spoke about his experiences building a desktop application development environment called Sidewinder, which combines XForms and XHTML to allow very efficient creation of desktop applications, not just web applications. Charlie demonstrated and spoke about the applicability of the XForms architecture to creating robust mash-up applications out of components that actually are reusable.

Perhaps my overall favorite talk of the evening was given by Erik Bruchez. You can find out more about it here. Erik presented yet another application with high business value and low development effort that he created by combining XForms with the eXist database. The main point of his overall presentation was to illustrate that the XForms-based web application architecture consists of a rich client tier that can speak directly to server tier applications like databases with no coding in the middle tier. Clearly given my blog entry of Oct. 18, I couldn't agree more.

Whenever I explain that the XForms architecture is "rich client", I always stop to remind the reader that this is a mindset, not a deployment strategy. You can deploy an XForms rich client, or you can deploy XForms functionality to a thin client (a web browser only) using a server product such as the Lotus Forms Webform Server. The point is that the XForms mindset allows you to factor out the deployment issue and focus on what the application is supposed to do.

And as an IBMer, I am duty bound to my colleagues over in Information Management to point out that the database involved in the above mentioned example could instead be the scalable and robust DB2 9.x with pureXML support. Using a package like Lotus Forms that supports the XForms 1.1 submission enhancements for web services and REST-like services, forms applications can directly consume web services set up by the database adminstrator without the need to bring in a java developer to write the code in the middle. This is an important point that I'll return to very shortly in the conclusion.

The final presentation was the keynote by Elliotte Rusty Harold, who is a gifted speaker and thinker. He began by explaining how hard it has been over the past 15 years to make do with today's web architecture for building web apps. He was more eloquent at saying this, but it's a bit like trying to incrementally build a 15 storey building on the foundation for a single family dwelling. Of course, the point was that XForms can do the job that today's web just can't, and it can do it really well, but... As Elliotte explains it, XForms is currently a "Cambridge" technology built by really smart people, and that it needs to build the bridge to New Jersey where all the fast-and-cheap, quick-and-dirty... and successful... technologies live.

This is a fancy way of saying that while XForms is now hardened for the most demanding of enterprise applications, more focus is needed on making XForms more accessible to the much wider audience of people who perhaps start out with simpler problems that work their way up into being the complicated beasts that become those unmaintainable piles of jumble-fuss (my word; Elliotte would've come up with a better one) that cause so much trouble today.

XForms can win, Elliotte says, if three things happen. First, native support in the browser. Second, good design tools. Third, XForms needs a killer app.

Regarding browser support, he points out that it is well-known that IE will be hard, but that we should start out by getting the Firefox/Mozilla XForms plugin to be moved to the core browser, and not an 'additional' plugin anymore. To some extent, I agree, but in the spirit of New Jersey thinking, XForms can already run in all browsers because we have server-side products that can boil it down to the HTML and AJAX that all browsers, including IE, natively understand. Elliotte felt that was a bit of a hack, but we have to remember that he's also a professor, so he sometimes also falls into the Cambridge trap. Still, the point is well-taken, as long as we view things in terms of transition. The server products show that XForms is a viable technology for all browsers even if you don't want to deploy rich clients. A great next step, of course, to garner direct browser support, and we're working on it, but it's not impeding our ability to provide the business value of XForms to customers today.

Regarding design tools, well, it turns out that the reason I demonstrated the Lotus Forms Designer is because I agree with Elliotte on the importance of the role of good design tools for accelerating XForms development. Actually, I believe XForms is at a deeper level. Good design tools are actually critical for accelerating web application development. Since XForms simplifies web application development, of course it also enables the creation of more flexible and robust design tools for web application development. Indeed, XForms easily enables design constructs that would otherwise be impossible to create or brittle at best given only a purely imperative language like Javascript.

Finally, there's Elliotte's point about needing a killer app. Let me start with a Cambridge thought: XForms doesn't need a killer app because XForms is the killer app. The first killer app of computing was the word processor; the second was the spreadsheet. In a conversation I had with Elliotte the next day, he commented that the spreadsheet was even more of a killer app because it had to move the coder out of the way in order to put the computing power in the hands of the people. Sound familiar? XForms is the killer app of Web 2.0. The New Jersey reality check is that it has to walk like a killer app and quack like a killer app in order to be a killer app. The spreadsheet only became the killer app once the people saw what they could do with it and how to do it. The 'what' and the 'how', that's what made the XForms + database presentation and the design tool presentation so important in our XForms special session at XML 2007. The call-to-arms is simple: make more what, make more how, ... make XForms Everywhere.

The World Wide Web Consortium, or W3C, produces the "standards" for all sorts of web technologies with which we are all familiar, including XML, XSLT, XML Schema, HTML, CSS, PNG, ..., and XForms. The W3C calls these things "Recommendations" because after due process and careful review and implementation, the World Wide Web Consortium recommends the technical specification for world-wide development and deployment. And this is what has happened to all the recommendations listed above.

But just as software goes through maintenance and refinement, so to do the specifications that describe the software. For example, XML 1.0 is itself currently at fourth edition, and XML Schema 1.0 is at second edition.

On October 29, 2007, the W3C approved the third Edition of XForms 1.0 as a Recommendation, superceding the second edition from March 2006 and the first edition from October 2003. This release of the specification is quite important to the XForms community as it represents over 300 refinements, clarifications and corrections that have arisen in the last 18 months. The W3C Forms Working Group has worked very hard these past 18 months on what I would call "enterprise hardening" of XForms, and the numerous differences produced by the group can be viewed in the second edition errata page as well as the third edition diff-marked version.

Next up from the W3C Forms Working Group is an expected transition of XForms 1.1 to Candidate Recommendation. Stay tuned for more on that...

A lot of talks about XForms are a bit technical in nature because the people who manufacture XForms processors tend to be technical people who understand the business value of XForms in terms like "model-view-controller" architectures, a superset of AJAX, and software engineering benefits like abstraction. But the C-level executive cares not about these things. It is important to connect them to what the C-level executive does care about.

The C-level exec is about efficiency, flexibility and accountability.

The CEO wants

• efficiency via reduced operating costs and decreased time to close deals
• flexibility to react to new business processes and changes of business partners
• accountability for control of expenditures and compliance with regulations

The CIO wants to achieve

• efficiency through end-to-end business process integration and ability to leverage business objects as IT assets
• flexibility through a malleable system architecture that can be rapidly reconfigured with replaceable, reusable components
• accountability through transaction record auditability

We need to speak about how XForms helps the organization to achieve better results along these metrics. This is where the global picture of what XForms does against the backdrop of the classic 3-tier architecture comes in handy.

The C-level exec is not up at night worrying about the server tier, where databases, content managers and workflow engines live. These may be expensive systems, but they are designed to be robust and highly scalable systems whose metrics relative to the size of the anticipated user base of a system are easier to quantify. In other words, they are low risk numbers that are easy to budget for.

The C-level exec is not as concerned about the client tier, where the browser and OS live, again because the metrics are easy and stable.

The C-level exec has the most uncertainty and risk at the middle tier. There are two parts to it. There is the part that uses APIs to talk to DBs, CMs and workflow engines to implement custom application logic as needed. This part is not as scary because we have reliable, robust, scalable, stable APIs, and because the components we're talking to are those reliable, robust, scalable, stable, expensive systems sold by companies with important people's ties to yank when there is a support problem. Then there's the scary, assembly-language-of-the-web-part of the middle tier. This is the part that has to juggle a dynamic, multistep end-user experience on the anything-goes client side, where the web browsers are free and you get what you pay for. The upfront cost of the thin client is low, but the time to market with new offerings is the most highly affected because a lot of unpredictable, time-consuming work lives here.

Enter XForms.

• It allows you to standardize and consolidate the end-user experience into a single business object that represents the overall transaction.
• It allows that business object to access the web services of an SOA as a natural part of the process of going from empty transaction data to completed transaction data.
• It fully represents the transaction and therefore can function as an integral part of the records needed for auditability.

As a final thought, it should be clear from the exponential scales of complexity and cost that the diagram above argues that XForms-based systems have business value through complexity containment, and that said value is rightly reflected in software product cost.

I am pleased to announce that IDEAlliance has now announced the XForms Evening at the XML Conference.

This event brings together a number of the leaders in the XForms community to present the business value of XForms, including design experience, end-to-end solution development, case studies and driving business value through Web 2.0 integration. The session culminates in a keynote address by Elliotte Rusty Harold, who offers his vision and advice on the future of XForms.

You can find out the details by going to the conference main page and then clicking the XForms Evening link, or just click here.

Lotus Forms 3.0 is now available. Click here for details and trial downloads.

Perhaps the biggest news overall is the performance enhancements to Web Form Server, which is a "Rich Internet Application" platform that provides the capabilities of Lotus Forms directly to web browsers without the need to install "rich client" software.

We have also added digital signature capability to Web Form Server. This is, of course, one case where a small amount of native code must be deployed by the server in order to perform the core cryptographic functions allowed by the end-user with their private key material. But this is a small and seamless download needed for digital signatures only. Again, no "rich client" deployment to maintain.

I am excited by the ease-of-authoring improvements in the Forms Designer, such as the Form Designer's enhanced datatype awareness. I am also very pleased with enhancements made to the many ease-of-authoring wizards, including the XForms-savvy Digital Signature wizard, the table calculation wizards, and the compute wizard. And the performance improvements made for large forms rival the improvements made to the run-times. No blog entry could do justice to the improvements to the designer; you just have to download the trial version and test drive it.

Perhaps the biggest overall cross-product change has been the performance improvement made available via a new language feature called on demand page loading. This feature is available in both the Web Form Server and the rich client Forms Viewer plugin (the latter of which is available for trial download). If you have multipage forms, this feature greatly reduces the time spent bringing up a form, and it avoids loading data structures for pages not visited by the end-user's navigation experience. This feature is combined with a new capability to do page-level validity checking on page-change operations so a user cannot progress through a wizard experience unless the validity criteria for that page are met.

The biggest overall language change goes to the support of bidirectional text and components for Hebrew and Arabic locales. This is a deep integration with the XFDL language that encompasses nested component orientations, text direction options, and even support for visual versus logical data representations.

Finally, on the XForms front, there are several XForms 1.1 features now supported in the product line, including having Lotus Forms that communicate with the web services of an SOA. This release also supports half a dozen new functions that facilitate everyday programming, such as date math and better string comparisons for sorting and searching of data. The release also supports the new XForms datatypes that allow empty content and simplify authoring.

To be sure, no blog entry could cover all the news of a major release, but these are some highlights I have found to be meaningful, and I hope you will download the trial versions and start finding out what you like best about Lotus Forms.

Well, now that I can finally log in to the blog system again, it's time for an update on XForms 1.1.

The working group has been addressing the "last call" comments of the community since last call officially ended back at the end of April. Although we got a pretty healthy review during the March and April review period, it so happens that we have received an equal or slightly greater number of review comments after the official review period. The working group has been treating these comments with the special "last call" status as well. This not only maximizes our responsiveness to our community but also, frankly, makes for the highest quality specification, which I think is quite important at the current stage of XForms within the big wide world.

As a side comment, I would say that the bar of technical precision is extremely high, in my opinion, relative to most other W3C groups, and in XForms 1.1 (as well as the new editions of XForms 1.0), we have had great success at tightening up our most sophisticated and powerful features, which is to the benefit of our users.

Since last call started on XForms 1.1, there have been nearly 150 comment records created in our database, which is quite a lot, but the number of records does not tell the full story because very many of those records report half a dozen or a dozen issues at a time, usually amounting to one person's feedback about an entire module within the spec. It is more telling to have a look at the diff marks in the editor's draft of XForms 1.1. There are over 650 and counting!

The working group met in Madrid last week to continue work on the last call. Many thanks go to our gracious hosts at SATEC, who not only provided all the facilities we needed to do the work (and a great dinner on Thursday), but whose members went out of their way for us after the meetings by driving us back to our hotels in the evenings!

When the meetings began in Madrid, we had 48 remaining last call comment records to address. I organized them into related categories, provide a sensible order of attack, and did my best to predict how long to spend on each topic. It was pretty daunting, though, because some of these issues were still open because they were the hardest to deal with and were deferred to the face to face meeting for that reason. I was mightily pleased with how hard the Forms Working Group worked in Madrid. Everyone did their best to elucidate all sides of each technical issue, and to compromise in order to reach consensus. We had a number of issues that were hard because both sides of a debate were crystal clear and each side could even see the sensibility and consistency of the other side, depending on the fundamental set of "axioms" one accepts about XForms as a language and application platform. Yet still, we managed to produce resolutions and then action items for 39 of the last call comment records.

Of the 9 remaining, 3 were resolved on the latest teleconference. Suffice it to say that the next month or so will be busy indeed as we complete all the action items related to the working group's decision and also come to consensus on those last six issues. Hopefully, we can advance XForms 1.1 to "Candidate Recommendation" before the publishing moratorium that is coming before the W3C Technical Plenary in Boston in November. It'll be a close call, but either way the long-awaited "CR" for XForms 1.1 should occur in the next two months at the outside.

Candidate Recommendation is a very important step in the W3C process. It says that the working group thinks the language is done, and the concerns of the wider community about it being done have all been addressed. This is the level of stability in the specification that is needed to support the second meanning of "Candidate Recommendation", which is that it is the "Call for Implementations". To be honest, many implementers have already added support for half or more of the new XForms 1.1 features, but this is the time period in which the working group must formally show a full implementation and at least two interoperating implementations of all features. XForms has no shortage of implementations, but starting to "dot the i's and cross the t's" on the implementation report before the year end.

Clearly, there has never been a better time to be working on XForms!

XForms is an important standard for encoding the core XML data processing asset of a forms application for multiple reasons.

For one, it fills the gap of schema languages, which are predominantly focused on describing what constitutes correct data that should drive server-side transactions. XForms codifies what it takes to get from an empty initial instance of a schema to a completed correct instance of a schema.

Another reason is that XForms provides an efficient language for doing the above, one that is based on such well-known and time-tested techniques as the Model-View-Controller design pattern, declarative formulas based on the spreadsheet algorithm (the second killer app of computing), and asynchronous event-based scripting. Frankly, the scripting capability is very effective for two reasons. First, each command is fine-tuned to the data by all the declarative constructs, so each line of script can do the work of 10 to 100 lines of purely imperative code. Second, the scripting in XForms actions are quite focused on mutating the XML data, so you don't get the baggage of a general purpose language that can create arbitrarily complex data structures that exist only in the memory heap of the run-time processor.

On the one hand, one could conclude that all of these language efficiency and simplicity aspects of XForms are beneficial to application authors who write XForms. But on the other hand, it turns out that the crown jewel is that the language efficiency and simplicity of XForms reverberate into the design tools that can all authors to write the application for a schema without much exposure to pointy brackets, much less coding.

To illustrate the point, I've created a 12 minute video of an XForms design experience on developerWorks to help show you more about what can be done with XForms based on a wizard-driven, point-and-click visual design experience. This video is similar to what I presented in the developer track at WWW 2007 in Banff. Please check it out and let me know what you think.

A rich internet application (RIA) is a web-based application that has more sophisticated capabilities normally associated with desktop applications. A typical RIA deploys little or no code beyond the HTML, JavaScript and AJAX calls needed to make the rich web application run. The AJAX calls are needed to keep the client-side browser experience in sync with the user's data content, which is maintained on the server-side by servlet or portlet code. One important aspect of RIAs is that they tend to be highly cross-platform because they only rely on features available from web browsers.

A rich client platform (RCP) is a framework of widgets, widget containers and other processing capabilities that facilitate sophisticated application development and deployment. One important aspect of RCPs is that they tend to be highly cross-platform because platform support is provided by the framework.

In both cases, you get sophisticated capabilities and cross-platform support. In the case of RCPs, you get these benefits by learning a framework, deploying the platform to the client-side and developing applications targeted for the platform. The one negative here is that you are invested heavily in the platform, so if some aspect of the platform is a showstopper (such as large download size), then you won't be able to deliver the application. In the case of RIAs, you get the benefits by learning a framework (browser HTML/JS/AJAX + J2EE), deploying the platform to the server side and developing applications targeted for the platform. The one negative here is that you are heavily invested in the platform, so if some aspect of the platform is a showstopper (such as no offline operation of the application), then you won't be able to deliver the application.

XForms solves the RIA/RCP conundrum by providing a framework for expressing the core data processing asset of an XML-based application independently of the deployment paradigm. There are XForms implementations of both types, RCP and RIA. In particular, the IBM Lotus Forms Webform Server is a product that converts an XForms-based Lotus Form into a rich internet application (RIA) automatically. The IBM Lotus Forms Viewer is a rich client platform (RCP) that provides the run-time for XForms-based Lotus Forms. The application developer is free to think about the application without thinking so much about the pecularities of how it will be deployed.