1st part in a series of tutorials on XRX - building a RESTful delicious client in eXist and orbeon
rest
XRX: Performing Updates | O'Reilly News
Submitted by administrator on Wed, 07/30/2008 - 15:15.XRX: Performing Updates | O'Reilly News
XRX: Simple, Elegant, Disruptive - O'Reilly XML Blog
Submitted by administrator on Wed, 07/30/2008 - 12:44.XRX: Simple, Elegant, Disruptive - O'Reilly XML Blog
XRX
Submitted by administrator on Thu, 05/29/2008 - 22:09.Bumped into XRX today. XForms + REST + XQuery. I like the sound of this, and XForms on the client just got a whole bunch easier…
I’m seeing multiple signs that the confluence of XForms and XQuery has legs. (And REST just plain makes sense in any situation). -m
XRX: Simple, Elegant, Disruptive
Submitted by administrator on Tue, 05/27/2008 - 12:33.I recently started writing for O’Reilly Media. I posted an article on XRX.
Here is the link:
http://www.oreillynet.com/xml/blog/2008/05/xrx_a_simple_elegant_disruptiv_1.html
We are just getting started moving to a new MoveableType (MT) system so not all of the features (like keywords and feedback) are working.
If you have problems commenting on the site, please feel free to post your comments here.
Here is a comment from Arun Batchu:
An elegant introduction to an elegant architecture, Dan. Thank you. The essential takeaway from what you describe is the exploitation of XML from one end to the other end - especially from the Developer's perspective, for, how XRX actually manifests in runtime could be left to implementation technologies. Thus the logical architecture of XRX could be realized by a few variations of concrete technology - which is great. The XForms could be realized by XForms server technology (such as the excellent Orbeon stack), the ReST could be realized by any middle tier and the XQuery could be realized by an XQuery engine (such as Data Direct) that may actually be driving any one or a combination of datastores (XML, SQL, file system or ...) . The symmetrical and consistent leverage of XML as a data model from creation to transport to rest and back eliminates a whole lot of wasteful work. Like you point out, XPath is one of the most powerful query systems I have encountered; you can pack so much in so little and reuse it across the board with little if any change from the drawing board to production. In such a system as you describe, a business rule expressed once can be reused anywhere - from one end-to-end , however long the travel, as long as the architecture is XRX, like you have described. Thanks for expressing it so well. A few of your readers will not get it - it is one of those things that once you experience it, you are left wondering why this did not happen before. Oh, well!
You are welcome! Thank for your feedback Arun!
XRX and MDA
Submitted by administrator on Sun, 01/06/2008 - 18:02.Summary
This posting describes how using three technologies (XForms, REST and XQuery or XRX) you can dramatically transform your organizations development methodology. You can move true model-driven architecture (MDA) that dramatically reduces the temptation to duplicate code.
In January of 2007 I was working on a complex real-estate forms project for a state agency in Minnesota. Our job was to get 87 counties across the state of Minnesota to agree on the over 250 data elements that describe real estate transactions. We had spend months developing a complex XML Schema and web forms with over 50 one-to-many relationships. And out next task was to be able to save the form
data into a structure that could be quickly queried by any of these 250 data elements. But the prospect of "shredding" the documents up into 50 distinct SQL INSERT statements and then reconstituting the documents with 50 distinct SQL SELECT statements was going to require doubling our small team of four to at least eight developers. We didn't have the budget or schedule to do this. After chatting with Kurt Cagle he suggested saving the data to a database that supported XQuery. Both the open source eXist native XML database and our corporate standard, DB2 supported XQuery. It turned out that we could indeed save the entire document with a single command and still perform complex queries on any element in any document. Our project could proceed with a small team and stay on schedule.
And then an interesting thing happened. Our team started to understand that if we saved additional metadata in these native XML data stores we could accelerate our project even further. Every pick-list in every form could dynamically call REST-enabled web services to get their values. XML Schemas could be stored in the databases and be queried. We had found a simple and elegant solution to the pervasive problem of where to store the model in a model-driven development project.
In the past I had been on large teams of developers that attempted to use model-driven development. But although we started out each project with idealistic goals, each time we were faces with a classic option: copy, paste and edit the model or write a transform. The copy and paste solution was quick but it would have to be redone each time the model changes. The transform took longer to write but then could be rerun each time the model changed.
Like most developers, I always was over-optimistic that I had done my homework and done accurate and detailed modeling of the problem. I thought the models were stable and would not change very often. And I was almost always wrong. Models do change and sometimes in totally unexpected ways.
But by storing our models in a native XML database an interesting started to happen. We could quickly transform the models (stored in XML) into other artifacts with simple XQueries. It became far less tempting to commit the sins of copy/paste/edit when it only took five minutes to create another transform. And most remarkably, each of these transforms was another REST enabled web service that could be reused by many web clients or other development tools.
So how do you get started? You might try the following steps.
- Download the exist database from http://www.exist-db.org
- Use a WebDAV interface and copy some XML Schema documents into a collection
- Write a few small XQuerys that pull metadata out of your XML Schemas. For example pull out all the enumeration values out of an element.
- Use those XQueries to drive some aspects of your development such as an XForms selection list.
- Compare the amount of code you wrote with any other MDA system. If you don't have at least a 10 to 1 savings I would be very surprised.
What do you think? Do you have a more efficient way to query your model? Let me know!
- Dan
skimming at XML 2007 (and The Cloud's Silver Lining)
Submitted by administrator on Fri, 12/14/2007 - 16:51.I've not yet had a chance to write up my experiences at XML 2007 in Boston last week, but an announcement today by Amazon prompts me to at least summarise one of the talks I did, called XForms, REST, XQuery...skimming. (Slides)
It's a theme I've touched on a little in this blog, and have spoken on before (at an XML UK meeting, last year, and at XTECH 2007, this year), and the idea goes something like this; building web applications by dynamically generating user interfaces that contain data, is awkward, and difficult to maintain. Web services (which includes REST and APP) removes some of the dependency on databases by hiding the data behind a simple abstraction layer--a kind of 'ODBC for the web'--and XQuery takes this further by giving us a standard language to get at the data.
But XForms is the icing on the cake, since it gives us a rich-client language that can be deployed like traditional web applications, yet is one that ensures a clean separation of the data and the user interface.
During the course of the presentation I showed how incredibly easy it is to create an application with XForms, since forms can be first created on the desktop--loading and saving their data from local XML files--before being deployed to any type of web server. By creating static forms that load their own data, rather than the more common model of dynamically generating static files that contain data, we end up with files that are completely agnostic about where they are delivered from. My demonstrations took them from the desktop, to a simple web server, to eXist (an XML database).
But these applications could even run in 'the cloud', with no web-server. That may sound odd, but I could store any of these XForms applications in SVN or Amazon's S3 and they would work as well as if they were in IIS or Apache. This is the ultimate in simple maintenance and future-proofing, since you are no longer reliant on any particular operating system or server-side language...the forms just 'are'.
GData
The final step of my presentation was to take this a little further and show an XForm that interacts with Google's GData; the form is simple, and accepts a longitude, latitude and comment which is then inserted as a row into a Google Spreadsheet, using Atom Publishing Protocol (APP). Another form is then used to retrieve the entire spreadsheet, and each row is used to create a pin on a map.
The great thing about this demonstration is that it shows that not only are our forms now agnostic about where they run--desktop, server-side, S3, etc.--but now the data becomes agnostic; we don't care how the data is stored, all we are concerned about is the protocol used to interact with it.
And crucailly, XForms' ability to get at this data shows that it really is the missing link when it comes to building a new kind of web application.
SimpleDB
But this generalisation about data just took another leap with the announcement today by Amazon of a closed beta called SimpleDB. This will allow programmers to store and query for data in a manner similar to the way that GData works.
By putting data into the cloud, and then using an easily deployed, declarative rich-client language like XForms, it is possible for the architecture of web applications to be altered in a quite fundamental way, in a direction that is completely different to what we're used to.
Introducing the XRX Architecture: XForms/REST/XQuery
Submitted by administrator on Fri, 12/14/2007 - 14:08.At the XML 2007 conference there were many people that seemed to have independently discovered that if you combine XForms/REST and XQuery you can create a software development environment that circumvents the need for middle tier objects and conversion to and from relational databases.
The conference also had a great deal of discussion of how XForms is a great architecture but as Elliot Rusty Harold pointed out, we need something to happen for XForms to replace things like
What XForms needs is a label around the collection of ideas that makes XForms really give you an order-of-magnitude improvement over other web development architectures. I would like to suggest that we get behind one "label" that brings many of use under one big tent. I would like to suggest we use "XRX".
XRX to me would stand for the ideas behind XForms/REST/XQuery web applications. But if people are using XHTML as a container for XForms you could use XHTML/REST/XQuery. Just spare use the "J" which is associated with a hard-to-use language for über geeks.
You could also use XRX for the format of the data in the client and on the server. In this case it would be XML on the client (in the XForms Model), and in a native XML database that supports a REST interface.
We can still build awesome tools and frameworks that make the XRX architecture really fly. IBM Workplace Forms Development tools, Orbeon Presentation Server, Kurt Cagle's new frameworks that use the eXist native XML database, and the hundreds of other XForms development tools all fit into the XRX strategy.
But if we make XRX into a "big tent" strategy we can also include XQuery on the server as part of the architecture. This would include people like Jason Hunter at MarkLogic. Jason is an experienced Java developer that has started to move away from Java on the server and toward XQuery because of its richness. And I could not agree more.
I have been reading the book The Tipping Point: How Little Things Can Make a Big Difference by Malcolm Gladwell and trying to apply the ideas in it to the adoption of XForms and XQuery. I think that the book has many example of how a careful packaging of the ideas behind a contagious meme can make ideas spread like widefire.
So when we hear the term "XRX" we want to have the following concepts come to mind:
- A web development architecture with a 10x productivity improvement over traditional JavaScript/OO/RDBMS methods
- A development architecture based on international standards that is designed to minimize the probability of vendor-lockin
- An architecture that gives a rich user experience without creating mountains of spaghetti procedural code on either the client or the server
- A system that leverages the REST architecture to take advantage of high-performance and simple interfaces using web standards
- Portability on both the client and the server using a variety of forms players and XQuery databases
- The option of avoiding costly shredding (and reconstitution) of complex XML documents into RDBMS tables
- A community of standards/tools and a "complete solution" ecosystem that can give you a proven ROI on your IT investment
I don't expect everyone to buy into the XRX meme ideas. Everyone has their own favorite tools/standards and acronyms. But everyone that was at the XML 2007 conference could feel the excitement in the room at the XML presentations. Now we need to give that excitement a name and see if it sticks. Hopefully a few years we will be able to walk into a bookstore and see a bunch of books with "XRX" on the cover.
What I now need is for people to start to test the XRX meme and see if it slicks. Let me know what works and what does work. Send me your 30-second elevator pitch. How big should the idea tent be? What do we have to do to get more people in our tent? What evidence (case studies), example programs and resource do we need to convince the world we have seen the light …and its so elegant and beauty to will keep us from ever going back to the stone age of web development.
Impressions of XML 2007 in Boston
Submitted by administrator on Sat, 12/08/2007 - 23:23.I returned from the XML 2007 conference in
Here are some of the highlights for me.
XForms appears to be getting lots of traction. The XForms sessions were all packed with standing-room only in most of them. The presenters we passionate that the combination of XForms, REST and XQuery has had a huge impact on their projects and may soon impact the rest of the development world.
Microformats are definitely starting to make huge strides where Metcalf's Law will start to kick in. The existence of 450 million tags and five FireFox microformat extensions indicate that within a year it will be just part of web publishing best practices. Even Microsoft is said to be supporting Microformats in IE 8, just a few years from now.
Taylor Cowan (who works for Sabre) gave an excellent presentation on how Microformats can be extended into the travel space with an Atom based format for publishing trip ideas. His use of Atom and semantic web technologies was right in line with my vision of how personalized agents are going to leverage microformats.
John Boyer's demo of IBM's Workplace Manager showed that you can now purchase top-shelf XForms GUI design tools that does not lock you into a proprietary vendor solution. I hope the people from Microsoft, Adobe, FairIsaac, Altova, Google GWT and other vendors take notice. Locking your users into a Java or JavaScript client is no longer an acceptable strategic option for developing web applications. Wake up everyone! Are you listening? We have a standard that works and lets now start building more great tools that use this standard. XForms is the only system that is creating as John puts it "Order of Magnitude" increases in developer productivity.
Mark Birbeck's Sidewinder demo showed that XForms is breaking out of the browser an on to the desktop. Showing web and iPhone web pages directly in desktop widgets was a great demonstration of the power of XForms.
Kurt Cagle's presentation about problems with programming with the fragile DOM model in JavaScript was interesting in its format. The presentation was actually an XForms application running on eXist. Kurt "eats his own dog food" on the client and the server! I am humbled by the man.
Jason Hunter from MarkLogic (an XML database vendor) gave an nice demo of the MarkMail.org application and showed that, yes, XQuery databases really do scale to the terabyte range. He has about 4 million e-mail messages indexed and you can do combinations of database and text searches in a few seconds. In other words a demonstration of how real-world analytics comes to XQuery. I also found out that MarkLogic is free if your database is under 100MB. Perfect for 10MB of metadata in a metadata registry.
The people from IBM research labs appear to be starting to take the XForms MVC architecture and really run with it. We saw some very interesting presentations on how future standards for data driven XML Application Components (XACs) and State Chart XML (SCXML) might be used in the future to make browser-based mash-ups easier. Componentization, composition, customization and reuse are all part of this architecture and it would not be possible without XForms MVC architecture. Hopefully Charles Weicha from IBM research will post his slides so you can see more.
Norm Walsh also talked about how the new XML Pipeline standard (XProc) is starting to take shape. Even though the standard is still in working draft there are already at least five implementations of the standard. This single standard could have more impact on the shape of server-side processing then any other standard coming out of the w3c. Integrating this into XQuery-based application servers seems to be the next step.
The eXist database also seemed to clean up all the open source native XML database talks. eXist was mentioned in over a half-dozen presentations and there were no other native XML databases even mentioned in the presentations I attended. It is not surprising that eXist is now the top hit on Google when you search for "XML database".
One of the most interesting presentations that I did not get a chance to see (but I did talk to the presenter afterwards) was Thomas White's presentation on using XSLT in the browser to get a 10x speedup in form rendering. It turns out that implementations of XForms players like FromFaces use heavyweight JavaScript objects in the browser. Thomas has found that a donkey cart can be turned into a sports car by implementing an in-browser XSLT centric event scheduler. If his architecture is implemented we could see vastly better XForms players in the future. Lets all hope that Thomas's architecture comes to the XForms world sooner than later.
I did not attend any of the XML publishing tracks. But from the titles it appears that the Darwin Information Typing Architecture (DITA) standard is a hot topic. Presentations on using XSL-FO and CSS for book and other print publishing were also on the agenda.
I also did not attend too many of the XML in the
I am disappointed that the conference organizers did not do anything to require that people post their presentation on the web site before their talk. There were many very interesting presentations that could have a larger impact the world and the sloppiness of people not publishing their slides seems to be a little unprofessional. There were some people that just brought their digital cameras and took pictures of each slide. Smart.
In summary I think that the conference had two types of people:
First there were people that had not done extensive work with XForms. The first group I call the "evolutionary proceduralists". Theses were people that were trying to beat the dead horse by patching the old-style procedural web application architecture with JavaScript, JSON and
The other type were "the enlightened innovators". People that have seen the real world benefits of dumping object middle-tier stacks and relational databases and going with a pure declarative approach to solving business problems based around XForms/REST and XQuery. These were the people that are going to lead innovations in application development.
I was also impressed how a large percentage of XML innovations came from outside the US. I attribute the fact that some of the best presentations came from the UK to be the "Michael Kay" effect.
Beauty is in the Eye of the Beholder
At one of the XForms sessions a young Ruby advocate stated that he thought his Ruby code was "beautiful" but he did not himself think that XForms code was "beautiful". Most of the people in the audience agreed that XForms was beautiful but like any new language, it takes getting used to. Beauty is the process of your visual cortex recognizing familiar patterns and signaling your pleasure centers. If you have never read Japanese then the most profound Haiku written in the most beautiful script will not have beauty.
XForms/REST/XQuery to me is indeed beautiful. Its beauty comes from its ability to quickly map real-world requirements into working systems with high fidelity. No other system in the world approaches its elegant architecture. And after XML 2007 I realized I was not the only one that had independently reached this conclusion.
Skimming, and an Open Source project for a GData XForms client
Submitted by administrator on Tue, 05/01/2007 - 15:51.For me, one of the most exciting developments on the web in recent years has been the growth in services that let you manage raw data. Many of the most useful such services originate when some web-based application--such as Google calendars or spreadsheets--exposes the underlying data. For example, with Google Spreadsheets, it's possible to use ATOM to query spreadsheets for certain values, or add rows and columns, making this a very powerful way of storing data 'in the cloud'. (See Web 2.0, Copernicus and Spartacus: Moving the centre of the web.)
We're building quite a few XForms applications that make full use of this power, and it's something I'll be talking about at XTech 2007, in my session XForms, REST, XQuery...and skimming. Since the approach we use could work on any XForms processor, talking to any ATOM-based server, we've decided to create an open source project to host the forms, and document their use. If this is an area of interest to you, check out the GData XForms client project on Google Code, and its discussion forum.
Metaphors for Declarative Systems
Submitted by administrator on Fri, 04/06/2007 - 14:31.I am working on my presentation for the 2007 semantic technology conference. My topic is the semantics of declarative languages.
I am attempting to present to business strategists that may not appreciate how powerful the new XForms and XQuery standards are and how entire rich user experience web 2.0 applications can be built using these tools as long as the tools have RESTful interfaces.
My first consideration is to find the right metaphors. I am working with the evolution metaphor and the puzzle metaphor right now and it seems to be working well.
I have also done some research on Domain Specific Languages and I see that they have also used the evolution metaphor but I find that they are focused on building small languages that are specific to a group or project and they don't really look at the external semantics of the problem.
Please write me if you are also trying to explain the business benefits of either declarative languages to a non-technical audience and if you have found metaphors that get the key points across.
Thanks! - Dan
XForms is coming of age with version 1.1
Submitted by administrator on Sun, 02/25/2007 - 22:07.A new draft of XForms 1.1 was made available last week. It's a last call working draft, which means that once comments made on the document have been taken into account, it will hopefully be possible to advance the specification through candidate recommendation, and on to a full recommendation.
To be brutally honest, in terms of features, XForms 1.1 is nothing to write home about. The bulk of the new stuff was needed long ago, and we could have done with having it in there before now: looping and conditional constructs are crucial to any language; the ability to set URLs and headers at run-time is a 'must-have' when using SOAP, WebDAV, ATOM, and REST; and being unable to insert nodes into a nodeset when all the nodes had been deleted was painful! Thankfully, those days are now behind us. :)
But just because none of these features are unanticipated, doesn't mean that this document is not an important one.
For a start, in many places it's clearer than XForms 1.0; although W3C documents are always a team effort, the improving coherence is due to a lot of hard work from John Boyer, as he's often prepared to go the extra mile when writing an explanation or giving additional examples. (A sense of what has changed can be seen by looking at the version with all differences highlighted.)
Secondly, whilst I might have a moan that the new features were slow coming, the point is that they indicate the commitment on the part of the working group to solve real world problems, and more importantly, they show how the underlying XForms architecture easily lends itself to the addition of new features in a natural way.
And finally, with the new submission features of version 1.1, and the already powerful XML-handling that was a hallmark of version 1.0, XForms becomes the obvious choice for quickly creating rich internet applications that use ATOM, WebDAV, and REST. That XForms does 'come of age' is crucial if, alongside the proprietary offerings of Adobe and Microsoft, there is to be an approach to building internet applications that is based purely on open standards.
When should I use XForms?
Submitted by administrator on Wed, 11/22/2006 - 10:34.The question about how XForms should be sold to customers often comes up, and generally we reply "why would you?". As in any IT project you need to use the right tool for the job, so why would you try to sell XForms in and of itself? Sometimes XForms is the right thing, and sometimes not, but either way--we say to those who ask--it's a tool in your armoury (if you want to use it), not your customers'.
If you have a system that uses web services a lot (for example), then a rich client that can handle XForms will be ideal--that's where most of our use cases are coming from for formsPlayer deployment. I've mentioned before a system that we've built for a customer that involves using the OpenWFE workflow engine and the eXist XML database, and we've written practically no code on the server! All we've done is used a few workflow templates, and the REST/XQuery interface to eXist, and everything else is achieved via the XForms themselves. Maintaining such a system is not only very straightforward, but is easy to move to other platforms, as well as swap components in and out (since the interfaces--such as XQuery--are all standard). This is all at the heart of the skimming approach we've been talking about.
But unfortunately, at the other end of the spectrum, if you have a system that uses a server-side scripting language, with direct queries on a database to populate list-boxes (that whole nineties thing!) then you might not find XForms that useful. You'll have a maintenance nightmare with such an application, either way. For XForms to be of benefit in this scenario it would have to dramatically improve the user experience, rather than only being 'as good' as what you have already.
I'm not differentiating here between using client and server technologies, but rather between the use of high-level constructs versus painstakingly creating pages with server-side script. So in the approach I'm suggesting, you will also get productivity improvements if you can use a pipeline-based system like Orbeon, since it too uses 'big constructs', even though it delivers HTML to the user.
At least in the short term, the main benefits of XForms will generally accrue to the developers and not to the users, and will be of most use to those who have a service oriented architecture, rather than an old-fashioned web-site. So trying to 'sell XForms' is probably not only a waste of time, but actually unnecessary--if it fits your system, use it.
