Avenue Q and French Open XML

I'm spending a few days in New York with the family and we just saw Avenue Q tonight - absolutely fantastic. I haven't laughed so hard since ... well ... since ... uhm ... probably since seeing Spamalot two years ago.
In an unrelated story, I just saw that Julien Chable has recently published three French articles on his blog about Open XML and using Altova products like XMLSpy and DiffDog:

• Travailler avec Open XML : Altova XML Spy (Partie 1)


• Travailler avec Open XML : Altova XML Spy (Partie 2)


• Travailler avec Open XML : DiffDog de Altova

Recent Reviews of XMLSpy and DatabaseSpy

In the past week I came across two interesting recent reviews of our products in blogs that I wanted to mention briefly:

• Michael Testi wrote Software Review: XMLSpy 2008 From Altova on his All This And Everything Else blog, where he reviews the latest version 2008r2 of XMLSpy.
• Gandalf wrote DatabaseSpy: Project Management: Part 1 Data Sources on his myDezigns blog, where he is looking into the newest features of DatabaseSpy.

As always you can download a free 30-day trial version of any of our products to try it yourself.

New BIG "minor" release of Altova tools

It's called Version 2008 Release 2, but in reality it should be a new major version. Our "problem" is that each year has 12 months whereas our talented engineers are practically cranking out a new major version every 5-6 months. So we have to call one of them the major release and the other one a minor release - but this one is BIG!

We've updated all the tools in the popular Altova MissionKit bundle with tons of new features and usability enhancements that our customers have asked for. I am most excited about the following, which provide big benefits to our users:

• Very Large File Support: XMLSpy 2008r2 contains a number of advanced optimizations for working with very large files. These result in a reduction of memory consumption by up to 75-80% compared to the previous version when opening and validating XML documents in Text View. This means that you can now open and work with files that are about 4-5 times larger than those supported in the past!!
• Extended Open XML (OOXML) Support: XMLSpy was the first XML Editor to directly support Open XML in April 2007 and today we are introducing more Open XML support in these products:
• MapForce 2008r2 now directly supports SpreadsheetML and allows the user to place any Excel 2007 document inside a mapping project to directly transform data from EDI, XML, databases, web services, and legacy text files to Excel 2007 and vice-versa. This new support for Open XML and Excel 2007 is, of course, also available in the automatic code-generation capabilities of MapForce, allowing developers to generate application code for recurring data transformation scenarios in Java, C# and C++.
• StyleVision 2008r2 now directly supports Open XML output in Word 2007 (WordprocessingML) to allow the user to generate multiple rich output formats from one single stylesheet design. StyleVision supports the generation of stylesheets via an easy-to-use drag&drop interface from XML documents as well as from databases and is the ultimate report designer that can produce output in HTML, PDF, RTF, and Open XML from one visual design. In addition, it allows developers the creation of Authentic forms from the same design to facilitate XML-based data entry across an organization with no deployment cost.
• DiffDog 2008r2 now supports detailed XML differencing between Open XML documents, including the ability to directly edit and merge changes across those files. In addition, the directory comparison feature now also supports ZIP file types so that directories and ZIP archives can be compared as well.
• Expanded Modeling Capabilities: UModel 2008r2 now supports the OMG's BPMN (Business Process Modeling Notation) and is also the first UML tool to ship full support for C# 3.0 and Visual Basic 9.0 - including accurate parsing of new language constructs in these programming languages that directly support XML. UModel does, of course, also continue to fully support Java 6.0 and provides full reverse-engineering and round-tripping for all the above languages.
• Better Integration Through Global Resources: developers using multiple Altova tools - for example as parts of the MissionKit bundle - can now take advantage of increased integration between these tools. The new Global Resources feature lets a developer define directories, databases, and ancillary files in one central location and those are shared between all applications. In addition, a developer can define multiple deployment scenarios (e.g. test, staging, production) for their XML projects, and also directly connect the output of one application to become the input for another.

The above list has just a few of the highlights that I find most exciting. More details and all the other cool new features can be found on the "What's New" page on the Altova website. There is also a press release being issued today about the new version.

I will also be covering some of these features in more detail on this XML Aficionado blog in the next couple of days - stay tuned...

Open XML is now an ISO Standard

The official press-release came out of the ISO offices on April 2nd and Open XML (OOXML) is now an ISO Standard with the official designation IS 29500.

Microsoft issued a press release today, and states that 86% of all voting bodies and 75% of P-members approved the standard - both measures being above the needed thresholds of 75% and 66.7% respectively.

See also the following blog and media reactions today:

• Brian Jones: "The end of the file formats war"
• Jason Matusow: "Open XML is and international standard"
• Peter O'Kelly: "Ecma Office Open XML Document Format appears to win Approval as an ISO Standard"
• Computerworld: "Unofficial vote tallies show Open XML being ratified"
• CNet: "Unofficial document: Open XML passes ISO vote"
• ZDNet: "Microsoft wasting no time rejoicing in its OOXML win"

There is also an interesting story floating around that Norway allegedly seeks to reverse its Open XML vote to No - but that seems to be irrelevant given the high margins that the tallied outcome has over the minimum requirements for approval as a standard.

To get an early start working with Open XML (IS 29500), check out Altova's support for Open XML in our XMLSpy XML Editor.

You are also invited to read all previous articles on Open XML on this XML Aficionado blog - especially my January 30 tutorial post on Content reuse with Open XML and XSLT.

New York International Autoshow

I spent a day at the NY Autoshow this weekend with my son to take a look at the latest concept cars and the newest production cars from the world's largest and most exclusive automobile brands, and to check in on the progress of UIMA, an XML standard that has the potential to one day be used by cars to communicate with the road and with other cars around them.

Being a technology aficionado, I was very impressed to find more evidence of a move to carbon-fiber and aluminum construction evident with many manufacturers this year. Similarly there was are a lot of new developments with respect to electric and hybrid vehicles that could be seen in New York.

Here is a quick gallery of some of my favorite cars at the show:

Mazda Furai Concept Car

Saleen Raptor

Audi R8

BMW Concept Car

And my favorite of the entire show - and a great example of the use of carbon-fiber and aluminum in a high-end production vehicle - is the Aston Martin DBS:

Aston Martin DBS

Sadly, a few revolutionary cars that I would have liked to see were absent from the show, such as the Tesla or the Gibbs Aquada. And a few other high-end car manufacturers were also missing, for example the Koenigsegg or the Pagani Zonda.

But there was a flying car on display. Well, at least a prototype of one. You just can't have a car show without a flying car:

I was also looking in vain for evidence of any production applications of UIMA or other similar software approaches that would allow XML to be used by cars to communicate with the road and other cars around them. But that technology still seems to be a few years in the future before it reaches production vehicles, as UIMA is presently being standardized by and OASIS TC.

The exciting thing about UIMA is, of course, that it is not only based on XML, but being described by a specific domain model in XMI (which can be used in the UModel UML Modeling Tool), and contains specific Web Services descriptions in WSDL (which can be viewed in the XMLSpy XML Editor and utilized in our MapForce data integration tool). Last, but not least, there are semantic web implications with respect to knowledge integration in UIMA (which result in RDF and OWL information that can be viewed and processed in our SemanticWorks RDF and OWL editor). I will be following UIMA closely and will report on any progress in this XML Aficionado blog.

More photos can be found in this gallery on my photo website. And for more news from the New York Autoshow, please refer to the various in-depth reviews on Popular Mechanics...

OOXML vs. ODF - the "battle" is heating up as we get closer to the ISO BRM date

I wrote about the Burton Group's report "What's Up, .DOC?" before on the XML Aficionado blog, and it didn't take long for the ODF Alliance to write a scathing rebuttal to the Burton Group report. Ironically, that rebuttal was published in PDF format, not ODF...

Before we take a look at what happens next, maybe it is time for a short review of the various acronyms and abbreviations that are commonly used in these reports, discussions, and in related blogs:

OOXML	Office Open XML is an XML-based file format specification for electronic productivity application documents, such as spreadsheets, charts, presentations, and word processing documents. Originally developed by Microsoft, it is already an Ecma standard and widely used due to its implementation by Microsoft Office 2007. It is currently in the process of being proposed as an ISO standard.
ODF	Open Document Format is a file format for electronic office documents, originally developed by Sun for the OpenOffice.org office suite and then later standardized through OASIS and ISO.
ISO	International Organization for Standardization
BRM	Ballot Resolution Meeting is the ISO process by which comments received during the previous ISO FastTrack vote and letter ballot phase are resolved by the meeting, during which national bodies and the submitting entity (Ecma) will possibly agree on a set of revisions to the originally submitted standard text. The DIS-29500 BRM is scheduled for February 25-29, 2008, in Geneva.
DIS-29500	The official ISO name and standard number for OOXML
OASIS	Organization for the Advancement of Structured Information Standards; a non-profit consortium that defines open standards for the global information society
Ecma	Originally this was called the European Computer Manufacturers Association, but their new name is Ecma International - European association for standardizing information and communication systems.
XML	eXtensible Markup Language as defined by the W3C in 1998. Probably the most important standard of them all, because both OOXML and ODF are built on top of XML. If you don't know it already, you should definitely learn XML... :)

 

So what's new with the OOXML vs. ODF debate now that we are only two weeks away from the ISO BRM? Earlier this week, the Burton Group responded to the ODF Alliance's rebuttal in a series of three postings by Guy Creese on the Collaboration and Content Strategies Blog, and you can find them here: Part 1, Part 2, and Part 3. In this response, the Burton Group addresses each criticism from the ODF Alliance point by point.

Also, Slashdot reported this week on the Ecma response to the ISO comments and the recent blog post from Russel Ossendryver (an open source and ODF advocate) criticizing the Ecma response.

If you prefer some demos over reading thousands of pages of specifications, you may find these videos interesting that have been posted on YouTube recently: a video of Open XML on the iPhone, as well as a video of Native Open XML support on Mac OS X. Both videos show support of OOXML on Apple's platforms, yet Martin Bekkelund (a proponent of Norway's "no" vote on DIS-29500) writes on his blog today about some headaches he's had with OOXML on the Mac and his iPhone. I was curious about his allegation that a .DOCX on the iPhone produces an error message, so I had to try it myself - and I am happy to report that any .DOCX attachment received on my iPhone (running the 1.1.3 software) displays beautifully and works pretty much exactly like it is shown in the YouTube video above.

More commentary and further information can be found on Michael Desmond's blog, as well as in previous OOXML-related posts on this XML Aficionado blog. Also keep in mind that the best way to learn OOXML is to start experimenting with it, and I recently wrote a longer article on Content reuse with Open XML and XSLT to show exactly how easily it can be done using the built-in OOXML support in your favorite XML Editor.

One thing is certain: everybody will be watching the outcome of the ISO BRM very closely...

XML-People

Tim Bray posted a great story "XML People" about the early days of XML, and the people involved in creating the standard. It's hard to believe that XML is 10 years old already.

Talking of which, XMLSpy is just about 9 years old this month: we launched version 1.3 on 1/24/1999 and version 1.4 followed on 2/15/1999. There was never a version 1.0... :)

Content reuse with Open XML and XSLT

While Open XML may not yet be an ISO standard, it is already standardized by ECMA and - even more important - all documents created by Office 2007 are already stored in Open XML by default, so there is an abundance of documents whose content you can now reuse much more easily and productively than ever before. So instead of waiting for the ISO vote or paying too much attention to all the political battles being fought around it, I want to show you how you can already take advantage of Open XML (sometimes also called OOXML or Office Open XML) today.

This is the first article in a series of blog postings that I plan to write about practical Open XML tips & tricks, so I encourage you to subscribe to my XML Aficionado blog (via RSS or via e-mail), if you haven't already done so. This will ensure that you get future articles from this series automatically as soon as I post them.

So let's look at an Open XML document in our favorite XML Editor. For this example I am going to use a WordprocessingML document (.docx) that I have created with Microsoft Office Word 2007. When I open the .docx file in XMLSpy, I immediately get to see the contents of the package file, which is structured according to the Open Packaging Convention.

That's a fancy way of saying that it is a ZIP file that contains specific files and directories that make up the content, structure, styles, relationships, and other parts of the document. Using XMLSpy's built-in capability to open any ZIP-formatted archive, I can directly browse any directory structures inside the ZIP package, add new files to the package, or open any existing XML file contained in the package:

For the purpose of reusing the content from this WordprocessingML example file, I am going to open the 'document.xml' file, which contains the content of the document.

As soon as I double-click the file in the ZIP archive, the XML is displayed in a separate window just like any other XML document and I can use the powerful grid view or text view features of XMLSpy to view or edit the XML data (sometimes it may be useful to invoke the pretty-print function in text view to make the file more easily readable):

This is, of course, a live editing view, so you can not only view the Open XML data, but make any changes to the XML and save it back into the package file.

But now let's look at how we can easily reuse content from this Open XML document using XSLT. XMLSpy ships with a few Open XML example documents as well as example XSLT stylesheets for just that purpose. Let's look at the 'docx2html.xslt' stylesheet, which takes a WordprocessingML document and extracts all paragraphs to turn them into HTML. This example stylesheet is by no means intended to be a fully-featured conversion tool from .docx to HTML. Instead it serves as a blue-print of how to reuse content from a .docx file and hopefully will serve as a starting point for your stylesheet development efforts.

At the core of that XSLT stylesheet we need a <xsl:for-each> loop to iterate over all the <docx:p> elements, which it turns into simple HTML <p> paragraphs. The text inside the paragraphs is grouped into runs of characters that share common attributes, and so we need an inner <xsl:for-each> loop to iterate over those <docx:r> elements and extract the text from their <docx:t> text node children. Thus the most primitive content reuse that only extracts the text of all paragraphs looks like this:

Once we have constructed those loops, we can start to think about perhaps extracting and reusing some style information. To do that, we now emit a <span> HTML element for every <docx:r> run of characters and give it a style attribute, whose value will depend on the <docx:rPr> element, so we use <xsl:apply-templates> to decide what HTML style we want to apply to the <span> elements:

The corresponding templates for the three most common styles (bold, italic, underline) are trivially easy to construct and look like this:

With just a few lines of XSLT and a few templates we have already written a stylesheet that extracts the basic paragraphs and most important styles from a WordprocessingML document and turns them into HTML that can be viewed in the browser view - here is the result produced from running the above XSLT stylesheet on the example WordprocessingML document that you can find in the XMLSpy examples directory:

Similarly, it is quite easy to extend the stylesheet to extract meta information, other styles, or image information from the WordprocessingML document and reuse the content for any modern application scenario, from web publishing via HTML, RSS, or social media formats to mobile web applications and beyond.

"But wait! How can I apply an XSLT stylesheet to an XML document that is stored within a ZIP file?", you might ask.

You can, of course, extract all the XML files using a regular ZIP expander, but there is a much better solution: when you use the document() function in XSLT 2.0 within XMLSpy or with our royalty-free XSLT engine AltovaXML, you can directly access files contained in a ZIP archive by using the "|zip" pipe operator within the filename, e.g. "MyDocument.docx|zip\_rels\.rels" will address the Relationship file ".rels" in the archive directory "\_rels" inside the ZIP package with the file named "MyDocument.docx".

The benefits of using XSLT to reuse content from Open XML documents are obvious: because XSLT is a cornerstone of the core set of XML standards from the W3C, you can apply all your existing XML, XPath, and XSLT know-how and you can use the excellent tools support that is available for these standards. For example, you can easily develop and debug your XSLT stylesheet using the powerful XSLT debugger in XMLSpy, which allows you to single-step through the transformation, set breakpoints on XSLT instructions or even on data nodes in your Open XML document, view the partially generated output, and inspect the state of the XSLT processor in detail as the output document is constructed:

Using the XSLT Debugger eliminates a lot of the pain that is normally associated with XSLT stylesheet development and allows for a very iterative approach to creating and improving stylesheets that facilitate content reuse and repurposing.

To sum it up, reusing content from Open XML documents for a variety of web applications, mobile scenarios, or social media and Web 2.0 contexts is very easy and can be achieved with standard XML-related technologies, such as XSLT.

For additional information on Open XML and how to take advantage of all the content that is now already available in that format, please refer to the following sites:

• Open XML support in XMLSpy
• Download a free 30-day evaluation version of XMLSpy
• MSDN Open XML Formats Resource Center
• OpenXMLDeveloper.org
• Brian Jones: Open XML Formats (blog)

Using XML Catalogs in XMLSpy

Jerry Sheehan has recently posted a useful article "A simple way to re-direct schema locations in XMLSpy using XML Catalogs" on his XML Scoop blog. In it he describes how to extend the CustomCatalog.xml file in your installation directory to redirect PUBLIC or SYSTEM identifiers in DTDs as well as URI references in XML Schemas to local copies of frequently-used schemas or DTDs to reduce loading time. XML Catalog support in XMLSpy is based on OASIS XML Catalogs.

OOXML and ODF report from Burton Group: "What's Up, .DOC?"

Burton Group has released a new report "What’s Up, .DOC? ODF, OOXML, and the Revolutionary Implications of XML in Productivity Applications" this week. Written by Peter O'Kelly (blog) and Guy Creese (blog), the report provides a deep and insightful analysis of the current state of ODF, OOXML, and other document formats (W3C, PDF), the history of those formats, and then continues to parse through the FUD and standardization games to arrive at a set of projections regarding the success of OOXML and ODF, as well as a set of practical recommendations.

I've read the report already, but am not going to spoil the fun for you and reveal all the conclusions here - the report is well worth your time and you should read it yourself! However, I will say this much: the report validates some of my thinking on the subject that I have expressed in various previous blog posts on OOXML here.

There is a quick overview of the report on the Burton Group's Collaboration and Content Strategies blog, and you can download the entire report for free after filling out a registration form.

Other early blog reactions to the new report are here:

• Burton Group report on ODF/OOXML will "ruffle some feathers"
• The coming OOXML showdown

And I am sure more blog reactions will follow next week.

As always, if you are interested in working with Office Open XML (OOXML) files, a great place to get started is to look at the OOXML support in XMLSpy and to download a free evaluation version of Altova's XML Editor and give it a try.

XMLSpy awarded Certificate of Excellence for IBM CTO Innovation award

XMLSpy 2008 was selected today as one of only three finalists for the IBM Information-On-Demand CTO Innovation award.

It was a very diverse list of nominations for the CTO Innovation award and a very competitive judging process - with Altova being the only company in the XML and developer tools space among the semi-finalists and finalists.

At the award breakfast today the actual "winner" didn't even show up, but we received the above Certificate of Excellence. The judging was done by a panel of CTOs, experts, and IBM executives, and they were positively impressed by the new DB2-specific functionality of our XML editor.

XML Aficionado in the News

Two articles hit the news today that are based in part on interviews with the XML Aficionado:

• The X Factor (by Michael Desmond) in Redmond Developer News
• Am Anfang War Das Schema (by Dr. Peter Zschunke) in Associated Press Germany

The former is a detailed analysis of the OOXML vs. ODF controversy from a developer's perspective, including interviews with Burton Group analyst Peter O'Kelly, Microsoft's Brian Jones, and me.

The latter is a review of the new Altov XMLSpy 2008 release (in German).

XMLSpy Tip: Using the XPath analyzer to quickly find information in files

One of my favorite features in the Altova XMLSpy XML Editor continues to be the XPath analyzer, which lets you quickly locate information in any XML file by typing an XPath 1.0 or 2.0 expression. Introduced first by Altova in 2001, the XPath analyzer gained XPath 2.0 awareness in version 2004 and was turned into a new dynamic window with multiple output tabs in version 2007.

The XPath analyzer is really convenient and easy to use. Located within the bottom information/output window, the XPath analyzer is found in a separate tab, right next to the Validaton results and the Find in Files function. You can configure it for XPath 1.0 or 2.0 with the click of a button, and can immediately see the result of any XPath expression, by typing it into the text input line - the XPath expression is evaluated immediately and the resulting nodeset is displayed in the output window:


In this example I have used the XPath expression //expense-item[@type="Lodging"] to locate all <expense-item> elements that have a type attribute with the value of "Lodging".

The resulting nodeset is evaluated dynamically as you type, so you can always see whether or not you have entered a correct XPath expression (in addition, the text entered turns red, if there is any error in the expression). You can quickly navigate to any of the found nodes by clicking on them, and the corresponding node in the document will be highlighted in the main editing window (irrespective of which view you are using).

One of the cool new features that was just added in the new XMLSpy 2008 version this month is the ability to also evaluate XPath expressions across multiple files. This is great if you are working on a larger project and need to quickly see where certain information is located. Sure, there is always Find in Files, which has powerful Regular Expressions, but a true XML Aficionado always prefers to use XPath :)


In this screenshot I was searching for all elements in any of my project files that have an xml:lang attribute - this can be expressed in XPath as //element()[@xml:lang] - and the resulting nodeset shows nodes found in several different files in my project. I can, of course, open any of these files and immediately view the found nodes just by clicking on them.

Here is another nifty trick: if you often work with larger files and want to persistently bookmark certain positions in those files, you can also use the XPath analyzer to quickly navigate to those locations. All you need to do is add comments to the XML file to mark those locations, e.g. <-- Bookmark #1 --> and then you can use the XPath expression //comment() to show all comment nodes in your file. Now you can click on each one and you will see them displayed in the editing view. You can even quickly reestablish bookmarks by clicking on each one and hitting Ctrl-F2 to set a new bookmark:


To see more useful information about the XPath analyzer, watch this cool flash video or read the XPath Evaluation chapter in the online manual.

Altova releases Version 2008 of the entire product line

I am very excited! This new release is packed to the brim with new features like XInclude and XPointer support, two-step XSL:FO transformations to allow use of XSL:FO with XSLT 2.0, evaluating XPath across multiple files, direct editing of database data, extended support for Office Open XML file formats, and speed and memory footprint improvements when working with large files in XMLSpy; aggregate data processing and value-map functions in MapForce; refactoring and modularization support as well as enhanced database functionality in StyleVision; Visual Basic .NET code engineering and real-time integration with Visual Studio and Eclipse in UModel; support for editing database views and stored procedures in DatabaseSpy; one-click directory synchronization and binary differencing in DiffDog; and much more.

We've really listened to our customers and implemented many of these features based on their direct feedback. I'm sure that you will find many of these enhancements and new features indispensible for your every-day work with XML, UML, and database technolgies. Make sure that you check out the MissionKit for Software Architects bundle, which includes all of our products in one convenient package.

In addition, you'll find common enhancements across the entire product line, such as upgraded support for Unicode 4.1, support for Widows Vista, an improved database connection wizard, and more.

For complete details see the press release, our What's New page, or simply download the free 30-day evaluation version. Also seen on the Altova News RSS feed.

Let me know what you think about the new features - post your comments directly on my XML Aficionado blog.

Office Open XML fails to win ISO approval (so far)

This week Microsoft failed to win ISO approval for the Office Open XML (OOXML) standard in the 2nd round of the ISO standardization process. While the Wall Street Journal published a critical article about this, this is by no means the end of the road, nor is it even a major setback - the ISO process commonly requires multiple rounds, and the 3rd round (expected for early 2008) will very likely see Office Open XML becoming an ISO standard.

As Burton Group analyst Peter O'Kelly observes in his blog: "FWIW I still expect Open XML to become an ISO standard -- and it's reassuring to see the spec/design improved by the standardization process." - I couldn't agree more.

Irrespective of the timeline of this ISO standardization process, I expect OOXML to quickly become a de-facto standard as more and more corporations and enterprises upgrade from Office XP or 2003 to Office 2007 and start generating tons of documents in the new OOXML formats (which is the default setting for new Office 2007 file).

Having all these Word documents, Excel spreadsheets, etc. available in OOXML lays the groundwork for a huge increase in content reuse and repurposing, because the data - being XML - can now be accessed through other applications, transformed through stylesheets, and integrated with other enterprise data.

To get started with OOXML, software developers can simply open OOXML documents in XMLSpy and start working with the XML data inside, then use XMLSpy to create and debug XSLT 2.0 stylesheets that directly access the OOXML data through the AltovaXML processing engine.