TechEd, Open XML, and HDR Photography

While being at TechEd in Orlando, FL, last week, I had lunch with Doug Mahugh and we talked about the upcoming ODF support in Office 2007 SP2, the new features in the Open XML SDK, Altova's new support for Open XML diff/merge in DiffDog, creation of Open XML from StyleVision, and data integration and mapping for Open XML in MapForce, as well as various other XML-related topics.

We also talked about some other industry topics and finally came to chat about HDR (high dynamic range) photography. Doug sent me a few links to some of his recent photos, and this one impressed me the most.

I couldn't help it and had to get the software the same day. However, as I had left my Canon SLR camera at home for this trip, I wasn't able to test-drive HDR imaging until I got back home today:

Obviously, this isn't a particularly exciting scene - I just shot from our balcony towards the end of the cul-de-sac. I used an automatic exposure bracketing of ±2 and loaded all three images into Photomatix and then played with some settings in the tone-mapping to create some vibrant and surreal colors.

But I still like the result quite a bit - it makes me want to go out and take some HDR photos of Marblehead harbor and experiment with other local scenes where the high dynamic range can come into play nicely.

Creating Open XML (OOXML) Spreadsheet Documents

As Office Open XML (OOXML) gains more wide-spread adoption and popularity - and since it is now an ISO standard - developers will be interested in how easy it is to create Open XML documents directly in their applications, e.g. spreadsheet documents that are compatible with Excel 2007. Most approaches require quite a bit of hand-coding and worrying about the actual OpenXML specifications, but what I want to show you today on the XML Aficionado blog is a way to use MapForce to auto-generate all the source-code (for example in C#) that will produce the desired .xlsx document so that you can integrate it into your applications (and use it royalty-free within your organization).

I will use a very simple example to demonstrate how you can turn some raw sales data in an arbitrary XML format:

into a pretty business graph in Excel 2007:

For such a simple use-case you could, of course, simply open the XML file in Excel 2007 directly, but I am only using a simple example to illustrate the process. The true power of this approach is that you can easily work with very complex data in a visual and intuitive manner - and that you can auto-generate the source-code to implement this as part of your application to automate such processes.

So let's open MapForce and insert the XML data file into our working surface where we are going to define the mapping:

Next we are going to insert an OpenXML spreadsheet document into the work surface of our mapping project - we can either insert an empty spreadsheet, or we can use an example document that we have previously created in Excel to indicate what sheets and what data ranges or labels should be receiving our data:

Now it is time to define how the source XML data should be mapped to the target OpenXML document. This particular mapping is just one example - MapForce lets you map between any combination of XML, relational database, EDI, flat-file (e.g. legacy text files), and OpenXML spreadsheet documents. In our case we are going to convert from start-date/end-date ranges in the XML source to months in the OpenXML document and from states to regions:

Once you've defined the whole mapping, this is how your project will look in MapForce - note that underneath the blue-gradient working surface the "Mapping" tab is the one that is presently selected, because I've just defined my mapping between the input and output files:

To test my mapping - before I auto-generate my program code, I can click on the "Output" tab underneath the working surface, and MapForce opens up Excel 2007 embedded within the same application frame to show me the result that is produced by my mapping:

This Excel table is then used to produce the graph that I showed earlier.

Now I want to auto-generate code in C# for my data integration project that will automate this generation of Excel 2007 OpenXML documents, so the next step is to check the code-gen settings to ensure that I generate it for the correct development environment - in my case Visual Studio .NET 2008 - but MapForce supports many other environments and can also generate code in C++ or Java in addition to C#.

OK, now we are ready to generate code. All that is required is using the corresponding command on the File menu, and all the source-code files are placed in a designated output directory, and the corresponding solution file for Visual Studio is generated as well:

The auto-generated source-code can now be integrated into any application and can be used royalty-free within your organization to automate the creation of Open XML (OOXML) spreadsheet documents.

If you would like to experiment a bit more with this example yourself, you can find all the files used here in the MapForceExamples directory when you download the free 30-day evaluation version of MapForce.

Also, keep in mind that you can use Excel 2007 files (or any other OpenXML spreadsheet documents) in MapForce both as input and output files, so you can create data integration applications and mapping or conversion code for any possible scenario that involves OOXML spreadsheet data, XML, EDI, or relational databases.

New BIG "minor" release of Altova tools

It's called Version 2008 Release 2, but in reality it should be a new major version. Our "problem" is that each year has 12 months whereas our talented engineers are practically cranking out a new major version every 5-6 months. So we have to call one of them the major release and the other one a minor release - but this one is BIG!

We've updated all the tools in the popular Altova MissionKit bundle with tons of new features and usability enhancements that our customers have asked for. I am most excited about the following, which provide big benefits to our users:

• Very Large File Support: XMLSpy 2008r2 contains a number of advanced optimizations for working with very large files. These result in a reduction of memory consumption by up to 75-80% compared to the previous version when opening and validating XML documents in Text View. This means that you can now open and work with files that are about 4-5 times larger than those supported in the past!!
• Extended Open XML (OOXML) Support: XMLSpy was the first XML Editor to directly support Open XML in April 2007 and today we are introducing more Open XML support in these products:
• MapForce 2008r2 now directly supports SpreadsheetML and allows the user to place any Excel 2007 document inside a mapping project to directly transform data from EDI, XML, databases, web services, and legacy text files to Excel 2007 and vice-versa. This new support for Open XML and Excel 2007 is, of course, also available in the automatic code-generation capabilities of MapForce, allowing developers to generate application code for recurring data transformation scenarios in Java, C# and C++.
• StyleVision 2008r2 now directly supports Open XML output in Word 2007 (WordprocessingML) to allow the user to generate multiple rich output formats from one single stylesheet design. StyleVision supports the generation of stylesheets via an easy-to-use drag&drop interface from XML documents as well as from databases and is the ultimate report designer that can produce output in HTML, PDF, RTF, and Open XML from one visual design. In addition, it allows developers the creation of Authentic forms from the same design to facilitate XML-based data entry across an organization with no deployment cost.
• DiffDog 2008r2 now supports detailed XML differencing between Open XML documents, including the ability to directly edit and merge changes across those files. In addition, the directory comparison feature now also supports ZIP file types so that directories and ZIP archives can be compared as well.
• Expanded Modeling Capabilities: UModel 2008r2 now supports the OMG's BPMN (Business Process Modeling Notation) and is also the first UML tool to ship full support for C# 3.0 and Visual Basic 9.0 - including accurate parsing of new language constructs in these programming languages that directly support XML. UModel does, of course, also continue to fully support Java 6.0 and provides full reverse-engineering and round-tripping for all the above languages.
• Better Integration Through Global Resources: developers using multiple Altova tools - for example as parts of the MissionKit bundle - can now take advantage of increased integration between these tools. The new Global Resources feature lets a developer define directories, databases, and ancillary files in one central location and those are shared between all applications. In addition, a developer can define multiple deployment scenarios (e.g. test, staging, production) for their XML projects, and also directly connect the output of one application to become the input for another.

The above list has just a few of the highlights that I find most exciting. More details and all the other cool new features can be found on the "What's New" page on the Altova website. There is also a press release being issued today about the new version.

I will also be covering some of these features in more detail on this XML Aficionado blog in the next couple of days - stay tuned...

LANSA middle-ware integration builds on MapForce

Here is a cool story about an Altova partner, who recently integrated the MapForce mapping and data transformation user interface into their product.

LANSA is a development environment and suite of eBusiness solutions that organizations use to rapidly implement business systems that make effective use of new technologies. From its beginnings as a 4th generation language and repository-based development environment, LANSA has evolved to a family of products and solutions that support IBM iSeries (AS/400), Windows, UNIX and Linux platforms.

LANSA Composer is built on top of LANSA Integrator, the company’s integration toolkit that offers bi-directional data integration through XML, SOAP, and Java services, on IBM System i and other middle-ware platforms.

At its core, LANSA Composer utilizes the MapForce application as its transformation component:

For more details, see the LANSA Case Study on the Altova website for further information. Also, this integration is getting great reviews in the press. For an example, read this article in Database Trends and Applications.

New York International Autoshow

I spent a day at the NY Autoshow this weekend with my son to take a look at the latest concept cars and the newest production cars from the world's largest and most exclusive automobile brands, and to check in on the progress of UIMA, an XML standard that has the potential to one day be used by cars to communicate with the road and with other cars around them.

Being a technology aficionado, I was very impressed to find more evidence of a move to carbon-fiber and aluminum construction evident with many manufacturers this year. Similarly there was are a lot of new developments with respect to electric and hybrid vehicles that could be seen in New York.

Here is a quick gallery of some of my favorite cars at the show:

Mazda Furai Concept Car

Saleen Raptor

Audi R8

BMW Concept Car

And my favorite of the entire show - and a great example of the use of carbon-fiber and aluminum in a high-end production vehicle - is the Aston Martin DBS:

Aston Martin DBS

Sadly, a few revolutionary cars that I would have liked to see were absent from the show, such as the Tesla or the Gibbs Aquada. And a few other high-end car manufacturers were also missing, for example the Koenigsegg or the Pagani Zonda.

But there was a flying car on display. Well, at least a prototype of one. You just can't have a car show without a flying car:

I was also looking in vain for evidence of any production applications of UIMA or other similar software approaches that would allow XML to be used by cars to communicate with the road and other cars around them. But that technology still seems to be a few years in the future before it reaches production vehicles, as UIMA is presently being standardized by and OASIS TC.

The exciting thing about UIMA is, of course, that it is not only based on XML, but being described by a specific domain model in XMI (which can be used in the UModel UML Modeling Tool), and contains specific Web Services descriptions in WSDL (which can be viewed in the XMLSpy XML Editor and utilized in our MapForce data integration tool). Last, but not least, there are semantic web implications with respect to knowledge integration in UIMA (which result in RDF and OWL information that can be viewed and processed in our SemanticWorks RDF and OWL editor). I will be following UIMA closely and will report on any progress in this XML Aficionado blog.

More photos can be found in this gallery on my photo website. And for more news from the New York Autoshow, please refer to the various in-depth reviews on Popular Mechanics...

Altova releases Version 2008 of the entire product line

I am very excited! This new release is packed to the brim with new features like XInclude and XPointer support, two-step XSL:FO transformations to allow use of XSL:FO with XSLT 2.0, evaluating XPath across multiple files, direct editing of database data, extended support for Office Open XML file formats, and speed and memory footprint improvements when working with large files in XMLSpy; aggregate data processing and value-map functions in MapForce; refactoring and modularization support as well as enhanced database functionality in StyleVision; Visual Basic .NET code engineering and real-time integration with Visual Studio and Eclipse in UModel; support for editing database views and stored procedures in DatabaseSpy; one-click directory synchronization and binary differencing in DiffDog; and much more.

We've really listened to our customers and implemented many of these features based on their direct feedback. I'm sure that you will find many of these enhancements and new features indispensible for your every-day work with XML, UML, and database technolgies. Make sure that you check out the MissionKit for Software Architects bundle, which includes all of our products in one convenient package.

In addition, you'll find common enhancements across the entire product line, such as upgraded support for Unicode 4.1, support for Widows Vista, an improved database connection wizard, and more.

For complete details see the press release, our What's New page, or simply download the free 30-day evaluation version. Also seen on the Altova News RSS feed.

Let me know what you think about the new features - post your comments directly on my XML Aficionado blog.