Monday, October 6, 2014

Extracting useful data from HTML pages with XQuery

When building in-house solutions or mobile enterprise applications, you are often faced with having to deal with legacy systems and data. In some ancient systems, the data might only be available as CSV files, in other cases it might be arcane fixed-length text reporting formats, but if the legacy system is less than 20 years old, chances are pretty good that someone built and HTML front-end and so the data is available through a browser interface that renders it in some poorly formatted HTML code that loosely follows the standard. And very likely you will find the data intermixed with formatting and other information, so extracting the useful data is usually not as easy as it sounds.

In addition, when you are building mobile solutions, you may sometimes need some government data that is not yet available in XML or another structured format, so you again are faced with having to extract that information from HTML pages.

Common approaches to extracting data from HTML pages, such as screen-scraping and tagging are cumbersome to implement and very susceptible to changes in the underlying HTML.

In this video demo I want to show you a better way of extracting useful and reusable data from HTML pages. In less than 15 minutes we will build a mobile solution that - as an example - takes Consumer Price Index data from the US Bureau of Labor Statistics, parses and normalizes the HTML page, and then uses an XQuery expression to build nicely structured XML data from the HTML table that can then be reused to build a CPI chart. I will walk you through the creation of the XQuery expression step-by-step so that you can easily apply this method to similar problems of HTML data extraction:



As you can see in the above video, it was fairly easy to create nicely structured XML data from a table in the HTML page and to create a first simple chart that plots the CPI data over time.

But the true power of this approach is that you have much more flexible charting capabilities in MobileTogether and the XML data is now reusable, so you can calculate annual inflation rates directly from the underlying CPI data and plot it as well.

Sunday, October 5, 2014

moto 360 Review

A while ago I wrote about my somewhat disappointing experience with the original Galaxy Gear, Fitbit, and Google Glass in an article "The (Broken) Promise of Wearables". It seems that I may have to revise my opinion a bit based on the new moto 360 smart watch:


First of all, I will admit that I'm a huge watch aficionado and have a collection of several beautiful mechanical timepieces and complications, as well as functional sport watches. So the Galaxy Gear  just hurts from a design perspective - both in its original form as well as the Gear 2 and the new Gear S. I have also been less than impressed by the new Apple Watch. Despite all the claims by others that it is beautiful, in my eyes it has the same flaw as the Galaxy Gear: the watch is square. Most display screens are rectangular, so they just built a watch around a square or rectangular screen.

However, there is a reason that the majority of watches have evolved with a circular dial. It is the most comfortable to wear, because it doesn't limit the movement of your wrist. And it has a timeless elegance to it.

Wednesday, October 1, 2014

Building a stand-alone mobile solution with MobileTogether

In a recent blog post I introduced our new MobileTogether platform for building mobile in-house solutions. Today I would like to give you a little demonstration of how easy it is to build a mobile solution with MobileTogether Designer.

As an example, we're going to build a simple tip calculator app for your next restaurant visit. Since this particular solution doesn't need any back-end data, we're going to create it as a stand-alone mobile solution so that it can be used even without a server connection once it is deployed.



As you can see, it just took about 8 minutes to build this app. MobileTogether lets you focus on what is really important, and handles everything else for you. If you want to try for yourself, you can download MobileTogether Designer here.

You can also watch more MobileTogether Designer video demos here.

Friday, September 26, 2014

Pixelstick

In December last year I contributed to the Kickstarter campaign for Pixelstick - an interesting new photographic tool for light painting. When our Pixelstick arrived in early September, it was immediately clear that it would need to go to New York with my son when he went back to college.

Calvin is a photography student at NYU's Tisch School and so I knew he would put the device to some creative use... and indeed he just posted a "How to" video on YouTube:



For more information on Pixelstick go to www.thepixelstick.com

For more updates on Calvin's work, follow him on Twitter @EpicFalkon or check out his website discover.calvinphoto.pro

Wednesday, September 24, 2014

XQuery Update Facility in XMLSpy

A really cool new feature in XMLSpy 2015 is the interactive XQuery Update Facility support, which lets you make changes to XML instance documents in a programmatic way - using XQuery statements - that exceed the typical regular expression based Find/Replace capabilities by far. The XQuery Update Facility specification provides a mechanism to insert nodes, delete notes, and modify nodes within instance documents. In XMLSpy 2015 you can now apply these updates either to the current file, all open files, all files in a project, or to entire directories.

This video explains the most important XQuery Update Facility commands and demonstrates how easy it is to put the power of XQuery Update Facility to work for you:



The new XQuery Update Facility support is one of the many new features introduced in the new Altova version 2015 product line last week, which includes new versions of XMLSpy, MapForce, all the other MissionKit tools, and all Altova server products.

Tuesday, September 23, 2014

Introducing MobileTogether - Build mobile in-house solutions faster

It's about time that I start talking about our next major product here: Altova MobileTogether is an exciting new cross-platform mobile environment that lets you build in-house mobile solutions for your workforce much faster and more productively than any other mobile cross-platform method out there. You can use MobileTogether to bring your in-house data — be it in SQL databases, XML, available as web services, etc. — to your employees on the device of their choice, be it business intelligence dashboards, elegant enterprise forms, or any other business processes from graphs for sales analytics to monitoring of business-critical data.


There are, of course, many ways to develop mobile solutions, and for external customer-facing apps the native platform approach or other multi-platform SDKs may make sense. But for in-house solutions the math just doesn't work. You need to be able to build these in a few days rather than weeks or months in order to stay on budget.


Monday, September 22, 2014

New XBRL Formula Editor in XMLSpy

I'm very excited about all the new features in XMLSpy 2015, and in particular about the new XBRL Formula Editor, which now lets you build formulas more intuitively using a XBRL Table Linkbase layout preview. The XBRL Table Linkbase specification provides a mechanism for taxonomy authors to define a tabular layout of facts. The resulting tables can be used for both presentation and data entry. In XMLSpy 2015 we use these tables as a way to specify variables in XBRL Formula definitions.

However, XBRL Table Linkbase is a fairly young specification, so not many published XBRL taxonomies include Table Linkbase definitions yet. Please see my previous blog post, where I have demonstrated how to add a Table Linkbase to an existing XBRL extension taxonomy.

Once we have a taxonomy with a Table Linkbase attached to it, we can then proceed to create some assertions or calculations using XBRL Formula Editor. In this video, I will demonstrate how to do this using a recent SEC filing as an example instance:



The new XBRL Formula Editor is one of the many new features introduced in the new Altova version 2015 product line last week, which includes new versions of XMLSpy, MapForce, all the other MissionKit tools, and all Altova server products.

Sunday, September 21, 2014

XBRL Table Linkbase Editor and Layout Preview

This week we launched our new Altova version 2015 product line, including new versions of XMLSpy, MapForce, all the other MissionKit tools, and all Altova server products.

One of the cool new features in XMLSpy 2015 is the real-time XBRL Table Linkbase layout preview. The XBRL Table Linkbase specification provides a mechanism for taxonomy authors to define a tabular layout of facts. The resulting tables can be used for both presentation and data entry.

However, XBRL Table Linkbase is a fairly young specification, so not many published XBRL taxonomies include Table Linkbase definitions yet. This is where XMLSpy can greatly help: in this video I will give you a quick demonstration of how to add a Table Linkbase to an existing XBRL extension taxonomy, using an XBLR filing that was submitted to the SEC as an example:



Learn how the graphical XBRL Table Linkbase editor in XMLSpy makes it easy to define XBRL tables for the presentation of multi-dimensional XBRL data. You can determine whether your table produces the desired results in the real-time XBRL Table layout preview, which is new starting in XMLSpy 2015.