MobileTogether

Extracting useful data from HTML pages with XQuery

When building in-house solutions or mobile enterprise applications, you are often faced with having to deal with legacy systems and data. In some ancient systems, the data might only be available as CSV files, in other cases it might be arcane fixed-length text reporting formats, but if the legacy system is less than 20 years old, chances are pretty good that someone built and HTML front-end and so the data is available through a browser interface that renders it in some poorly formatted HTML code that loosely follows the standard. And very likely you will find the data intermixed with formatting and other information, so extracting the useful data is usually not as easy as it sounds.

In addition, when you are building mobile solutions, you may sometimes need some government data that is not yet available in XML or another structured format, so you again are faced with having to extract that information from HTML pages.

Common approaches to extracting data from HTML pages, such as screen-scraping and tagging are cumbersome to implement and very susceptible to changes in the underlying HTML.

In this video demo I want to show you a better way of extracting useful and reusable data from HTML pages. In less than 15 minutes we will build a mobile solution that - as an example - takes Consumer Price Index data from the US Bureau of Labor Statistics, parses and normalizes the HTML page, and then uses an XQuery expression to build nicely structured XML data from the HTML table that can then be reused to build a CPI chart. I will walk you through the creation of the XQuery expression step-by-step so that you can easily apply this method to similar problems of HTML data extraction:



As you can see in the above video, it was fairly easy to create nicely structured XML data from a table in the HTML page and to create a first simple chart that plots the CPI data over time.

But the true power of this approach is that you have much more flexible charting capabilities in MobileTogether and the XML data is now reusable, so you can calculate annual inflation rates directly from the underlying CPI data and plot it as well.

In this next video demo I want to show you just how to do that in less than 10 minutes. We will add a year-range selector to our chart where we can define which years to plot, and we will add an overlay chart that derives the annual inflation rate based on the underlying CPI data using XPath calculations and the plots that data:



Using this technique, you can not only extract data from singular HTML pages, but easily build a modern mobile front-end experience for many legacy systems that just offer an HTML-based browser interface at present. This will enable you to make your workforce a lot more productive and efficient, as they can now use a friendly mobile app experience to access your system rather than having to deal with a couple of HTML pages and forms in a browser on their tiny smartphone screens.

Building a stand-alone mobile solution with MobileTogether

In a recent blog post I introduced our new MobileTogether platform for building mobile in-house solutions. Today I would like to give you a little demonstration of how easy it is to build a mobile solution with MobileTogether Designer.

As an example, we're going to build a simple tip calculator app for your next restaurant visit. Since this particular solution doesn't need any back-end data, we're going to create it as a stand-alone mobile solution so that it can be used even without a server connection once it is deployed.



As you can see, it just took about 8 minutes to build this app. MobileTogether lets you focus on what is really important, and handles everything else for you. If you want to try for yourself, you can download MobileTogether Designer here.

You can also watch more MobileTogether Designer video demos here.

Introducing MobileTogether - Build mobile in-house solutions faster

It's about time that I start talking about our next major product here: Altova MobileTogether is an exciting new cross-platform mobile environment that lets you build in-house mobile solutions for your workforce much faster and more productively than any other mobile cross-platform method out there. You can use MobileTogether to bring your in-house data — be it in SQL databases, XML, available as web services, etc. — to your employees on the device of their choice, be it business intelligence dashboards, elegant enterprise forms, or any other business processes from graphs for sales analytics to monitoring of business-critical data.


There are, of course, many ways to develop mobile solutions, and for external customer-facing apps the native platform approach or other multi-platform SDKs may make sense. But for in-house solutions the math just doesn't work. You need to be able to build these in a few days rather than weeks or months in order to stay on budget.


MobileTogether makes this rapid development possible by using a unique system architecture that consists of the following three components:


MobileTogether Designer

MobileTogether Designer is the IDE where you build your mobile solutions. It comes with full database-support for all major database servers as well as the ability to connect to any XML files, web services, HTML pages, or other data sources directly. If that's not enough, you can connect to FlowForce and MapForce server as an interim data transformation platform to get your data from EDI or other formats into XML easily.


Once you've defined your data sources, you then drag & drop UI controls onto your design surface and connect them with the data model. You can build powerful program logic using visual ActionTrees and for data manipulation the full power of XPath and XQuery is at your disposal.

This way you can build a powerful mobile solution in just a few hours and can test it right inside MobileTogether Designer using the built-in simulator that supports the iOS, Android, Windows Phone 8, and Windows 8 look&feel as well as many screen sizes and device options.

We have put together a few video demonstrations that show how easy it is to build a mobile solution with MobileTogether Designer.

MobileTogether Mobile App

Once you are satisfied with the way your mobile solution looks, it is time to get it onto your mobile device. The MobileTogether Mobile App is what runs your solutions on your device, and it is available for free form the respective app stores. The Mobile App is available for iOS, Android, Windows Phone 8, and even desktop Windows 8, so you can deploy your solutions to all mobile workers, no matter if they prefer a smartphone, tablet, or laptop!


Normally the MobileTogether Mobile App connects to a MobileTogether Server (see below), but for an initial trial run, you can simply connect the MobileTogether Mobile App directly to your MobileTogether Designer. Instead of starting the simulator in the designer, you select "Trial Run on Client" from the toolbar, and then you can see your mobile solution on your device and test it there, provided your mobile device is on the same local network as the computer where you are running MobileTogether Designer.

MobileTogether Server

Once you're ready to deploy your solution to your entire workforce, it is time to install and configure MobileTogether Server. This server acts as a conduit between your mobile clients and your database servers and other data sources in your IT infrastructure.


If you only want your mobile solutions to be available while the mobile devices are connected to your corporate Wi-Fi network, then it is sufficient to install the server in-house and your employees can immediately run your mobile solutions, once you've deployed them from the Designer to the Server.

If you want your employees to also be able to access your solutions while you are on the road, you will need to designate and open a port in your firewall so that the client devices can reach your server from the public Internet when they are traveling. We recommend installing an SSL certificate for that purpose so that the data connection between the clients and your MobileTogether Server is encrypted. In addition, we recommend securing the MobileTogether server with user authentication. You can choose between built-in user management, or the MobileTogether server can talk to your Active Directory server to integrate with your enterprise user management.

Alternatively, you can also install MobileTogether Server into a private cloud with a cloud provider of your choice, if you prefer to have your server running in a cloud rather than on your on-premises infrastructure.

Timeline

In May this year we first introduced Altova MobileTogether at TechEd in Houston, TX. In July we launched beta 1 of the MobileTogether Designer. In August we launched the beta 1 version of all the MobileTogether clients in the respective App Stores. And last week we launched beta 2 of MobileTogether Designer, Server, and the Apps. We expect MobileTogether to be commercially available later this fall.

Getting Started

You are invited to participate in the beta 2 and try it for yourself.


Just download the MobileTogether Designer from our website, download the MobileTogether Client from the App Store on your device, and you can be up and running and have your first solution on your device in 1-2 hours. Then, when you want to scale out to have your colleagues run it on their devices, you can download MobileTogether Server so that others can connect to it.