News Stay informed about the latest enterprise technology news and product updates.

Data services mashups emerge for SOA

Kirstan Vandersluis, founder and chief scientist for XAware Inc., the open source data integration vendor that offers tooling for creating a data services layer, works on the issues surrounding the creation of data services mashups for service-oriented architecture (SOA) applications. He explained what data services mashups are, the standards such as XML Schema, and industry applications in an interview with SearchSOA.com conducted earlier this month at JavaOne.

Can you explain what a data services mashup is?
It's a mashup in terms of pulling data from multiple sources into a logical unit. So it's a mashup in terms of data. It's not a visual mashup. That's a whole different ballgame. We're pulling data from virtually anywhere into a logical unit. We're trying to take a bunch of backend data systems, which are typically very complex, and make it look like something much more rationalized in terms of an XML Schema that somebody put some thought into designing. When you take all the data sources and put them together what kind of business functionality do you end up with?
In the end, you're going to have an XML data set conforming to a schema or you can invoke the service and basically get the data back. Can you give us an example for something like a travel company?
For a travel company you might be interested in seeing different views of a customer based on their travel history. So the website is going to need information. There's the GUI piece that we don't get involved with. The scenario would be that the front end application is going to make a request and say: "Get customer view. Here's the customer ID." Then the data mashup is going to essentially be a combination of data from multiple sources. There's going to be a CRM system in there, so they are going to look at the basic customer profile from the CRM system. There will be a query to the travel management system to get trip history. As that data comes back, it's probably some kind of raw format. Maybe it's a flat file from the CRM system, maybe from an API. The travel management system might be a mainframe and it's coming back over a queue. Then that raw data is automatically converted into XML data sets. It's those XML data sets that get combined into the larger structure, which is the customer view. That's what is returned back to the application. Is there any comparison between a data services mashup and what can be done with data warehouses and data marts?
It's just a different way to do things and there are trade offs. We're certainly not saying that we're replacing data warehousing. You can apply it in a similar application. The one benefit to our approach is it's all real-time. We're sending the sub-queries out in real-time to whatever systems you want. To that extent you could be incorporating data warehousing technology into your mashup, but very often some portions of the data need to be more real-time. That's where the advantage is for data mashups. You can query operational systems directly. It is to some extent driven by the requirements of the application. If you need real-time data, that's what we do. You can query any system, pull the data back and combine it. Being an abstraction layer is also a good thing because you can build your applications according to the schema. Then you can make changes over time without affecting the applications. The applications are relying on the contract, which is defined by the XML Schema. So if initially you pointed all your data services at your data warehouse, and you find out later that's not real-time enough, you can change the data sets that are being mashed up, so one of them hits an operational data source. What is the job description for people using your tool? Is it a data architect? Who actually does this work?
Generally it is either a data architect or a Java developer, someone who is familiar with the data. Certainly the data sources that are relational, any data architect or DBA is going to be comfortable in that scenario. The non-relational data sources that are a little more complex, maybe that's a Java developer, but a data architect very often is a typical scenario. Is this skill set something you find in the typical data architect's toolbox?
Yeah, I find that. Generally, we get very good feedback from people in that kind of position. We get very good feedback from data architects because it is very visual. We expose the metadata whenever that's available to us. Then it's just a matter of drag-and-drop, so I think it is well within their skill sets. Generally, they think it's better than what they've had to do in the past. Do you find that the people on the business side understand data services? Do they understand the value of it when you talk to them?
I think it's spotty. Some do. Some don't. I'd have to say the majority maybe don't at this point, but I think it's more a matter of how well they understand service-oriented architecture, which is the typical use case of a data services layer. To the extent that they've bought into service-oriented architecture, they understand a data services layer. I'd say the majority of them don't. Some of our customers who deal with the business people are putting a lot of energy into enlisting the business side into their SOA strategy, but they are having moderate to limited success. In terms of data services adoption within SOA where are we at in the curve? Are you finding a lot of interest, or actual application?
It's definitely growing. We've had 50,000 downloads. That's just in the last two months since we've had a GA product. So there's a tremendous amount of interest there. Of course, in a two-month time span people are not going to go from conception to deployment. To that extent it's still early on, but I think data architects and developers understand it in terms of the SOA side. Then the other big area where a data services layer applies is the rich Internet application side where screen sections using Ajax are now being populated by services. That does appear to be a different area in terms of granularity of the data. It's an area where you're going to want highly granular data and many, many services, rather than the SOA side, which ultimately should be driven by the business and services should be defined by the business. That typically leads you down a road where you have very coarse-grain services, but I see it growing in both areas. Are there standards that data services still need to mature or do you have the standards you need?
Our world is very much centered around XML Schema. So that is a key standard, building services conforming to XML Schema data types. Then there are standards around the invocation of a service: WSDL and SOAP standards and REST. How about the industry specific standards like ACORD for insurance, are those standards mature enough?
I think ACORD is in very good shape. The insurance industry embraced XML and XML Schema probably more than any other industry and for a longer time. They've been working on ACORD since shortly after XML was announced. Maturity does vary depending on the industry, but insurance in particular is very mature. In terms of your focus on XML, is the XML data pretty clean or do you run into problems doing data services because organizations aren't always adhering to the standard?
From a standards perspective it's pretty clean. The problems you run into are from a customization customers want to apply on top of the standards. ACORD is one standard that has taken extensions into account. Every major information group has an extension area for customers to plug in. Of course, the more you customized it, the more you run the risk of not being able to interoperate with other companies. From a standards perspective XML is pretty clean, but there certainly are customizations out there that inhibit interoperability. But those are all design considerations that people have to deal with as they are designing their application.
For more information
Making sense of data services mashups

Learn more about SOA data integration
Are there issues when an application has to cross industries, such as needing data from insurance, financial and healthcare systems?
There are no issues there. You can create services that can conform to any schema. It can be any number of schemas translating into any number of data services. What are the worst practices in data services development that can really cause problems?
Well, bad XML Schema design is an issue. Sometimes people don't even design a schema. When they start with an XML instance it's not very logically organized. What that leads to is having to make changes throughout the design cycle. If your organization is not very logical then you have trouble mapping to your backend systems. There's always going to be some impedance mismatch between your logical view and the actual physical structures. So you can help yourself out a lot by making sure your logical structure is logical and well organized.

Dig Deeper on Development implications of microservices architecture

PRO+

Content

Find more PRO+ content and other member only offers, here.

Start the conversation

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

-ADS BY GOOGLE

SearchSoftwareQuality

SearchCloudApplications

SearchAWS

TheServerSide.com

SearchWinDevelopment

DevOpsAgenda

Close