Open-source-based social-networking website kaChing is using Xignite financial information services and some Amazon cloud computing cycles to power a virtual stock market. By working with Xignite rather than traditional market data providers, kaChing saved hundreds of thousands of dollars and months of development time, said Pascal-Louis Perez, CTO of kaChing.
KaChing's vision has been to create an environment in which hundreds of thousands of individuals can attempt to prove their investment savvy. There is a crunch time: kaChing must calculate the value for close to 400,000 portfolios at the close of every day's market. Working with Xignite, it was decided that this large task could be efficiently uploaded to the much discussed Amazon Elastic Cloud for processing as needed.
Services run the kaChing environment. The architecture is said to assure that every subsystem communicates with other subsystem as a service. Perez said that this architecture makes it easy to integrate with services like Xignite.
"We really like decomposing our problems into services and having multiple services that can solve lot of different problems," he said. "It is good to scale. People focus on one service at a time."
Financial market data turns out to be an ideal candidate for the elastic nature of Amazon Web services (AWS) because of the significant fluctuations in traffic levels. "We are looking at writing a lot of our services so we can move them from our own servers to Amazon and back depending on demand," said Joel York, CMO at Xignite.
Xignite began working with AWS in late 2008 when Forbes.com selected Xignite to deliver real-time stock quotes. Using AWS, Xignite reduced time-to-market for the Forbes deployment to only two months and saved personnel costs by eliminating the need for a full-time operations engineer to procure and install servers at an additional collocation space.
One of the challenges was incorporating the AWS platform into the development cycle. York said, "From an engineering standpoint, if you are a software provider, you need to integrate that platform into your release and development cycles when you do the testing and final imaging."
"The other thing is that a lot of the tools for managing what you are doing on the cloud are in the early stages," he continued.
AWS as fit for kaChing
In the kaChing instance, AWS is used for both storage and computation. Some applications only run on the cloud, while others only run locally. The big task is to calculate the value for portfolios when the market closes.
"Doing this computation in the cloud is great because we can turn on the cloud, perform the computation, and then turn it back off," said Perez. "We are trying to move more and more into that model, although we are still far away from being able to fully use the elasticity of those cloud computing systems."
Other applications that deal with managing individual accounts or mirroring successful managers are only done in-house owing to heightened security concerns.
The move to AWS was a natural fit for kaChing because of its reliance on a network of low-cost machines. Perez explained, "We use unreliable hardware and inexpensive machines, so the shift to having lots of little machines in the data center to lots of machines in the cloud is similar. This makes it easy to write and move applications between kaChing's servers and the cloud."
"It is much different for people used to building solutions like big Oracle databases running on a $100,000 machine. The main problem is decomposing the problem into something you can do on lots of machines," said Perez.
The financial services industry has been getting more and more real time, particularly with the advent of the Internet and more computer-based algorithmic trading, which requires low latency on the order of 3 milliseconds (ms). For most Web services, in which transactions are carried out by humans, delays of up to a second are considered real time.
York said, "What is going on with market data is the same as in the music industry where the originators of the data have less control. "Traditionally the originators of the data were paid for every user on every data feed. But smaller publishers that provide free services like Forbes and kaChing need a more cost effective pricing model for the consumer Web.
Real time market data has only traditionally been available through service providers like Reuters, which provide market feeds consisting of trillions of transactions per day. But this data is both expensive and difficult to integrate into other applications. Still a lot of information can be handled outside of the sternest performance bounds. Xignite has developed an SOA architecture that encapsulates over 50 different categories of data as a service.