Data, data everywhere, not a byte to use
As we move further into an era where speed is (almost) everything and trading firms and platforms alike are trying their best to improve their latency, one facet often does not get the attention it deserves. That is data management. Practices such as program trading and high frequency trading (HFT) put pressure on the ability of firms to handle the relevant information in the market for trading, something the orders being generated in the market are a part of. Recent studies have shown that the amount of data being generated has gone up by 40-50% in the last couple of years. The short-term strategies of different types of firms, including prop traders and hedge funds, which require continuous placing, amendment and cancellation of orders at a rapid pace mean that there is much more market data that needs to be handled than ever before. In developing markets, it is not just the requirements of HFT, but also the fast rising volumes in the exchange-traded derivatives markets, that are posing a challenge for the market participants. Growth rates for market volumes of more than 50% are not considered very surprising. Similarly, the volatility in the market due to the financial crisis earlier and now due to the European debt crisis means that these pressures are going to rise globally. On a firm level, the move towards multi-asset trading in a multi-market environment also means that more data is being handled than ever before.
We have discussed the various reasons for the rise in the data volumes. Now the question is whether there is any way of handling this data growth. Different firms are trying to come up with different solutions regarding how to efficiently manage their data requirements. In the leading markets in the US and Europe, complex event processing (CEP) has increasingly become a means of meeting a firm's requirement of processing large volumes of data and analyzing it for patterns. CEP has widespread uses such as testing, trading, risk management, compliance and market surveillance. It is important that regional players in markets such as Asia-Pacific which have not considered CEP till now look at it as an option. It is fast becoming an integral part of the landscape of low latency trading and will give its users the confidence to handle the continuous growth in data volumes in the future as well.