Introduction
In this article, I would like to share some experience about building a financial information service. On the above diagram, the data is flowing from left to right, from exchange data source or other data providers for example: Reuters, Bloomberg, to the backend system which is in the middle of the system and named as a data lake. Finally the data will be passed to client UI running on desktops or mobile phones.
This system is constructed by multiple layers. The bottom layer is the infrastructure serving the communication purpose. On top of this layer, that is the network layer. Multicast communication has been using in the data lake for data transmission internally. Multicast high availability is implemented. For the data receiving from exchange or data vendors, the protocol is depended on the providers. For the output of the data lake, data bridge will be used if the distance is too far for example a cross continent communication.
The third layer of the system is a private protocol which is used to normalize the data so that we can use the data in different format. This layer is also used to encapsulate the client from the data source so that data changing in the exchange will not affect the client UI applications.
Multicast high availability
This is one of the key feature that is supporting the system so that it can perform green blue deployment and high availability in a multicast environment.
Here is an article discussing the implementation. link.
Data encapsulation layer
For a big system, encapsulation is important since it can break out a big system into multiple modules so that it is easier for it to adapt changes from outside world.
As I stated in the introduction, there is a data encapsulation layer in this system. To implement that, we picked a distributed data cache and a normalized data structure for communication.
A similar idea is used in the other project and here is more details --- link.
Client UI
With the financial information service, the data will be converted into a general format so that different client UI can read it and render their own presentation.
Comments
Post a Comment