Use Case | Book Replay
TickSmith’s Book Replay Solution optimizes National Bank’s algorithms and trading calculations and enables quick access to data from specific marketable points
The Global Equity Derivatives Group at National Bank is a high frequency trading firm and a leader in market making of exchange-traded securities such as stocks, exchange traded funds, futures and options. The bank collects and processes a fast-growing volume of stock-market financial data, such as trade and quote history information. In order to keep up, the bank required an analytics platform that could scale to match their data growth and intake, while procuring a more efficient way to analyze the financial data.
“We wanted to optimize our system so that we could make it as fast as possible,” says Alfredo Villar-Sbaffi, Director at National Bank. “Basically, the way we make money is by being fast. We use the data for trade auditing- tracking and analyzing trades, to calibrate our modelability, for compliance, and for backtesting.”
Processing the raw data for different uses proved to be a major challenge for the bank. “Before, we would basically process the raw data ourselves,” says Villar-Sbaffi. “The problem was that it was time consuming. There were a lot of issues that were breaking, and no one had the time to maintain it. We didn’t need to access the data daily, but when we needed it, we had to spend time to fix the bugs before we could use it.”
It was obvious that the bank had outgrown their outdated on-premises IT environment, and were in need of an updated big data solution to compete in today’s capital markets.
National Bank chose to work with big data solution provider TickSmith, rather than building a new solution in-house. “When we learned about TickSmith, we noticed that they were a good match for us, in terms of the experience they had and the needs we had to fulfill our duties,” adds Villar-Sbaffi. “They take the raw data that we record, which comes directly from the exchanges and they normalize it. They put it in a more ‘human-friendly’ and ‘reader friendly’ format. In the past, we did the job ourselves, but it wasn’t a very robust process we had in place.”
- Cost effective infrastructure, only up when necessary
- Zero Vendor locking: Open source formats of Parquet/Avro
- Transparency on how the data is treated from the original raw form to curated form
- Data quality as a metric for confidence indicator of the ingested data
- All data is available to the client (SQL, Query API, Spark)
- In web portal
- First-class entitlement system: decide who sees what data
- Visualize and browse trading history per symbol
- Shows updated statistics/analytics at any point in time
- Works with any source of data
- Supports Level 1 and Level 2 data
- Can combine data from multiple sources into unique books
“We have engineered our platform to do all the dirty work,” says Tony Bussieres, CTO of TickSmith. “The data is prepared in a way so that it’s cleansed, normalized and the behaviour of the data is consistent to the end-user. Our client doesn’t have to deal with the cloud or maintenance. They can focus on doing what they do best- the analytics and algorithm testing.”
Privacy and security are also a priority for National bank, who manage more than $219 billion (CAD) in assets. Therefore, one of the requirements was that the data remain authentic and transparent. “We’re running our Book Replay solution on their data, so they know the source of the data and can derive what they need from it,” says Bussieres. “They don’t have to tell us their secret sauce calculations or algorithms.”
TickSmith’s Book Replay solution allows National Bank to access the data in a straightforward, web-based, point and click application using a configurable interface. The platform is adaptable so it can access data from any exchange or a combination of exchanges in any programming language. “All the exchanges that provide us with data have their own protocol,” adds Villar-Sbaffi. “Previously, when an exchange would change their language, we would need to readjust our code to reflect those changes and our system would just break down.”
The platform is deployed on AWS for National Bank, who were one of the first organizations to migrate into cloud territory. “Going to cloud was something new for us. TickSmith was the expert, and they helped us set it up,” says Villar-Sbaffi. “We had questions about costs, but the cost was right in comparison to what we were doing in-house. We also needed the scalability and the on-demand aspect. For example, we’re not backtesting every day, so maintaining a bunch of in-house servers that were idle most of the time was not a cost-effective set-up.”
Among the many benefits of using AWS is the ability to integrate open-source software along with their offerings. For example, the flexibility of AWS allowed TickSmith to fine tune the installation of Kafka through AMI, autoscale through scheduling, and to separate storage and compute.
Kafka, an open-source streaming platform, was recently installed to enable near real-time live streaming ingestion. Being elastic, which is the ability to run specific storage machines when needed, would result in a more efficient and faster workflow. National Bank benefited from this implementation with further cost savings, along with the confidence in knowing their cloud platform is future proof.
The final product is an end-to-end platform solution that includes ETLs, business logic and analytics, workflow process monitoring and data delivery. National Bank is able to use it to centralize terabytes of data received from quote feeds, trading systems, event feeds and more. The open platform gives them easy access to their data, to produce analytics and time-series via accessible interfaces and APIs.
As part of National Bank’s strategy, it is critical that they maintain control of the data source because their algorithms are tested using this data. The purity of the data will help the bank to make crucial decisions and send timely orders to the market. “Our Analytics platform allows National Bank to use their own data instead of having to deal with all the different exchange feeds,” says Bussieres. “They can trust their own data, and if they want, they can reverse engineer it to get results pertaining to their business.” Thanks to TickSmith’s agile platform, and with the help of artificial intelligence and algorithms, the process of analyzing large volumes of data is no longer a headache. “One of our needs in this field is to be able to replay data from the past,” says Villar-Sbaffi. “Our in-house solution was breaking down often, and something as simple as getting to a specific marketable point- like a certain day and time- just getting there would take us several minutes, or even hours. Now it’s almost immediate.”