Index Management: From Excel to Electronic Studios

An index is a benchmark for evaluating the effectiveness of investment decisions. There are the large indices that everyone knows; S&P 500, Dow Jones Industrial Average, Nasdaq Composite, etc., and there are also agencies, big and small, that create their own indices. They gather the data from sources (a specific industry, or some idea) then they compose it, review the data, and publish the index centered around that idea.

Many entities have a huge number of indices and each index has its own specific rules. Each index goes through phases of ongoing maintenance where groups of analysts need to conduct very specific sets of actions in regard to the different components of a particular index. To do this manually, you’ll have to manage long excel files of data with numerous tabs to navigate and notes to follow with several people involved in layers of checks and rechecks. This substantial amount of work is unreliable and outdated as most of these maintenance steps can be fully automated. With an electronic index management system to automate the lion’s share of this tedious effort, analysts are left with only a final verification process to be responsible for.

An electronic index management system also helps with the utilization of similar index calculations. If there is only one interactive document, an analytic environment, that works the same way for all indices. Most calculations can be abstracted away to be reused on other indices as many use similar calculations and so, by creating a centralized calculation set of libraries, we can be certain that similar calculations are performed in a similar fashion. For example, if you are looking at the data you will most likely need volume aggregations and you’ll want to take a look at market cap. Most of these calculations are straight forward, so analysts can pull the raw data and then do those aggregations by hand or in excel.

A proper electronic index management solution will have a set of libraries that aggregate for you.

It removes the low-level work that nobody wants to do.

Say you want an aggregation for some fundamental; you specify the fundamental, specify the time period, you get the data and don’t have to calculate it on the fly.

Another step here is the weight method where you have a list of constituents and you have to assign a weight to those constituents according to some rule. These rules are also typical for the most part. And with convenience functions, if you say “I want them to sum up to 100 and prioritize them according to market cap” – you just call the function and it returns the set of weights.

If an agency wants to create an index. They have an idea. That is their product. They need the data now, they need a studio to work with that data, to form the idea, back-test it, to create a product that is reliable. With an Electronic Management System integrated with data streams, an agency could build an index themselves, with realtime data to calculate, and historical data to backtest their index.

Every Index Management Solution should contain the following:

1. Common data layer: otherwise you’ll have to pull from different sources which is tedious, unreliable, and the sources may expire or become inconsistent with each other. With consolidated feeds, those inconsistencies between sources are taken care of.
2. Monitoring services: If something goes wrong, then a management solution should be able to track it and act accordingly. For instance, there may be erroneous quotes coming from the market and this makes sure it doesn’t affect the calculations.
3. Workflow maintenance: rebalancing schedule, for instance, so you have to go back to your index from time to time to review the components and you have to do it by hand, but instead there will be a maintenance schedule. If once or twice a year you have an automated process that reboots itself and there is a report about what has changed, you can review it from a more thorough perspective. You don’t have to dig deep and do it from scratch by pulling the raw data and doing it all over again. Instead, you just make sure the results are correct and then approve them and the inconsistencies will be fixed and you’re off again for the next half-a-year.
4. Index Scheduling: An index is an artificial entity usually aligned to some set of instruments. Most indices like the S&P 500 are disseminated at one tick per second and an index management solution can take care of that calculation and set a schedule for the active hours usually aligned to some trading window. In addition, an electronic management solution should make sure the index has a fixed calculation schedule, which is determined e.g. by its constituents.
5. Calculation Engine and Distribution Infrastructure: Calculations themselves may be trivial or fairly complex. Most indices are based on simple arithmetic formulas and everything is basic math. That should be covered as well as derived calculations, such as the settlement price and charting. Ideally, the engine should combine symbols from a market data provider on to further more complex components like based on the current yield or rolling average in one environment.
6. Monitoring and alerts: An automated system that controls the quality of index distribution and reacts to rare events that require attention, such as corporate actions. As well as, a set of filters for recovering from occasional erroneous quotes or price manipulation attempts. 

Indices are valuable tools in estimating the economic health of an industry. In addition they are used for easy portfolio diversification and work as benchmarks for evaluating investment decisions. With advancements in technology, manual maintenance of indices is rapidly becoming outdated. An electronic index management system alleviates the workload required by this process and having a solution that contains all of the proper components will provide agencies with the ability to offer more reliable, and therefore more valuable, indices.

Based on the interview with Anton Antonov, quantitative analyst at dxFeed.