SmartStream published the findings of a new survey showing the growing importance of reference data investments for financial firms and the strides being made in data standardization and automation.
Participants were surveyed from across the derivatives market, including: hedge funds, proprietary trading firms, energy traders, bank and non-bank futures commissions merchants (FCMs), and brokerages. Results indicate that in the last five years firms have made significant investments or changes in managing and processing reference data for trade data management, with automation emerging as the primary driver.
Key findings from the report:
- 71% of firms have invested in reference data during the last five years
- Standardizing external data remains a significant challenge, cited by 46% of respondents
- 56% of respondents reported significant reductions in manual processes through automation, with notable improvements in reconciliations and trade workflows
- 24% of firms have fully automated their reconciliations systems, however this area continues to pose challenges despite recent technological advancements
- AI adoption is on the rise, with 25% of firms already leveraging AI or machine learning for data management, with 30% planning to integrate these technologies soon
Linda Coffman, executive vice president at SmartStream RDS, said in a statement: “Despite progress, challenges do persist, with fragmentation and complexity – when it comes to integrating data from multiple vendors and addressing the inconsistencies of external counterparties and exchanges. That said, the adoption of AI is transforming how reference data is utilized, driving more sophisticated applications across the trade lifecycle. This reflects a shift toward continuous investment in reference data, moving away from one-off major upgrades”.
The report highlights an increasing trend towards artificial intelligence (AI) and machine learning, with a quarter of firms actively leveraging these technologies for reference data management and just under a third planning to do so. AI adoption is still in its early stages, but momentum is building for innovative applications of reference data. While initial costs and cautious approaches remain barriers, these are expected to diminish as adoption scales, unlocking broader use cases and efficiencies across the financial sector.
Space has opened up to explore new use cases for reference data that feed into alpha generation. While still a trend in its nascent stage, there is already significant innovation taking place in how firms marshal reference data. Transaction cost analysis has been the most popular use case so far, but firms are also using reference data for decisions on trade selection.