The European Securities and Markets Authority (ESMA) is responsible for regulating how financial services firms within their jurisdiction collect and report market and trading data. ESMA uses a legacy system to handle the large data volume generated by reporting as required under MiFID II and other regulations. The regulator is now reportedly planning to upgrade this operation by using Microsoft’s Azure cloud platform.
What might get corrected when ESMA completes this change?
There are several issues that may be corrected when ESMA completes this change, if the experiences of industry firms who have gone through the same transition are any indication. Many legacy data systems were not designed to handle the volume and variety of data that has become the norm today and therefore can be plagued with problems in their overall design and architecture. This in turn overwhelms in-house resources and erodes data quality. Legacy data systems are also often less agile and unable to support predictive analytics.
Globally, billions of shares are traded daily. When other required data is included – client documents, market data, data management coding, etc. – this amounts to terabytes, petabytes, exabytes or possibly even zettabytes (a trillion gigabytes or a billion terabytes) of data. These volumes are way beyond what an in-house, on-premises legacy data system are likely to be able to handle.
As just mentioned, the different types of data also complicate matters. Different cloud services and different APIs – even if they are doing the same function – can generate data in varied formats, which will require integration. This would be another data processing function piled on to a strained and overburdened in-house system.
Also, data processing is now a real-time game (and already has been for years). To keep up, a company would basically need its own Google Cloud, Microsoft Azure or Amazon Web Services at this point. If a financial firm is counting on real-time prices or market data to keep up with business, the average in-house resources will not be sufficient on that front either.
Aside from legacy data systems, legacy data itself – meaning data coming from legacy applications or previously existing databases – can cause other problems. These include data quality challenges, database design issues, data architecture and process challenges. Data quality issues can come down to something as simple as inconsistent formatting of values and spreadsheet columns. Design issues can include lack of naming conventions or documentation. Data architecture problems can occur if different in-house systems need to interact and each has a different design, formats or any one of numerous characteristics. Without proper attention, legacy data issues can lead to faulty data processes when working with legacy data or legacy data systems.
So, how well can Microsoft Azure address ESMA’s needs – as opposed to other major cloud platforms?
The main factors that might lead any company or entity to choose Azure over AWS are that they already are using other Microsoft products for their operations – making Azure easier to integrate, or Azure’s convenient combination of data integration and storage functions. Azure also appeals to those seeking a platform-as-a-service (PaaS) format rather than the infrastructure-as-a-service (IaaS) approach of AWS and AWS’s variety of tools.
This doesn’t necessarily mean Azure is better or best for every company, but could indicate that ESMA’s operations already used Microsoft technology or were better suited to a PaaS approach.
Analysis
Typically – and globally – financial services regulators have often found themselves playing catch-up to the industry they regulate. Whatever the outcome of ESMA’s plans, its decision to drop an outdated legacy system, at minimum, puts it on the same playing field. Having the capability to more efficiently process relevant market data will make the regulator even more effective in its oversight.