3D Monitoring
A lot has changed over the last few years in the monitoring tools space. The fast pace of technology innovation has demanded quick alignments of monitoring tools to ensure that new technology can be properly monitored. It also introduced opportunities to use these innovations in the development of the monitoring tools themselves.
Another relevant and significant change over the last few years has been in business culture. From being seen as a ‘nice-to-have’ layer in technology operations, business sponsors are now aware that without the ability to monitor and measure their technology, progress and innovations will be compromised.
Alongside changes to technology and culture, monitoring terminology has also quickly advanced.
While the old naming conventions were fairly simple, covering the basic technology layers such as infrastructure monitoring and application monitoring, the jargon nowadays changed to APM (Application Performance Management), EUEM (End User Experience Monitoring) and ITOA (IT Operations Analytics).
This article will look into monitoring considerations from 3 different angles or dimensions and expand on the challenges and opportunities large enterprises, especially in financial services, face when considering their monitoring tools strategy.
The 3 dimensions of Technology Monitoring
Not to be confused with Gartner’s 5 dimensions of APM, the 3 dimensions this article refers to represent different ways of looking into technology monitoring in the IT organisation.
- Technology Stack: monitoring the different strategic architecture components. These can be looked at in isolation, but many will share some overlaps.
Examples of this include: hardware, OS, virtualisation, databases, network, middleware, scheduling, processes, log files, interfaces, web, mobile, big data, storage, memory, workflows and many more.
- Development Lifecycle: monitoring considerations should be addressed from the early design stages and then implemented as technology products go through the different stages of the lifecycle.
Examples include: requirements analysis, design considerations, development environments, integration and test environments, production environments, DR environments etc.
- IT Disciplines: the third dimension represents IT groups and functions that need to consume and use monitoring data for their deliveries and operations. This dimension can also be cyclical and highlight continuity between different parts of the IT organisation, but some of these disciplines are more segregated.
Examples include: information security monitoring, infrastructure monitoring, application monitoring, capacity management, performance testing, operations analytics, business monitoring, incident management, problem management etc.
Monitoring in financial services
When considering monitoring strategies in large financial firms, the description above represents more challenges than opportunities. This is often caused by the complexity of the organisational structure, the difficulty in removing legacy dependency and the aggressive reprioritisation of business requirements.
Here are some of the key challenges broken down into these 3 dimensions:
Technology Stack – if we are considering the opportunities, then most attention will be on reusability. But given the fact that technology in financial services is partly driven by legacy and partly by strategy, many technology products or architecture layers are monitored by certain tools for ‘historical reasons’, so rather than reusability, large financial organisations are more likely to experience duplication.
Other than the obvious issues introduced by duplication, large organisations might also experience performance risks due to running multiple agents which consume similar types of data as well as potential conflicts between IT managers as to which tools should or shouldn’t be used.
Development Lifecycle – some industries addressed the lifecycle challenges by adopting DevOps practices. A DevOps culture has strong emphasis on tools and lifecycle so monitoring strategies are likely to introduce more opportunities than challenges.
One of the reasons for the challenges in achieving monitoring continuity in large financial firms is the significant differences between the IT environments (e.g. Dev, QA and production). Very rarely environments are built in similar ways, have similar specs and use the same tools. This then complicates the usage of the same monitoring tools and the promotion of the tools’ configuration from one environment to another.
Also, IT in the financial industry is known for its aggressive outsourcing culture which is not always coupled with strong collaboration between the different vendors. Let’s assume that in certain projects, development is managed in-house, testing is outsourced to one vendor and production services to another. In this scenario, three teams from three different vendors (and probably locations) are not likely to follow similar methodologies and have preferences of the same tools to ensure continuity.
IT Disciplines – a few years ago, most monitoring tools were associated with specific technologies or disciplines. Products either came with their own monitoring tools (like Tibco and Hawk), or were associated with specific market leaders (like Tivoli for infrastructure).
This created a reality where different IT disciplines relied upon and had experience of different monitoring tools, and due to the large number of tools and technologies, integrating all these tools required heavy investment and difficult decisions.
In recent years, leading monitoring vendors have widened their strategic plans and are covering a much larger share of the technologies, lifecycle and disciplines. Also, as the latest monitoring technology focuses on real-time analytics, it means that once the data is consumed, tools can be configured to process and visualise it for any type of monitoring.
Monitoring tools selection
One of the questions often asked when financial firms select their monitoring strategy is whether one tool can deliver all these features.
The answer is probably yes – the extensions, API’s and toolkits most tools include in their standard feature list are partly designed to ensure that the vendor’s sales representative can answer with ‘Yes!’ to any potential client’s question that starts with ‘Can…?’.
But in reality there will be two limiting factors. One is the risk of putting all one’s eggs in one basket (too much dependency on one tool or vendor). The other is having the commitment and buy-in from too many departments, managers and engineers to agree, work together and make it happen.
Another consideration is using open source solutions to develop an in-house product that fits all and can be flexibly extended based on internal requirements. It might be tempting but often it creates an even riskier dependency on individuals.
Making progress
Based on the above, it is easy to conclude that large financial services have significant challenges when looking into end-to-end standardisation of their monitoring solutions. Innovative tools and opportunities are certainly out there but the journey to achieve ‘nirvana monitoring’ will always be long and hard.
So how can progress be made?
Having a clear definition of the target state of monitoring is the first step for achieving progress.
When defining the target state, the following should be considered:
- The target number of preferred monitoring tools – a realistic number which should later drive tools reduction initiatives
- Up-to-date list of the technologies, lifecycle stages and disciplines the monitoring strategy should cater for
- Tools integration – approved technology in which monitoring tools can share data (e.g. RESTful APIs or central Event Management tools)
The next step should include a conscientious decision as to how to breakdown or segregate the organisation into manageable sections for the implementation of the monitoring strategy. This can be done by vertical / departmental breakdown of the organisation or by any split based on the dimensions highlighted above.
Successful definition of the target state monitoring will also signal capability to collaborate, which is one of the biggest hurdles in large and complex organisations.
About the Author
Peretz Shamir is a Service Delivery Manager at Mansion House Consulting, leading services for Tools Evaluation, Implementation and Operation with emphasis on Application Monitoring and APM.
About Mansion House Consulting
MHC is an international business and technology consultancy, focused exclusively on the financial services sector. We provide high quality, practical and robust solutions for the industry through our team of highly experienced consultants and subject matter experts.
We specialise in change and transformation management, toolkits and regulatory and governance frameworks. We deliver solutions globally to the transaction and investment banking communities, including leading Tier One clients from the financial services industry.
We have a niche expertise in Capacity, Performance and Event Management, Application Monitoring and Data Centre Optimisation Services. We provide both outcome based propositions and managed service solutions.
Established in 2009 we have been expanding and evolving ever since, with a team in excess of 300 and listed in the Sunday Times Tech Track 100 on four consecutive years 2013-2016, the London Stock Exchange’s 1000 Companies to Inspire Britain 2015 as well as the Investec Mid-Market 100 list in 2016. Headquartered in London, we have a global presence through offices in Frankfurt, Singapore, New York, Jacksonville (Florida) and Bangalore (India).
To find out more about our services, explore our website www.mansion-house.co.uk or contact:
Mansion House Consulting Limited
Disclaimer
This publication has been prepared for general guidance on matters of interest only, and does not constitute professional advice. You should not act upon the information contained in this publication without obtaining specific professional advice. No representation or warranty (express or implied) is given as to the accuracy or completeness of the information contained in this publication, and, to the extent permitted by law, the Mansion House Consulting Limited Group, its members, employees and agents do not accept or assume any liability, responsibility or duty of care for any consequences of you or anyone else acting, or refraining to act, in reliance on the information contained in this publication or for any decision based on it.
© 2016 Mansion House Consulting Limited. All rights reserved.
In this document, “MHC” refers to the UK entity, and may sometimes refer to the MHC group network. Each MHC entity is a separate legal entity. Please see www.mansion-house.co.uk for further information.
