About KuppingerCole's Leadership Compass
KuppingerCole Leadership Compass is a tool that provides an overview of a particular IT market segment and identifies the leaders within that market segment. It is designed as a compass for navigating the market by providing a detailed analysis of vendors and available products or services to help organizations to identify which ones best meet their requirements.
However, the information contained in this report is insufficient on its own for choosing a vendor, product, or service for a specific scenario. Organizations should carry out a thorough and comprehensive analysis of their requirements and map these to available products or services.
Types of Leadership
We look at four types of leaders:
Product/Service Leaders: This helps identify the leading-edge products/services in the particular market. These products/services deliver most of the capabilities we expect in that market segment. They are mature.
Market Leaders: This identifies which vendors have a large, global customer base and a strong partner network to support their customers. A lack of global presence or breadth of partners can prevent a vendor from becoming a Market Leader.
Innovation Leaders: This identifies the vendors that are driving innovation in the market segment. They provide several of the most innovative and upcoming features we hope to see in the market segment.
Overall Leaders: This ranks vendors based on a combined rating, looking at the strength of products/services, the market presence, and the innovation of vendors. Overall Leaders might have slight weaknesses in some areas, but they are typically above average in all areas.
For each area, we distinguish between three levels of products/services:
Leaders: This identifies the Leaders as defined above. Leaders are products which are exceptionally strong in certain areas.
Challengers: This level identifies products that are not yet Leaders but have specific strengths which might make them Leaders. Typically, these products are also mature and might be leading-edge when looking at specific use cases and customer requirements.
Followers: This group contains vendors whose products/services lag in some areas, such as having a limited feature set or only a regional presence. The best of these products/services might have specific strengths, making them a good or even best choice for specific use cases and customer requirements but are of limited value in other situations.
Our rating is based on a broad range of input and long experience in that market segment. Input consists of experience from KuppingerCole advisory projects, feedback from customers using the products, product documentation, and a questionnaire sent out before creating the KuppingerCole Leadership Compass, and other sources.
Product Rating
KuppingerCole Analysts AG regularly evaluates products/services and vendors. The results are, among other types of publications and services, published in the KuppingerCole Leadership Compass Reports, KuppingerCole Executive Views, KuppingerCole Product Reports, and KuppingerCole Vendor Reports. KuppingerCole uses a standardized rating to provide a quick overview of our perception of the products/services or vendors.
KuppingerCole uses the following categories to rate products/services:
- Security
- Functionality
- Deployment
- Interoperability
- Usability
Security is primarily a measure of the degree of security within a product/service. This is a key requirement. For products, we look for evidence of a well-defined approach to internal security as well as capabilities to enable its secure use by the customer, including authentication measures, access controls, and use of encryption. The rating includes our assessment of security vulnerabilities, the way the vendor deals with them, and some selected security features of the product. For services, we look for evidence of secure and consistent processes and pertinent certifications.
Functionality is a measure of three factors: what the vendor promises to deliver, the state of the art, and what KuppingerCole expects vendors to deliver to meet customer requirements. To score well, there must be evidence that the product/service delivers on all of these. For products, functionality is exemplified by technical capabilities and features. For services, functionality can include a range of discrete tasks such as installation, configuration, customization, and support.
Deployment, for products, is measured by how easy or difficult it is to deploy and operate the product. We consider the degree to which the vendor has integrated the relevant individual technologies or products. We also look at what is needed to deploy, operate, manage, and discontinue the product. For services evaluations, deployment is a measure of the number and complexity of services and products deployed.
Interoperability refers to the ability of the product/service to work with other vendors’ products, services, standards, or technologies. For products, we consider the extent to which the product supports industry standards as well as widely deployed technologies. We also expect the product to support programmatic access through a well-documented and secure set of APIs. For service providers, this evaluation considers their ability to support relevant standards and deliver solutions that integrate with customer applications and infrastructure.
Usability/Customer Experience are measures of how easy the product/service is to use and to administer. For products, we measure usability by considering the quality of user interfaces, looking for those that are logical and intuitive. We also look for a high degree of consistency of user interfaces across the different products from the vendor. For services delivered by system integrators or managed service providers, customer experience considers the quality of the services rendered as represented by the service providers and their customers.
We focus on security, functionality, ease of delivery, interoperability, and usability for the following key reasons:
Increased People Participation: Human participation in systems at any level is the highest area of cost and the highest potential for failure of IT projects.
Lack of excellence in Security, Functionality, Ease of Delivery, Interoperability, and Usability results in the need for increased human participation in the deployment and maintenance of IT services.
Increased need for manual intervention and lack of Security, Functionality, Ease of Delivery, Interoperability, and Usability not only significantly increase costs, but inevitably lead to mistakes that can create opportunities for cyberattacks to succeed and services to fail.
KuppingerCole’s evaluation of products/services from a given vendor considers the degree of Security, Functionality, Ease of Delivery, Interoperability, and Usability to be of the highest importance. This is because lack of excellence in any of these areas can result in weak, costly, and ineffective IT infrastructure.
Vendor Rating
We also rate vendors on the following characteristics:
- Innovativeness
- Market position
- Financial strength
- Ecosystem
Innovativeness is measured as the capability to add technical capabilities in a direction which aligns with the KuppingerCole understanding of the market segment(s). Innovation has no value by itself but needs to provide clear benefits to the customer. However, being innovative is an important factor for trust in vendors because innovative vendors are more likely to remain leading-edge. Vendors must support technical standardization initiatives. Driving innovation without standardization frequently leads to lock-in scenarios. Thus, active participation in standardization initiatives adds to the positive rating of innovativeness.
Market position measures the position the vendor has in the market or the relevant market segments. This is an average rating over all markets in which a vendor is active. Therefore, being weak in one segment does not lead to a very low overall rating. This factor considers the vendor’s presence in major markets.
Financial strength is a key measure because even though KuppingerCole does not consider vendor size to be a value by itself, financial strength is an important factor for customers when making decisions. In general, publicly available financial information is an important factor therein. Companies which are venture-financed, for example, are in general more likely to either fold or become an acquisition target, which present risks to customers.
Ecosystem is a measure of the support network vendors have in terms of technology partners, resellers, system integrators, and knowledgeable consultants. It focuses mainly on the partner base of a vendor and the approach the vendor takes to act as a “good citizen” in heterogeneous IT environments.
Please note that in KuppingerCole Leadership Compass documents, most of these ratings apply to the specific product/service and market segment covered in the analysis, not to the overall rating of the vendor.
Rating Scale for Products and Vendors
For vendors and product feature areas, we use a separate rating with five different levels, beyond the Leadership rating in the various categories. These levels are:
Strong positive: Outstanding support for the subject area, such as product functionality, or outstanding position of the company for financial stability, for example.
Positive: Strong support for a feature area or strong position of the company, but with some minor gaps or shortcomings. Using Security as an example, this can indicate some gaps in fine-grained access controls of administrative entitlements. For market reach, it can indicate the global reach of a partner network, but a rather small number of partners.
Neutral: Acceptable support for feature areas or acceptable position of the company, but with several requirements we set for these areas not being met. Using functionality as an example, this can indicate that some of the major feature areas we are looking for are not met, while others are well served. For Market Position, it could indicate a regional-only presence.
Weak: Below-average capabilities in the product/service ratings or significant challenges in the company ratings, such as very small partner ecosystem.
Critical: Major weaknesses in various areas. This rating most commonly applies to company ratings for market position or financial strength, indicating that vendors are very small and have relatively few customers.
Inclusion and Exclusion of Vendors
KuppingerCole tries to include all vendors within a specific market segment in their Leadership Compass documents. The scope of the document is global coverage, including vendors which are active only in regional markets such as Germany, Russia, or the US.
However, there might be vendors that do not appear in a Leadership Compass document for various reasons:
Limited market visibility: There might be vendors, products, and services that are not on our radar yet, despite our continuous market research and work with advisory customers. This is usually a clear indicator that Market Leadership is lacking.
Declined to participate: Vendors might decide to not participate in our evaluation and refuse to become part of the Leadership Compass document.
Lack of information: Products/services of vendors that do not provide the information we have requested for the Leadership Compass document will not appear in the document unless we have access to sufficient information from other sources.
Borderline classification: Some products/services might have only a small overlap with the market segment we are analyzing. In these cases, we might decide not to include the product/service in that KuppingerCole Leadership Compass.
The aim is to provide a comprehensive view of the products/services in a market segment. KuppingerCole will provide regular updates on their Leadership Compass documents.
We provide a quick overview of vendors not covered and their offerings in the chapter on vendors to watch.