Let me present a modern problem: The bank sends me a new debit card, after the previous number had been exposed in a data breach. It’s great that they are protecting me and my money. It’s not so great that I now must update all of my subscriptions with my new card number. And I have a lot of subscriptions, especially for media. Prime. Fubo. Netflix. HBO Max. Showtime. Crunchy Roll. Peacock. And on and on and on.

So during a recent July evening, we sit down to enjoy a true crime documentary about a murder in New England (my favorite). It’s on one of my streaming providers, sold via a bundle with other services. I update the billing info on the parent service, go back to documentary and…no deal. I get a message to update my billing with the parent company of the bundle, which I had done. No problem, try again. The parent company showed I had indeed updated my billing info. Back to the bundled service for the murder show…and., once again, nothing. After about five tries, a light bulb went off.

These services were still using a high latency — probably batch — process to connect their systems and distribute data for billing and subscriptions. That means they were not syncing in real time. If real time is like sending a text message, their process was like sending a letter. 

Now as far as stakes go, this one was relatively low. The streaming service missed out on ad revenue, while annoying me. But what if the stakes were higher? What if I was hoping to catch a live, once-in-a-lifetime event? What if this was a service that kept my lights on? Or helped me get somewhere after my car broke down? What if I real-time was imperative to my customer satisfaction?

And now that I am saying this out loud (or typing it), I wonder if any customer interaction can truly be considered low stakes.  Losing out on ad revenue is not nothing. When it comes to media, I subscribe to way too many streaming services as it is. Did this negative, high latency interaction make me miss my old cable company? And in light of inflation concerns and tightening my household budget, a bad customer experience might have me thinking, “I can always find a true crime documentary about a New England murder on another streaming service. They are prolific.

Today, customers expect a real time experience. Not to sound like Veruca Salt, but we know what we want and want it now. High latency isn’t just a week or a day; it can also be minutes. Delays in data can put the business at risk.  

Business happens in real time. People buy things in real time, banking happens in real time, goods are shipped in real time, and bad players try to access your data in real time.

Enterprise intelligence means making decisions based on data, so don’t we want to use the freshest data available? Companies with higher levels of enterprise intelligence report greater revenue growth and customer acquisition. In fact, the top three goals realized by the intelligent enterprise are revenue growth, increase in innovation, and improvement in customer experience.

Considering these uncertain economic times, IDC recommends that companies invest in digital-first technologies to accelerate transformation and meet the headwinds of inflation. That includes investment in services that augment agility and enhance customer engagement. Real-time experiences can do that. In a recent survey, IT leaders told IDC that for 2022, investing in technology to achieve real time decision making is a top priority.

Change data capture (CDC) is a great example of implementing dynamic data movement for real-time results. CDC is very popular within the enterprise for populating their streaming data pipelines, and there are new offerings in this space overt the past 12 months to accommodate a growing list of data sources and sinks.

CDC is just one entry point into real time streaming data. And not every use case requires zero-latency solutions. But we all know that improving the customer experience (CX) directly impacts the organization’s bottom line. With phones in our hands and digital assistants in our kitchens, we want relevant experiences that satisfy our needs with the least effort.

Streaming data use cases exist across all industries, from manufacturing to financial, from retail to healthcare. Our research shows that strategic use of streaming data across departments aligns with higher levels of digital maturity, i.e. the more mature organizations are implementing a streaming strategy across departments and throughout the enterprise, while the more immature organizations are still siloed in their use case implementations. When business runs in real-time, scheduled batch data is often out-of-date, and use cases we see around security/threat management, customer activity tracking, and real-time financial data plays that theory out.

I always say, “Let the use case lead.” It should direct how you think about real-time architecture, which ideally, is an expansion of your existing framework to avoid creating data silos. The top questions to ask are: “What are the business benefits we are hoping to achieve?” and “What insights do we need to achieve those goals?” But it’s also important to consider how a use case can drive insights at scale by asking: “Who needs those insights and where do they need them?” and “What other systems might we need to integrate with for context or to operationalize insights?” Asking these questions will help your company figure out the technology requirements, as well as where you need to be able to deploy those capabilities and what applications and systems you need to connect.

Amy Machado - Sr. Research Manager - IDC

Amy Machado is a Senior Research Manager for IDC's Content and Knowledge Discovery Strategies program. She examines the evolution of the content and unstructured data supply chain and its impact on mission-critical operations. Driven by demand for using data to make business decisions and building AI-ready data sets, modern technologies are creating new opportunities for automation and insights for what has traditionally been locked in undefined unstructured data. Ms. Machado's coverage includes enterprise content management, content sharing and collaboration, knowledge discovery, knowledge management, intelligent document processing, and customer communications management software. She uses primary research to size, forecast, and segment these application markets. Ms. Machado has over 20 years of analyst experience and has been at IDC since 2011. In addition to content and knowledge services, she has also focused on real-time streaming data and time series analytics, production printing, and industrial printing markets.

In this session of the Digital Leaders Community, we had experts from Nutanix and IDC Metri on hand to share their experience with the community.

The interactive session started as usual with a roundup of the reasons attendees from the community had joined. There was a variety of interests among the digital leaders from around Europe including one company that has made the decision to move to the cloud, some that have plans based on the maturity of different workloads, and others that wanted to hear about peers that have already moved to cloud.

One public organisation told how it had become completely cloud based. Did it save money? The attendee commented on the need to make sure the CFO and finance team understood that opex would increase. There is a need to make sure finance sees the savings as well. They talked about the datacentre no longer being needed, so that space can be used as offices. It was concluded that FinOps is tricky, but especially important as it is hard to forecast costs as more people move to cloud.

Nutanix has many customers moving to cloud, and sees how cloud costs have evolved for clients. The expert from Nutanix, Steen Dalgas, in particular has been helping companies figure out the TCO of cloud and other environments. Cloud is great for companies in innovation/growth mode — to rent capacity when you don’t know what parts will work. It is better to keep mature areas on-premises, as it is much more cost effective.

You can compare with on-premises with infrastructure as a service (IaaS). Platform as a service (PaaS) is quite different, and is hard to compare with on-premises. It is also extremely hard to repatriate from PaaS. The majority of cloud use now is IaaS, but PaaS will start to become more dominant.

A word of caution — the attendees mostly agreed that “By moving to the cloud, you don’t get rid of problems, you just move them somewhere else.

There followed a discussion on cloud-related skills. One attendee noted that they are outsourcing the skills they need to core suppliers. Some people feel more secure in the cloud than on-premises. However, cloud provides an opportunity for people to be lazy — sloppy code with data leaks was the example — that is a problem on-premises that in the cloud can become very expensive very quickly.

There is major demand for FinOps — managing the different environments, figuring which workloads to use in which environment. And this is a key skill along with the software skill of internal cloud governance; what users buy from the cloud on company credit cards can be significant if they don’t understand the issues in terms of variable costs and security.

The meeting discussed whether you could prohibit people from using the cloud, or do you educate people about costs and let them get on with their work? You need to monitor cloud bills very closely, to educate people, and you need people with knowledge of cloud biz models. There is always a need to ask many questions about the cloud invoice and figure out what’s happening internally.

One suggestion was that line of business people need to be aware that what they do costs money. Ideally, make sure costs come straight back to users. This can be done by tagging cloud resources with an app name, environment name, or resource tag so you know who is using what and how much it costs. One comment was that unfortunately sometimes even IT doesn’t know how the cloud business model works. ­­

 

Two years ago, one global bank said it would move out of datacentres. Now it says it will keep DCs in major global centres but make them a cloud-like experience. One attendee related how their company tried a shared private cloud, but consider it as on-premises. They can control a lot, but don’t need to pay for the whole thing when capacity is not used.

The IDC CIO Advisory team would like to thank everyone who came to the call for their input. It is always inspiring to hear from those making changes in their business and taking the tough calls with their management colleagues. We hope this session was valuable and provided many takeaways for you.

If you already receive invitations to our sessions, I hope to see you there. If you would like to join this community, please email Marc Dowd mdowd@idc.com.

 

IDC Digital Leadership Community: Topics 3Q22

Brainstorming Out of the Box

Thursday, August 25, 2022, 17:00 CET

Sometimes doing something unexpected can really pay off. This is true of being a digital leader, just as it is for disruptive innovation.

In this light-hearted but serious session we will look at the “hacks” and experiences that we collectively can share. We will be dealing with subjects in all areas from leadership “masterstrokes we have witnessed” to unusual policies that work:

In this session you can expect:

  • Amusing anecdotes with a deeper meaning
  • Car crash policies and how to avoid them
  • The heroes of digital leadership, and why we think they are great

Data and Analytics/Data Culture/Data Management Technology

Thursday, September 29, 2022, 17:00 CET

I for one thought it would never happen: IDC research shows that organisations are looking to move to being data-driven. I see movements among my clients to gather all the data in the organisation that is relevant and make it available.

Is the dream coming true? Are we moving to a world where data will at last come into its own and become the “new oil”?

In this session, we will discuss how to move towards a better use of data. Topics will include:

  • Developing trust in data — is that the key?
  • Governance of data
  • Managing the complexity of data initiative
  • The security and sovereignty issues around the use of data

We hope you will join us. Please feel free to suggest digital leaders who might benefit from these discussions.

https://www.idc.com/eu/digital-leadership-advisory

Marc Dowd - Principal, Client Advisory - Research and Consulting - IDC

Marc Dowd is the principal for IDC’s European client advisory practice. Dowd has over 25 years of experience working with the leaders of corporate IT across a wide range of industries. This includes 9 years as principal for EMEA advising CIOs of large international companies and government bodies for Forrester Research. Recently he has been focusing on Digital Transformation (DX) and the use of emerging technology such as AI, IoT and blockchain to develop new business models and business capabilities. His experience enables him to provide CIOs and strategic business planners within organisations who use technology, with market and customer insight, analysis, tactical advice, forecasting and technology trend intelligence to senior management teams at local, regional and worldwide levels.

Let’s start with a question.

When was the last time you picked up a phone to call a coworker?

Odds are, it was a while ago.  Instead, you reached out through Teams or Slack, email or a workflow of some kind.  You might have pushed a task to them through a service portal or used an analytics tool to give them data.  And you probably didn’t think anything of it.  Maybe you used Dall-E to create a custom meme then folded it into fluid document.

Each of these scenarios, and many more, are components of what IDC calls the “Intelligent Digital Workspace” (IDW).  Well, we actually call it an “intelligent digital workspace ecosystem”, which is a phrase intended to convey:

  • Intelligent – the environment uses AI/ML and scripted automation to adapt to the user’s requirements
  • Digital – the environment coordinates electronics and software both local to the user and remote to them to meet the user’s needs
  • Workspace – the environment is used to do work, as opposed to, say, playing games
  • Ecosystem – the environment is deliberately constructed of many components (data, electronics, software, and more) which work together in a coordinated fashion

If we ask people directly if they have deployed an “intelligent digital workspace ecosystem”, not many respond positively.  But, if instead we look at the functions of such an ecosystem, we see that over 70% of companies have enough functions to say they have already deployed it.

How did we get here?

So, how did we end up deploying an entirely new operational concept of the workspace without, well, noticing?  Did vendors trick us into it?  Was it the result of a Friday night that we just, collectively, don’t quite remember but photos ended up online?

Honestly, like any technical ecosystem, the IDW concept emerged in response to the collective efforts of tens of thousands of companies to provide employees relief from:

As we all struggled to adapt to these challenges, we also discovered that those inefficient processes and the technical challenges with the applications supporting those processes spurred further issues.  Teams became increasingly isolated within themselves.  IT support simply could not respond quickly enough and small problems lingered for hours or days.  Security systems designed for on-premise use, or to protect systems for occasional remote work, were untenably obscure and difficult to use over an extended period of time.

So, of course, we fixed it.

We deployed management tools which helped smooth out end user security (1st ranked IDW management tool) and operations.  These tools came from major vendors and small players, grew up in-house and evolved quickly to meet our expanded needs.

We deployed team collaboration and workflow tools to connect employees (1st and 2nd ranked IDW end-user argumentation). 

We deployed intelligent service management to provide support (2nd ranked workspace integration) with cross-group collaboration and the ability to report on activity across teams and the enterprise being the two most important attributes we designed for.

When it wasn’t enough, we empowered our teams to develop their own solutions and deployed those enterprise wide, accomplishing more for “agile transformation” in two years than we did in the previous two decades.

What’s Next?

Although we did well, solving those problems brought with it a host of new opportunities.  The results are often extremely complex, interweaving systems never really intended to work together.  Each enterprise has to manage the complexity on their own, and frankly they seem tired of it.  Now over 75% of organizations say that management and maintenance of this elaborate ecosystem is the responsibility of the vendors not the enterprise. 

This is, in part, because the world does not stand still.  The seemingly impossible challenges of allowing people to work in a dispersed, hybrid environment have given way in the hierarchy of problems to questions of geopolitical instability, to a collapsing return on energy investment for fossil fuels, and labor-shortages/job shuffling across a broad demographic range.  These pressures will only continue – with storms of disruption sweeping through the technology industry.  Simply put, enterprises will expect their vendors to step up and manage these ecosystems because the IT staff will have other things to do.

Meeting these “other things” will require the enterprise to clarify how it intends to use the IDW and for chosen platform vendors to respond, allowing employees for all enterprises to deliver faster, more consistently, and with greater productivity than traditionally organized and supported teams.

Now What?

So, that’s a lot.  We de facto created a new kind of digital workspace, founded in a wide range of technologies ranging from virtualization and endpoint management on one side to enterprise portals and content management on the other.  We cannot spend the time to manage the complexity of it anymore, so will turn to outside vendors and AI/ML to reduce operational load.  So, what do we do about it?

The question isn’t if you have an IDW; it’s whether you are actively coordinating it to deliver the right outcomes.

Advice for the enterprise: Enterprises need to step back and assess their existing environments.  Many will find that they have, de facto, deployed elements of an intelligent digital workspace as part of their ongoing efforts.  Those that have deployed the IDW de facto need to rapidly fill in any missing capabilities and work with their existing vendors to get out from under the operational burden the new environment creates.  Those that for whatever reason, lagged behind need to move quickly, coordinating with new or emerging vendors to deploy enough capabilities to support new ways of working.

Advice for the vendors: Vendors need to stop and assess their current offerings in terms of where they fit into the intelligent digital workspace ecosystem.  Coordination with partners, the ability to quickly solve emerging problems, and lowering operational burdens are likely to be the key messages customers need to hear over the next 12 to 18 months.  Many companies will also look for AI/ML enabled support and operations services, as well as the ability to deploy AI/ML models to support their employees.

The IDW is now a permanent feature of the enterprise.  It will, whether we think about it or not, continue to expand.  By being intentional and focused, enterprises and their vendor partners can derive the best value from this evolving ecosystem, shaping it to their needs.

Shannon Kalvar - Research Director - IDC

Shannon Kalvar is Research Director for Enterprise Systems Management, Enterprise Client Platforms, Observability, and AIOps at IDC. His current research addresses the evolution of digital work, AI in both an end-user and IT operational contexts, and the transformation of IT management in a rapidly changing world where humans must master the exponential increase in data and complexity while navigating tumultuous economic and geopolitical times. Shannon is a member of the Future of Work and AI councils, as well as a contributor to IDCs quantum computing research. He is based in Indianapolis, IN.

Many senior managers who have gone through a transformation from a traditional software development organization to an organization with self-organizing Agile teams, understand that this was a necessary exercise to ensure that value is delivered faster to their everchanging business environment. Where the survival of companies is often a direct result of delivering new user functionality faster than the competition, these managers are usually happy they went through this, but they also recognize that adopting an Agile way of working is not a silver bullet or a guarantee for success.

Although the concept of self-organizing teams sounds great, they still need to be managed in the context of the entire organization, its mission, vision and strategy.

Senior management needs to make decisions regarding budgets, investments, staffing, what to do, and what not to do. They need to decide on the strategic directions, the targets and goals, the operating model but also monitor progress and make corrections when necessary.

What we see in the market is that organizations struggle with the latter, and this is largely due to a lack of objective management information. Many Agile teams use progress metrics that make sense in their team, but which can’t be aggregated into useful management information. However, it’s important to understand the performance of the Agile teams, as they are providing the value to the business and its customers. We see that companies operate on different levels of maturity when it comes to the control they have over their application development teams. The question is: on which maturity level is your company?

A well-known model to look at maturity of processes is the Capability Maturity Model Integration (CMMi) model.  This model assesses the development processes on 5 different levels (see figure).

CMMi maturity levels
Source: https://en.wikipedia.org/wiki/Capability_Maturity_Model_Integration

This is a good principle and can be used to assess control over Agile development teams as well. The Agile Control Maturity Model (ACMM), based on the CMMi levels, allows senior management to understand the control level regarding their agile teams and the value they deliver. The following table shows the way IDC Metri looks at the different levels of Agile control maturity.

Low maturity levels (0, 1 and 2)

Most organizations currently are on level 0 or 1. They went through the transformation, and CIO’s and other senior management got advice from Agile coaches and other consultants on how they should manage the new organization where self-organizing teams have a lot of power to decide, and the business is very closely working with the teams to create value. However, on level 0 or 1, you have no idea how much value is created for the budget spent, and how this relates to the competition. Furthermore, predictability is low and it’s hard to manage the interdependencies between teams and to forecast when certain pieces of software will be ready. External teams are contracted based on time and materials (hourly rates) and not output-based (maximum value for money).

High maturity levels (3, 4 and 5)

As an organization gets to a higher maturity level, senior management gets more and more in control. Objective metrics are displayed in up-to-date dashboards, enabling them to make decisions based on facts. This brings along many benefits, like for instance:

  • The ability to demonstrate to stakeholders that you (the CIO and Senior Manager) is in control of the performance and value creation of the Agile teams.
  • Focus on actual value creation by improving productivity and delivery speed, while improving the quality of the product.
  • Objective measurements are used to understand which are the high performing teams and which are the low performing teams. Then the reasons this can be investigated and improvement actions can be carried out.
  • Software Cost Estimation accuracy improves enormously, as data from the  organization and teams can be used to calibrate the parametric models.
  • Leading indicators result in increased predictability of the teams, eliminating schedule or cost slippages.
  • Based on objective measurements and targeted removal of critical violations, the risk in the application portfolio reduces significantly while quality, maintainability and cost levels improve.
  • Output-based metrics can be used in contracting Agile teams, enabling organizations to select the team that provides the most value for money (productivity times average rate) instead of the cheapest one that may not be productive at all.

And there are many more benefits.

In the next figure, typical productivity improvements we observed are shown, where 0% is the market average for comparable teams/applications.

Being an expensive function, Agile development teams should be managed to ensure value is optimized for the investment. But, how do you measure the value? When IT leaders are struggling with this common challenge, IDC Metri has prepared this checklist to help you manage and measure your teams’ performance.

IDC’s Digital Leadership Community (DLC) July 2022 session was much more positive in outlook than I had expected. The participants shared their experiences and plans openly, and it was quite remarkable to be part of a discussion that really focused on “war stories” from the field and great suggestions for action plans.

Inflation is making the war for talent more difficult and is impacting salary expectations for new hires. One member spoke of their finance department telling them that when the recession comes, salaries will fall. But will that really happen? One CIO told us it has already happened in Israel, where a crash in the start-up market has eased the situation from just months ago when requests for salary increases were in the order of 30%–50%. This could be a sign that the overheating in the salary space will subside somewhat due to recessionary pressures.

Nevertheless it was acknowledged that IT skills are in high demand and will remain so. This is reinforced by IDC research that says the gap in available skills is still growing in all markets. The discrepancy in salary between new hires and existing employees is widening due to the need to pay new hires more due to inflation, which is a potential source of friction between employees. One company in a market with inflation running very high said it has resorted to salary reviews every six months to stem the tide and ease the suffering of its IT workforce. Proactively offering wage increases to retain IT talent was discussed.

There was a consensus that inflation is hitting opex contracts hardest, as might be expected, but there were also warnings from participants that negotiations across the board are becoming harder as suppliers brace for rising costs. Many supplier companies are trying to raise prices to adjust to higher costs (including energy costs). One CIO spoke of an SD-WAN contract negotiation where he saw clauses in the contract to ensure flexibility in prices between now and when the contract is actually signed. It was the first time he’d seen such a clause. The consensus was “be ready to walk away from key suppliers if they continue to hold a gun to your head.”

All participants agreed they are expecting more such difficult discussions with vendors until the end of the year. These discussions will have a big impact on IT budgeting in October. For some, who have had contracts in place for more than 10 years, vendors are now asking for additional clauses to provide flexibility for prices.

A positive note was raised by a CIO from Turkey, which is currently at 78% inflation. He said they are used to inflation. “It makes us flexible,” he said. They are focusing on fixed-fee agreements and projects to try to push inflationary risk on to suppliers. They are also migrating to cheaper or free products — they used MS SQL previously, for example, and are now moving to NoSQL and alternatives. He also said, “Big inflation rates make the business side dynamic.” His organisation is consolidating IT across the international group and is planning to have most IT handled from Turkey, as salaries are quite low there, and this reduces costs in opcos in other countries. Essentially, for that CIO, inflation is a challenge but also an opportunity.

One attendee — who had just finished budgeting — said their budget had been agreed with less discussion than ever because the finance department knows that inflation is raging. For now, the company can pass on prices to the market. As a distributor you can benefit from inflation. IT has even provided a special calculation tool to optimise pricing for the distribution business unit. It was noted by participants that price increases are even more drastic due to the dollar-euro exchange rate changes with the strengthening of the dollar.

Attendees also discussed how to decide which projects to postpone. A number of participants said they are looking for higher ROI than usual, in effect taking a harder look than usual, with IT deciding together with business which projects would pay back quickly. An attendee said his organisation is keeping IT “on prem” to delay the shift to opex. Others have moved to outsourcing to save money by avoiding upfront investment.

Will there be a big drop in IT budgets with the expected recession? No one on the call was expecting a big drop. The recommendation from attendees was to slow down strategic projects that have a longer payback period (three to five years) and most have already done that. I asked if anyone expects budgeting to become more dynamic/frequent. The answer from the group was, “No, not really.” One attendee did note that their cloud/FinOps team is looking to forecast cloud costs more frequently to avoid cost surprises and suggested looking for support from AWS and MS for that. Some suggested conducting this assessment on a monthly basis, and getting finance team support for the IT team to do that. The need to educate the executive board about the volatility of cloud costs was also noted.

Energy prices are a crucial factor for some participants. In certain industries the business is doing scenario-based planning based on different exchange rate outcomes and energy price thresholds.

The meeting ended with surprising positivity among the attendees. It was good to see that the Digital Leadership Community is pulling together to help each other in times of adversity. We look forward to you joining us in our next meeting. If you would like to access the DLC board for this quarter and add comments or read the IDC research, please do so here.

If you already receive invitations to our sessions, I hope to see you there. If you would like to join the community, please email Marc Dowd (mdowd@idc.com).

 

IDC Digital Leadership Community – Think Tank Topic August 2022

Brainstorming Out of the Box

Thursday, August 25, 2022, 17:00 CET

Sometimes doing something unexpected can really pay off. This is true of being a Digital Leader, just as it is for disruptive innovation.

In this light-hearted but also serious session we will look at the “hacks” and experiences that we can share. We will be dealing with subjects in all areas from leadership “masterstrokes we have witnessed” to unusual policies that work.

In the session you can expect:

  • Amusing anecdotes with a deeper meaning
  • Car crash policies and how to avoid them
  • The heroes of digital leadership, and why we think they are great

We hope you will join us. Please feel free to suggest Digital Leaders who might benefit from these discussions.

https://www.idc.com/eu/digital-leadership-advisory

News about data breaches – when enterprises ‘share’ their data without consent – make headlines and strike fear into executives’ hearts, send IT departments into overdrive and shake customer and employee confidence. However, little is said about when enterprises share data purposefully. There is a strong case to be made about how sharing data can make you and your ecosystem partners stronger, more resilient to volatility and uncertainty, and better able to serve your customers.

Did you know that 50% of US and Canadian of enterprises are already sourcing data from their partners (suppliers, distributors, retailers, etc.) and an additional 19% say that they are likely to begin doing so in the near future? (See material from IDC’s March 2022 Data Buyers Study) This means that it is highly likely that your competition is doing this. It is not just that so many others are doing this, it is that there can be a lot of benefit to sharing with trusted partners.

Shared data and insight is one of the foundations of successful industry ecosystems. IDC’s most recent global research finds that more than 1/3 of enterprises find that shared data positively impacts KPIs for innovation rates, business agility, and transforming their individual enterprise to more digital operation (IDC’s Future of Industry Ecosystem Survey, June 2022). This type of sharing is widespread and can take many forms. There are obvious opportunities for suppliers to share data with the companies that make finished goods, for example. When the finished goods company has accurate information about supplies and transit times for inputs to their products, they can foresee potential out of stock situations. This can allow them to qualify additional suppliers and/or inform their customers of potential bottlenecks. Sourcing and sharing data outside the individual enterprise helps provide insight that one entity cannot effectively collect and analyze on its own.

Of course, after you answer, ‘Why share data and insight?’ you have to figure out, ‘How?’ IDC has been looking at this through a series of research, starting with The Why and the How of Shared Data in Industry Ecosystems where we describe ways to create value from data and insights through fit-for-purpose solutions such as data marketplaces. Next, we followed that up with case studies of how groups have made this work in a presentation on Shared Industry Ecosystem Data: How Ecosystems Made It Work with Data Sharing and Marketplace Technology. In this research, you can learn the types of technologies that others have used to make data and insight sharing more available. These tools also assist with curation, governance and entitlement – which are key to any shared data situation.

Many enterprises still struggle with the ability to share data and insight across divisions and geographies. However, you don’t have to wait until you get all of your internal information sharing perfect before you look to how outside collaboration might provide advantages. Wins with external stakeholders could pave the way for more effective collaboration internally!

Data sharing has existed since before there were technologies in place. Vendors and customers have often had dialog where they shared information, whether these arrangements were formal or informal. Most often, solving the technical side of data sharing meets with less skepticism than the governance, change management and pure inertia issues. The gap has been less that potential ecosystem participants are unable to share and more that they have been unwilling for one reason or another. IDC receives a lot of inquiries about topics such as:

  • What if there’s no industry ecosystem for me?
  • How do I know what to share?
  • How am I going to share this?
  • Do I have to share everything?
  • How do I get others to share with me?
  • Should I charge for access to my data?
  • Can we make money doing this?

Your enterprise is not alone in wanting to learn more but being cautious about the potential exposure of shared data and insights. And, even though a few ecosystems have been able to create an outside revenue stream for these outputs, it should not be a primary consideration when potential partners come together.

Reach out and let’s talk about how to make shared data and insight in industry ecosystems work for you! There a quite a few examples we can point to, across industries and across the globe. You can learn from other ecosystems and how they’ve used technology, process and teams to derive mutual benefit.

Next on our research agenda, we are crafting a TechBrief on Data Clean Rooms – a way to share data while preserving privacy and competitive information, while benefiting the two or more parties who are looking to tackle common challenges ranging from fighting fraud to improving supply chain issues to helping to gain more insight about customers and potential customers. This is a way to be building your own ecosystem and gaining valuable insight while keeping the legal and security teams satisfied with the level of care you put in place.

We will continue to write about the topic of shared data & insights within our Future of Industry Ecosystems practice as well as our Data as a Service practice.

Lynne Schneider - Research Director - IDC

Lynne Schneider is Research Director leading IDC's Data Collaboration & Monetization, and Location & Geospatial Intelligence market research and advisory practices. Ms. Schneider's core research coverage in DaaS includes data sourcing and delivery services from traditional and emerging data providers along with evolving data aggregation and dissemination platforms. The breadth of coverage includes services that enable an organization to externally monetize data generated as part of the organization's ongoing operations, value-added information derived from this data, and the marketplace for combining data with other solutions. This research analyzes the supply and demand side business and technology trends of this emerging category.

Organizations overcome challenges in quantifying Agile value by embracing a solution that assesses, benchmarks and course-corrects Agile development teams. How is this done? Through agile value management.

Agile development is challenging to manage and measure, especially when compared with traditional development models like Waterfall

Management is often uncertain when functionality will be delivered and at what cost, whether it delivers required quality, and with what inherent risks. Due to its inherent nature of using Story Points to assess progress, Agile value is challenging to quantify.

If something’s difficult to measure, it’s hard to evaluate, manage and ensure value delivery. Through an example, this blog sets out to show that it’s possible to counter these challenges and restore balance between the relationship of Agile teams and management.

Imagine this common scenario: A major client-facing application that digitally transforms the company is a year into development with delivery postponed every quarter. Sprints are chewing through the backlog, but it keeps re-filling with refactoring, bugs and newly discovered requirements. Turn-over on the team is creating challenges in productivity and quality, and onboarding new team members takes too long, which impacts overall productivity. 

Management keeps receiving progress planning updates that show a smooth, Agile development machine with sprints and stories paced to deliver, but then the budget keeps increasing and the delivery date shifts to the right. Management is struggling to reconcile the reports versus actual cost, productivity, quality, and deadline progress. They’ve replaced the project manager once, but are frustrated by conflicting messages and are distrustful of quality, timely progress. 

The development team is confused and team members are leaving the project (and the company) due to management pressure to correct a project, where the team feels it is moving through the Agile process effectively. They feel that “management” doesn’t understand how Agile works and its value proposition; coming from Waterfall project management, they believe that management simply doesn’t get Agile. 

The development team also feels hampered by staff turnover, resulting in lost productivity, institutional memory and decreased quality. If management got off their backs and let them work, they could deliver a successful product. 

Behind management’s interest in seeing the product done and delivered is that they see it as key to their digital transformation, and a competitive and defensive necessity versus their market peers. If Agile is so much better than Waterfall, why are costs, duration and quality spiraling out of original forecasts? At least with Waterfall you knew what the milestones were and could prioritize deliverables. With Agile, it’s all about backlog and Story Points. 

Management eventually lost faith in the development process to deliver the application in the competitive timeframe needed with a minimum viable product and a semblance of cost controls. They feel like they’ve lost grip over a project that has become out of hand with no end in sight. Both groups need a concrete measurable process that allows all parties to objectively and consistently assess the team performance, product health and quality. They need a dependable way to visualize progress (productivity, cost, delivery speed, quality, security, maintainability). 

The company previously subscribed to research from IDC and heard about the IDC Metri Agile Value Management (AVM) solution that assesses, benchmarks and course-corrects Agile development teams. AVM is a systematic, quantitative, comprehensive, data-driven, and repeatable solution, based on ISO standards, and has demonstrated quantifiable value to clients. 

The IDC Metri AVM solutions team began the process of breaking down the product development process and product. 

Measures of value creation occurred across multiple sprints. Enhanced function point analysis, based on ISO standards, is to assess progress and efficiency rather than Story Points. Source code analysis assed code quality and security, to reduce refactoring and the number of CVEs. This analysis not only identified the gaps, but provided specific, tactical recommendations on how to address them. Further, the benchmarking team gauged their performance to understand how competitive their team was versus market peers. 

After six weeks of work, management and team received dashboards and analysis that clearly identified gaps and remediation. 

The engineering dashboards clearly identified poor code and CVEs that allowed the development team to better and more rapidly address quality issues. As they adopted the guidance from the engineering dashboard, the development team found their practices improving. Quality and performance improved due to lower testing efforts attributed to enhanced coding practices. Also, improving practices and identifying better practices reduced team stress, and enabled newly onboarded team members to more rapidly become productive. 

The benchmarking team let management identify areas to invest and realize team improvement. The management dashboard provided managers the opportunity to check key project factors, notice and correct negative performance trends.

Based on the provided forecasting assessment, management realized the development team wasn’t ready for the required business delivery milestones. Even if they performed to peers. So, management made the decision to augment the team with third-party staff to ensure success.

Both the development team and management needed the same thing – a common, agreed-upon definition of “reality”. They started off with two different frames of reference. For the development team, their frame was backlog, sprints, velocity and Story Points. For management it was cost, time, quality and milestones. The two groups were speaking different languages and couldn’t reconcile. By bringing in AVM, management and the development team now had one frame of reference everyone could agree with. They had a complete analysis that brought together technical and business needs.

The team had one language from which they could understand development health, address gaps and deliver agile value.

Interested in learning more about IDC Metri? Let’s schedule an appointment for an introductory meeting.

To discover more strategies CIOs will need in order to plan for contingencies and technologies, to create a dependable infrastructure that enables fault tolerance, read IDC’s new eBook, CIO Sentiment Survey: Finding the North Star in a Turbulent Environment.

The adage “the only constant is change” is truer than ever in today’s disrupted world – a world where the velocity, intensity, and frequency of change continues to push us to new heights, compelling IT to change itself and revisit how CIOs must lead this transformation.

The IDC CIO Sentiment Survey 2022 was conducted in July 2022 (with 289 worldwide respondents) and revealed the extent to which digital business is accelerating and the global IT landscape transforming. The survey found that many CIOs face vital challenges in achieving their goals in these turbulent times. The good news is that many organizations are responding to these changes with agility, creativity, and innovation. But how are they doing it? How did they move from being reactive to proactive? What are some of the best practices for survival and success?

Why Does IT Matter?

The percentage of business coming from digital products, services, channels, or digitally driven improvement to operations has reached 50% of total business in 2022 and is similar in each region of the world — demonstrating that digital business is a rising global phenomenon.

Today, IT teams compete globally for talent and can expect a comparable impact on their organization’s business. On average, each full-time IT employee supports $13.29 million in revenue — a data point that is relatively consistent worldwide, with $14.70 million in North America; $13.96 million in Europe, Mideast, and Africa; and $12.39 million in Asia-Pacific. 

It’s not just the revenue generated per IT employee — it’s about how they add value to product or service offerings by enabling digital business, improving customer experience, or making internal operations more efficient. IT is clearly the core engine for the success of a digital business.

But beware of your perceptions. Our survey found that most CIOs overestimate their performance and how much others appreciate them. For example, when looking at their business process performance, 77% of IT executives estimate their overall performance as somewhat better or much better than the competition. This is significantly above the normal 50% average and highlights a clear, statistically demonstratable overestimation of performance.

But responsibilities are increasing. Fifty-five percent of CIOs apply influence they gained during the pandemic to expand their role by adding new responsibilities in areas such as intelligence, privacy, innovation, transformation, safety, sustainability, or resilience.

A minority of CIOs have established a stronger relationship with their CEO (16%) or have gained recognition as a business leader (18%). As the digital revolution accelerates further intertwining of business and IT, rising to a business leader is the only path forward for most CIOs.

Making IT Count

CIOs have constantly sought to deliver the largest and most numerous contributions from IT activities but, more than ever, they must now choose their battles and focus their efforts on helping the business deliver differentiated business outcomes.

The IT group’s most important business impacts across the organization are enabling it to respond more quickly to change (cited by 85% of respondents) and taking the business online, virtual, and contactless (84%). To do this, the IT organization must apply that differentiated approach in the current context of uncertainty and volatility:

  • Survey respondents think that Agile portfolio management brings the most impactful business outcomes (45% of respondents).
  • Value stream mapping remains a highly effective approach (35%) as it helps identify where customer value is added and where it’s consumed.
  • CIOs need to focus on simplification of  business processes with increased self-service options (44%) and set an automation-first initiative aiming to improve all business-critical systems (35%).

Be Practical

For Future CIOs, navigating the winds of change requires a new approach — no longer is more and faster a viable approach. Now, a focus on high-impact initiatives with discerning business outcomes is the key:

  • Get closer to customers. IT executives stay as far as they can away from customers, partners, LOB executives, or the CMO, spending less than 12% of their interactions with any of these stakeholders. They spend far more of their time with the IT leadership team (43%), the CTO (42%), and the CEO (35%).
  • Automate. On average, 40% of business processes have been automated, with IT being the most automated (42%). However, IT executives estimate that finance and supply chain will take over in the next three years (43% and 42%, respectively).
  • Care for people. Retaining talent has always been a hallmark of strong leaders; now it is a differentiating leadership trait crucial to the success of an IT organization. And the best way to boost motivation and retention is by providing more opportunities to work with and learn from the business (cited by 29% of respondents). While still valued, salary and bonus incentives seem to have a lower impact (24%).

Conclusion

IDC’s CIO Sentiment Survey 2022 has disproven the old age, “The more things change, the more they stay the same.” Digital business has demonstrably become a worldwide phenomenon with opportunities and inevitable challenges, for customers, employees, and organization leaders — but none more so than CIOs. To discover more strategies CIOs need to plan for contingencies and technologies to create a dependable infrastructure that enables fault tolerance, read IDC’s new eBook, CIO Sentiment Survey: Finding the North Star in a Turbulent Environment. Click the button below to download the eBook now.

Learn more about the Fourth Agricultural Revolution (Agriculture 4.0) and technologies supporting food quality and production efficiency, read IDC’s new, complimentary eBook, The Digital Agriculture Revolution: A Survival Guide.

Predictions Previously Estimated Agriculture Could Only Support 5 Billion People

The 1973 movie Soylent Green, starring Charlton Heston and Leigh Taylor-Leigh, describes a future where uncontrolled population growth, ecological disaster and climate change have caused agriculture to collapse, turning the Earth into a dystopian landscape. In this future, food riots are a common occurrence throughout the world, and the main source of food is a wafer made of plankton called Soylent Green, which in the end turns out to be made from recycled humans. The population of that dystopian world was set at a hyperinflated, unthinkable 7 billion people.

Soylent Green is set in 2022.

If you read science fiction from before 1970 that mentions world population, it will usually cap the populace of Earth at about 5 billion people. At that point the Earth either resorts to draconian population control, cannibalism, or extra-planetary colonization. Currently, our population is estimated at 7.96 billion people. Cannibalism is still frowned upon in polite company and despite the best efforts of billionaire playboys, our current extra-planetary population sits at around zero people. Why were these predictions so far off?

The Third Agricultural Revolution, also known as the Green Revolution, started in the 1950s. It saw the development of technologies including mechanization, chemistry, and breeding techniques that dramatically increased food production. The Green Revolution has contributed to the reduction of poverty, prevented hunger for billions, eradicated widespread famine, reduced GHG emissions, and tripled yield per acre, thus leaving more land for natural habitats and averting ecological disaster. This revolution silently changed the world in ways that were unimaginable even in the 1970s, 20 years into the Green Revolution.

In 2022, we are facing similar predictions. Food production must increase by 60% in 30 years. We are facing a future defined by climate change, ecological collapse, and worldwide famine. The insatiable modern diet for food and energy is consuming the planet, leaving nothing for the next generation. Some predictions claim that we only have 30 harvests left before agriculture collapses.

What many are unaware of (or deliberately ignorant of) is that we are solidly into the Fourth Agriculture Revolution.

We are on the cusp of a much brighter future

The Fourth Agricultural Revolution (Agriculture 4.0) is the incorporation of IoT, automation, data collection and data analysis into food production systems at scale. Agriculture 4.0 is enabling food producers to do more with less and create new digital commodities which are revolutionizing the way we produce and distribute food. As the world becomes more affluent, markets will demand more diverse foods, higher protein, increased quality, and environmentally sustainable foods at an affordable price. Data is the key to meeting these demands. Technologies to collect and use data are being incorporated at every level of food production. The Fourth Agriculture Revolution will be the collection, movement, processing, packaging, and marketing of data at scale.

What does that mean? Better, cheaper, and environmentally friendly foods.

Waste Not, Want Not

An excellent example of the power of tech is the reduction of food waste. Roughly 33% of the food produced in the world is wasted either along the supply chain or in the home. In the US, estimates suggest that enough to feed the population of Italy (67 million ton) is wasted at the retail and consumer level alone. The use of IoT, data technologies and data analysis is already reducing spoilage in storage and transport, helping match the right crops to the right processors, lengthen shelf life or shorten supply chains to connect farmers to consumers. The enormous potential to build a better world (and make a bit of money) has agriculture and technology companies hungry to work together.

Let’s Clear the Air

Agriculture contributes up to 25% of GHG emission worldwide, but the industry could become carbon negative in the next decade. Widespread adoption of practices that encourage soil carbon sequestration could remove up to 5 Gigatons of carbon from the atmosphere each year, turning agriculture into one of the best mechanisms to fight climate change. Agtech is rapidly reducing GHG emissions by creating emission benchmarks, improving production practices, and creating digital commodities that put a price on environmental services. These environmental service markets provide incentives speeding research, development, and adoption of environmentally beneficial technologies.

Robots and Their Vision of the Future

Autonomous machinery has the potential to revolutionize agriculture. Swarm farming, in which multiple autonomous machines work in concert, will allow for farming to be highly efficient, with timely seeding, automated crop scouting, judicious crop protection applications, and quicker harvests. Better timing of processes will increase yield and crop quality. Simply reducing soil compaction from heavy machines could raise yields by 10%. Automation will also allow for smaller, more agile equipment, while automating processes will also allow producers to focus on higher order tasks and promote more diverse cropping systems. These machines will be able to gather a wealth of information which can be used to refine best practices, improve predictive models, and provide actionable insights.

Computer vision (CV) has been gaining momentum in agriculture. Satellites, drones, and other cameras scout field, predict disease, and enable farmers and agronomists to work efficiently. “See & Spray” technologies using CV can cut herbicide use by 95% and other autonomous weeding machines bypass herbicides altogether. CV in animal agriculture can detect illness or stress before it would be detectable by humans. By identifying stress and diseases in plants and animals faster, CV will allow for quick, precise action, thus minimizing waste or treatments while helping create optimal growth environments for crops and livestock.

What’s Next?

Revolutions are times of massive opportunity to build. What we have covered is only the tip of the iceberg. Other technologies are being incorporated into agriculture that will reduce transaction costs, increase nutritional value, connect farmers to consumers, enable smallholder farmers, revolutionize genetic selection, and change the face of food production either subtly or radically. The cost of technological development and deployment has been vastly reduced by investments in connectivity and integration of cloud computing from major players. This opens the way for new companies to enter this space, find partners and build a brighter future. Optimism about agriculture building a brighter future is historically the most profitable trend for any company looking to change the world.

To learn more about the Fourth Agricultural Revolution (Agriculture 4.0) and the technologies supporting the food quality and production efficiency, read IDC’s new, complimentary eBook, The Digital Agriculture Revolution: A Survival Guide. Click the button below to download the eBook now.

The latest indicators point towards a further decline in GDP in the second quarter in the US. If that is indeed the case, many in the financial community are ready to declare that the US is already in a recession (given that we saw a decline of -1,6% in US GDP in Q1). However, the select committee of economists tasked with making the call are unlikely to label this a recession in the short term. The reason: very mixed data. 376 000 jobs were added to the US economy during the month of June. In parallel, inflation in May reached its highest level since late 1981. This is a very unique version of stagflation, and economists are expecting interest rate rises throughout the rest of 2022.

This situation is the latest in the storms of disruption affecting the market right now. The litany of storm keeps growing: ongoing pandemic management challenges, major supply constraints, significant skills shortages in IT and broader technical areas, rapidly increasing cybersecurity threats, evolving climate regulations, and assertions of sovereignty over digital business triggered by the Russia-Ukraine War. With these developments, business leaders are now facing rising inflation, and many are anticipating a significant slowdown in economic activity. The US may be earlier to sound the alarm, but the EU is forecasting a similar situation by early 2023. And while in country recession fears are not a major focus in Asia/Pacific, there are question marks around the potential knock-on effects across the region.  IDC’s recent poll of 100 CIOs globally indicated that 79% expected a recession in their own country, or in important buyer countries in the next year. 95% of those believe it either has already started or will hit us in the next 12 months. Of 77% of them think it will last at least a year. Most, however, expect it to be a modest or moderate slowdown, not a deep recession.

As Jamie Dimon (CEO of JP Morgan Chase) explained recently, ‘I said they’re storm clouds. They’re big storm clouds or – it’s a hurricane. That hurricane is right out there down the road coming our way…and you better brace yourself.’

For the technology markets that IDC tracks, this is also a unique situation. This will be the first potential recession in the context of a ‘As-a-Service’ Technology Landscape[1]. Cloud now dominates tech spend across infrastructure, platforms, and applications. In fact, globally the cloud market will represent 40% of WW enterprise IT spend in 2022 (excluding devices and telecom services). An even more telling point is that over 50% of all spending on all software will be as-a-Service by the end of 2022, rising to almost 2/3rds by 2026.

While 63% of the CIOs indicated that they would be either making targeted reductions in IT spend or reduce overall IT budgets, many organizations are continuing to invest in strategic technology projects to support business initiatives (e.g. Analytics, Automation, CX, EX and supply chain transformation) as part of broader digital transformation efforts. In many tech-savy enterprises, up to a third of all spending (not just IT spending) is for technology. Given the focus on cloud when it comes to Tech, IDC expects that the dynamics of that spend will look very different to the past.

Phil Carter and I want to share five key parameters to monitor:

  1. Consumer vs Enterprise: IDC spending data already shows a significant difference between consumer IT spending (focusing more on devices and media/content services) versus the enterprise, where spend is proving to be much resilient (driven primarily by cloud). For more details on these trends check out IDC’s recent Global Recession Scenario for IT Spending webcast.
  2. Enterprise CAPEX vs. OPEX spend: In previous economic downturns, IT spend followed GDP trends closely as large-scale CAPEX purchases were put on hold or cancelled altogether. Now, we are dealing with much longer-term multi-year contracts that tech buyers have put in place across the IaaS, PaaS and SaaS layers. Hence overall IT spend will not see the same level of spikiness as in past downturns at a market level. In fact, the real pressure will be on the Cloud Providers themselves as they still need to invest capital in IT hardware and datacenters to support all those long term commitments.
  3. Cloud Contract Flexibility: IDC is already seeing examples of more flexible cloud contracts being negotiated – minimal short-term pricing increases to drive longer term or larger consumption commitments over longer periods, using reserved cloud instances as a way to get better pricing terms or very favorable licenses and maintenance conversion options in exchange for longer term contracts. The goal for both sides being to reduce cost volatility over an extended period, making capacity planning and cost accounting more predictable.
  4. Cloud Margins and Cloud Price Increases: For the first time since their inception, many cloud vendors are increasingly under margin pressure. This is linked to increased input costs (infrastructure and energy) and the strong dollar impacting international revenues. As a result, IDC expects the prices of the cloud offerings overall will increase in the range of 5-7% over the next 12 months. This will be an especially important development for major software providers who are at the mid-point of their own journeys to SaaS business models. Managing changing cost expectations for the underlying infrastructure whether owned or “rented” from IaaS providers is now their business challenge, rather than their customers’ operational challenge.
  5. Cloud Value Realization: Over the past 12-24 months, the pandemic-driven tech spend exuberance has highlighted the cost of cloud. At the platform level, every development decision is a budgeting decision. The focus on FinOps and scrutiny on the ROI has ramped up. The current economic situation will accelerate this trend. One of the key challenges will be to ensure that they aren’t spending on an “As-a-Service in name only” solution that simply alters the financing calculus. The real benefit of an effective As-A-Service offering is that it reduces and even eliminates the long term operational and upgrade costs associated with software and infrastructure lifecycles. Observability will be a critical part of FinOps.
RELATED READING: Enable your sales team to sell value rather than competing on price and features. IDC's new checklist is available now.  Start Value Selling Today. Optimizing Customer Value Best Practices. 

Looking back at history, the 2007-2008 financial crisis was the trigger for cloud’s to move into the mainstream. However, that was a real market crash. One software executive recently highlighted to IDC how $1 Billion in pipeline evaporated in the matter of days. The current situation is more of a dripping tap than a burst pipe. There is a long run up for policy makers and it’s the inflation vs. growth balancing act – with interest rates being the lever – hence why the market is watching the monetary policy makers very closely.

For the IT market, IDC expects continued resilience in the enterprise space – driven by this ‘As-a-Service’ technology landscape. Cloud vendors new and old will have to think about how to recalibrate their business in an uncertain demand climate. Second quarter earnings will be a early litmus test for the winners versus the losers. And tech buyers will have to find ways to continue delivering accelerated time to value from their cloud spend – or else their investment taps will slowly be turned off.

For more insights, watch our on-demand webinar: State of the Market Special Edition: Global Recession Scenario for IT Spending


  • [1] You could argue that the initial impact of the Covid-19 pandemic also fell into this timeframe, but the tech spend spike we saw in that period was unprecedented, and was driven by the need to keep businesses afloat where everything literally was required to be digital. Hence it is very unlikely to be repeated, so it is impossible to use a reference point in this regard.

Rick Villars - Group VP, Worldwide Research - IDC

Rick is IDC's chief analyst guiding research on the future of the IT Industry. He coordinates all IDC research related to the impact of Cloud and the shift to digital business models across infrastructure, platforms, software, and services. He helps enterprises develop effective strategies for using their diverse portfolio of cloud investments and applications. He supplies early guidance on implications of critical innovations such as the shift to cloud-based control platforms for deploying/managing infrastructure, data, and code delivery as well as the emergence of AI as a critical IT workload and part of all IT products/services.