Growing AI capabilities in the organization

Avoid pitfalls and set your organization up for success.
Author

Sjoerd de Haan

Published

February 13, 2025

Do you need to develop AI as a core capability in your organization? In some cases you can buy a solution. In other cases you need to develop it yourself. This article get you started with developing models in your organization.

Building models is not for the faint of heart. So fasten your seatbelts and prepare for challenges, setbacks, failures and, if successful, great heights.

Indeed the rewards of the machine learning journey can be high. Success means that AI capabilities are embedded in the organization, that collogues appreciate them and use them. And they contribute to the bottom line.

Imagine the feeling when you get there with your team after a long and courages journey! I say courage because some courage is needed. Some models are hard to build, but getting the models used is even harder.

Pitfalls when developing AI capabilities

I have been there with many organizations, and I can say that it s absolutely hard to get it right.

The most common pitfalls are:

  • Strong focus on technology
  • Deliveries get delayed
  • The business is not engaged
  • Ambition level is low; there will never be a return on investment

Some of the underlying cause are:

  • Not having a clear vision
  • Not having a smart strategy to reach that vision
  • Not having the right people
  • Not understanding the complexity of AI in the context of the organization

Vision
Vision affects the value proposition and directly affects the ROI that you could possibly achieve. Vision is important, but it is not rocket science. Believes, convictions, trends and opportunities have equal weight in the equation. The equation then is simple:

If you put your rocket up to the sky if lift, if it pointing towards the ground, it will crash.

Strategy
Strategy is about how to get there. In the business case a strategy will emerge.

People
Ultimately, people make the difference. While not the focus of this post, I will say something about this right now.

Complexity
For a start you need to have someone that understands and manages complexity; see people. This will become apparent from the business case.

Business case: developing AI capabilities in a sales organization

A client of mine is a sales organization that wants to develop AI capabilities. A closed sale generates revenues at low marginal cost.

With conversion rates between 10% - 30%, a small improvement in conversion has a large impact on the bottom line. If we assume that the total costs are zero, lifting conversion rate from 10% to 11% results in an increase in profit of 10%. In reality the increase will be higher. Sales organizations spend money on every opportunity: think about marketing, sales people, sales tools, etc. If cost take out 50% of the revenue, the increase in profit will 20%, if we assume the marginal cost is zero.

The organization expressed a vision to let AI help the sales team to increase conversion rates. And for the future maybe even let AI take over parts of the sales process.

At this point in time now there is not much use in developing a perfect vision. If there is enough potential to get started, get started with a small project In this case the potential is evident. The organization has a substantial technology department, and developing data science capabilities makes sense.

Below we continue with the example of this organization.

The first AI project better be a success

The first project is crucial. Why? It is hard to appreciate in advance how though AI development is.

During the first project is will become apparent key players in the organization that

  • The investments for developing AI are substantial
  • The organization needs to change
  • The people need to learn and adapt

If this first projects fails, there will be not much enthusiasm left.

The meaning of success

The meaning of success is not always clear, so let me be clear here: technical achievements are not a measure of success. The more technical deliveries you deliver, the harder it will be for the project to become a success.

Do yourself a favour and stop counting technical deliveries. Even companies like Google and Open AI want embed their technical innovations into their products at some day. You should aim for no less.

Use the following intermediate milestones to create focus:

1. AI produces quality output
2. Adoption by the organization (AI output is used by sales agents)
3. ROI of AI is demonstrated (AI output is proven valuable or at least made plausible)

Scoping and downsizing the first project

A typical initial machine learning project has several stages; all of which are scoped to big.

For the sales organization in our buisness case, a next best action system would be a great first project.

Here we have the components that make up a next best action system for sales:

1. Recognize patterns in current opportunities from previous sales data 
2. Understand how actions influence sales outcomes
3. Recommend the best action at each stage, given the available information
4. Validate the the results

From step 4 we go all the way back to step 1 in a cycle; see Figure 1.

---
config:
  look: handDrawn
  theme: forest
---
graph TD
    A[Recognize patterns] --> B[Understand actions]
    B --> C[Recommend actions]
    C --> D[Validate results]
    D --> A
Figure 1: Next best action system

So far so good. The diagram is pretty simmple. Here is the culprit: each stage is composed of multiple sub-stages.

For example, stage 1 could be a classification model with the following sub-stages:

  1. Reproducible project structure
  2. Data engineering / cleaning
  3. Exploratory data analysis
  4. Exploratory feature analysis
  5. Feature engineering
  6. Base model
  7. Feature selection
  8. Model selection & tuning

And there is more… if you buy into stage 1, you get for free

  1. Three separate data sources
    1. structured data
    2. transcripts, chats and emails
    3. one bonus dataset
  2. The structured dataset has over a 100 columns, which gets you 200+ features
  3. The sales funnel has about 10 stages, so you get
  • One model for each stage
  • Different features apply for each stage

And that’s not all… If you say yes today, we give you a free set of data quality issues:

  1. The data is likely not clean, and there will be missing values
  2. The data is likely not balanced
  3. The data may be not stationary
  4. The data may be not independent
  5. The data may be not identically distributed

You get it. If we are not selective in what we deliver, we end up with 3 data sources, 200+ features and 10 models (one for each stage). If we include careful data checks, this will amount to a year of work. Just for stage 1.

The minimum viable project

The problem with scoping too big is that it detriments the chance of success. It leads to the technology focus pitfall that most companies fall into.

It is not easy to avoid waste entirely in a pilot project. I am happy if, after a successful project, no more than 50% of the technical development efforts turn out to be waste. I bet for most companies the percentage of waste is closer to 90% or 95%.

Again, the purpose of the first project is to look for a heart-beat; a sign of ROI. The 95% of waste produced by most companies includes everything that is not the bare minimum required to measure a pulse.

We want to go from pattern recognition to validation is quick as possible. If we find no pulse, we can always decide to add later.

Strategy

The mantra for the initial project is minimalism.

  • Work with a minimal set of features, actions, models
  • Engage the organization in the lightest possible way (but early on)
  • Reduce tuning, model selection and optimization efforts
  • Use milestones that culminate in proving ROI.

People

The core roles that you need for a first project are roughly:

AI champion
A person that understands the business, the organization and technology. Is enthusiastic to drive the project go back and forth between the business and the technology. Evangelist. Is able to make decisions on the spot.

Data scientist
Someone who can handle data, write code, make reports, make models, test hypothesis and validate results.

Complexity guru
Managing complexity is key in the first project. This is something that requires experience. A combination of analytical skills, non-linear thinking, and soft skills is required navigate the landscape of AI project development.

Early adaptor
Someone in the organization who is committed to experiment with the AI output at an early stage. This person makes an effort to sport opportunities no matter how poor the AI output is. This early adaptor is crucial to get the feedback needed to steer the project in the right direction.

Conclusion

Developing AI capabilities in an organization is a complex combination of business, people, and technology.

For the Dakar rally, you want a aggressive car to navigate the bumpy terrain; not a the save and polite commuter car.

“Playing save” by focusing on extensive technological deliveries is a sure way to fail. An aggressive minimal approach works best to steer around common pitfalls.

The first project is crucial and the mara is:

cut, downsize, cut.

The first project is about finding a sign of ROI; a heart beat ❤️.