Innovation and machine learning
11 years of machine learning projects
This is my 11th year as machine learning professional. Why then am I writing than about innovation? Machine learning projects often part of a larger innovation effort. I have seen many times that a lack of understanding of innovation leads to failure of machine learning projects.
This lead me early on to broaden the scope of my work:
Machine learning -> organizational change, innovation and machine learning.
Innovation
Innovation is complex and poorly understood, but it does not have to be. Here is a simple definition:
Innovation is the use of creativity to raise the performance of you business
What are performance and creativity? Instead of aiming at water tight definition, I give you some examples.
Performance
What outcomes can qualify as innovation? Here are some examples:
- Entering a new market
- Reaching more clients
- Expanding your product line
- Reducing costs
- Increasing sales
- Increasing margins
What outcomes do not qualify as innovation?
- Increased profits due to external factors like inflation, increasing demand, competitor failure, etc.
- Recovery from downturns that resulted from internal failure; e.g. a defunct manger is replaced.
Creativity
What is creativity? In the context of innovation, any idea that shows human resourcefulness and that does not obviously miss the point is creative. This does not have to be complex.
Here are some ideas that I consider resourceful:
- Start to follow up with clients, if this is not custom in your organization
- Track one or two key metrics that are not tracked yet
- Implement a new process that will save time or money
You may be surprised that no rocket science is needed to be creative. On the contrary, complex ideas often miss the point. Complexity itself is not a sign of resourcefulness.
Here are some ideas that I don’t consider creative, simply because they miss the point:
- We introduce a complex computer interfaces for elderly people, so they don’t need to face humans
- We optimize something that doesn’t need to be optimized
- We use advanced math when simple math will do
- We will collect all data and then analyze and predict everything
Let’s call these “in your head” ideas. Overthinking is dumb and not a sign of creativity.
Innovation and machine learning
Let’s apply my definition of innovation to machine learning.
Some machine learning efforts are part of regular operations. This is the case when a model needs to be trained periodically due to data drift. Another example is training an existing model for a new client, when new clients make up regular business.
In the majority of projects I do, machine learning is part of a larger innovation effort.
The goal should then be to raise the performance of the business; not to train a machine learning model This can be done by increasing sales, reducing costs, or improving the product.
Once the goal is understood. The next step is to be creative in achieving this goal. Forget about machine learning for a moment and think about what you can do and need to reach the goal. Often this is not just machine learning.
Innovation and machine learning in practice
With one of my clients I am developing a lead scoring model to increase sales. Right at the start we identified that a model will not increase sales by itself. A behavioral change is needed, which may be more challenging than building the model itself.
We decided to:
- Integrate a proof of concept in the sales process
- Find a group of sales persons to test the model and get feedback early on
- Reduce modeling work to a minimum until we validate the use