Spending on AI is exploding. IDC predicts today’s $12B/year investment will nearly quintuple over the next three years.
But while spending on AI initiatives is mushrooming, relatively few enterprise projects are being deployed. There are lots of reasons for this. Far too many data science teams, for example, are trying to build their own massive training data sets, which eats up budget and project cycles.
There’s another major obstacle to AI deployment, however, that we see all too frequently. It’s the outdated “waterfall” approach that teams take to planning, building and testing their AI systems.
The waterfall method was once the dominant approach to building application software for mainframes and workstations. Adherents to waterfall methodologies regarded applications as monoliths moving down an assembly line and not released until complete. First the entire application was mapped out. Then it was architected, and then coded. Finally, the entire application entered testing, and ultimately, deployment.
The waterfall method is very vulnerable to complexity. Complexity makes applications bigger, with more interdependencies. In waterfall these interdependencies aren’t fully explored until the penultimate testing stage, at which point any oversights or miscalculations that are discovered drive the entire project back to the architecture or coding stage.
While application development approaches have moved away from waterfall, AI projects have not. And these projects are not immune to the risks that waterfall creates.
In the case of AI projects, the waterfall method forces data science teams to identify all requirements and potential issues upfront. This is a restrictive approach that leaves room for error and risk. It is not until the team reaches the final stages of the project that they discover the mistakes made along the way or learn that they’ve built the wrong model.
This “get it all right before you use it” approach to AI is holding enterprises back from enjoying the benefits of machine learning.
Want to talk to Alegion about Agile AI? Contact us.
Next time we’ll talk about an alternative to waterfall…