Lessons learned running AI-powered solutions in production

Abstract

This paper examines lessons learned from running artificial intelligence (AI)-powered solutions in production. It starts by describing the evolving approach the author’s company has developed to gathering training data, followed by its experiences running AI-powered solutions at scale, some particular tips around managing algorithmic complexity and, finally, lessons learned regarding testing and regression. The author also shares insights gained while performing proofs of concept (PoC) for clients. A common theme throughout the paper is the critical role of the subject-matter expert (SME) in automating complex business processes. The team’s SMEs may not even have detailed knowledge of the technology in play, but their expertise in the business domain is crucial to success in these endeavours.