Validation Methodology of Large Unstructured Unsupervised Learning Systems

Snapwiz's Data Science team uses multiple levels of validation that cover various models in the domains of adaptive learning, skill networks, assessments, professional development and career recommendations. The models that power our applications includes psychometric, machine learning tools, natural language processing and custom built. I will describe our best practices and show validation of specific examples in our applications. This includes a suite of specific unit tests that cover specific logical implementations of algorithmic equations to automated cron jobs that daily run thousands of simulated user cases through the end-to-end application. Once failures are detected, the QA team has a suite of tools that they can use to do first level debugging before passing off to developers.