INSUBCONTINENT EXCLUSIVE:
Machine learning is a complex process
You build a model, test it in laboratory conditions, then put it out in the world
After that, how do you monitor how well it’s tracking what you designed it to do? Arthur wants to help, and today it emerged from stealth
with a new platform to help you monitor machine learning models in production.The company also announced it had closed a $3.3 million seed
round, which closed in August.Arthur CEO and co-founder Adam Wenchel says that Arthur is analogous to a performance-monitoring platform like
New Relic or DataDog, but instead of monitoring your systems, it’s tracking the performance of your machine learning models.“We are an
AI monitoring and explainability company, which means when you put your models in production, we let you monitor them to know that they’re
not going off the rails, that you can explain what they’re doing, that they’re not performing badly and are not being totally biassed
— all of the ways models can go wrong,” Wenchel explained.Data scientists build machine learning models and test them in the lab, but as
Wenchel says, when that model leaves the controlled environment of the lab, lots can go wrong, and it’s hard to keep track of that
“Models always perform well in the lab, but then you put them out in the real world and there is often a drop-off in performance — in
So being able to measure and monitor that is a capability people really need,” he said.Interestingly enough, AWS announced a new
model-monitoring tool last week as part of SageMaker Studio
IBM also announced a similar tool for models built on the Watson platform earlier this year, but Wenchel says the involvement of the big
guys could work to his company’s advantage as his product is platform-agnostic
“Having a neutral third party for your monitoring that works equally well across stacks is going to be pretty valuable,” he said.As for
the funding, it was co-led by Work-Bench and Index Ventures, with participation from Hunter Walk at Homebrew, Jerry Yang at AME Ventures
and others.Jonathan Lehr, a general partner at Work-Bench, sees a company with a lot of potential
“We regularly speak with ML executives from Fortune 1000 companies and one of their biggest concerns as they become more data-driven is
model behavior in production
The Arthur platform is by far the best solution we’ve seen for AI monitoring and transparency…” he said.The company, which is based in
New York City, currently has 10 people
It launched in 2018, and has been heads-down working on the product since
Today marks the release of the product publicly.
