You walk into a restaurant that claims to have the world’s best chef. Once you’re seated, you ask for the menu. The waiter says, “There’s no menu here. We cook whatever you want.”
The next time you’re out for dinner, you walk into a restaurant and the menu is handed to you. You order the dish that best fits your likes and preferences.
Of these two instances, which do you think is more time-and energy-efficient? We’re going to go ahead and answer for you: the second.
That, in a nutshell, is what AI model stores do for enterprises. Let’s get down to details.
What is an AI model store?
An AI model store is a centralized repository of pre-trained AI models. It serves as a marketplace where users can discover, access, and download the models they need, for a wide range of applications and tasks.
This includes models trained for tasks such as image recognition, natural language processing, speech synthesis, recommendation systems, sentiment analysis, and more. They are typically built using ML techniques and are trained on large datasets to achieve high performance and accuracy.
These models are developed by different researchers, companies, or open-source communities. They provide a convenient way for developers and data scientists to leverage pre-existing models rather than building models from scratch. Users can save time and effort in training models, especially for complex tasks that require significant computational resources and labeled data.
Model stores typically have the following features:
- Version control: Track the history of each model, including the model files, artifacts, and metadata. This allows users to compare different versions of a model and track its performance over time.
- Model lineage: Track the lineage of each model, showing how it was created and which data it was trained on. This helps to ensure the reproducibility of models and to identify potential sources of bias.
- Model search: Provide a way to search for models by name, tags, or other criteria. This makes it easy to find the right model for a particular task.
- Model deployment: Deploy models to production. This ensures that models are available to users and that they are updated when new versions are released.
This library of datasets will help AI/ML enterprises churn out projects more efficiently. Other benefits include:
- Improved model management
- Increased model reproducibility from source code and data
- Improved model deployment and tracking
Why model stores?
1. Better tracking
Versioning is a constant in machine learning. For an enterprise that handles more models than it can count, it makes more sense to build upon existing models than to create an entirely different model for each client, as long as the models fit the purpose. Such versioning requires crisp model management.
Global ride hailing app Uber, an enterprise known for its heavy employment of ML, took the help of Comet’s ML platform to manage their experiments. Thanks to their wide geographic range, the ride hailing app holds ginormous amounts of data. Olcay Cirit, research scientist at Uber AI, says, “We can’t simply look at a global accuracy metric. Rather, we must use sliced analysis to gauge the improvements or degradations in each market and product category.”
With Comet, experiment tracking became exponentially easier. The teams could access / add product-specific metrics without having to make changes to the platform
2. Better access
Upgrades and developments to AI models get better visibility in a model store. They can be compared and contrasted. This aids the selection of the appropriate model for the solution. This comes particularly handy when there’s an overabundance of models.
Inverted AI, an organization based in Vancouver, British Columbia, is working on developing game-changing APIs in the automated vehicles industry. This simulation-based project is being developed on the ‘Imagining the Road Ahead’ model.
Needless to say, the number of road traffic simulation models soon snowballed. With Weights & Biases Artifacts, Inverted AI could catalog various models and make it centrally accessible for the entire team to work on. “We got to the point where we had so many models and data versions that we simply couldn’t manually keep track of all of them. Once we started taking advantage of Artifacts, it’s been very helpful,” said Adam Ścibior, CTO at Inverted AI.
3. Better efficiency
Without proper categorization, searches could consume a huge chunk of time. When the models and the relevant data are properly librarized, the time spent on finding something could be channeled towards something more productive.
CareRev, a healthcare staffing solutions platform, found this point particularly advantageous once it started using Comet’s ML platform.
CareRev acts as a marketplace that connects healthcare professionals with facilities and patients. With Comet’s model management tools, CareRev could successfully cut down the time improper model management costed them.
“From training to pushing the new model out, it took us a couple of hours because of the CI/CD pipeline we integrated with the model registry. So that could have been a one or day two thing if we didn’t have Comet,” says Kes Crandall, ML Engineer at CareRev.
So, are AI model stores a necessity? Or just another layer?
The answer to that depends on the amount of data an enterprise handles. For organizations that have fewer models, model management may not be an immediate problem to attend to. For enterprises with plenty of models, not having a model management in place could prove to be an expensive mistake.
It also depends on how fast they want to churn out innovative solutions. If they have increased access to existing AI models, there’s less time spent reinventing the wheel. And more time spent on bringing in additional revenue.
With the growing datafication of enterprises, it is only a matter of time before model stores are a part of daily lexicon.
