The State Of MLOps
Monitoring Companies Market Review
Hi! The following is an exhaustive list of companies in the world of MLOps Monitoring and their segmentation by personas, supported data type (tabular, image, audio, etc), product features (data-integrity, data-quality, health, drift, bias & fairness, XAI, etc), product focus (data-centric or pipeline-centric), total funding to date (Aug 2021), company type (startup, open-source, corporate) and more. I gathered the data by researching documentation, blog posts, product demos, and marketing materials. The table is hosted on AirTable and is best viewed on a computer. Please click on the "Views" Icon to see all the other available views, and feel free to filter and sort, or click "View larger version to see it on a full screen".
If you would like to add your company to the MLOps-monitoring or upcoming MLOps-tools list, please fill the monitoring form or the tools form, respectively, or find them at the bottom of this page. If you find any inaccuracies please contact me.
Please note that I take care in manually validating every application, and I include only public information & released features.
TLDR; By analyzing the current space of MLOps Monitoring we see that most of the companies in this space are startups who are focused on data scientists, machine learning engineers, and developers and are focused on tabular data, and many are Israelis. The majority of companies are focusing on data monitoring, and a small minority are focusing on data-pipelines (e.g., DAGs, don't be confused with traditional APM). The three big cloud providers (FAAMG) are providing relatively basic to intermediate features, and don’t seem to have plans to become best-of-breed. We can see that the market has exploded and that there are many new players who share the same functionality but having their own point of view. The amount of money invested in the MLOps space is a staggering $3.8 Billion. We see small but mature startups that did not scale and therefore, exited prematurely.
I’m foreseeing a consolidation in the field, and I believe that eventually, the big guys will buy the small ones, the small ones will buy the tiny ones in an effort to be appealing to the big guys. If you are asking yourself which solution should you choose, the answer is not an easy one, although they all share the same basic functionality, there isn't a clear winner right now. Each company chose its own path, whether orchestration, insights, custom metrics, configurability, integrations, ML features, etc. My suggestion at this point in time is to understand your model monitoring needs, read their docs, ask for demos and decide for yourself.
Apply for the MLOps Market Review
*In AirTable, for the category & sub category column, I have used values taken from the 'List of tools for MLOps_v2_Dec 2020' by Chip H.
Monitoring Articles On Medium / Towards Data Science
On The Media
Andrew Ng answers my question on who will be the market leader in MLOps Observability in the coming 2 years.