Deepchecks can integrate with common databases (For example: S3, HDFS, BigQuery, Snowflake) for batch processing. We also have a python package that can integrate with real-time use cases with a few lines of code.
For more information, click here.
For a basic monitoring solution all we need is the model’s input. In order to receive more informative insights, we recommend to monitor multiple data points across the data pipeline (e.g. raw features, predictions, labels, training data).
There are a few types of issues we detect:
We also enable configuration of custom alerts, since you know your data better than us 🙂
Some platforms began to develop basic monitoring solutions, although they are far from providing an adequate solution to this critical problem.
From our experience, most of the problems that come up in production can only be detected with a comprehensive monitoring solution, that takes the model’s properties into account and compares different phases across the data pipeline.
We support any models adhering to the scikit-learn model api. This includes: TensorFlow, CatBoost, LightGBM, Keras, XGBoost, Caffe and scikit-learn models (as well as many others).
In addition we also support multi phase models, ensembles and models combined with business logic.
Deepchecks is capable of monitoring a single data point or multiple data points across a pipeline. From our experience, many valuable insights are only detected while monitoring multiple data points (this is necessary for both inter and intra pipeline monitoring).
Yes! Our robust architecture enables comparing different pipelines or live versions with each other. One application of this capability is monitoring A/B tests via our system, although there are various other applications.
There are two solutions for this important (and common!) issue:
Deepchecks offers both a SaaS solution designed for anonymized data, and an on-prem deployment option for non-anonymized data (which even works on air-gapped environments).
Yes! Apart from our own dashboard, our metrics and alerts can be sent to common monitoring and alerting tools, including Slack, Mail, Teams, Datadog, New Relic, Graphite, PagerDuty, Splunk, ServiceNow and more!
Yes, Deepchecks was built to support streaming data pipelines of extremely large volumes.
Please reach out to receive specifications and benchmark results.
Just a few more details before we answer your question.