Cleanlab: Labeled Datasets that Correct Themselves Automatically // Curtis Northcutt // MLOps Coffee Sessions #105
MLOps Coffee Sessions #106 with Curtis Northcutt, CEO & Co-Founder of Cleanlab, Cleanlab: Labeled Datasets that Correct Themselves Automatically co-hosted by Vishnu Rachakonda. // Abstract Pioneered at MIT by 3 Ph.D. Co-Founders, Cleanlab is an open-source/SaaS company building the premier data-centric AI tools workflows for (1) automatically correcting messy data and labels, (2) auto-tracking of dataset quality over time, (3) automatically finding classes to merge and delete, (4) auto ml for data tasks, (5) obtaining and ranking high-quality annotations, and (6) training ML models with messy data. Most of the prescriptive tasks (finding issues) can be done in one line of code with their open-source product: https://github.com/cleanlab/cleanlab. // Bio Curtis Northcutt is the CEO and Co-Founder of Cleanlab focused on making AI work reliably for people and their messy, real-world data by automatically fixing issues in any ML dataset. Curtis completed his Ph.D. in Computer Science at MIT, receiving the MIT Thesis Award, NSF Fellowship, and the Goldwater Scholarship. Prior to Cleanlab, Curtis worked at AI research groups including Google, Oculus, Amazon, Facebook, Microsoft, and NASA. // MLOps Jobs board https://mlops.pallet.xyz/jobs MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links https://github.com/cleanlab/cleanlab https://cleanlab.ai/blog/cleanlab-history/ https://labelerrors.com/ https://l7.curtisnorthcutt.com/ https://nips.cc/Conferences/2021/ScheduleMultitrack?event=47102 https://www.youtube.com/watch?v=ieUOv1sQPlw https://cleanlab.typeform.com/to/NLnU1XZF Cameo cheating detection system: https://arxiv.org/ftp/arxiv/papers/1508/1508.05699.pdf The Cathedral & the Bazaar book: https://www.amazon.com/Cathedral-Bazaar-Musings-Accidental-Revolutionary/dp/0596001088 --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Curtis on LinkedIn: https://www.linkedin.com/in/cgnorthcutt/ Timestamps: [00:00] Introduction to Curtis Northcutt [00:30] Difference between MLOps and Data-Centric AI [04:04] Realizing the problem of data quality in ML manifesting [05:11] Computer vision problems [06:54] War story that got Curtis into Data-Centric AI [13:50] Overview of Curtis' vision [14:45] PU Learning [21:25] Consistency Rate and Flipping Rate [25:25] One line of code [29:48] Models makes mistakes [33:09] Cleanlab play with the environment [36:30] How ML Engineers should approach data quality problem [42:42] Quantum computing [46:39] Result of confident learning [52:31] Utility for small data sets [53:53] Cleanlab's huge success stories [56:13] Rapid fire questions [58:58] Cloudy and mystified space [1:03:46] Cleanlab is hiring! [1:05:06] Wrap up