MLOps community meetup #44! Last Wednesday, we talked to Savin Goyal, Tech lead for the ML Infra team at Netflix.

// Abstract:

In this conversation, Savin talked about some of the challenges encountered and choices made by the Netflix ML Infrastructure team while developing tooling for data scientists.

// Bio:

Savin is an engineer on the ML Infrastructure team at Netflix. He focuses on building generalizable infrastructure to accelerate the impact of data science at Netflix.

// Other links to check on Savin:

https://www.usenix.org/conference/opml20/presentation/cepoi

https://www.youtube.com/watch?v=lakPlz8GJcA&ab_channel=RConsortium

https://www.youtube.com/watch?v=-oMZAS9qfrE&ab_channel=AnalyticsIndiaMagazine

https://www.youtube.com/watch?v=yyWirT279tY&ab_channel=FunctionalTV

https://www.youtube.com/watch?v=QkRJ24Q0E-k&ab_channel=Matroid

----------- Connect With Us ✌️-------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Savin on LinkedIn: https://www.linkedin.com/in/savingoyal/

Timestamps:

[00:00] Background of Savin Goyal

[02:41] Breakdown of Metaflow

[05:44] In the stack, where does Metaflow stand?

[13:23] Where does Metaflow start in Runway Project?

[15:27] What tools or storage does Netflix use for DataOps, ie: the front-end management of data sets and how does that integrate with Metaflow? [18:56] Recommender Systems: Can you explain the other areas that you're using Machine Learning?

[22:27] What do you feel is the hardest part of building an operating Machine Learning workflow? [28:45] 3 Pillars: Reproducibility, Scalability, Usability.

[36:05] You give so much power to people. How do you keep them from going overboard?

[37:47] Can you explain this Pillar of Usability?

[41:09] Road-based access control has been coming up a lot recently. Does Metaflow do something specific for that?

[44:49] What are some learnings that come across that you didn't have since you open-sourced when you were working at Netflix?

[48:10] What kind of trends you have been seeing? Where do you feel like the market is going?

[50:33] Have you seen some companies really interested in Metaflow? How have you been seeing them combine other tools that are out there?