Committing to Cloud Native: Recent Episodes

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer Guest Alex Ellis Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about the confluence of Cloud Native technology and Open Source. Today, we are super excited to have as our guest Alex Ellis, who is the Founder of OpenFaaS, which is one of the most popular open source serverless projects, as well as a CNCF Ambassador. Alex takes us on his journey on how he Founded OpenFaaS. He talks about how important independence is to him, OpenFaaS, and other projects he’s worked on, and shares some influential books that he read that helped him in his journey of setting up a company. We also hear his views on how to build a sustainable open source community. Alex goes in depth about some of his other projects he created, recently being invited to join GitHub Stars program, and three eBooks he self-published that you should check out online! Go ahead and download this episode now! [00:01:47 (https://podcast.curiefense.io/25?t=107)] Alex tells us what OpenFaaS is, how adoption has gone, and how many people have used it and committed to it. [00:04:27 (https://podcast.curiefense.io/25?t=267)] Richard wonders how Alex funds the project, if there’s a business model, and if it’s big enough to have people afford to work on it. [00:07:31 (https://podcast.curiefense.io/25?t=451)] Justin brings up a keynote that Kelsey Hightower did on the benefits of Amazon’s Lambda at KubeCon and wonders if that or just other presentations in the community have an effect on OpenFaaS to become a project of its size now. [00:11:06 (https://podcast.curiefense.io/25?t=666)] How do people react to using OpenFaaS in their organization since it’s not in the CNCF as an incubation project? [00:12:52 (https://podcast.curiefense.io/25?t=772)] Alex talks about independence and how important it is to him, OpenFaaS, and all the other projects he’s worked on. He also talks about a few good books he read that helped him with his journey in setting up a company such as, Million Dollar Consulting. [00:17:22 (https://podcast.curiefense.io/25?t=1042)] We learn from Alex his views on how to build a sustainable open source community and another great book he learned from called, The Right It by Alberto Savoia, who is the Head of Innovation at Google. [00:22:42 (https://podcast.curiefense.io/25?t=1362)] Alex is known for creating other projects which seems to go against the idea of doing something small and seeing if it works, so we find out why he likes to create other projects. [00:27:51 (https://podcast.curiefense.io/25?t=1671)] Alex tells us about being very active on Twitter, he talks about Daniel Vassallo who created a course on how to create a Twitter following, and about writing his blog posts. [00:31:38 (https://podcast.curiefense.io/25?t=1898)] Alex got something from GitHub twenty-three hours ago. Find out what he got and why. [00:36:47 (https://podcast.curiefense.io/25?t=2207)] What is GrowLab? [00:39:07 (https://podcast.curiefense.io/25?t=2347)] Find out where you can follow Alex online and three eBooks he self-published. Quotes [00:04:46 (https://podcast.curiefense.io/25?t=286)] “Yeah, I mean it isn’t big enough to have people afford to work on it. That’s probably the biggest lie of open source is, the bigger something is that the more money is rolling into it.” [00:05:00 (https://podcast.curiefense.io/25?t=300)] “Even community contributions, as lovely as they are, there aren’t people with full-time jobs who said on their CV that says ‘Full-Time OpenFaas Contributor.’ That just isn’t the case with something like this.” [00:05:42 (https://podcast.curiefense.io/25?t=342)] “It basically says something like open source isn’t about you.” [00:06:52 (https://podcast.curiefense.io/25?t=412)] “In marketing framework, you’ll read is that a sustainable business has value exchange or value capture that is equal between all three parties: the consumers of it, the company or the creator behind it, and the community of partners, contributors, and sort of third parties.” [00:08:28 (https://podcast.curiefense.io/25?t=508)] “I actually think that managed Cloud functions are a really smart idea. They’re great to use. The cost cannot be beat in any way.” [00:10:35 (https://podcast.curiefense.io/25?t=635)] “What you don’t want to create, something I’ve really learned, is a commodity. And further than that, you don’t want to create something where there’s no capability for you to capture value from it.” [00:13:00 (https://podcast.curiefense.io/25?t=780)] “I mean, for me, what I mean by independence is not being employed by any one company.” [00:14:17 (https://podcast.curiefense.io/25?t=857)] “I had created some insights in the industry and the only way I could get the job that I wanted was by creating it myself.” [00:25:22 (https://podcast.curiefense.io/25?t=1522)] “Well, some of the earliest clients I was working with, their audience wasn’t really at the stage where they were ready for full on Kubernetes, but K3s was nascent, it was stirring a lot of hearts, and people really like the idea of simplicity.” [00:30:41 (https://podcast.curiefense.io/25?t=1841)] “And what I learned in the industry was you really want to focus on the impact that you’ve had rather than on the detail.” Links Alex Ellis Twitter (https://twitter.com/alexellisuk/) Alex Ellis Linkedin (https://www.linkedin.com/in/alexellisuk?originalSubdomain=uk) Alex Ellis Website (https://www.alexellis.io/) OpenFaaS (https://www.openfaas.com/) Open Source is Not About You-GitHub Gist (https://gist.github.com/g1eny0ung/9e7d4d0f72547a8d156452e76fa8f7a3) GrowLab (https://growlab.dev/) Million Dollar Consulting: The Professional’s Guide to Growing a Practice by Alan Weiss (https://www.amazon.com/Million-Dollar-Consulting-Professionals-Practice/dp/1259588610/ref=sr_1_3?crid=ELRL5Y8FVNF8&dchild=1&keywords=million+dollar+consulting+by+alan+weiss&qid=1634322457&sr=8-3) The Right It: Why So Many Ideas Fail and How to Make Sure Yours Succeed by Alberto Savoia (https://www.amazon.com/The-Right-It-Alberto-Savoia-audiobook/dp/B07MWD2GKL/ref=sr_1_2?crid=3UIUG8KNB14XU&dchild=1&keywords=the+right+it+alberto+savoia&qid=1634323412&sr=8-2) Everyone Can Build A Twitter Audience Course by Daniel Vassallo (https://dvassallo.gumroad.com/l/twitter-audience) Daniel Vassallo Twitter (https://twitter.com/dvassallo?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Serverless For Everyone Else by Alex Ellis (https://openfaas.gumroad.com/l/serverless-for-everyone-else) Netbooting Workshop for Raspberry Pi with K3s by Alex Ellis (https://openfaas.gumroad.com/l/netbooting-raspberrypi) Everyday Golang by Alex Ellis (https://openfaas.gumroad.com/l/everyday-golang) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript Processing... 😅 Special Guest: Alex Ellis.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman Guest Justin Garrison Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast sponsored by Reblaze where we talk about the confluence of Cloud Native technology and Open Source. Today, our guest is Justin Garrison, who is a Developer Advocate at Amazon. He also worked at Disney Animation and Disney Streaming Services, and we found out what he did there before going to Amazon. Justin created bashScheduler, and he tells us why he said, “This is a really bad idea!” We learn more about his book, Cloud Native Infrastructure: Patterns for Scalable Infrastructure and Applications in a Dynamic Environment, a blog post he wrote about “The Economics of Writing a Technical Book,” a talk he did for DevOpsDays Portland 2021 called TikTalk, and his cool “manpage” resume. Also, he shares some insight on how he sees recognition as something we could really bring into the software industry. Go ahead and download this episode now to find out so much more! [00:01:18 (https://podcast.curiefense.io/24?t=78)] Before Amazon, our guest, Justin, worked at Disney and he fills us in on what he did there and what he does now. [00:04:36 (https://podcast.curiefense.io/24?t=276)] Justin tells us more about his book, Cloud Native Infrastructure: Patterns for Scalable Infrastructure and Applications in a Dynamic Environment. [00:08:46 (https://podcast.curiefense.io/24?t=526)] What’s going on with remote conferences and talks? [00:11:17 (https://podcast.curiefense.io/24?t=677)] Find out more about a talk Justin recorded for DevOpsDays Portland 2021 called TikTalk. He also tells us if there is an open source community on TikTok. [00:14:27 (https://podcast.curiefense.io/24?t=867)] Justin created bashScheduler and we find out what it does and why he said, “This is a really bad idea!” [00:20:50 (https://podcast.curiefense.io/24?t=1250)] How did Justin get a Linux.com email address? [00:21:43 (https://podcast.curiefense.io/24?t=1303)] Justin Dorfman is impressed with Justin’s man-page resume and he fills us in on the details and some tips on doing a resume. [00:24:26 (https://podcast.curiefense.io/24?t=1466)] Justin mentions a brag document that he used by Julia Evans and her website called Wizard Zines. [00:26:25 (https://podcast.curiefense.io/24?t=1585)] Will Justin write a follow up book to the Cloud Native Infrastructure? [00:29:12 (https://podcast.curiefense.io/24?t=1752)] Justin tells us about a blog post he wrote three years ago called, “The Economics of Writing a Technical Book.” [00:30:43 (https://podcast.curiefense.io/24?t=1843)] When Justin was at Disney, he got a movie credit for working on Zootopia which won an Oscar, and he talks about recognition being something he could see bringing into the software industry. He also shares something interesting about movie credits and recognition. [00:34:33 (https://podcast.curiefense.io/24?t=2073)] The guys chat about how CNCF has done a great job about having the “Chop Wood Carry Water” Awards at KubeCon and the people behind the scene that have such a huge impact in the foundation. [00:36:46 (https://podcast.curiefense.io/24?t=2206)] All Things Open 2021 is coming up and Justin will be giving a talk called, “Internet Scale, Open Source with Kubernetes,” that you should check out. Links Curiefense (https://www.curiefense.io/) Justin Garrison Twitter (https://twitter.com/rothgar) Justin Garrison Linkedin (https://www.linkedin.com/in/justingarrison/) Justin Garrison Website (https://www.justingarrison.com/) Justin Garrison TikTok (https://www.tiktok.com/@justinleegarrison) Infrastructure for Entertainment-Justin Garrison (YouTube) (https://www.youtube.com/watch?v=VtedIghTPzI) DevOpsDays Portland 2021- Justin Garrison- Ignite- TikTalk (YouTube) (https://www.youtube.com/watch?v=kQJYlZUhBt8) Cloud Native Infrastructure: Patterns for Scalable Infrastructure and Applications in a Dynamic Environment by Justin Garrison (https://www.amazon.com/Cloud-Native-Infrastructure-Applications-Environment/dp/1491984309) The Economics of Writing a Technical Book by Justin Garrison (Medium) (https://rothgar.medium.com/the-economics-of-writing-a-technical-book-689d0c12fe39) bashScheduler-GitHub (https://github.com/rothgar/bashScheduler) Committing to Cloud Native Podcast-Episode 22-Thoughts on Bash Becoming Interplanetary and More with Brian J. Fox (https://podcast.curiefense.io/22) Justin Garrison - man-page resume (https://www.justingarrison.com/resume.html) Wizard Zines (https://wizardzines.com/) Get your work recognized: write a brag document by Julia Evans (https://jvns.ca/blog/brag-documents/) All Things Open 2021 (https://2021.allthingsopen.org/) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript [00:02] Justin Garrison: It's slightly different than infrastructure is code environment where you have a repo of YAML files or something. Whenever you change them, it pushes out and it says, okay, now I'm going to make the world look like that. But this infrastructure of software has a tighter loop. It has more consistency to say, hey, if anything changes, right now, I'm going to revert that change, I'm going to make sure that the world concept looks like this with running software, and not just a repository full of code. That really was like the key thing that we came out of cloud native infrastructure. We saw that pattern over and over again, at successful companies. Amazon, Netflix, Google, Microsoft, like all these companies are focusing on how do we run our infrastructure, not by people, always reverting changes, but software doing that, and Kubernetes was, again, the controllers pattern was kind of the Northstar for us to say like, this is how it's done. [00:52] Announcer: Hello, and welcome to committing to cloud native, the podcast where we talk about the interface between open source and cloud native. We're super excited about our guest today. I really hope you enjoy this conversation. [01:07] Justin: Hey, everyone, I'm here with Justin Garrison. He's a developer advocate at Amazon. He lives in Los Angeles like me, we share a first name. Hey, Justin, how are you doing? [01:16] Justin Garrison: I'm doing just fine, Justin. [01:18] Justin: Awesome. Before Amazon, you were at Disney, what's under the hood there. You work on Disney plus talk about this. [01:26] Justin Garrison: I was at Disney animation for four and a half years doing infrastructure for the future animated films, working on render farm artists, workstations, internal servers, which is a great eye opening experience. I learned a lot of deep Linux technology and kind of how to run things at scale, especially on prem. But we're also using multiple different services and that was really when containers were starting to come around. I started focusing more and more on how that can enable developers to deploy things on prem or in the cloud. It was really impressive for me just to be able to work in that environment. [02:00] Then I moved over to Disney streaming services, which created Disney plus ESPN plus a lot of different smaller streaming services in there. I write infrastructure for them. Behind the scenes of Disney plus as Amazon. It's a lot of Amazon, I really started learning a lot more about how to run services in Amazon at scale, and how to do things with native services. My team was specifically in charge of the centralized container infrastructure we had. With the containers that we were running in Amazon, what does that look like? How did we enable developers to deploy in simple ways, so there's actually quite a few different Disney plus talks that reinvent from last year. [02:37] If you want to find out some of the details about how Disney plus runs very specific things, I did three or four talks at reinvents, which go deep into how we were deploying things, how we were running things, what native services we were using at Disney plus. I actually, I did a talk with my old co worker Zack. We were on the same team at Disney plus, and then I had switched over to Amazon. It was great talking to him about how we created the deployment tools and how we were managing the centralized infrastructure. Lower the cognitive load for a lot of developers to not have to learn everything about the entire stack of like, okay, where's the Linux running? Where's the container fit in? Where's the scheduler fit in? Where's Amazon components and VPC in there. [03:17] We kind of took care of all that for them, and gave them more of a platform style environment of just saying, hey, I have an app, here's my container, deploy 100 bucks, and we'd make sure that gets done, it's highly available and all of the best practices that we had for logging and monitoring and some defaults for dashboards. All that stuff just got created for them out of the box, and they didn't have to really worry about it or think about it. Then they could tweak or change things as they had need. [03:44] Justin: Did they just like poach you from reinvent? [03:49] Justin Garrison: I was really excited learning so much about Amazon from Disney Animation, and Disney streaming, and just saying, hey, like I have learned a lot in the last six years at Disney. I could really turn that around and help more people do these things in Amazon do very similar things in Amazon. I was first and foremost a customer that was just like, telling people about it. I was going into my own conferences and telling people hey, this is Kubernetes. Here's EKS, here's how containers help and doing those things day to day. Then now I do it for my day job. It's not actually something I do on the weekends or at nights or anything. It's like, hey, when I reached out to Amazon and said, I love these services, can I help other people use them better? I had a great interview experience and onboarding, to say like, this is how we're doing these things here. If this is the right way, I would love to help more customers and more people just be successful in these areas. [04:36] Justin: I mean, you literally like wrote the book on cloud native infrastructure. Was that before or after you joined? [04:43] Justin Garrison: That was before I was at Disney animation. That was before Disney plus. Someone in the Kubernetes community reached out and said, hey, I saw you were doing some things and I was doing writing on the side my own blog. I've written for blogs in the past. I really liked writing. I like written communication and someone reached out, looking for an author for this style book, and it was really around CNCF tooling and how that looks. The first time I started outlining it, it was early days of Kubernetes. We weren't really sure what we wanted to put in the book or anything. I reached out to Kelsey at the time was writing Kubernetes up and running. I asked him, hey, what advice do you have for writing a book, and the first thing he said was get a co author. [05:23] I was like, that's great advice. I didn't know a co author at the time. I don't know who to reach out to. I started talking to people, especially within the CNCF community. Multiple people recommended Chris. Chris, and I had never met before, we had known each other like, in passing in the community, but never met. With that, I reached out to her, and she brought so much great experience. She was a SIG chair lead for SIG AWS inside of the Kubernetes community already. I was like, hey, you're doing a lot in AWS, she was a co maintainer for cops, which is a Kubernetes deployment tool on top of AWS. [05:58] We started talking, and it was great just to be able to take her experience from what she was doing in open source and the community, and what I was doing sort of in an enterprise side of things, to be able to say, like, hey, how does this work for anyone, and the first iteration of the book changed a lot, because at first, it was really about Kubernetes. It was about how to like set up projects and where they fit in. We wrote the first draft, and we threw away most of it, because it was already out of date. It was something that was just like, these projects change too fast, this environment changes too fast. We went back to say, okay, what are the patterns? [06:29] What are the things that really will make the book last more than two years? What can we put in here to teach someone, hey, don't look at like a new tool, or a new, you know, option for something, look at how it works, and look at where that gets applied. That really got us down to what we call infrastructure as software, which is what you see inside of you using Kubernetes, the control loop, it has software that runs. It takes some data, and it constantly looks at that state of the world and says, what's the desired state that you want me to change? What's the current state of the world? How do I make that happen, and that's what the controller constantly does over and over again. [07:08] It's slightly different than infrastructure as code environment where you have a repo of YAML files or something. Whenever you change them, it pushes out and it says, okay, now I'm going to make the world look like that. But this infrastructure of software has a tighter loop. It has more consistency to say, hey, if anything changes, right, now, I'm gonna revert that change. I'm going to make sure that the world constantly looks like this with running software, and not just a repository full of code. That really was like the key thing that we came out of cloud native infrastructure. We saw that pattern over and over again, at successful companies, you know, Amazon, Netflix, Google, Microsoft, like all these companies are focusing on how do we run our infrastructure, not by people, always reverting changes, but software doing that. [07:54] Kubernetes was, again, the controller's pattern was kind of the North Star for us to say, like, this is how it's done. But also calling out other places where it's done. Like Netflix has chaos monkeys, that's like a controller pattern. That's like, doing the wrong thing. Essentially, it's causing chaos, but it's still infrastructure of software. In the case of like, it's not waiting for a git commit, it's not waiting for someone to make a change. It's always looking at the environment, always measuring the environment. It says, hey, what happens if DNS goes away, or this instance goes away or something, and it can do that as running software. You want them more productive infrastructure as software patterns, not necessarily destructive, but chaos engineering has grown into its own to say, oh, hey, like, there's a lot of great tools that will also allow you to verify that your infrastructure tooling will correct things on the fly and not wait for a human intervention. [08:47] Justin: What about remote conferences and talks. What's going on there? [08:53] Justin Garrison: With COVID a lot has changed in that space. I've tried to be innovative. My room is actually tore apart right now because I'm recording a couple of conference talks. I'm trying to do it in a different way. People like watching remote conference talks. This is not like a PowerPoint slide deck isn't always the most entertaining thing. But if I look at things like YouTube, a normal YouTube video and say, like, hey, what's successful here and it is more engaging, there's a little more polish and and producing put into it. Then I also think of like, if I don't want to watch a conference, what is something that I do want to watch? [09:28] To me, it was like, well, I'll watch TV episodes. I watch 20 minutes of a TV episode. I might learn something from it. I'll do that anytime. Like I'll just keep binge watching them. But I won't do that with conference talks. What if conference talks were more like TV shows? What if we actually change the format? What if there was no slide deck? What if it was tell a story? What if it was entertaining first, and if you learn something that's good and there's actually a quote from Walt Disney where he says if you make something entertaining, if someone learns something from it, that's great. But if you make it you know, educational first and it's not entertaining, what's the point/ [10:01] That's not his goal. His goal was entertainment first, hopefully someone learned something too. I've really taken that of like, that's what I'm trying to do. I had a cube con talk last year, which got a lot of great feedback of there was no slide deck, I literally filmed the mini movie in 20 something minutes or 30 minutes. I tried to teach people what it was like to run infrastructure for this entertainment business, because it's not well known for a lot of people. I literally started with like, what is it like to render a movie? What does that infrastructure look like, then what does it look like to stream that movie? What are different design considerations? Why is that infrastructure different? Why is it something that is important to have in both sides, like you can't have the same specialty of just like, oh, the render infrastructure can't also stream the movie that needs to be separate for various reasons and why. [10:47] Justin: Well, it's interesting, because obviously, with the past couple of years, everything's been remote and in terms of events, and I saw a few people give recorded presentations, and they were really good. There's some people that are just like, oh, it doesn't count if it's recorded. I'm like, why? You know if it's interesting, and it keeps you glued, like you were saying, like a TV show. [11:17] Justin Garrison: I don't know when this podcast is airing, but I have a talk that I recorded that's going live next week for DevOps, in Portland. The talk is called Tick Talk, but I recorded five Tick Talks. Five Tick Talk videos, and I stitched them together and I said, hey, I could tell a story with Tick Talks. Because that as a format is really interesting to me, and it adds some things that you can't do in a normal style. Like, here's my slide deck, here's some things to talk about. Like I'm actually able to switch scenes, I even switched clothes, I recorded some of them on different days. There was a lot that went into doing that style format that was different, but being able to be creative in that space, to me has been just a ton of fun. I love bringing just that like different view on like, how do we make this better for the audience. Still, they can learn something from it. It doesn't have to be just here's a bunch of data. Oh, like learn one or two things and be entertained and more people will see it and more people will learn. [12:16] Justin: How is the open source community on tik tok? My fear with Tik Tok is I'm gonna get addicted because every time I see a Tik Tok video, I'm like, whoa, where the hell did the time go? I was like, I can't install this. [12:30] Justin Garrison: It's definitely an outlet for a lot of people. What are you watching Tik Tok for. Most people, I think are watching it for entertainment. They want to be entertained, they want to feel some emotion. A lot of times that's laughter, especially in the last year and a half of you know, being on lockdown and being remote. It's like people need to laugh. It's something that is kind of core to who we are and tick tock in a lot of ways brings that. In small chunks of one to three minutes, I can just keep saying that wasn't funny, keep going. But it's important for everyone. There are some people that do really cool, like developer content. I really admire the Dev Talk sort of space where a lot of people want to learn, like change their lives. [13:05] Being a developer breaking it technology for so many people is life changing. It's life changing for what they're able to do, where they're able to work, how they get jobs and being able to afford things. There's quite a few people that I've seen on there that some of it is, the funny side of it, where like, hey, I'm a developer, only a developer would understand this joke. Also the other side of like, hey, if you want to learn how to be a developer, here's some links. Here's some things you should learn. Both sides are valid, because it's needed for depending on where people are at. But it's cool, I follow some of them and I've seen those videos on my for you page. [13:42] It's really great just to see how people are using that. My content that I've posted is mostly like on the funny side, not sometimes on the educational side, but I really try to like, figure out the format and how you can use these different formats, podcasts, YouTube videos, Tik Toks, there's so many different mediums and they all have pros and cons. [14:01] Justin: Yeah, no, I mean, I think that's interesting. I've seen a few that were front end focus, like the CSS community, like they always have some good memes on Twitter, but like I'd never like thought cloud native and like low level programming would have like a market there. But please drop your Tik Tok link in the chat. We put that in the notes. That's something I'm definitely gonna check out. I'm not going to install Tik Tok but I'm going to check it out. One thing that caught my eye was bash scheduler and you say it's an open source project that you wrote, and I'll let you go more into it. But you said it's a really bad idea. I'm biased because I really like bash. I love bash. I interviewed the creator of it not too long ago, Episode 22, by the way, and what is bash scheduler, what does it do what you make it for? [14:50] Justin Garrison: I made it to learn. I'm a project based learner. I wanted to learn the Kubernetes API. I didn't know about extending Kubernetes. I started it five years ago, four years ago, something like that when Kubernetes was still early, and someone told me you can add third party schedulers to Kubernetes. I said, really? How does that work? Well, where do you do that, and I just started diving into the API. One of the cool things about Kubernetes is the structure of the APIs. Once you understand the patterns, you can do a lot, you can figure out oh, this is repeated every time and even for native objects, or CRDS or whatever. It's just the same pattern over and over again. Sure, there's libraries and things to help you do more advanced, scalable things. [15:36] But I want to say, can I take a bash script and take a pod and put it on a node and make a pod, and that's what bass scheduler was for. It's just okay, well, let's just randomly take some pod and put it on a node. I was doing this in my free time years ago, just saying, okay, and I wanted to do it with native tools, the first iteration of it actually didn't use GQ at all. I was parsing the JSON output with a [crosstalk 15:58]. But that's already. I was like, I want to be like a bash purist here, and then not use those easy tools like JQ. When I rewrote it this past year, I was like, you know, like, I make this way shorter and more understandable for other people to also learn if I just use JQ and say, hey, this is where the object is. At that point, it turned into I wasn't learning from it anymore. I wasn't using it. *\ * [16:24] To me, it was three really cool things. One was just how easy and structure the API is to how easy it is to extend pieces in Kubernetes, by looking them up by saying like, hey, a scheduled pod is just a pod that has some metadata on it that says where it's supposed to go. Then the cubelet, everything else happens after the fact. The third thing actually was something I didn't think I would learn, but how easy it is to get a feedback loop of Kubernetes, and how much that enables learning. that was really exciting for me, because I could do a cube control proxy. I had access to the API server. [16:56] I had used tools in the past, I have to used mezzos, I'd use these other things in the past, and I could never do it that easily. Being able to just all of a sudden, no matter where I was, or where the Kubernetes cluster was, create a cluster and then access API with my own authentication, all that stuff was just taken care of, and I just curling localhost, and then I was like, wait a minute, it's that easy. Okay, now, let me see what this does when I actually package this up in a container and put it in Kubernetes. It essentially has similar access to give it a service account, tell that you can call the API server and then I was like, okay, now that feedback loop was so powerful, because that's where people learn. It's not like reading a doc, it's not necessarily watching the video. [17:37] It's doing something and say, having that aha moment of, oh, this is different, I get this. Now, when I've created bash schedule, that's what I want to do. I didn't want to do it with another library. I wrote a lot of Python in the past. I was like, I don't really want to do it with Python, I want to try it with bash and just see like, if I can do it with bash, it's simple enough that I could do it with anything. Just being able to do it with curl and JQ was very powerful. [18:03] Justin: I said in the past, it always starts with a bash script. It just stays with the bash script. I mean, bash and YAML is like an every single cloud native related repository. I thought it was really interesting. [18:17] Justin Garrison: Of every programming language, I've learned, I've gotten the most mileage out of bash. Yeah, that's the thing that I can start to. [18:22] Justin: It's everywhere. It's on Mars, for God's sakes. It's just really great tool, I think for anyone who uses, even Unix like or Windows now it's just a great scripting language. [18:34] Justin Garrison: For me, years ago, I set limits for myself on bash, and knowing when to turn to the tool or when to turn away from it was really important. I learned through doing my own development and working in various environments that basically I have a couple rules for when I stopped using bash. That's typically when the script itself becomes more than 100 lines. If it's 100 lines, more than 100 lines, at some point, I probably should change to something else. I can condense things that I can make it easier for humans to read after the 100 lines. The second one is if I have to accept more than two arguments on the script. Once I have a switch statement that has three plus arguments, maybe five, it's just gonna keep growing at that point. If I reach two, I know I'm gonna have 10. At that point, I'm like, I want a better arc [inaudible 19:18] library, I want something that's going to do some better options just on that Unix style strings and set ops just isn't great. Then the third thing was really around if I needed to use an array, if I have to turn to any sort of bash array like in bash scheduler, I use one but once I start hitting an array, I know I'm doing something a little more complex for a scripting language and I try to look for something else. I can do this a lot easier in Ruby or Python or go, maybe I should just switch it up now. Because once you invest all the time, it's gonna be harder to switch. Those are kind of my three rules of like, I will use bash all day until I hit those three, one of those three things and then I will look for the next best tool. [19:58] Justin: I like that. It's like the three rules to succeed in scripting. [20:03] Justin Garrison: It's helped me a lot, because by starting with bash, I could build things so much faster, I can do and see things. I can't tell you how many internal tools and some open source projects, I've started that were a bash script. I use those same rules. I said, hey, this is the POC, the next version should be written in some other language for these reasons. Let's rewrite from there. We didn't invest too much time in that POC, but we got the user experience. We got some of the functionality, and then we can see how it would grow from there. [20:33] Justin: Yeah, I mean, your disciplined. [20:36] Justin Garrison: I failed so many times. By failing and saying once I was managing the 500, line bash script then I was like, I've made some mistakes. That's when I started learning the hard way these rules. [20:48] Justin: That's cool. One other thing I saw, which was pretty cool is you have a linux.com email address, like, how do you get one of those? [20:55] Justin Garrison: It was something that I didn't know about. But the Linux Foundation has individual memberships that you can pay for. For $150, you get a lifetime email address from the Linux Foundation, which is linux.com email. I paid for one along time ago, and I don't know how many like I got Justin, just because it was there. It's Justin at Linux. I was like, that's cool. It's just an email redirector it's not like, you know, anything special, but I have, you know, a lifetime membership with the Linux Foundation and it forwards that email. I thought it was cool. I saw someone else have one and they had a really short one. I was like, I think I want that. When it was a first name only short domain. That's really cool. I think I want one of those. Yeah, so it's a redirector for me. [21:38] Justin: Still made me go wow, that's pretty impressive. Speaking of impressive, I looked at your resume, and it's a man page. Did you think of that? Or do you like getting an idea for someone else? I want to just kind of do mine like that now, but I feel like I'd be ripping you off. How did you get the idea? You just like, you know what man page, I think would be a great resume for that. [22:00] Justin Garrison: Someone else. I got it from someone else. His name is Major. [22:03] Justin: Oh Major Hayden? [22:02] Justin Garrison: Yep. [22:04] Justin: Oh, from Rackspace. [22:06] Justin Garrison: Yep, he had at first, he's the first place I saw. He has it as an HTML, it was on his website. I condensed it, I've been using that same style for a long time, because I was so familiar with the format. I think I've had more comments in my career about the format of the man page, then the content on the resume. [22:25] Justin: It looks legit. I was like, whoa, wait a minute, it's his resume. [22:30] Justin Garrison: It did teach me a lot about being unique to like, get some attention. It doesn't do great with like, automated parsers for man pages, and that sort of stuff. Because I would have to export a PDF, make the HTML, I make it a single page, and then I export a PDF. That's what I would send, but also top tip for anyone doing a resume. Store it in a Git repo. Put your resume in a Git repo. I've been doing that for, I don't know, a decade now almost. I have forks of like, hey, here's the job path that I was looking at this other company. I have, you know, tags that would show hey, here's the one I submitted to this company on this date, and being able to review that history was super valuable for me as I was, you know, growing in my career and looking at other opportunities, and even just learning from my own mistakes in the past or just like, hey, this wasn't resume. dot v2 docs, like this was actually like, hey, git log. [23:24] Oh, there's that fork that I had, that had some content that I remembered. Or here's the tag of if I applied at the same company multiple times, I could see the versions that I applied on a fork. Having that history is invaluable, because you forget so much, or at least I do. [23:40] Justin: Especially like the older you get. I would always like be amazed that my mom wouldn't remember certain things, because I remembered everything when I was younger and then once you like, get in your 30s you just like start to go, I never said that. Did I say that? Then it's like they play it for you. I'm like, Oh my God, I totally forgot. I'm gonna start doing that. I think that's actually a really great idea because you can just see the evolution of your resume and your life, your professional career. [24:07] Justin Garrison: Even if you're doing a Word file, right? Like you do Microsoft Word or something like that's still text. It's harder to diff HTML, the man page resume was really easy because it's just HTML, CSS, but there's definitely value. I would still commit the PDF, which was just like a blob, eventually, hey, here I can see the snapshot of the thing. I really think that I had a lot of value out of doing get from a resume, but also I would keep what's called a brag document. Julie Evans I think is her name. She does the wizard zine, the comics, online of like different Linux tools and networking tools, stuff. She's amazing at pushing all this stuff. [24:42] I first read about using a brag doc from a blog post she wrote, I have a quarterly reminder, fill out your brag doc. I just put a couple summaries of the things I'm proud of that I work pn. Not necessarily all the projects. Not necessarily like everything I did. Just like hey, if I look back in three months, like what would be the two things that I'd say like I was proud I did that, or I was happy I worked on that, or something big I learned. I put those in the brag doc. I've been keeping those for years now. Four times a year, I fill out a section of that, and it helps with yearly reviews, all that stuff, but also it helps with the resume. Because then I look back hey, my six years at Disney, what did I work on? What was I proud of? I can review that. Hey, you know what I was really proud of like, I was proud that I worked on Moana and Wreck It Ralph. All these things are like, awesome, but like, hey, what did I actually contribute? [25:28] What was the thing I was doing for those projects? What was it that I would put on my resume, and that's where the brag doc came in. Then I can see that tracking in the Git repo of resume. But also in those brag docs, great career advice that I learned from two different places. [25:44] Justin: I do on my website, I post things that are public, but I don't keep track of like internal things that it can't link to. I think that's actually a really good idea. The way I version it, I do my age. I do like 3.6 and then the amount of times I've edited the site in my 3.6, 36 years, mark. That's been working good. But I think that there's a lot of things that I forget, because they're not like something you can just link to. I like that. The brag doc, I got to look that up. Is there going to be like a cloud native infrastructure, follow up book, or you kind of done with being author? [26:32] Justin Garrison: I enjoyed being an author, I enjoyed writing the book. Chris and I debated writing version two, because there was a lot missing. When we wrote the book, it came out [inaudible 26:40] We were writing it early 17. Taking everything we learned from the 2016 through mid 2017, and put it in the book. Even things like lambda and serverless were like just coming around and security in the cloud. There was a lot of things in here that weren't necessarily available. People hadn't really failed at it enough to have a way to learn and teach people. There's things we would love to put in there as like a v2 to extend a lot of it. The pattern, I think is still the same where even with security, even with some other thing you're trying to do, if you want to run it at scale, having software manage that thing through some sort of declarative state is probably the way to go. [27:26] But we haven't really scheduled or started working on any sort of v2 for it. We both done a lot since. She has gone on to, you know, take the idea of infrastructure software, and really implement it in different tools in different areas, especially in the Kubernetes community. Things like cluster API, are kind of direct descendants from the idea of how do you manage infrastructure, grazers great at managing pieces of infrastructure for the cluster, but how do you manage Kubernetes itself? How do you manage more things inside of the greater scheme of what your infrastructure means. Including things like pager duty schedules, and on call. [28:02] Those sorts of things are also part of your infrastructure. People don't think of it that way. But it's really critical to have those aspects be rolled into these changes. Because once you have a disconnect between the documentation, and the runbook, and the code that's running and who gets paged, you're going to have a hard time. There's going to be some disconnect there where nothing lined up properly. Treating all of those things as critical infrastructure are really important. We still talk about it a lot, we still try to keep people educated and know what the next thing is. Like how they can be successful with the patterns. But we haven't really done anything with writing another version for the book. [28:42] Justin: We've had a couple authors on the podcast and one fellow, he did a self publish. Then he just did like a publisher. He said, just the back and forth with the person who checks. The technical editor. He just said it was just like, I'm never gonna do that. I'm just gonna do Self Publish. It's just so difficult. But I think O'Reilly is such a great brand. It's like, it's got to be such an accomplishment. Was that your first book? [29:12] Justin Garrison: Yeah. I wrote a blog post a while ago about the economics of writing technical book, because a lot of people ask about, like, well, did you make money, what was the process like I try to actually lay that out a lot in this blog post. It's a couple years old now from that, but actually 2018, so three years old, but I really tried to put any information that I had at the time in there because it was great writing my first book with someone, a group that was professional, and they knew the process and they knew the timing for things. I didn't have to get an editor and do all the reviews myself because as much as I like writing, I'm not great at that grammar itself, and knowing consistency for capitalization, or a comma versus an M dash or something like that. [29:52] There's a lot of benefits in just having professionals review that work and help make it better in so many ways. The CNCF was great with the book as well because it came out right around that time of a cube con. I was working with Chris and Dan from CNCF at the time. They love that we were writing a book around this ecosystem, Dan has a quote on the back of the book, which I was forever thankful for him being able to, like be part of that and promote it as well. It was a great experience. Going forward for me, I might do like self publish in the future, for something that is more low key or something that is not necessarily like a professional endeavor, I get really excited about remote work. I get excited about culture. I get excited about process for things. I love addressing some of those topics too, which isn't necessarily like the O'Reilly technical sort of background for like the audience. [30:44] Justin: So, you were at Disney? Did you get an Oscar? [30:47] Justin Garrison: My first movie credit is Zootopia, which did win an Oscar. I was able to hold the Oscar. The Oscar goes to the the directors of the film. [30:56] Justin: Are they heavy? [30:57] Justin Garrison: Yeah. [30:57] Justin: That's got to be really cool seeing your name in the credits. [31:03] Justin Garrison: The recognition, I think is something that we could really bring into the software industry. It's a artifact of the history of Hollywood where credits used to be at the beginning of films. They didn't used to be this like mile long scroll at the end. They had basically the top billing, you know, people would be at the beginning of the film, and there'd be nothing at the end. We don't really have that same organization or recognition inside of software, or inside of open source. But I guess the like, closest thing you can see is like a maintainers file or owners file or something at a Git repo. But it's not really celebrated the same way. I do think there could be a notion of having more celebration and having more like formal recognition for those things. Even if it's like, at this point in time, because movies are box software, modern movies. It's a snapshot in time of the development. It goes in a box, literally on a disk on a box. It sits in like in the bits get translated if you bring home a DVD or something like that. But even in streaming services, right, like it's a artifact in time. Software is not really like that software constantly changes. It's something that is always like, you can't take a snapshot of Disney plus. When I was there at Disney plus, like it's completely different, a lot of aspects from a year and a half, two years later. [32:16] Justin: Yeah, I think it GitHub is doing a pretty good job with their stars program. Daniel Stenberg, who founded Carolan, he just got like a real gold star sent to him with a bunch of swag and everything. Then they highlighted them on the blog and being a maintainer for 30 plus years. GitHub is definitely do that. But it shouldn't stop there. [32:39] Justin Garrison: Recognizing the I don't wanna say top billing, but like the real stars, the people that are very public and very much in front, they've done a lot of work for sure. They've put in the time. The first company that started listing everyone, like the first company that said, hey, we want to list our legal department, we want to list marketing, we want to list movie production, babies like that didn't come around until Pixar in Toy Story. They're like, no, everyone helped make this film. Even though there is voice actors, even though there was artists. They wanted to say, everyone should get their name in the credits. At the time, that was controversial. [33:12] That was something that the movie industry said no, like only top billing gets that, you can't make them too long. They actually have a break. If you go back and watch original Toy Story, there's a line break that says Pixar would also like to thank it's not part of the credits. That's how they kind of were like, hey, we want to get around this rule and give everyone credit for their input on this. Everyone from onboard training to HR to finance, like they helped make this film and they put all those names in the credits, which I thought was really fascinating just to see like how that's grown to now it's just everyone. That's what you do. Like everyone gets that here at the studio for whatever the rules are working on a film like you get your name and a credit. [33:51] Justin: It's got to be just a real big confidence booster. I don't know just something where it feels good to get recognized. I think that's definitely something that the open source community and cloud native community can take after. [34:09] Justin Garrison: Even if your name is eight minutes down the 10 minutes scroll, it's still that recognition. It shows that you had some ownership and you can actually point back to that, and that is important. [34:18] Justin: The thing is, especially with open source, there's so many people, non code contributors and non technical contributors that do amazing things behind the scenes, but how do we get them in? [34:33] Justin Garrison: I do think that CNCF has done a really good job about having the chop wood carry water awards at cube con where they say, hey, this isn't someone that necessarily was in front of the camera. They were doing the work and they were really pushing for this stuff behind the scenes and recognizing those people on stage with an award I think was a great first thing for them to just step up and say like these people are really important. We need to make sure that they have that recognition, even if no one else is seeing it, they were. [35:04] Justin: Yeah. Before I joined CNCF world, I didn't realize how many people were kinda in the background, Amy and Bill and all these folks that don't really touch GitHub or anything, but have such a huge impact in the foundation and just the ecosystem as a whole. There's other names I'm blanking on, but yeah, I mean, it's just the Christie, like all these people that are doing communications and getting everything organized for different events, and meetups and this and that. It's just really cool. It's just a cool community. I'll be honest, it's very organized. I mean, it's just really awesome to be a part of. [35:45] Justin Garrison: The contributor experience holds the CIG, dedicated to that to grow the community and make sure that people coming on board to the community are having a good experience. Like they actually gather data to say, hey, you onboarded recently, what was good and bad, like, what can we do better and constantly trying to improve and be inclusive, and sometimes, it might even become like at the discomfort of someone else. I remember when the Kubernetes community meetings were Friday at like, 9 am pacific time, and it's great, because I'm pacific time. But that's not welcoming for everyone. Not everyone is pacific time, some people are asleep. Being able to spread that out and say like, hey, we have a lot of people in different parts of the world. [36:26] Maybe 9am Pacific isn't the best time and actually taking that feedback and saying, well, we need to record these, they can be up on YouTube. But we also want other voices to be heard, not just people to listen to it. Being able to contribute there was important and contributes and all these other things that are focused on the community culture, I guess, is the word but the values that they have. I have all things open talk coming up next month that I'm actually recording right now about the communities specifically about how Kubernetes has scales and as a project and as code contributions and releases is absolutely important. [37:00] Growing the community constantly, because if you stop growing the community, you're going to stagnate and you're not going to get new contributors. You're not going to get new points of view. Constantly getting that feedback loop is important. [37:12] Justin: When she said all things open, I went there and like 2016 is a really great conference. Todd, if you're listening, you're awesome. What's your talk on? [37:22] Justin Garrison: It's called interest scale open source of Kubernetes. [37:24] Justin: Nice. Okay, cool. Awesome. Well, everyone, check that out when you hear this. [37:30] Justin Garrison: I mean, this has been a lot of fun. It's been great just seeing these podcasts and other channels that have this sort of content that people that maybe are getting started or trying to understand how things work inside the broader CNCF ecosystem. It's not just about like being part of that org or being directly tied to that Kubernetes or Like you could be a part of the community and not contribute code or you can just have some other open source project that uses their projects. [37:56] Justin: You're very right. Especially in CNCF, there are so many different tasks that need to be done that don't require a text editor. If you are on the sidelines, want to get in the game, then there's plenty of opportunities. Anyway, that's all we got today, folks. I gotts go get a haircut. [38:20] Announcer: Listeners. I hope you enjoyed this one. Do Tune in next time. We're really excited about our lineup of guests. We have a super exciting guest next week, as well. Check out the show notes for this podcast at podcast.curiefense.io for the community to cloud native podcast. Thanks again for listening. Tune in next week. Catch you later. Special Guest: Justin Garrison.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer Guest Andrew Martin Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast sponsored by Reblaze where we talk about the confluence of Cloud Native technology and Open Source. We have a great guest today, Andrew Martin, joining us from London. He is the CEO of Control Plane, a Cloud Native security consultancy training and pen test firm. We learn more about Andrew’s background, how he got involved in Kubernetes and Cloud security, and more about Cloud Plane. In 2019, Andrew made some Kubernetes predictions, and we find out today if any of them came true. We also find out how he keeps updated on what’s going on with open source in Cloud Native and other things. Since he has such a wealth of knowledge, Andrew fills us in on his book coming out soon called Hacking Kubernetes: Threat-Driven Analysis and Defense, and what chapter he’s most looking forward to people reading and why. We couldn’t let Andrew go without asking him for his “Predictions for 2023!” Go ahead and download this episode now to learn so much more from Andrew! [00:01:34 (https://podcast.curiefense.io/23?t=94)] Andrew tells us what Control Plane is, what does it does, and how many people they have working there. [00:02:13 (https://podcast.curiefense.io/23?t=133)] What is the average size of company in this space and why would someone need extra security on top of Cloud Native? [00:06:58 (https://podcast.curiefense.io/23?t=418)] Andrew tells us how he got involved with Kubernetes, Cloud security, and more about his background. [00:10:22 (https://podcast.curiefense.io/23?t=622)] We find out why Andrew thinks Kubernetes succeeded and Docker Swarm didn’t. [00:11:57 (https://podcast.curiefense.io/23?t=717)] In 2019, Andrew made some predictions and Justin wants to see if any of them came true. First prediction, did hosted services catch up with GKE? [00:12:59 (https://podcast.curiefense.io/23?t=779)] Second prediction, did non-container VM-based isolation improvement happen? [00:16:39 (https://podcast.curiefense.io/23?t=999)] With Andrew’s vast knowledge Richard wonders what he uses to keep updated on how open source works in Cloud Native and if there’s a Medium Blog that he’s subscribes to. Also, he shares which conference he will be attending this year and others he recommends. Justin gives a shout-out to TAG Security and their meetups. [00:20:05 (https://podcast.curiefense.io/23?t=1205)] Andrew’s book he co-wrote with Michael Hausenblas, Hacking Kubernetes, is discussed and he tells us the chapter he’s most looking forward to having people read. [00:23:49 (https://podcast.curiefense.io/23?t=1429)] Justin wonders if any of Andrew’s colleagues reviewed the book or if it’s all done with O’Reilly. [00:25:26 (https://podcast.curiefense.io/23?t=1526)] Andrew explains what he does to make sure that people at Control Plane are actually getting the best of the open source world without which it wouldn’t exist. [00:29:03 (https://podcast.curiefense.io/23?t=1743)] Richard is curious to know what method Andrew uses to find an interesting problem and how does he do security research in a way that makes him feel really excited about doing that sort of work. [00:32:22 (https://podcast.curiefense.io/23?t=1942)] We hear one last 2019 Kubernetes prediction and that is, if the tangle of YAML was going to unravel by 2019? He also talks about image and build metadata security matures which was another prediction. [00:35:53 (https://podcast.curiefense.io/23?t=2153)] Richard asks Andrew if he’s worked with Dan Lorenc in the Sigstore Project and Justin gives a shout-out to Dan and Episode 20 on this podcast to check out. [00:36:14 (https://podcast.curiefense.io/23?t=2174)] Andrew shares his predictions for 2023. [00:39:27 (https://podcast.curiefense.io/23?t=2367)] Find out where you can follow Andrew and the work he does. Quotes [00:03:21 (https://podcast.curiefense.io/23?t=201)] “The shared responsibility model gives us a different level of interaction with our cloud provider based upon what is ultimately platform as a service or infrastructure as a service or software as a service as well.” [00:04:03 (https://podcast.curiefense.io/23?t=243)] “But when it comes to how we behave operationally the cloud provider can make no guarantees that we’re not shipping bad code to production.” [00:10:51 (https://podcast.curiefense.io/23?t=651)] “And service meshes were being shipped by Docker Swarm before they were cool.” [00:11:29 (https://podcast.curiefense.io/23?t=689)] “So, from a networking perspective, Docker Swarm was much better out of the box because it was batteries included, but changeable, and came with its own networking paradigm.” [00:11:40 (https://podcast.curiefense.io/23?t=700)] “However, the inability to run multiple containers in a pod meant that there was no flexibility of application to Pology, and that’s really where Kubernetes stole the show.” [00:13:14 (https://podcast.curiefense.io/23?t=794)] “Google had this huge infrastructure, all these Borg cells, Flexible Compute to host Gmail and Google Search and calendar, and maps.” [00:14:10 (https://podcast.curiefense.io/23?t=850)] “And so still definitely harboring loads of zero days that are probably being exploited somewhere by somebody.” [00:25:32 (https://podcast.curiefense.io/23?t=1532)] “Democratization and open sourcing of security, tooling, and information has been a constant source of utter amazement to me.” [00:26:13 (https://podcast.curiefense.io/23?t=1573)] “At some point, I have wondered, and I noticed the case for other security researchers in the Cloud Native space as well, if actually we’re a bit further ahead with the art of the possible than the current state of the art.” [00:26:36 (https://podcast.curiefense.io/23?t=1596)] “So we try very hard to open source everything.” [00:29:23 (https://podcast.curiefense.io/23?t=1763)] “I think the most important think speaking personally, but also infusing teams and displaying leadership, is to infuse or inculcate a shared sense of passion for thing.” [00:37:11 (https://podcast.curiefense.io/23?t=2231)] “So actually, one of the things that I really love is the function as a service approach, on top of STO, on top of Knative, on top of Kubernetes, because you then get the full observability of the whole platform.” [00:37:23 (https://podcast.curiefense.io/23?t=2243)] “You can apply this intrusion detection, you can use your namespace, aware tools in order to introspect more deeply, and you can also satisfy what ultimately the kingmakers, the developers require because there’s no point building a secure system if the developers can’t ship business functionality through it.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Curiefense Blog (https://www.curiefense.io/blog) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) jdorfman@curiefense.io (mailto:jdorfman@curiefense.io) podcast@curiefense.io (mailto:podcast@curiefense.io) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?lang=en) Andrew Martin Twitter (https://twitter.com/sublimino) Andrew Martin Linkedin (https://www.linkedin.com/in/andr3wmartin/) Control Plane (https://control-plane.io/) Hacking Kubernetes: Threat-Driven Analysis and Defense by Andrew Martin and Michael Hausenblas (https://www.amazon.com/Hacking-Kubernetes-Threat-Driven-Analysis-Defense/dp/1492081736/ref=sr_1_3?dchild=1&keywords=Hacking+kubernetes&qid=1633728820&sr=8-3)_ Hacking Kubernetes: Threat-Driven Analysis and Defense by Andrew Martin and Michael Hausenblas (Amazon UK) (https://www.amazon.co.uk/Hacking-Kubernetes-Threat-Driven-Analysis-Defense/dp/1492081736/ref=sr_1_3?dchild=1&keywords=hacking+kubernetes&qid=1630843908&sr=8-3&pldnSite=1) Committing to Cloud Native Podcast-Episode 20: Taking Open Source Supply Chain Security Seriously with Dan Lorenc (https://podcast.curiefense.io/20) SANS SEC584: Cloud Native Security: Defending Containers and Kubernetes-Course with Andrew Martin (https://www.sans.org/cyber-security-courses/cloud-native-security-defending-containers-kubernetes/) O’Reilly Kubernetes Threat Modeling Course with Andrew Martin (https://www.oreilly.com/live-events/kubernetes-threat-modeling/0636920055610/0636920055609/) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript [00:02] Andrew Martin: One of the things that I really love is the function as a service approach on top of STO, on top of K native on top of Kubernetes. Because you then get full observability of the whole platform, you can apply this intrusion detection, you can use your namespace, aware tools in order to introspect more deeply, and should also satisfy what ultimately the kingmakers, the developers require. Because there's no point building a secure system if the developers can't ship business functionality through it. Platform for functions with Kubernetes is underneath. [00:33] Richard: Hello, and welcome to committing to cloud native, the podcast where we talk about the confluence of cloud native technology and open source. Super excited to talk to you today, been a while since we've had a podcast, very happy to be back at the track or whatever. I don't know something. Anyway, our other panelists we had today besides the illustrious Richard [inaudible 00:55]. That's this guy, is Justin Dorfman. Justin, how you doing? [01:00] Justin: Doing good. How are you Richard? It's good to see you again. It's been a while. [01:04] Richard: Always good to see you, too. Oh, the hat still has something on it. It's a smudge, kind of looks like LA, should say New York City. I don't know what's going on there. Anyway, we have a guest on this podcast, which is good for all of us listeners. Because it'd be the worst if Justin and I just talked. I'm talking about Andrew Martin. Andrewio is, Andrewio? [01:23] Andrew Martin: I'll be whoever you need me to be. [01:27] Richard: We have Andrew Martin on today who has just said he'll be whatever he needs me to be. Andrew, I need you to be the CEO of control plane, can you tell me what control plane is and does. [01:38] Andrew Martin: Control Plane is a cloud native security consultancy, training and pentest firm. If it looks like cloud native, and it smells like security, we are interested. [01:49] Richard: Awesome. I understand what that is pretty intuitively. How many people do you have there? [01:53] Andrew Martin: We have about 30 people and still expanding, taking on security architects and threat modelers and all that good stuff. [01:59] Richard: I'm familiar with the security field. I kind of know what pentesting means. I know that there are people out there who are literally hired to like, take axes and wire clippers and go through fences and figure out how to get to servers, which is the coolest job ever. For my benefit, because I don't know, what's the average size of company in this space? Is it like five people? Is it like a solo Polish dude and working out of his apartment? Does that have like 1000 people? What's going on? [02:26] Andrew Martin: There's a huge variance. Actually, we should be just as concerned about the solo dude in his basement, who may have an incisive way into our systems as some of the bigger organizations, of course, in the UK for penetration testing, you have organizations like NCC who do it, a huge proportion of the market. But then you also have the smaller boutique firms that focus very specifically on the cloud native aspects and the interactions between the cluster and the clouds. The IM integrations, how things configured in CICD, getting into the supply chain and all that good stuff. Yeah, we we like to think we operate on more of a intuitive than check mark based basis. [03:06] Richard: Cool. that makes a bit more sense. Now, dumb question. Security for cloud native, I thought cloud native depends upon actually using other people's code. When I'm using AWS, I'm dependent on AWS security, why do I need extra security on top of that? [03:22] Andrew Martin: The shared responsibility model gives us a different level of interaction with a cloud provider, based upon what is ultimately platform as a service, or infrastructure as a service, or software as a service as well, where Gmail, you don't see the servers, but for the infrastructure from for example, GCP while you're actually interacting with the servers, and you have a responsibility to secure part of that stack. We trust the cloud provider to encrypt our data in transit, to make sure that there's suitably advanced encryption and folds envelope encryption, so the keys can't be retrieved and so that our data at rest is safe. But when it comes to how we behave operationally, the cloud provider can make no guarantees that we're not shipping bad code to production. [04:09] First line of defense for any cloud-based application is the socket that faces the internet that's behind the load balancer, or whatever that would be. As we go to just serve our application code, well, that's developed by us, not developers. If we have a struts like remote code execution, in that web facing tier, well, then all of a sudden, our adversary can get in, to our infrastructure. As I say, that's not something that the cloud provider is able to give us any guarantees over because they have no insight into what we're deploying. That's very much the delineation of the model, how it's supposed to work. [04:45] What do we do? Well, we can use the raft of modern cloud native, specifically kind of containers, namespace aware tooling, to run lots of processes on one host, let's say like a Kubernetes style. Lots of containers. For each of those containers, we can attach a granular security profile. If we think back to the LAMP stack, we would have a VM with, obviously, it's running Linux, we have an Apache web server, which needs capabilities and permissions to bind to a port below 1024. That's a privileged operation, then we've also got my sequel, which wants to bind to a port above 1024. It's a 3306 if memory serves. [05:25] Applying same policy to the whole VM, isn't practical, you have to do that on a per process basis. That just gets laborious. Traditionally, we've seen, we've got stuff like Dan Walsh, turn off SC Linux, it's a kind of lifetime crusade, SE Linux has this level of granularity, but it's difficult to figure. Cloud Native gives us containers, containers gives us a single process per container model. That container is an easy, and relatively well hardened security boundary. We can configure that per process. We have very high fidelity of policy, very granular application or per process level. When we get that remote code execution into our container, because we've got that vulnerable application library, we can go in and save the baseline behavior for this process has deviated, we've spawned a shell, we're making other strange network requests we wouldn't do normally. That's the delineation between the cloud security that we get from the vendor, which those underlay encryptions, and data at rest. The lifeblood, the data when it's actually passing through the system, and narrowing down the attack surface, and then running cover with tight policy for each process that run. [06:43] Richard: Oh, sure enough CEO slash Chief Marketing Officer, cause it's just very clear. Thank you so much. You mentioned Kubernetes at one point, and I do want to get to your book that you wrote Hacking Kubernetes, which is going to be very exciting. Before I get there now that we have enough background to know that what you're talking about. How did you get involved with Kubernetes? How did you get involved with cloud security? What's your background? [07:05] Andrew Martin: I was very privileged to come straight out of university and into a startup where I did the traditional hat juggling that start-ups are well known for. That gave me the opportunity to migrate bare metal servers on to Amazon, AWS at the end of 2008 to deal with PCI audits to do database query tuning, to expand into multiple different types of programming language. That gave me the kind of generalizing specialist angle on things. That really placed my inquisitiveness, which is probably a superpower and a huge distraction, and meant that I had the ability to kind of learn about various different domains. I moved to London about 10 years ago. [07:42] We're just kind of looking for difficult problems, you can build high traffic websites, and fine tune them and deal with your requests per second that all the different layers of the stack. But actually, when things get really difficult, that's when regulation turns up that specifically for example, it puts an extra hop in between things, and you're looking at trying to do a low latency service. But actually, you have to deal with some sort of reverse proxy, some sort of step up gateway kind of thing. The nature of those problems really appeal to me as an engineer, because of course, all engineering is a compromise. It's about finding the right thing to fit into the hole. When that set of compromises is regulators, then it becomes a slightly more difficult and interesting set of problems. [07:38] Subsequently ended up moving through various financial institutions ended up at the UK home office, and they were ahead of their time. Thanks to what was called the GDS, government digital services, oh and incidentally, the home office in the UK is a governmental department. It's a ministry of the interior, rather than sort of the ubiquitous back room, which we all share these days. The home office were advanced by virtue of these previous projects, and they were running Kubernetes 1.2. I turned up, dealing with critical national infrastructure deployments onto Kubernetes. At that point, it was clear there was obviously 1.2, kind of pre our back, pre network policy, pre dynamic admission control. There's a huge number of things that were widely known security foot guns in Kubernetes. [09:15] But because I've been deploying containerized systems for a couple of years preceding that, it was clear to me that this was the direction of travel. The pod concept enabled any application deployment topology. That really differentiated from Docker Swarm, which looked like it had a horse in the race for a while, and so I double down individually and sort of commercially. Seeing that we had this fully declarative interface of Kubernetes where YAML as far as the eye can see, I built a static analysis tool called cube sec, at cubesec.io, which uses a beyond call risk models style for Kubernetes resources. This pattern of static analysis on resources on plans kind of terraform style infrastructures, code, as well as the deployment code, sort of brings us to a place where we have stuff like opa today, that generalized open policy agent. That sort of talk on that about the same time, it was very clear the direction of travel of cloud native security. It takes the journey to almost four years next week. For those of us listening in the past or the future, so well, yeah, perseverance is key. [10:23] Justin: You brought up Docker Swarm? Why do you think Kubernetes succeeded and Swarm didn't? [10:32] Andrew Martin: That's a great question. Potentially, the answer is to do with the lack of pod support. With a single process per container, Swarm was only designed to run single containers, which meant that we had to run more processes in one container, so we don't have to use full in it. Side cars, we don't have side cars for logging or security or service mesh envoy style, and service meshes were being shipped by Docker Swarm, before they were cool. They created a mutual TLS overlay network across the base network between the Docker Swarm nodes over which the micro segmented application networks were able to communicate. That's a pretty advanced networking model, compared to the more flexible, but less well defined CNI model in Kubernetes. Of course, we only got network policy in Kubernetes, because Calico shipped it and as with other things, the interface was extracted from the Calico implementation and baked into Kubernetes core itself. From a networking perspective, Docker Swarm was much better out of the box because it was batteries included, but changeable, and came with its own networking paradigm. However, the inability to run multiple containers in a pod meant that there was no flexibility of application topology. That's really where Kubernetes stole the show. [11:54] Justin: Thank you. That's a really great answer. You're an expert with Kubernetes. In 2019, you made some predictions, and I want to see what came true and what didn't. Hosted services, catch up with GKE. Did that happen or not? [12:12] Andrew Martin: To some extent, I think GKE, and full disclosure, we wrote the CIS benchmarks for GKE, in this year, and contributed back some sort of thoughts and ideas around the supporting services. GKE still continues to push the envelope in terms of how Kubernetes is run, and how it integrates with the cloud. A good example would be, we have the kind of [inaudible 12:30] hybrids in Amazon AWS, Google's kind of GKE response to that is GKE autopilot, which removes the node abstraction. It's advanced thinking, and it's definitely pushing the paradigm. I think GKE continues to sort of lead slightly at the things like binary authorization, supply chain security before it was cool again. I have a soft spot in my heart for GKE of course. I still think it edges ahead somewhat. [12:59] Justin: Shout out to Dan Lawrence there, the second one, non container VM base isolation improvement. Did it happen? [13:07] Andrew Martin: Oh, yes, this blew up. We've now got things like gVisor, which is ultimately for App Engine to run. Google had this huge infrastructure, all these Borg cells, flexible compute to host, Gmail and Google search and calendar and maps. That was how Google deployed itself globally, and the self-contained cells. Amazon turns up and starts renting compute in the form of AWS and Google, say, we can do this, why aren't we doing this, and the first thing that they build, the app engine is running untrusted third party code. That is my code and your code on Google's Borg infrastructure on the backplane. In order to isolate those workloads from the underlying Linux kernels, Google built first version of gVisor, and gVisor is a user space kernel reimplementation, which means it catches system calls before they hit the kernel, and handled as much as that is possible, and a golang based kernel emulation. [14:04] This moves the untrusted code further away from the Linux kernel base, which of course is written in C, and still definitely harboring loads of zero days that are probably being exploited somewhere by somebody. It gave the Google security team, that warm, fuzzy feeling of security. They do a couple of other things with gVisor including dedicated IO dedicated TCIP stack. You can run it in various configurations, but they realized they were onto something here and gVisor is now also runnable as a container runtime for Docker. It will run in GKE. If it's installed on a host node, you can start containers that are wrapped with gVisor. If an untrusted process is running, and by that I mean stuff perhaps or video transcoding because they're very complex is often somewhere that you can see bugs. [14:56] Image tragic was a good example of how difficult it is to do that safely all time. Maybe it's also particularly risky workload, as in, perhaps we allow untrusted code to be executed within. If somebody gets remote code execution, or we gift it to them, or whatever it is, they can't run exploits against the kernel, because they've got this Shim, this gVisor layer in between. It's a kind of VM based isolation. We've also got things like firecracker, which is the AWS runtime for lambda, which is the same kind of idea, it's a very stripped down Virtual Machine Manager inside the virtual machine. It also creates container namespaces. It hangs set comp off not only the VMM process, but also the process inside the VM. These are really crazy hybrid approaches that take the best of both, they stripped down the QEMU, and KVM, virtual machine drivers, and give us this kind of hybrid existence where we can still take advantage of some of the container guarantees of horizontal scalability and the resistance to churn and add these virtual machine guarantees, which include a more difficult runtime to escape. Yes, it developed and improves, I think, is suitably in specific to claim victory. [16:15] Richard: Andrew, you have a lot of knowledge, right? I feel like a lot of this podcast of ours listening to you, well, you're very articulate and may just be the English accent, I don't know. What's like interesting to me is that you're basically saying all this extra stuff. You're giving all this context for where this team moved here. This team moved there. I think about the average listener, and wondering what there may be wondering, and I think the main thing is, where do you find this stuff out? Like, what do you use to keep updated on how open source works in cloud native? How do that say, Google realized that we're on to a good thing with gVisor. How do you know? Is there like a medium blog I should subscribe to what's going on? [16:56] Andrew Martin: It is a great question. There are multiple layers. One is that I've been very fortunate to attend many conferences and meet some of my heroes. I mean, I met Eric Brewer at one of the cube cons, obviously, the guy invented CAP theorem. But even so many really remarkable minds have contributed into open source and code bases. Just going to conferences, and having enough of grounding or an understanding of people's work to ask them incisive questions, is a great way to just kind of dig in a bit deeper and get some context. We're very fortunate in that we've been able to provide services, some of the cloud providers and some of the big industry entities that also sort of provides them access to this, but really good and assiduous use of Twitter lists is really key. [17:43] Instead of just following my timeline, I have a tech list. I have a sec list. Between those, I also trigger some RSS notifications. Sifting through those things, top Hacker News comments, not all the Hacker News comments, just helps. There's definitely some diamonds in the rough. [18:02] Richard: So staying keyed into the ecosystem is a good way. Twitter really does continue to be in fantastic way if you can avoid timeline fatigue. Using list is a good idea using conferences is a good idea. Which conferences are you say the best ones to go to? Or do you have any that you're looking forward to this year? [18:19] Andrew Martin: Well, if all the stars align, I'll speak at cube con in person. Just prior to that is the cloud native security day, the final masterstroke, the community understanding is of course, getting involved and places like tag security as it is now and the CNCF six security, the open SSF. These are all open-source organizations that helped to push the envelope by virtue of people just contributing their time and kind of collaboration. Those are the most fascinating and interesting things. Even just combing through meeting notes to catch up on a month being away is a useful way to maybe there's a GitHub issue. Maybe you can chase a pull request down and kind of dig through some of those things. In terms of conferences though, yeah, absolutely keep on the tag security day. Prior to that, we'll run the CTF there as well. Sign up for tag security. You also have the opportunity to run some nefarious, I don't know how much I can reveal but red pill blue pill, style CTF. [19:21] Justin: Do you ever attend the tag security weekly meetups? [19:23] Andrew Martin: Yes, I do. [19:26] Justin: I learned a lot. My boss was like, check this out. I'm like rolling my eyes like oh God. But then I went to him. I'm like, wow, it's a lot of great information. I haven't run into you yet or maybe we did. I just didn't see you because there's so many people on it. Yeah, shout out to tag security for sure. [19:44] Andrew Martin: Yes, I managed to be a member from early doors. Then my attendance is not what it could be. [19:50] Justin: I wasn't trying to throw you under the bus or anything like that. I think Richard brings up a great point like you're expert in Kubernetes and I think that's why O'Reilly probably went to you and said, Hey, bro, we need to write a book. Let's talk about this Hacking Kubernetes book. It's not out yet. You wrote it with a fellow named Michael. When does it come out? Let's talk about this book. [20:15] Andrew Martin: I discovered by virtue of searching for it on Amazon the other day, that is due out on the 30th of October, November. Yes, November, I think those briefly concerning and yeah, I wrote it with my excellent and studious co author, Mr. Michael Hudson [inaudible 20:31], who is a prolific writer already, and took me under his wing to guide me through the process. It was tremendously enjoyable to write. The opportunity for consistent research and making sure that I understood a concept sufficiently to write a few sentences on it is really positive reinforcing learning cycle for me. Really as I said, this acquisition of knowledge and interesting things, very much powers my kind of intellectual engine. I really enjoyed that part of the process. [21:05] On the flip side, it required excessive dedication to the cause. Some sacrifices of various things, such as enjoying a drink from time to time, I was just able to hand in the final draft a few weeks ago. The cognitive unbundling that occurs when that happens, because everything that turns up is potentially something for the book and how do I contextualize this? Cross reference. Did I make that CV in there? It was a tremendously enjoyable exercise to undertake. What we've attempted to do is analyze the threat landscape against Kubernetes bathed in the light of historical vulnerabilities, and then try and paint some kind of future ideal hardened security state based on the impact of remediating risks, rather than kind of Iron Maiden lock everything down and cut move. Hopefully, people find that in some way useful. [21:59] Richard: What do you think the most accessible chapter is that you look forward to having people read? [22:03] Andrew Martin: I really enjoy the first chapter. Because it starts off with the pod and it decomposes the pod looks at a container. The great thing about Kubernetes container and Linux security is they are microcosms and microcosms of each other. The more that you zoom in on Kubernetes security, but it's actually container security for the process, of course, rather than the orchestrator. Then you drill down there and container controls are just Linux kernel controls, and then we go all the way down, and then we're back into some archaeological spelunking through why specific design decisions have persisted for however long. The book features my archetypal eight bit nemesis, the nefarious Captain hash jack. His prime directive is to plunder the data, that courses through the veins of the example organization of which the reader is a fictional seizer. [23:00] We contextualize a number of attacks with the intent of a threat actor of Captain hashtag description. He's state sponsored, he's organized crime level, he's where a lot of organizations today will find they potentially have problems. Hopefully, this despite being slightly fantastical, anchors it somewhere close to reality and makes it more of a useful set of recommendations, rather than something a bit more abstract. Starting with the pods, is my favorite piece. We actually had to move Captain hash jack attack on a container from the first chapter into an appendix, because it's a content of 12 pages to truly explore the periphery. Combination of chapter one, and the appendix helps to set the scene for the rest of the book and ecosystem. [23:48] Justin: Do you have any colleagues of yours, like review the book? Or is it all done with O'Reilly? [23:55] Andrew Martin: I was so grateful for some incredibly detailed reviews. Editor, O'Reilly, [inaudible 24:00] was awesome. She connected us with some really established and advanced authors who were very competent with their reviews, and obviously preeminent in the space. I then reached out personally to a couple people in the scene who really, again, that they will get called out beginning but provided some invaluable feedback and also some philosophical views on some points that were made with a kind of higher level generalist perspective, which was, it was challenging to read. It meant that I had to sort of exercise my own thought patterns on the thing. That was one of my favorite parts of the whole process. Then finally, I obviously disseminated the "work" to the control plane team internally, and a number of people came back, what I would describe as critical bugs in parts of the book. We ran an internal bug bounty, and numerous people scored. [24:57] Richard: This reminds me of one of the questions I was asking myself earlier while you were talking, which is control plane has 30 people in it. Security firm, obviously, you can't have all your tools be open source, because that doesn't work for security at all. One of the questions I have is how do you disseminate open source knowledge about cloud native infrastructure, cloud native programs through control plane, you just mentioned internal bounties, which is a really great idea for things like a book. But then a book is also a closed product that doesn't have to be re updated every time. What do you do to make sure that people that people at control plane are actually getting the best of the open source world without which it wouldn't exist? [25:33] Andrew Martin: Democratization and open sourcing of security, tooling and information has been a constant source of utter amazement to me, ever since sort of starry eyed, I found, OWASP, when I was about 20, or so. As I said, I had this very privileged position of being in this company and having responsibility. I would drive to London, Bristol, which is 100 and something miles to attend OWASP events. At one point, I received the big thick book, and the last sort of 100 pages for all XXS attacks. People printed out all the Unicode strings that would cause problems. It just blew my mind. At some point in control planes existence, I have wondered, and I noticed the case for other security researchers in the cloud native space as well. Actually, we're a bit further ahead with the art of the possible than the current state of the art. There is potentially potentially exposing things that are not yet target for attackers. [26:33] But clearly they will be in some period of time. We try very hard to open source, everything. We open sourced the threat models that we built for the financial services use group for Kubernetes. Of course, CIS benchmarks, open source, we've just built a new training course for O'Reilly called threat modeling Kubernetes, which has all of our ways and means to do threat modeling. Including, kind of default list of threats. Obviously, we have absorbed work that community's done, I folded that in with our own particular slant. I mean, we claim no credit for it, we're just happy to sort of push back. Where the difficulty comes, I think is the open core versus hosted security tool balance. [27:13] We see this with a number of tools, whereby the core enforcer or scanner, or security tooling is open source. You can take that and you can deploy it in your infrastructure, and you can have a good result. But if you want to do that at scale, and have an awareness of what these things are doing over multiple, perhaps clusters, or nodes, or availability zones, that's where the secret sauce kind of SAAS hosted piece comes in. The only thing we don't open source is the infrastructure behind the accumulated simulator project, which is what we use to run the Capture the Flag events at cube con. We've been running that with tag security for the last two or three events. We have a wonderful time that the community contributes and totally good fun. [28:03] Justin: Yeah, no, I hear that's like one of the biggest side events there. Now it's so cool talking to the creator. [28:12] Richard: You don't mind, I want to see if we can get behind the creators brain a bit more to help out future participants of capture the flag. Earlier on in this podcast, you were talking about how you enter the back control plane, and you said I was looking for a difficult problem. You've also mentioned several times that you're incredibly privileged, which this is to me, those go together, once you've solved the bread and butter, once you have rents covered. A lot of people, in our circumstances, I include myself in this, are looking for difficult problems. I think for you is a particularly interesting question because you're a security researcher, and security researchers, by default are looking to find ways around things. Are looking to find problems, questions, answers, which are not public, which are hidden, which are not going to be written somewhere in an easy one to three step guide. I'm curious, I'm not sure how to best phrase this question and thinking about it, whether to ask what's interesting to you? Or what method do you use to find an interesting problem? How do you do security research in a way that like, makes you feel really excited about doing that sort of work? What does the difficult problem mean to you? How do you approach that question? [29:20] Andrew Martin: That certainly triggers some self reflective thoughts. I think the most important thing, speaking personally, but also in fusing teams and displaying leadership, is to infuse or inculcate a shared sense of passion for a thing. I'm very fortunate again, that everything in cloud native is shiny, and so it's easy to find things that are, and we have this concept of flow, from a psychological perspective, something is a little bit more difficult, and we're a little bit less comfortable with it than the average. We can get into the state of psychological flow where we're kind of chasing the dragon of intellectual stimulation, and being in that quadrant of the graph is where I try and organize my time as an individual, where I try and infuse the organization that I'm very lucky to be a part of. [30:08] Finding those things that inspire passion in people, and then performing research on them, it's kind of a happily, conducive feedback loop into itself. Finding that level of challenge, rears its head up with the interest levels of technology. Right now, there's a lot of interest in next generation, process isolation, and those kind of gVisor and capture containers and all that firecracker, good stuff. There's also a lot of interest in EBPF base kernel instrumentation and process analysis. There's been so much momentum building up for this, and the tooling is all available. We've just been waiting for the kernel versions in our deployed servers to catch up with the kind of five mainline, which brings in all this new functionality. Additionally, this gives us a whole new way to rewrite a number of tools that have had durational problems. [31:05] I mean, of course, when Kubernetes started, IP tabled rule sets that were kind of in the hundreds or 1000s, when they reloaded, that was a two to three second downtime on all netfilter traffic, and you just have an outage. They had to rewrite IP tables with EBPF, which is the NF tables kind of shim. We can still interact by IP tables. But the netfilter code in the kernel that deals with that is now this hyperfast performance EBPF. These kind of things where we know the problem space, but there's something revolutionary, can really lead to lots of new code, we want new code because it brings us new features. But of course, new code is less tested or more potentially subject to bugs. Then we're into the constant software is never finished. It's part of an ecosystem that's always moving Moxie marlinspike can be credited greatly for that. These are the things that really piqued my interest and kind of keep me entertained from a professional perspective. [32:00] Richard: Love that answer also [inaudible 32:01] is the coolest just straight out saying that. Excellent. You've personally answered one of the questions I was going to ask next, which is what are you most excited about in the cloud native security space coming up. You've already covered a few things, this may be a good point to go back to your 2019 Kubernetes predictions. The last one of the ones that we're going to get to is that the tangle of yellow was going to unravel by 2019. Did that happen? [32:28] Andrew Martin: To some extent, there were, I don't know, leaps, bounds and steps that these taken in the right direction. We see things like customize, that make things at least easier to template across different environments. But I think the underlying issue of kind of meta templating or like higher order templating over YAML is still almost where we are. [32:52] Richard: Okay, so it has nothing yet but maybe in the future, we hope. What about image and build metadata security matures? Has it matured? [32:57] Andrew Martin: Yeah, that's a great one. Well, looking back on it, I appreciate the fact that I wrote it, let's say. What we seen is tools like cosine move into the space. Control plane have been hugely supportive of work out of the NYU cybersecurity labs, like tough and in toto over the past few years. But they were always lacking adoption. Which was able to bring content trust, as they called it into Docker, that required a secondary server to be deployed alongside an OCI registry. That overhead it was too much for most organizations. It's not a not a great deal of difficulty. But there was no stimulus. Suddenly, we see the solar winds loss of attacks. We see the kind of rebill pipeline attacks as well this year. Broadly, supply chain just enters the public consciousness at 1000 miles an hour, and starbursts when it's there. [33:54] Suddenly, everyone's talking about supply chain. What we've now seen is this use of image and build metadata in all sorts of different ways. There's this amazing project called recall, that will take the hashes of things that we've built, and put them in a append only transparency look. We can now do this for binary artifacts. Then you've got tools like cosine, which can integrate the update framework with container signing, and give us this brand new signing style that actually encompasses all of the existing state of the art with one slight innovation, which is to put the signed metadata and signature into an OCI image and push it to the same registry as the image that it's signing. [34:41] Because we're using that unique identifier for the image means that we have an immutable or a hash based link between the two. We can then deploy to production, validate that some evidence lake or recall or some external system has proof that thing was signed revalidate the signature to ensure that it's signed by somebody that we trust. Then finally, that's our admission control into Kubernetes. In production, the state of the art is now integrating tecton chains, whereby in toto is baked into every build step, you get the end of the build, build a container, you sign it with six store that implicitly trusts all of the cumulative in toto stages before that. We're into an amazing place where we can add software bills of materials into here and sign those two. While we may not have fixed the ultimate problem, which is third party code risk is very difficult to hedge against. [35:36] GitHub is full of potentially untrusted code. We're now in a position where we are much stronger against insider threats. in the event of a discovery of potentially malicious code, we know which containers or which systems have that code deployed, and we can operate and affect patching strategy. [35:52] Richard: Are you working with Dan Lorenzo on that? The six door? [35:55] Andrew Martin: Yes, we have worked with Dan and Luke Hinds in the six door project. [36:03] Justin: Shout out to Dan and Episode 20. You can hear more about that. [36:07] Richard: Cool. You had one more prediction in 2019. But I think in the interest of time, I want to ask a different question. Which is predictions for 2023? [36:18] Andrew Martin: Okay, that's a great question. Kelsey Hightower has been saying that Kubernetes is for building platforms. It's not the platform itself, building platforms, I think we're moving closer and closer towards that. What's happened over the past five years is that big companies who've built platforms on top of Kubernetes have been subsumed into larger organizations. We still haven't quite seen the thing. Rancher is doing an amazing job, but again, sort of [inaudible 36:40] last year. Still some sort of abstraction over Kubernetes, I think kind of makes that a little bit easier. I really love the GCP cloud run functionality, restates container and just runs it by itself. We're back into the same Swarm question there because of course, it's a single container, you don't have the lambda, secondary security container thing. But GCP cloud run runs on App Engine, so through gVisor onto App Engine, but it's k native compatible. [37:11] Actually, one of the things that I really love is the function as a service approach on top of SEO, on top of K native on top of Kubernetes, because you then get full observability of the whole platform, you can apply this intrusion detection, you can use your namespace, aware tools in order to introspect more deeply. You can also satisfy what ultimately the kingmakers the developers require, because there's no point building a secure system if the developers can't ship business functionality through it. Platform for functions with Kubernetes is underneath. [37:43] Justin: When we had Kelsey on a couple months ago, that's what he was saying. He says lambda and in open FAAS Functions as a Service, that's the future of like software development. I don't know if it'd be 2023. But who knows, I mean, maybe it will be. It could totally shift. [38:01] Andrew Martin: We certainly see that with some of our customers now, there is a hybrid deployment approach when they're trying to build global platforms in multinational organizations, where they're interested in targeting everything from a virtual machine, through a Kubernetes into a server less platform. Of course, developers will go for the least friction, operations teams probably want to have some balance of uptime and availability and observability for things, which traditionally have been slightly more difficult with Functions as a Service. We've had to do things like keep functions warm by pinging them every so often, just because the underlying runtime garbage collect them. There are still operational and developer frictions when it comes to running these things in production. Certainly, it's the direction of travel. [38:47] Richard: Awesome, great answer. Thank you so much for coming up with that on the fly. In fact, thank you for talking just in general during this entire podcast. It's been great to have you and to learn about your background, learn about control plane does learn about your new book coming out October, November 30 to you who are listening, this is going to be called Hacking Kubernetes. You have to say it with a British accent from now on. You can't say Kubernetes that's over. That'll be very fun. Please check it out. That's coming out of O'Reilly it will be available on Amazon and we hope that your predictions 2023 come true along with Kelsey Hightower. Now, Andrew Martin before you leave, we know that we can access you on Twitter and we can put the Twitter handle sublimino on our favorite Twitter list for tech security and probably other things if we wanted. We know we can see control plane @control-plane.io. Is there anywhere else on the internet where we can follow along with your thinking your words and the work that you do. [39:52] Andrew Martin: There are a couple of interesting escapades you may wish to follow along with. One is sans sec. 584. Which I was very lucky to have the opportunity to write with some excellent co authors and co trainers. That is attacking cloud native and Kubernetes. It's a three day deep dive into red teaming, and then blue team responds. It ends up being a purple mix of the two. There is also a course for a rally called threat modeling Kubernetes, which is the paper-based view of the hands on sans lab. It teaches you things like connectivity matrices, how to model systems, and that default set of attacks and threat objects that we look to remediate against. It should be a jumpstart into Cloud Native defense. [40:43] Justin: Is that based off their [inaudible 40:44] [40:47] Andrew Martin: It's not actually in tune with [inaudible 40:46] yet. We do have one exercise in there. It will all be embedded in the future, though. At the end, we set up a lab, shout out to Ben Hall, the [inaudible 41:00] [41:03] Richard: Well, follow along with those audience. Thank you so much for listening. Andrew, thank you so much for talking. It was great to have you on. [41:11] Justin: I've learned so much like I don't think I can hold it all in my brain. But that's the beauty of this podcast is I can listen over and over. T hanks so much for coming on. I really appreciate it. [41:23] Andrew Martin: Absolutely. It's been my pleasure. Thank you very much. Special Guest: Andrew Martin.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer Guest Brian J. Fox Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about the confluence of Cloud Native and Open Source. Today, we have an amazing guest with a long history of open source in the space and that is the legendary Brian J. Fox, who is the Co-Founder of Orchid, a blockchain company that started in 2017. Also, he created the Bash Shell and he was the first employee of the Free Software Foundation. Brian shares the story of how he ended up at FSF, his thoughts on the success of Bash after all these years, which includes running on Mars currently. We learn everything he did before he Co-Founded Orchid, he tells us all about Orchid and how it works, his thoughts on the open source movement and where he sees it going, and more about the value of cloud companies. We also find out Brian is a bassist in a band, so if you want to find out more go ahead and download this episode now! [00:01:51 (https://podcast.curiefense.io/22?t=111)] Brian tells us how he ended up at the FSF. [00:05:08 (https://podcast.curiefense.io/22?t=308)] Justin wonders if Brian thought Bash would still be around and he tells us it’s running on Mars in the helicopter. [00:07:08 (https://podcast.curiefense.io/22?t=428)] Richard brings up that Bash is on Windows and asks Brian to talk about how that happened and what his reaction was. [00:09:00 (https://podcast.curiefense.io/22?t=540)] At some point Brian left FSF and went to Orchid, and Richard wonders how that started. Brian fills us in on all the things he did in between FSF and Orchid. [00:14:01 (https://podcast.curiefense.io/22?t=841)] We learn how Orchid works and its physical infrastructure. [00:18:51 (https://podcast.curiefense.io/22?t=1131)] Brian tells us about the protocol being strictly peer to peer, and he explains more about the Orchid network and the bandwidth. [00:22:11 (https://podcast.curiefense.io/22?t=1331)] Justin asks if Brian still seeds or if he has enough users where it’s just kind of self-sustaining. Brian mentions OXT which is the name of the Orchid cryptocurrency. [00:23:36 (https://podcast.curiefense.io/22?t=1416)] Richard is curious and wants to know what Brian thinks about open source as a movement in the last two or three years, where does he think it’s going, and how does he think he’s leveraging that in Orchid in as best a way possible to make sure the success of the system that he’s building. [00:27:57 (https://podcast.curiefense.io/22?t=1677)] We find out from Brian that he’s all about problem solving and the architecture that goes into the problem solving and it’s about the expression. [00:29:40 (https://podcast.curiefense.io/22?t=1780)] How does Brian thread the line between being an open source diehard and I run a capitalist firm. [00:32:33 (https://podcast.curiefense.io/22?t=1953)] Justin does a U-turn to the conversation and goes back to the VPN industry and wants to know Brian’s thoughts on the current market of traditional VPN’s that are not crypto powered. [00:33:33 (https://podcast.curiefense.io/22?t=2013)] Brian tells us how he deals with requests from law enforcement agencies. [00:35:54 (https://podcast.curiefense.io/22?t=2154)] We end with Brian telling us where you can find him online, he tells us about his band Chillpoint that you should checkout, and he leaves us with thoughts on cloud companies not going away. Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Curiefense Blog (https://www.curiefense.io/blog) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) jdorfman@curiefense.io (mailto:jdorfman@curiefense.io) podcast@curiefense.io (mailto:podcast@curiefense.io) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?lang=en) Brian J. Fox Linkedin (https://www.linkedin.com/in/brianjhanfox/) Brian J. Fox Twitter (https://twitter.com/brianjfox) Chillpoint Band (https://chillpointband.com/) Bash (https://en.wikipedia.org/wiki/Bash_(Unix_shell)) GNU Bash (https://www.gnu.org/software/bash/) Free Software Foundation (https://www.fsf.org/) Orchid (https://www.orchid.com/) Orchid OXT (https://www.coindesk.com/price/orchid) nixCraft (https://bash.cyberciti.biz/) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript

[00:01] Richard: Bash is awesome, I use bash every single year as it is. [00:03] Justin: I do too, it's always open. [00:05] Richard: So I'm also just incredibly grateful for your work. [00:07] Brian J Fox: Thank you guys so much. I use Bash all the time and I just want to quickly say that the people who provide documentation and instructions on how to use these tools that have been around a long time, they are doing a fantastic service and NixCraft is one of those. You should check them out if you can online. [00:26] Richard: Hello and welcome to Committing to Cloud-Native, the podcast where we talk about the confluence of the cloud and open-source. Today, we have an amazing guest with a long history of open-source in this space. Super excited to have him on. Before I introduce our guest, I want to make sure that the audience knows who was talking. I'm Richard Littauer, I'm your normal host and then we also have Justin Dorfman, the other normal host. Justin, how are you today? [00:57] Justin: Normal. [00:57] Richard: Yes, me too, just totally normal, just in no way exceptional or interesting. [01:02] Justin: I'll be honest. I'll be honest, Richard, I am a little nervous because who we have on right now is a legend. So go ahead. [01:10] Richard: Yes, but he hasn't done that much, right? Like Brian J. Fox, he's smiling right now. He's just a normal guy, right? Wearing a black t-shirt, got glasses on. [01:18] Brian J Fox: Yes, I would say of the three of us I am definitely the most normal. [01:21] Richard: That's true. That's true. [01:22] Justin: Could be. [01:23] Richard: Audience, you can't tell I'm wearing a pizza for a hat, not true. Brian has a long history in this space. He's the co-founder of Orchid, a blockchain company, started in 2017, but he's also done a lot more than that. So he created the Bash Shell. He was the first employee of FSF, otherwise known as the Free Software Foundation. That's about where my brain just stopped and was like, wow, I don't even know where else. How did you end up there? How did that happen? [01:54] Brian J Fox: How did I end up at the Free Software Foundation? [01:56] Richard: Mhmm. [01:57] Brian J Fox: So, I was excited about working with computers and I started writing some software on an Apple two computer around, I don't know, 1980, 1981 and I was teaching a gifted and talented; I'm ancient, by the way, I'm about a million years old. [02:14] Richard: He looks like it too. [02:15] Brian J Fox: Yes, I do. I appear quite old. I'm just kind of a bent-over, decrepit old man. I was teaching gifted and talented seventh and eighth graders and the language that they were learning, I was doing some computer stuff with them, and the language that they were using was called terrapin logo and the terrapin logo software was somewhat broken and I used to delve in there and fix it in the Apple two, you could break into the monitor and make changes to your running software by actually just patching the code live. So I was doing that one day and the owner of the Terrapin company saw me doing that and said, wow, would you like a job? I said, sure, why not? And I already had relationships at the MIT AI lab, Marvin Minsky was a family friend and I went to school a few years ahead of his youngest twins, Henry and Julie Minsky, who are still great friends of mine by the way, wonderful human beings. [03:12]So anyway, I started working for Terrapin and the two other guys that were working at Terrapin were also grad students at the AI Lab. So there was a lot of overlap between things that were happening at the artificial intelligence lab and this job I was doing, and I wrote a full-blown version of Emacs for the Apple two called A-Macs. It was an 80 column editor with dynamically loaded libraries and printing modules and multiple languages. It was free. It was pretty full-featured and I was very excited about it and I wanted to show it to my, then at that time hero Richard Stallman, because he had written Emacs and Emacs to me was the embodiment of how to think about software architecture. I could see from using the program, I could kind of understand the architecture behind the way in which it was built and I thought that was super elegant. So that kind of made me want to write a version of Emacs, so I did that and then I ran over to the lab and I saw Richard Solomon coming down the steps and I said, Richard, I have to tell you something. He said, what? He didn't know me at all. I said, I wrote this complete version of Emacs for the Apple two, and it's called A-Macs and it's got all medics completion and everything, it's super good and he says, well, I don't think I'm a good distribution point for that and then left. [04:25]He kind of had completely missed that I was saying, you're my hero and I tried to emulate you and thought that maybe I was talking about distributing software. So the guys I was working with a Terrapin about six months after that said to me, you should talk to Richard Stallman, he's trying to start this free software thing and I think it fit in really well with that. I said, well, he didn't really want to talk to me when I talked to him the last time, but I'll try. So they had told Richard that I was coming over. So, I came over and Richard said, I hear you're a real wizard and we'd love to have you working in the free software foundation. I said, that sounds exciting and Richard and I hit it off pretty well and then I started writing software for the Free Software Foundation and that's one project I knew that started. [05:06] Richard: Wow. [05:06] Brian J Fox: Yeah. [05:07] Justin: Did you ever think Bash would be still around when you were first developing it were usually like, oh, this is going to be a game-changer or is it just like, ugh, I have to do this task? [05:20] Brian J Fox: Software doesn't really have a good shelf life just in general and that was true in the early eighties that was still true. So I never thought that any piece of software I wrote would have any longevity at all and I'm super surprised to find out that many pieces of software I wrote have some longevity and of course, Bash is the most famous of those and is now available on multiple planets as well as basically every computer that exists on the earth today. [05:49] Richard: Is Bash running on like on Mars right now in like one of the rovers? [05:54] Brian J Fox: It's running on Mars in the helicopter, yes. [05:57] Justin: Nice. [05:58] Richard: Cool, okay, that's awesome, you're literally in multiple planets, that's really cool. [06:02] Brian J Fox: Yes. Yes. Interplanetary software, yes, that's pretty good. I never really thought that it would have this lifespan and I think it's just the nature of the very specific thing that Richard and I and others were doing with this project Canoe I did, this completely free version of a Unix replacement that was completely free. I think it's just because of that, that this piece of software has this longevity. If I wrote a brand new piece of software that delivered brand new functionality, then I think it would have some lifespan and other people would then take that and improve upon it and make something even better or different. But in this particular case, we were trying to recreate a legacy system and it had to be perfectly matched to the legacy. It had to be a hundred percent backward compatible. So we could add things to it, but it had to be a hundred percent backward compatible and when that happens, when we delivered that it became the defacto standard. So all the software that came before that uses the shell to do stuff, continue to rely on the shell and that's why it's been around 35 plus years. [07:07] Richard: So I know that it's now on windows, that was announced in 2016. Can you talk a bit about that story, how that happened? [07:14] Brian J Fox: I don't know how that happened. Well, what I think really happened was Bill Gates left the scene, stopped kind of pushing hard on what he was pushing hard on and he's a relatively proprietary guy and his successor was kind of proprietary and then third guy came along and he's realized the modern world is not really looking to stick with only proprietary solutions and they realized that free and open-source software is important, which is why Microsoft did their version of helping free and open-source software by for example, buying, GitHub and delivering the PowerShell, which is... [07:51] Richard: Classic. [07:51] Brian J Fox: Yes. [07:52] Justin: What was your reaction when you heard that? Because for me, I remember where I was, I was at my desk when I heard this and I was like, how did this, what, like, to me, it was just like a total shock. How did you deal with it? [08:07] Brian J Fox: I was kind of like, well, it's about time. I mean, for the previous 20 years, we had a Cygwin, which was basically a Unix-like environment that you could run on a windows machine and it was just mildly painful to get set up and to use and to try to get it to interoperate with all the other windows stuff. So when Microsoft started making noises about embracing open-source, it made sense to me that one of the first things they would do is kind of take the staple pieces of software and make sure they were running on their own. [08:41] Richard: You mentioned Bill Gates changing his mind, which I kind of have to ask, Richard Stallman is obviously a very controversial figure today, some of the stuff he's done in the past few years totally sucked and various; it's like not someone I even want to talk about anymore. I'm kind of ashamed I share his first name. So at some point you left FSF then you moved on and you started working on Orchid. I'm curious how that started. [09:06] Brian J Fox: So I did not move from FSF to Orchid. There were about 30 years in between and believe it or not, I've done other things since then. So here are some things that I've done since then. So I moved to California from Boston and I continued to work for the Free Software Foundation, writing software for Project Canoe and championing open-source and free software. And then I started to travel around and hang out with all these people that I'd met at MIT in different parts of the world and I did that for about five years. Then in '95 I had come back and a friend of mine wanted me to work with him at Wells Fargo and I resisted quite strenuously and then eventually got convinced to. So I then wrote the first version of Wells Fargo's online banking and Wells Fargo was the first bank to go online here in the United States and after they didn't get ripped off immediately, other banks went online, which was great. In the course of doing that, I realized there were no good tools for building database-backed websites. We wrote that in C and you wanted to change the color of something, the designer would say, I want to make that thing be a darker blue and you would get into the C code and change the color in the C code of the thing that's going to make the web page. [10:18] Richard: Wow. What year was this? [10:19] Brian J Fox: '95. [10:21] Richard: Wow. Okay. [10:22] Brian J Fox: And then in August of '95, I wrote programming language for programming the world wide web called Metta HTML and it was a HTML compliant language, meaning that the text of the language looked like HTML, could look like HTML and you could have, but it was functional language like Lis-Pro scheme, so you could have functioned calls and do all these things. It was executed server-side, not client-side. This was so early on that some browsers didn't support the table tag. I mean, it was like that, like we had the blink tag [Inaudible10:56]support. So this language would allow you to do things like define a function calls table that looked like an HTML side called table and you could just pass it through for browsers that supported that or you could turn it into a bunch of pre-statements and make and draw a beautiful ASCII table. [11:16] Richard: Polyfills before polyfills. [11:18] Brian J Fox: Yes, exactly, exactly. [11:19] Justin: Did you talk to Tim Berners-Lee about it or did the W3C know about it or did you work with them? [11:26] Brian J Fox: W3C knew about it, they had started to do some server-side scripting stuff. Everybody was busy writing Apache and I talked to the main guy who was working on Apache at the time in the lab and I was like, hey, look at this cool thing that I've got, this should just be a module inside Apache and he's like, yes, I'm working on something else. I don't really want to do that right now. I was like, oh, okay, fine. So, then I made a web server [Inaudible11:50]all this stuff to kind of cover that and then several companies, this was the mid-nineties, mid to late nineties, so several companies made their gigantic tens of millions at the time that was a gigantic number, tens of millions of dollars building products using this programming language. It was pretty well received and worked pretty well. And I was interested in start-up companies. So around 2000, I started a company that works with start-up companies and invest in them and that was called the ACORE group and it is now called The Opus Logic but basically, it's an extension of that. We invest in start-up organizations and we provide technology to those start-up organizations and take out equity. [12:30] Justin: Got it. [12:31] Brian J Fox: Then in 2017, I had this idea and I drew something on the board and the idea of basically was I had a friend who was living in China and he couldn't get to Wikipedia and I was like, you should be able to get to Wikipedia. He's like, well, every time I use a VPN, the great firewall of China just closes it down. And I was like, well, maybe you can just use a VPN that I set up at my house. Why don't you do that? And I thought, well, the general case of that is we can have a network of participants and people who use services will pay for them and people who provide the services will get paid and it's all just a peer to peer network sitting on top of our physical network infrastructure that we have. So now what we need is a way to smuggle bits back and forth so nation-states like China, can't see the data that's being moved around. And the great firewall of China is a fantastic IT project. It employs the most people and they spend the most amount of money on it and they're very technically savvy. There's nothing schlocky about what China does there, but they do allow certain types of traffic to go back and forth. For example, peer-to-peer zoom calls are allowed to go back and forth. So imagine if I package up all the data inside my zoom call. [13:43] Richard: Are we doing that right now? Is this data going to China at the moment? I mean, we're using Riverside, but basically, I'm just curious. [13:50] Brian J Fox: Well, Riverside is owned by China, edit that's false. [14:00] Richard: So how does Orchid work then if you're bundling stuff in Zoom calls, or do you have a fake Zoom call proxy that you're just sending everything through or what? [14:06] Brian J Fox: No, I said zoom call, but actually just think of any other protocol that's not proprietary. For example, WebRTC is an example of a protocol that'll let you do these things back and forth. So you can put interesting bits of data in there, but you could put interesting bits of data in any kind of thing. See, the network internet protocols are packet-based, which means that they're discreet. You put a bunch of data in a packet and then you send the packet out and then eventually somebody receives the packet and if you have good sequencing, so you know which packet comes first and which packet comes second, you wait until you have the packets all put together and then you have this bundle of stuff. So I can actually use email as an internet protocol, it's just very slow. [14:46] Richard: So email is super inefficient? [14:47] Brian J Fox: As a method of doing packet network communications. It's actually rather efficient for moving conversations back and forth. [14:54] Richard: Yes, my mom loves it, works great. [15:00] Justin: You have like some sort of infrastructure for this project for Orchid. What are we looking at? Are you full cloud, your hybrid? What are we looking at? [15:11] Brian J Fox: Okay, good. Let's talk about Orchid for a minute. So, one of the drivers was to allow access to what we like to think of as the free internet from behind restrictive firewalls, that was one driver for doing it. Then that wasn't really the only reason it was like, oh, wow, there's a bunch of, this is the way that the internet used to be back in the late eighties and early nineties before we decided to go with DNS and have a whole bunch of centralized stuff. The internet was really just a bunch of people connected over DSL or dial-up or whatever we have in order to do peer-to-peer communications on things. First protocol we had for sharing information like that was FTP and the very next thing was Gopher and Gopher was exciting because you could look up and see what information somebody might have and then go get that information from them. That was amazing. But you had to know how to get to the person's place. It's kind of like the way tourists today, you have to know somebody's onion address to get to their server. [16:11]So then of course people invented the DNS protocol, which was a dynamic name lookup, and it allowed you to name, give your nice logical name and then all these servers could know the address associated with that name. Well, that was great, but that was also the end of what I would call privacy and the internet the way we knew it. And then the way it is right now at my house I have Cox cable and Cox knows my name, my address, my social security number, my credit card number, and every single website I visit and I don't think that's really Cox's business, number one. And number two, even if I totally trust Cox in every way, shape, and form with that information and that they're going to do their best job to protect it, some nation-state or some bad hacker comes along from Russia, for example, hacks into Cox and takes all the data because all of this data is stored in one central location. So now they know all my information and they know yours too Justin and yours too Rich, and they know everything right. And this has already happened time and time again, just take a look at all the, I mean, we only hear about it when it's a credit card hack. Like, oh my God, at Target, all the people who've ever used a credit card at Target and I'm like, oh, that's not me. I'm not one of the 50 million people who had their credit card stolen. Of course I am. I mean, everybody, I think everybody in the United States and many other countries have had their information compromised. [17:37] Justin: Absolutely. I think even for ISPs they have retentions for, if someone needs to check, like if they get a warrant to check someone's internet history, which is just really creepy, but that's what Orchid is like doing. [17:51] Brian J Fox: I mean, that's really the point. So it was big news when Apple said to the FBI, no, we're not going to help you break into this phone. We were all like, yay, they're our champions, they're protecting us. But the point is they could have helped them break into the phone, they have that technology. So if they know how to break into the phone, other people know how to break in the phone and that's what happens when you have a cell phone, everybody who wants to can kind of tell where you are and who you talk to. [18:20] Justin: Yes, it's a freedom... [18:22] Brian J Fox: Yes, it's the nature of these network devices. [18:24] Justin: Yes, it's the payoff, the contract you make. [18:27] Brian J Fox: Yes. So, we, the humans that use these convenience features are in the habit of trading our privacy and our agency for convenience and it's a very short-sighted view in my opinion. So an Orchid network and an Orchid-like network is an excellent way to combat that and still deliver convenience. [18:49] Justin: So is it strictly peer-to-peer or are there servers that you need to run to kind of, if there's not someone on the network, how does that work? [18:59] Brian J Fox: So the protocol is strictly peer-to-peer. If you're asking, did we seed the network with some servers, so that was some nodes that are running Orchid protocol? The answer is, yes, we did do that, that's definitely a good way to get started, but that's not the ultimate way in which this works. The more people that use Orchid, the more people provide services for Orchid and the more people get paid. And so this whole thing was just an ecosystem to have the method of payment to incentivize peer-to-peer to communications. So I'm kind of excited about it still. [19:29] Richard: Yes, it is super exciting. I think what Justin is trying to get at, how is this related to cloud technologies? Because that's kind of one of the things we're interested on here, we're about committing to cloud-native. What I'm hearing is you built a super awesome protocol for sharing bandwidth between users and basically to allow for like another way of doing VPNs and then you also have a cryptocurrency that's built on top of that protocol, which incentivizes people to actually share their bandwidth. It's very similar to me to protocol labs, which for instance is going around and sharing data and you have like a content-addressable system and then you have a crypto on top of that, which incentivizes people to use it. Does that sound accurate? [20:04] Brian J Fox: I think that's very close. I'm going to make a slight change to that, which is the Orchid network is really a way for computers to participate in a network and to buy and sell services from each other. And bandwidth happens to be the lowest hanging fruit easiest thing to sell, it is a wasting good. Everybody has extra bandwidth, nobody is on a hundred percent of the time using up every little bit of bandwidth that exists on the planet. So that just happened to be the low-hanging fruit, but selling compute services, selling CPU time, use of memory, the use of storage, all those things are completely reasonable things to do inside the Orchid network. And there are other networks that are out, other protocols that are coming out that are also doing that. I'm an advisor to Acache network and Acache is fabulous. I love what Acache is doing and I love the supermini, which is a small device that any person can put in their house and it can run various protocols, various blockchain protocols. [21:03] Richard: So ideally at some point, this would actually replace a lot of the cloud computing stuff that we currently have, right? Because cloud computing is based around the idea that I have a lot to run, I'm going to run into someone else's servers, largely one of the big four, but you're saying was, well actually, why don't we just have everyone running stuff all the time and we could just find a way for people to incentivize and pay for that and then have it run on their local computers. [21:23] Brian J Fox: Sure. I mean, I pay for my use of AWS. Why wouldn't I be happy to pay less for my use of your computer? And the fact is that the protocol allows for, like IPFS, for example, allows storage across a wide number of nodes. So my data is automatically backed up. So I don't have to trust that Richard will always have his computer on and it will always be working well and it'll never run out of disc space or anything. He's just one of the participants in this network and he gets paid accordingly to the amount that he participates. [21:55] Richard: What's also interesting about IPFS is a lot of the early nodes are actually backed up by AWS [Inaudible21:59]because that's one of the ways of seeding the network. It's like, well, let's just have the cloud run it for a while. [22:04] Brian J Fox: Right, no problem with that. Those are useful machines as well. [22:09] Justin: Do you still seed or do you have enough users where it's just kind of self-sustaining? [22:14] Brian J Fox: I can't tell you how many users we have, not because I don't want to. [22:18] Richard: On purpose. [22:19] Brian J Fox: No, because I don't want to, but because I literally cannot do that. I don't know how many users we have. I can tell you how much OXT it moves back and forth, that I can tell you but I can't tell you the number of users. I don't know that answer. I don't know anything about the network utilization except for how much OXT moves around. [22:40] Justin: What's OXT? [22:41] Brian J Fox: OXT is the name of the cryptocurrency, the Orchid cryptocurrency. [22:45] Justin: Got it. Got it. [22:46] Brian J Fox: It's an ERC 20 token, which means it's based of an Ethereum. [22:50] Justin: Okay, great. [22:51] Richard: If I can take a step back, what I love about this conversation, Brian, is that there's just so much. You've been around for so long, you've done so many things that like, I don't even know where to start asking questions because I'm just like, wow. Even IPFS, Bash is literally IPFS in the sense that it's running on another planet. So, like, way to win there. [inaudible23:09]good for you, you win. What's really interesting to me is that you started out at FSF and then you've also gone on and had this long chain of things. You built languages that were early compilers for websites. You were around as we saw the dot com come and the dot com go, then you were around for cryptocurrency rise and now you're an investor, you're an ecosystem level thinker. What I'm curious about now is what do you think about open-source as a movement in the last, like two or three years? Where do you think it's going and how do you think you're leveraging that in Orchid in as best a way possible to make sure the success of the system that you're building? [23:51] Brian J Fox: So free software and open-source, once again, like this internet stuff used to just be the way things were. I mean, software is an expression, where as humans, all we do is manipulate symbols. We don't do anything else and we're really good at it. We're good at pattern matching and we love it, that's what we get stuck doing. I think I can break someone's brain by giving them unsolvable patterns in some way. I think that's like a possibility, don't do that, but I think it's a good idea, but don't do it though. We're basically pattern matching semantic machines when, for the technical people in the audience who know what a byte is and what a memory location is. If I take a memory location and a computer and I put the number 65 in there, what does that mean? Does it mean the number 65? Does it mean a pattern 0001 0001? Does it mean the letter a, the small letter a? What does it mean? Does it mean some dots on a screen? It's only what we perceive it to mean and just what we're doing right now, sitting here in front of a computer, we think we're talking to each other and we are but the symbols are coming in, we're just interpreting these signals to mean something. [25:05]I'm just looking at some dots on a screen and Richard's nodding his head and you can hear my voice and it's all just electricity. But as people, we map it into our human experience. So that's number one. So then think about that as you write computer software, really all you're doing is expressing some idea and computer software, the beauty of it for me is in the architectural design, is in how you craft your thoughts to encapsulate the solution to a problem. This is not very different from what mathematicians do when they make mathematical proofs. They craft their thoughts in order to give an articulation of a solution to a problem. Mathematical formulas are not secret, they're open, they're free, they're open source and in fact you cannot patent a mathematical formula. [26:00] Richard: I didn't know, it makes sense. [26:02] Brian J Fox: You can patent a process, but you can't patent a formula. So why can we even think about patenting software, for example, or making it be secret? What is the advantage of making it be secret? Where nobody else knows how to do it, but in fact, the instantiation of this thing we're looking at where we're doing this video call, it's the idea of this and the execution of it that makes this thing valuable, not the code that went into it because now that I've seen this, I can write code that will do this. The hard part was thinking of this, right, was thinking that this could be a real experience that we could use. And then wow, now that I see that it can happen, large numbers of people are capable of writing software that can do this. So what's the advantage in slowing them down? Am I really going to keep my thing proprietary? Is there a good business reason for that? I think not. I think there's not a good business reason for that and all I'm doing is slowing down the advancement of this technological field, which in this case is slowing down the advancement of semantical understanding and the way people interact with each other. So I don't want to be part of that. So everything I do is free and open-source. [27:11] Richard: I like it, it aligns really well with your story about your friend in China, who wanted to go on Wikipedia. One of the things that we want to have happen everywhere is people are able to access cool knowledge. If I want to go learn about the horse caller, I should be able to just read a Wikipedia article about the horse caller and if my government says, that's not okay, it's like, well, why, it's just technology. So it seems like you're really aligned there, which I just, I love. That's awesome. I'm sensing a lot of what you're saying has to do with protocols and most developers these days don't work on the protocol level and don't think about how the internet was made originally. I would say most developers are thinking about JavaScript widgets and trying to figure out how to make this work with react in some other way and how to make a site look a certain way. So for you, it's all about engineering. It's all about software at scale. Does that sound accurate? [28:01] Brian J Fox: For me it's about problem-solving and the architecture that goes into the problem-solving. So sure that is software at scale. I mean, you can't have software at scale without that kind of problem-solving mentality. Literally it's about the expression of a solution and the other thing is like, physicists do this and mathematicians do this and I think that people who are not just academics, but people who are into this kind of problem-solving space do it a lot. The idea is what is the general problem that needs to be solved and what's the general solution for that and once we have a general solution, then you can start to do things, which is why we have computers. They are general purpose computing devices and we use them, everybody nowadays uses them for everything. They don't think about it. If you put your beverage in the microwave and you push the beverage button, the last thing you think about is what language was used to write the software that operates my microwave. I mean, you just don't think about it and most people don't think about the programs that they run on their computers, which are extremely valuable in their lives for communicating with other human beings and for discussing and sharing ideas. [29:05] Richard: It's great that most people don't have to look at the abstraction of their microwave. It's great that they can just press a button and go. A lot of that's partly because there's been a proprietary company that's figured out how to make the microwave and how to sell it to people. We use a system of money where money is used as a way of saying, I put this amount of effort in, and if you want to not have that effort, you can buy this thing. I know that you're a capitalist in some sense, you have a venture fund, all that is using these weird levels and balances for using money to control how information flows. I'm how you thread the line between being an open-source diehard information should be free, why am I just slowing people down and I run a capitalist firm? I know that's maybe a tough question, I'm just curious how you do that. [29:50] Brian J Fox: I don't see that there's any conflict at all. I mean, just think about this. You have a guy mow your lawn, there's nothing proprietary about that. He does some labor, he gets paid. There's nothing proprietary about it. If you asked me to think about your problem, that's fine. I'm going to think about your problem and you should pay me for that. You're going to get a solution to your problem. That doesn't mean that you're going to be the only guy in the world getting the solution. So thanks for contributing to the solution, to that problem, that's very useful. And the only reason why you would want to be the only guy with the solution to that problem is if you wanted to prevent other people from having that solution, which I philosophically disagree with and I don't think it's good for humanity and I don't even think it's good for your business. You're a brand new start-up and you've got this great idea, you think that cars should be able to fly and that's your thing and you're going to make the car that flies and then everybody will want a car that flies and everybody will have to buy your car. [30:48]The truth is for you to succeed in your business you need an ecosystem. You need a large number of people to believe that flying cars is a good thing. You need choices in that ecosystem so people who have a lot of money can buy the expensive version and people who have a small amount of money can buy the cheaper version. You're not the guy who can make all those things at once because you're a start-up and you have to focus on a specific target market. So what you really want is for this to be a valid market and for you to be a useful player in that market. Well, you've got first-mover advantage and that's really important. You're going to be one of the guys solving this problem. If you can take that information about how to solve this problem and give it away to everybody, you will create this massive ecosystem in which you will be a major player. And if you don't do that, your business may fizzle and die because not enough people can buy your product, because your product is super expensive, or the people who would give you a lot of money, you've targeted them, you haven't made a product that really fits the masses. It's only fit this smaller group of people, and you're only getting a small amount of money. So it's better to create an ecosystem than it is to try to have your own specials proprietary niche solution. [32:01] Justin: I want to make a quick U-turn and I want to go back to the VPN industry. I watched a lot of YouTube, most of the shows I watch, they're all sponsored by VPNs. I hear mixed reviews about VPNs, they are scummy, they are lying to you, they're not completely encrypted, whatever. I'm not naming company names, but just a lot of money goes towards YouTubers to promote the VPNs. What are your thoughts on the current market of traditional VPNs that are not crypto-powered? [32:37] Brian J Fox: The issue I have with traditional VPNs is you have moved the centralization issue that I talked about for the ISP and I will mention names like Cox, for example, we've moved from the centralizing of your information at the ISP to the centralized area of your information at the VPN. So the VPN now knows who you are, your social security number, your credit card number, and every website you visit and they're susceptible to the exact same issues that the ISP was susceptible to. A nation-state comes along and steals all the information from the VPN and now all that information about me is in somebody else's hands, perhaps in somebody's nefarious hands. So that's the downside of a VPN. It doesn't solve the problem. [33:26] Justin: Right, it just basically makes it someone else's problem, in a sense. [33:29] Brian J Fox: Exactly. [33:30] Justin: Now, how do you deal with requests from law enforcement agencies? Is it just like a, hey, we there's nothing we can do? [33:39] Brian J Fox: If the police come, if some government agency comes to the board of Orchid, which I'm on the board of Orchid and says, give us all the information you have and all your users, I say, okay, I'm done now. I don't have any information on my user, I can't help you. They're not my users, I created a protocol, Orchid created a protocol and people use that protocol. It would be like going to the internet protocol organization and saying, give me all the information on your users, they can't, they don't know who users are. They just created this protocol that users use, but they don't have a list of users. They can't see who's using the protocol. I mean, this was one of the; very early on, I was very clear on this, the idea was to create a complete open-source solution that stands on its own and doesn't even need a company. There should be no company involved in the execution of this plan. The only company that we needed was we wanted to put a group of developers together and get them focused on solving a problem. [34:46] Justin: And make really cool bunny illustrations. [34:51] Brian J Fox: Yes, I will tell the honest truth, I had nothing to do with the bunny illustration. [34:58] Justin: I like them. I'm saying those costs money. [35:02]: Richard: So I like the idea of just not having things about users, not tracking that and just building better protocols for people to share and I do see in the future at some point, I mean, this will replace cloud if it's done well, right? So a lot of the cloud stuff now is basically run by large companies like Google, Amazon, and it doesn't need to happen, it could also be run by users. That doesn't mean that the cloud space is going away, oh, listeners, it just means that it's going to be a different end point that you're hitting. We're always going to still need to have how to do a software at scale, how to figure things out, and how to have good VPNs or how to use things currie fence to make it work. But Brian, I love how you talk about ecosystem level and I think it's really important message that just needs to be hammered home. Every time I hear it, it's like, yeah, of course, why don't I say this every day? [35:46]We are running up on time and I want to make sure people have the ability to listen to you further than this podcast. So where can they find you online, on the web, other podcasts, tweets, Twitter, what do you do? [35:57] Brian J Fox: I tweet very rarely. My Twitter handle, I think is Brian J. Fox or Brian John Fox, I'm not sure. I think it's Brian J. Fox. I never Instagram. If you see a band called Chill Point Band, chillpointband.com, if you see a band called Chill Point Band playing somewhere, then go there and talk to me, I'll be playing in that band. I'm a bassist. [36:16] Justin: Oh, nice. [36:18] Richard: Sweet. [36:19] Justin: I didn't know that. [36:20] Richard: Covering all the bases. Nice. [36:23] Brian J Fox: Oh boy, edit, get rid her that joke. [36:25] Richard: I'm not sorry. Listen, I did want to say one thing though, you were talking about cloud companies, not going away. We have television content and the world pays a lot of attention on television. They watch TV shows and that they watch that and then streaming content providers came along like HBO and Cinemax and so forth and people started watching those streaming content providers and then new television stations appeared, which were streaming stations, streaming content, but content made for television. Then YouTube came along and as Justin pointed out, he watches a lot of YouTube. I also watch a lot of YouTube. There's a lot of content on there that I find interesting and it's like I get my on-demand streaming television fix. But TV didn't go away and I appreciate the fact that somebody makes Game of Thrones and I get to watch that and I don't think that individuals have the wherewithal or the incredible skill sets to gather the skillset together, to make something like Game of Thrones on YouTube. Maybe I'll be proven wrong in the future. I'll be very happy about that, but it means there's room for both. As long as the legacy providers pay attention to what's happening in the motion and the world around them and try to keep up with that and move forward that way, then the legacy providers will still be providing some value and people will use them as well. [37:49] Justin: Great point. [37:49] Richard: As a consultant who works within that ecosystem, I'm not sure exactly how much I can say, but I'm making a language right now for a series that you may be able to get in the streaming service. I appreciate that it's not going away and no individuals cannot do everything because I know nothing about lighting. I can make a language for your creatures, but that's about it. [38:07] Brian J Fox: Oh, that's great. I'm excited. [38:10] Richard: It's kind of fun. I'll let you know what it is when I can, NDAs are important. [38:15] Brian J Fox: Fully understood. [38:16] Justin: Richard's a boss. He's a language guru. [38:19] Richard: Bash is awesome. I use Bash every single day as it is. So I'm also just incredibly grateful for your work. [38:24] Brian J Fox: Thank you guys so much. I use Bash all the time and I just want to quickly say that the people who provide documentation and instructions on how to use these tools that have been around a long time, they are doing a fantastic service and Nick's Craft is one of those. You should check them out if you can online. [38:43] Justin: Yes. [38:43] Richard: That'll be $14.95. Excellent. Thank you so much, Brian. Special Guest: Brian J. Fox.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Tzury Bar Yochay Guest Snow Pettersen Envoy Proxy Senior Maintainer Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about the confluence of Cloud Native and Open Source. Today, our special guest is Snow Pettersen, who is an Envoy Proxy Senior Maintainer working at Lyft on the Resilience team. Snow has done Cloud Native at Square, Netflix, Lyft, and he tells us how it’s changed over the years and a particular challenge he had recently. He also shares with us about problems with the release and rollout with sidecars in Envoy. Speaking of Envoy, Snow explains exactly what it is and what it does. We also learn the architecture of Envoy, the new contrib folder proposal, extensions coming out, and the “golden rules” to follow when reviewing a code. Go ahead and download this episode now to hear more and thank you for joining us today! [00:02:06 (https://podcast.curiefense.io/21?t=126)] Snow has done Cloud Native at Square, Netflix, and Lyft. Find out how it’s changed over the years. He also tells us about a recent challenge he had. [00:03:47 (https://podcast.curiefense.io/21?t=227)] We learn from Snow that the biggest headache he’s seeing with people using Envoy has been the release and rollout problem with sidecars. [00:06:47 (https://podcast.curiefense.io/21?t=407)] Tzury wonders how Snow would explain Envoy to someone. He also tells us how it switches to the new set of configurations while processing and Envoy’s scalability on a single machine. [00:13:16 (https://podcast.curiefense.io/21?t=796)] Snow goes more in depth about the architecture of Envoy and the new contrib folder proposal. [00:20:24 (https://podcast.curiefense.io/21?t=1224)] Find out how many people are actually maintaining, monitoring, and moderating the process. [00:24:02 (https://podcast.curiefense.io/21?t=1442)] Justin asks what Snow anticipates on extensions that will be coming out that can’t make it to core and what is it that people want that they can’t get right now. [00:26:43 (https://podcast.curiefense.io/21?t=1603)] Tzury wonders what the most obscure, unexpected use of Envoy was in production that Snow came across. [00:28:17 (https://podcast.curiefense.io/21?t=1697)] Over the years that Snow has been at Envoy, he tells us how much of his time he spends writing new code versus reviewing others versus answering emails and file or responding to issues on GitHub. Justin shares some stats from Snow’s GitHub profile. [00:29:54 (https://podcast.curiefense.io/21?t=1794)] Snow shares the “golden rules” when you review a code. [00:33:04 (https://podcast.curiefense.io/21?t=1984)] Find out where you can follow Snow online, and he gives a shout-out to the entire Envoy community! Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Curiefense Blog (https://www.curiefense.io/blog) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) jdorfman@curiefense.io (mailto:jdorfman@curiefense.io) podcast@curiefense.io (mailto:podcast@curiefense.io) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?lang=en) Snow Pettersen Twitter (https://twitter.com/snowypeas) Snow Pettersen GitHub (https://github.com/snowp) Lyft (https://www.lyft.com/) Envoy (https://www.envoyproxy.io/) Episode #17: “99.99999% Uptime with Anna Berenberg” (https://podcast.curiefense.io/17) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript [00:00] Snow Petterson:There was a period of time around this time when I started being a maintainer and a bit before when I was writing a lot of code, just because again, I think it aligned very well with what my company needed at the time. Now, over time I've just gotten review ownership over more and more codes and being brought into more and more like, hey, you know how this works, so can you chime in? So I've definitely like drifted away more towards the side of communication. It's always nice to get some code written every now and then, but there's so much other stuff that happens that I always have to be careful about making myself the blocker for the code landing. [00:42] Intro: Hello, and welcome to Committing to Cloud Native, the podcast where we talk about the interface between open source and cloud native. We're super excited about our guest today, can't wait to introduce him. Our panelists today are Justin Dorfman and Tzury Bar Yochay, and they're going to have an awesome conversation. I really enjoyed listening to it and I really hope you enjoy this conversation. [01:06] Justin: Today we have Snow Peterson joining us from Lyft. He's on the Envoy Proxy Project as well, senior maintainer. Tzury, you're here, what's up? I thought you almost had a COVID, but you're good. [01:18] Tzury: Hey JD. Hey Snow. How are you guys? I'm all good. I'm fine. Thank God. [01:22] Justin: Okay. Thank God and Snow, how are you? Are you doing good? [01:26] Snow Petterson:I'm doing great. yes. Happy to be here. Thanks for having me. [01:30] Justin: I really appreciate you coming back because for the audience that doesn't know the backstory, Snow was on like a month or two ago and the audio was so bad that we had to pull the plug. So we rescheduled and Snow, thank God said yes and that's where we're at. And we just want to basically go over what we talked about, but this time with a new recording platform and new equipment. So thank you again, Snow for really taking the time to do that. [02:04] Snow Petterson:Yes, no problem at all. [02:06] Justin: So cloud-native, you've done it at Square, you've done that Netflix, you've done it at Lyft. How has it changed over the years? [02:13] Snow Petterson:It's definitely matured a lot. I think a lot of the stuff we were doing early on at Square, particularly in the Envoy spaces, which is how I ended up in this whole space. It was rough around the edges and it took quite a while to ramp up on things and things didn't always work the way you wanted and I think now things have definitely matured. I guess it's been four or five years at this point. So more problems are solved, things are easier to do, but still a lot of challenges. [02:42] Justin: What's a major challenge that you've recently experienced, whether it's at Lyft or just maintaining the project? [02:49] Snow Petterson:I think one of the interesting [Inaudible 02:51] there's been this push towards like a [Inaudible 02:58]approach where a lot systems are relying more and more on these open source projects that run next to their services and Kubernetes and assessments as well and this has been like a trend in cloud-native where more and more problems have been sold via site cars, which on its own has cost like a bunch of new problems around like management of these site cars. And I think a lot of people who jumped on the site car bandwagon early on are now running into issues with managing all of these site cars with companies having 5, 10, 15 site cars running and their pods resulting in a whole set of new difficulties that people didn't realize would be this bad once when they were preaching about the value of site cars. [03:48] Justin: Is it like a performance issue or is it more of a security? What's the biggest headache that you're seeing with people using Envoy and site car to loading? [03:57] Snow Petterson:It's a release and rollout problem, that's a huge one where it's tricky to have a good release policy for site cars because you're kind of torn between two sides. One which you want to get new code out quickly and safely, but it's hard to do quickly if you have to roll your entire fleet, there's a lot of work to do this safely because you can try to roll your entire fleet, what kind of stats are you monitoring, what kinds of systems are in place to make sure that things don't go wrong and just the idea of having a gradual rollout of site cars can be very tricky because you end up having to often the build your own systems if you want something more granular than like per Kubernetes cluster, for example. So what you get per cluster, it's probably not too bad because you can just kind of do them one at a time, but taking down an entire cluster can be pretty bad as well, depending on your setup, not everybody runs with a bunch of redundant clusters. [05:01]Then you have this problem of once you start building automation for rolling the fleet to up-to-date site car, if you have a lot of site cars that need constant updating because you not only do you have like the open-source readily available site cars for you also building your own internal site cars, you end up having to roll your fleet quite a lot and it just creates a lot of churn and each one of these can cause a lot of issues. Sometimes you bundled multiple site car updates into the same update and then it gets very complicated because the person doing the actual rollout might not have any context around the site car being updated. And then the other way of doing it, where you don't roll the fleet, you put the onus on service owners to manage it whenever they update, they got a brand new set of site cars as its own set of problems where like, oh, you updated your app, you have seven new site cars, something's wrong. What do you do? [05:55] Justin: That's defeating the purpose of the whole microservices architecture. Microservice is supposed to make it easier and then this is just like monoliths all over again, kind of. [06:05] Snow Petterson:That's a nice analogy because you're going back to like a more monolithic setup in a way. But the app owner doesn't really know because they're just looking at their own code and they're like, oh, my code has 500 lines of code. This is barely anything, so I'm just going to deploy this rapidly because it's such a tiny amount. But if you're then also factoring in the tens and thousands of lines of codes, you are deploying every time a site car upgrade goes out, things become tricky and it's harder to move at the same velocity that you would have expected given how small your supposed microservice is. [06:40] Justin: Right. Tzury, thoughts? [06:42] Tzury: I think we have the perfect candidate that should answer the following question. How would you explain Envoy to a new, I wouldn't even say new commerce, somebody who knows nothing about Envoy, he here is the name, he/she, they heard the name and they went, what is Envoy? But they have no connection to the internet. They cannot type it in Google and get the answer. What would you say to them on a cocktail party Saturday afternoon? How would you explain to them Envoy? [07:15] Snow Petterson:Usually what I do, if I'm talking to somebody who might not even be familiar with microservice and disparate systems is kind of tell the story of how you end up here, which is once upon a time, there was a bunch of mainframes and everything around on one machine and things were easy, if you wanted to have different subsystems call each other but you just call the function or it's just all in there, it's easy. Then to scale out, you ended up building this microservice world where you have all different component talking to each other, and then you have to teach each of these components how to talk to each other and also then became difficult because you had a lot of different components wanting to talk to each other and they all had to like understand how to talk to each other. So, Envoy in the capacity of service [Inaudible 07:58] in other ways as well, but as a service mesh, which I think is how most people use it, it serves a role of centralizing how these services talk to the each other into a single process that can run alongside all of the applications, allowing them to communicate with each other and this is like the very basic of what it does. [08:22]Where its real attractiveness came from its ability for dynamic reconfiguration, which provides a consistent way for people to interact with it. Many people have done similar things with things that relied on very like laborious ways of updating the console or complicated conflict management systems. So the big thing that Envoy brought in to make this problem more attractive was all of these APIs and XTS mechanism that allows for a management server to push all of the configurations that it might need and update without restarting and in a very efficient manner and react to changes really quickly. In addition to this scale of really well, have a relatively low performance overhead non-zero, but it's fairly low and also be able to scale up to very large use cases. [09:19] Justin: So when Envoy gets an updated figuration from XTS server, whatever that might be, how does it gracefully switch to the new set of configurations while processing? By the way, what is the Envoy scalability on a single machine or a single corp to your knowledge? [09:40] Snow Petterson:In terms of concurrency, that's an interesting one, because the way that we suggest that we run it is that, and this will also go into play in explaining how this conflict update works, where Envoy will generally run one, if you want to run on a single machine, you run one [Inaudible 09:57]for core, and there are very few cross-thread interactions. There's some, but for the data paths where you're handling requests and proxying them, a single worker thread accepts it and handles it for its entire lifetime. And Envoy will basically, each thread I think is able to handle like, that's a really hard question to handle the scalability because it depends heavily on which features you're using, TLS versus TLS makes a huge difference for example. I recall in the past there being talks about like, I think 20 or 30,000 RPS synthetic benchmarks being run, but that's obviously very like you strip down everything and you like run it with like nothing. [10:39] Tzury: So, it wasn't HTTP two, it was HTTP one, for example, with HTTPS, no TLS. [10:45] Snow Petterson:Yes, like no modification of the page load, no access control, no nothing, which is not how people would use it. So, it becomes of the value of such a benchmark becomes questionable because there's this whole other set of features that I might use. [11:02] Tzury: Well, that's still an impressive number. [11:05] Snow Petterson:Going back to how the conflict update works it's generally, there's the static and SIG that it comes up with, the calls we've struck in SIG, which is required run time. This has to be, you need to do a process [Inaudible 11:18] changes, but the typical way you would define this static config is that you will basically say get all of my actual config for the management server and then the management server is responsible, we'll receive a request from the client requesting, hey, I need to know about all of my configuration and so the server is [Inaudible 11:39] of these and the [Inaudible 11:42]comes up with my internal object [Inaudible 11:46] resource he wants. So these are like listeners, which define which port, protocol and transport socket TLS one knock configuration for each listener clusters, how to communicate with all their systems, endpoints, the IP addresses associated with these clusters and then as any new conflict comes in, this gets handled on the main thread, which is used exclusively for this control plane interaction and it gets processed, it gets validated and then it gets posted to all of the worker threads, telling it to update as thread local version of this data. [12:19]Then there's another mechanism where generally [Inaudible 12:22]snapped to a stream or a request. So if you get a request and while you're processing the request you get a conflict update, it also acts on the old configuration just to make sure that each request has a consistent view of what the configuration looks like. [12:42] Tzury: So in terms of the Envoy architecture, which is something that when you dive in a little, you find it quite quickly. The beauty of having a stripped-down core barebone, I would say, minimal core of Envoy while almost anything you want to implement and do, things of which you consider obvious and ground level are pretty much implemented as filters or as extensions and I believe that was done by choice at the time. Can you elaborate a bit about this architecture? [13:17] Snow Petterson:So if we just start off with the filters for processing an [Inaudible 13:22] requests you can define like a list of filters, which is like each filter will process the incoming request and the outgoing response and gives you a chance to like modify the request. I think early on this predates me, but I assume it was natural to just implement the routing mechanism as one of these because what is the actual routing mechanism? Well, it's the thing that accepts the request and it generates a response. So this fits neatly into the filter mechanism and I think in constructing this extension mechanism, that was an early choice to make it very generic so that the way extensions are done can be reused for basically anything. Basically, you have some C plus API that you can implement and you register a factory that accepts [Inaudible 14:10]that defines it and this very generic extension mechanism means that you can do basically anything that's very easy to make anything an extension point. So a lot of things were quickly [Inaudible 14:26] where you'd take something that was previously not, for example, TLS used to be baked in, but in order to better support other ways of transforming the data on that level, it was extracted into a transfer socket extension so that now TLS is just an extension and this just opens up for so many other extensions we built that kind of like fits in the same spot in the stack as TLS. [14:55]So this is definitely a fantastic choice and has allowed us to do stuff like have different security postures for different extensions. We can say that we have built-in say 50 different filters. Only 10 of them are robust to trusted downstream sort of upstreams and whatnot, which allows us to better evaluate. We can tell people, we can guarantee that this has been vetted via fuzzing and via other production testing. We expect this to be secure. Others, newer extensions generally will be flagged as we don't know, or have a less robust one, which means that if we then get issues coming in and saying that, oh, I found a bug, if this, this and this happens, we can crash the process. If it falls into one or our like less trusted extensions, we say, well, that's okay, we'll just fix it while the other ones will go through like a security release, file a CV for it, because we've already made the promise that these things are secure and so we have to go through the right process to make sure that we disclose it in a responsible manner. [16:07] Justin: Does this have anything to do with the new contrib folder proposal, or is it completely different? [16:15] Snow Petterson:So the contrib folder is basically a proposed new way of structuring extensions, where in addition to having basically core and extensions, instead we'd have core extensions and contrib and contributing like a new collection of extensions that are held to, like not the same bar as core extensions and this is to address in terms of like coverage, making sure that it's been signed off by a core maintainer and whatnot, the idea being that there's a lot of very useful extensions that we love proposing, but some of them are, they show up with like a 5,000 lifeline PR and they're like, hey, I made this extension. It's super useful, we've been using it in production, but in order for us to get it into a state where we'll be okay putting it to core, it just takes so much time and efforts unless we have somebody willing to do that work it just sits there. So contrib is a way for us to say, yes, we would happily take it, we're going to put into contrib, which means that we'll do some like due diligence, make sure it looks sane and then we'll probably put it in there. [17:24] Justin: I mean, yes, that's definitely going to help. It's kind of like the WordPress model where you have this plugin directory and I could just see this taking Envoy probably to the next level if this proposal gets accepted and then built upon because it has to be very discouraging for developers coming and sending a pull request and then getting it declined. It's not like you want to do that, it's just got to find the time and there are other stakeholders that are going to be affected by this inclusion in the course. So I think this is probably the best way to kind of combat this issue that you're having. [18:01] Snow Petterson:I think it's going to help a lot. I think at the moment, there are so many different paths towards a very extensible platform that the web assembly work that is being done it's a little bit too immature, I think for a lot of people to be willing to use it, but it's getting there. There's also a proposal around adding better support for Sego. So you could ride a shelter that like call send through a Sego API into a real goal binary, as opposed to like goal web assembly extension. There's a lot of interest in making the platform more extensible and so I think [Inaudible 18:40]will help quite a bit and if I'm select the space of like native extensions, but there are other things which I think will help a lot as well, which is like web assembly, in particular, I think that that has a lot of potential. The Sego one is also very interesting. [18:54] Justin: I'm looking at a gid hub Envoy project and I see over 700 developers contributing code and patching and so how do you guys maintain the time utilisation all this takes, the efforts and the time to navigate between the community users, developers and I believe even without that, you would have the roadmap and the utilities already set out for the next upcoming years. I mean, Envoy is its own roadmap, I mean, let me put it this way, when someone comes to a live project and Envoy is definitely one of those who right now is super cool, super popular taking over cloud vendors. We had the projected Joshi from Google last week and we just talked about how Envoy was actually embedded within Google cloud products and we know Azure and Microsoft Azure under AWS simply do the same. So, envoy has its own roadmap and even without our community involvement, it will have its own tasks, projects, priorities, features upcoming and so on. Now we come in with our own ideas, some of them matching what is already on the roadmap, some really cool ideas, some of them less cool, probably. How do you guys prioritize, manage, maintain these and found a balance between all of this? How many people within the community, if I say, how many of you guys are actually maintaining and monitoring and moderating all these processes? [20:37] Snow Petterson:We have maybe 10 or 12, something around that, maintainers. We're weekly on call rotation, where every week there is a new maintainer who's on call who will be responsible for triaging issues and PRs assigning them to a review room and I think that works reasonably well. It's always tricky because sometimes there's very hard questions to ask and we don't know the answer and that requires investigation time but a lot of them it's about finding the right person to tag on the issues and kind of understanding who the domain expert on the different ports are and just knowing who might have opinions of things. And there's a prioritizing the work, that's always tricky because as you can imagine, most people working on it, generally have their plates full. Smaller things can often be, you kind of just get it done. Like if somebody asks for like a very small addition and I know exactly how to do it, I can go easily do it right there and then there's sort of a PR, but a lot of things are much larger and then a lot of the time we're kind of reliant on somebody from the community being able to step up and take ownership over mending it, or somebody from like a maintainer or whatnot that has some company internal motivation for getting it done. That tends to be very helpful. [22:03]Where for example, Google did a lot of work in order to reduce the footprint of stats because they had a lot of like issues internally, as I understand it with certain large deployments hitting memory issues. So for them it was easy to just kind of show up one day and say, hey, we're going to rework how the stats subsystem work in order to improve things and a lot of the work, we're seeing a lot of work now coming in from Google as well on implementing quick and because that's in their interest. If somebody shows up and they have a feature request around quick, where they might want to, whatever it is, something that might not be on the immediate roadmap, given that there's already interest in, there's a lot of people in the community who are working on it, it's a lot easier to prioritize that kind of work, but there's not a lot of like requests around stuff that none of the maintainers have a strong desire to work on either via their company or personal that you basically fall reliant on some reporter or somebody else stepping up and saying, yeah, I'd like to implement this. [23:12] Justin: So I hate to like go back a little, but I'm really interested in contrib. The reason I found it was I subscribed to the Envoy mailing list and I saw the Google doc base submitted and you see excitement in the Google doc. So I believe that this will just be a new chapter for Envoy and because we did have Anna Brandenburg on and she's very pro Envoy and in also running it very lean without too many extensions. So this whole new way of using extensions that might not be Google would never install and to their infrastructure, but many others might, what do you anticipate on extensions that will be coming out that can't make it to core? What is it that people want that they can't get right now? [24:12] Snow Petterson:Small extensions have typically been a lot easier to get in. So, if somebody wants an extension that does like a very small modification [Inaudible 24:22]as long as it's somewhat generic, that it's okay. So I think the bigger thing will be like larger filters and extension. So, a great example here, I think are like protocol parsers that will like generate stats and was like statutory ones and also like ones that can give routing. So various protocols that aren't currently supported, I said, this is because I've reviewed some of the PRS to add support for other protocols. I helped align the support for suite keepers that generation, that's like a three, 4,000 line PR, which is fairly hard to get in because it definitely requires aligning interests between maintainers and contributors without requiring this maintainer sponsorship. I think it will be a lot easier for people to add in support for parsing data of various other protocols. So I think that that's probably a big one because I definitely seen a lot of issues where people asking about support for some protocols that I'd never heard about. Stuff in like Telecom and all those things where they want it to be able to have some kind of like understanding of the protocol and without a sponsor that's been really hard to get in. [25:31] Justin: Are you talking like there are four protocols or higher ones? [25:36] Snow Petterson:I think they're all higher. Yes, I think they run on top of TCP. I don't think the room for extensions and implementing layer four protocols is a bit tricky, I think just because we're kind of where they essentially lay today, but I'm sure there are people who will be interested in doing that too. [25:52] Justin: But they can basically implement that say as a GCP filter, right? [25:57] Snow Petterson:Yeah, exactly. [25:58] Justin: On top of the GCP? [25:59] Snow Petterson:Yes, like that's how the zookeeper filter is implemented. It's a TCP field networks filter that sits before the TSP proxy. So all it does is inspect the byte as it flows through a proxy and parses the protocol, which then allows us to generate stats based on which commands are used, which I think it's mainly around which commands are used before it gets passed to the TCP proxy, which just does a standard TCP proxy off the data. [26:31] Justin: None of this is considered site car loading? This is just... [26:36] Snow Petterson:Yes, you're beefing up your site car. [26:38] Tzury: Okay. Yes, I like that. What was the, I would say the most obscure, unexpected use of Envoy in production that you came across that you'd say, oh, we never imagined that people will use Envoy that way or for this purpose, then you just, oh, that's awesome, look to people's imagination? [26:58] Snow Petterson:I think there are some cases I've heard of them running it inside of vehicles. I don't think it's fully autonomous, but as part of some system, I forget the details, but I've definitely heard cases of distributed systems being operated within some kind of vehicles. I forget the details, I'm sorry. [27:17] Justin: All good. Like autonomous, like in the car. [27:20] Snow Petterson:Yes. But if you really think about it, it's also not that weird because I'm sure they used to have some microservices running and they need to connect them somehow. So they're probably running Kubernetes too, I don't know. [27:32] Justin: Like a raspberry pie stack. [27:34] Tzury: Well, why would you have microservices inside your Tesla? How many services are running within Tesla, for example? [27:41] Snow Petterson: [Inaudible 27:42]I'm sure at some point you got the same problems in that space too or like you want one team to work on these components and they want to be able to have a nice contract so that they can make changes to their system without having to affect the other ones. So then yes, now you're talking about API contracts between components and you're not too far away from the microservice architecture. Maybe you're not going all over the internet but... [28:09] Justin: They are probably using portable data center, I would say, mini data center, right? Interesting. So if I'm asking you a like over the years, your roles in Envoy, how much of your time is actually has to do with writing new code versus previewing others versus answering emails and file or responding to issues on gid hub, which is accrual into emails and communication in general? [28:41] Snow Petterson:There was a period of time around the time when I started being a maintainer and like a bit before it, when I was writing a lot of code, just because again, I think it aligned very well with what my company needed at the time. Now, like over the time I've just gotten review ownership or more and more code and being brought into more and more like, hey, you know how this works so can you chime in? So I definitely drifted away more towards the side of just communication. It's always nice to get some code written every now and then, but there's so much other stuff that happens that I always have to be careful about making myself the blocker for code landing. [29:21] Justin: Looking at your gid hub profile, 72% of your time is on code review, 12% on commits and 16% on poll requests. So, you're doing quite a review. [29:33] Snow Petterson:Yes, no, there's I think just for my first time, I like six months like a review at work and I pulled my status and I think I had like three or 400 reviews, which is the number of times I like hit submit review or whatever over six months. So it's quite a bit. [29:51] Tzury: What are the golden rules for code review that you would share with the public when you review a code? What are the do's and the don'ts? [30:01] Snow Petterson:One really important part is making sure that, assume that the person that wrote the code has good intentions and that they're doing their best. So be respectful in that way and make sure that you're not, if they do something wrong, tell them gently. There's no reason to be mean. So tell them gently and explain what's happening and ask questions and don't be too arrogant. I think I see a trend somewhere where people say you see comments, like this doesn't make any sense at all, why are you doing it like this? And sometimes they're wrong because sometimes it's because the thing that they did is perfectly reasonable if they just like miss the context because even if you're the supposed expert doesn't mean everything. So I'll do much more of a cautious approach where if there's something that doesn't make sense to me, I'll ask them why this was done and try to understand it and kind of present it like that. I think having that kind of, showing that kind of respect for the people who wrote the code is very helpful because it helps the PR you're working on to proceed smoothly. But it also means that they're more likely to come back because they've had a good experience. So I think just we're creating like a very inclusive environment. Let's see, there's a running thing and Envoy PR reviews where the maintainers and 99% of the time always like includes the word things when they approve a PR, always have to show your appreciation. [31:30] Justin: No, it goes a long way. It really does because it's really hard to get into someone's head on the other side and you'd be like, are they mad at me? Are they annoyed? So yes, that definitely goes a long way. [31:41] Snow Petterson:I thought this was super helpful when I was like ramping up on the project as well and also seeing sometimes a PR drags on for a long time and also including like a, hey, thanks for iterating on this, this will be great. Just making sure that people are encouraged to keep working on it because yes, like you said, it's hard to communicate your own state of mind to get up issues or PRS. So like including like a very clear like, hey, no, I'm very happy that you're doing this work. I think it's very helpful. [32:15] Justin: Definitely. I mean like the emoticons, those are cool, but sometimes people just like to do a thumbs down and that's just like a burn, but I think overall it definitely goes a long way, no doubt about it. It's very important that people do communicate their gratitude and it would actually be a really interesting college thesis to see which projects, what the language is like, thanks versus not thanks, how healthy is the project in terms of adoption? That's not, I don't know, if we have a listener that wants to do that that'd be really interesting and I'm sure Snow, it would help you out. Anyway, it was so great having you on. We're going to do a little after show after, but before we go on to that next phase, how can people find you online? Is it Twitter? Is it gid hub? Where can people find Snow? [33:10] Snow Petterson:I'm on Twitter, I think, what am I @snowypeas? And my gid hub is snow peas, there are some links there to find me as well. [33:20] Justin: Yes, it will be in those show notes for sure and is there anyone in the Envoy community specifically in the Envoy community that you'd like to give a shout out to so they can be like, oh my God, I was mentioned on the podcast? [33:31] Snow Petterson:The entire Envoy community, thank you. Thank you for everything. [33:35] Justin: Awesome [33:36] Outro: Listeners, I hope you enjoyed this one. Do tune in next time, we're really excited about our line-up of guests. We have super exciting guests next week as well. Check out the show notes for this podcast at podcast.curiefense.io. That's C U R I E F E N S E podcast.curiefense.io for the community to cloud native podcast. Thanks again for listening, tune in next week, catch you later. Special Guest: Snow Pettersen.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer Guest Dan Lorenc Software Engineering Lead, Google Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about the confluence of Cloud Native and Open Source. Today, we are very excited to have as our guest, Dan Lorenc, who is a Staff Software Engineer and the lead for Google’s Open Source Security Team. Also, he founded projects like Minikube, Skaffold, TektonCD, and Sigstore. Dan will take us back to how he got into open source, Google, Cloud, and how he ended up being a lead for Google’s Open Source Security Team. We learn more about one of the bigger attacks that happened when Codecov Bash Unloader got compromised, what SGET is, what Google is doing to stop dependency nightmares, zombie dependencies, vectors, and why people should not sign Git Commits. Dan has written several blog posts and he talks more about some of them, and he shares some tips on the easiest way to get your security up if you are using cloud providers for working on open source projects. Download this episode now to find out much more from Dan! [00:01:53 (https://podcast.curiefense.io/20?t=113)] Dan tells us how he got into open source, Google, Cloud, and how he ended up being a lead for the Open Source Security Team. He tells us about his first open source project called Minikube. [00:05:07 (https://podcast.curiefense.io/20?t=307)] Justin brings up the safer curl URL pipe to bash which has been a topic on Hacker News. We learn more about the attack that happened earlier this year when Codecov bash installer got compromised and Dan explains more about that. Dan goes in-depth about what SGET is. [00:11:04 (https://podcast.curiefense.io/20?t=664)] Richard asks Dan if he thinks it’s important that people sign their Git commits and he talks about a blog post he wrote a couple of weeks ago about this. [00:12:40 (https://podcast.curiefense.io/20?t=724)] Dan explains how we can deal with security with stuff in the cloud and he tells us one of the biggest concerns he has right now. [00:15:12 (https://podcast.curiefense.io/20?t=912)] Find out more about the security leads across Google, and he tells us about an amazing paper that he recommends reading called “Reflections on Trusting Trust” by Ken Thompson. [00:17:23 (https://podcast.curiefense.io/20?t=1043)] Some people at the PSF got a $300,000 grant for supply chain security and Justin asks Dan if he had a role in that. Also, Justin mentions the reports going to Congress and the powerful XKCD graphic. [00:19:57 (https://podcast.curiefense.io/20?t=1197)] Learn what Google is doing to stop dependency nightmares, zombie dependencies, and vectors hitting that area. Also, Richard wonders if you can know as a cloud user what the dependencies actually are that you’re able to be exploited by. [00:26:54 (https://podcast.curiefense.io/20?t=1614)] Richard wonders how Dan stays sane, and how does he decide what to work on next. Also, Dan wrote a blog post called, “Procrastination Driven Development” and he describes how this all works in his brain. [00:31:07 (https://podcast.curiefense.io/20?t=1867)] One thing Justin wants to know is what repository or what package manager keeps Dan up at night. He wonders if there are any out there that need attention, or are they getting the attention that they need. [00:33:30 (https://podcast.curiefense.io/20?t=2010)] Find out where you can follow Dan on the internet and also some great tips to get your security up if you are using cloud providers at the moment for working on open source projects. Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Curiefense Blog (https://www.curiefense.io/blog) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) jdorfman@curiefense.io (mailto:jdorfman@curiefense.io) podcast@curiefense.io (mailto:podcast@curiefense.io) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?lang=en) Dan Lorenc Twitter (https://twitter.com/lorenc_dan) Dan Lorenc Website (https://dlorenc.medium.com/) “Codecov Bash Uploader Dev Tool Compromised in Supply Chain Hack” By Ryan Naraine (Security Week) (https://www.securityweek.com/codecov-bash-uploader-dev-tool-compromised-supply-chain-hack) SGET (https://sget.org/) “Should You Sign Git Commits?” By Dan Lorenc (https://dlorenc.medium.com/should-you-sign-git-commits-f068b07e1b1f) “Reflections on Trusting Trust” By Ken Thompson (https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf) “Securing Open Source Software at the Source” By Ashwin Ramaswami (https://www.plaintextgroup.com/reports/securing-open-source-software-at-the-source) “Zombie Dependencies” By Dan Lorenc (https://dlorenc.medium.com/zombie-dependencies-77c34740a7a8) “The Dependency Jungle” By Dan Lorenc (https://medium.com/swlh/the-dependency-jungle-841bd1c7bce0) “Procrastination Driven Development” By Dan Lorenc (https://dlorenc.medium.com/procrastination-driven-development-56f0aa17c0d4) “Open Source is Under Attack-Dan Lorenc (YouTube) (https://www.youtube.com/watch?v=ejcrUDaH1ZQ) Dependency - XKCD #2347 (https://twitter.com/jdorfman/status/1417605268708331524) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript [00:01] Dan Lorenc: Open source is starting to become a worry because the tigers are starting to attack it. It was known about these attacks and supply chain attacks in general for decades, right? They go back to at least 1984 I think when Ken Thompson published this amazing paper called reflections on trusting trust, he pranked a bunch of his coworkers at Bell Labs by putting a backdoor into a compiler, that back door was so smart that it would insert backdoors into everything it compile. His coworkers were very smart though. So they know how to disassemble these binaries and [00:27 inaudible], but his backdoor was so good that it also inserted a backdoor into all the disassembling tools. So it would hide the back doors when his coworkers looked at it. So he really baffled everybody and kind of showed that unless, you know, the tools that built all of the tools that built the tools you built all the way down, it's hard to build up trust and software at all. [00:44] Richard: Hello and welcome to Committing to Cloud Native, the podcast where we talk about the confluence of cloud native and open source. Very excited to introduce our guest today. Before I introduce him, I want to make sure we get to the other panelists on this episode. I am, of course, Richard, Littauer the man without the plan. And then we have Justin Dorfman, the man with the Dorf, Justin, how you doing? [01:08] Justin: I'm doing great. I'm really excited to talk to Dan. [01:12] Richard: Me too, Dan, as Justin just said, is our guest. This is Dan Lorenc. He is calling today from Austin, Texas. Dan is a staff software engineer and the lead for Google's open source security team. He looks very secure where he is. I can see lots of, no he's just a normal guy in a t-shirt. So don't be overwhelmed by how awesome his title is. Dan, how are you doing today? [01:36] Dan Lorenc: I'm having a great day. Thanks for having me on. [01:38] Richard: Is it really hot in Austin right now? It's really hot everywhere else. [01:41] Dan Lorenc: Yeah, it's pretty warm. It's not as hot as it can get here in the summer, so don't want to brag too much, but yeah, we're warm. [01:48] Richard: Pretty good. Now you live in Austin, but you've been in the cloud space for eight years. Tell us how you got where you are. How did you get into open source and Google and cloud? [01:58] Dan Lorenc: So I've been at Google for about nine years now, in the cloud space for pretty much the entire time since before it was called cloud. My first shop at Google was on the app engine team when that was kind of the only product Google had in the cloud area. If you're not familiar with it. It was a kind of platform as a service product before any of those buzzwords existed. So cloud kind of popped up around it, and around me as I was working on kind of this developer tooling, developer experience area. I got more into open source probably right around when Kubernetes and containers started to pick up. I started playing around with Docker pre 1.0, I remember all the old glory days there and got into the Kubernetes ecosystem right when that started to take off. So it's been a fun ride since. [02:38] Richard: So how do you end up being a lead for the open source security team? If you're first in a space, someone has to hire you or something, right? So like how did that happen? [02:48] Dan Lorenc: Yeah. So in the open source, in the supply chain security area, and it was a case of just kind of being really worried about it before anybody else was, I guess. And actually, yeah, that kind of connects back to how I got into open source. When I started at google, I was doing some work on our cloud platform and stuff, but from the inside and Google’s got a pretty different development process than a lot of other companies. Everything's in a big mono repo. And you've probably heard some stories about that. When you step outside of that mono repo and start building things, open source and all the tooling disappears. You're not using these same internal build systems and everything. And it was kind of scary at first. [03:20] My first open source project was mini cube. I started that project. I don't remember how many years ago now, but that was kind of the standard way to get Kubernetes set up on a laptop. And I was building it on my laptop, publishing this binary on github just from my laptop, that people were just taking and running as route on their computers all over the world. And it was kind of terrifying compared to, you know, the internal stuff I was used to. And they're making that face now, but that's still how most people are doing development. [03:43] Richard: When you say people. How many people are we talking? [03:47] Dan Lorenc: Most of them and it's scary. And so, yeah, I started to worry about that, looked around, not a lot of better options in open source. So I've been worried about supply chain security there for awhile, got really serious about it. Maybe two or three years ago. And still it wasn't a big topic. Maybe about a year ago other people starting to worry, the attacks started to pick up, people started to get scared other than just me, I stopped looking like a crazy person. It was a sign by the side of a road saying the end is coming. And then, you know, solar winds and some of the other big attacks happened earlier this year. And now all of a sudden, you say, why haven't we cared about this for longer? [04:20] Justin: Now you win. First they think you're crazy and now you win. [04:25] Dan Lorenc: Yeah. If you're too crazy for long enough, eventually people will believe you. Is that the motto? [04:29] Justin: Let me just tell Richard something. So Richard, I reach out to Dan and I'm like, Hey, I really want you on the podcast, he responds with, oh, here, talk to this person. No, I want you on the podcast because I've looked at your background with minicube and scaffold and six store. I was just blown away. I was like, I didn't realize you were involved with all of that. So the one thing that we've been seeing a lot, especially with security is a safer curl URL pipe to bash. It's been a topic on hacker news. It's always been a thing. So can you tell me and the rest of the world about ESCA? Cause I just learned about it. [05:13] Dan Lorenc: The curl pipe to bash thing, it's been a meme on hacker news forever. People love to complain about it. There isn't really a lot better you can do in certain circumstances. I think it's also kind of misunderstood too. Some of the problems with it, it's not quite as bad as a lot of people say, but it is really bad in some other ways, I guess. And then I think there's some stuff we can do to improve it overall or we can start I guess. The curl pipe to bash minature, not meme kind of pattern, if you're unfamiliar with it, is a way to install a piece of software. You go to a website and it says, here's a little script. This will install a software for you, just hit this with curl and then execute it on your system. And it will do some smart stuff and install our code. [05:45] People love to do that as developers because it's pretty easy to set up. You set it up once, you can have a bash that's pretty smart. You don't have to worry about bundling for all 37 different Linux distributions for match, getting into Homebrew and ports and the Mac app store and windows executable and all the different platforms and everything like that. You can kind of just write one bash script [06:02 exists]. The problem though, is that there's not as much security there. There's kind of different security constraints when you're installing software that way. At least now for the most part, people are doing, curl of an HTTPS URL, pipe to bash that wasn't always the case, you know, four or five years ago, before [06:17 inaudible] encrypt took off. And so that was really one of the biggest problems for doing that, then you don't even know what script you're getting or if somebody's kind of man in the middle and modifying the script before it gets to you. [06:26] But if you do have a script that serve from a pretty good website and there's SSL set up, curl pipe to bash isn't the worst thing in the world. What it is missing though, is a lot of benefits that package managers can give you or a lot of the protections that package managers write to the developers too, which is kind of one of the parts that's misconstrued a bit. When you're app get install a piece of software and debbian say curl for example. You're not taking curl from the curl maintainers’ repo, you're not getting it from github.com/curl. You're getting a fork of curl or the WN maintainers have forked curl and they have promised to apply security patches within a timely manner for a set period of time. That's just how debbian works. The tradeoff there comes from though, is that they're going to be applying patches and they're not going to be taking new features in, it's really just security patches for that time until there's a new debbian release. And then they will bump the version of curl to pick up all the new features and then apply patches again, going forward. [07:18] So there's really no option in the middle for people today. You either get something maintained by a group of maintainers or you take it directly from upstream. If you get it directly from somebody upstream, then who knows how you're installing it and all that stuff, upstream maintainers don't have signing keys. Tampering can happen even in the places twhere hey publish stuff. So of the attacks that we've seen there, one of the big ones earlier this year was when code coves bash installer got compromised. Are you both familiar with that one? [07:42] Richard: Our listeners may not be. [07:43] Justin: Yeah, exactly. Go more into it. [07:45] Dan Lorenc: Code cove is a popular code coverage standing and they wanted to make it really easy to install this tool and scrape your code coverage metrics as part of CI, so you can track it over time. Just like Linux distributions there are a hundred different CI systems out there. And so they package this installer up in a bash scripts basically. You could curl this bash script. It would install a little reporter in you CI system. And that's all you had to do is really one line. So awesome developer experience. And it's used all over the world. Unfortunately, they had some credential leak happen, which can happen to anybody. It probably does happen to tons of people and somebody found those credentials and use them to tamper with the script that was served to users. [08:21] And nobody noticed for months, they did some of the best practices they were supposed to do, like publishing hashes for this script and telling people how to verify them. But nobody was doing that basically. Easy way was too easy and people didn't follow the instructions for how to do it correctly. So it took months before anybody noticed this script which was supposed to be uploading code coverage was actually stealing all the secrets in your CI system and exporting them to some IP and we still don't know exactly where they were going. So that was one case where the script was downloaded from the correct URL. They publish hashes, people didn't check those hashes and something in the middle got screwed up. So we thought, how could we make this a little bit easier? And one of the ideas is this new ESCA tool, which I think you were just talking about, right, Justin? [08:58] Justin: Correct? [08:59] Dan Lorenc: Yes, so this is a tool we read started as part of the safe star project to try to make the easy way to download stuff. Also the safe/secure way. So safe get or secure get or something like that is where the name came from. The problem with code cove that they ran into really is that if you're going to distribute those hashes for the file you're downloading, you have to put them somewhere and you can't put them right next to the binary or a script you're distributing because then whoever gets the credentials to change that script can just go and change those hashes. So they put them somewhere else, which is the right thing to do. They put them back on the github repo, but that also means nobody's going to go check them because something else have to do. It's another URL you have to figure out and go learn. [09:36] So the idea we came up with is to use OCI registries. If you're not familiar with the term OCI, it's the same thing. It's like a Docker registry. OCI is the name of the protocol that Docker and everybody's standardized. Now the cool thing about an OCI registry is that you get a shot 2, 5, 6, built into the URL directly. So it's a content addressable API. When somebody uploads a script, you get a URL with the digest built right in and all the tooling enforces that it's correct and all the servers enforces that's correct automatically. So if you put a script there rather than a random S3 bucket or GCs bucket or something like that, and it can't really be tampered with from the time you give somebody a link to that thing. So it makes the digest checking built-in and automatic and easy. [10:16] Richard: It's like Sri and it's like IPFS, right? It's like basically the same, like content addressable. How many people have to share that link before that's not like a vector, right? Cause if I share a link just with you and then everyone in this podcast know that it's me and you then surely they could just come to our house with hammers and then they have the link. [10:32] Dan Lorenc: Yeah. So, you know, you still have to get that link to people correctly the first time. And then there's some other stuff you can do there. But yeah, so if you're handing people a link, you get that shop built and you can check it. The ESCA tool will do that. And there's some other cool stuff the ESCA tool can do though, like help you with digital signatures, which are also pretty hard to do and are in pain today. You do want to sign your contents with a physical key, it's got a bunch of YubiKeys here and there are [10:53 inaudible] people can't see the video, but yeah, if you don't have a YubiKey, are little thumb drive size things, you can plug into your computer, they can get some secret credentials on there that you can sign stuff with. And unless you lose that device, people can't really forge those signatures. So we've got integration for all of that built in too. [11:08] Richard: Do you think it's important that people sign their commits? [11:10] Dan Lorenc: I do not think signing commits on github and there's so subtlety here. I do not think sign commits on GitHub actually adds a ton of value. [11:18] Richard: Why not? [11:19] Dan Lorenc: I wrote a blog post a couple weeks ago about this, it's a similar issue to the one about publishing a digest for the content you're going to download right next to that file. And github when you sign a commit, the way signatures work in general, you have a private key and a public key, and you give everybody your public key beside stuff with the private one, and then they can check that signature against the public key. With github you just have to log in with your password and you upload whatever public keys you want for your account. And if you sign a commit correctly, github gives you [11:46 inaudible]. So if you're trying to protect against people compromising your password for your account, by signing commits, it doesn't really do much cause anybody that compromises your account can also just change the public key on there. And so you'll still get that green badge. So it's kind of protected by the same password that's protecting the account in the first place, [12:03] Richard: But that's the same logic as Google protecting my password because anyone could just hack into Google and then my password isn't protected. So is that the argument you're making here? [12:10] Dan Lorenc: [inaudible] a signing doesn't do much on top of the password. So if you're trusting the password then signing is kind of a false sense of security I guess. There are ways you can make it more secure and get yourself some value, but that really comes down to using something else to distribute that public key on top of just github. You've got to come up with some other system, like you could tweet it, you could post it on Keybase, you could do a lot of these things. But for the most part, people just look at that little green badge on github and assume it's more secure when you get that little green icon saying it's verified, which is sort of the problematic aspect of it. [12:41] Richard: Yeah. That makes a lot of sense to me. One of the questions I have that comes right off the bad is, we are on the committing to cloud native podcast. So we're all about cloud native here. How do you deal with security with stuff in the cloud when like tweeting your private keys doesn't make a ton of sense and then you're depending upon Amazon's ability to keep those keys sacred. What do I do basically? [13:03] Dan Lorenc: Yeah there are a couple aspects to that I guess. I think one of the biggest concerns at least that I have right now in cloud native security is Docker images in general. They're huge, there is tons of stuff in there. And for the most part, it's impossible to figure out what's in them. With our techniques, you know, you can start signing your images now. We've got a bunch of tooling and [13:21 inaudible] start to help with that. But even if you do that, you're just signing this big opaque blob for the most part that you're losing visibility into. So if you do a Docker build, you might have a curl pipe to bash inside of Docker file and nobody will know by the time you hand them that Docker image. [13:37] There's a whole bunch of case studies that have been done. You know, these things go stale, you build an image it's immutable, which is great, but it's immutable, which means it never updates. And so you have to build a new one to update it. And then, so there's a whole bunch of problems that I think will be pretty easy to solve eventually, once we start taking it seriously, but aren't quite solved yet. If you look in your Kubernetes cluster, there's going to be hundreds, thousands of images in a big one. And in each of those, you're probably going to find 20 or 30 vulnerabilities when you start running them through scanners and they start to add up quickly. [14:03] Richard: But it's your job to take things seriously early. So you're already taking it seriously. So I know you already have a fix for this. What is it? [14:10] Dan Lorenc: It's kind of another philosophical change people have to start to make in the way they build software. You know, going back to that minicube example I talked about earlier, I was building these minicube binaries on my laptop and handing them to people. Eventually we graduated to building them in a build system that was just, check-ins, sitting on a Mac mini under my desk in the office. You wouldn't run production infrastructure on a bunch of Mac committees sitting under people's desks but for some reason, we're okay with building software on them and then putting that software right into our production environment. I think [14:37inaudible] shifts we have to make, and we saw this with the solar winds attack in general, is that you need to start treating your build environment as seriously or more seriously than the production environment you're going to deploy into. [14:47] You would go build a huge fence and then slap a cheap block in front of it, that doesn't really make any sense. But the build system really is the doorway into our production environment. So that's not, at least secure the environment that you're going to go put that stuff into, then it just becomes a natural attack point and attackers are starting to figure that out. [15:05] Richard: So not to go into vectors of attack for Google, but to totally go into vectors of attack for Google, you are the open source security lead. There's also probably a security lead and there's also probably a physical security lead. Do you three all meet up in like a dark room with cigars and figure out like what to do and where to put locks? [15:26] Dan Lorenc: There are a lot of security leaders across Google. We take security pretty seriously. There are thousands of people that work on it. Open source has started to become a worry before and because attackers are starting to attack it, open source now is under attack. It was known about these attacks and supply chain attacks in general for decades. They go back to at least 1984. I think when Ken Thompson published this amazing paper calls reflections on trusting trust. If you haven't read this or watched any videos about it, you should really look it up. It's amazing. He pranked a bunch of his coworkers at Bell Labs basically, which was full of very smart people by putting a back door into a compiler, that back door was so smart that it would insert backdoors into everything it compile. [16:04] His co-workers were very smart though so they know how to disassemble these binaries and [16:06inaudible], but his backdoor was so good that it also inserted a backdoor into all the disassembling tools. So it would hide the back doors when his coworkers looked at it. So he really baffled everybody and kind of showed that unless you know the tools that built all of the tools that built the tools you built all the way down, and it's hard to build up trust in software at all. But for some reason we forgot about it after the eighties, until the last couple of years. The best answer I've gotten to that one. I don't have a definitive answer. I haven't sat down all the attackers and asked them why they've only started inserting vulnerabilities and open source software in the last couple of years. [16:35] Richard: Dude that sounds like you should do that. Why haven't you done that? Come on. [16:39] Dan Lorenc: Let's get the next episode of the podcast. The best answer I've gotten is that we finally gotten so good at locking down all the other ways and the software security in general was terrible. You need to go and hack the compiler to do these cool tricks. Even in the last decade security has gotten so much better in general, that these supply chain attacks and using open source as a supply chain vector haven't really become easier on their own. They've just become relatively easier because we're using password managers and two factor [17:03 auth] and HTTPS and all of these other things that we should have been doing this whole time. But now that we are doing those, the supply chain attacks are becoming relatively easier. And they're also more insidious in a lot of ways, [17:15] Justin: After solar winds, it seems like all of a sudden your position is just like totally validated. With that said, we know some folks at the PSF who got that $300,000 grant for supply chain security. What was your role in that if you had one or how was it? [17:36] Dan Lorenc: The PSF as actually gotten a couple of grants lately. One came from Google. I think they got another one is from Bloomberg. A bunch of organizations are starting to take this seriously and fund this critical infrastructure. This one, I'm sure you've probably seen the XKCD. I can't remember the number now, but it's got this picture of all these crazy complex things built up, all modern digital infrastructure. And then it shows one person holding up one tiny corner of it. And it says one maintainer maintaining this [18:00 inaudible] in Nebraska by themselves for the last 20 years. I think PSF runs some critical infrastructure to our industry, which is the Python package index warehouse and all of that stuff. [18:10] I think a lot of huge companies that take critical production dependencies on this don't realize how underfunded, understaffed a lot of these things are. And so we've been trying to find a lot of those across the industry that people are relying on without realizing it and get the visibility and get them funding that they need to better maintain and secure and support this infrastructure. Those grants are also going to do some other cool stuff like, you know, integrated vulnerability scanning into the Python tool chains and make it easier to understand what your dependencies are across different requirements dot TXT files. [18:40] Justin: I got to say I think probably the most prominent non code contribution to open source in the past fiscal year is that XKCD graphic with the Nebraska. [18:53] Dan Lorenc: [Inaudible] [18:54] Justin: Yeah Nebraska, like I'm seeing it in reports that are going to Congress, like it's out of this world. So we'll put it in the show notes, but it's just a powerful graphic. [19:05] Richard: I just keep wondering who it is. I mean, I know the [19:07 crosstalk] lives in Kansas City, so [19:08inaudible] is in Kansas City. That's the closest I can get to Nebraska, but who is a developer in Nebraska? There's gotta be one. There's gotta be one somewhere down the dependency chain. I want to find that person and thank you.] [19:19] Dan Lorenc: Reach out from Nebraska, wherever you are. [19:22] Richard: Sosupply chain checks are the equivalent of taking a couple of horses and a few pistols and hitting up a stage coach as it goes between bank one to bank C across Nebraska, back in the old days, right? That's like let's hit it here. In open source that kind of works by hitting the dependency tree and dependencies are really complex things that are really easy to hack. Justin and I have interviewed Dominic Tara before on our other podcast Sustain where we talk about these sort of issues a lot. And he was the owner of left pad, which had a whole thing happened to it. And that was when we realized, oh crap, what's going on with our dependencies. Dan, what is Google doing to stop dependency nightmare, zombie dependencies, vectors hitting that area. [20:04] Dan Lorenc: So open source is a critical piece of most software supply chains today. Nobody's running software without open source coming in at some point. And so that's why we're starting to see an attack to the combination of how widely used it is. You know, left pad was a huge wakeup call there. How many different dependencies show up in your transitive tree that you might only install one in a node application, but then that installs 10 and they each install 25, that each installing another 30 and the commonatory start to blow up. And so if you look at each one of those as a potential attack vector and attack point, you start to get pretty scared. I don't remember the exact left pad story. I think it was just deleted. It wasn't actually attacked or something like that. It was deleted or untagged or yanked from NPM or something. [20:43] Richard: Someone asked Dominic Tara to be the maintainer. He's like, yeah, I don't need to maintain this thing. I live in a sailboat I'm done and gave it to him. And then that person then deleted it. And then everyone's like crap, all of our builts broke. [20:55] Dan Lorenc: Right, right. Yeah. So it was attacked. There was no malware inserted, no Bitcoin mining, you know, it could have been much, much worse, but it was a big wake up call to everybody that had something depending on that, that didn't know they were depending on that. So there have been some other scary studies showing that, you know, the transit of reach of some of these packages on NPM is huge. If you can get like, kind of make up the stats somewhere, I think it was in a Sona type report. But if you can get the credentials for 10 people on NPM, then that's like 80% of all packages. The fan out ratio is pretty huge. So it's pretty scary that way. What Google is trying to do is just come up with some best practices, visibility and insights around that. So you can actually see how large a transitive dependency tree is. [21:33] You know, we can't just magically go in and wave a wand and get every project and start reducing these things or cutting off their dependency tree or taking fewer dependencies or fixing all of the code. But it's a cultural shift again, just like the rest of it. And so a lot of it is around showing, visibility how important it is to know, and not just blindly add things to your gomod file or your a package locked out chase on. Just because somebody wrote a library, it doesn't mean it's going to be better for you in the long run than finding a smaller one, assuming it's a little bit more work for yourself. We're not the only ones in the industry working on this stuff, you know, dependent [22:04 bought it] github has been a huge help to people just to kind of automate and make that update process easier than sending people smaller changes constantly is way better tactic than only updating once a year and having the resolve dependency conflicts for the next three months. I mean, we're seeing huge projects like Kubernetes in the cloud native space to actually start to actively go and reduce their dependency tree, which is awesome to see. [22:26] Richard: So that's an interesting point about Kubernetes, because in the cloud what I'm often doing is having someone else run code that I don't actually see the dependencies. I just say, you go run this code. So can you even know as a cloud user, what the dependencies actually are that you're able to be exported by? [22:42] Not all of them and that's a big part of it. If you're importing something directly, then maybe you can look in its dependency pile and see everything all the way down. But if you're just grabbing a binary or using a SAS product or something over the service, then no, you can't. And that's pretty scary in a lot of cases. The Kubernetes one is also, you know, there's kind of two sides to it. Kubernetes is this huge application with thousands of its own dependencies. People also import Kubernetes. Almost anything in the cloud native space you can reuse as code from Kubernetes directly. [23:10] And so by actively reducing their dependencies, Kubernetes is giving themselves a much better footprint and reducing their attack factor. But it also unbox everybody up and down the dependency tree from starting to be able to reduce these things. And that's why this is so hard to kind of prevent and mitigate overall, right. You're kind of at the mercy of your downstream dependencies and you might not have merged rights and no one was merged rights anymore. We see this a lot with zombie dependencies we were talking about. [23:34] A project could be super well-maintained until it's not anymore and somebody, you know, just moves on and deletes their github, or it leaves it there without responding to email and that kind of thing, it's impossible to those. Cause you know, a really good stable project doesn't get a lot of work, which is awesome. And those are the types of things you should depend on because they're not breaking constantly, but that looks from the outside exactly like a project that has 35 CVs inside of it where the maintainer just stopped responding. So it's kind of hard to tease those apart in a lot of cases. [24:01] Justin: I think maybe it is you bring a good point about dependency bought because I get pings on just by personal repos, like my website for instance. And I update them because it's very simple to do that. And I think maybe another thing github/Microsoft should be looking into preventing these zombie-type repos and dependencies, is once the project gets to a certain size, whether it's downloads or stars or whatever, is to have a successor or to convert that into an organization. I think that would probably be the next step to combat this whole dependency hell security vulnerability. So I think, you know, some people that github and Microsoft, I mean, you can make this happen. [24:51] Dan Lorenc: Yeah. I haven't actually heard, I thought of it like two seconds before you said it, the way you let up that sentence. I had never thought of like converting it into an organization idea before. [25:01] Justin: I forgot who said, I think it was Andrew from libraries.io, who suggested that, but I'm not sure. I just know they ran a study of most dependencies in the Ruby community. They just did Ruby because that's their world was one to two maintainers under a personal account. So that's where they kind of got that idea. But sorry, I didn't mean to cut you off. [25:25] Dan Lorenc: Yeah, that's a great idea. It's a subtle difference and so if you've never played around with GitHub that much, you might not understand the meaning there, but a lot of these things start out under a personal github repository, something like github.com/myusernames/myrepo. I mean, there are certain permissions you can do in a personal repo. Like you can have other maintainers, but there's still only one admin. And it's the person whose name that's under. When you convert something to an organization, you usually give it a different name, but the overall pattern looks the same. So it's github.com/some organization which could look like you user name or not. And then that repo and the organization can have multiple admins that are all kind of equal. And if one steps down and somebody else can kind of, you know, they're also now an admin and you can add other people and configure billing and do all of that cool stuff. So it's a really easy way to increase the bus factor I guess for a lot of these [26:10 stripped] the direction that one goes. [26:13] Richard: Having worked in a lot of organizations, I wish that were true. But most of the time like the lack of clarity in the actual like organization itself can lead to that being more of a bus factor, not less. I know there are projects working on this, like clearly defined. Clearly defined has helped funded by Google as well as like AWS. And they're trying to basically build a giant database of all dependencies that are like legal or not using SPDX licenses. I don't know if they're already doing stuff like security. They say they're on the website. I forget like whether or not they have say a list of this many maintainers. And this many vulnerabilities. At some point that would be a thing that's going to have to be done at scale, which just takes a lot of effort. I have kind of a different question if you don't mind, which is, this is all horrifying. This is really scary. I'm scared, right? Like everything is going to break. Everything probably broke yesterday. I just don't know it yet. How do you stay sane? Because it sounds like if everything is an order of magnitude as an issue, how do you decide what to work on next? And I know you've written a blog post called procrastination driven development. So this was a bit of a lead on question, but I'm just curious. Can you describe how that works in your brain? [27:25] Dan Lorenc: Yeah. I mean, it's a great time to be in this field. There are so many problems, you know, all of it is impactful, but a lot of people get frustrated or turned away and they say, you know, there's no magic bullet here. There's not one thing we can do that's going to fix it all. But that means there's a hundred things that we have to go do and we've got to do them all. And so there's plenty of room, plenty of opportunity for people to help out and get involved. I think one of the biggest areas, like you talked about having data and visibility and knowing where all of these dependencies are coming from, that's kind of where I'm focusing now most of my time at the sixth floor project, because I think that's kind of a prereq for a lot of the other analysis that we can start doing. [27:57] The supply chains right now are you have one, whether you know it or not, your code is coming from somewhere, that code is coming from somewhere, but you probably can't write it down today. Like if I asked you, what are all of your dependencies, you might be able to write those down. And I say, what are all those dependencies? Probably can't write those down. You know, getting that transitive tree out there. There's another problem though, which is where even if you can start to write those down, especially in the Docker in cloud native world, you might think this container came from that container, which was built in the system, which came from this github repo. [28:24] But none of that is verifiable today. All of that metadata is, you know, it's just best effort. You're guessing that a Docker image is built from the repo that the person said they built it from and pushed it from. And so we're trying to focus on there is verifiable supply chain metadata, but it doesn't mean it's good or bad. We're not telling you the repo is safe and that the maintainer isn't putting malware and, you know, Bitcoin miners in it. What we're trying to do is build up a supply chain of metadata that we can trust. And prove that if something came from this repo, it actually did come from that repo. The repo might still have malware in it. That's a separate problem. It might still have a whole bunch of known vulnerabilities, but at least we know it came from that repo. [28:57] And then we can start to do the same for maintainers and see who the maintainers are. And if there's only one and then we know that's a problem, but a lot of the data today, you just can't trust the data at all to even go and start making a lot of these informed decisions. So that's where I'm starting but because I think it makes it harder to attack supply chains and because it starts to unlock a lot of these other types of things you can do when you have all this data. [29:17] Richard: Going back to Ken Thompson earlier, he installed backdoors to make it easy for him to show that securities were bad. Can't we just admit that there's always going to be Bitcoin miners working in my dependency tree and just install a back door to have the Bitcoin go to me instead of them like, surely that's the better fix. Just admit it's going to be bad. Make sure the badness goes to you. [29:37] Dan Lorenc: Yeah attack, the attackers. That's a good one. I think in a of ways the Bitcoin mining and mining stuff has been a nice occurrence and a [29:43 boot] in a lot of ways. It's a disruptive way I can think of as an attacker making a profit off of getting into somebodies infrastructure. It compared to the ransomware attacks we've seen over the last couple of weeks. I think most people would take Bitcoin mining, which is just burning a CPU over, you know, losing customer data or losing access to your service. So in a lot of ways it's nice that this has happened a little bit and nobody wants to have coin miners running on their infrastructure, you know, github even had to go and do a whole bunch of work to reduce quotas and everything then benefit with CI was harder to use for open source projects, but because people were abusing it with clean mining. So it goes both ways. I think we're going to keep seeing it as long as it's that easy to get arbitrary code execution in people's projects. But I'll take that over a bunch of the other attacks we've seen any day. [30:25] Richard: Yeah. I think what's scary is that this isn't really fun and games at the end of the day. Like it's interesting, but people have died because people have hacked hospitals and demand of ransomware. People have not gotten their chemo because they couldn't get into their systems and didn't know their data. And that's really hard to deal with. That's super stark. I'm glad people like you are out there doing this work. How do you stay sane? I mean, your hair is pretty massive, but like, how do you deal with that pressure? [30:49] Dan Lorenc: Pressure really, you know, if I don't do anything, then it stays this bad. You know, it's not getting worse. I'm trying to avoid getting one of those Nebraska scenarios, holding everything up myself. And I see it as a whole bunch of small wins we can make every day. So it's kinda nice and motivating that way. [31:03] Richard: I like that answer. Awesome. [31:05] Justin: One thing I want to know is what repository or what package manager keeps you up at night? You know, you've got so many different package managers, that's touching pretty much every developer in some way or another. Are there any out there that are like, oh wow, these need attention, or are they getting the attention they need? [31:28] Dan Lorenc: I think [31:28 PIP] was, I don't want to say it's in a bad state, right? I don't want to blame people. It was something that people didn't realize needed funding over the last year. And I think, you know, through a bunch of different grants and stuff that have gotten us into a much better shape, which is great. I think from a package manager perspective, I think I start to worry more about the kind of OS and distro level ones. The things like Appget, the things like young DMF and the fedora ecosystem. Most of those were built for actual machines that would install things and update things, pin things and downgrade things. [31:59] And they're built in a different era where the keys are distributed in a different way and all this stuff. So they don't support a lot of the declarative installation mechanisms like you can start to do now in PIP and NPM where you can install fixed versions of things. You lose a lot of metadata when you're doing those old ways and they're at the very bottom of most people's container images. So attacking one of those I think would be really hard to spot, really hard to recover from and pretty wide reaching. Thankfully they're run by professionals and most cases people that take security really seriously, but those are the ones that I do really worry about the most. [32:32] Justin: Do you think using something along the lines of six store or upgrading their infrastructure or their backend to support today's standards of security is something that could be easy to implement or is it just, they're going to have to like think from the ground up again? [32:50] Dan Lorenc: I hope so. Six store is only a couple months old at this point, we are seeing people start to use it kind of in parallel with package managers, like their maintainers of WP packages that are signing individual packages with some of our tools. And I hope we get to a point where people can start to use it for those lower level package managers. They've spent years working on their infrastructure, which is great and to make it easy ao people don't have to do that again for the next generation of package managers and everything. [33:15] Richard: So we're running up on time. So before we head out and I just want to say, thank you so much, but you have so much good advice and you're very eloquent. I think not only in how you talk, but also in how you write, where can people find you on the internet? [33:30] Dan Lorenc: Yeah. My medium page is where most of the stuff I write ends up D L O R E N C.medium.com. I think I should get you all of that stuff. [33:39] Richard: And if you had like a couple of tips for anyone out there, who's using cloud providers at the moment for working for open source projects, what would you say is the easiest way to get their security up a tiny bit? [33:49] Dan Lorenc: Turn on dependabot and start updating your dependencies as quickly as possible and that one's kind of controversial. I'm going to say it anyway. I think, yeah, update as frequently as possible. Take the small version bumps when you can, because it means when there is a CV found later, it's way easier to update your way out of it than dealing with a year or two's worth of backlog of updating things. That goes against that, if it's not broken, don't fix it philosophy, which I get. I think when you weigh it out, though, just take the updates and roll with them as they come. That's a big one. And then just pay attention to what's in your dependency tree, write it down. Somehow every language, every package manager is a little bit different, but useful lock files. Don't let left pads break you, that kind of thing and vendor stuff where you can use proxies if you have to, depending on the ecosystem, but yeah, insulate yourself as much as possible while taking the updates as fast as possible. [34:35] Richard: Thank you so much, Dan. It's been a really pleasure to talk to you. I really appreciate you coming on the podcast, everyone that was Dan Lorenc, if you have any questions for him, he has a medium article and he may respond to comments. If not, well, I don't know what to do. Maybe his twitter. [34:48] Dan Lorenc: Yeah. Twitter works too @lorenc_dan. [34:52] Richard: Got it. Thanks. [34:53] Justin: Thanks. Special Guest: Dan Lorenc.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer | Tzury Bar Yochay Guest Prajakta Joshi Group Product Manager, Edge Cloud for Enterprise and Telecom Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the Cloud Native space. Today we have as our guest, Prajakta Joshi, who is a Group Product Manager at Google driving Edge Cloud for Enterprise and Telecom. She joined Google in 2015, has been working with Tzury since 2016, and really knows all about the history of Cloud Native and how it really started. Prajakta tells us about her background and how she ended up in the Edge Cloud space. We also find out where open source fits in, revenue streams going towards the open source ecosystem to make it more sustainable and more useful to Enterprise and Telco users, and more on the evolution of server less gRPC service mesh. We also hear Tzury’s really interesting idea of the future Cloud, and Prajakta shares her perspective on how she handles the good and the bad and where the focus should be. Download this episode now to learn much more! [00:02:10 (https://podcast.curiefense.io/19?t=130)] Prajakta tells us about her background and how she keyed into Telcos. [00:04:27 (https://podcast.curiefense.io/19?t=267)] We learn more about the Edge Cloud product, how Prajakta ended up there, and how she manages consistency. [00:10:37 (https://podcast.curiefense.io/19?t=637)] Prajakta mentions Kubernetes and Richard asks where open source fits in. [00:15:12 (https://podcast.curiefense.io/19?t=912)] Richard asks Prajakta if she has any thoughts about revenue streams going towards the open source ecosystem and how that would work. [00:20:33 (https://podcast.curiefense.io/19?t=1233)] Prajakta explains more about how we have revenue streams going from Enterprise and Telcos back into the open source projects to make the entire system more sustainable and ultimately more useful to Enterprise and Telco users. [00:25:27 (https://podcast.curiefense.io/19?t=1527)] Tzury wonders instead of having so much manual labor, he thinks the real future Cloud would be fully automated cloud, bottom up from the infrastructure level itself and Prajakta tells us what she thinks about this. [00:28:31 (https://podcast.curiefense.io/19?t=1711)] Prajakta elaborates more on the evolution of server less gRPC service mesh. [00:34:51 (https://podcast.curiefense.io/19?t=2091)] Tzury wonders what the oldest service is in Google that Prajakta knows of that is still running the same way it was running at the beginning, and it was not migrated to any fancy schmancy new tech that she can share with us. [00:38:06 (https://podcast.curiefense.io/19?t=2286)] Find out about the two parts of service mesh and what the traffic director does. [00:39:08 (https://podcast.curiefense.io/19?t=2348)] Tzury asks how Prajakta how it feels being the greatest on one end and still the underdog on another. Also, how does she deal with this frustration or excitement and affect her day to day. [00:44:27 (https://podcast.curiefense.io/19?t=2667)] Find out where you can follow Prajakta on the web. Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Curiefense Blog (https://www.curiefense.io/blog) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) jdorfman@curiefense.io (mailto:jdorfman@curiefense.io) podcast@curiefense.io (mailto:podcast@curiefense.io) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?lang=en) Prajakta Joshi Linkedin (https://www.linkedin.com/in/prajaktasjoshi/) Prajakta Joshi Twitter (https://twitter.com/prajaktaplus) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript [00:00] Justin: Hey, it's Justin co-host of this podcast. We just released Curiefence version 1.4, which includes support for NGINX. It has UI improvements, security improvements, and much, much more. So just go to curiefence.io/blog to see what else we improved. Now enjoy the show. [00:21] Prajakta: When I started off as well. Like a lot of the focus was on, okay, what's the next cool tech we built. And slowly as you start building it, you start realizing like for example, service mesh, it doesn't solve all problems for customers. You really need to bring other bits and pieces and you need to keep evolving these technologies. And I think the thing, the pivot that I made was okay, let's not start from the tech. Let's start from the problem we are trying to solve, or the new experience we are trying to deliver or whatever business value you're trying to build and then build backwards, or integrate the stuff that exists. I think that is sort of the thing nobody really talks about. [00:58] Richard: I love that perspective. That's brilliant, everyone that is Prajakta Joshi. She has joined us today from Google and she is so on top of her game that she immediately launched into one of the most insightful things we've had on this podcast. So welcome to the Committing to Cloud Native Podcast. This is the podcast where we talk about the confluence of cloud native and open source, how we get them together, how we build great things. Today, I'm joined by Justin Dorfman as usual Tzury Bar Yochay . Hi, as usual. Thank you so much for joining us. Both of you, other hosts. I'm Richard Littauer and we have our guest today is Prajakta Joshi. Prajakta Joshi is a group product manager at Google, driving edge cloud for enterprise and telecomm. She joined Google in 2015. Been working with Tzury since around 2016, and really knows all about the history of cloud native and how it really started. So Prajakta, you were just saying that what's happened already is that we just start building stuff, but we're not stepping back and saying, okay, who we're building this up for? How can we solve the problem? And what is the problem in the first place? Can you expound a bit more about your background? Because I know that you don't just think in terms of business needs and in terms of like the community, but you also are actually really keyed into telcos. Can explain how that happened? [02:24] Prajakta: So a little bit about my background, I started off as an engineer. At the time, you know, this was in the early two-thousands, the problems for point problems, if you will, where somebody has an application and they need to scale it up because they have this massive growth of users. And at the time it was just load balancing technology. So I started working on that and then along the way, as different customer problems started cropping up, there was technology like CDN that I built out and perimeter based security and so on and so forth. And then slowly what happened was customer workloads themselves stop being in a data center or in a single place. And they started proliferating and appearing wherever they showed. For example, somebody could not migrate all at once to cloud and they were sitting on premises and slowly the kind of technology that you needed to solve these problems got more and more complex. [03:18] You needed multiple pieces to go solve a customer problem. And then, you know, often like you brought up the case of telcos. For cloud providers, our first set of stakeholders that we solve for enterprises like your retailers and your financial institutions and healthcare and so on and so forth. In the last few years, you've seen a lot more interest of telcos to go adopt those same cloud native paradigms. So, which is where the telcos came in the mix. And I think one of the best things about cloud native is that it is breaking down very traditional silos that have existed, say between enterprise and cloud or how we solve for enterprises versus how we solve for telcos. And a lot of what I'm building or doing right now is about building common technology that can solve for each of these stakeholders in a consistent way. At the same time, we have to meet them where they are. So not all traffic of telcos is going to be HTTP HTTPS, and we have to build for that, just as an example. It has been an evolution. Even on my part, I went from building products to actually starting with the solution and building products backwards. And I'm still on the journey of learning how to do that as well. [04:27] Richard: So I'm new to the cloud native space. And so I'm not entirely sure what edge cloud is in particular. Can you describe what that product is and how you ended up there? [04:38] Prajakta: Let's talk a little bit about edge. When you say edge different people will probably have a different version of what edge is and a really simple way to think of edge is, it's everywhere. It's like we have this massive global distributed edge. And so the edge, if you're on premises in your data center, that's an edge. If you really think of it that way, or if you're actually sitting in a retail store, or if you're even, even when you have a cell phone, the device actually could be an edge. And then obviously you've got other edges. Like we have [05:08 our pops], the telcos have their network edge. We have third-party providers who have their edges. So I think very simply when we say edge cloud in Google cloud, what we really mean is, you know, you bring your workloads to the cloud. You need to have similar constructs and similar technologies and similar paradigms. [05:28] Even if you don't deliver your workload in cloud. If you need it on premises, that's where we bring cloud to you. Or if you need it in an oil and gas location, that's where we bring it to you. Or if you need it in your data center, that's where we bring it to you. I think of it more like distributed cloud. This is a term that [05:43 Gartner] introduced. It's an insightful term because it is really about bringing cloud to wherever your workload needs to be, as opposed to forcing the workload to come to cloud. And so that's basically edge cloud. You bring Google cloud and you bring the related technologies paradigms. And then at the end of the day, you map these to the use cases and business value that you want to deliver to the customers. [06:06] Richard: So you're describing sort of hybrid uses as well, not just that in data centers, but also to end-users. How do you manage consistency while doing this? [06:16] Parajaka: I think that is one of the most interesting and challenging problems to solve. Now let's talk about all of the things right in the mix. So one is, you've got the customer premises. The second is you've got Google cloud. The third is you have our edge locations. The fourth is you have also other cloud providers in the mix because a lot of our customers are also multicloud. Managing consistency across this is why you see a lot of these cloud native technologies spring up in the first place. So for example, when you say manage consistency, as an end user, or as an enterprise, or even as a telco, what are you trying to do? You want to go and spin up your compute, whether it's containers, whether it's VMs, whatever you need in the same way. You want to apply policies in a same way and as much as possible. [07:04] And this is still an evolving space, more at the service level than the infrastructure level. So instead of saying, apply my firewall here to this VM and that container, and then put these other hundred services. You should be able to see service A can talk to service B. It's a much easier thing for all of us and the admins to go grow. The third part of it is observability. So now you've deployed these workloads in all of these different locations, whether it's edge, whether it's your data center average, you need to be able to see what's happening to the traffic end to end. So I think observability, which includes monitoring automation and so on and so forth. I think that is [07:41 inaudible] we talk about consistency. [07:44] And I think the last bit is I sort of alluded to that in observability, but it is automation, the more everything as code. So like configuration as code, security policies as code, that lends itself to being automated well. And the more you can get humans out of the loop, the lesser the errors and so on and so forth. So a lot of the consistency also is related to almost ways to make this thing automated. And now you ask how to do it? That is basically what all of us are trying to solve for it, which is why, again, I talk about the solution itself. Like what does consistency mean? If you look at somebody who's trying to go say migrate their workloads out of their data center, they may not be able to do it one shot. And so they take some of their key workloads or the ones that they can move fast. They move them to cloud. Some workloads may never move to cloud due to compliance and other reasons. [08:38] So now you've got this workload running in two places that is where not everything in the world is containerized or is going to be containerized. Like there are different things that benefit for containerization and some workloads will never containerize. So the consistently, like, which is why, you know, Tzury and I often talk about service mesh. The good thing about a paradigm like service mesh is it as agnostic to the type of compute, which means I can talk about services that are based on VMs and services that are based on containers exactly the same way. And that is an example of consistency. When I apply my policies, whether the workload is in my data center, in my cloud, if I can say enable MTLS between service C and service B, that is a level of consistency. If I can easily orchestrate traffic routing. So for example, I introduced a new version of code or of my service send 99% of the old versions and send 1% traffic to the new version*so I can test it. [09:37] If I can do it similarly, both in cloud and anywhere else, the workload is, other workloads are deployed. That is consistency. So there is not one thing that's going to deliver us consistency, but I think one overarching thing, or like just a tenet is the more you start upleveling where you apply policies and where you automate, the easier it is to make things consistent, which is why you see a lot more conversation around service mesh, or even previously Kubernetes and service mesh. And then to abstract out everything that's top of server lists and so on and so forth, but it is one of the most interesting and challenging problems to solve for. [10:15] Richard: Thank You. That's an excellent explanation. I love how you go into service meshes and how they're really important for consistency. One of the questions I have is that you're presenting this rosy-eyed vision of like corporate projects, large projects, enterprise, telcos, doing all of this work and providing use cases to all their customers. This is committing to cloud native. It's about open source too. You mentioned Kubernedes briefly, where does open source fit in? [10:39] Prajakta: You know, the way I have started looking at open source and the way I have started looking at even all of these technologies is what does it enable? If I'm an enterprise why should I care about open source? Like, what is the benefit to me, or if I bring in Kubernetes or service mesh, what is the benefit to me? So I'll give you an example of some of the interesting things that open source brings, and it's not open source for the sake of doing open source. So for example, like Istio, I'll just take the example of an open source service mesh or Kubernetes, which is an open source technology. First, the sheer number of different stakeholders who are building the technology generally leads to diverse inputs into building the technology. And so you will land up with a better product because you don't have a one-sided view of what the solution or this product should look like. [11:29] I think that's one of the biggest benefits of open source. And we have seen that in the case of Kubernetes, which has now become the de facto standard for container orchestration. I think the second big reason to really look at open source seriously, and you know, there is open source and then there's open interfaces. So for example, you had a guest Anna who spoke about this product, we call traffic director. Now when we build traffic director, which is a control plane, it's a service mesh control plane. And it communicates with a proxy called Envoy in the data plane, the interface between those two, we adhere to the open interfaces, which means tomorrow if you don't like traffic director, you can swap it out for any other implementation. So one of the good things about open source and open interfaces is it gives you a choice. [12:18] If I, as a customer, test something out and as long as I can swap it out for something else that preserves the same interface, it makes my life easy. And I can change my mind after I've deployed a technology. And I think the third thing is a lot of the open-source technology, especially in the cloud-native bucket, they are built for applications which need to be automated at scale. They are built for applications to scale inherently. Security is a first class citizen. It's not an afterthought where like I was involved in building some of the early load balancers in two thousands. Our consideration was not at all security to start with. It was like, okay, I have this hardware box. How many queries per second can it take? And how can it, how much can it scale out to the backend servers, then when the service started getting DDoS is when we were like, oh, we need to go add a DDoS defense and then so on and so forth. [13:12] But one of the newer paradigms, like you look at mesh or Kubernetes, they treat security as a first class citizen, and there is more and more awareness around embedding security traffic management for the developers to automate, write code, a separation of boundaries. Like a lot of these cloud native technologies, they separate out like, you know, service mesh is a very good example, like applications from say the networking logic. So I think that cleanliness and hygiene of interfaces is something that is seen more and more in the cloud native products. So I think overall that is the reason to adopt open source. And, you know, if an organization decided to build something in-house. If you look at Kubernetes, it came out of board, like that board was a technology in Google, but taking it to the community, enrich that whole product, like it became 10X of what it was at Google. [14:03] And I think you get the power of the community with open source. You're not going to be able to build such a big R&D team for every component you need. So I think the more you get plugged in, and one of the things, and this is specifically now from personal experience. The more you contribute back to open source, the more you can get out of it because you can then influence the direction. If you simply consume it, in some ways you benefit from what others are doing, but you never get to influence the roadmap. So it's just a two-way street, in some sense, like as you leverage open source as an enterprise or through our products, we, as well as the enterprises are contributing back in a bigger way, like, you'll see a lot of enterprises starting to contribute back to open source. [14:45] Richard: I love that answer. Brilliant. Couldn't agree more. I mean, open source is what I live and breathe, and that's just spot on. At the beginning of this podcast and this conversation, we mentioned telcos, and you've written in our notes doc here, which is an awesome document. It's basically good enough in itself to publish at this point about telcos trying to create new revenue streams, using cloud native technologies and opportunities. I'm curious if you have any thoughts about revenue streams going towards the open source ecosystem and how that would work? [15:16] Prajakta: So I have this very interesting and strange background where I keep switching between working with enterprises and telcos. And so I have also seen the thought process evolve over the last 10, 20 years. If you really look at what telcos are trying to do. If you look at maybe more recently, they essentially have three things to solve for, one is they have their own IT workloads. So these workloads are very similar to, you know, enterprise IT workloads where they need. For example, they're big users of our data and analytics product. So they need compute and they need storage and they have applications that serve their own ID. The second big chunk of almost use cases or products that the telcos have, is their core network, that's sort of there, you know, that's their lifeline. That's what actually drives their business. And these networks have existed way before any of us did. [16:13] I mean, if you look at how far Indian bell go back, like they are a hundred plus years ago. So I think the second part is the network. And then the third part is where, think of your cell phone bill, right? Even if I gave you 5g on a cell phone, you're only going to pay so much more for your bill. You're not going to pay 10 X of what you pay for your monthly. So you need to deliver new experiences or you need to go and solve for new use cases to generate revenue streams. And these are sort of the three things that are top of mind for telcos. Now, if you look at Telco IT the journey to becoming cloud native is very similar to enterprise because the IT workloads look like enterprise workloads. [16:54] With telco network. It's a different ballgame, in that a lot of these telco networks, for example, are VM based. Like, especially if you look at the 4g networks, a lot of these networks have existed for so long that making them cloud, we cannot just tell telcos go cloud native. It is not easy as that. We have to start maybe by deploying a greenfield side-by-side by converting some of these brownfields to greenfields and so on and so forth. And then I would say the most interesting bit, or at least the most interesting happening in recent times, is what we have done with telcos. So three, four years back, if you told people AT&T and Google cloud or AT&T and some of the other announcements or Google, and some of the other announcements we've done are happening. People wouldn't believe it because cloud providers and telcos did not partner so closely. What we have realized is there is a win-win to be created that with the beneficiaries being the enterprise customers. [17:51] So for example, take the case of a retail store. With COVID everybody's budgets are slashed. The retailer wants to go deliver these amazing experiences where they need to bring you back to the stores. So now you enter the store and you find your app at a mannequin and it chops the look for you, and it gives you recommendations and so on and so forth to deliver this you need to run AI models, compute and so on and so forth. The retailer says I have no budget to run it inside the store. So what we did is we partnered with a leading telco and we essentially use their network edge. We built out the edge compute and placed it there. And we run all of these models close to the retailer there. So this became a new revenue stream for the telco and us. And at the same time, it solved a very critical problem for the retailers. [18:39] So when you talk about these new revenue streams, it's generally about three things. You're either trying to make money for the end customer, or you're trying to save money for the end customer, or you're trying to deliver a new experience for the end customer, which helps them increase customer engagement as well as obviously make money eventually. So this is a very new and evolving partnership if you will, between telcos and cloud providers. And then it spans various industries like retail, finance, healthcare, you will see that these solutions are fairly vertical focused, because like I said before, like, what is the problem to solve right? In a retailer, they may be trying to go and deliver these experiences without CR CapEx, heavy, experienced, or CapEx, heavy investment, or somebody like a healthcare. Like if you take oil and gas, as an example, there are these remote oil and gas fields. [19:31] Like there is no edge computing or cloud nearby. There is really no 5g connectivity then how can we bring it there? So that is another set of problems we are solving with them. And a lot of this is great because the underlying technologies used here are for example, Kubernetes or service mesh or 5g and edge just happens to be wherever you need to run the workload. But the end result is it solves something important for the enterprise. And then this is where the telcos can make money because they're delivering a service, not just 5g, but an actual end solution that solves a business problem. So that is why I say, like, if you start from there and go back, that is where the value is. [20:13] Richard: That's very valuable for the end customers being the telcos and very valuable for enterprise. I try to ask a different question. Maybe I didn't ask clear enough, how do we have that money go back to open source itself? You mentioned we use Kubernedes to do this work, and that's great. It's wonderful to use the opensource ecosystem to leverage enterprise needs. What I'm curious, how do we have revenue streams going from enterprise and telcos back into the open source projects to make the entire system more sustainable and ultimately more useful to enterprise and telco users? [20:44] Prakakta: I think first of all developers. So instead of only leveraging open source, it is important that we put developers because those are actually the most expensive resource. If you think of it and high quality developers who will keep the code base clean, who will at the same time, be inclusive and accept different trains of thoughts and so on and so forth. I think that is the first biggest investment anybody can make. Google has one of the largest contingents of developers working on Kubernetes. That's how we contribute back to that community. You will also see telcos increasingly doing that. Some of it is also because if you really want to influence something, you have to put your skin in the game. Otherwise you're going to be a passive consumer of it, which works to a certain extent. But if you really want to move the needle, you have to put skin in the game. [21:33] And Google in general. I mean, it's nothing new. We've been committed to open source for a really long time. So I would say first as developers, like, you really need to put people who build the stuff. I think the second thing is the communities themselves. And you can see Kubernetes, for example, has a [telco21:48], just as an example, or they have these different stakes where essentially they are looking at it from a use case problem or a vertical problem, or like, who are we solving for lens? And that is where the money will come back. So as an example, let's say you want to go use Kubernetes to run the telco network. It's not perfected on telco networks. It's good enough, but it could be much better. And this is where telcos and us. We are investing time to go participate in these sites and then bring some of the use case ideas in the enhancements that are needed. [22:21] And then obviously put developers and money where we are asking for features. Like we need to make sure we are also contributing back developers. So we'll build them out. It's true of telcos as well. They are newer to the game, but if you look at enterprise, telcos us, there's a lot of participation. It's like, if you just look at the curve, it's a very, like, it's like a hockey stick in terms of participation in open source. I think those are the two biggest ways. And then one of the biggest things that help our technology evolve is real-world validation. I think the third thing is sort of facing back lessons learned into the community. It's not so much about monetization or money. It is about evolving the product. And so saying I used Kubernetes and here are the three features I'd like to see whether it's for enterprise or telco. I think all of us need to do more of that, but that really helps the product itself or the solution. [23:13] Richard: You know I have to agree with the developers aspect because we have a guest coming on next week, Dan Laurenec, he's a Googler as well. And he's working on supply chain security side. And I think when you brought up earlier that keeping the code clean and all these other things along those lines, it's very important because I mean, let's just say a hundred percent of the fortune 500 use open source in some kind of capacity. All of those would be vulnerable if we didn't have a supply chain security and very trained eyes looking at the code. And that is what Google and Microsoft and all the other Fang companies are hiring to kind of look at, but it doesn't end there. And it just needs to continually be invested in because each day millions of lines of code are being added to various, high used open source projects. [24:11] Tzury: Prajaktayou mentioned everything is code that principle. But I'm thinking if you work at Google cloud, the actual cloud, you don't use a cloud. You actually working in a dentist center, it's a building, they're hardware, servers, cables, routers, electricity, generators, backup, and all of that. So, first of all, we always need to remember that the fascinating tech that we call public cloud that's available for us, it's thanks to lots of people who are doing really hard work to make it so smooth, automated, and simply, you know, manipulated and managed by few lines of code, a Terraform that we just put together and all of a sudden you have superpower computing that you can calculate anything you want. [25:11] But that led me to think, and if I'm by mistake uncovering like Google X project or whatever secrets cut me off. So imagine that a code that will actually run army of robots that will simply keep building and expanding the cloud manufactory more storage and plugging in more compute and taking care of everything, instead of having so much manual labor. I think that the real future cloud would be fully automated cloud bottom-up from the infrastructure level itself. What do you think about this? [25:57] Prajakta: You should patent it before anybody gets to this. So that is an extremely interesting thought. Like I was just reading this article on Elon Musk, where he's staying in this small little house built by a company called Boxable. Just got, it's not even out as a product yet. And these are these modular units, like 3, 400 square feet with a kitchen and bed and everything built in there. And if you could actually put any of those together, like with some modifications to build a whole house. In general, if you really think of how we've built data centers, that is the way we have scaled as well, like there is a recipe of using basically just genetic cockpit, where the value is actually in the software. And as much as you can simplify and make your hardware consistent and start moving the value into software, the easier it is to build what you are saying. [26:51] When we have a pretty massive footprint of our regions and our edges and so on and so forth, that is how we have scale. Like it is really infrastructure in a box, it's just not visible to people, but that is how we scale. So when we bring up a new data center, there is essentially that same hardware that goes in, there are these massive software systems that actually bring the value. So to your point, it is true that some of this could be packaged as a product. Like you could literally have data center in a box, which you can make more consistent with the use of essentially, you know, like maybe commodity hardware or more generic hardware, which is quite open interfaces and put the value in the software and also modular software where you can plug and play things out, depending on, you know, what it is that you're trying to accomplish. [27:38] I still think though Tzury like, you know, imagine you launch this amazing product. The world is what it is. Like people will still have their data centers and people will have these new, amazing data centers and people will still have their edge locations on their devices. So some of the problem statements, you and I will still need to sit and solve, which is the world is just heterogeneous by design. And I think we still have to solve for that world. But to your point, this is basically what you're talking about is consistency at a solution level. Like instead of just making, you know, some genetic design for say servers and so on and so forth, you're saying can we actually package a whole data center or a region in a box? And I think that that's a great idea. We do it personally for our own data centers, but Tzury, I think that's your next product right there. [28:28] Tzury: Thank you. Probably we should talk a little bit more about the massive contribution if opensource was mentioned, massive contribution Google has made, is making to almost everything we call today, cloud native, almost everything that we use within the cloud native. And one of the, I would say the enigmatic terms that was mentioned by Anna and also popped in your notes, is the server less GRPC service mesh. So I'm thinking service mesh server less, GRPC. Can you elaborate about this a bit? [29:09] Prajakta: Let's maybe talk about why they came about. So it was sort of an evolution if you think of it. So GRPC came from a Google internal technology called stubby, like that was the basis of GRPC. Then we've got Kubernetes which came out of org, service mesh like we've been doing, we didn't call it service mesh, but that's been the construct used internally. And then obviously serverless. We are very strongly involved in the [29:34 inaudible] of other communities. So each of these, so let's maybe trace the evolution. It didn't all happen at once. The first problem Google really solved was about building load balancers at scale, which means when I worked on load balances, it was a box. And if you needed more horsepower, you bought a bigger box. That's how it works. They said, that's not the way to do load balancing. And they tried to do it. And it scale for search in Gmail and so on and so forth. [30:01] So they essentially put this commodity servers like humongous scale across the globe, and then they build software that could balance using those servers. And so they built this massive global distributed load balancing. So that was the first innovation if you will, that happen, then there was a problem of, you know, how do you efficiently do message to message communication. And that led to stubby and which eventually evolved into a more standardized GRPC. Then Kubernetes came out of being able to schedule jobs at scale, like that's where it started. And then obviously this predated containers and so on and so forth. And that evolved into orchestrating compute, which is like containers at scale. [30:42] So Kubernetes was about containers, but containers and Kubernetes definitely brought about the notion of a service, but it didn't necessarily have everything, which is why now you see there's a very strong correlation between Kubernetes and service mesh, but service mesh itself is just a paradigm if you think of it, right, what it's really saying is let me bring together some compute. It doesn't matter what it is, whether it's VMs, whether it's containers or so on and so forth. Let me bring a set of these together. Let's call this a service. And when I call this a service, all the things that I need to do for networking, whether it is figuring out how to route traffic or whether it is how to do like, you know, enforce MTLS, let me not put it in the application. Let me go put a separate proxy there and let me go and delegate that responsibility to the proxy. And the goal is to separate application from the networking. [31:36] And then this just gives you a service fabric and the brains of it as a service mesh control plane. To the service mesh control plane or the service mesh came about because you wanted to, or you as an end user, should be able to apply policies at the service level and in a very simple manner without having to touch your code. So I should be able to say, when service A goes to service B, like I said before, maybe send 99% to version A and 1% version B or enforce MTLS between all services or for every packet that goes out, go put it in a dashboard and let me see the end-to-end flow. That is why service mesh was built. It's sort of an overlay on top of Kubernetes. [32:15] Now there is one misconception people have about service mesh, that it's only for containerized workloads, actually, that is not true. It includes VMs. It should be able to work for serverless. It should be able to work for bare metal. It is that one very key misconception that we need to clear up because the world is all of this and a big chunk of the world is VM based. And some of it will always stay in VMs. So I think that is how service mesh came about. And then people said like, do I really need to know, not all people, but a subset of the end-user said, do I really need to see the guts of how this product is and so on and so forth or how the solution is? That's where serverless came about, like to abstract away all of this stuff away from the end-user, like as a developer, I want to run my application. Why do I need to care about these hundred things? So that is where serverless came about. [33:06] In a way people will use one or more of these. It's not like one thing will solve everything. Or some people may just use one of these technologies, but that was sort of our evolution path. Now, eventually the hope is that in all of this, you know, serverless as a paradigm is probably the [33:25 mental model] we need for most, which is if I can just state policies in terms of services, if I have to think as little as I can with networking, that is when this thing becomes easy for me to use like cloud and anybody's cloud. I just don't say Google cloud, but anybody's cloud, there is ample opportunity to actually simplify it, like we have not reached the simplified cloud yet. We still need to make that happen. So that is where technologies like serverless are extremely important. [33:54] And also being able to code, like I always spoke about use cases. So in terms of serverless, there's a variety of used cases, like you want to do processing at the edge, let's say, I want to like, you know, Tzury in your world, like in the world of security, if I want to do something to the traffic and it is very custom, I have multiple ways to do it. I could spin up a serverless or a function, I could actually leverage [34:18 inaudible] of a service mesh, or maybe I build out a site process, which the load balancer points to. So I think depending on the level of sophistication, the solution needs, or the level of sophistication the developers of the enterprise have, we have to give choice to customers. But again, like, you know, focusing back on what is the problem we are trying to solve and what is the best stool to go solve. It means that we will require all of these technologies. It's not like one makes the others obsolete. [34:49] Tzury: Talking about, you know, VM, et cetera. What is the oldest service in Google that you know, that's still running the same way it was running at the beginning and was now migrated to any fancy schmancy new tech that you can share with us. [35:06] Prajakta: You know, it is very interesting that this is how Google runs, like Google does run in containers. Google does have a service mesh. Google does do serverless. It's just that we didn't call those things that, like we called it Borg instead of Kubernetes. So in a way, if you can think of it, the services are running like they used to, because that is where we started from. And it was not that we set out to invent Kubernetes or anything like that. It was like there was a genuine problem of scale, where, how do you schedule something at scale, our own jobs at scale. And so in a way Tzury you can think that almost all services, obviously there's evolution, but they are running with similar paradigms from start to now, like in fact, this paradigms were invented to solve the problems we hit as we drove these services. So when you see our load balancers, it is how we run our own load balancing. Like that's how you scale your request for search, or when you look at containers, it's boring. So in some ways, every service is running the old way. If you can think of it that way. [36:07 ] Tzury: So is [36:08 inaudible] use Kubernetes, what is more in use between Google itself? [36:12] Prajakta: Yeah, it is basically the same set of people now who are doing both of those things. So it is basically the same tech. So like, for example, if you look at, I'll give you another example since Kubernetes, we spoke about it now. So if you look at say something like a service mesh, if you look at something like GRPC, I think the key thing to realize is we often mix the paradigm with the implementation. So Kubernetes, the implementation that is outside, that came from us opensourcing board. So it is one and the same thing. With service mesh, you have a lot of products in the industry. Like we ourselves have two very interesting products. One is a [36:50 steel] and the managed version, which is called ESM under service mesh. And then there's a slightly different variant called traffic director, which pulls in all of our global systems. [37:00] So in a way like a lot of these systems, at least the tech that we adopted came from what we do internally. So it is basically one in the same thing. Like traffic director uses the same systems we use for a bunch of internal Google services. The flip side of this I wanted to call out though, is we've also been careful to award the not invented here syndrome, which means, and I think you and Anna spoke extensively about Envoy proxy. When we started off, there was discussion on whether we should go build a proxy like that. And when there was an evaluation done by several people, like several key people at Google, the conclusion was that Lyft built out something, Lyft in my client, build out something really interesting. There is no reason for Google to reinvent it. So we're also being mindful that it's not just about our tech, it's about the best tech that we can use internally as well as enough cloud. So that's a good example of us using an external tech. So it is whatever is the most optimal way to do it. And then going and using that. [38:05] Tzury: So for example traffic director is built on top of Envoy. Is that something that you can share? [38:10] Prajakta: Yes so this mesh basically has two parts to it. There is essentially your data plane where you run the proxy, like next to your application, you run the Envoy proxy and then you have applications and you control the proxy through a brain, which is what traffic director is. So we do an and the protocol between the control plane for traffic director control plane, and the Envoy proxy is called like XDS. And now there's a new variant of it. We stick to the open-source variant of it. We don't make any modifications. We don't fall off, which means that we are supporting exactly the same industry standard protocol between the control plane and the data plane, and yes, like our key sort of deployments are all with Envoy proxy, the traffic director, which is the brain itself, uses a bunch of Google systems. But since it uses the open protocols to [38:58 inaudible] data pin, you could swap it out or swap it back in because the others also use the same protocol. [39:05] Tzury: How does it feel Prajakta being the greatest in one end and still the underdog on another? Let me explain what I mean, you and I and many people in this world know that the Google cloud actually from tech perspective has degraded so [39:23 inaudible]. Google cloud is by far the most sophisticated, intelligent, amazing, faster than any piece of technology, but still there are others more popular. I'm not even taking into account all the contribution and the birthplace of all those technology, such as GRPC, service mesh and Kubernetes and so on. To know that what you do is by far on top of the game. But from, I would say commercial side is still the underdog. How do you go on your day-to-day with this? Is it frustration, excitement? How does this affect your day-to-day? [40:14] Prajakta: That's an interesting question. So maybe I give a slightly different perspective. First let's talk about like, you know, as always, since this is focused on the end customers, I would say most of the world's workloads haven't yet moved to the cloud. Like that's actually the very sobering reality of things. The opportunity ahead of us is huge, like I would start from there. I think the second part of it is it is not truly about being number one or number two, or number three, the question is, are your customers coming and using the cloud. Like what are the problems you're really solving for them? Why are you the best suited to solve for them? I think two or three very simple things in there. We've had, especially if you've been tracking, since Thomas came on board, there's a very strong focus on solutions based approach to solving things. [41:06] So Tzury, when you asked me what, number one, or number two, or number three, it is almost [41:11 material] to this discussion when you put this big picture, like herere in front of you. The question is, where are the workloads? The workloads are in cloud? The workloads are on premises that folks are asking us to put, bring cloud to wherever they are, whether it's in their data centers, whether it's in their edges. A lot of our focus is on that. And it is super exciting because this is sort of the world we've been actually advocating for. Like, not just now, like since, you know, you've heard the anthers announcements, you've heard our work around edge cloud, you'll hear a lot more about it in the coming months, but you've seen all these announcements with telcos. I think it is that, it is very important. [41:50] You know, when you are running a marathon, if you take the point snapshot at the beginning, it's super not useful, like at the end who wins and the market is big enough. I think the right thing to focus on at this point of time is, are we really solving customer problems? And that is why some of the things we are focusing on. One is obviously to go and bring technology. And from the starting point of the customer, like, you know, now there's a new project called Lantos for VMs coming out. The reason for that is a lot of workloads are VM-based. We cannot ask people to containerize overnight. There's a good realization of that and solving for that. That's an example. [42:29] The second thing is a very good understanding of who your customers are. Our customers are enterprise and industry verticals. And we have a new set of customers with telcos. I think the third thing is to take the industry along, which means partnerships. We are spending a lot more time on partnerships like if you see our telco announcements, we are partnering with all kinds of ISB. We're partnering with telcos themselves. If you look at enterprise there is a huge amount of emphasis on that. And I think the last bit is take all of the cool tech we have, which you brought up as well, but it's not really about technology. Like what is the problem we are solving? Which means we have blueprints for various solutions. All our products work well together, there is simplicity in how you can configure them. There is visibility into what's happening when you deploy them. [43:17] I think those are sort of top of mind. So some of the things you brought up are not even crossing our minds because really the problems you have solved are just a tip of the iceberg. I think the opportunity lies ahead. And that is where basically we are focusing on and it is super exciting. Like daily, you wake up and there is a, you know, there's a new problem you hear from the customer or at the same time you also hear about, you know, how do you say, like the [43:41 inaudible] went down or essentially you help them deliver an experience which doubled their customer engagement. Those are the real wins actually. Those are the things to track. And like the rest will follow is how we look at it [43:55] Tzury: It is indeed***a marathon. And we barely even started. [44:00] Richard: I wish we could talk forever about this. I mean, this is like a marathon. Listen to this conversation I'm learning so much. It's really great to hear. I wish I could talk to you about Aurora, you founded for women and leadership in Google cloud. I wish we could talk about your work in the DEI committee there. I wish we could talk about the 12 Peyton's you have. Tzury when she said get a Peyton, she was serious. She was mentioning something from her own experience. Unfortunately, we do have a time limit on this podcast. Prajakta, for people who are interested in hearing more about you and your awesome work, where can they find you on the web? [44:31] Prajakta: I think the easiest way to find me is on LinkedIn. That's where I'm most active. I think for the rest, a lot of it is through the announcements that you see from Google cloud in the areas of edge, telco, some of the past products. I mean, those are great products to track as well, and from the cloud network insight and people can always reach out to me on LinkedIn and Twitter. That's the best place to find me. [44:54] Richard: What's your handle on Twitter? [44:56] Prajakta: It is Prajakta plus. [44:59] Richard: Thank you so much that Prajakta. It has been great having you on. Thank you so much, looking forward to having you on future podcasts, if we can manage that because you have so much to say, and it's just wonderful hearing about what you're doing there at Google. Thanks again. Thank [45:16] Prajakta: Thank you, Tzury, Justin and Richard. It's been a pleasure being here. [45:18] Tzury: Thank you Prajakta. [45:20] Justin: Thank you. Special Guest: Prajakta Joshi .

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer | Tzury Bar Yochay Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the Cloud Native space. This episode is a little different than others because we decided after publishing seventeen episodes this year, we want to take a trip down memory lane and look back at our most popular ones. Today, you will hear clips from our “Top Five” episodes which include the following guests, Kelsey Hightower, Les Jackson, Sergio Méndez, Chris Ferreira, and Richard Li. Sit back, relax, and enjoy! Oh, and go ahead and download this episode now! [00:01:05 (https://podcast.curiefense.io/18?t=65)] We start with Episode #8 which was our most popular downloaded episode. Our guest was Kelsey Hightower, who is Principal Engineer and Principal Staff Advocate at Google in the Google Cloud Platform Division. We learn how he ended up in the Cloud Native space, joining Google, and his job offer at NASA. Kelsey shares an abundance of information and we find out the amazing story behind “No Code.” [00:04:58 (https://podcast.curiefense.io/18?t=298)] Our next top episode was Episode #9 with Les Jackson, who is Developer Advocate at Marketplacer. Les explains his creative process and why he was interested in using Envoy to begin with to make microservices. We also hear his cool experience with writing his book, The Complete ASP.NET Core 3 API Tutorial, which he self-published versus going with a publisher. [00:09:41 (https://podcast.curiefense.io/18?t=581)] Episode #4 brought us a super fun guest, Sergio Méndez, who is an SRE, Professor, and CNCF Ambassador to Guatemala, as well as the organizer of the Cloud Native Guatemala Community Group and is working to get students and people from Central America involved into the CNCF ecosystem. Sergio is a “Linkerd Hero” and he is working on two contributions for Curiefense, which are Linkerd and Rancher, which we learn more about. [00:14:26 (https://podcast.curiefense.io/18?t=866)] Kubernetes is the topic of Episode #10, and we brought in Chris Ferreira, who is the Principal Engineer and Architect for the WebEx Platform at Cisco. Chris shares how he started out in culinary school and ended up in the IT world, working in startups on the front end and the back end, working for Microsoft, and now Cisco. He tells us about getting introduced into Kubernetes and becoming a code contributor to Istio in a pretty big way. [00:20:33 (https://podcast.curiefense.io/18?t=1233)] We end with Episode #7, with our guest Richard Li, who is the Co-Founder and CEO of Ambassador Labs, which builds popular open source tools for Kubernetes. We learn about the “Golden Rules” to building a successful open source company or project, and things you must do from the very beginning to keep the project interesting to the community and to get it out and reach out to those achievements. Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) jdorfman@curiefense.io (mailto:jdorfman@curiefense.io) podcast@curiefense.io (mailto:podcast@curiefense.io) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?lang=en) Committing To Cloud Native Podcast-Episode 8-Learning in Public with Kelsey Hightower (https://podcast.curiefense.io/8) Kelsey Hightower Twitter (https://twitter.com/kelseyhightower?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Committing To Cloud Native Podcast-Episode 9-Microservices are Interesting with Les Jackson (https://podcast.curiefense.io/9) Les Jackson Twitter (https://twitter.com/binarythistle?lang=en) The Complete ASP.NET Core 3 API Tutorial: Hands-On Building, Testing, and Deploying by Les Jackson (https://www.amazon.com/Complete-ASP-NET-Core-Tutorial-Hands-dp-1484262549/dp/1484262549/ref=mt_other?_encoding=UTF8&me=&qid=) Committing To Cloud Native Podcast-Episode 4-Sergio Méndez: SRE, Professor & CNCF Ambassador to Guatemala (https://podcast.curiefense.io/4) Sergio Méndez Twitter (https://twitter.com/sergioarmgpl) Committing To Cloud Native Podcast-Episode 10-Kubernetes, (Almost) Love at First Sight with Chris Ferreira (https://podcast.curiefense.io/10) Chris Ferreira Linkedin (https://www.linkedin.com/in/chferrei) Committing To Cloud Native Podcast-Episode 7-Building a Business Around Popular Open Source Tools for Kubernetes with Richard Li (https://podcast.curiefense.io/7) Richard Li Twitter (https://twitter.com/rdli) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Special Guests: Chris Ferreira, Kelsey Hightower, Les Jackson, Richard Li, and Sergio Méndez.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer | Tzury Bar Yochay Guest Anna Berenberg Google Cloud Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the Cloud Native space. Justin has some shirts and stickers that he wants to give away to fans of this show, so listen right now to find out how to get some! Today, we have a super exciting guest, Anna Berenberg, who is a Distinguished Engineer at Google. Anna goes in depth about her position at Google and what she does there. She tells us about the “Five nines” applications, how proxies are used on a day-to-day cloud basis at Google, and how security and reliability come together with what they’ve done at Google for Envoy. Also, Anna explains how listening and being attentive to people and customers who contribute to open source plays an important role. Download this episode now to learn so much more from Anna! [00:02:15 (https://podcast.curiefense.io/17?t=135)] Anna tells us what she does as a Distinguished Engineer, what her specialty is, and what Uber TL means. [00:03:30 (https://podcast.curiefense.io/17?t=210)] Richard reads some things from Anna’s Bio and she describes in depth what it all means. [00:05:35 (https://podcast.curiefense.io/17?t=335)] Justin asks if Google Search ever goes down because he’s never seen it down, and it seems very critical that it’s always up. Anna tells us all about the “Five nines" applications and she mentions a paper that was recently published called, “Deployment Archetypes for Cloud Applications.” [00:09:35 (https://podcast.curiefense.io/17?t=575)] Anna explains how she thinks about her role and what her goal is. [00:11:50 (https://podcast.curiefense.io/17?t=710)] Richard wonders how proxies are used on a day-to-day cloud basis at Google. [00:13:41 (https://podcast.curiefense.io/17?t=821)] Anna explains what proxyless means and how it works. She mentions a single general-purpose RPC infrastructure called Stubby that Google uses. [00:19:20 (https://podcast.curiefense.io/17?t=1160)] Tzury asks Anna how we can make software more secure, especially critical pieces of actually handling infrastructure, and Anna tells us how security and reliability come together and what they’ve done at Google for Envoy. [00:23:41 (https://podcast.curiefense.io/17?t=1421)] Tzury wonders what kind of things Anna is doing within Google or outside of Google, involving open source related products to make the entry point easier for newcomers and developers who come from different platforms and different technologies to adopt new technologies. [00:26:43 (https://podcast.curiefense.io/17?t=1603)] Speaking of extensions and platforms, Justin asks Anna when Google is going to adopt Curiefense. ☺ [00:29:17 (https://podcast.curiefense.io/17?t=1757)] Richard asks Anna how do they incentivize clients, communities, coders, developers, projects, and products like Curiefense to get involved in the planning stage, and what are her teams doing to make sure the needs of Curiefense and other projects, like our listeners may have, are taken into account at a very high level. [00:32:21 (https://podcast.curiefense.io/17?t=1941)] Find out where you can follow Anna online. Quotes [00:03:55 (https://podcast.curiefense.io/17?t=235)] “So, we had our own proprietary load balancing proxies and control planes.” [00:04:01 (https://podcast.curiefense.io/17?t=241)] “And then a time came when we realized that for Google Cloud to achieve its principle, Google Cloud considered itself to be Open Cloud where we embrace open source technologies, and we basically essentially can think about cloud without borders.” [00:04:33 (https://podcast.curiefense.io/17?t=273)] “And at the same time, Lyft came out with this new, brand new proxy called Envoy Proxy, which had an amazing architecture.” [00:05:42 (https://podcast.curiefense.io/17?t=342)] “Well, this is actually another passion of mine is a design of Five nines applications.” [00:11:04 (https://podcast.curiefense.io/17?t=664)] “People should be thinking about policies and not the infrastructure that allows them to propagate and allows them to enforce.” [00:12:13 (https://podcast.curiefense.io/17?t=733)] “So think about it as like a gateway, the place where I trust the mains meet together.” [00:14:05 (https://podcast.curiefense.io/17?t=845)] “And what we developed actually, a service mesh before service meshes were cool.” [00:15:09 (https://podcast.curiefense.io/17?t=909)] “So when I looked at Cloud and how important gRPC is to cloud because it allows for much better productivity and velocity of cloud developers when using Protobuf’s and how well it feeds with the Kubernetes as modernization.” [00:15:50 (https://podcast.curiefense.io/17?t=950)] “We actually reuse the same API’s that we are using for Envoy.” [00:16:46 (https://podcast.curiefense.io/17?t=1006)] “So, if you think about Gmail or something like this, some big application, or customers that big, that if you put proxy in each home, and you have hundreds of thousands of microservices, the daily job will be restarting proxies.” [00:27:11 (https://podcast.curiefense.io/17?t=1631)] “An interesting conversation to have is how Envoy as a platform can allow co-existence of a functionality, in some cases could be competing, in some cases complimentary.” [00:30:00 (https://podcast.curiefense.io/17?t=1800)] “Well, we’re always listening. I think listening is under appreciated activity and it’s important to listen and important this time, not a single requirement but a collection of requirements.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman) jdorfman@curiefense.io (mailto:jdorfman@curiefense.io) podcast@curiefense.io (mailto:podcast@curiefense.io) Richard Littauer Twitter (https://twitter.com/richlitt) Tzury Bar Yochay Twitter (https://twitter.com/tzury) Anna Berenberg Twitter (https://twitter.com/kniga) Anna Berenberg Linkedin (https://www.linkedin.com/in/annaberenberg/) “Deployment Archetypes for Cloud Applications” by Anna Berenberg, Brad Calder (https://arxiv.org/abs/2105.00560) gRPC (https://grpc.io/) “gRPC Motivation and Design Principles” By Louis Ryan (Blog post) (https://grpc.io/blog/principles/) Envoy (https://www.envoyproxy.io/) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/)

Transcript Justin Dorfman 00:00 Hi! it's Justin cohosts of this podcast, we got some new shirts and stickers and we really want to give them away to fans of the show. All we ask is you share one of your favorite episodes on Twitter and DM the link to at Cariefence and if you prefer email drop a line to podcast @curiefence.io. Thanks and I really hope you enjoy the show. Anna Berenberg 00:25 It's actually very interesting point about core versus extensions. Which brings us back to the question of reliability. The reason that the core is not kept figure limited is to ensure stability, reliability and security of the core and so let's say Google doesn't need all this extensions, then it can compile them out. It doesn't compile them in as core, and then they can guarantee quality. Now everybody needs extension [phonetic 00:56] right because Envoy is not just a proxy to platform. Richard Littauer 01:02 Hello, and welcome to "Committing to Cloud Native, the podcast where we talk about the consequences of open source and cloud native technology. We have a very exciting guest today from Google. But before I get around to introducing her, I want to make sure that the listeners know about the panelists, because our voices are going to come up and it's nice to know who we are. So I'm Richard Littower [phonetic 01:25]. I'm your host today with me is my co-panelists, Justin Dorfman. Justin, how are you? Justin Dorfman 01:31 I'm great, Richard, how are you? Richard Littauer 01:33 Always good and my other co-panelists, Tzury, how are you doing? Tzury Bar Yochay 01:38 I am great, Richard. How are you today? Richard Littauer 01:41 Also, still good, hasn't changed in the last 10 seconds. Thanks for asking. All right. Getting to our guest today. Our guest is a Distinguished Engineer at Google, who when I asked her to describe in more detail what that means use a lot of words that I would like you all to hear. So I will ask her soon. Again, we have Anna Baron Berg today calling from San Francisco. Anna, how are you doing? Anna Berenberg 02:04 I'm great. How are you all? Richard Littauer 02:06 I think we're all still good. Okay. All right. So when I asked you, how else I might introduce you, you said a lot of things about load balancing and so on. What is it that you do as a Distinguished Engineer? What is your specialty? Anna Berenberg 02:20 I am Uber TL for load balancing products and technologies at Google, I work to make sure that load balancing works for both Google products as well as Google Cloud product and I've been at Google for 15 years doing just that. Richard Littauer 02:39 So I have a silly question. 15 years is awesome, by the way, wow, I'm unfamiliar with the term Uber TL and that may just be me. But in cases, any of our listeners, can you describe what Uber TL means? Anna Berenberg 02:49 Well, there are very useful people as engineers who implement code, and then you have TLs, who actually guide them to build and design services and systems and then when the scope becomes too big for one TLs to cover, then you have multiple TLs and then we have Uber TLs, for work with the TLs. So like your article [phonetic 03:14] system, where Uber TLs now guide, and design and architect a whole area, and the families of products. Richard Littauer 03:24 Technical Lead, got it. Technically, what you've been doing at Google Cloud-- I mean, since 2015, you've had this job is you drove the modernization of Google Cloud-- Application Networking by embracing Immersion Cloud Native Technologies, using onboard proxy with traffic directors, Open Service Mesh and Universal Control Plane that powers all cloud applications, load balancing products, I just read that from your bio, which is why I was able to rattle it off so fast, can you describe in more depth what that all meant? Anna Berenberg 03:55 So we had our own proprietary load balancing proxies and control planes and then a time came when we realized that for Google Cloud to achieve its principle, Google Cloud considered itself to be open cloud where we embrace Open Source Technologies and we basically, essentially, you can think about cloud without borders because once you use Open Source, then everywhere where the customer runs, they actually can become part of this open ecosystem of Google Cloud and so with that, it became apparent that we need something different than just google proprietary product and at the same time, Lyft came out with this new brand new proxy called Envoy Proxy, which had an amazing architecture and I totally fell in love with that, to be honest, because it was such a beautifully design proxy and with that, I made a proposal and the proposal got approved that we actually rebate and build all our new products and rebates, existing ones on our wall as a proxy and it's been quite wonderful experience and ride that philosophy because it allows actually to embrace bigger planes that run pure Envoy on somewhere else, whether it's on premises on the cloud, and use our own control plane to manage them, as well as our managed LB products. Justin Dorfman 05:27 So, ever since I've been using Google's it's like 98, it's seems like it's never gone down and is that true? Like, does Google Search ever go down? I've never seen it down. It seems like it's very critical that it's always up. Anna Berenberg 05:42 Well, this is actually another passion of mine is a design of Five nines applications and what five nines means? It's basically never done. It's the applications that basically always up 24x7, and then how do you deal with that? What kind of architecture would you build and with that, Google embraced what's called Mobile Applications and that's why searches never down because it could be down somewhere? But the traffic management and load balancing and routing, and everything is bound around places, zones, or regions are basically everywhere, where they stack potential maybe unhealthy, then it works around it and serves it from other Healthy Places and the same thing inside of the application itself, the same technology is being used to make sure that you can always build around faulty hardware faulty software, everything can fail and then you still build an application to be able to survive it. Just out of interest, not for self-promotion. We published recently, a paper titled deployment archetypes for cloud application that talks specifically about all sorts of deployment and global deployment as part of it. Justin Dorfman 07:05 Thank you, because that makes a lot of sense and was your team responsible for building that out? Because it seems like you know a lot about that type of part of the search engine? Anna Berenberg 07:17 No, not my team but I build and work with teams then build infrastructure for the services, but the services become our customers and what we want to do is actually solve customer problems, and how would you solve customer problems unless you understand what problems they're facing and so we get to listen both the internal Google customers as well as external Google Cloud customers to understand what are their requirements? How do they want to build this Five nines applications? What is the compliance requirements that also comes with that? Yeah, so it's all about listening. Tzury Bar Yochay 07:56 I would say to me, Anna, your ESDN [phonetic 08:00], and you actually defining this software defined network, which people may not be aware of, because those tremendous trends and revolutionary takes 99% in the backend side of the story, at backbone of the infrastructure of the cloud, but Cloud ability, agility, scalability, and all superlative and we can attach the cloud is mainly available for the fact that data center, were used to be based on appliances and hardware are became what we call cloud, which is mainly based on software stock, performing, doing all the routing and all the great work of networking and switching and whatnot, firewalling, etc., etc., etc. The world is stepping towards these Software Defined Network, I believe, I don't know numbers or predictions, but they talk about ISPs, one day also transforming to that type of infrastructure, 5g and so on, and so forth and then we are talking to you while you're actually leading teams and doing design and defining, eventually the software defined network. So I wonder sometimes you get to think of your part in the evolution of the significant role you've been playing. It's such a humble way, all those years. Anna Berenberg 09:38 I'm thinking about my role in it that way. I think we live in a very interesting time period And like you said, we add the causal [phonetic 09:50] of SDN becoming requirement everywhere, because all the customers and all the users want to have policy based workflows, I would say. So, everything has to become policy base. How do you propagate policy? How do you get this policy enforced? How do you do it in uniform ways, so that the consumers don't have to think, 'Oh, I'm going to enforce one policy on a hope and another policy and another hope because it comes from different providers or different functionality? How do we bring networking together under one umbrella in a way that customer can define a single policy, let's say, Access Policy...? Who can patch this bucket? Or who can talk to the service? Or who can actually go to internet and how all these policies can be defined in a very simple way and enforced in all network paths, no matter how many network paths are there? And what equipment or what products do they use? If again, I think this will change how people think about networking. In fact, they probably stop thinking about networking, they start thinking about security and that's the goal of it. People should be thinking about policies, and not the infrastructure that allows them to propagate and allows them to enforce. So yes, my goal is for people to stop thinking about network. Richard Littauer 11:21 So I have a very silly question, because I'm just not the expert and I know that Justin and Tzury work on this stuff all the time with Curiefence. I don't... I'm largely here on this podcast helping to guide guests along but I do need to ask me, and you seem to be the expert. So I wanted to ask you, when I think of a proxy, I think of what I use to stop people knowing that I'm using the Pirate Bay to download Star Trek episodes. That's obviously not the only use for proxies. How our proxies use on a day to day cloud basis at Google. Anna Berenberg 11:54 There are multiple reasons for using proxy. At some sense, you can think of it as a choke point. So anywhere where you want to choke, access to something and enforce policy and make sure that there is a control over traffic you put the proxy in there. That's one way. So think about this, like a gateway, the place where trust domains meet together. Let's say you have one team of people who trust. Within the trust domain, you have another team of people who trust and then you want to connect this two together. To connect this two together, you need a thing in between that would allow basically somebody authoritatively say, OK, this traffic can come in, this traffic cannot come in, this traffic should be augmented, this traffic should be thrown away, whatever. The other part of the proxy is, especially of reverse proxies are more balanced and so behind proxy becomes a load balancer. So you are able to hide actual workloads and size of workloads and deployments or workloads from the people who consume the service and then you can scale up and scale down and basically all of that happens on the back end intelligently and no matter how much traffic is, uncommon proxy can distribute it in an optimal way to actually serve the consumer, as well as not overload a workload behind. Richard Littauer 13:26 Awesome. That's actually really helpful for me, because I was confused how proxies interface with load balancers. Now I see that they're basically the same thing. It's just another term. That's really great. I know that you define one of the first gRPC proxy lists mesh, if not the first. Proxy less seems to-- If proxies are so useful, what is proxy less mean, and how does that work? Anna Berenberg 13:46 Well, actually, me defining it is probably too much of taking a credit what I've done, I looked at how Google internally used service to service communication, and we have a proprietary protocols study that has historically been used at Google and what we developed actually a service mesh, before service meshes were cool and what that service mesh, what it did? It had a control plane, and it has a data plane and then control plane actually manages data plane, which is part of the study transport and framework that is linked into the client code and the server code. So it's a little different from regular service mesh. And the reason regular service meshes as we know it day needs two proxies on both sides, the protocol is mostly HTTP. So you cannot change the protocol in a way to fit into the traffic management as part of the application. So what happened here in our case, we have that is service to service communication that has been all micro services and services within Google and you can imagine, there are 100s of 1000s of micro services and services and it's proven itself very successful. So when I looked at Cloud and how important gRPCs to cloud because it allows for much better productivity and velocity of cloud developers when using protobufs, and how well it feels with a Kubernetes as modernization, while Kubernetes, modernization of compute and control plane, the gRPCs is a modernization of application network. So looking at this together, it was pretty easy to make a next step and say, OK, we are going to take a service mesh that we developed inside of Google, which is Proxima Service Mesh, and we're going to put it on gRPC. How do we do that? We actually reuse the same API's that we using for Envoy and so you have a single control plane, and you have a data plane that is compliant to this API, similarly to Envoy proxy and now you can mix and match one, you can have a proxy service and client communication, where all the traffic management, security etc., is on this side. So the developer doesn't have to worry about them, it's done for them. While actually, it doesn't also need a proxy and what it helps with in a large, it helps with 2 points: It improves latency, because going through the proxy. As latency, it's not that important for regular application, it is super important for latency critical application that measure every millisecond and less than millisecond, as well, it's important for a very large deployment. So if you think about Gmail, or something like that some big application or customers that big that if you put proxy in each hope and you have this 100s of 1000s of micro services, your daily job will be restarting proxies. So the proxy lists solve the problem of Lifecycle Management of proxies also. So the way we've done it, we actually allow them to coexist. So in a Single Service Mesh, you can have both proxies [phonetic 17:19] and proxy. Justin Dorfman 17:21 So it's not just like some big corn job. It's actually like, we're optimized. Tzury Bar Yochay 17:26 This is mind blowing JD, this is my blowing issue, I'm telling you. What's going on right now. Anna Berenberg 17:31 So, because API's are the same, you're going to get a feature parity, not full feature parity, because obviously, a proxy is--- By definition, we can have more functionality, because it's out of the process, it can restart independently and if it crashes, the process, doesn't crash itself, etc. Tzury Bar Yochay 17:54 I know one of the things that actually scares me, sometimes literally keeps me up at night, is that we live in a world of; we know that hardware is less bugs than software. That's a fact and there are reasons for it, logic gates, etc. Anna Berenberg 17:54 So we have to be lots more careful than [unintelligible 17:56] but it indeed allows for coexistence of applications that requires super high latency requirement, a super low latency requirement, as well as, cannot afford to have so many proxies in deployment. Tzury Bar Yochay 18:34 Software, as they put it out in understand orbits [phonetic 18:37] at a time, software is eating the world. Now software is everywhere. Security defined by software. Networking is defined by software, even proxy and proxy lists are software defined, transportation and routing and so on, and then we all do some amazing tech stack he just described. How can we sleep at night, knowing that software, which not to be-- You know that every software has bugs, and those bugs can easily be exploited, taken advantage by malicious entities to do all those attacks and breaches and so on? How do we make software more secure, especially critical pieces of actually handling infrastructure? Where are the guarantees the gatekeepers you put in place in your work and your team work, day to day to ensure robustness and safety and security? Anna Berenberg 19:38 Yes, this is an excellent question and that's where security and reliability come together and one of the things that we've done, let's say at Google for Envoy, re-founded the whole team that is responsible for security or Envoy and the focus on reliability for us, it's very important. That Envoy doesn't have exploits that, it will never be crashed because we're using it for as our as foundation for our product. That's one thing. Another thing is in general culture of the teams and I think Google has a culture very much tilted towards reliability. People think about reliability when they design systems, when they hold it up. There are a lot of tools to make sure that testing is properly done. There is a fuzzing tools. Fuzzing is required for interfaces. There are all sorts of paranoia that is healthy paranoia that is being deployed and required when you build infrastructure, and what to say that reliability is more important than any functionality we can add. In some cases, we can think that the integrated just because we wanted to make it more reliable. So it's the integration is not necessarily comes as the features that customers see how reliable we make the systems for the cost. Tzury Bar Yochay 21:24 Can we talk a little bit more about Envoy in--- I'm not sure how far can you go in terms of things which are proprietary within Google, because it's not a secret that Google has its own flavors? Floros [phonetic 21:38] is to be a flavor of hungry now it's probably, I'm assuming flavors of fan boy when we started curious fans, we looked at OK, what platform do we want hooked on as a first iteration of the product of this solution, and we picked Envoy not being aware of the very fact products, Google Cloud products, and apparently AWS and Azure, they are all fallowing Google as usual and rewriting they're on cloud stock on top of Envoy. So Envoy to some extent, if not yet will soon become the operating system, sort of the operating platform of the cloud networking application networking. Now, there is a barrier though, Envoy in terms of having it that in one end build the architecture, you mentioned that he's providing API and extensibility in first place. So, Envoys on feature as we had snow one of the maintainer so far we explained to us that. When they discuss a feature of Envoy they first look; can we implement the feature as an extensibility, using the Envoy core API [phonetic 22:54], and very rarely, they will change the core. In most of the cases, Envoy development is done as Envoy extensions, this is how we have built and extend and expand. So these very, I would say, extendibility, multiple API and extension need to take into account in order to get started, provide some sort of barrier to beginners to begin with but on the other end, it's like using the Veeam as an editor, if you go with the first day or two of the hustle, then you don't give up, then you find yourself in heaven. In a peaceful mind, you know, you get to the right place. So, what are things that you doing, I would say, with a single girl are probably also outside of Google, having involving Open Source related products to make the entry point easier for newcomers, for developers, who come from different platform, different technology to adopt new technologies. Anna Berenberg 24:04 It's actually a very interesting point about core versus extensions which brings us back to the question of reliability. The reason that the core is not kept very limited, is to ensure stability, reliability, and security of the core and so let's say Google doesn't need all this extensions, then it can compile them out, it doesn't compile them in, has core and then they can guarantee quality. Now everybody needs extensions because Envoy is not just approximate platform, as he said each of all the functionalities built as an extension. One of the things I think is going to simplify in the future onboarding and extensibility. There are two things: One is standardized interfaces for remote filters. There is one type of authorization remote filters, like extra amount of Z that has predefined API. So for that you weren't going to need to touch Envoy proxy all together, you can run your filter collocated if it's authorization filter, so you can build the services based on that. And the other one that is now in development from Google Developers actually, is called External Processing. So that one also is a gRPC callout from proxy and I don't call out to complicated service and this service can actually do a modification of request, if needed. So that would allow onboarding of people without ever touching a proxy itself, the only thing that does need to change is a configuration. So the configuration is given by control plane to proxy and then the proxy will call out remote services. This is like totally mind boggling. And the second option is Wasm proxy, that will allow developers to actually compile their code all together independently, the ABI, are going to be standardized also. So it's going to be a standard way of getting the data in and getting the data out and that's it. So I think you rightfully saw yourself that usability should be a concern, because more and more people using Envoy to implement-- Again, SDN, right? It's all about value added services on top of this platform. Justin Dorfman 26:40 Speaking of extensions and platforms, when is Google going to adopt Curiefence? I mean, we have a thriving community and you know, we have room for one more organization to start using Curiefence. [crosstalk 26:55] Oh, come on. Richard Littauer 27:00 You're going to scare away all our guests. Tzury Bar Yochay 27:05 This is my KPI. This is [crosstalk 27:11] Anna Berenberg 27:11 The interesting conversation to have is how Envoy as a platform can allow coexistence of a functionality, in some cases, could be competing, in some cases complimentary, how can they build the offering, where the consumer can decide I want some functionality from this product for Cloud Provider Native Offering versus I want to add on, like, in this case, Google Cloud has a product, which is in the same space as Curiefence, which is called Cloud Armor. So it's an interesting proposition to develop an ecosystem like that. Tzury Bar Yochay 27:53 Justin, you were not aware of but you should be aware at this point that Anna is a great inspiration on us while we were in the very early days of Curiefence. I had a call with Anna project that probably will follow you up on this podcast app and Matt Klein, that when we discuss actually cure Fest, 5g, its architecture around boy and so on and so forth. So the contribution from Google is given already JD on a day to day, Justin Dorfman 28:22 [crosstalk 28:23] You know what, I don't know that and it should be probably in our about us, because that's all I know, I joined two months after the release, and I was half joking. I mean, obviously, we would love it. But no, I wasn't trying to put you on the spot or anything. Richard Littauer 28:44 Justin, You're ridiculous. Alright, here is a follow up. So you do amazing work at Google, you lead many technical leads and you work really hard to make sure that this work is happening and a lot of what you're doing has been groundbreaking, which is awesome. A lot of the stuff is Open Source, gRPC is Open Source, which is really great. Anyone can get involved and when you look at the code, I mean, part of the standards and the principles for gRPC available, gRPC.io. is free and open, allow anyone to look at what's there. My question is how you incentivize clients, communities, coders, developers, projects, products, like Curiefence to get involved in the planning stage, in the stage where you're working, because Google is doing all this work at the top strategizing and it's easy to say it's Open Source at the bottom. It's easy to say, Well, we've made it open source. Go ahead and use it if you want. But it's much harder to integrate feedback from the community at the stage where it's how we're building all of this stuff and strategizing our knowledge base. So what are you doing? What are your team's doing to make sure that the needs of your offense and other projects like your listeners might have are taking into account I'd like to very high level? Anna Berenberg 29:38 Well, we always listening, I think listening is under appreciated activity and it's important to listen and important to understand, not a single requirement, but a collection of requirements because when you have requirements from multiple people, then you're not looking at solving upon problem, you are looking at solving an actually generic problem. So we always listening and being attentive to people who both contribute to Open Source as well to our customers and interestingly enough, our customers also contributors to Open Source. So this creates an ecosystem in which we provide value added services on top of Open Source, while our customers take advantage of our value added proposition, they actually improve Open Source offerings as well and they make them more acceptable to them what especially on the consumption side, are they going to produce Kubernetes operators then configure whatever we want, whatever they need, while we providing GCP API's. There are a lot of this collaboration, natural organic collaboration happening between Open Source community, that let's say a lot of our customers and the community that are our customers, as well as customers who actually are not very much interested in Open Source, but they will use Cloud Native. So you have a spectrum of customers, and all you need is to listen, to understand what they're missing. Richard Littauer 31:46 I love that answer. Thank you. We are coming up on time. So one of the questions I have Oh, aren't we started? I want to keep going, she's fine. Okay, well, then go ahead and edit your question. You know, Paul, keep that in, I want it to be real and show that we want to keep going with Anna? [crosstalk 32:05] Something Anna, I don't want you to like-- Well, I want to abuse this privilege. Anna Berenberg 32:14 You all funny. Richard Littauer 32:17 See, Mom, I'm funny. Justin Dorfman 32:19 So my final question was, where can people listen to you? And do you have any final thoughts and if not, where can people find your final thoughts elsewhere? Your other thoughts on this sort of stuff, do you have a Twitter account blog? Anna Berenberg 32:31 I want to think of myself as a person anybody has to listen. Justin Dorfman 32:38 In fact, listening is an under estimated activity that is from you. Richard Littauer 32:43 Also, I think this was really interesting to listen to you have such a depth of knowledge and your ease of explanation really shows that you're able to take these really complex stuff and just say, 'Yes, here's how we do it. It's pretty cool and it's like, really great. So you're definitely someone worth listening to. Anna Berenberg 33:00 Yes, so I have to return. I'm posting some of it. I have LinkedIn account, but now I don't have a blog. Richard Littauer 33:07 That's okay. What's your Twitter account? Anna Berenberg 33:09 It's [unintelligible 3:10] it's K-N-I-G-A and it's from Russian. It's book in Russian. Richard Littauer 33:19 I love that. Yeah. Well, thank you so much for coming on this podcast. It was really great and I really appreciate you sharing your knowledge. I'm thinking about the etymology of Envoy because Envoy means messenger and as we all know; the Roman god of messengers was also the Roman god of flowers. So we talk of flavors, we could also think of a bouquet of different Envoy things and so I was trying to show like something around this was just really great and now I feel like I've walked through a garden of beautiful flowers. So thank you so much. Justin Dorfman 33:52 I love how Richard you go. Everyone knows. I don't know and now [laughter] I always learned something like either with the linguist or a couple of new words that you define. It's a very interesting, not only do I learned about committing to Cloud Native, I also get to learn about words, big words with Richard. Richard Littauer 34:11 Thank you, so much. Anna Berenberg 34:13 Thank you. Richard Littauer 34:14 Thank you. Special Guest: Anna Berenberg.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer Guests Tom Kerkhove Zbynek Roubalik Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the Cloud Native space. We are super happy to have two guests with us today, Zbynek Roubalik, a Senior Software Engineer at Red Hat, and Tom Kerkhove, Azure Architect at Codit. Also, Zbynek and Tom are both Maintainers for KEDA. They are here to tell us more about KEDA, why it’s so cool, what they do as Maintainers for it, their challenges, and personal goals they have with helping them become better Cloud Native Engineers. There is a discussion about how it’s not just about code, but all the other pieces that really make a project, and knowing how to manage everything in between. Go ahead and download this episode to hear much more! [00:02:00 (https://podcast.curiefense.io/16?t=120)] We start with finding out what KEDA is, how long it’s been around, and how large the open source community is. [00:04:20 (https://podcast.curiefense.io/16?t=260)] Justin wonders if Tom and Zbynek were there from the beginning of the sandbox onboarding or if they came after. [00:06:32 (https://podcast.curiefense.io/16?t=392)] Tom tells us about doing case studies they’ve done for Alibaba Cloud and the CNCF blog. [00:08:35 (https://podcast.curiefense.io/16?t=515)] Richard asks Tom and Zbynek why this project and what is so cool about KEDA compared to anything else in the Cloud Native space, and they tell us what sort of stuff they do as Maintainers for KEDA. [00:11:47 (https://podcast.curiefense.io/16?t=707)] Justin brings up that he saw “Merch” in their store, and he wonders how Tom and Zbynek are fulfilling that and if they use a service. [00:15:36 (https://podcast.curiefense.io/16?t=936)] Richard wonders about Tom and Zbynek getting paid to do this work, and Tom fills us in. [00:16:33 (https://podcast.curiefense.io/16?t=993)] Tom and Zbynek talk about the major challenges that they’re facing, which include “commit and run.” [00:24:00 (https://podcast.curiefense.io/16?t=1440)] Richard asks Tom and Zbynek to share their personal goals for being with this project, where do they want to be in five years, and how is KEDA helping you become a better Cloud Native Engineer. [00:27:52 (https://podcast.curiefense.io/16?t=1672)] Richard mentions Tom’s website you should check out and he asks him if people could only read one of his blog posts which one would it be. [00:30:28 (https://podcast.curiefense.io/16?t=1828)] The guys all discuss how it’s not just about the code, but about all those other pieces that really make the project and knowing how to manage everything in between. [00:33:09 (https://podcast.curiefense.io/16?t=1989)] Find out where you can follow Zbynek and Tom online. [00:33:45 (https://podcast.curiefense.io/16?t=2025)] We end with Justin asking Tom and Zbynek who they would like to thank at the CNCF or anyone in the Cloud Native community. Quotes [00:02:05 (https://podcast.curiefense.io/16?t=125)] It’s basically aiming to make application autoscaling that simple on Kubernetes because it isn’t. So, we try to make it super simple for you so you can focus on your application and not the scaling internals of Kubernetes.” [00:05:07 (https://podcast.curiefense.io/16?t=307)] “Funnily enough, that’s a very hard piece of maintaining open source, knowing who is using it because you cannot measure it in any way.” [00:24:29 (https://podcast.curiefense.io/16?t=1469)] “By not having to worry about how Kubernetes scales.” [00:24:37 (https://podcast.curiefense.io/16?t=1477)] “I bluntly set this one. We would present the KEDA for the incubation graduation. I want this to be the standard application autoscaler so that everybody uses the same thing, that you don’t have to worry about it, and we just handle everything for you, frankly.” [00:24:46 (https://podcast.curiefense.io/16?t=1486)] “I like the simplicity because I like when things are from the user perspective because usually, we are developers, engineers, and we are focusing just on the codes, technical side.” [00:25:49 (https://podcast.curiefense.io/16?t=1549)] “And now I’m sometimes exaggerating, but they still understand how to just scale this freaking thing, while it’s just the helm install away and you’re good to go.” [00:26:30 (https://podcast.curiefense.io/16?t=1590)] “That’s also a good example because we are working on a new reference case with CAST AI and they basically use KEDA to make Kubernetes cost efficient and make sure that we don’t waste electricity and all this kind of stuff and make the environment a better place just by optimizing your workloads.” [00:28:58 (https://podcast.curiefense.io/16?t=1738)] “And that’s why for my projects I always use the mantra which is, It’s just a pull request away.” [00:29:48 (https://podcast.curiefense.io/16?t=1788)] “There’s so many things you need to do which is not code, it’s ridiculous.” [00:30:54 (https://podcast.curiefense.io/16?t=1854)] “I haven’t written a single letter of code for KEDA. I’m doing all the other things.” [00:31:41 (https://podcast.curiefense.io/16?t=1901)] “I would like to say that this is super important, like not just doing the code as mentioned, but all the stuff around, it is very important to have like active project and the health, because if you don’t have the healthy environment around, the project was stale basically.” [00:31:56 (https://podcast.curiefense.io/16?t=1916)] “I fully agree on one the typical things that I see is you go to a project and you look for documentation. If you’re lucky you find documentation, then you ask, do you have any documentation.” [00:34:09 (https://podcast.curiefense.io/16?t=2049)] “So, Chris Aniszczyk, the CTO of CNCF, because although he’s super busy, he’s always supporting us definitely. The fact that he’s helping KEDA from the early days when we were so small project and he believed in us from the start.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) jdorfman@curiefense.io (mailto:jdorfman@curiefense.io) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?lang=en) Tom Kerkhove Twitter (https://twitter.com/TomKerkhove) Tom Kerkhove Linkedin (https://www.linkedin.com/in/tomkerkhove/) Tom Kerkhove Website (https://blog.tomkerkhove.be/) Zbynek Roubalik Twitter (https://twitter.com/zroubalik) Zbynek Roubalik Linkedin (https://www.linkedin.com/in/zbynek-roubalik/) KEDA (https://keda.sh/) Codit (https://www.codit.eu/en/) Red Hat (https://www.redhat.com/en) Committing to Cloud Native Podcast-“How to manage a successful CNCF project” with William Morgan of Linkerd-Episode 5 (https://podcast.curiefense.io/5) Sticker Mule (https://www.stickermule.com/) CAST AI (https://cast.ai/) Chris Aniszczyk Twitter (https://twitter.com/cra?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript Tom [00:01]: We have some case studies on the blog, for example, Alibaba Cloud. We did one even for the CNCF blog, but I never taught I would say this about myself, but I'm really one of the PMs for this project. And I don't mean this badly to a PM, but I never saw myself as a PM, but that's just all I do reach out to people. Hey, cool stuff. Can we do something together? Because that really helps people convince that the project is used. People rely on them. Certainly, if you see big names, if they use it, then we probably should use it as well. It must be good. And that really helps. In terms of the CNCF incubation, luckily there end users can talk off the record. So we have some of the people with the big companies that we cannot list that are apart of it. Still you want to talk to your friends and colleagues about the ones. And obviously there are still ones that we do not know about because it's free in open-source Richard [01:00]: Hello and welcome to committing to cloud-native, the podcast where we talk about the confluence of cloud-native and open source. Super excited to talk about our guests today and to talk with them even cause that's what we do here on the Committed To Cloud Native podcast, besides me, Richard Littauer. We also have Justin Dorfman as a panelist. Justin, how you doing? Justin [01:20]: I'm doing great. How are you? Richard [01:22]: I'm doing excellent. Happy to be here talking with two guests today. One of them is calling from my favorite city in the Czech Republic from Bruno and we have Zbynek [name], which I probably totally butchered his name for which I apologize sincerely. I do not speak Czech. I am so sorry. We also have Tom Kerkhove. Zbynek is joining us today as a senior software engineer from Red Hat, whereas Tom is joining us as the Azure architect at Coded. Both of them together are maintainers for KEDA, capital letters please. What is KEDA? Which one of you wants to go first? Tom [02:05]: Well, I can get started. It's basically aiming to make application auto-scaling that simple on Kubernetes because it isn't. So we try to make it super simple for you. So you can focus on your application and auto scaling internals of Kubernetes. Richard [02:22]: I thought Kubernetes wasn't natively made to scale. Isn't that the entire point of like Kubernetes? It's made to help make it easier for you to do that? Tom [02:29]: Definitely. If you went through the whole internals and the guts of Kubernetes, and you understand how to get your external metrics inside the cluster and all these kind of things. Like if you just want to scale CPU and memory fine, you're good to go. But as soon as you want to get it from, let's say Prometheus's or some cloud provider, you need to do some more work and we try to give you a consistent approach to do it for all your applications, be it with cloud vendor, be it with some open source technology, we give you a consistent experience. Richard [03:02]: Okay. That makes a lot more sense. How long has this project been around? Tom [03:06]: It really was created in 2019. So I just kind of go by that. Zbynek [03:09]: I would say it's like [03:11 inaudible] Richard [03:12]: How large is the open source community? How many developers you have working there and how many maintainers? Tom [03:19]: We have four maintainers, so two people from Microsoft and us too. And I think, how many contributors do we have? Zbynek [03:27]: More than hundreds. Tom [03:29]: It originally got announced in May, 2019, indeed. So as a partnership between Microsoft and Red Hats to close the gap and given event driven, autoscale it and then early last year, so that's 2020, we donated it to the CNCF as a sandbox project to basically stimulate the vendor naturalness and then actually now we are working on our proposal for incubation and we're actively doing end user interviews. So it is a great time to have this conversation because we are also learning from people how we are helping them make that auto-scaling simpler. Justin [04:08]: That's awesome. I mean, the reason I reached out is because you're a little ahead of us in the program. We just joined sandbox and I noticed that you're getting ready to go to incubation. Were you there from the beginning of the sandbox onboarding or did you come after? Tom [04:26]: I jumped in around sandbox, I think Zbynek, you're part of KEDA from the start. Zbynek [04:32]: I joined couple months later, so maybe four months later in the summer of 2019. Tom [04:40]: So it's mainly presenting what the project is, how it helps people and why it should be part of the CNCF. We came at the moment where they were just changing the whole process. So it was a bit finding out what the new approach is, but I think it was fairly straightforward to get it done. And that also helps you think about how you run the project, helps you to think about security, being a good member of the community. I know now with incubation, more about what's the adoption, what's the community and who are the end users. Funnily enough, that's a very hard piece of maintaining open source, knowing who is using it because you cannot measure it in any way. So you really need to reach out to people. I noticed you use it. Can we list you as an end-user? And I was actually very surprised how many times you hear, okay, I need to check with legal if we can do it, this and that. Justin [05:31]: Yeah. That's so true. Tom [05:34]: Because you're only using free stuff. So for me, it seems obvious to just give back the logo, let's say, but of course a logo has. Justin [05:43]: They are strict. We had eBay's logo up on our site and they're like, we didn't say you could use that. We're like, oh, okay. Our bad we'll take it down. Like, it wasn't a big deal, but it was just, they track it down and they're like, oh, we can't have that onsite. Tom [06:00]: Like we have some really super cool end users. We already have some listed publicly, but we have some other great ones, but we cannot talk about them. But man, they're so cool that it's really annoying because people use it from day to day. Justin [06:16]: Yeah. Then the most annoying thing is that in order to go to incubation from the sandbox is you have to have user studies. And you know, we can't write them because they won't let us, it's like this kind of catch 22. So I totally feel ya. I mean, you've got some really big brand names. You've got Microsoft, Alibaba Cloud and a few others, but are they doing the case studies? How do you go about doing that? Tom [06:41]: We have some case studies on the blog that we have, for example, Alibaba Cloud. We did one even for the CNCF blog, but I never thought I would say this about myself, but I'm really one of the PMs for this project. And I don't mean this badly to a PM, but I never saw myself as a PM, but that's just all I do reach out to people. Hey, cool stuff, can we do something together because that really helps people convince that the project is used. People rely on them. Certainly, if you see big names, if they use it, then we probably should use it as well. It must be good. Right? And that really helps. In terms of the CNCF incubation luckily there end users can talk off the record. So we have some of the people with the big companies that we cannot list that are part of it. So that helps. Still, you want to talk to your friends and colleagues about ones. And obviously there are still ones that we do not know about because it's free in open source. Justin [07:40]: We recently had a guest on Avi Press co-founded and CEO of scarf.sh, check that out. Him and his team are figuring out the problem to see who's using your stuff. It's pretty cool. We're going to implement it pretty soon. So this is cool. Like we can share, you know, tips and stuff. I like that. Richard [08:01]: Also a [08:01 inaudible] KEDA is for those of you who want to go to the URL, Tom, I have a question for you. So you've been working for Coded as an Azure architect for a while, but you're also speaking of big names. You have a lot of like, you know, medallions to attach to yours. So you've been a marketer of Azure and MVP and advisors since 2014. You're one of the first GitHub stars. You've been a CNCF ambassador since 2020. And you do a lot of different stuff around GitHub, [08:25 inaudible] Promitory, kubernetes event, grid bridge, Azure deprecation. Arcos, I'm literally just reading your bio, but partially because the people in the audience haven't read your bio and it's my job to make sure they know who they're listening to. So one of the questions I have for you is why this project out of other projects, what's so cool about KEDA compared to anything else in the cloud-native space. Tom [08:45]: Honestly, I have a customer who went from an Azure service to Kubernetes. They said, we'll use Kubernetes. I think that was four or five years ago. It's great. It's contained. And it really scales very well. And then they deployed it and then they said, okay, let's auto scale. And then I was like, this is not supported out of box, good luck. Back then, there were only two metric adapters, which was Prometheus and I think GCP and I was like, okay guys, you can build it yourself or you manually scale? They said, well, manually scale. But that's how I started [09:19 inaudible] to bring Azure monitor metrics and Prometheus. [09:23] Fast forward to the partnership with Red Hat and Microsoft, I was like, man, this is what my customer needed. This is just closing such a big gap. And actually I was already working with Jeff Holland before as an MVP. So I reached out and say, Hey, how can I help? Because this impacts all my customers and yeah, I really believed in the project, I started contributing and then the team asked if I wanted to become a maintainer. So that's the story. It's just what I needed, basically. Richard [09:54]: Awesome. Zbynek what about you? Zbynek [09:55]: Oh, it was just like totally different because as I mentioned, the project originally started as a collaboration between Microsoft [10:02 inaudible]. So there was the [10:04 inaudible] the guy from [10:06 inaudible] that told me, okay, we are working on this project and we are looking for some additional workforce. So I was looking at it and I was like, this is cool because this brings something new to Kubernetes, something that's missing because all the scaling is not simple to mention. So that's why I joined this project because I like the idea. Richard [10:27]: So you two are maintainers, but maintainers as a term, which covers all manner of sins. What do you say that you're jobs are on a day-to-day basis. Are you merging issues? Are you doing big strategy planning? Are you community facilitators? What sort of stuff would you say you do? Zbynek [10:42]: I wouldn't say that we are doing everything that you mentioned. So basically we started with the communication, we have issues, we have discussions, on slack we have [10:50 inaudible] on Kubernetes space. And we are making decisions, [10:57] like feature, features that they would like to have, would like to implement and all this stuff. And it's complex so you can try different aspects of project development as I mentioned, slowly transferred into [11:10] because [11:11] well this kind of [11:12]. So that's just perfect. Justin [11:18]: No, that's how I know of Tom is Twitter. And you guys, especially, you are like besties on Twitter. Like it's this constant quote retweets and hearts and it's really cute. I just want to go into something. And the other thing with the CNCF is obviously built in the community and that's something that I am, where it's like I have multiple hats. The biggest thing is great in the community. And I saw you have merch in your store, like, how are you fulfilling that? Do you use a service or what do you do? Tom [11:57]: So that's a good example of the benefits of CNCF, where they have a service desk. And I don't know how it originally arose, but we thought, yeah, why don't we have Merch? So he started digging, and there's a lot of options. There's like these websites where they can upload your logo and then people can buy merchandising and they just pay for everything I think. But I just said, Hey, let's go to the CNCF service desk and see what the guidance is because we cannot be the only ones with this request. And then they told me that CNCF projects, which are part of graduation or incubation, get this for free. So they become part of the CNCF store. So I said, Hey, no problem. I fully respect that. Do you have any recommendations? And then they just ask, okay, what's the quantity? And we said, yeah, we just want to do a limited amount. And then they said, okay, let's just do it for you. So they picked it up for us. So that's a good example of how a foundation helps. [12:55]Also in terms of the websites I think it was with the 1.0 release. I created the first ugly version, just to have something up. And then we involved CNCF as well. And they had a person come up with the kid and design and built it for us. So they really help with these typical non-project specific things, which save us a lot of time. Justin [13:17]: The docs are beautiful. They just flow really well. They're really well done. I was pretty impressed with that. So it's interesting. So you're a sandbox project, but you're kind of getting incubator slash graduation perks. Am I going to have to talk to Chris [Name]about this? What's going on? Tom [13:36]: I always say one can only ask, if it's a no, it's a not. Yes, it's even better. Justin [13:43]: Yeah, you're absolutely right. And that's a good lesson for everyone, is just, all they can do is say no, there's really nothing to lose. Interesting. Okay. That's good to know. So you don't have to do any of the fulfillment, they handle everything. The Shopify store, I believe is what you use. Tom [13:59]: Yeah, I think so. But they did everything for us indeed, but there are actually a lot of ways where you can offer this. And so I was actually a bit overwhelmed and they wanted to use what is typically used. So we got involved. Justin [14:12]: We use Real thread, they do a really great job. We could provide them, they make a Google form and people fill it out and they put t-shirts stickers. Anything you want to put in there and they ship it anywhere in the world. So they've been a really great partner and yeah, we just basically get our stickers from sticker mule, ship them to Realthread. And then they just handle everything. It's a really great service. Richard [14:37]:Justin used to work at sticker mule. So just take whatever he says with a grain of salt. Justin [14:42]: I think sticker mule makes the best stickers like really do, but what do I know. Tom [14:47]: But you still need money to be able to make stickers. We don't have any funding so, well at least to my knowledge. we really rely on the good people, such as Snake, Azure dev, GitHub. I don't know [15:01 inaudible] we have. Zbynek [15:04]: It's [15:04 inaudible] for hosting our [15:06] for testing. Justin [15:08]: Well, are you allowed to like, do GitHub sponsors or does the CNCF prohibit that? Tom [15:15]: That's actually a good question. Justin [15:16]: Cause like, if you did like a GitHub sponsor, you can put it into an open collective and then get like a virtual card and then pay for stickers out of that or just pay for them yourself and then get a reimbursement. Tom [15:26]: Yeah. That's a good question. I don't know if we are allowed to. Justin [15:30]: Got to have stickers or it's not an open source project. Richard [15:35]: So you two are paid. You have money coming in. You're not doing this of your own initiative. It looks like you're in nice apartments. So I'm just making sure, I mean not too nice, but nice. Tom [15:44]: I do this in my spare time. So that's also why I'm super slow and delaying everything. So my company gives me some budget too, as an MVP to do whatever I want. So that helps. But the majority is while my kid is sleeping or when my wife is out for work. Justin [16:05]: How many times has Microsoft tried to recruit you, Tom? It seems like you are one of their biggest advocates. Richard [16:14]: Well, Justin is trying to get a referral fee there again, sneaky guy. You got to watch that dude. You've got to watch him. So funding aside, CNCF has given you a lot of opportunities and obviously, you know, your companies allow you to do work on this stuff. That's great. It seems like a sustainable project. You have awesome docs. You have tons of contributors. What are the major challenges that you're facing? Like what are like the top two or three things that come to mind where it's like, oh, this is the problem that I have, and I don't know how to solve it? Tom [16:43]: Commit and run. So we allow you to contribute scalers. And while it's super simple to add them, it's also, I think the downsides, because we have scalers to systems that we're not experienced with. I'm looking to see if I have an example. Richard [17:01]: So on your homepage, you have this huge list of projects. Those are the scalers you're talking about, right? Tom [17:06]: Yeah, so we rely on the community. For example, we have one for who are way clouds. If there are issues. I mean, nobody used that before. So that's why Zbynek now said okay every new scaler needs to have those end to end tests, if they're not there, sorry, we will not be able to merge them because we have no idea how it should be. Richard [17:26]: You must be this tested to ride the ride, is what you're saying. Tom [17:30]: I think that would be my remark. Zbynek [17:34]: I agree this is the biggest issue we have because as you can see [17:39] scalers is huge and it is not in one person. So two persons to do all these kind of stuff. So we need to keep the people engaged that are using the specific scale. So usually [17:55] is that there is somebody that, okay, I would like to have this scaler for KEDA what can we do? So we encourage them to continue [18:01] and would really appreciate it if thIS person can take a little kind of [18:06] and from time to time [18:09]. Richard [18:11]: I wonder if maybe a document policy talking about scalar maintenance and what's sort of required of you, if you become someone who submits a scalar might be a good idea, because then you can say, if you aren't reactive to issues, we'll remove the scaler from production because it's totally going to. Zbynek [18:23]: If I'm not mistaken, we have the [18:24] kind of [18:27] documentation software. So once those scalers are implemented [18:32 ] . So that it's fine. Richard [18:35]: How many users would you say you have? Tom [18:37]: It's open source, we have no idea. Richard [18:39]: No idea at all. Okay. But tens of thousands, hundreds of thousands of people using it. No idea. Super cool. Tom [18:45]: We only know about the people that are listed at least what I know. I don't know about you Zbynek, but that's why they becoming a maintainer makes you see these kinds of things. But that's why, if you use an open source project, it's the least you can do. Zbynek [19:02]: I know that Jeff mentioned the number of installations of Kubernetes but I'm not sure if I'm allowed to talk about this. [19:12 inaudible] was pretty impressive I would say for this kind of small project. Richard [19:18]: What's that thing about Huawei cloud. I don't know, I hadn't heard of all Huawei cloud before. I presume I will hear from it again, but I have no idea what Huawei's user base would be. I don't speak Chinese. I can't read Chinese documentation and I don't know a lot of Chinese developers. And so I was like, it could be freaking massive and maybe there really is one person keeping all that up and you wouldn't know, super tough problem. Tom [19:42]: For example the Alibaba Clouds, they are using it for, like scaling the applications that are being deployed on the Cloud. So I would say this cloud is pretty huge, like in China, so we like [19:51] number of customer that are being like the customers of Alibaba service, is just one customer for us. Justin [19:57]: We had William Morgan from Linkerd on and one thing that he does is he has an adopters.md. I don't know if you have that, but he basically just goes out and anyone he finds out that is using the project. It's not about so much the logo on the site, but the easy way for someone who's using it can do a pull request and then add their company. That's kind of something he's very obsessive about. Tom [20:25]: Yeah, I think that's another good approach, but I'm just thinking, what's the difference. Of course it's still a logo, but I think we would still have the bigger company saying it's not okay that you list us. It will be easier of course, but I think the way this needs to be fixed is not on our project. But for example, if you have a look at GitHub, you have the star. Star says nothing. I mean, this is just reds bookmark is repo. But what if there would be something new, like a button. I use this project and then you as an individual could just click that button. I could click it and then you at least have a better measurement. Richard [21:06]: There's no way that's not coming. There's no way that Microsoft doesn't know that's something that should be implemented for GitHub. I mean that's been obvious for years. Tom [21:13]: I've suggested it but let's say that it's not getting much traction. Justin [21:17]: Maybe we have the power Richard to make this happen because we know some people. Richard [21:24]: We can figure it out. I'm wondering if it's possible. I used to have the New York times logo on my personal website because I was [21:32 crosstalk] random like throwaway article about dothraki the language in the movie Game of Thrones and then New York times emailed me saying, you're using our logo on your website in a way to advertise. You are doing it to increase your cloud. That will cost you $3,000 a month or something. And I'm like, ah, no, thanks. So it was something, I don't even remember. I just took it down. Like, my commit on GitHub was like, these guys are jerks, but I realize now what I could have done is made a section of the website saying I can neither confirm or deny these people, let me use their logo or not. Tom [22:09]: But how do they know so quickly? Richard [22:11]: No they didn't know very quickly. It took them years to find out, but they basically have armies of minions who go out and just try to make money. I mean that person's making his pay grade and you know, I could have been nicer. I wasn't, but that's cool. But like that's just someone's job. Justin [22:26]: They have like the same type of job of someone giving out court orders to people. Richard [22:31]: Just like a patent troll almost, you know, numbers. Justin [22:38]: You A hole. Zbynek [22:40]: Regarding the patent trolls this is one thing I like about [22:43] is i can say the same, but basically [22:46] what is reasonable and it's been like developed by some [22:52] he can file for the patent and it's like some report from the company, but the company will take over the patent and will put it outside for everyone to submit [23:03]. So you can work on the patent, you get paid if your name is still on the patent [23:08] the company [23:10] and there is like [23:10] it will be like free forever basically. Justin [23:14]: I'm not sure if, how would IBM feel about that? Because that is the parent company of Red Hat. Tom [23:21]: Well we are still different company. So they are just owning the shares. So basically we have different policies. And this is still [23:30]. I know that in the agreement when IBM was buying Red Hat, there was like this note, just the patent will remain [23:37] as promised, this is good. Richard [23:46]: So you can get different perspectives. This is a question I haven't asked a lot, but you two seem super personable. And it's an interesting podcast just with maintainers, which we often don't have. Sometimes we have people who are more like sales reps or advocates or evangelists, my least favorite word in the world. What are your personal goals for being with this project? Where do you want to be in five years? How is KEDA helping you get there? Specifically in relation to cloud native? Not like I want a house somewhere in the mountains of Poland, but like specifically like, what is KEDA helping you do? There's a couple, there are small Hills. They are not very big. Okay. I'm sorry. I'm sorry. The mountains that are [24:23] Tom, where you live. I'm just curious, like how is KEDA helping you become a better cloud native engineer? Tom [24:28]: By not having to worry about how Kubernetes scales. I bluntly set this one, we would present KEDA for the incubation graduation. I want this to be the standard application autoscaler so that everybody uses the same thing. They don't have to worry about it. And we just handle everything for you, frankly. Richard [24:48] Nice. Zbynek [24:48]: I would say the same, I like the simplicity because I like the way things from the user perspective because usually we have the [24:57] engineers and we have [24:59] technical side [25:03]. So I do like that it should be as simple as possible even for people who don't know Kubernetes, don't know containers that much and don't know all the internals. They just have a application [25:16] they just want to scan it. So this is the end goal I would say, it shouldn't be very simple. Tom [25:23]: I fully agree. And then I typically hear a lot of people say, Hey, but you don't need it is because you could do it this way or that way. And I'm like, that's fine. But you understand Kubernetes, your highly involved in the community. So you know all the projects, but imagine you're a person who's new to cloud native and he's responsible to deploy his application. Why should he spend days, sometimes exaggerating, but days to understand how to just scale this freaking thing while it's just the [25:55 home install] away and you're good to go. And that's one less thing to worry about. There is already so many things at Kubernetes you have to worry about. Zbynek [26:06]: Kubernetes is there [26:07] for [26:07] little things, so there is like this [26:09] just called horizontal auto scaler, which allows you to scale your application based on the CPU and [26:17]. There is one big difference because this component doesn't allow you to scale to zero and KEDA can scale to zero. So you can scale your application down to zero. The [26:26] can just scale to one. This is the big difference I would say from a typical perspective. Tom [26:33]: Yeah and that's also a good example, because we are working on the reference case with cost at AI and they basically use KEDA to make kubernetes cost efficient and make sure that we don't waste electricity and all this kind of stuff and make the environment a better place just by optimizing your workloads. And that's a nice reference case because that way we're also helping the environment. Justin [26:59]: Elon Musk would love that. And as you said, it would just be a great marketing angle. Like, you know, good thing, you know, you're not flexing. You're just saying, Hey, look, we're saving energy. Not only are you saving money, but you're saving the planet. Richard [27:14]: For other listeners who may not know, Justin and I are both panelists on another podcast called Sustain. We're literally talking about sustainability of open-source and every now and then it goes away from maintainers, stopping burning out to sustaining the environment. What I love about this podcast is that it basically could go on either one of Justin's and I podcasts. It's basically the same. It's like, how to sustain maintainers and the environment super cool. It makes me really happy. Justin [27:39]: It's like a parallel universe in a way. Richard [27:41]: Crossover. Justin [27:43]: Yeah crossover. What is this a crossover episode? Anyone who watches BoJack would understand that. Richard [27:50]: So Tom, I'm looking at your website right now for listeners. This is blog. Tom Kerkhove, that's K E R K H O V E .BE. And it seems like you're a prolific writer. What blog posts do you have down the line? What are you most excited about, telling people, if people could only read one blog post, what would it be? Again really open-ended question cause mostly I just love hearing your responses. Tom [28:14]: It would be one that's on my to-do for almost one to two years, which is about my adventures of doing open source as a maintainer. Richard [28:24]: Can you talk a bit about what's on that? Tom [28:26]: First I need to have some time to write. It's basically going to talk about what I learned from being on the other side of the fence. And one of the biggest lessons to me was instead of writing a gibberish issue, with a vague bug report, really take the time because it will also help you get the bug fixed faster. But instead of opening a bug, sometimes it's just as simple as opening a pull request. And that's why for my projects, I always use the mantra, which is, it's just a pull request away. So that's why documentation, automation, quotes or whatever has to be in the git repo. So you can just send the pull request and that's why CI/CD and all of those things, which is now all in, and [29:17] is such a great help because you don't have to rely on the human making a manual change before the CI succeeds, for example. But yeah, I need to write about that first. Richard [29:27]: No, thank you for sharing. That's awesome. I can't emphasize enough how important it is for people to know that in writing open source, isn't about your intent and it's not just about just showing in coding, but it's about actually thinking about how to be effective and how to be effective at being a nice person. Tom [29:47]: There are so many things you need to do, which is not code, it's ridiculous. Even if it's replying to issues and discussions. Zbynek [29:58]: Those people should sometimes bare in mind that basically we are spread across the globe. So there are different time zones, certain days of week. This is another aspect of being maintainer. To respond toward report, at least I try to respond. Justin [30:13]: Richard's working on a code of conduct issue on a project that we work on. Richard [30:18]: No it's fine,it's so draining, man. Justin [30:22]: That's the thing it comes with the territory. Like, as you said, Tom, it's not just about the code. It's about all those other pieces that really make the project. And if you want to be going from the sandbox to graduation, those are the things you're going to have to deal with. So anyone listening to this, they're thinking about bringing their project to the cloud native, to the CNCF. These are things you have to think about. It's not just writing the best code. It's how do you manage everything in between? Tom [30:52]: I haven't written a single letter of code for KEDA. I'm doing all the other things. It's not nice, but. Richard [31:04]: No that's just so great to admit, like that's exactly what maintainers do, is not write code. Justin [31:11]: The only code I write on the main Curifense project is the MD, you know, the markdown files, updating the read MEs and getting the release notes together. And that's fine because that's what I'm good at. I'm good at doing the other things that are not co-related. So building community from scratch, very difficult, but talking to you and everyone else that also has CNCF projects, it makes it a lot easier. So thank you both. Zbynek [31:39]: I would like to say this is super important, like not just doing code as Tom mentioned but all the stuff around it is very important. Like [31:48] project and healthy because if you don't have the healthy environment, the project will [31:52]. Tom [31:55]: I fully agree. One of the typical things that I see is you go to a project and you look for documentation, and if you're lucky, you find documentation, then you ask, okay, do you have any documentation? Yes, it's the code? And I'm like, man, I'm not going to go through the code to understand how to use it. It should be documented. Otherwise I'm out. I mean, I'm here to use your free stuff, man. I don't want to waste my time. I'm just kidding, but you really need to think from an end user perspective, otherwise you will not build a good product in my opinion. Richard [32:29]: I like what Zbynek was saying earlier about UX or the DX, as they're sometimes calling it the developer experience super important work. Zbynek [32:36]: Let's say you would like to try some new project and they [32:38] documentations, [32:41] it fails and you don't know what to do because the documentation is not very good. So this is something that is very important for projects. Even like [32:52] projects but all the projects that are [32:54]. Richard [32:55]: Every listener who has had this experience, if you're not driving, raise your hands. Yes, that's right. That was all of us. Cool. Thank you so much. That was really awesome to get your insight into being a maintainer for KEDA, to being a maintainer in general, it's being good people. Where can people follow you both on the web? Zbynek where can people follow you? Zbynek [33:15]: I have a Twitter account, which is ZROWALEK, which is pretty hard so you're going to find it somewhere. Yeah this is the main point for reaching out to me. Tom [33:31]: I fell like drinking from a fire hose. You can follow me on Twitter. For links of the show notes. Justin [33:41]: Before we close out, I want to do maybe like, maybe this is like a new segment, but basically who would you like to thank at the CNCF or anyone in the cloud native community? I think that's important because then people can follow them and make sure that they're getting more points of view. So Tom, who would you like to thank? Tom [34:03]: I would say Liz Rize because she's helping us with the incubation. She's really nice to work with. But also Chris, so Chris, the CTO of CNCF because although he is super busy he's always supporting us. Definitely the fact that he's helping KEDA from the early days when we were so small project and he believed in us from the start. He's really nice, given he's that busy. And he's also the one that said to me guys, why are they not incubation yet? You need to start moving. So that's a true sign that he believes in what we're doing. So I think he's a really nice person. Zbynek [34:44]: Inaudible Justin [34:45]:You're for Chris as well. Yeah, I agree. He's always on a service desk. I'm like, how do you find time to do that? Tom [34:53]: I don't know if he ever sleeps. Justin [34:55]: I don't think so. Zbynek [34:57]: Well Tom if you say it's [34:58] I'm not sure because you don't sleep as well because you're up doing all this stuff. Justin [35:05]: Robot conspiracy, Tom [35:08]: Maybe or it's just about priorities. Richard [35:13]: Awesome. Thank you both so much. It was great having you on, let us know in the future if we can help out at all. Listeners, if you're curious about them, you can follow them in the show notes. If you have any comments about this episode, please send me or Justin a line. Justin@Curiefense.com. That's JDorfman@Curiefense.io. Again Tom, Zbynek [35:41] whichever you are. If you're in Brussels and a [inaudible]. Thank you so much. Thank you. Thank you for having us. Special Guests: Tom Kerkhove and Zbynek Roubalik.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer Guest Avi Press Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the Cloud Native space. We are super excited to have as our guest today, Avi Press, Co-Founder and CEO of Scarf. We learn all about Scarf, how it works, and what it has to do with Cloud Native. Also, Avi tells us about the newest product, The Scarf Gateway, how Linkerd and Rocket.Chat are using it, and a new longer-term project they are working on called Nomia. Avi shares what he’s most excited about for Scarf in the future, his mission for Scarf, resources Scarf is providing to users, and he tells us more about why they are focusing more on the tooling right now and not the monetization. Download this episode now to find out more from Avi! [00:01:10 (https://podcast.curiefense.io/15?t=70)] Avi tells us what Scarf is, how it works, and what it has to do with Cloud Native. [00:02:28 (https://podcast.curiefense.io/15?t=148)] We learn how Linkerd is using the new product, Scarf Gateway, and what they’re doing. [00:05:23 (https://podcast.curiefense.io/15?t=323)] Justin asks if there’s a company or something they do at Scarf, that tracks uptime of different container rate registries. Avi explains one of things they are working on with the Gateway. [00:06:11 (https://podcast.curiefense.io/15?t=371)] Avi talks about their infrastructure and CloudStack. [00:07:43 (https://podcast.curiefense.io/15?t=463)] AWS billed Scarf for $482 for the “support” needed to fix their original monthly bill and Avi explains. [00:09:25 (https://podcast.curiefense.io/15?t=565)] Since Scarf is the middle layer for interfacing with other large cloud providers that gives more data to the users, Richard wonders why the cloud providers are working with Avi and why doesn’t AWS just block all of their ports and requests. [00:11:51 (https://podcast.curiefense.io/15?t=711)] Richard is curious to know Avi’s perspective on the Legion aspect and wonders how Avi helps show Google is using his product and how does he connect people to people at Google without violating GDPR or other massive privacy concerns. [00:14:25 (https://podcast.curiefense.io/15?t=865)] If you’re a developer who’s running a Cloud Native project, find out how easy it is to set up the process to start using Scarf. Justin mentions some of Scarf’s users and Avi talks about Rocket.Chat using the Gateway. [00:16:38 (https://podcast.curiefense.io/15?t=998)] Avi fills us in on a new longer-term project they are working on called Nomia. [00:18:54 (https://podcast.curiefense.io/15?t=1134)] Avi tells us how they found Shea and all about his vision when he came to Scarf. [00:21:59 (https://podcast.curiefense.io/15?t=1319)] Besides Nomia, Avi shares what he’s most excited about for Scarf in the future and Justin asks him how his friend Havi Hoffman is doing. [00:25:32 (https://podcast.curiefense.io/15?t=1532)] Avi talks about how they’re focusing on the growth of their tooling right now, not the monetization. [00:26:24 (https://podcast.curiefense.io/15?t=1584)] Justin asks Avi to describe his company in five words, or more, his mission, and what it would be like. [00:28:27 (https://podcast.curiefense.io/15?t=1707)] Richard asks what Avi is doing to ensure that he’s not myopically focused on helping developers only focus on what companies that they can get something out of, how is he making sure that the tools he builds empower disempowered communities of developers and people of color and women, and what is he doing to make sure that these users are also being cared for and are a priority in what he’s doing. [00:29:35 (https://podcast.curiefense.io/15?t=1775)] Avi tells us what he’s doing to listen to other voices in how he develops these dev tools. He also tells us what resources Scarf is providing to users. [00:34:14 (https://podcast.curiefense.io/15?t=2054)] Find out where you can follow Avi online. Quotes [00:10:03 (https://podcast.curiefense.io/15?t=603)] “And so in that way, we’re basically an element of letting projects and companies choose the best registry.” [00:10:44 (https://podcast.curiefense.io/15?t=644)] “And so by being an agent of shifting some of that leverage back to the project owners we are creating a space where the best registry should win and not the registry that’s just most established should continue to win.” [00:18:21 (https://podcast.curiefense.io/15?t=1101)] “And so our goals for Nomia is that by offering a very generic framework and ecosystem around resource management, we can help maintainers with even more information about how their dependencies that they maintain the distributor getting used kind of regardless of how and where and from what registries, etc.” [00:19:10 (https://podcast.curiefense.io/15?t=1150)] “Can Scarf help me monetize it and like figure out how I can make a sustainable business out of Nomia?” [00:22:07 (https://podcast.curiefense.io/15?t=1327)] “I think in the long run what I’m really excited about for Scarf’s future is how we can really help the businesses behind a lot of these Cloud Native projects and help them be more successful.” [00:23:49 (https://podcast.curiefense.io/15?t=1429)] “And I think that the secret to that success is really not anything that I’m doing, but really just the problem we’re working on and trying to solve these problems in open source of how we share data with each other and how that can be a driving force for change and improvement in the Open Source and Cloud Native spaces is something that resonates with a lot of people.” [00:25:00 (https://podcast.curiefense.io/15?t=1500)] “You can never really put enough effort into the docs.” [00:25:36 (https://podcast.curiefense.io/15?t=1536)] “We’re really focusing right now on the growth of our tooling.” [00:26:49 (https://podcast.curiefense.io/15?t=1609)] “We want to empower open source developers to have sustainable businesses and projects.” [00:31:05 (https://podcast.curiefense.io/15?t=1865)] “And so, I think that’s what I would point to that not everything that we try is going to be popular or stick, and what we do is we just listen, and we adapt, and that’s how documentation insights came to be.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) SustainOSS Podcast-Avi Press and Scarf-Episode 70 (https://podcast.sustainoss.org/70) Avi Press Website (https://avi.press/) Avi Press Twitter (https://twitter.com/avi_press?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Avi Press Linkedin (https://www.linkedin.com/in/avi-press-4437a356) Avi Press-GitHub (https://github.com/aviaviavi) Scarf (https://about.scarf.sh/) Scarf Blog (https://about.scarf.sh/blog-2) The Scarf Gateway (https://about.scarf.sh/scarf-gateway) The New Stack-“Scarf Takes Aim at Package Manager Lock-In with Scarf Gateway.” (https://thenewstack.io/scarf-takes-aim-at-package-manager-lock-in-with-scarf-gateway/) Rocket.Chat (https://rocket.chat/) “Announcing Nomia and the Scarf Environment Manager,” Scarf Blog by Shea Levy (https://about.scarf.sh/post/announcing-nomia-and-the-scarf-environment-manager) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript Avi [00:00]: A lot of projects will even go publish containers to multiple places. And it's good to do that because registry sometimes go down and depending on what you're pushing out there, like that could be a runtime dependency that might may or may not be depending on kind of the nature of how people run their containers, but being robust to that and being able to actually understand usage across those registries and being able to switch what you need to, is something that ultimately just gives maintainers more control and more ownership of our software. Richard [00:31]: Hello, and welcome to Committing to Cloud Native, the podcast where we talk about the confluence of open source and cloud native technology. Super excited to have a guest on today. Justin Dorfman and I, Justin is the other panelists, have already talked to Avi Press before on another podcast called Sustain, about his amazing company. Avi is the co-founder and CEO of Scarf spelled like scarfing something down or like the fancy neck wear, which he is unfortunately not wearing today is open department. Avi, how are you doing? Avi [01:07]: I'm doing well, how are you? Richard [01:09]: Doing great. Could you give us like a two minute pitch for what Scarf is and how it works? Avi [01:14]: Yeah. So Scarf as a company, that's all about trying to help open source developers and projects and get better understanding of how their software is being used and to connect with their commercial users. We're all about trying to help open source developers support themselves and helping businesses leverage open source dependencies as effectively as possible. And just connecting the two sides of that. Richard [01:35]: What does this have to do with cloud native? Avi [01:36]: The initial product that we are working really hard on right now is a very cloud native focused technology, where we try to help specifically anyone that distributes Docker containers, understand how are your Docker containers getting pulled? How are they being used, which companies are using them. And this is something that the open source supply chain right now, I think is a very hot topic with some of the responsibilities that have come up. And I think this is really highlighting the conversation that I think the cloud is really exacerbating. That's, you know, we're pulling in dependencies from all over the place and that leads to both security implications, as well as just data sharing problems and reliability problems and how we deliver software to each other. And so we're kind of right in the heart of that right now, by trying to help open-source projects actually distribute those containers to their users directly and having more control over how they distribute those containers. Richard [02:31]: I know that Linkerd is using this new product. Can you kind of go into how they're using it and what they're doing? Avi [02:41]:Yeah. So they're users of the Scarf Gateway, Linkerd and Buoyant, the company behind it. So all of their Docker containers, you fetch them through a Linkerd domain that they own, that domain is connected to the Scarf gateway. And so Linkerd uses Scarf really for two things. So one is that the gateway is redirecting traffic tour, wherever those containers live. And now, if in the future Linkerd wants to move their container registry from one place to another, they can do that. It's a their domain that you're fetching from. And then all the stats associated as something that, you know, I think it's kind of the original reason they wanted to get on board with it. [03:19] So even knowing like what versions of what tags of the container are actually getting pulled over time, where in the world are they coming, and which companies and which clouds and container runtimes? All these sorts of things, that your registry has that data. They don't share it with you for a myriad of reasons. And we at Scarf think that maintainers should have that information. That's very helpful, both from project maintenance to know, did people get the bug fix that were just pushed out, who's using it, but also to support the businesses behind these projects, to know here's the commercial users. [03:50] I think one of the very common business models here is having an open source project and like say an enterprise grade cloud service that you try to upsell those users to and having an understanding of which companies use the open source version, is really helpful for your lead gen, for kind of just assessing project health. I think we tend to heavily weight stars on GitHub in a lot of cases, but that's not really actually as important as, well, how many companies are using this in production? Richard [04:19]: I'm such a star person, you know, like this is just such a vanity metric. However, if you look in your GitHub stat and you see clone per day, that's where I'm like, oh, holy moly. That's pretty, pretty cool. And so yeah, stars are cool, but they don't really tell you that much. I think if I remember correctly all Oliver the CTO said, it's the missing admin dashboard for container hosting? Is that something like, how he said it? I thought it was like dead on. Avi [04:50]: Yeah. And I think, cause you know, a lot of projects will even do a published containers to multiple places and it's good to do that because registering sometimes go down and depending on what you're pushing out there, like that could be a runtime dependency. It may or may not be depending on kind of the nature of how people run their containers, but being robust to that and being able to actually understand usage across those registries and being able to switch when you need to. And that's something that ultimately just gives maintainers more control and more ownership of their software. Richard [05:23]: Is there a company or is it something that you do that tracks uptime of different container rate registries? Avi [05:31]: It's not something that we are actively tracking right now, but one of the things that we are working on with the gateway is basically detecting when the registry goes down and automatically failing over to one of our mirrors and the gateway largely just redirects to wherever the registry. But in some cases, certain container runtimes don't actually respect those redirects. And then we have the proxy. Then when we proxy, we have our own like pull through caches that we use. [05:56] And the idea is that if we detect the [05:58 inaudible] to that cache, which is kind of a neat way to have just, you know, more robust, just another layer of redundancy. And when we have the automatic mechanisms will come in just by default. We're tracking that kind of thing as well. Richard [06:11]: Well, can you go more into that? Like what's your infrastructure? Like if someone's going to switch your service, what are you doing? What's your cloud stack, whatever. Avi [06:20]: Yeah. We've spent a lot of time on this infrastructure as we keep building out, it's ever more complex. So basically if you use the Scarf gateway, you just really sign up on our site. It's fully managed by Scarf, though we will be open sourcing the actual like core things. You can run it yourself if you want. Our stack is a very robust thing. It's all on AWS and in multi-region. So it's kind of fast regardless of where you are in the world. And the idea is that when someone says, Docker pull your domain.com/whatever your image identifier is, first it hits us. And then we look up and like a redirect caches, well where is this container actually? And then by default, we just issue a redirect and the client just follows that redirect and pulls the container and we log that it happened. And then we can process all that data downstream and expose that to the project owners. [07:08] There's a bit more complexity in terms of just making sure that stuff is always available. So we use Kubernetes very heavily to orchestrate, you know, the actual app servers running. We have like a global readiness cache that is keeping track of these things and all the backups to make sure that stuff never goes offline. Richard [07:25]: Have you ever considered multi-cloud? Avi [07:28]: It is something that we are considering for the long run. I think right now, especially at our size multicloud is definitely challenging to do right now, both for just like implementation complex and also just costs. But it's definitely something that we aspire to do. Richard [07:43]: Speaking of cost, I think AWS literally billed you for $482 for support, for fixing your original monthly bill. Please go into that. I saw it on Twitter. Avi [07:55]: I mostly Tweet about, yeah I thought it was funny. They're resolving it. I think it's going to be okay. So it's a really hard throw shade at AWS, but I thought it was funny cause we've been moving towards this more global infrastructure for the gateway. At the end of the day, if this is where your containers host, it needs to be like fast, everywhere available at the time and stuff. [08:14]: So we put a lot of investments into making sure that infrastructure is robust. And so yeah, we had a few resources that just didn't get for whatever reason, just like didn't get cleaned up and we're kind of just sitting around and we just saw this huge bill that was alarming. We contacted them and they took care of it and it was fine. But then they left us with just another bill for the support to do that, which was kind of crazy and I just tweeted and kind of something to make a joke about it. But in all seriousness, AWS has been great. And yeah, the instruction and building out is very rock solid as a result of all the tooling they put out there. So perhaps AWS, they do a great job. I think they're easy to gang on in kind of these ways, because they're just so successful, but for good reasons. Richard [08:59]: Yeah,*there's a reason they're number one. It's because you can't get fired for choosing AWS. Avi [09:05]: I think the multi-cloud thing is definitely very interesting for us in the long run because, yeah, you want to be as redundant as you can be. And so for us at this point in time, being multi-region was kind of the middle ground that gives us that redundancy and speed around the globe but was not quite as complex to take on at our size because we're still a startup. Richard [09:27]: If you're just the middle layer for like interfacing with other large cloud providers that just gives more data to the user, why are the cloud providers working with you? Why doesn't AWS just block all of your ports, all of your request? Avi [09:40]: Why don't they just block everything that we're doing? So I think there are a few things behind this. I mean, one is that, if you are a cloud provider say of like a registry service, so yeah, AWS has their own public container registry products. Anyone using the Scarf gateway, using their own domains to distribute their containers is more easily able to switch between the providers. And so in that way, we're basically an element of letting projects and companies choose the best registry. So if you believe we have the best registry and we want to actually like make the case, well you should switch to us because we offer the best product gateway, Scarf users are more easily able to switch. And I think that's a problem right now because if you're on a hosting provider that introduces a new feature that makes their product worse or, you know, makes it more expensive or in any way makes it so that, you would want to move, their incentive is to just lock you in. [10:38] And the mechanism of that lock-in is that the URL is theirs and not yours and everything about your hosting is at their discretion and not one that you have any control over. And so by being an agent of shifting some of that leverage back to the project owners, we are creating a space where the best registry should win and not the registry that's just most established should continue to win. And so I think that's a really key thing that we provide. And having more of that data downstream, I mean even though the registries have this information or holding onto it, we are also processing that and building tooling on top of it, which I think is valuable even to really anyone but not something that those registries do now. Richard [11:23]: You mentioned helping out with Legion as well. This is something where like, NPM, for example, doesn't tell you all the people who download your package, but they know that stuff. And then they can use that internally to figure out what [11:34] best and the best tool [11:35 inaudible] space. It's probably even better with large monopoly, like companies like Google or Amazon, which has the ability and the developer teams to say, okay, this is better, so let's go there or let's take over that project, et cetera, et cetera. NPM probably doesn't have that sort of cloud yet, although they might in certain circumstances. Well, I think NPM will [11:51 inaudible] Microsoft anyway, so yeah, sure. But one of the things which I'm curious about from your perspective is, the lead gen aspect. How do you help show that Google is using your product and how do you connect people to people at Google without violating GDPR or like other massive privacy concerns? Avi [12:10]: So we addressed these privacy concerns by not actually holding on to any personally identifiable information. So when someone pulls a container through Scarf, we do look up metadata associated with the IP address, which can tell us like, oh, this was from a company or this happened from this cloud provider, et cetera, which we then expose to the maintainers, but then we just delete the actual IP address. So like we can't tell you, Justin went and downloaded this, here's Justin's email. That's not something that we can do. That's not something that we're trying to do. But just knowing that someone from the company is downloading is useful. [12:47] And I think the way that we phrase this, is that Scarf try and track companies, we're not trying to track people and in a world where it's usually the other way around, I just love that that's something that we can do, that we can kind of just track companies on behalf of people and organizations and open source. Richard [13:01]: Well, you're not providing as much value then because the people, companies is the people you want to reach. I don't want to reach Google's letterhead. I want to reach John Schmidt or something. I don't know, Eric Schmidt, you know, I want his email so I can be like, please fund us. Avi [13:13 ]:. And there's plenty of services that help with that information. So given a company and there's lots of third-party services that can say like, well, depending on the role that you're trying to reach, like here are the emails that we know, and those are on our roadmap to have those kinds of integrations with Scarf is planned. But that doesn't mean that from a privacy perspective, that does not mean that we're like actually kind of capturing that and then raw exposing literally that we help you identify downstream from that. Richard [13:42]: Is your stuff open source as well? Avi [13:44]: The gateway itself is not quite open source yet. Although it definitely, it needs to be because, we're not really trying to lock in maintainers into doing anything, we're trying to free them up. And so it is important that we do open source it and that is planned. Other parts of the Scarf tool, chain, bits and pieces are open. So we open source everything that we can. So we have like a JavaScript library that we've open source. We're in the long run, working on package managers and components for package managers that will be open source. But right now a lot of the value that Scarf provides is the infrastructure of what we've built, which is kind of less useful on itself to open source. But as we build out the feature set and that stuff stabilizes, it is very much on the roadmap for those components to be open-sourced as well. Richard [14:28]: So if I'm a developer who is running a cloud native projects, how easy is it to set up process for me to start using Scarf? Avi [14:36]: It's pretty quick. So it's just making an account on our site. For the gateway it's literally just, tell us where the container is, tell us where you'd like it to be, and then you're done. You can optionally tell us what [14:46] you'd like to use. And then you got to go to your DMS and actually set that C name up. By default we give you a URL to use, but we encourage projects to connect their own domain because that way you're not even locked into using us. So that's really about it. I think that often when we talk to projects, like they can get set up in like 10 or 15 minutes and they're off to the races. Justin [15:07]: Yeah. It's something we're evaluating after we fixed some other things that are more pressing, but we had a call with Avi and that's how we got on this podcast. One thing I was looking at the Scarf gateway, you do have some really good users. I mean, as I said, you have linkerd, which is Buoyant and then Rocket Chat, which I haven't heard from for a while, but I know they still have activity. Are they just using Docker things? Avi [15:35]: Yeah, they're using the gateway right now. So I think that's their main installation for Rocket Chat if you're hosting yourself, just pull the container down and run it. And so, yeah, Rocket Chat has been using it in that capacity as well. They have people from different parts of the organization that are all checking Scarf to just see activity and use it however they want to make decisions. [15:55]: That's a really exciting one. I think that I love when companies like that have a host it yourself option and especially for something like chat where like having it locked down in your own cloud is such a valuable thing for a lot of different domains. And so being able to actually empower those companies to actually understand their software usage, but in a way that doesn't actually sacrifice the privacy of the users, is really is what keeps me getting up every day, really excited about what we're doing. Justin [16:19]: That's pretty awesome. And you have like a new open source project. Avi [16:24]: Oh yeah. Nomia. Justin [16:25]: TheWhite paper was really impressive. I mean, it kind of went over my head, but still, by our VP of engineering I believe, go into what it is and what the future of it is. Avi [16:37]: Yeah. Nomia is a longer term project, that we are working on kind of alongside everything else that we're doing. We brought on Shay, who's now our VP of engineering to work on. And it's a project he's been kind of stewing on for over a decade. He's been a core maintainer of the next package manager. If you're not familiar, Nix is a purely functional package manager and it takes a lot of, kind of the, you know, the cutting edge research from package management and makes it so that packages can be deterministic from their inputs, like given a set of inputs that go into a package, you know, that you're going to build the exact package output, given the inputs. [17:10]: And so what Nomia does is it generalizes those ideas from Nix. The Inputs here are all packages and here's how you build them. And here's the derivation for the packages. What Nomia does is generalize those ideas so that you can really express much more arbitrary things at different levels of abstraction. So you could think of kind of the exact ways that one package might be a product of several inputs or dependencies. You can kind of take those same ideas and then extend it to the spinning up cloud services of, here's all of the, you know, my database and these different Nomia services. I just kind of want to send them all. I want to have a developer environment with all of these packages implemented and here's how those compose. [17:54] And so Nomia is really this very generic engine for how you references resources, how you compose them and how you can just spin up an environment, whatever that might mean, given those resources and so in the long run Nomia is something that we will be building our package management tools on top of and will be available for other people to build package managers and build systems, et cetera, service managers on top of. And so our goals for Nomia is that by offering a very generic framework and ecosystem around resource management, we can help maintainers with even more information about how their dependencies that they maintain and distributor getting used kind of, regardless of how and where, and from what registries, et cetera. [18:40] So it's a very ambitious and long-term project, but it takes a lot of the very cool ideas that have come out of Nix and even things like Git use these like content addressable, deterministic principle, and applied them to very broad sets of domains. Justin [18:53]: And how did you find Shay? I mean, he seems like a unicorn. Avi [18:58]: So he actually came to us probably about a year ago and said, here's an open source project that I want to build, Nix [19:06 inaudible] for a really long time, I want to generalize them with this project. Can Scarf help me monetize it and like figure out how I can make a sustainable business out of Nomia? And the more we thought about it, the more we realized how much Shay's vision for Nomia overlap with what we were trying to do. Help people distribute their software and understand how it's being used and connect with their commercial users. The more it became clear that she should be building Nomia at Scarf, and we should kind of align them and combine forces and kind of just put that forward and get it out there. And so we brought them on to build it for Scarf. Justin [19:39]: That's awesome. This reminds me of the MRNA pioneer, Dr. Kericho I believe it is. And she had a really interesting, it's kind of similar to Shay's story, where she spent decades of her life working on MRNA, failing, not getting grants, took demotions just to continue the research. And they finally found the CEO of, I think it was the German company that hooked up with Pfizer on the newest vaccine. She was the lead on that. And I think this is kind of similar where you have this passion of an engineer of 10 plus years of going into this. I think this could turn out to be like the MRNA story of the package ecosystem. Avi [20:31]: I really hope it does turn out that way. It's very early days for Nomia, like in terms of the scope of the ambitions of the project, really just getting started with it. Nomia, have already had a lot of good interest from a lot of the Nix community and as we kind of build this stuff forward and see how it takes shape. It's really exciting to see how it evolves, it's a very ambitious vision. Richard [20:49]: Must***be a lot of GX, which was a package manager built by Jeremy Johnson at protocol labs, which uses Merkle trees to identify package managers or any packages that you have. So you can use the entire IPFS ecosystem basically to pull down packages. Avi [21:04]: That's super cool. Yeah. I feel like [21:06 crossword] so many applications to this. Yeah. The content addressability story for package management generally is just a really good idea. And like Docker makes really heavy use of that with great success for like, you know, how you can cache different layers for caching in the container space is so, so important. Justin [21:26]: And expensive I'm in the CDN business. I know how expensive that can get. Richard [21:31]: I think it's one of the benefits of using a package manager that basically runs on top of IPFS, it's centralized. So anyone who has that package makes it easier for other people to get it faster, including closer to you geographically. So it's sort of just invalid as a whole CDN model right now and also the cloud native model, where you end up with people going towards the major three companies. But instead, it sort of just, whoever's using a thing. We can use the thing too. So I'd be curious to watch Nomia evolve. I don't know if you'll be able to answer questions right now. What's different about it or so on, but that does leave me on to another question, which is besides Nomia, what are you most excited about for scarf in the future? Avi [22:06]: I mean, I think in the long run, what I'm really excited about for Scarf's future is how we can really help the businesses behind a lot of these cloud native projects and help them be more successful. So the tooling that we're going to be building out, to help with those businesses, you know, like things like support agreement facilitation, and selling of licenses and all sorts of commerce downstream from open source, I think is really going to just help the overall ecosystem in the long run to just be more sustainable. This is what, you know, we got into on the Sustain podcast a lot. But yeah, I think what I want to say is just that the sustainability issues that face the open source community as a whole are equally crucial for the cloud native community as well. [22:47] I think that open source powers of cloud native, like just directly, all this stuff is open source all the way down and whatever we do to help the businesses and people and projects in this space are just going to yield to better and better software for everybody up and down. And so by empowering both users and the people building this stuff, that's how we get a vibrant community. Richard [23:08]: I couldn't agree more, like I don't know where to go after that. That was just perfect. Justin [23:11]: I do have a very important question. And that question is, how is my friend Hovi Hoffman doing? Is she doing good? Avi [23:19]: She's doing great. She's been really helpful as we've been getting just kind of a Scarf community going. She's been kind of spearheading that charge for us and all of our content efforts and stuff. It's all her. Justin [23:32]: I mean, you got some really great talent. Like you got Hovi, you have Shay. I mean, you seem to have a knack for getting people that really get things done. That's pretty awesome. It's a good skill. Avi [23:45]: Yeah. I'm really proud of the team that we've put together already. And I think that the secret to that success is really not anything that I'm doing, but really just the problem we're working on and trying to solve these problems in open source of, you know, how we share data with each other and how that can be a driving force for change and improvement in the open source and cloud native spaces. It's something that resonates with a lot of people. And I think, yeah, this is just a hot topic in general, open source sustainability and have better incentive structures to enable that, is something that, it just attracts a lot of good talent and we're doing that by building off dev tools. And that's the other thing too, building fun dev tools and helping people out, like it's an easy sell. Justin [24:28]: Yeah. You got the JSS, TK I believe. I think that's what it comes down to, is who has the best tooling and the developers get attractive through there. Yeah. I think this is really great, also great documentation, which I saw you have good documentation as well. Avi [24:46]: That's an ongoing one to do. Justin [24:47]: It never stops. It's never done. Avi [24:52]: We've been putting more and more investment into it. And the farther we go, the farther I'm like the more I think, wow, we have such a long way to go with docs. You can really never put enough effort into the docs. That's what Havi has been really helpful with as well. She is now just like just opening [25:07 inaudible] requests all the time, improvement docs across our various repos and yeah cannot put enough resources into that. One of the things that we do is trying to just help create a blueprint of how you have a commercially viable open source project. And I think a piece of that is just docs, docs, docs, just keep going on the docs. If you think you've put enough resources into it, just put a little bit more probably is the way to go. Justin [25:31]: Already cache positive? Avi [25:34]: I hope we get there soon. We're really focusing right now on the growth of our tooling. And so the monetization of that has not really been the focus right now because the strategy here, is that by building on a really good foundation developer tools and software distribution tools and analytics, that puts us in a good position to actually start to help these projects, you know, monetize with their commercial users. But it's not what we've focused on so far. I mean, now we're kind of getting into a little bit of more startup in business ideas, but yeah, I think at our size we just really have to focus on what we do, rather than try to do a bunch of things all at once. [26:08] And so right now, dev tools let's make good dev tools. And so the gateway has been the main thing that we're putting our efforts behind and, you know, right now it's containers, but we'll be expanding it to just arbitrary kinds of packages behind it as well. So we can just broaden the reach and we can help with it. Justin [26:28]: If you can describe your company in five words for your mission, I guess I'm trying to get at what would it be? What's the north star that all your team members are marching towards. I guess what I'm trying to say. And it doesn't need to be five words. I don't know, talking about arbitrary is horrible. Avi [26:47]: We want to empower open source developers to have sustainable businesses and projects. Justin [26:51]: That resonates with me. Avi [26:54]: And our approach I think is what makes us different. There are a lot of companies and a lot of efforts to try to build more sustainable, open source. Our approach is start with the distribution of that software, help the people who build that software understand how it's being used. And that's the wedge to, how do we connect them to the commercial users? And then it all starts with the data, the distribution layer. Justin [27:18]: You know what I just thought of, do you know Patrick and Josh over at Orbit? Are you familiar with them? Avi [27:25]: Yeah. Yeah. I met Patrick a few months back at his [27:27 inaudible]. I think they just raised more money. Richard [27:30]: Yeah, he raised a 15 million A round. It's very awesome, I'm really proud of them. I think you guys could probably hook up and have you already been in talks with that? Avi [27:41]: We talked about it very initially like nine months ago. I think that if you're looking at the health of your community, I think the missing piece of that as well, who's actually using it, not just, I mean the contributions and stars, it's just like, these are all pieces of the puzzle. And so the better view that we can give to these maintainers, the better off they'll be and the better off their users will be. Justin [28:05]: Yeah. I mean, I use Orbit every single day. I mean, five days a week. And just looking at the dashboard, it would be amazing to see what stats scarf is giving me because they emphasize a lot on organizations and individuals that are using your thing, your projects. So if this gives another data point, I think. Richard [28:26]: What are you doing to ensure that you're not myopically focused on helping developers only focus on what companies that they can get something out of? How are you making sure that the tools you build actually empower disempowered communities of developers, people of color women, what are you doing to make sure that these users are also being cared for and are a priority in what you're doing? Avi [28:46]: I love that question. And I think what I'll say is that I see our mission as directly aiding that, because there's a whole group of people who would be working on open source more if it was sustainable for them to do so. And so by building tools that help them actually jump into an open source project, you work on it full time because there's actually a living on the other side of it. That is directly what is going to help more underrepresented people get into more cloud native technologies, not just like, can you get a job at a company doing that? So I think the more we empower sustainable open source, the more we are going to get in all kinds of people into open source. Richard [29:29]: So I really liked that answer. And I agree with you, one of my, I guess, questions, concerns, thoughts, is you said that your approach is different. How are you ensuring that your approach isn't directed by the kind of thinking that led to the problem in the first place, which the people thinking that what they do will help out, like what are you doing to listen to other voices in how you develop these dev tools? Avi [29:49]: What I would point to is how much we've been listening to the feedback and community about our analytics strategies. It's no secret that analytics and open source has not been the norm thus far and in a lot of contexts, extremely unpopular. And so with our initial analytics tools, the JavaScript library that we offer, Scarf JS, we ran into a lot of pushback with some very big maintainers that were using Scarf. And the feedback that we had around that was a extreme distaste for a third party hook, phoning home in a context where you weren't otherwise. [30:30]: And we really dug in with them, with people who were like, you know, thinking like this is the worst thing that you could be doing. Why would you do this? And when we dug in and talked about it with them, came to a few conclusions and agreements, actually, which is that A, even people who really, humanly don't like that, CUY maintainers might need this information and how it can help them. I don't think that there's a lot of disagreement about that. It's more about like, okay, well, how are you actually collecting this information? It's also the case that the registries already have this data. Like even right now, they just don't share it. So like, well, how do we bridge this gap? And so I think that's what I would point to that, like not everything that we try is going to be popular or stick. [31:12] And what we do is we just listen, we adapt. And that's how documentation insights came to be, which is interesting I think. People don't like a very transparent, like, the Scarf jazz logs where it's like, Hey, Scarf Jazz is here. Here's the dependency that's using it. Here's what it's reporting. Here's how you can opt out. Here's another way you can opt out. People were not okay with it. Pixel tracking on the other hand, something that I think is a lot more subversive and, opaque in a lot of ways, it's in line with people's expectations and therefore they were okay with it. So we offer pixel tracking now. And I think what was a bigger learning of that is, people are okay with the fact that registries have this information. So that's really one of the things that had us lean in on the Scarf gateway as much as we have, which is that this integrates with the registry, there's no additional like phoning home that needs to happen. [32:02] All that needs to be is that the registry and the maintainer people who actually distribute software through that registry need to be aligned and need to have incentives that are aligned. And so as a business, we align with the maintainers as much as we can. And we align with the people who use that software as much as we can. And the result is that actually this data is something that's useful and we're going to expose it. Richard [32:22]: Awesome. Good answer. How are you doing viral marketing? How are you making sure that it's not just Scarf, but actually having users go out and be like, okay, you should use Scarf. Like what resources are you providing to those users? Avi [32:35]: One line of product features that we're kind of just getting started with right now, is tools to help projects market themselves with their usage data. So we just launched a dynamic, read me badges that can either show like here's how many times these packages have been downloaded, but also here's how many control users are being identified each month. And the idea being that, you know, a lot of projects have a logo wall on their read me. And we think that's a really great approach and highly recommended if you have commercial users using your cloud native open source project, then having kind of dynamic tools that help aid that, is something that also markets Scarf as well. Like here's all the great companies that use my project. Also, we got this data from Scarf and other people come in and say, oh, actually I want that information too. Why don't I have that? Richard [33:20]: Well, you just launched Buttons recently, didn't you? Avi [33:24]: Yeah, the read me badges is, we just launched there. We'll have more tooling around this as well, just so you can kind of dynamically market yourself and use Scarf data to power it. Richard [33:35]: Where can people find out about that tooling and where can they read more about Scarf on the web? Avi [33:38]: So check us out online. We are at scarf.sh. We have a blog there, I highly recommend everyone, take a look at and you can read more about our ideas on this topic and what we're building in the space. And we also have a newly launched slack community where people can come and discuss both Scarf products, but also just as well, like open source and cloud native technologies as a whole. And you know how that fits into sustainability and just making your project successful. Richard [34:03]: Can I talk about giraffes, I'm just curious. Avi [34:08]: You can always talk about giraffes. Richard [34:09]: Excellent. I believe Avi, last name Press that your website is still avi.press. I that accurate? Avi [34:18]: That's my first name. Richard [34:19]: I said it once and I'll say it again, it's the best domain in the world. Avi [34:24]: Thank you. Thank you. I'm glad you think so. When I first saw that dop press TLDs came out. I think I was in college at the time. I stopped what I was doing. Richard [34:35]: I hope you bought them for all of your family members as well, every single person. Avi [34:40]: I didn't but I should. No, I need. Richard [34:45]: You're so selfish Avi. Avi [34:47]: I am andmy brother has a birthday coming up and that's what I'm getting him. Richard [34:55]: My website is of course Richard.liptower. No it's not Richard is not a CLP. Avi this was great, thank you so much for coming on and talking to us any last message you want to leave with the listeners about Scarf? Avi [35:11]: I think my last message would be that if you distribute software to anyone on any technology, drop us a line, tell us how we can help, whether it's analytics, whether it's commercialization, whether it's just want to chat about open source. We are here if you want to talk. So yeah. Drop us a line. Richard [35:26]: Awesome. Thank you so much. Cool. Thank you guys. Special Guest: Avi Press.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer | Tzury Bar Yochay Guest Jérôme Petazzoni Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the Cloud Native space. Our amazing guest today is Jérôme Petazzoni, Founder of Tiny Shell Script, LLC, a long-time operator in the Cloud Native space, and was part of the team that built, scaled, and operated dotCloud PaaS before that company became Docker. We learn how Jérôme became involved in Docker, his involvement in Helm, how he loves teaching complicated things to people because he finds it a motivating challenge, and how he has a ReadMe driven development. Jérôme tells us more about what his company Tiny Shell Script does, what he’s doing with Cloud Native Islamabad, and if that’s not enough, he tells us more about how a Launchpad became his travel instrument, and how he can play the theme of Zelda with a Launchpad, Raspberry Pi, and 3000 lines of Python. Download this episode to find out much more from Jérôme! [00:01:33 (https://podcast.curiefense.io/13?t=93)] Jérôme fills us in how he got involved in Docker, which was dotCloud PaaS when he started, how he ended up doing the job he did there, and what parts of the code base of Docker he’s touched. [00:09:21 (https://podcast.curiefense.io/14?t=561)] Justin wonders how many talks Jérôme has given in his career at Docker. [00:13:35 (https://podcast.curiefense.io/14?t=815)] Justin asks Jérôme if he feels that Docker paved that path to the Cloud Native space or if he thinks there’s more to it. [00:14:58 (https://podcast.curiefense.io/14?t=898)] Richard wonders where Jérôme sees the role of the developer evangelist of people like him, who accidently randomly do all of this marketing and work in building open source communities in the Cloud Native space. [00:17:48 (https://podcast.curiefense.io/14?t=1068)] We find out how Jérôme is a massive proponent of good documentation. [00:19:42 (https://podcast.curiefense.io/14?t=1182)] Learn about Jérôme’s involvement in Helm. [00:22:53 (https://podcast.curiefense.io/14?t=1373)] Jérôme tells us what Tiny Shell Script, LLC does. [00:24:52 (https://podcast.curiefense.io/14?t=1492)] Justin wonders what made Jérôme just go and do his own thing. [00:29:51 (https://podcast.curiefense.io/14?t=1791)] Richard asks if Jérôme trains his developers in French as well, and Tzury asks him if he can come do training at Reblaze. [00:31:37 (https://podcast.curiefense.io/14?t=1897)] Since Jérôme has done a lot of work elsewhere, Richard is curious in what’s going on with Cloud Native Islamabad. [00:33:18 (https://podcast.curiefense.io/14?t=1998)] Jerome tells us about his launchpad that became his travel instrument and making music. [00:35:55 (https://podcast.curiefense.io/14?t=2155)] Justin asks Jérôme if Kubernetes be what it is today without Docker, and Justin says no and wonder if he agrees. [00:37:12 (https://podcast.curiefense.io/14?t=2232)] Find out where you can follow Jérôme online. Quotes [00:15:17 (https://podcast.curiefense.io/14?t=917)] “And I would even say for any company who’s going to interact with developers because in a way we have a lower attention span.” [00:15:29 (https://podcast.curiefense.io/14?t=929)] “So, you know if I have five different containers storage products to choose from in ten minutes, basically I expect all of them to give me a two-minute video that’s going to explain why they’re the best solution.” [00:18:13 (https://podcast.curiefense.io/14?t=1093)] “Now in a way, I think it may be Lauri Apple who gave me this notion of maybe she doesn’t call it like that, but ReadMe driven development in a way.” [00:18:34 (https://podcast.curiefense.io/14?t=1114)] “Kind of start with the ReadMe because that’s how you’re going to explain to the world what that is and how it works and how folks should get started.” [00:21:43 (https://podcast.curiefense.io/14?t=1303)] “Sometimes I say, maybe I stole that from someone, that is kind of the missing package manager for Kubernetes.” [00:25:11 (https://podcast.curiefense.io/14?t=1511)] “At some point in 2015 when I was kind of questioning myself and what I wanted to do, I went around an interviewed at a few places and that was really interesting and I totally recommend to do that once in a while because it helps to kind of ground your expectations and everything.” [00:26:10 (https://podcast.curiefense.io/14?t=1570)] “That was part of their greatest course, but I don’t remember a single thing about them.” [00:34:43 (https://podcast.curiefense.io/14?t=2083)] “Until at some point again, I’m going to say accidentally stumbled upon the right combination while I realized I could very easily process media messages with that thing.” [00:36:09 (https://podcast.curiefense.io/14?t=2169)] “It’s one of these things where you need a “what if machine.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Jérôme Petazzoni Twitter (https://twitter.com/jpetazzo) Jérôme Petazzoni Linkedin (https://www.linkedin.com/in/jpetazzo) Jérôme Petazzoni Website (http://jpetazzo.github.io/) Tiny Shell Script, LLC (https://tinyshellscript.com/) Docker (https://www.docker.com/) Helm (https://helm.sh/) Cloud Native Islamabad (https://community.cncf.io/islamabad/) Cloud Native Islamabad Twitter (https://twitter.com/CloudIslamabad?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) FluidSynth (https://www.fluidsynth.org/) Container Training with Jérôme Petazzoni (https://container.training/) Zelda with a Launchpad, a Raspberry Pi, and 3000 lines of Python- Jérôme Petazzoni-YouTube (https://www.youtube.com/watch?v=pzBBoJhk1tY) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript Jérôme [00:01]: When I joined dotCloud, I was joining with a strong infrastructure background. I knew how to run servers and whisper in their ears when things weren't going well. And I did that for maybe two, three years, moving from managing our EC2 Fleet. Then being part of the big engineering mutation, then managing a small team of SREs and then suddenly Docker happened. Richard [00:28 ]: Hello, and welcome to Committing to Cloud Native. The podcast where we talk about the confluence of cloud native and open source. We have an amazing guest on today. Before we get around to introducing him. I want to introduce the other panelists. So we have Richard Littauer hello, everyone, that guy. And then we have Justin Dorfman. Justin, how are you? Justin [00:47]: I'm great, Richard. I'm just so glad you're back. Never leave again. Richard [00:51]: I'll try not to. And Tzury Bar Yochay. Tzury [00:55]: How are you today? Richard [00:56]: And our guest today is Jérôme Petazzoni. Jérôme is calling today from Berlin. Jérôme, how are you? Jérôme [01:03]: That's great. Yeah. Thank you for having me. I'm doing well today. I just got my first [01:08 inaudible] actually this afternoon. So looking forward to positive things in the future. Richard [01:14]: Well, that is awesome. Welcome. Jérôme is the founder of Tiny Shell script, LLC. He's been a long time operator in the cloud native space. He was part of the team that built scaled and operated dotCloud Paas and before the company became Docker. So you've been around forever. How did you get involved with that company? Jérôme [01:36]: How did I get involved with a Docker and more specifically dotCloud since it was not called Docker when I joined. Well, I happened to coincidentally work as a soldier of fortune, a long time ago with another consultant named Solomon Hikes. We both worked for the same company back in France, and I don't it was maybe 2005 ish, something like that. And then we stayed in touch. He was starting to do this container thing. And I had this hosting company and he was looking for servers to host his [02:12 weird chritam] channels to do his container things because back then that was complicated. [02:18]: And on the other hand, I had servers and I was looking for things to run on, said servers. So that was a good match. So basically every around six months we would get together and talk about containers and hack on things. And until eventually at some point in the summer of 2010, I was in Paris and I got a phone call like, "Hey, this is Solomon, long story short, we just raised some money after YC and we are trying to build a team. Would you like to work in California? And I was like, yeah, sure. Sign me up. And six months later, I was in a big plane on my way to San Francisco. And that's how it started. Richard [02:57]: So many questions. So how did you end up, where were you in this role? How did you start? Where did you segue to? I know that you've given tons and tons of talks and that you're really big on teaching other people how to scale stuff, but you're also a coder in your own right. So like what parts of the code base of Docker have you touched? Jérôme [03:12]: So initially when I joined dotCloud, I was joining with a strong infrastructure background. I knew how to run servers and whisper in their ears when things weren't going well. And they did that for maybe two, three years, moving from managing [03:30 inaudible], then being part of the bing engineering mutation, then managing a small team of SRS and then suddenly Docker happened. [03:40]: And we still had this platform that we wanted to maintain because we didn't want to leave all the customers, you know, like, okay, we're closing the past business and put all their energy on this new open source thing. No, we didn't want to do that. So I kind of stayed behind to keep the lights on with another engineer, Andrew [Name04:00] and we did that for maybe a year or so until we transitioned that dotCloud part of the business to someone else and eventually, it's kind of what I'm doing now, should I stay? Should they go? Is this new fangled go thing that we're doing? Is this something that I want to be a part of? And I wasn't the only one in that situation, like many folks had been kind of sold on the dotCloud project and now it was Docker and it was a completely different company and future. [04:32]: And then I accidentally gave a talk at the scale conference in LA cause I was living in Pasadena back then, and that was like maybe one hour drive from my home. And it was like, this seems to be a group, like local community conference. I'm going to give a presentation about containers. And it was pretty funny because back then nobody was really excited about containers at all. But just before me, there was a talk by [Name 04:58] who back then I believe was working maybe at [05:03 Parallels]. I don't know exactly how the company was called back then, but one of the heavy hitters in the container space back then, and it was pretty interesting because he was really deep in the technical details because here is an extremely badass [05:17 hacker]. And at the end he was like, eh, well folks, if you have ideas of what we could do with this containers, like hit me up because it's cool but we don't know exactly what, what we could do with that technology. [07:28]: I'm like, okay, what do you want to do? Well, can you come to our office and give this talk that you just gave to our team? I'm like, yeah, but we have Friday 4:00 PM. And my flight is, I don't know, like maybe Saturday morning or Sunday morning, something like that. So that seems complicated. No, that's fine, let's hop in a cab and go to the office. And so we do that in a couple of hours of heavy Beijing traffic later, something that Beijing traffic has nothing on any traffic. I'm in this office doing this container presentation in a room of maybe 20/30 Chinese engineers from Baidu. And that was also a very special experience because I was speaking in English, there were a couple of folks taking turns to translate. And then after each let's say paragraph, they were translating and then folks are asking a ton of questions and translators were asking back a bunch of questions as well. [08:27]: So, you know, very special experience, but fast forward a few months, and we were able to announce some kind of partnership or at least some kind of joint announcement with Baidu, where they said, oh, we're using Docker. And it's great. So for the company that was awesome. Now you take the same story, but you replace a China with Russia. I ended up being invited to the [Yandex conference] in Moscow a few months later and we did the same thing, go there. I didn't need an interpreter. Not because I speak Russian, but because Russians actually speak good English and we can understand each other very well. And we ended up having a joint announcement with Yandex seeing, oh, we're using Docker. So then the joke started to be, oh, we need to send you to a conference in Mountain view so that Google can say they they're using Docker. So that's how I became kind of Docker's evangelist accidentally, you know, almost of a misunderstanding. Richard [09:21]: You've done a talk when I was at MaxCDN, you did like a little talk there, but how many talks do you think you've given over your career at dotCloud or was it called Docker at the time? Jérôme [09:33]: At that point, I think it was Docker. And I think the answer is way too many. So I know how many talks I gave between something like July, 2014 and July, 2015 if I remember correctly, because we counted them because we were with my manager, then we were thinking like, okay, let's set our goals for next year. What kind of goals should we set? I was the only [10:00 inaudible] /advocate, evangelist, whatever there. And I didn't know many other advocates or evangelists back then. So it was like, what should my targets and numbers and whatever be. [10:12]: And so we count and over one year I had the 90 talks, almost 100. Now to be fair, they were alot of optimization there. So I had a few talks that I would give each time I would go to a different city. So for a while it was just intro to docker. At some point I had the Docker and storage drivers one that also, you know, when I was coming back the second time, in a city then, okay that's going to be the dotdoc. So there was a lot of contents we use, and I'm going to the city for a conference and then I'm going to do as well, like speak in the local meetups. So you kind of do a two for one each time, but still that was a lot. And I think that's kind of, part of the thing that led me to burnout. [10:59]: In a year about 100 talks and that lasted for maybe three or four years, but not the same rhythm. In 2016, for instance, I was doing a workshop every month, plus a talk almost every month or every two months, maybe, something like that. So over the course of time, I was speaking less and putting my energy into different things. And for instance, by 2017, I was doing very little speaking, like just workshops. And I was helping a lot to shape the technical program of Docker con in particular, which we call the black belt tech track. And that was an amazing experience. Richard [11:41]: I always say for some reason that you were employee number four at Docker, I don't know why, but you were there from like very early on. Jérôme [11:51]: Yeah, but that's absolutely correct. Well, maybe plus minus one, but basically there were like the two founders back then. So when Sebastian and I joined in the first pool let's say with Sam Alba, Eric [Name] was, if it's weird to say his name I can the English way [12:07 Name] the marker of man. So definitely single digit number of employees. Richard [12:13]: Yeah. And, you know, it's just interesting because like anyone remembers when you first started, it was like a rocket ship. It was like this company kind of came out of nowhere and kind of define this new way of doing dev ops and which eventually led into cloud native. How was that on the inside looking out, do you feel that Docker kind of paint that path to the cloud native space? Or do you think there's just more to it? Jérôme [12:45]: I mean, Docker is definitely part of the story and I don't know which metaphor would be the best one to describe its role. You know, is it one of the mini bricks from which we built the pillows that lead us to cloud native or something like that? Maybe it's one of these bricks, but it's kind of stuck in a really tight place right now and then the comfortable one. But yeah, I think using containers the way we do now and kind of bending them to the will of developers if we want, making them super easy to use seeing potential there. That was a pretty strong defining moment for Docker. And then I don't know why it is in the place where it is today. [13:28]: So some folks are like, well, Docker should be a huge player now. For many folks it feels like now it's more like a, you know, there's the whole enterprise business that [13:38 inaudible]. And then what remains at Docker, all the developer tooling. And I think it's great in a way, because that's where Docker we shown our Docker itself. I mean, as in the Docker engine and then later Docker desktop. So I think it's great that the company is now putting all its energy into these things on which it executed really well in the past. Of course, as a minor shareholder, I'm disappointed that you didn't do casually and digits exist, but such as life. Richard [14:09]: That's interesting to me that you started off describing yourself as a soldier of fortune, which is like one of my favorite phrases just cause it sounds awesome. I mean, it means a consultant or a technologist at some point, unless you're actually a mercenary for some military and it didn't like it. Jérôme [14:24]: No that was absolutely non- violence. Richard [14:28]: Well, it's also funny because you talk about, you know, I accidentally stumbled into Pasadena. I accidentally went to China and Russia. And to me what's really amazing about cloud native space and Docker in particular is the strength by which open source and the community has really helped lift those containers. It's just really helped make that entire movement possible. And it's because of people like you going forward and doing 90 talks in one year, which would totally lead to burnout because that is a incredibly fast burning fire. Like that is a lot of work. So I'm curious where you see the role of the developer evangelists of people like you, who accidentally randomly do all of this marketing and work in building open source communities in the cloud native space. Jérôme [15:15]: I think these kind of roles are super important these days, especially for open source companies. And I would even say for any company, who's going to interact with developers because in a way we have a lower attention span. So, you know, if I have five different container storage products to choose from in 10 minutes, basically I expect all of them to give me a two minutes video that's going to explain why they're the best solution. And I'm kind of exaggerating a little bit here, but you get the idea. There's so many information, so many things to pick from that we have a really hard job to kind of distill that information and making it super short, to the point so that folks can be like, okay, I understand, you know, Docker or [16:01 inaudible] or this or that. And now we understand that it is what I want to use or what I don't want to use.So that's part of the thing. [16:11]: And to me, that was interesting and exciting because I think I loved teaching from relatively young age. I was a TA back in college and I really enjoyed it, for instance. And then later I realized, well, I like explaining things to people, especially something complicated. To me, it's an extremely motivating challenge to be like, all right, this thing, maybe it's complicated or at least we perceive it as complicated. And I'm going to try and break it down in a way that you can understand it and that you can use it. And I want to empower you to use that complicated thing. You know, whether it's Docker or Kubernetes or tomorrow is something else. I think that's something that I really enjoy doing, almost the same way that, you know, when you build some code and you run it and it runs and you're like, yeah, it works. [17:07]: Almost the same way. It's like, Hey, I'm building this explanation. I'm building this example, I'm building this tutorial, this training content. And when I see people get it, it's kind of the same, maybe like the dopamine rush that you get when your code actually works and does the thing. So here it's like, okay, the code is the combination of the slides, the good samples, everything we put together and the end result is not like computer goes [17:34 berr] but folks understand it and get it. Then the lights goes up and you're like, okay, I've done it. Richard [17:41]: I mean it's the wetware. It's like, the software is cool and fun to make, but also it doesn't exist without the other technology that humans are really good at, which is language and sharing and talking about how we do things together. It sounds like you're probably a massive proponent of good documentation. Any thoughts on that? Jérôme [17:57]: I am, yeah, absolutely and I admit that maybe it was a little bit late to the game in a way, because when I was a younger developer, maybe I didn't hold the same opinions. I used to think, well, you know, well commented code might be enough etc. Now in a way I think it might be a [Lori18:16] who gave me this notion of maybe she doesn't call it like that, but read me driven development in a way, which is the idea like when you're going to, especially for, let's say you're a small library or some code or something like that, like something that you're going to put them on GitHub, let's say and share it to the world, kind of start with the read me, because that's how you're going to explain to the world what that is and how it works and how folks should get started. [18:42]: And I think that's fairly important. Maybe like some folks insist on fried the tests first and then fried the code so that it passes the tests. And to me that would be the documentation. Maybe don't fry the whole documentation first of course, but at least the read me so that you get an idea of how things are going to go together because if it's just a double of code and some files and some comments, it's going to be pretty difficult for folks to make anything useful out of that. So, yeah. Good documentation is important. Richard [19:17]: Like you're saying, read me driven development. RD, I like that. I'm stealing. Jérôme [19:24]: And it might have to be attributed to [Lori Name19:26]. Richard [19:27]: I'm stealing that but I'll give you credit for sure. Justin [19:30]: So one of the things when you're doing RDD or documentation driven development, or any of them is you're steering the ship, you're saying, this is where we want it to go. How do we do that? Which leads me to my next one. That was a non subtle transition. Can you tell us a bit about your involvement with helm? Jérôme [19:45]: Initially Helm was part of the contents that I added into my training content, because at first it's like, Hey, we should have an option to install things on our Kubernetes [19:56 inaudible]. And from there you go to, okay, now we want to write helm charts. So it gets to be a bigger and bigger chunk. And then a couple of years ago, helm was maybe one hour chunk of content in my training. And now we have a whole thing, like the whole day just dedicated to helm and CACD pipe line and github and things like that. So like continuation of all that. [20:24]: And the more I got involved with it, the more I appreciated pretty much everything around it, in the design, the way it works, the community, the way that you have like gazillion of helm charts out there, the artifact hub etc etce. And also sure, maybe it's not perfect. And they are probably a few things that folks maybe would do differently otherwise, but it's still pretty good and I'm glad we have helm to manage stuff in the community specifically. Richard [20:57]: My only beef with Helm charts is charts. It just always threw me off. It's like when I first got into the space, I was like, oh, so you mean like Grafana stuff. You know, it's just very confusing. However, once I'm in the space, I get it. And I've been told many times over that it is the best package manager for Kubernetes apps. Is that the case, like, I don't think there's any other package manager that stands up to it. Jérôme [21:28]: I would agree. And I could even, in way tone down that a little bit by seeing, well, anyways, the only one we have, so obviously it has to be the best. Richard [21:38]: Yeah okay, my bad. Jérôme [21:41]: But yeah, sometimes I say, maybe a stole that from someone that it's kind of the missing package manager for Kubernetes, and sure there is a bunch of things that we can install with just like cube CTL apply some EML and you'll done, but even very simple things, very quickly it turns out that you want to change one little tiny parameter, one little thing. Then you just really appreciate, you have a helm chart rather than curl this CML pipe said [22:13 gypsy TLO, client Euro]. So I really appreciate that we have Helm and even for stuff that is very easy to install now. Lots of projects are putting the work and providing Helm charts. So yeah, I think it totally deserves the title of package manager for kubernetes. And I don't even know if we need to say the best, because again. Richard [22:37]: Not so arbitrary. Jérôme [22:39]: But I'm glad that is the one we have because things could have played out worse. Justin [22:45]: You mentioned that you have like a whole day dedicated to Helm in your trainings, but we also started this by saying you're the founder of Tiny Shell Script LLC. So what is it, the Tiny Shell script does? You go around and train people how to use Docker? Jérôme [22:58]: Yeah, Tiny Shell Script LLC, it's a one person operation. So when I quit Docker, after taking something like six months off, I was like, okay, I'm going to try training and see if I can actually live from that. And that's doing great. And at some point we're like, okay, how do we do this? I need to have some kind of entity or whatever, I need the name. So basically it was either kind of a [inaudible] or finding something cute and silly like that. So basically it was either kind of JP Kudzu or finding something cute and silly like that and I took that as a kind of wink to the, you know, we say that maybe, the best [23:36 inaudible] operator from hell was supposed to be like this old grumpy unique, and super silly. When it's super angry it threatens to replace folks with a Tiny Shell Script. [23:47]: Okay I'm a Tiny Shell Script. So hopefully not in the sense that I want to replace what people do, but in the sense that hopefully what I do is going to eventually be fully automated. And, even if I don't know exactly how we're going to automate training, even though I spend a lot of time thinking about it. Tzury [24:09]: I would say you will always need eventually a Tiny Shell Script to glue components, which for some reason I know with all the helm in place and all the CIC, you always need to go. You always have these little bash snippets that you're actually using. Right. Richard [24:26]: It start with the shell script. Always starts with a shell script, not to cut you off, but I totally agree with you, sir. It just starts with the shell script and it never leaves, you know, it just works sometimes. And I just got to ask you something. So I know from doing my three or four talks, anytime I did a talk, like I got a recruiter saying, Hey, saw you at X. Just want to ask you if you like this position? You probably have offers from every type of company, all in the fangs, Facebook, Amazon. Why go Indy? What made you just say, you know what, I just want to do my own thing. What was it? Jérôme [25:09]: First as a really quick sidebar. At some point in 2015 when I was kind of questioning myself and what I wanted to do, I went around and interviewed at a few places and that was really interesting. And I totally recommend that folks do that once in a while, because it helps to kind of ground your expectations and everything. So some of these were amazing because you get along super well with everyone and you're like, that's awesome. And sometimes you completely like aced the interviews and sometime you completely [25:37 inaudible], but you know, it's life. And actually the Google interviews were really interesting because everybody's said, okay, it's going to be green, it's going to be like stupid white boarding, when they ask you to put, you know, like crinkly sideways, et. [25:54]: And I actually found them pretty interesting because yes, there was a lot of white boarding, not going to deny it. But for instance, somebody asked me something about [26:03 inaudible], which is a thing they hadn't touched in 20 plus years. And so I was super honest, I was like, well, I think I've done [26:08 inaudible] when I was in college a little bit, that was part of their grading scores, but I don't remember a single thing about them. They ate Like, okay, fine. [26:16 inaudible] is like this and that. Now can you write functions to the lesson, whatever. And that was fine. [26:22]: And very interestingly it was, I don't know, half a dozen of interviews. And the only one that went really bad was with somebody who asked me, okay. It was basically something about a scribbling algorithm, but they use containers as an example and then started to ask me, okay, do you know about containers? And I'm like, yeah a little bit. And I was kind of resisting. Richard [26:44]: Google me,he didn't google your name.Yeah I mean come on. Jérôme [26:49]: But I'm like, yeah, I know container as a little bit. I worked at Docker. I don't know if you know, Docker. He was like yes. So I was like, okay, good. And then the question was completely unrelated to containers and it was something about the skidding algorithm and I could not come up with the answer we wanted, but what was kind of interesting to me that even though, you know, I tanked one of these, they still gave me an offer, which I didn't get, but it was closed initially. They told me, oh yeah, we can hire you in our office in San Francisco. And I was like, yeah, that's going to be a five minutes walk from my place. And then the offer was something in Mountain view. And I was like, no, that's a one hour commute each way each day. [27:27]: So no, that's not worth all the money in the world. But that was interesting because that thing helped you kind of, you know, lots of misconceptions I had about the process about the company, about, you walking in. I was effectively a nobody, because even that guy didn't know about, I mean, the link between what I was doing at Docker and then [27:51 inaudible]. So that, was very interesting. Now to go back to answer your question, like, why go Indy and why not join a big company like that? Part of it was because I wanted, at least for a while to work at my own pace after the depression and burnout that I went through in 2017, in the beginning of 2018. I was like, okay, now I want to have the possibility. You know, if at some point I want to kind of take two months off, no questions asked. If I'm independent I can do that. I mean, depending of commitments, etc, of course, but it's easier to do than if I'm in a company. [28:27]: I also wanted freedom to work both with US-based customers and Europe based customers, which in a way, when I was evangelist [28:36 dev] advocated to the raffle Docker, that's kind of what I was doing because I was very often spending like a few months in Europe and doing round of conferences and workshops, etc. And that was like an incredible opportunity and I wanted to keep that, plus the fact that they have so many family and friends in France, of course. So I wanted the freedom to spend time with them without having to negotiate with my manager etc. Then things worked out really well since 2019. I've been booked, maybe not like solid back to back, but a lot, like more than I was hoping to be. [29:16]: And I wouldn't even say that since the beginning of the pandemic it's like training business has increased because I mean, my personal theory is that I help folks scale things. And with so many things going online with the pandemic, so many of these companies actually need to scale more and Kubernetes and containers are a huge opportunity for that. So more training needs. Since the beginning of this year, well I took January of, but otherwise I've been training almost nonstop since then. Justin [29:51]: Do you train your developers in French as well? Jérôme [29:54]: Both, actually the training I've wrapped up this month. Well, last month in May was in French. We've done a thing with some of my French partners, where we do a kind of training, where we start with, you know, container basics. And we ended up with building Kubernetes cluster from French, and folks can take if they want like 1, 2, 3, 4, or 5 things and some folks get the whole, how do we say the grand slam maybe, which is pretty ambitious. And we do that one in French. But it's the same content, like the slides are in English. It's the same slide as I deliver usually except it's in French now. Richard [30:29]: So I want you to come to do your training for our company [foriblaze 30:33]. What will be the earliest time slot that you can do it, assuming you go fully booked ahead. Right? This is an exclusive. Richard [30:43]: This is a first, we've never had a business offer. This is pretty cool. Go ahead. Jérôme [30:51]: I believe the earliest would be September. I think I have a week available in September. Richard [30:57]: Lock it up for us. Guys, listeners, you can't do anything before November. We took the September one. Jérôme [31:06]: And then October is actually, I have one week free in October because we aligned the next edition of the French one. And we avoided the vacation period. French folks take alot of vacation. So I know we have a week free in October because that's the French vacation. And then things open up again in November. Richard [31:27]: This is awesome. Did you think you would get like a deal? Like I didn't that [31:32 inaudible] that's pretty cool. Jérôme [31:33]: We never know, it's the first time it happens like that but you never know. Justin [31:38]: I know you've done a lot of work around elsewhere. I'm curious. What's going on with cloud native Islamabab? Jérôme [31:44]: Yeah, I think it's Patricia from Traffic Labs, connected me with the organizer of the [31:50 inaudible] Islamabad and asked me, Hey, would you like to go in and split [31:54 inaudible] We're like, sure. Why not? And that's it. Justin [31:58]: Cool. We'll have to have the link for that. I mean you give so many talks that you must have so much, like out there to talk about but we'll drop the link in the show notes, for those of you who want to see that podcast, I think it's really cool that you do stuff globally, that you're not just stuck in San Francisco and you're not just stuck in France. I mean, even though you're calling from Germany. How man languages do you speak? Jérôme [32:17]: I'm only fluent in French and English. I'm trying to be better with German, but it's a little bit hard and I can say bad things in Italian, Russian, Spanish, not actually communicate. Justin [32:32]: I was wondering if your Mandarin got any better since that visit to Baidu? Jérôme [32:37]: Unfortunately. No. So when I was at Docker, I was doing a lot of intentional speaking and I kind of the rule of beginning each doc by saying hello, I'm Jérôme and unfortunately I don't speak the local language, but saying that in the local language. And unfortunately Beijing was when I broke that rule because even though like practicing for hours, I was never able to say it in a way that would be remotely acceptable. And that was weird for some of my friends, because they're like, well, you play music. You have a really good ear for chords and melodies and stuff. So you should be able to pick up languages easily. I'm like apparently this is using different wires in my brain. Justin [33:19]: We should talk about music. What are we talking about, Docker? I just thought his photo of yours with, is it a drum machine, it's a beat. What is it? Jérôme [33:28]: It's a launch pad and that became my travel instrument when I was in that like depression, burnout phase. And you know, it's kind of, how do we stay in a kind of hanging to things, et. One of my friends from France come and visit and he had that launchpad and we just tinkered a little bit with it. And I was like, oh, this is really cool. And I feel like I could do some music with it. When I drove him back to the airport on the way back, I stopped by, like guitar center or whatever. And I bought one and I started to use it and make music with it. At some point when I was traveling again for conferences, that was a way to recharge, you know, like you want the conference and you're stemming the bar, it's kind of getting low. And I would kind of dash to my hotel room and plugged the thing to the twig, which is a [34:20 inaudible] do some music and recharge. [34:24]: And at some point I started thinking, wow, it would be nice if I didn't need the computer to do that. Technically, maybe I could have like a raspberry PI and we would connect the thing to the raspberry PI and the raspberry PI will make the sounds. But that really seemed like science fiction because I couldn't see any combination of software to make that happen. Until at some point again, I'm going to say accidentally stumbled upon the right combination while I realized I could very easily process media messages with that thing. So that was the really easy part. But making sounds, I found something called fluid synth, which takes sound fonts, which are like sound bings, something that I knew like in the late nineties, on the soon blaster [35:10 inaudible] 32 records, that's super old, but that's still around and it works great. [35:15]: And that way I had my sound making module, so to speak. So then it was just a matter of writing a bunch of code to turn that into a self-standing solution. And yeah, there we go. We have like this little instrument. I spent a lot of time in 2018, like building it mostly by code and then in 2019 kind of playing it around. And now that I'm traveling less, I prefer to play, in like bigger instruments, like a piano, guitar etc. But I still have it handy for when going out in the park or hopefully when travel will be a thing again. Richard [35:53]: I need to make a quick U-turn and go back to when we were talking about like Kubernetes and Docker. Would Kubernetes be what it is today without Docker. I say no, what do you say? Jérôme [36:07]: I think I would agree. Yeah, because it's a really good question. It's one of these things where you need a, what if machine, because they have no idea. I mean, would we end up with another kind of container engine or whatever, or, you know, maybe we would have kind of jumped to Kubernetes without the containers or something. I mean, we had Measus, we would Nomad have happened. Things like that. I don't know, like we need a what if machine to answer this question. Richard [36:39]: Like what Rick and Morty have, like the green gun, where they can go and see what happened in the different dimension. Tzury [36:45]: I like it. Thank you for asking that question, Justin, because during that time I was able to ascertain that yes Jérôme does have a YouTube channel and yes, there is a video of him playing Zelda with his launch pad, using a raspberry PI and 3000 lines of Python. And it's totally awesome. So everyone should check it out. I just drop it in the show notes and we are running up on time. So before we close up everything, and that is so awesome that you did that by the way. So awesome. Where can people find you online besides this YouTube channel? Where can they hear your words? Jérôme [37:20]: Well, I have container.training, where I usually announce when I'm going to do new, like conferences, workshops and things like that, except conferences and workshops aren't actually a thing for me anymore this year. So I rant and complain on Twitter, on JP kudzu and I'm also JP kudzu on every other social media out there, basically. Tzury [37:43]: Awesome. Thank you so much, everyone. Go check that out. Thank you for talking about your training and your work and also for all the work you did in the early days. And like until now, right. It's because of dedicated people like you that cloud native exist at all and because of you, that people learned about it and kept going and could ask you really dumb questions. Like have you heard about containers before, doing interviews? So thank you so much. Jérôme [38:09]: Thanks for having me. Richard [38:10]: Thanks, it was great seeing you again. Special Guest: Jérôme Petazzoni.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Tzury Bar Yochay Guest Alexis Richardson Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the Cloud Native space. We are super excited to have as our guest, Alexis Richardson, who is the co-founder and CEO of Weaveworks, the creator of GitOps that unlocks cloud native agility, reliability and scalability. Previously he was the TOC chairman at the CNCF, and co-founded RabbitMQ. Today, Alexis shares his advice on how to build a successful open source community and an open source company. Some key things he talks about if you want to build a successful business around open source are trust, communication, and incentives. Also, we learn more about Weaveworks, the products, and the solutions to make your life easier. Go ahead and download this episode now to learn much more from Alexis! [00:01:23 (https://podcast.curiefense.io/13?t=83)] Alexis shares with us his history of what he did in the tech industry before he co-founded Weaveworks. [00:08:12 (https://podcast.curiefense.io/13?t=492)] Tzury asks Alexis if people are preferring open source as sort of an insurance policy to get more confidence with the software they’re using, or are they actually preferring the open source so they can actively contribute and get involved in the development and patch. [00:10:00 (https://podcast.curiefense.io/13?t=600)] Alexis tells us that open source coincides with two phenomena that they’ve seen that are very important, self-service and agile. [00:13:40 (https://podcast.curiefense.io/13?t=820)] Alexis shares his thoughts on what an open source company is and how an open source company operates. He talks about Confluent and how they work on Kafka, which is an open source product. [00:16:34 (https://podcast.curiefense.io/13?t=994)] We learn from Alexis his recommendations if you want to build a successful business around open source. [00:21:00 (https://podcast.curiefense.io/13?t=1260)] Alexis shares his advice on the secret of building a successful community, and he talks about trust, communication, and incentives. [00:22:52 (https://podcast.curiefense.io/13?t=1372)] Justin asks Alexis how it went in terms of relying on the community to come up with those plugins he mentioned. [00:25:29 (https://podcast.curiefense.io/13?t=1529)] We learn more about Weaveworks, the products, and the solutions. [00:28:44 (https://podcast.curiefense.io/13?t=1724)] Alexis shares his opinion of the role of CNCF within the Cloud Native movement and the Cloud Native Open Source. [00:29:59 (https://podcast.curiefense.io/13?t=1799)] We learn who coined GitOps, Alexis tells us why he’s a big fan of Kelsey Hightower, and Tzury asks Alexis how long he thinks it will take Kubernetes to be really simple to use. [00:34:17 (https://podcast.curiefense.io/13?t=2057)] Alexis fills us in on the industry being cyclical, the new buyers, and if technologies are growing that rapidly. [00:41:41 (https://podcast.curiefense.io/13?t=2501)] We end with Alexis saying to buy from Weaveworks and use GitOps to make your life easier. Quotes [00:02:55 (https://podcast.curiefense.io/13?t=175)] “One of the lessons we had learned was that open source software was going to be how people would consume infrastructure, at least at some base level.” [00:06:05 (https://podcast.curiefense.io/13?t=365)] “Instead, we focused on, you know, solving problems for the cloud generation of apps represented by the Silicon Valley companies.” [00:12:13 (https://podcast.curiefense.io/13?t=733)] “And if you think about software developers as people, which we do now luckily, then it’s really easy to see that people want to have convenience in their work.” [00:13:48 (https://podcast.curiefense.io/13?t=828)] “An open source company is a company that primarily works with or produces open source tools in order to engage with customers who are users first because they have a free experience as well as potentially a paid experience.” [00:15:17 (https://podcast.curiefense.io/13?t=917)] “The question then becomes what is open, and for some companies, the answer has been everything has been open because we sell a complimentary product.” [00:16:34 (https://podcast.curiefense.io/13?t=994)] “Anyway, today my recommendation would be that you know, you really need to think that if you want to build a business around something open source, number one rule, your open source project has to be successful.” [00:18:14 (https://podcast.curiefense.io/13?t=1094)] “So, you need to be a leader for your project to be successful and people need to be aware that you’re the leader. And for that to be true, you really need to be pretty fully featured.” [00:20:13 (https://podcast.curiefense.io/13?t=1213)] “And obviously everyone knows that engineering the results those are not infinite, and nobody expects you to have all the features that they could possibly imagine, and you should always be prepared to say no.” [00:21:19 (https://podcast.curiefense.io/13?t=1279)] “So the secret of good community is trust and communication, and I think thirdly, incentives.” [00:21:46 (https://podcast.curiefense.io/13?t=1306)] “You can’t be a leader if you’re not actually telling people what’s going on.” [00:22:30 (https://podcast.curiefense.io/13?t=1350)] “You need to find a way to encourage contributors to take ownership so then you can have an ecosystem.” [00:22:58 (https://podcast.curiefense.io/13?t=1378)] “Well you have to seed it yourself. So, you need to go out and be prepared to go and find people to build first ones, and then you create the momentum.” [00:24:48 (https://podcast.curiefense.io/13?t=1488)] “And once you have that, you’ll have enterprise series users coming along and you won’t just have small scale use or people with no money, but also people who actually have real business problems that need solving around operations, and scale and security and governance, automation, BI, AI, everything.” [00:29:12 (https://podcast.curiefense.io/13?t=1752)] ”That is why we put our GitOps application CD tool Flux and our progressive delivery tool Flagger, for example, into the CNCF, so we can co-invest in them with Amazon and Microsoft and Alibaba for example.” [00:36:22 (https://podcast.curiefense.io/13?t=2182)] “But if you’re building a business, it’s a really good chance to find areas where you can solve problems that generate revenue.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Alexis Richardson Twitter (https://twitter.com/monadic) Alexis Richardson Linkedin (https://www.linkedin.com/in/richardsonalexis) Weaveworks (https://www.weave.works/) Confluent (https://www.confluent.io/) GitOps (https://www.weave.works/technologies/gitops/) “Kelsey, Kubernetes, and GitOps”-GitHub Universe 2020-YouTube (https://www.youtube.com/watch?v=yIAa5wHsfw4) “A Hands-on Walk Through of GitOps”- Kelsey Hightower-GitOps Days 2020-YouTube (https://www.youtube.com/watch?v=jbDidLauGtQ) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript One of the lessons that we have learned was that Open source software was going to be how people would consume infrastructure, at least at some base level. We weren't sure exactly what the business model was going to be, but we could see it was going to be important. For example, Metalogic in Finland, we've been introduced them all to Martin Micus at mysql and he'd say, yeah, sure I'd be very happy to distribute your app server with my database, but you'll need to make at least some source, we weren't sure what that would mean. Intro [00:28]: Hello and welcome to committing to cloud-native, the podcast where we talk about the interface between open source and cloud-native. We're super excited about our guests today, can't wait to introduce him. Our panelists today are Justin Dorfman and Dory [Name] and they're going to have an awesome conversation. I really enjoyed listening to it, and I really hope you enjoy this conversation. Justin [00:52]: How are you doing today, Alexis? Alexis [00:54]: Very well. Thank you. Nice to be here. Justin [00:57]: Thank you, Justin. Thank you Alexis for coming out. I'm very excited and the very early days of thinking, considering and planning out to make your [01:07inaudible] as a public officers project. We have a really insightful and effective conversation and some of those great ideas I hope you can elaborate today and repeat them to the benefits of our audience. So Alexis, would you mind sharing with us your professional history before you came about co-founding and running Weaveworks? Alexis [01:34]: I'll try. I mean, I've been in the tech industry for about 20 years, before that I was in financial services. My first job after leaving university was with Goldman Sachs where I was a derivatives trader for a while, and I had a background in mathematics and computation and things like this and an interest in technology. [01:55] So when I was working in the city, I became very interested in how technology was affecting, how people made money and did their business. We could see that, increasingly things were getting automated, not just in the single-user XL decision support sense, but just across the whole firm. And decided that it would be fun to sort of get into technology full time. So I left and started the company with friends, which became Metalogic, which was an early-stage technology venture for me in Europe building what we will now call the front office app server, I guess, in Java. And we were one of the first companies that really made an effort to kind of cater to the ordinary developer who didn't want to learn about enterprise APIs and enterprise this, and enterprise that or just write things as if they'd just done a University course. [02:44] Unfortunately, that was way early and a lot of crazy things about the company meant that we didn't succeed, but after we packed it in, I decided to do another company building on the lessons learned. So one of the lessons that we learned was that open source software was going to be how people would consume infrastructure, at least at some base level. We weren't sure exactly what the business model was going to be, but we could see it was going to be important. For example, at Metalogic in Finland, we've been introduced to Martin Micus at mysql and he'd said, yeah, sure I'd be very happy to distribute your app server with my database, but you'll need to make at least some of it opensource. So we weren't sure what that would mean. [03:22] So I decided to learn about opensource and in sequence met up with a lot of people, some of whom are still very active in the industry like James [Name] who's now I think it CloudBees doing Jenkins X. He was doing Activemq at the time. Ross Mason, who went on to do MuleSoft, then symphony soft. Rock Johnson who did spring source, which was in phase 21 at the time. And many other similar people and realize that, there was a way to build a company here, but making money was something that we were going to have to figure out. We could all see right hand mysql pioneering, but we weren't sure where that would go either. [04:00] So I started various companies to try and tackle the problem of open-source support in a world of lots of components that enterprises wanted to consume. And not settle down into building an open-source stack for finance cause that was my background. I got together with a bunch of finance guys and we did a company called cohesive, which became Cohesive Ft because it was financial technology. [04:24] And Cohesive we soon realized that the way that software was going to be used, wasn't just open source, but also through cloud and virtualization. So we built a tool called elastic server, and then we built a bunch of cloud tools which became somewhat popular. But while we were doing that, I hadn't lost my interest in opensource and was working with JP Morgan as part of my project at Cohesive. And they had launched a new initiative called AMQP, which was supposed to be like HTTP or TCP for networking in the web, but aimed at integration and messaging like MQ from IBM and TIBCO rendezvous, which was 30 or 40% of the typical financial services IT budget. [05:05] And so they wanted to have a protocol like HTTP that anyone can implement, using different implementations, just like we have many web browsers and web services today. And they all talk to each other despite being made by different people, because the protocol makes it work. So we decided to make a tool called rabbitmq, which would implement [05:23inaudible] and then can do integration and be part of the open-source stack and finance. But actually, I soon realized that would be while a fun thing to do, it would be more relevant to the industry if we took this product, which is opensource and provided it as a tool to Silicon Valley companies, building websites, people like Heroku engineers and others. [05:45] I mean, this is the time when Gihub, Twilio, everyone was getting started with these big products. So we sort of pivoted our attention towards those companies. And instead of building rabbitmq with a C shop client, receive cost client, so that some finance person could try and put it into that execution engine and then not pay us any money because it was open-source. Instead we focus on solving problems for the cloud generation of apps represented by the Silicon Valley companies. [06:14] And as a result of this rabbitmq find a niche and the niche was one it could grow from, it became very successful. So we ended up with the company, Rabbit technologies, which I also started alongside Cohisive, getting acquired by VMware. From there, I went on to Pivotol, where I ran spraying [06:31 inaudible] fabric, I was responsible for the generation of spring boot and spring cloud, speaking as a product owner rather than as a tech, I'm not a developer. And then after that, I decided to leave and do Weaveworks because I could see that the application developer and the cloud open-source were going to continue to be the thing, which is driving the market. [06:52] Today, we've seen this explosion of tools around Docker and Kubernetes, a whole ecosystem of hundreds of tools, which is exactly what we hoped would happen. I think that we're in a new world where finally we can see the potential for a second generation of technology like the internet in the nineties, but for applications and those applications will run anywhere connected by things like 5g networks. So a very exciting time to be in the industry. Justin [07:20]: You just mentioned so many great tools that all of us are familiar with, many of us have been using or still using. And perhaps we can just hang on those and talk about that exciting era. You mentioned starting Engineyard, Heroku and so on. I do want to ask you though Alexis, before we move forward to weaveworks and talking about Cloud Native, Kubernetes and so on. I do want to hang on this open-source topic for a second. So you said that Finland was, you met it through myqyl guru creator and he said, I'll be happy to distribute your software, but you have to make at least some of it open source. Alexis [08:05]: That was Martin. Yeah, Martin's view was we can partner, but there will need to be an open source over that. Justin [08:11]: From your experience over the years, are people preferring open source as a sort of insurance policy to get more confidence with the software they're using or they actually preferring the open source so they can actively contribute and get involved in the development and patch and so on. Alexis [08:31]: So I think the answer to this question changes at the time. I mean, today, I think people prefer open source because almost all the best infrastructure software is open source. And as a result of this, it has widespread support in the industry, which means that you can choose multiple vendors to get you help. I mean, not always, but mostly. And that means that you get a fair price and you get to compare and make your decisions. And we're also seeing new generation of opensource coming through where the developers at big end-user firms who are not traditional software vendors, but companies like Uber or Lyft or SoundCloud or Spotify. Justin [09:08]: Netflix and so on. Alexis [09:10]: All of those folks, the Netflix's of the world are themselves not only consuming, we're producing opensorce. So that's become part of the understanding about how infrastructure software is done today. It's good. Not only because everybody is building it on the vendor side of the fence, but also because everybody is building it on the end-user side of the fence. But originally I would say that wasn't the primary reason why people would use it. And I think that the main reason that I saw that really drove the change to what we see now is when you're trying out software for the first time, it's a lot easier if you don't have to talk to salespeople or pay for things or downloaded limited trial edition or worry about the licensing or really any of that stuff, because this is getting in the way. [09:58] So I would say opensource coincides with two phenomena that we've seen, but very important, one is Selfservice and the other is Agile. Now I know that Agile today is a little bit of a dirty word and talk about other ways of, dynamic IT and [10:15 inaudible] and so on. But fundamentally moving to a more iterative, faster-moving way of planning and executing with type decision loops or OODA loops some people call them, versus the old way of doing technology in the nineties, where we would set the budget and then spend two years spending it. And then after two years, we'd figure out if we'd got the project right or wrong, which was often wrong, whereas with the iterative method, you're constantly adjusting your plan, like a yacht in a race. And then Selfservice, you know, coincides with the web. [10:45] It's not an accident that amazon.com was successful. You can go to something that you don't even need to get out of your chair. You can look at a screen which is already in front of you, maybe you're at work and you can do this while you're supposedly working, which has spent hours and hours browsing books and then you can buy one, it gets sent to you and you don't even have to get out of your seat. So you can get it sent to you. Maybe somebody has to collect it from the front door, the doorbell. And this Selfservice makes it just about convenience. And convenience is very important for progress because every time you put something in somebody's way that prevents them, it gives them a chance to say no to something. Then they might not do it and you lose users that you can build a big business if you make things convenient. [11:26] So Netflix, they talk about the moments of truth or something. The moment of decision. This idea that before a person watches Netflix, they have an opportunity to decide to do something else. Like, go for a walk or make a meal or watch television or play with their dog or talk to their family. And Netflix, you shouldn't do any of those things in the moment of truth, you should have this decision to use Netflix because it will always work, it's always convenient and always comes on with something that you want to see, which if you think about it is a little sinister, but they already put their finger on this idea that people want to make decisions in a convenient way. And self-service is really critical to how users, as we call them how people want to do things. [12:12] And if you think about software developers as people, which we do now, luckily, then it's really easy to see that people want to have convenience in their work and tools that are familiar. You know, philosophers in the early part of the last century make careers, writing books about the relationship between people work and tools and how it needs to be authentic. And I think that the selfservice iterative way of working in teams that open source lends itself to and SAS as well, is really the right way to work in technology. Otherwise, it becomes [12:46 inaudible]. Justin [12:48]: So you mentioned Opensource company. Some would think this is a contradiction within the same sentence. Open Source mean non-profit but the core idea it's free. A company purpose is to earn revenues to shareholders. So I'm assuming the building, company and organization on top of an open source technology, the small key layers of, I would say operations that you're running has to be a thought. You need to think about each and every aspect of the operation and what should go into the open source to each point or which parts of your solution should be allocated on the premium paid side. And so on. Would you mind share with us your thoughts about this question? Alexis [13:39]: There are many different answers to this question overtime again, just as with the other question. I think today it's a little different from how it was 5 or 10 years ago, but an open-source company is a company that primarily works with or produces open-source tools in order to engage with customers who are users first, because they have a free experience as well as potentially a paid experience. And we can see a lot of open source companies in the world today. Confluent is a great example. They primarily work on CAFCA, which is an opensource product, but they also make money by selling an enterprise edition. [14:11] Some years ago you could have argued that a company like Thoughtworks was an open-source company cause they often recommended open-source tools in their consulting. I'm not sure if that's still true, but I've also seen other consulting firms that are primarily opensource focused. But I would say that this is for the benefit of the user or the user that doesn't have lock in or has choice. But often the real reason was developers just want to use those tools and work with them. But from a software product side, it's very clear that it's more of a company like Confluent, whether the product is some kind of support services or enterprise product or SAS, fundamentally you're building a capability where the end-user starts their journey through an opensource experience, almost certainly with an opensource tool. Be a CAFTA or Kubernetes, or Weaveworks. [15:01] We've been doing a lot of business around flux as well as Kubernetes and flaga, which provide people who github's capability. Other companies are building a business around other things in the CNCF, I think Pessary are building a business around Curifence, which is insecurity. So you'd be an opensource company in that sense. The question then becomes what is open? And for some companies, the answer has been, everything has been open because we sell a complimentary product. Here's a good example of, but isn't even opensource but similar, Norton antivirus, do you remember you'd buy your laptop and it would have windows on it. And for some reason it would also have Norton antivirus on it. And it was really difficult to uninstall it. Then this thing would sit there and it would tell you volubly constantly gregariously and you've got a virus or you haven't got a virus. [15:47] And if you're a normal person that's [15:48 inaudible], anyway, you can figure out if you've got any sort of tech sense at all, that you need to tell things to keep updating itself otherwise it's going to get out of date and think that everything is evil on your computer and [15:59 inaudible] until the end of time. So what they've done there is they've given you a free product, which contains free intellectual property, which is the virus list. And the patterns they're looking for. And then they forced you to upgrade to a commercial subscription to get that thing updated. Otherwise this thing turns into a hassleware. So that is an opensource business model, par excellence. I think users have pushed back on that now. Thank goodness. Justin [16:24]: Golo.com and cnn, those days. Alexis [16:28]: The good old days. Justin [16:29]: The tool bars, don't mention. Alexis [16:35]: Anyway, today, my recommendation would be that, you really need to think that if you want to build a business around something opensource, number one rule, your opensource project has to be successful. You can't build a business around an opensource project that nobody uses. You can't make money, if you know, everybody says, well, I'd love to use your opensource project, but I'm actually using Kubernetes. This is ultimately why D2IQ changed from Mesosphere to D2IQ and from Mesos to Kubernetes, because at some point they realized there just aren't enough Mesos users who want to go commercial anymore, and they're all going into Kubernetes. So the other thing that's very important as a consequence of being successful is you have to be seen to be successful, which means that you need to be clear leader in your space. [17:28] Now, some spaces are very big and have many leaders. Databases is a really good example, within the world of databases you have categories. And in each one you have typically two or three or four good opensource projects. Some are older, some are new, some have different features. I think this is a very healthy situation. And then you have another 200 that generally don't get used. There might be special purpose or a labor of love for somebody working alone or something that you know was somebody's fun project for a few years, then ran out of gas. We saw this with web frameworks in Java in 2000. It was a time when there were 30 or 40 of these and then mobile frameworks, 2010 onwards. Now we just have React and Angular and a couple other things. So you need to be a leader for your project to be successful. [18:15] And people need to be aware that you the leader and for that to be true, you really need to be pretty fully-feature because trust me if there's more than one company or team or group or whoever it is, whether it's a commercial project or if it's a pure community play really important to remember projects do compete for attention and users can become contributors and they support the projects over time. All of that is a competition in some sense. It can be a friendly competition, but it's still a competition. And for that's to go well, it doesn't help if you choose to remove a critical feature from your projects. If you do, and you have enough opensource users who want that feature, then you have to choose between one of three things. One, you put it in, or your community puts it in two you tell people to use something else with your project. [19:10] So for example, Kubernetes doesn't have a gooey because there are plenty of other GUIs that you can use. And some of them are opensource. There also commercial gooeys of course, but if nobody had a Kubernetes GUI, I'm sure that Kubernetes would have build [19:22 inaudible] inside the project. And then the third option is just to tell your users to go and f--- themselves. Trust me, it's a really bad idea if you're trying to build a business. Now, if you're not trying to build a business, maybe it doesn't matter. Theere might be all kinds of reasons why the users are happy with the limited feature set. But if there's a critical feature that users want, you shouldn't hold it back. And that's really important when you look at closely governed, opensource projects in a single company around them, which deliberately hold back features they can get into trouble if they end up in competition with that community. Justin [19:56]: See if I get it right, the foundation of a successful opensource company, is the open source layer, the opensource core, the opensource community, the product, feature set, support, and so on. Alexis [20:11]: Correct. So then you should provide the features that people need right now, if you can. And obviously everyone knows engineering the results is not infinite, and nobody expects you to have all the features that they could possibly imagine. And you should always be prepared to say no, a really good way to do that is to have some kind of steering committee or project management group, when you are opensource, which can say yes or no to features and produce roadmaps and stuff. Justin [20:36]: So for this part to be successful, it has to do with, you mentioned being a leader. So to be a leader, you need those who follow you, right? Those that follow you, meaning you got to build a community. Alexis [20:50]: You got to build a community and you also have to have competitors. Justin [20:53]: Now Alexis you have been billing thorough communities over the years. I believe you were also a chair of the CNCF for a few years, right? Could you share with us a few great insights of how to build successful community around a tech product? Alexis [21:10]: So this is obviously one of the kind of million-dollar questions, and you can build a great community and still not make money. So be careful what you do with this advice. So the secret of good community is trust and communication. And then I think thirdly incentives. So, you can establish trust by for example, always responding to users. When I did rabbitmq, we had a rule, anyone who asks a question on the mailing list gets answered within 48 hours of business time. And that created the impression that we were a very responsive community. The communication is really important. You can't be a leader if you're not actually telling people what's going on, they can't trust you either. [21:50] So you need to have folks writing about the projects, going to conferences, talking about user cases. When you have users who are successful, you need to encourage them to talk about that. So it really helps if you're a natural sort of networking person or a kind of party person is also a good person to have. I think a great example of this done right, is Alex's job without openfast. Alex is kind of tirelessly awake, nonstop, promoting openfast that putting people in touch with each other and helping people along. And then I think a third factor is incentives, where this is where you get scale because you really can't do everything. Not everyone can be Alex. [22:28] You need to find a way to encourage contributors, to take ownership. So then you can have an ecosystem. And with rabbit, we copied mural, which basically have a load of consideration plugins. So he said, we'll manage the core, which will allow us to have a fairly small engineering team before we can start trying to worry about commercialization. And then we'll have an ecosystem of adaptors plugins so about 250 in the end, which is quite a lot. [22:55] Justin: How did that go, In terms of relying on the community to come up with those plugins? [23:00] Alexis: we have to seed it in the [23:01 inaudible] so we need to go out and be prepared to go and find people to build the first one. So then you create momentum. So that was something we did with the CNCF. We knew that the secret was having a CNCF community that was out of control and that would force everybody to jump on board because there would be a sense of unstoppable momentum, but that was much more important than any other kind of marketing in the beginning anyway. So we have this strategy of attaching successful projects. So the CNCF, which meant that we had to go and find those projects and convince them that if they're looking for a foundation, the CNCF was a good choice or that a foundation would make sense for that particular project, which it didn't in every case. [23:41] And that led us to things like Kubernetes and [23:44 inaudible] and envoy and so on, which meant that by the time you get to [23:47 inaudible] rights of these people realize there's something special happening here. And then you can move into a different phase marketing in a different way, but you know, how do you get good communication and good contribution? You got find contributors. They need to be outside your team. You ideally look for tribal leaders, people who are very well known in the Ruby community or whatever. Sometimes you can have several people. One person does the coding. One person does the evangelism. Another strategy is to tie yourself to platforms so that your project is being used alongside another widely used thing? Now that gives you a platform for success. Those are all strategies for building community. Justin [24:25]: Thank you because that's what I'm currently doing with curiefence. So this is really great advice. Thank you. Thank you. Thank you. Alexis [24:32]: Another thing that's really important is if you're think of commercialization, is thinking about users on a journey and for that you need documentation. Good examples of the success stories and integrations and then help them with things like analytics, to start to understand how the project is going. Once you have that, you'll have enterprise serious users coming along and you won't just have small scale use or people with no money, but also people who actually have real business problems that need solving around operations and scale and security and governance, automation, BI, AI, everything. All of those things start to become more and more of a pain point as you grow. [25:12] And as you grow, you're obviously generating revenue in your business. So then you have a way of paying for features from the vendor. And that's where if you want to build an opensource business, you come back and you say, okay, we are selling you capability that can help you to scale up. So that could be services or it could be enterprise products. Justin [25:32]: Do you mind elaborating a little more about Weaveworks, the product, the solution? Alexis [25:41]: So Weaveworks in the light of this conversation can be seen very simply actually. We're fundamentally offering a productivity solution for free to developers in Kubernetes because in Fox and gitbubs, we're making it super easy to deploy applications, undeploy applications, update them, scale and patch them and manage them on Kubernetes clusters. In fact, we're doing that for the whole stack. So we can, through configuration, which is how githubs works, configuration lifts and get. The more you can describe it, the more you can alternate and control. [26:14] So we enable you through agents to apply your configuration to running systems and automatically verify that everything is converse to the correct state. And then far or less, if it hasn't, which means that you can run thousands and thousands of stacks, even if they're different with just one person, because the operations are automated, if you want. And that means that you can do things like if you're a telco, you can have 5g in all of 40,000 telco towers around the country, without having engineers visit them, to fix things when they go wrong, which is awesome. [26:46] And you can imagine many out of high-scale user cases, I'm sure by cars and drones and IOT, train, cloud, et cetera. So we are offering a productivity solution because we're making it so that developers can very quickly recycle applications and clusters and stacks, any kind, wherever they like. But then by making people productive, our expectation is that people will then do more. So then we're selling commercially things to protect them if they do, so security, governance, compliance, policy that protects you in a world where all your teams are using the same high productivity tool. Justin [27:26]: You mentioned githubs, interesting enough. I'm not sure if you're aware of, but within curiefence the entire configuration engine is simply a repository. So we were making a day or a year ago, year and a half ago. We were thinking that that would be probably the best solution. I'm not sure if githubs was a term used at a time, but that was a natural choice from our perspective to say, okay, let's use github. So each branch represent a different platform, QA, production and pre-prod and so on. Then every change that you make within your configuration, every role that you change, basically become a commute that you [28:06 inaudible]. [28:07] So then I find out about githubs, which is a thing and a methodology and a way of life. And I was so excited to see how people coming from different directions, different angles, or actually ending up in pretty much same principles, same concepts. So you mentioned a community, you mentioned principles as I believe the foundation of transparency, communication, leadership. We skip over probably a CNCF, the role of CNCF nowadays, the way you see it within running and moving forward to cloud native. So what is in your opinion, the role of CNCF within the cloud native movement, the cloud native opensource? As a company, if somebody now we're thinking of this idea of opensource project. Alexis [28:56]: What does it mean for us you mean, for weaveworks? We want as many companies as possible to have the simplest possible time using Kubernetes and cloud native. And therefore we want everybody to be using ideally a very simple, common set of very popular tools. That is why we put our githubs application CD tool flux and our progressive deliveries, farga example, into the CNCF. So we can co-invest in them with Amazon and Microsoft and Alibaba, for example, we are all using these things in production and drive it forward as a common piece of infrastructure across the whole industry. [29:34] And then enterprises, big companies like HSBC or apple or fidelity, you know, big organizations that are using these open source tools can say, oh, I'm doing continuous deployment or I'm doing progressive delivery. Let me use these tools to do that. And oh I'm also doing githubs, which means I can be secure and also great. Justin [29:54]: Who coined this githubs term? Alexis [29:58]: Yeah. I want to know. Justin [29:58]: Oh, you did. How awesome is it when like Kelsey Hightower talks about it on stage? I mean, that's pretty awesome. Alexis [30:08]: I'm a very big fan of Kelsey and all of his work. He's the first person to really hone in on how to explain githubs in a really simple way to developers, which, you know, we come from an operations perspective, building a sauce, and Kelsey wanted to rethink it through and he understood that it wasn't actually about git as much as it was about hubs on this idea that you have this reconciliation, but automatically fixes your stack for you every time it goes wrong, which means that you can do deployments and updates and patches and rollbacks all automatically. It's awesome. So, anyway, going back to what's in the CNCF for us, you know, being in a foundation is making a commitment to a community in return for hopefully building your commercial product out of standard tools. [30:53] And by standard, I don't mean what ISO standards, but just tools that everybody recognizes the use by lots of people, they don't even have to be individually leaders, they could be one of three or four commonly used tools. But when you talk to a customer, they don't say go away because we're using Oracle. They say, please come in and talk to us because we're using the same tools that you've built your business from. Now, if you're a software company, that's a really good thing. Just think about back to the nineties when you know, it was like, oh, piss off you're not Microsoft, or you're not Oracle, or you're not some. It's harder to make yourself stand out. But if you're in the opensource world, the foundation creates a platform, but not only big companies can succeed, but also small companies. Justin [31:37]: So how long do you think it will take Kubernetes, not to mention eco to be a real simple [31:45 inaudible], like real simple? Justin [31:48]: While there is such a thing as fundamental complexity, right? You know, like chasing common core of complexity. What is the smallest number of bets that you could write Kubernetes in? And the answer is obviously 10, but you know, the 10 characters that is, what can I say? I mean, Kubenete is fundamentally a bit complicated because fundamentally it's quite richly featured. If you don't use all of Kubernetes, you can have a simpler experience. And there are tools like K3S and trust me, I know a lot of people who've thought about writing their own simpler Kubernetes, but you're taking something out in order to be simpler. So that's the first thing. [32:24] I think the roots of simplicity is not in a removing cycle, atomic complexity, the code is going to stay, but we can simplify it in other ways. For me as an end-user Android is simple because I had a phone in front of me and I can turn it on and off on. I can install applications and make phone calls. And if there's no fluff in the bottom, I could even charge it. So that's good. I don't need to care about Android version typically, or a lot of things about Android. So in that sense, to me as a user, it is simple. So I'm experiencing the joys of platforms and embedded software now, but I believe that for Kubernetes, people will still think in terms of applications. The cost will be part of the infrastructure you start the application with. [33:05] Like an app server on demand. So you'll go to the cloud, you'll say, give me my cluster and my stack. You'll say, I want to do machine learning. Okay. Here's a cluster with machine learning stuff on it already, off you go straight away. You're instantly productive where if I'm an independent software vendor, like, you know, say I'm selling a business application, I will bundle that as a set of containers and maybe Kubernetes clusters so that somebody can run it wherever they want to, without the whole installation rigmarole that we used to have in the enterprise. [33:32] So all of that is good, so everyone operates much more like a kind of enterprise app store. So all of that will mean that things for most users seem simple, even though they will still be some devops people and some platform operations engineers, and some Linux people, and some other people, who are still concerned about how Kubernetes works and what the architecture should be for version three. But the rest of the world's not going to care about just like, you know, I'm quite certain that you can spend hours of your life arguing about [34:02 inaudible] about things on the Linux lists on the days when he's around there. But you know, for the rest of us, it's not relevant. The world is simple enough that we can use Linux through other tools. Justin [34:15]: One routine is I like to discuss before I'm running out of time, is the new buyers, I would say the decision-makers, the text choice nowadays seems to be made by different audience than it used to be. So do you think this is one of the reasons for the opensource and cloud native and degrade moment that we're experienced in right now, those technologies are growing that rapidly? Or is it the vice versa? Alexis [34:52]: I think the industry is cyclical, most industries are cyclical imperative creation, expansion, and then a period of structural collapse, which proceeds a new area of construction. And that's often in business associated with things like M&A, consolidation and growth. You know, the internet in the nineties, I think is the best analogy for today's era. And for a while, everybody was building the same stuff on the same tools. And then you get to the end of the nineties, around 2000, you have the Java apps, and it's a more destructive time, but from business point of view, the market is still growing. And then you kind of get to about, I don't know, 2006 or seven, and they started to consolidate. [35:31] Some people have moved on to a new tool set because they've realized that you can power a big internet sites like Facebook on PHP, and suddenly the world is different. And people are thinking about the rich funds, the mobile apps and much more. So right now, I think we're in a period of expansion and creation. So I guess a Cambrian explosion would be another analogy people like to use that a lot of strange new tools and hope for [35:54 inaudible]. [35:56] And then I think that we'll soon see a period when some of the land mammals emerge and not so many creatures are around maybe a handful. And then there'll be in each category two or three good tools that everyone uses. The real winners out of this will be the people who build business. Well, okay. No, that's not true. The end users will win because they'll have a chance to really drive the industry. The developers will win cause they'll get to work on some sort of stuff. But if you're building a business, it's a really good chance to find areas where you can solve problems that generate revenue, whether it's kind of operations or, you know, governance or something else around the [36:33 inaudible]. Justin [36:35]: Help me understand something, not that I'm anti or God forbid, I'm apart of this, working on this project opensource and cloud native in a low debt. However, to me, it still seems like something that I'm still not getting, the fact is that we talk about infrastructure as code. We talk about automation, talking about continuous integration. It could use deployment and githubs and all those fancy great automated and infrastructure and automation, et cetera, et cetera. However, seems to me that probably it's the middle phase. Like it will evolve over a period of time where it could be a really simple, but right now this is really like the overhead of the devops world, is sometimes takes longer than, let's say in the short term than to benefit immediately, if you understand what I was saying, if I want to put, for example Linux with an [37:36 inaudible]. It should take me know less than a minute to get it done but now if I want to do it through Kubernetes, now I need to go through [37:46 inaudible] and configurations. Like simple stuff get more complicated, even though you would expect this automation will save us time. We're going to work less hours, in fact, working much more, many hours. Alexis [38:00]: How many hours you work is tied to other factors. I mean, if the tool makes you more productive, often you find yourself doing a lot more with the tool, as far as those, like Jevons paradox about that. Justin [38:15]: So because you enjoy it you're using it more. So you're spending more time in the computer because you like the technology? Alexis [38:20]: Well see now that the customer resource is to make them more committed, we'll just use more resources. So the network and cost goes up. Who led to Bitcoin because they are real idiots. I believe that, we haven't moved the barrier to entry out of the way enough. So a lot of the benefits, day two benefits, operational benefits, they all are in production with larger scale systems. But for the ordinary developer, just putting, you know, two Netgear bricks together to make a wall for a demo or a POC or a small app, you don't need all of that stuff. So we don't have a continuous experience between the ordinary developer and the large scale application at the moment. That is a shame. I think that will get solved in time. I think we are still very early in this particular revolution or this particular phase. We haven't, for example, seen, I was saying the analogy is the nineties. If you think back to the nineties, we had Warp at the end of the nineties, which was for five or six years considered to be the future of mobile, along with things like Symbian, all of which turned out to be utterly pointless. [39:25] We have the Blackberry phone with all the keys to do email, and then we have the iPhone and suddenly everybody knew what the internet was for. The iPhone is that for music, photographs and tools and talking to each other, and now we have all this powerful technology, and we don't know what it's for. So it could be for something that hasn't been seen yet, or it can be for an evolution of something we already use, like a kind of car thing or a home screen saying, I mean, I'm looking on my desktop iMac screen from a few years ago. And I'm thinking, how long will it be before I walk into every home? And there's a wall dedicated to communication, and there's all this kind of future 3D technology that Google was showing at their conference last week, which makes it seem like the person is really standing next to you. Justin [40:13]: That was awesome. Alexis [40:14]: It was pretty incredible and combination of computers, bandwidth, connectivity, and compression. Well also, you know, machine learning to iterate outside of the gaps. So many different things that made it seem like a natural experience, so all of these things are becoming possible with this new platform. And eventually people will have some wow moments. And then they'll say, oh, well, you know, I can't remember what it was like before we have, you know, the [40:43 inaudible] or whatever that will be. Early days. And the developers just have to wait through the sludge in the meantime and shave the yacks. Always shave the yacks. Justin [40:53]: Usually I like to chime in a lot, but I learned so much, especially from the community stuff. And I really appreciate you coming on. I'm an early rabbitmq user when I was at a company called Mahalo. So it was just kind of cool to see that you're like one of the creators. I would say great conversation, Alexis, but you just started talking about the future in visionary. You talk about technology in the future. I really want to have another conversation about this. That seems like another episode. So thank you very much, Alexis, for being with us today and sharing from your experience, how to build a successful open source community and open source company. Is there anything you would like to mention before we conclude? Alexis [41:39]: We sell absolutely anything you want, please use githubs first of all. It really is about making your life easier and if you do that, then you'll find that there are things that we sell as well that will help you scale. Justin [41:52]: We'll definitely start using, githubs. We're already githubs, but definitely weaveworks tools. Alexis [41:59]: Thank you very much. [42:01] Listener. I hope you enjoyed this one do tune in next time. We're really excited about our lineup of guests. We have super exciting guests next week, as well check out the show notes for this podcast at podcast.Curiefence.io. That's C U R I E F E N S E podcast.curiefence.io for the community cloud native podcast. Thanks again for listening tune in next week. Special Guest: Alexis Richardson.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer | Tzury Bar Yochay Guest Flavio Percoco Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the Cloud Native space. Today, we have a special guest joining us from Italy, Flavio Percoco, who is Senior Principal Software Engineer at Red Hat. Flavio tells us about working at Red Hat, why he uses OpenShift, and how he stays relevant and chooses what projects he’s interested in. We also learn the moment Flavio realized he made the right choice about his profession, where he thinks Cloud Native is going, the project he is most interested in now, and his thoughts on why Curiefense is the best community right now. Download this episode now to learn so much more! [00:02:30 (https://podcast.curiefense.io/12?t=150)] Flavio tells us about working on OpenStack and Elastic. [00:04:47 (https://podcast.curiefense.io/12?t=287)] We find out why Flavio uses OpenShift versus another. [00:06:21 (https://podcast.curiefense.io/12?t=381)] Flavio tells us about the CNCF project Kubernetes. [00:07:40 (https://podcast.curiefense.io/12?t=460)] Richard asks Flavio since he’s building the tools for people, how does he stay relevant, how does he know what the clients need, and how does he choose what projects he’s interested in. [00:12:51 (https://podcast.curiefense.io/12?t=771)] Justin wonders what made Flavio want to go back to Red Hat. [00:15:05 (https://podcast.curiefense.io/12?t=905)] Tzury asks Flavio to share his “moment” in life when he realized he made the right choice with his profession. [00:17:50 (https://podcast.curiefense.io/12?t=1070)] Flavio tells us where he thinks Cloud Native is going and how he thinks we can get there. [00:27:14 (https://podcast.curiefense.io/12?t=1634)] Richard wonders why Flavio is doing the exact thing he is doing now, how did he decide that this was interesting to him, and why Cloud Native. [00:30:45 (https://podcast.curiefense.io/12?t=1845)] We learn what project Flavio is most interested in right now. [00:31:54 (https://podcast.curiefense.io/12?t=1914)] Flavio talks about Curiefense, and why he thinks this is the best community right now. [00:35:31 (https://podcast.curiefense.io/12?t=2131)] Justin talks about what really impressed him about Flavio when the GitHub thing came out when they changed from master to main. [00:36:25 (https://podcast.curiefense.io/12?t=2185)] Find out where you can follow Flavio online. Quotes [00:03:56 (https://podcast.curiefense.io/12?t=236)] “You work with like fewer resources and you’ve got to get creative on how to build your regions and data centers, and how to automate that in a way that you can deploy thousands and thousands of those bare metal nodes.” [00:04:26 (https://podcast.curiefense.io/12?t=266)] “The real hard problem comes from day-to-day operations, where you have to actually manage the whole cluster, and all of your regions, and all your zones, and all that kind of things.” [00:08:57 (https://podcast.curiefense.io/12?t=537)] “It’s one of those stories that it works in my laptop, but like, there’s so many gotchas when you actually run yourself, then you put it in production and you actually have people using your stuff that you don’t realize then until you’re actually running it.” [00:09:11 (https://podcast.curiefense.io/12?t=551)] “And many times people don’t run your software through the way you think they’re doing it, and many times they’re not even using it for the stuff you created it for.” [00:11:57 (https://podcast.curiefense.io/12?t=717)] “It’s good to have good processes so that engineers know you know what things to work on and what things are important, but it’s also true that it’s also the engineer’s responsibility or the developer’s responsibility to go and try to find this information because at the end of the day you’re building a software that someone else has to use.” [00:13:02 (https://podcast.curiefense.io/12?t=782)] “To me it was a lot of people, culture, and the fact that Red Hat is an open source company.” [00:13:26 (https://podcast.curiefense.io/12?t=806)] “I have no time to be fighting with people. I have zero patience for people that are not willing to contribute and cooperate and stuff like that.” [00:16:33 (https://podcast.curiefense.io/12?t=993)] “That was the thing that really just like made me fall in love with computers is the fact that you’re bringing in this access to technology, you have the power to bring technology to many people in different places and improve people’s lives and have a huge impact in what society looks like today and what the world is going to look like tomorrow.” [00:19:23 (https://podcast.curiefense.io/12?t=1163)] “It’s extremely important for people to have the right infrastructure to be able to develop whatever they’re doing research on and be able to do it also without having to do a massive upfront investment, which is something that tends to be a massive impediment for people to actually move on into the stuff they’re interested in.” [00:19:47 (https://podcast.curiefense.io/12?t=1187)] “And this is the other thing that to me, not only in the Cloud Native world, but like all the open source is extremely important because I didn’t go to college.” [00:32:20 (https://podcast.curiefense.io/12?t=1940)] “It really excites me that from day zero where we’re all being open to making the right calls that will favor community, and it will favor contribution over just pushing features and just like adding stuff to the software.” [00:33:08 (https://podcast.curiefense.io/12?t=1988)] “But, even at the cost of moving slower or taking care of very important things like having proper CI, and I don’t mean CI in terms of we need to test our software, I mean CI in the sense of if someone comes and submits a PR and that PR is not correct, what is the best way for us to communicate that to the person that is contributing that PR.” [00:34:37 (https://podcast.curiefense.io/12?t=2077)] “I’m extremely excited about this and this is something that it’s close to my heart, like in communities and cultures are really close to my heart way more than software is.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Flavio Percoco Website (http://flaper87.com/) Flavio Percoco Twitter (https://twitter.com/flaper87) Flavio Percoco Linkedin (https://www.linkedin.com/in/fpercoco/) Red Hat (https://www.redhat.com/en) Red Hat OpenShift (https://www.openshift.com/) OpenShift Assisted Installer-GitHub (https://github.com/openshift/assisted-installer) Metal3 (https://metal3.io/) OpenShift Assisted Installer #278-GitHub (https://github.com/openshift/assisted-installer/issues/278) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript Flavio [00:02]: I wasn't into computers until I was probably 18 years old. In my childhood I only played outside in the mountains, in the rivers. I used to run with a machete in my hand, just like cutting grass. Man I'm from South America. That's how we do it. It is like that. I say this because I'm actually happy that I had this childhood when I could just like interact with nature and eventually just find this thing that is going to be the tool and the stuff that I want to. Richard [00:33]: Hello and welcome to Committing to Cloud Native, the podcast where we talk about the confluence of Cloud Native and open source. Super excited about our guest today, which is kind of what I say for most guests, but this one in particular, just because he is awesome energy and helped us deal with some squabbles in the beginning with technology, which makes me think that open source and cloud native will also be dealt with appropriately. Very excited about that. Before I introduce Flavio. Oh no, I already did. I want to introduce the other panelists here. We have Justin Dorfman. Justin, how are you? Justin [01:05]: I'm doing great. How are you, Richard? Richard [01:07]: I'm good. Thanks for asking and Tzury Bah Yochay. Tzury how are you doing? Tzury [01:12]: It's bah-yo-hi, not bah-yo-hay. Richard time for you to get it right. Richard [01:18]: Tzury Bah Yochay Tzury [01:18]: There you go. Episode number eight or number 10, who knows and you got it correctly. I'm fine by the way. Great day today. Richard [01:28]: Glad to hear it. Flavio Percoco. Did I get that right? Flavio [01:33]: You did yeah. Richard [01:39]: It's so good to have you. Flavio is joining us today from his beautiful mansion in Lake Como, because I can't imagine anyone living in Lake Como and not having a giant awesome house on the lake. He's a software engineer at Red Hat. Flavio, how often do you just sit and look at your lake mansion and think this is it, I've made it? Flavio [01:58]:Quite often. I'll have to invite you over to show you what a mansion actually means in Italy. It's like very packed. It's not what you're thinking right now. The only people that have mansions here is like [inaudible 02:11] those people. I guess it's fine, but like, it's awesome. I go there quite often when I'm in town. It's great to sit there, eat an ice-cream. Richard [02:23]: You're not always in town, because you often do a lot of conferences and you're a speaker and your travel the world all the time. Right now, you're working on OpenStack. You talked a bit about that. Flavio [02:32]: I used to work on OpenStack. I worked in OpenStack for like six years. I don't know. Someone was asking me this the other day, like when did you start? Like, what was the name of the release that you started? I don't know. [inaudible 02:46] one of those. Then worked there for like six years more or less. Then I moved on to working on OpenStack and Kubernetes at the same time because OpenStack was not busy enough, I guess. I started working on integrating and eventually just went on from there for a couple of years, took a break from that and then now I'm back. I moved to Elastic and I started working on something completely different. It was always related to infrastructure, but I do not have to build the software that provided that infrastructure. I was just able to use it and build stuff on top of that infrastructure and kind of put my hands back into dev ops, which is something that I hadn't done in like several years ever since I started working at Red Hat and then spent a couple of years at Elastic and back at Red Hat. Full-time there working on OpenShifts, mostly. I had mostly focused on OpenShift deployment workflows on bare metal nodes. Everything that has to do with dealing with their data repair centre or bare metal nodes, and how to make that easier for customers and all the TACOs and edge cases and that kind of thing. It's an excited fill. You work with like pure resources and you gotta get creative on how to build your regions and data centers and how to automate data in a way that you can deploy thousands and thousands of those bare metal nodes without actually bringing everything down, I guess. Also the hardest part I think, of this whole process is not so much how to automate the whole process, but they're zero operations. To some extent they are easy. There are like a self problem. This is a problem that we're trying to like make easy for everyone. The really hard problem, comes from data operations, where you actually have to manage the whole cluster and all of your regions and all your zones and all that kind of things. That's when the actual real infrastructure management really starts. If you don't provide the right tools in a way for people to actually monitor infrastructure easily, then it's hard to really scale it at the levels that we want people to scale this thing. Richard [04:48]: How does OpenShift provide that? You said like it helps you deploy your bare metal nodes. I'm pretty sure I know what that means, but why OpenShift versus any other? Flavio [04:57]: Well, so this is something that everyone had with deliveries, that is a very important problem to solve. The only reason I say OpenShift is because that's Red Hat's product, really. That's what we go and sell to customers and people that are using it and still open source so we really sell support. But it's all based on Kubernetes. This specific tool that I'm working on is still open source. People can use it. It relies a lot on some of the OpenShift internals, but I will build their pieces around this tool that we call a system installer, assisted service, all the related pieces around it, that all pieces are all part of the obscene community. They all have an option community, part of the CNCF. They all follow the Kubernetes release cycle and try to be as community friendly and as open source of friendly as possible, which is perhaps full fashion upstream first. Richard [05:50]: Nice. So it's just an installer is what you're mainly working on at the moment. It sounds like you've touched like everything, which is awesome. Flavio [05:57]: Yeah the assistant service is just a piece of the system installer. It's mainly the thing that I'm working on right now, spending most of my hours there and trying to get the whole story straight, because a system installer can be not only bare metal, it can also do [inaudible 06:11] from cloud providers and do all that stuff. the part that I'm mostly interested in honestly, it's all there. They're [inaudible 06:17] not use-case of system installer. Richard [06:24]: You mentioned CNCF there, which one of these is the CNCF project? Flavio [06:27]: Well, Kubernetes is. I'm just kidding. That is a big part of it. There's another community, it's not so much which of the projects is a hundred percent part of the CNCF, but rather what parts of these projects are already part of the CNCF. For instance, like you see Kubernetes as part of the CNCF, but then there are a bunch of internal APIs, like the cluster API, the machine API, and some other internal interfaces that allow people to be all these tools and services that interact with the Kubernetes cluster and that allowed them to manage all the resources like their metal nodes and like VMs and networking and some other parts of the whole infrastructure environment. I would say the main project that is part of the CNCF that our [inaudibel 07:16] can work on is really Kubernetes and everything that is around it that will allow us to actually provide these basic install experience that we want people to have when deploying OpenShift. Richard [07:27]: I have this like existential question, which is, you're talking about all these projects you've been working in this industry for years and you're providing services for people who are setting up their infrastructures, but that's not what you're doing. You're building the tools for them to do that work. How do you stay relevant? How do you know what the clients need? How do you choose what projects you're interested in, seeing is how you're not a person throwing up all these instances every single day. Flavio [07:53]: Yeah, it's funny you're asking that. That is an interesting question. It's a question that I really posed myself the first time when I joined Red Hat, the first time back in 2013, where I started working OpenStack. The OpenStack story, really for Red Hat when I joined them, we were really few people compared to how big the team is right now. One of the things that we kind of like stress a lot in the OpenStack community when we started building new services and growing the community itself is, how do we get data from the field? Let's look like Red Hat for a second. Let's talk about the OpenStack community. As a community, assuming we don't have any company behind us, how do we get people to actually tell us how they're using software so that we can make it better? Something that is, I don't want to generalize, but I believe like many developers miss the fact that if you develop things in a box, if you develop things in your tiny oral without looking at how people are running your software, you're really developing a software that works for you. It's one of those stories that it works on my laptop, but like there's so glitches when you asked to run it yourself there, and then you put it in production and you actually have people using your stuff, but you don't realize them until you're actually running it. Many times people don't rub your software the way you think they're doing it. Many times they're not even using it for the stuff that you created it for. How do we get all these people to actually give us that information? What we started doing in the OpenStack community, it was made in this subset of the community, so to speak, that was the operators community. How do we welcome these operators so that they feel that they can be part of the community, like a hundred percent OpenStack community contributors, even if they don't commit any code to the OpenStack database. We pushed for this, established these and try to like make them part of their technical committee and try to make them a central piece of the entire community. And something very similar is what I try to do from within Red Hat so there's these things that you can do in an open source community where you try to like bring more contributors to it. Bring people that are actually using the software and being part of the decision-making process, because at the end of the day, they are the ones actually scanning your software. They bring like some use cases that you're like, I have no idea that you can actually do that with the stuff that I was spreading. Something similar is this for a company like Red Hat. But like Red Hat is a big company. Then it has like many people taking care of all these product management and all those things. As an engineer, I take care of basically chasing down product managers so that they tell me what customers want. When we were building this OpenStack team at Red Hat, I remember just going to the product managers of this software that I was working on and telling them, I want you to bring me all the customer's experience. I want to know how they're running the software, all the problems that they're running into. There was a lot of that. Also a lot of the tracking with our support team. Red Hat's support team is awesome. They take care of so many issues. We're level two, like engineering is like two or three, I don't know, like you put a number on it, but like many times we don't even know all the issues that they're working on, because they can solve them. But many times knowing what issues they are solving for the customers will bring more information while we can actually improve it. There's a lot of like chasing that, at least for me at the very beginning. It was a lot of chasing down product managers and making sure that it could be tracked more with support. I would collect all this information and either just build my own proposals, or like bring in up backup streams that people would know how some of their customers were using the software and what other things could be improved. Many of these conversations just end up being in requests for changes upstream, request for improvement and new proposals of new features or things that are bugs that need to be changed. It's still true. It's still like that. At the end of the day, it's good to have good processes so that engineers know, what things to work on and what things are important, but it's also a true belief, but it's also the engineer's responsibility or the developer's responsibility to go and try to find this information. Because at the end of the day, like they're building a software that someone else has to use, for various use cases. Some of them are mission critical. Some of them are just someone running OpenShift on the desktop. It's fine, but they're all important. But they are all people that are interested in running the software. Knowing this use cases, you're not going to be able to solve them all, but a least knowing that they exist is definitely gonna make you a better developer, I believe. Justin [12:37]: Got it. Like on Git Hub, you're part of two really big organizations. Kubernetes, and the Kubernetes special interest group. You could work pretty much anywhere with those in mind. What made you want to go back to Red Hat? What is it exactly that made you say Red Hat's the place to be and I've been there once and I'm going to go back and do the experience again? Flavio [13:03]: To me it was a lot of people, culture, and the fact that Red Hat is an open source company. To me, culture is something that is extremely important. People say, if you want to really be contributing to open source software, we have to be thick skin, be this super rough human being. I have no time to be fighting with people. I have zero patience for people that are not willing to contribute and cooperate and stuff like that. I want to work on things that excite everyone. I have an impact on digital work. If you look at this software that we normally work on when we're at Red Hat is, something that many times people don't even realize that they're using. People don't even know that they depend on the software. But then I'm like, why not? It's a software that makes me happy to work on. The people that I work with, they're great human beings. The culture of the company puts people first over many other things, many costs. That is something that for me, is like super valuable, especially in these times. Especially in this period of like given the tech industry where it's easy to burn out. It's easy to be fed up with many things. I would summarize it in people, culture, and definitely they try and just what we're working. Tzury [14:24]: Well Flavio, you mentioned that the software you're working on makes you happy. Well, I can tell from the little time I know you that you do what you like and you like what you do, which is a good thing. But everybody I know from our profession, people in our industry, most of them, at least two moments in their life. The first moment is the moment they knew they want to go into computers, software and so on. Then the second one is a moment when they look back and say, well, I think I made the right choice. This is indeed my profession. This is indeed my trait. Would you mind sharing with us those moments? Flavio [15:07]: Well, the first moment is probably easier for me. I tried many things. I have many ideas. I've always had many things I want to do. It's probably one of my, I wouldn't say my strength, it's actually the opposite. That's one of the things that I tend to spread myself too thin on many things, call it curiosity, call it whatever you want. When I finished high school, I guess I was looking into many things. I even signed up for psychology and medicine and a bunch of other things, but then eventually because of the same curiosity, I ended up taking an online course on it. I was all BHP, my SQL all the way down and some HTML on there and that just clicked with me. I started like playing with it a little bit, like doing all the changes and all the logic, but it was not that. It was not like love at first sight. The thing that really made me fall in love with this is what I did after. I did my internship, like working on web development and stuff like that. Then eventually I moved into some other company that was building a Linux distribution, mostly focused on assistive technology. It was built by people or people with like vision problems and people with motion problems to actually be able to use a computer. I was working on that distribution and I was working on the software back in the day. It was called Orca. I created this other software that was called mousetrap, which would allow you to move the cursor just with your webcam in your face. That was the thing that really made me fall in love with computers. The fact that you are bringing this access to technology. You have the power to bring to knowledge to many people in different places and improve people's lives and have a huge impact in what society looks like today and what the world is gonna look like tomorrow. That is a thing that really made me completely fallen in love with it. I ended up working on that for a while and then eventually ended up moving to a different country side, to switched jobs for obvious reasons, I guess. Then eventually I started doing something else, but it's always been important to be working on something that I believe will have a significant impact in the way we use technology today, in the way the world works. Richard [17:22]: You talked about going round to the product managers and making sure that they get feedback to you, because it's your job as an engineer to listen to your users, and to make sure that you're able to build tools that they could actually use. You talked about how you want to improve people's lives and improve their access to tech. You must have a very unique position. Being someone who has that mindset and has worked the places you've worked, to know how Cloud Native infrastructure can actually help people's lives in the future. I'm wondering where you think it's going and how you think we can get there. Flavio [17:55]: I would say it's going in the right direction. That would be my first answer. I think we're making many things right. It's interesting. In some of the cases like when we talked about telcos they're to users, because you end up enabling the infrastructure that eventually enables communication across the world. You're providing the software that these massive companies are actually using. If you think about it, when like I'm not going to say an NSP. But eventually when you're using this carrier and you're calling, they're making goals to use, like, just imagine in your head, it's like right now on this antenna, this is happening because like the software that I built and I worked on, is actually handling all this traffic. When people can go, I don't know, like emergency number 911. Like even though they're like more less traumatic use cases where you're just going to your loved one and you're enabling communication for people, those are very important. These days, it's Red Hat, Telco, Edge, that kind of things. But if you also look, [inaudible 18:56] which is something that Red Hat is starting to get into, it's pretty awesome to know that you're also enabling and improving the way all the automotive world is actually working and the way it's evolving. They're also like more, I don't want to say simplier, because I don't want this to sound like they're not as important or as impressive. But even just enabling research for universities, for colleges, like it's extremely important for people to have the right infrastructure, to be able to develop whatever they're doing research on and be able to do it also without having to do a massive upfront investment, which is something that tends to be like a massive impediment for people to actually move on with the stuff that they're interested in. Making this software that people are free to use at any point and that they can learn from. This is the other thing that, to me, I'm not only in the Cloud Native world , but like all the open source is extremely important. I didn't go to college. Everything I learned it by reading someone else's code. I learned it by working on open source. I learned it by contributing to open source. I learned it by being in RC and talking to people and having people actually explain things to me when I didn't understand them. To me it's extremely important to have all this software that other people can use. Cloud Native in particular is extremely important because that is what we want the world to move into. We want things to be accessible through the internet, which I believe should eventually become a human right. And just have people be able to just like scale and put everything that they need online and have easy access to all these technology, that for a long time was extremely expensive and really hard to get your hands on. Today you can just click on it and you will have access to all this technology be able to do what you need to do. Richard [20:51]: It's clear to you how you justify working in Cloud Native and how you tell the story to yourself that this is valuable work. It's clear to me that that's a valid story. That's awesome. It's really great to be able to do that stuff and to enable people to call their loved ones and research possibly at the same time. You answered it just fine. It's really cool to know that you didn't go to college, that you learned everything together because that really shores up this image I have of awesome developers being community-based first. Because you had to learn from other people. Flavio [21:25]: Totally. I'll tell you even like, I wasn't into computers until I was probably 18 years old. In my childhood, I only played outside in the mountains in the rivers. I used to run with a machete in my hand just like cutting grass. I'm from South America. That's how we do it. It is like that. I say this because I'm actually happy that I had this childhood where I could just like interact with nature, and eventually find this thing that is going to be the tool in the stuff that I want [inaudible 22:01] Justin [22:03]: I'm really glad that you did bring up the college thing because I don't know about where you were from, but for my parents, it was a very big issue between us because I knew what I wanted to do. I had the same thing with you. Like I saw the source code, I saw the future. So I thought, but it ended up turning out pretty good for me. I'm not going to lie, but I think maybe not so much now, but there's always been this kind of stigma, at least for me, especially like when you have a LinkedIn page and it says, what school did you go to do? Then you put Camino Rio high school. It just looks kind of weird. But at the same time, I think this day and age, that stigma is kind of going away because really the curriculum can't keep up with the technology in a way. Things are changing so quickly and more and more people are just learning on their own or learning through stack overflow and all these other channels that is not traditional education. Thank you for bringing that up. Because to me, I was like, oh, he definitely, probably has a CS degree and all that other stuff, because you're a principal engineer at Red Hat, that just got acquired for $38 billion. It's like a huge company that makes a huge impact in the world and the open source space. I hope more people listening to this don't feel bad about not going to college or not being able to afford it or whatever the case might be, because you can't succeed. Flavio [23:37]: You definitely can. I think people should definitely not feel bad about not going to college. But I also think that if you have gone to college, it's absolutely fine. They are things that you learn in college and there are things that you learn on the field. I think the only events that people didn't go to college have, or there are people that went to college is that they get experience from the field right away from being there. Now it comes with a cost and there are many basic things. By basic I don't mean like obvious things, but like there's many theories that people that don't go to college to know unless they actually need them. This is how I've been like learning the things that I need. But like you, you face an issue, it sparks your curiosity and you'd go to read about it, learn about it. Then you move on and you keep going and then you go as deep as you have to go to actually be able to do the job that you have to do. Now, these are many things that people that actually go to college and go to their regular curriculum. They will know from their studies, which is also like, it's all great information. I'll say it's awesome information that maybe eventually when I retire, I'll just go and read all these books because I still think it's very interesting. My fun anecdote about this is that I once tried to interview with Google, they rejected me right off the bat because they asked me a very basic question that you would learn in the first semester of computer science, which is what is the speed of rotation. I was like, writing this function. That is like, I don't know, like, O N or whatever. I was like, dam I got them. I don't know even know what this is. I was like, can I Google it? In the middle of the Google interview. Richard [25:16]: I've been rejected for more interviews for that question than any other. I kind of refuse to learn because I'm just so sick of being asked. It's not relevant. Flavio [25:30]: Right in the middle of the Google interview I was like, can I Google this thing and the guy was like, yeah sure. I did it. I was like, oh, okay. I understand the concept. I understand what it means. But it's clear from our current interaction that I didn't know this basic thing that you believe is extremely important for my job. If you ask me now, I will tell you, like, I would never ask anyone that question on an interview. It has to be a very specific use case or like very specific technology that you need to be working on. Like something that is extremely embedded. It has to be absolutely fully optimized, whatever you want to put it like in whatever way. But now years and years after that, I have a different view of that interview. That very same day I was like, had I gone to college, I would have known this thing, right? I would would have been able to answer this question. Richard [26:20]: We could not disagree more. Speaking to someone who has gone to college for more years than is really necessary. College gives you absolutely none of this. In fact, the ability to decide to know what you're interested in, dive down and research it as deep as you want and then move on to something else. It's something they don't teach you in college. They teach you how to do really badly at everything all at the same time. Both of you actually benefit from not having gone. I have two master's degrees. I know what I'm talking about. That all having been said, you mentioned earlier about diving in. You mentioned that like in the very beginning of the podcast. I forget the exact quote, but I've been wondering for a second now. Since you have this great experience of like being interested in some stuff, which meant you maybe don't have the background other people have. You haven't read code 101. Your code complete two may not be in your shelves, which is probably fine. It's a really big book, but I'm curious. Why are you doing the exact thing you're doing now? How did you decide that this was interesting to you? How is it more interesting than running around with a machete in a river next to the mountains? Why Cloud Native? Flavio [27:27]: First and foremost, I didn't say this was more interesting that running around with machete. I said, I used to do that. I wish I could go back to doing that, but I also have to eat. Making some money is part of the whole deal. Everyone is attracted to different challenges. I think the main thing under this whole thing that I really enjoy working in is distributed systems. Everything that has to do with multiple computers, multiple software, having to interact with other parts of the architecture, through the network, then having to synchronize and having to solve these issues either to work with concurrency or probabilism, or just distributing the load or whatever. This really the thing that attracts me the most. There's a lot of that in the stuff that I'm working on right now, then having all these notes, different hosts, and metal nodes coming up and how then all the different agents and tracking the service and making sure that the whole deployment process is working properly. I've always been interested and attracted by distributed systems. Because I guess it aligns a little bit with my obsession of communities and culture and having everyone actually agreeing with each other. That actually pushes me into trying to solve distributing issues and try to bring all these systems into our agreement and to state that we are all happy with. How did it get here? It's a good question. I actually started working on the search engine back in a different company. It was actually based on the last search before Elastic the company existed. It was very earlier days before Elastic started working on that and trying to build NLP natural language for a system, that goes on top of it. It was part of like a bigger architecture. I was not so much into writing the algorithm itself. It was not something I was super interested in. I wanted to deal with, what happens to the data for the moment it gets into our services, where it goes, what does it store? How are we going to process it after? how is it going to end in the search engine? How is we're going to use the results from the search engine and how we're going to propose this to the user and how we're going to do this. Back in 2010, it was like terabytes and terabytes of data. That was like a massive amounts of data. I started working with that. That was a startup like normal, regular startup story eventually ended up failing. But after that, I ended up moving to the Red Hat when I started working on OpenStack and was still attracted to the same issue site. It's a distributive computer provider and cloud provider. How are we going to make all these different pieces work together? In this case, it was very, there were all like very separate and different pieces. Like one was providing [inaudible 30:07] The other one was providing images. The other one is providing like [inaudible 30:12] and all those things. It all has to work. Within each of these services that were like different parts I had to travel to. It's always been around distributing systems in general, as the part that really attracts me. I think the whole theory behind consensus and all these algorithms and how to make systems interact with each other in a way that they are full tolerant is something that I find interesting, which also applies to security defense in great detail, in great extent. Richard [30:40]: [inaudible] is a huge field. Right now, this week, what are you most interested in? What project is most interesting to you that you've given into? Flavio [30:50]: I don't think I've had to deal with a lot of tolerance in the last months, I guess. Concurrency I think is definitely a part of what we're doing and the assistance, or because of the need of deploying all this nodes concurrently, and making sure that old data state is properly managed and that all the agents have the information that they need to move on. I guess the hard part comes when we need to make sure that all the network settings are set properly and all the nodes. It doesn't end up like when you're dealing with a list like everyone, you also have to deal with the probationing network, which is a separate network from the one that the actual [inaudible 31:34] end up in, which has its own challenges, especially because then you have to connect all these hydraulics for that. They have their own problems. I'm probably not going to go too much into that, but there's a different kind of distributive challenges that I'm dealing with right now that don't really have to do a lot with concurrence at this moment. Richard [31:54]: If you had to choose out of all your different projects, I know you went back to Red Hat because the community is great. If you had to choose a community that you think is doing the best right now, or that really makes you feel good and warm inside, fuzzy, if you will. Which would it be? Flavio [32:05]: I'm super excited with the community that, [inaudible 32:10] I mean it. It's still small, but it really excites me that from day zero, we're all being open to making the right calls, that will favor community, and that will favor contribution over just pushing features and just like adding stuff to the software bite. Cause being very focused on, we want to build a community like that. The technology is there, the technology's there, the challenges are there, the ideas on where to get it next, they're all there. Now we need to make it happen. How can we make this happen in a way that we're not only going to make it happen, but also gonna make it happen in a way that people will feel comfortable jumping into the project and jumping into slack or the repo and contributing to the project. We want it to make it happen like ultimately fast, which is like not making a community. We'll just code like crazy and just push everything. But even at the cost of moving slower or taking care of very important things like having proper CI, and I don't mean CI in terms of, we need to test their software, CI in terms of, if someone comes and submits a PR and that PR is not correct, what is the best way for us to communicate that to the person that is contributing that PR. It's not so much about telling them your code is wrong, or protecting that code to going into the repo. But it's about communicating properly to the rest of the community or the rest of the contributor's like, hey, this code is not working properly and this is why, and have that very expressive way to actually say this, because I don't know if that test suite is extensive enough and it's super clear, so people, when they click on the link and they see the loss, it's obvious to them what's going on and how they can learn more. The same thing applies for all the docs that are being built. How can we make this feature and document it in a way from day zero, that is clear to people, what it is meant to do not what we're after. The community is not too big, but again, at the cost of moving slower, many things are being done for the community and for the thing that we want to build and that we want to be like this security project, or like all the proxies out there, basically. I'm extremely excited about this. This is something that is close to my heart, like in communities and cultures, are really close to my heart, way more than software is. To me, this is extremely valuable is something that you would think that you would find pretty much everywhere nowadays, considering how much this has been talked about and social media, like Twitter, everyone is saying like community. You gotta be nice and we're done with all the jerks and we're all above that. Then eventually you go to some repo that is just full of those people. I don't want to rant a lot about it, but I'll see the glass half-full and say that there are good cases, like the pure fense community that are taking all the right steps, where community and people come first. Despite once again, at the cost of moving slower. Justin [35:30]: Just to add on that real quick, it really comes from the top because here's one thing that really impressed me with Flavio, is when the whole Git Hub thing came out, when they're changing from master to main, the fact that he brought it up first, I was like kind of hesitant. Because I was like, oh, I don't know if this is my place to do it. But he said from the get-go we just switched to main because we want more contributions and we want to be more inclusive to the community. I'm like, sorry, where did you find this angel? He gets it. Richard [36:02]: I just opened a drive by issue on a system installer, asking for a contributing guide. I'm really excited about that. Thank you so much, Flavio. I think we're coming up to the end of our time here, because podcasts are mortal things. Also food is very good. Flavio, thank you so much. This was awesome. Where can people find you online? Where can they read more about the things you do? Where can they find your tweets? Flavio [36:32]: Funnily enough, as much as I complain about Twitter, I'm still on Twitter. That would be flaper87. And yes 87 is the year I was born. Just in case you're asking. Justin [36:48]: My sister was born in 87. Flavio [36:52]: There you go. Best year ever. That would be all my links out there. i don't tweet as much. I used to tweet way more than I do these days. I'm still there. I still enjoy reading some of the tweets and blocking the ones that I don't like. Justin [37:05]: Tzury I think we should get him a mic and then he can be on this podcast as a panelist more often. Richard [37:17]: Thank you so much. Flaper87, everyone looking forward to our next podcast, Flavio looking forward to seeing you around. Catch you later. Special Guest: Flavio Percoco.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Tzury Bar Yochay Guest Connor Hicks Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the Cloud Native space. Today as our guest, we have Connor Hicks, Maintainer of Suborbital and Staff Software Developer, Product Discovery Lead at 1Password. We will learn all about Suborbital and how it’s going to make Cloud Native better, and the three projects that he refers to as the building blocks. We also find out more about 1Password, how they are using 1Password with Cloud Native with their projects, and their most recent launch of 1Password Secrets Automation. Connor talks about his partnership with HashiCorp, and we hear his thoughts on the complexity in the Cloud Native world. Go ahead and download this episode now because there is so much more we talk about with Connor! [00:01:41 (https://podcast.curiefense.io/11?t=101)] Connor explains what Suborbital is and how it’s going to make Cloud Native better. [00:03:04 (https://podcast.curiefense.io/11?t=184)] Microsoft, Google, Intel, and Mozilla want to move WebAssembly beyond the browser, and Conner explains what this means. [00:05:00 (https://podcast.curiefense.io/11?t=300)] Connor tells us a little bit about himself and how he got into computers and software, and why he decided iOS development was not for him. [00:09:46 (https://podcast.curiefense.io/11?t=586)] We find out more about how many front-end applications there are today using WebAssembly, and he mentions Figma and Internet Archive. [00:10:33 (https://podcast.curiefense.io/11?t=633)] Connor explains two ways you can take advantage of Suborbitals projects. He explains the three projects that he refers as the building blocks: Graph, Reactr, and Vectr, which together form Atmo. [00:14:28 (https://podcast.curiefense.io/11?t=868)] Justin wonders since Connor is not looking for sponsors, donations, or patrons, how is he sustaining this and what’s the plan. Also, Justin wonders if he’s considered submitting any of his projects to the CNCF. [00:15:25 (https://podcast.curiefense.io/11?t=925)] Tzury asks Connor if he’s doing all this work by himself or if he’s collaborating with others, and Justin asks how he’s managing his community. Connor talks about their recent launch of 1Password Secrets Automation, and how they’re now also offering Cloud Native software to their customers to run, which are the 1Password Connect Server, 1Password SCIM bridge, and 1Password command-line tool. [00:19:35 (https://podcast.curiefense.io/11?t=1175)] Connor tells us about his partnership with HashiCorp. [00:20:23 (https://podcast.curiefense.io/11?t=1223)] We find out how Connor feels about the complexity in the Cloud Native world. [00:25:05 (https://podcast.curiefense.io/11?t=1505)] Tzury asks Connor if Atmo is used to replace obligation from development or if it used to empower existing ones, and if it’s extendable and pluggable. Connor also tells us about using programming languages that they support such as Rust, Go, Swift, and AssemblyScript. Connor mentions using Unicorn Platform in helping him design his website. [00:29:05 (https://podcast.curiefense.io/11?t=1745)] Find out why Connor recommends not using Atmo for production use yet. [00:30:14 (https://podcast.curiefense.io/11?t=1814)] We learn what the available server-side technologies are to plugin and to integrate with WebAssembly modules or WebAssembly code. [00:34:09 (https://podcast.curiefense.io/11?t=2049)] Connor tells us where the 1Password name came from and where he came up with the name Suborbital and Atmo. [00:40:28 (https://podcast.curiefense.io/11?t=2428)] Connor tells us how he would like Atmo to replace Lambda in some ways. He tells us he would love to get feedback on the Suborbital projects so you can join the Suborbital discord, file issues in the GitHub repos, or join the GitHub discussions in their Meta Repo and reach out to Connor. Quotes [00:01:51 (https://podcast.curiefense.io/11?t=111)] “It’s a family of open source projects and each of them individually may not stream WebAssembly, but that is actually the main theme, enabling WebAssembly specifically on the server side.” [00:10:53 (https://podcast.curiefense.io/11?t=653)] “And that’s because the projects are organized to be a set of building blocks and then a platform built on those building blocks. So, the three projects that I refer to as the building blocks are Grav, Reactr, and Vectr.” [00:14:43 (https://podcast.curiefense.io/11?t=883)] “It’s something that I just love doing.” [00:15:14 (https://podcast.curiefense.io/11?t=914)] “Once WebAssembly as a whole kind of builds up a bit more steam and once the user base grows a little bit, it’s definitely something that I want to think about doing.” [00:17:22 (https://podcast.curiefense.io/11?t=1042)] ”Yeah, there’s a little bit of use of some of the Suborbital building blocks.” [00:17:52 (https://podcast.curiefense.io/11?t=1072)] “And so, we’ve built up quite a lot of experience running Cloud Native at 1Password.” [00:18:37 (https://podcast.curiefense.io/11?t=1117)] “There’s been a bunch of lessons learned, because not only are we running Cloud Native software to power 1Password, but we’re now also offering Cloud Native software to our customers to run.” [00:19:41 (https://podcast.curiefense.io/11?t=1181)] “So what you can do now is you can actually, there’s a plugin for HashiCorp vaults with 1Password Secrets Automation so you can actually make your 1Password data available through HashiCorp vault using their backend system.” [00:20:33 (https://podcast.curiefense.io/11?t=1233)] “So, I definitely have thoughts here and that’s one of the main drivers when I’m building Suborbital.” [00:22:36 (https://podcast.curiefense.io/11?t=1356)] “But you can imagine where a system that can intelligently decide where to run the different pieces of compute that comprise your application, and that’s something that I’ve been calling decomposed computing.” [00:23:54 (https://podcast.curiefense.io/11?t=1434)] “Absorbing complexity where it makes sense is always the right call.” [00:26:08 (https://podcast.curiefense.io/11?t=1568)] “And I won’t go off on too much of a tangent, but I do believe “Go” is the best thing to be using for Cloud Native right now.” [00:42:40 (https://podcast.curiefense.io/11?t=2560)] “And if there’s one kind of idea that I could leave off with is that I’m hoping a couple of years from now, WebAssembly will just be an implementation detail. I don’t want WebAssembly to be the headline feature on my website in three years, I want it to be a footnote.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Connor Hicks Twitter (https://twitter.com/cohix) Connor Hicks Linkedin (https://www.linkedin.com/in/connor-hicks-05044166/) Suborbital (https://suborbital.dev/) WebAssembly (https://webassembly.org/) 1Password (https://1password.com/) 1Password Secrets Automation (https://1password.com/secrets/) 1Password SCIM bridge (https://blog.1password.com/scim-bridge-release/) 1Password command-line tool (https://1password.com/downloads/command-line/) HashiCorp (https://www.hashicorp.com/) Go (https://golang.org/) “Microsoft, Google, Intel and Mozilla want to move WebAssembly beyond the browser”-ZDNet (https://www.zdnet.com/article/microsoft-google-back-bytecode-alliance-to-move-webassembly-beyond-the-browser/) Unicorn Platform (https://unicornplatform.com/) WebAssembly Summit 2021 (https://webassembly-summit.org/) WebAssembly Live! (https://www.webassembly.live/) Cloud Native Wasm Day 2021 (https://cloudnativewasmeu21.sched.com/) KubeCon/CloudNativeCon 2021 (https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/) Suborbital Meta-GitHub (https://github.com/suborbital/meta) Suborbital Discord (https://discord.com/invite/dyTQJhwqgW) Suborbital-GitHub (https://github.com/suborbital) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript Connor: 00:02 The number of layers that you need to be thinking about constantly is crazy. Orchestration, networking. Do I add a service mesh? How do I run my database? What are the differences between the cloud providers, Kubernetes distributions. What's this K3S thing thing over here. There's just so much to think about. It used to be so simple. You would throw up a VM, you would auto scale it and you would run your monolith. I don't think we should go back to that by any means, but I think there's a new generation on the horizon that gives us some of the best of both. Announcer: 00:39 Hello and welcome to Committing to Cloud Native, the podcast where we talk about the interface between open source and cloud native. We're super excited about our guest today. Can't wait to introduce him. Our panelists today are Justin Dorfman and Tzury [inaudible 00:54] and they're going to have an awesome conversation. I really enjoyed listening to it and I really hope you enjoy this conversation. Justin: 01:03 Hey Connor, how are you? Connor: 01:04 Hey, doing well. How are you? Justin: 01:06 Great, good to see you in the flesh. Sometimes it's weird because when you meet someone on Twitter, it's like you got the static image and now like you're alive, full color and everything. Yeah. I saw you host something. This thing called suborbital. Before we go into that, I want Tzury to say hello. Tzury, say what's up. Tzury: 01:29 Hey Connor, how are you? Connor: 01:30 Doing great. How are you? Tzury: 01:32 Thank you for coming on the podcast. Connor: 01:34 Yeah. It's my pleasure. Justin: 01:36 I got questions for days, but we only have 40 minutes. Let's get started. What is, suborbital tell the folks listening to this, what it is, why we need it, how it is going to make cloud native better. Connor: 01:52 It's a family of open-source projects and each of them individually may not scream Web Assembly but that is actually the main theme, is enabling WebAssembly specifically on the server side. It's split up into a couple of different modules. There's vector graph, reactor, Atmo. These are all different projects that when you put them together, they allow you to do some interesting things, when building server side technology. Justin: 02:25 Got it. Ever since I went into the cloud native space, all I hear about is WebAssembly. I think what's interesting actually, ZD net did a story about it, the byte code alliance. Are you part of the byte code alliance? There is this project part of it? Connor: 02:44 I meet with them regularly, but I am not personally a member just yet. I think they actually just opened up for new members like this week. I've been looking into doing that, but most of the members are companies and suborbital is not a company, it's an open-source project. I've been talking with them quite a lot though. I really love what they're doing. Justin: 03:04 Microsoft, Google, Intel, Mozilla. They want to move WebAssembly beyond the browser. What does that mean? Connor: 03:11 It was originally kind of designed as a technology to enable compiled languages other than JavaScript to be used in the web browser. That was honestly still what most people associated with. But in fact, WebAssembly is not specific to the web browser in any way. It was just the first thing that it was adopted in. WebAssembly at its core is a way of representing byte code, compiled code and where you run that, it's up to you. I mean, you can absolutely embed it in the browser and you can run it to power web applications, but there's a ton of other use cases for it. The ones that I work with are made for developing cloud native services software, but there's others such as IOT devices, embedded applications, there's non cloud native, non-browser things like desktop applications. It's really whatever you can think of. There's probably some way of incorporating WebAssembly into it. Tzury: 04:13 We could spend a day talking about WebAssembly, it's greatness. I hope we're going to do a lot with Web Assembling this conversation. I mean, just mentioning the facts, talking about just shortly type language, you can buy into any runtime application without recompiling takes a week, consider nearly impossible or really hard to get done in previous to study compiled languages. We can talk about V8 role in these revolutionary in JavaScript [inaudible 04:43] and WebAssembly falling after that. I would love if you can Connor to take us through the history and the evolution and the revolution of these technology in the recent, I would say, what 10 years is it around, a decade around us, more or less. But before we get to Connor, I'd like myself and our audience to know you a little better. If you can step back a little bit, tell us about your history. How did you get into computers, software, etc? Connor: 05:15 Yeah, for sure. I did a computer science degree in university and I don't think that's so uncommon, but definitely not by any means needed anymore. That's just the way that I went. During school I did a couple of internships. I worked for Airbus in their defense and space division. I worked for WordPress in their VIP division. Then my last year of school, I got an internship at 1password. I was actually in the iOS team that was building their iOS app. After I graduated, I decided that iOS development wasn't for me. I wanted to try something else. Just so happened that there was a prototype sitting in a get repo somewhere of a command line tool for 1password. I asked if I could pick that up and work with that. They thankfully said I could. That kind of took my transition out of the mobile development space and into more of the automation and tooling space. Then from there, I started working more on the systems and back-end side of things. I built a team around integrations and now I lead a team called product discovery there, which is all about research and development and trying out new ideas. Then for the last year and a half or two years or so, I've been working on this idea of suborbital and it started out as a functions as a service system for Docker. It was nice, but there was nothing truly special about it. There was nothing that Kubernetes, Docker, OpenFAST couldn't already do. That's when I kind of took a pause and ask myself, what could I build in the space? Because it was a very interesting space to me, distributed computing, job processing, all that kind of stuff. I love thinking about those problems. I was like, what can I build that would be unique and differentiate itself from everything else out there. That's where WebAssembly kind of came onto the scene. I will say, like I actually built server side WebAssembly before I ever even built WebAssembly that runs in the browser, which I think is strange, but I never even bothered running WebAssembly in the browser. I just went right to the service. That's how my brain works these days. I think in terms of distributed systems. Tzury: 07:31 Well, don't get me wrong. I mean, the world will benefit more from you doing cloud native than mobile native apps. But as you mentioned that you decided iOS development is not for you, I wonder, what was it in iOS development, I mean, I never experienced, I never even tried developing iOS application. I did so many all sorts of development platforms, but never did this one. What was in the process in the technology, in the tooling or that move you away from it? Connor: 08:08 It's none of those things actually believe it or not. I think Apple has some of the best software design and tooling available. I mean, X code has a little bit to be desired. At least it did when I was using it. I've heard it's gotten a little better since I last touched it, but it's none of those things that made me want to switch. It was actually the fact that I just had no desire to build user interfaces. That was the thing that wasn't working well for me. It just became, something that I lost interest in. Even to this day, like building web applications, like front end development, web application, it still doesn't really vibe with me. It's not something that I enjoy doing. It's very cool and it takes a lot of skill and it's something I respect highly, but it's definitely not my cup of tea. Tzury: 08:55 So you would rather developers will use your API application programming interface rather than users using your interface. Connor: 09:09 Yeah. I love having developers as my users and my customers. Tzury: 09:13 That's interesting the transition that language that was mainly design and mainly targeting browser makes its way eventually to the server side. I would say even more far more popular, more common than actually in the browser. I'm sure there's applications that we know is an end user to actually make use of WebAssembly. But every existing server platform. Every LLVM language will have it's WebAssembly. Is that the case? Connor: 09:47 You'll actually be surprised at how many front-end applications there are out there today using WebAssembly. It's definitely not totally saturated yet. There hasn't been a massive wave of people using it, but it's more than I think you would expect. Some recent ones that have come up is for example, Figma the very popular design application. They use that to help rendering. One of my favorites as the internet archive, using it to emulate flash, to be able to preserve old flash. A bunch of gaming applications as well that have come up in the last little while. But it's one of those things that you have no idea that you're interfacing with it, right? When it's running in the web browser, you have no idea that it's being used. But it's more popular than you might think. Tzury: 10:34 Where is the suborbital come into play when it comes to cloud native and service side that should protect applications in WebAssembly? Connor: 10:43 There's two different ways that you can take advantage of suborbitals projects. There's what I call it, the do it yourself way or there's what I call the batteries included way. That's because the projects are organized to be a set of building blocks and then a platform built on those building blocks. The three projects that I refer to as the building blocks are graph, reactor and vector. Graph is what I call a distributed messaging mesh. It's actually an obstruction over messaging protocols and discovery protocols that allow you to really easily write asynchronous code for communicating on the network. Then vector is an HTTP framework for go, not exactly consequential to WebAssembly, but it integrates really well with the rest of the suborbital ecosystem. It's one of my like passion projects. It's not for everyone. It's very opinionated, but I think it's great. Then there's reactor, which is the core WebAssembly runtime. It's a function scheduler or a job scheduler that allows you to run either Go or WebAssembly workloads, either one they're completely interchangeable. It provides the execution environment for WebAssembly, powered by the Wasm runtime under the hood. It allows you to run many WebAssembly modules all at once. It manages their memory. it manages inputs and outputs, and it provides APIs to the WebAssembly sandbox, that developers can use to actually gain the capabilities that they need to write applications for the server. That's called the runnable API. It's something that I've been working on quite a lot lately. It's something you have to do, to interact with that sandbox. Because WebAssembly is denied by default and it means that nothing can access the outside world unless you explicitly granted the permission to do so. Reactor manages all of that on behalf of the developer so you don't actually need to think about it. Then when you put those three together, those three building blocks together, they form Atmo and Atmo is the batteries included version if you will. It means that you don't actually need to set up a web server or TLS or any of the boilerplate that you would normally need to write when you're building a web service, you interact with it, using what I call the directive, which is a declarative format for defining your web services. It lets you define the end points and the routes and all of the steps that you want the application to follow, to handle those endpoints. You do so by composing together functions that are compiled to WebAssembly. A project for Atmo is a collection of functions written in various languages. We support Rust and Swift and TypeScript slash assembly script is coming very soon. It'll build all of your functions to WebAssembly modules. It'll package them all up together with your directive and with a set of static files. Then Atmo will take that bundle and execute the application as you've described it in the directive without you needing to write any boilerplate code whatsoever. There's a couple of different ways that you can approach it. I think it's a nice mixture of control. If you want to go the do it yourself route and have sole maximum control or simplicity and go with the Atm route batteries included. Justin: 14:16 All this is open source? Connor: 14:19 That's right. Justin: 14:22 I saw on your website, you're currently not looking for sponsor, by the way, your websites are beautiful. They're so polished. Love them. But if you're not looking for sponsors donations or patrons, how are you sustaining this? Like what is going on? What's the plan here? Connor: 14:39 I work for 1password. That's my job. I work on suborbital and evenings and weekends. It's my own time. It's something that I just love doing. That's something that I enjoy. I've been working on for almost years now. While it might seem that there's a lot there, it's been building up over a long period of time. It's just really enjoyable for me. I love interacting with other developers through open source. It's just fascinating problems that I like to solve. Justin: 15:06 Have you considered submitting any of your projects to the CNCF? Connor: 15:13 Yeah, it's definitely something I've considered once WebAssembly as a whole kind of builds up a bit more steam, and the user base grows a little bit. It's definitely something that I want to think about doing. Tzury:15:25 Are you doing all this work by yourself on your own? Are you collaborating with others? This sounds like a lot to achieve on a spare time? Connor: 15:35 Yeah. There's definitely some open source contributors that have come in and helped out. Especially on the tooling side with the suborbital, SCLI called Subo, and with a graph project and the reactor project as well. I definitely have spent a lot of time on it myself. Contributors started coming on board maybe six months ago as the popularity of the project started to grow a little bit. A good chunk of it is me, but there's definitely they're contributors as well. Justin: 15:59 How are you managing like your community? Just like have a slack? Basically, Tzury and I started this podcast because we want to learn from other open-source maintainers. If we ask you questions that might seem basic, they kind of are because we're learning. Connor: 16:16 Community is something that I'm learning. It's somewhat new to me. I've done some community stuff through 1password through a command line tool, but I am putting a bit more of a concerted effort on it for suborbital. We have a discord for example, that you can join if you'd go to chat.suborbital.dev, it'll take you right there. A fair amount of interaction honestly happens in Twitter. It's one of the most active places like I've been tinkering with some community analytics apps like Comscore and Orbit. When you look at the graphs, like, sure, GitHub is number one. But actually Twitter is even more active than Dysport and people love to, retweet things. That often lets you go a little bit viral on a particular day. Justin: 17:01 And then you're doing podcasts with the people that find you on Twitter. Connor: 17:04 Yeah. That's like, that's actually pretty much how it's done. I'll have a particular tweet that gets a bit of attention and then somebody will reach out saying, hey, come show off what you're working on this YouTube show or this podcast. I love doing them. There's so much fun. Justin: 17:16 Here's another question. 1password is your bread and butter. How are they using cloud native? Are they using any of your projects like what's going on? Connor: 17:27 Yeah, there's a little bit of use of some of the suborbital building blocks. One password actually started out. However, I think it was well over a decade ago now it was just a Mac app and it's now grown to being, we have an app on every platform. Then in 2015, actually, literally the day I joined the company, they launched one password for Teams, which is the cloud product that we now just know as 1password because almost everybody uses the cloud offering now. We've built up quite a lot of experience running cloud native, 1password did the entire service. 1password.com is a Go system that has treated us extraordinarily well. We've been expanding more recently into integrations. You might've seen two weeks ago, we launched 1password secrets automation, which is an extension and an expansion of what 1password normally does, which is the usernames and passwords and credit cards that you as a person use day to day. We've now expanded that to include infrastructure secrets, Kubernetes plugins, HashiCorp Vault plugin, all sorts of things to manage the secrets that machines and your business needs to run all sorts of different automations around that. There's definitely been some expansion there and there's been a bunch of lessons learned because not only are we running cloud native software to power 1password, but we're now also offering cloud native software to our customers to run. There's the 1password connect server that actually links 1password into your infrastructure. There's the 1password skim bridge that lets you connect with identity services like Okta, and the one password command line tool, which was kind of the way that I started out there. There's plenty of different things that have come in the last couple of years. That makes a lot of sense for 1password. Justin: 19:08 I would have never known that. Tzury: 19:10 I had no idea and I'm glad to hear about it. Justin: 19:12 Me too. This is awesome. Connor: 19:14 It literally launched on April 13th. Just a couple of weeks ago and it was a pretty great launch, got quite a lot of attention. Tzury: 19:21 We're too focus on our products. We no longer pay attention to what happens. Justin: 19:28 I'm like way too close to fire. You need to get back and like, see what else is going on. I don't get it. Are you competing with HashiCorp or you're working with HashiCor? What's going on? Connor: 19:41 No, absolutely not. We actually have a partnership with HashiCorp. What you can do now is you can actually, there's a plugin for HashiCorp Vault with 1password secrets automation. You can actually make your 1password data available through HashiCorp Vault using their back end system. Justin: 19:57 Now that's a smart idea because they are very ingrained into the cloud native landscape. Hooking up with them, I think that's a brilliant move. We've been doing this podcast for well over two months or so. Half of the people that come on this podcast say cloud native is way too complicated and little less than half that say it's easy. It's the most easiest thing in the world. I know you have some feelings on complexity in the cloud native world. Talk to me and the audience about this. Connor: 20:35 I definitely have thoughts here. That's one of the main drivers when I'm building suborbital I'm constantly thinking about the complexity needed to get this system up and running in production. It's a double-edged sword like no other where you have a massive amount of customization and options available to you in this space. But then that just flips around and becomes paralysis of like, what do I choose. Will making a certain decision ruin my system five years down the line if I choose wrong. The number of layers that you need to be thinking about constantly is crazy. Orchestration, networking. Do I add a service mesh? How do I run my database? What are the differences between the cloud providers and Kubernetes distributions? What's this K3s thing over here. There's just so much to think about. It used to be so simple. You would throw up a VM, you would auto scale it and you would run your monolith. I don't think we should go back to that by any means, but I think there's a new generation on the horizon that gives us some of the best of both. That's what I'm trying to do with Atmo. Atmo is designed to give you the best scaling properties of microservices and server list, while still keeping your code base and the architecture of your system simpler. It's enabled by WebAssembly. That's the great thing because an Atmo application is split up into what could be dozens or even hundreds of tiny WebAssembly modules, that are all working in concert to solve the goals of your application. Atmo can distribute that execution among many instances or even across multiple layers of the network, like Edge, Central Cloud, even devices one day. I haven't quite gotten that far yet, but you can imagine where a system that can intelligently decide where to run the different pieces of compute that comprise your application. That's something that I've been calling decomposed computing. That's not an official term or anything. That's just what I call it. Where if your application is already naturally broken up into these tiny pieces, you as the operator, as the developer, shouldn't need to care where those individual pieces are running, whether it's on an edge network in your AWS region or on a device somewhere. As long as it is achieving the business goals that you've described for it. My hope and whether it's through Atmo or through something else, my hope is that we can get to this place where "the network" tries to figure out more of this stuff for you. A lot of the complexity that we see today gets abstracted down a level and the developers and the operators who are dealing with this stuff every day don't need to care so much and they can focus more on what actually brings value to the application. Justin: 23:37 Basically, what you're saying is absorbing complexity and making it hard to screw up. Connor: 23:42 That's exactly right. It's something I believe in. I think especially if you're building products and tools and services for developers, and I mean for everybody else, but developers are kind of what I focus on. Absorbing complexity, where it makes sense is always the right call. Give them the knobs and leavers that they need to customize and use the application the way that they need to use it. But do it in such a way that they can't accidentally break something. There should be no foot guns to be had anywhere in your system. There should be no combination of configuration options that causes them to get into a bad state, whether that's with performance or security or scalability, it should be really hard to screw it up so that they can focus on building what they need to make their application work. Justin: 24:32 That's like the AWS way, if an engineer screws up at AWS, they don't blame the engineer. They blame themselves for that happening. It shouldn't have happened. I totally get that. Connor: 24:45 Yeah. if you would deploy an Atmo application and something breaks or something behaves in a way that doesn't make sense, that's my fault. That's not your fault. Justin: 24:53 Exactly. Accountability. Connor: 24:56 Accountability in cloud native. Tzury: 24:58 Just for me to better understand Atmo's I would say purpose and positioning, is it to replace existing application from development or is it to empower existing ones? Is it extendable? Is it pluggable? Connor: 25:16 Yeah, that's a great question. If you use Atmo on its own, it could very easily slot into an existing microservice system, for example, because it's just a web server and it's a web server powered by WebAssembly and a declared a format and all this fancy new fun stuff, but it is a web service. You could slot it into an existing setup if you want it to. Tzury: 25:40 Just for me to understand. You actually implemented your own web server or is it something that you're writing on? Like existing proxy or the networking? Connor: 25:50 Yeah. It's based on Go's networking stack and it provides all of the abstractions and all of the boiler plate that you shouldn't need to deal with. It handles all of that for you, but it does use the Go networking stack to power all of that. Tzury: 26:06 Well, that's quite powerful and stable. Connor: 26:08 That's right. It won't go off on too much of a tangent, but I do believe Go is the best thing to be using for cloud native right now, of course there's others. Of course, Rust is the fancy thing that everybody loves. But for building cloud native technology right now is the best choice. But to go back to the question of how it fits in, you can kind of think of Atmo roughly in the same vein of no JS where no JS is running JavaScript on the server and includes some APIs to make use of hardware. It makes use of the networking protocols that you need to use. It lets you run a technology that also just so happens to be traditional using the browser, lets you run it on the server. That's the comparison I like to make. If you want to use that same WebAssembly runtime and power to extend existing applications, that's where those building blocks come in. If you want to add WebAssembly execution to an existing Go app, for example, you could use reactor. If you want to add these network mission capabilities to one of your existing applications, use graph. It kind of depends on what your goal is. Tzury: 27:16 Well, this is fascinating. You're actually saying, if I hear it correctly, say an organization already have code base running Go base platform. Now they can add more capabilities and diversity in terms of languages. They can write say service with Rust, compile it into WebAssembly and get Atmo to play along on top of the Go run time, for example. Is that correct? Connor: 27:46 Yeah. If you want it to take some Rust code and run it as a standalone web service Atmo is fantastic. But if you want it to actually embed that Rust execution in an existing Go application, then you can use reactive. Tzury: 27:57 Can I use any language that is compiled to WebAssembly to write an Atmo or any specific frameworks in specific languages are available. Connor: 28:10 Yeah. That's a slightly tricky question because yes, it will run anything that compiles WebAssembly, but if you want to be able to take advantage of the capabilities provided by the runnable API, you have to use one of the languages that we support currently. Tzury: 28:22 Which are Rust, Go, Swift Connor: 28:27 And Assembly Script. Go will come shortly after Assembly Script. Tzury: 28:30 This is so cool, man. Getting it done on spare time. Justin: 28:36 It's so clean. Like you've been to the website, I just love design. I love talking about websites and how they look. Connor: 28:43 Our plug, unicorn platform, it's a site builder that I really like I'll take no credit for actually developing it. I had just put the site together with the tools available from unicorn. Justin: 28:52 Interesting. Wait, does it bright markdown and how does that work? Connor: 28:56 It's a static site builder. It's very much a gooey kind of like a web flow or something like that. Tzury: 29:06 Do you know of users using Atmo in production at scale? Connor: I'm specifically not recommending Atmo for production use quite yet? Not because of stability problems, but because the API is changing very rapidly still. I can't expect anybody to build their entire big system on Atmo when I'm going to be kind of pulling the rug out from under them every couple of months. I'm hoping that will change fairly soon. I'm hoping by Atmo beta five or so the, the API will have stabilized to the point where I can start recommending it for production use. We're on beta two right now. Beta three is right around the corner. However, the building blocks, they are quite stable and I would recommend those for production use. I do know of a couple of startups and a couple of companies that are using reactor, for example, to build custom functionality and run WebAssembly. Nothing for Atmo yet because I've been specifically telling people, try it out, please give me all the feedback that you can. people have built a small hobby applications and some side projects with it. But I'm explicitly saying that it's not quite ready for production use yet. Tzury: 30:06 Can you tell us a little bit about server side technology for runnning in executive web assemblies? Cause I know like Envoy as WebAssembly support for plugins first, it should be filters. What are the available server side technologies to plug in and to integrate with WebAssembly modules or WebAssembly cover? Connor: 30:27 Yeah, for sure. There's actually quite an interesting landscape right now. It ranges all the way from like the low-level runtimes. Think Wasm time, which is run by the bytecode Alliance there's Wasmer and there are some other runtimes like whabam and second state SSBM, these are kind of the lowest level execution engines. Then as you build your way up to like the more high level obstructions and that's kind of where Atmo lives at the high level obstruction kind of application layer. Things in the middle include things like the cross lit project from Microsoft, which is actually allowing you to schedule WebAssembly using the Kubernetes API, which is really quite fascinating. Then you've got other frameworks that are kind of in the same vein as Atmo like Wasm cloud and a [inaudible 31:14] These are other open source projects that are trying to accomplish some of the same goals using WebAssembly. Then if you go kind of even further past that there are some cloud providers and some actually hosted solutions that let you run WebAssembly. That includes Fastly's computer edge platform and the CloudFlare workers platform as well. Tzury: 31:35 I believe those were actually using VA. Is that correct? Connor: 31:39 That's right. Fastly platform is using Wasm time and Lucid because Fastly was one of the founding members of the byte code alliance. But CloudFlare workers is indeed using VA and actually most recently the denim project released support for WebAssembly and one of their fairly recent versions on their new cloud product is also going to be having WebAssembly support. Tzury: 32:02 It's looks like a promising landscape and promising technology is awesome. Justin: 32:09 The plethora of knowledge you hold around is Wasm is Wasm. It's crazy. It's awesome. Connor: 32:17 I'll say one thing is that I'm not a compiler developer. I actually don't know a whole lot about compilers and the low level specifications and all of that stuff. I know a lot of people working on those things. I know people at the byte code alliance and who were in the WebAssembly working group, WebAssembly community group. Compilers aren't my thing. I work at a higher level, like a higher level abstraction than that. The work that they are doing in the compiler space to support WebAssembly, especially across multiple languages is quite impressive. I would definitely suggest checking out talks from like the WebAssembly summit, the recent WebAssembly live event and the upcoming cloud native WebAssembly day at KubeCon Justin: 33:00 Are you attending that? Connor: 33:01 I am giving a lightning talk at Wasm Day. Justin: 33:05 That's cool, man. I was going to ask if you were going too. I'm glad it came up because I probably forgot, but in my notes I was going to say, bring up KubeCon and here it is. Are you representing 1password or are you representing your open source? Connor: 33:24 I am a developer from 1password as part of part of my career, part of my title and all of those things. But the talk is about suborbital so why don't I say both. Tzury: 33:35 Are they like sponsoring that event? Connor: 33:38 No 1password isn't sponsoring that event, but they do participate in lots of events. For example, GopherCon is one that 1password sponsors year after year, actually a couple of years ago we were the diamond sponsor. We were like the marquee sponsors at GopherCon a couple of years ago, which was really fun. But yeah, not this particular event. Tzury: 33:55 By the way, guys, 1password username for the brand of a company. Make sure you will never use the same 1password across all your websites. Connor: 34:04 That's right. Password management is very important. Yeah. Tzury: 34:09 I think that's where they got the name for it right? Connor: 34:12 Yeah. Tzury: 34:15 Because people use one password for everything they said, no, let us take care of it. Connor: 34:19 That's right. That's right. The master password is what I think they refer to, 1password the name is you use your master password to unlock the application. Then it stores, generates and manages completely unique and strong passwords for every different website. Tzury: 34:35 What if you forgot your master password, then you forgot the model of the first car your father had on 1974. What are you gonna do then? Connor: 34:46 We have something called recovery that allows one of your family members in your family account to recover you if you forget your master password, or if you're part of a business or a team, then the administrators of your business can recover your account. Since everything is N10 encrypted, we actually could not unlock your account, even if we wanted to, because the cryptography happens right on your device. We never hold the keys for that. Signing up for a family account and getting a couple of members of your family onboard is always a good idea. Tzury: 35:13 Make sure you keep the relations with your family members. Justin: 35:17 Don't get in a fight and then like, oh shoot. Tzury: 35:22 Connor where did you get this cool names suborbital? Connor: 35:28 I am a big space flight nerd. I love watching rocket launches and I follow all that stuff very closely. That was kind of the inspiration for it. But the main reason why I chose this name in particular is that the GitHub username was available. Justin: 35:40 That's all that matters at the end of the day. Is the domain available? Can you get the username on all major platforms? Tzury: 35:54 What about Atmo? Is it playing with atom. Connor: 36:02 Yeah. Atmo is actually playing on the word atmosphere. When something is coming into the Earth's atmosphere, they say entering Atmo kind of thing, all of the projects have a bit of a space flight theme to them. Something that I'm still a little bit of joy for myself as I give them the cool nerdy names. Justin: 36:16 Then you're using Orbit for community management. Connor: 36:20 Yeah. I've been playing around with that platform too. It's a fun little coincidence there. Yeah, Orbit's great. There's a bunch of products coming up in this space. Like I've been looking at Comscore as well and there's some other ones that I haven't even had a chance to try yet. But community management and community analytics is a big trend these days that I've been seeing. I'm really happy about it, especially because it kind of coincides with me trying to build a community. I've got a lot of brand new fancy tools at my disposal to try and build this well. Justin: 36:47 Yeah. I got to definitely get on Orbit. It's on my to-do list. It's just been pretty busy here. Pretty, pretty busy. Connor: 36:56 I recently actually got to meet Patrick one of the founders. Justin: 37:01 He's my boy. We hang out. Him and Josh, Josh is the Co-Founder really great team. Love them. Connor: 37:12 Yeah and then Mac and the team over at Comscore have also been doing really great work. They've got an investment firm like Alexis Ohanian, 776, like there's a big upswell of community tooling coming up these days. Tzury: 37:23 If I recall correctly, Firebase cloud function about two years ago or so they announced support for WebAssembly I'm talking about cloud Firebase, Google Firebase. Connor: 37:38 I'm not even sure I've seen that one. Tzury: 37:41 I'm just thinking how about for developers, an idea. If you have a Firebase operation and a very popular in a mobile space. Take Atmo as an option or at least if a developer would like to explore how to integrate Atmo in their operating platform. They can probably consider, I'm just trying to think for myself, which of running platforms that I'm in charge with, can experiment with it. Have you considered actually integrating with clouds functions around the world and making it easier for like server lists and all those frames, writing on those? Connor: 38:30 Yeah. it's an interesting question because there was actually a project called assembly lift run by somebody I know well, Dan. That project is specifically designed around running WebAssembly in AWS Lambda. There are projects specifically designed for that. I am myself not targeting the existing server list platforms because I don't agree. It's not something that I vehemently opposed, but I don't work well with the model of current server lists. It doesn't feel ergonomic to me the way that you have to organize projects and organize your code and the way that it's deployed, and the way that it's so tightly coupled to the vendors that provide them. That is not true for all of them. There's things like that are Kubernetes based and stuff like that. But I don't enjoy building on the current generation of server lists hosted server list platforms. That's why Atmo is function-based. It takes some of the parts that I do like about designing code for it. But it's not like the server list the way that you think about it with Lambda, where it is designed to be obstructed away from you. You don't have to care about how it's running and it's designed to manage itself, but it's not the current style of server list. I would consider Atmo pseudo server list platform. It's not quite server list, but it's not server list. I think it lends itself like the main differentiator for me is how do you organize your project? You as a developer, how are you fitting all of the pieces together? With current server list, there's just so much like popsicles and glue sticks that you have to use to thread logic through the different AWS or Google products to make everything work the way you want. That's just not how my brain works. That's not how I like to build systems. They've done some really amazing things on the technology side. It's just the ergonomics of it is not the way I like to work. That's why Atmo is slightly different. I would like Atmo to be able to replace Lambda in some senses. Like it could be the platform that you build on top of, instead of Lambda. It doesn't mean it can inter-operate with Lambda, especially like with something like assembly, you could build code that runs both on Atmo and on assembly lift and have them work together. There's no reason you can't do that. It's just not something that I feel like I need to be working on right now. Justin: 40:55 Anything else we want to cover? Connor: 40:58 I mean, from my end, I would just love to get feedback on the suborbital projects, the documentation, the tooling, the experience of building an application. These are all things that I care very much about. If you want to join the suborbital discord or file issues in the GitHub repos or join the GitHub discussions in our meta repo, these are all places that you can reach me. I'd love to hear anyone's thoughts about the tools and the frameworks. Tzury: 41:23 So it's suborbital.dev and it's github.com/suborbital and Twitter suborbital. Connor: 41:31 That's right. Yeah. If you go to chat.suborbital.dev it's take you right to our server. Justin: 41:35 That's smart, a redirect. Tzury: 41:38 We have learned a lot today Connor. Justin: 41:41 We have. I've never seen someone so excited about Wasm. I hear people talk about it, but you are a great brand advocate for Wasm. I'm really excited that you came on the show because since I've been in this cloud native space for four months or so WebAssembly is, but next to Rust is probably the hottest buzzword there is. It's cool to see that it's not just a buzzword and it's being applied at companies like 1password. It's pretty cool. Connor: 42:16 There's so many smart people working on these problems right now. I really believe like WebAssembly is going to be one of the killer technologies of the 2020s. Justin: 42:23 Yeah, I agree. I mean, once I found out that Sigma ran on WebAssembly I was like, oh, I get why WebAssembly is so good, but I never even thought that it was going to be used as a back-end server side at all. Connor: 42:40 If there's one kind of idea that I could leave off with, is that I'm hoping a couple years from now WebAssembly will just be an implementation detail. I don't want WebAssembly to be the headline feature on my website in three years. I want it to be a footnote. I want to be able to take advantage of all the great properties that it gives us, the security and the portability and the performance that it gives us. But it should be an implementation detail. You shouldn't need to care in a couple of years. Tzury: 43:08 So you mean suborbital on its own regardless of the actual under the hood languages, all of those details. Connor: 43:15 You should be using Atmo because of the systems that it lets you build and design, not because it's a WebAssembly. Tzury: 43:22 The security is simplicity and all those benefits. Well, I wish you will actually get to there which you spend majority of your time on this project without interfering with your career at 1password. Maybe they would love to have you on both. It's great potential, great project. Connor: 43:45 Thank you. Announcer: 43:47 Listeners. I hope you enjoyed this one. Do tune in next time. We're really excited about our lineup of guests. We have super exciting guests next week as well. Check out the show notes for this podcast at podcast.curiefense.io. for the community to cloud native podcast. Thanks again for listening tune in next week, catch you later. Special Guest: Connor Hicks.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer | Tzury Bar Yochay Guest Chris Ferreira Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the Cloud Native space. Today as our guest, we have Chris Ferreira, who is the Principal Engineer and Architect for the WebEx Platform at Cisco. Chris tells us how he started out in culinary school and ended up in the IT world, working in startups on the front end and the back end, working for Microsoft, and now Cisco. He shares the story of his first day at Cisco, the deal he made with his teammates, and the amazing results. We learn how he became a Product Advisor for Curiefense that started when he met Tzury, and what transposed after he drew his ideas on a napkin. Also, Chris explains the methodology behind the “Four Knows” that he believes the tech industry gets best. Download this episode now to find out so much more! [00:01:31 (https://podcast.curiefense.io/10?t=91)] We hear Chris’s unorthodox background and how he built his career. [00:05:54 (https://podcast.curiefense.io/10?t=354)] Chris tells us about getting introduced into Kubernetes and becoming a code contributor to Istio in a pretty big way. [00:12:24 (https://podcast.curiefense.io/10?t=744)] Justin wonders what was it that made Chris want to leave Microsoft to go work at Cisco. [00:14:25 (https://podcast.curiefense.io/10?t=865)] Chris shares his story about his first day of work at Cisco and what escalated from that day on. [00:17:13 (https://podcast.curiefense.io/10?t=1033)] Justin asks Chris if he would classify Cisco as a hybrid architecture. [00:18:31 (https://podcast.curiefense.io/10?t=1111)] We hear a great story from Chris when he met Tzury in London right before COVID. [00:24:31 (https://podcast.curiefense.io/10?t=1471)] Chris shares Cisco’s strategy in the Cloud space. [00:29:30 (https://podcast.curiefense.io/10?t=1770)] Chris talks about how layers of software add complexity and he explains what he does. [00:32:40 (https://podcast.curiefense.io/10?t=1960)] Tzury wonders why tech is the pioneer of this open source phenomena and Chris explains the methodology of the “Four Knows.” [00:33:55 (https://podcast.curiefense.io/10?t=2035)] Tzury asks Chris if there are features or capabilities that he would love to see in Curiefense for Cisco. Quotes [00:05:56 (https://podcast.curiefense.io/10?t=356)] “And at the time, Kubernetes was a whisper in the background.” [00:06:06 (https://podcast.curiefense.io/10?t=366)] “But, shortly after that I got introduced to Kubernetes and I don’t want to say it was love at first sight, but it was pretty darn close.” [00:07:14 (https://podcast.curiefense.io/10?t=434)] “The service mesh wasn’t really a defined term.” [00:08:24 (https://podcast.curiefense.io/10?t=504)] “And the response I got was no one’s ever moved a multi-billion-dollar AR business into microservices. He was probably correct, I’m not gonna lie, but like I said before, I’m pretty stubborn.” [00:08:53 (https://podcast.curiefense.io/10?t=533)] “And we started tinkering with the Linux containers doing code check-ins and doing things in the Kubernetes infrastructure, and implementing the Istio service mesh with the Envoy Proxy, handling communication paths and so on and so forth.” [00:12:32 (https://podcast.curiefense.io/10?t=752)] “However, an opportunity came over here in Cisco and it was like look, we want to run, not just in one cloud, we want to run in every cloud.” [00:14:01 (https://podcast.curiefense.io/10?t=841)] “They were building out data centers and geographic expansion would take months to do, to just stand up a single data center.” [00:14:59 (https://podcast.curiefense.io/10?t=899)] “I feel everyone should have skin in the game. If you don’t have skin in the game, then why trust the person?” [00:15:21 (https://podcast.curiefense.io/10?t=921)] “I don’t want to be building physical data centers, especially with the VM based infrastructure.” [00:15:44 (https://podcast.curiefense.io/10?t=944)] “We went down from a 6-7-month timeframe down to right around 60 minutes.” [00:16:20 (https://podcast.curiefense.io/10?t=980)] “We put the right tools in place that we could kind of track our capacity in a better way which allowed us to do that expansion so quickly.” [00:21:43 (https://podcast.curiefense.io/10?t=1303)] “WAF doesn’t have to be this overbearing heavy component that you have to run into your infrastructure that adds extreme amounts of latency and it can become a security filter in the entire communication path.” [00:24:43 (https://podcast.curiefense.io/10?t=1483)] “Cisco’s strategy in the cloud has been strategic, right, that you see certain acquisitions that have been done.” [00:30:36 (https://podcast.curiefense.io/10?t=1836)] “For example, everyone has Android and iOS phones. iOS phones from a hardware perspective are significantly underpowered here too, the specs you see in the Android device. But for some reason, iOS destroys Android in those benchmark cases.” [00:31:35 (https://podcast.curiefense.io/10?t=1895)] “Cloud Native has been one of the critical pieces to getting some of the things we have done recently, and it has allowed us to do so much more than we ever have before with so much less.” [00:31:50 (https://podcast.curiefense.io/10?t=1910)] “The open source community is just so incredible.” [00:32:49 (https://podcast.curiefense.io/10?t=1969)] “I always come back to the methodology that someone told me a couple of years back that there’s the four knows, and I think that the tech industry gets it the best. You know what you know what you don’t know, sometimes you don’t know what you know, and there are definitely cases where you don’t know what you don’t know.” [00:34:09 (https://podcast.curiefense.io/10?t=2049)] “So, security in Cisco is not a P1 or P2, it is a P0! And it’s really important that a company that does networking, that does hardware in certain ways, security is the fundamental base of everything that Cisco does.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Chris Ferreira Linkedin (https://www.linkedin.com/in/chferrei) Cisco (https://www.cisco.com/) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript Chris [00:02]: Shortly after that, I got introduced to Kubernetes and I don't want to say it was love at first sight, but it was pretty darn close. I really liked the simplicity. I'm a very simplistic person. If it's over-complicated and over-engineered people just didn't understand the problem well enough, in my opinion. So when I looked at Kubernetes, I was like, okay, just something simple, you pass it in, you execute, it dies. That's the way things should be looking like. Richard [00:28]: Hello and welcome to committing to Cloud Native, the podcast where we talk about the confluence of open source and cloud native infrastructure. Our panelists today are Justin Dorfman and [inaudible 00:38] and they're going to have an awesome conversation, I really enjoyed listening to it, with Chris Ferrera. Chris is joining us today from Austin, Texas. He's the principal engineer and architect for the WebEx platform at Cisco and I really hope you enjoy this conversation. Take it away, Tzury. Tzury [00:56]: Hello Chris, how are you today? Chris [00:59]: I'm good. And yourself? Justin [01:00]: Hey Chris. Tzury [01:02]: Doing great. Hey Justin. Justin [01:03]: Hey Tzury. So today we have Chris here, Chris, our friend from Cisco and before we going to jump into our short story, which is [inaudible 01:13] we would love to hear Chris's history in the world and he has a bunch of great stories, which when I heard him back in the day in London, when we met about year and a half ago, I was fascinated and I hope he'll share some of those with us today. Justin [01:31]: Chris, I want to hear what's your background, where'd you come from and how have you built your career?

Chris [01:37]: I have a bit of an unorthodox background, I think that's probably the best way to put it. When I actually went to college, I went for culinary school and I went there to become a chef. So that's kind of where my discipline and my rigor I think originated from because of the way you kind of operate in a kitchen is almost like a militant style. But I did go to obviously the IT world because I had spent a good amount of time in my youth and I always had a knack for it and I bounced around in a couple of startups to begin with and the startup story was pretty awesome because it was never boring, we got to wear a different hat everyday and you had to take on a different role every day. By doing that, I got to fill up my trivia pursuit pie if you will, of knowledge and I got to work on the front end, I got to work in the back end and then I started to kind of look into platform. But the last startup that I was at was a company called Parature, which was a CRM and we were completely cloud. Now we run our own data centers and [inaudible 02:41] and we had components at the time in AWS. AWS was pretty much the only cloud provider around then. It was really kind of getting off those around and so on and we started doing cloud before Azure was a thing, Google cloud was even around and we did pretty well the first year in product and we were always punching above our weight class, working with companies like IBM, Nintendo, and stuff like that. So it was a really cool experience, however, one day we all come into the office and there are about 20 people there and we're like, who are these people? And obviously, the announcement came out that we were being acquired by Microsoft and at first everyone was like, whoa, what's going on here, we're a startup, why do we want to go to big tech? And there were people there for sure that were at the startup because they wanted to be a startup and there were folks that were pretty excited about the acquisition with Microsoft. Now, Microsoft was at the time still in the Steve Ballmer era before Satya came in kind of the... Tzury [03:41]: Developer, developers. Chris [03:47]: Yeah. I mean, it was interesting procurement too and shortly after the acquisition was actually when Satya became the CEO of Microsoft and Steve Ballmer stepped down. But the culture change that you brought in, took a little bit of time to reach all perimeters of the company and when I first got there, when I went out to Redmond for the first time and I had this Apple computer and I work at Microsoft during the Steve Ballmer, you get a couple of head turns like what the F, it's okay. Tzury [04:16]: That's the culture he built. Chris [04:22]: That's correct. Tzury [04:23]: If you had an iPhone he would throw it against the wall but anyway. Chris [04:29]: I can't even tell you how many Windows phones I saw during that time and I was certainly still an Apple ecosystem kind of guy. So I was always an oddball from the moment I arrived and while some folks would adapt to that culture, I didn't feel the need. So I've always kind of stayed true. I've always been super stubborn. So I just always used whatever I thought worked and that's what kind of got me going at Microsoft to be honest with you. At the time Azure was starting up, Azure was painful, I think is probably the best way to put it. It wasn't very intuitive, some of the components were not necessarily easy to work with and we were trying to design at the time a CRM portal, which is the dynamics CRM product and we wanted to run it in Azure and we were having so much pain and difficulties in that area. We worked through it, we worked with the Azure team and I'm not getting to go into every detail, but it was a hectic process. We did some crazy stuff. For instance, one time we tried to upgrade all of the CRM portals in Europe, and apparently, we took up like 80% of Azure's compute in Europe. So they didn't quite enjoy that and to get on the line with us pretty quickly, and kind of some of the folks in charge of that area, some really talented people that I worked with over at Parature kind of had to put in mechanisms to protect ourselves and we did lots of other crazy things along that time and we kind of, we were in the frontier but we were kind of starting things. At the time Kubernetes was a whisper in the background and we were still doing BM-based infrastructure. We were using Microsoft service fabric at the time and it was okay. But shortly after that, I got introduced to Kubernetes and I don't want to say it was love at first sight, but it was pretty darn close. I really liked the simplicity. I'm a very simplistic person. If it's over complicated and it's over-engineered, people just didn't understand the problem well enough, in my opinion. So when I looked at Kubernetes I was like, okay, so something simple, you pass it in, you execute, it dies. That's the way things should be looking like and I adapted to it pretty quickly. I became a code contributor to the open source community, and I did a whole lot of things for Microsoft in that area for a good amount of time, until I'd say around 2017-ish, I get wind of this product that was being developed by Google and IBM and folks over at Lyft in SEO. I'm looking at it and I'm like, okay, I see what they're trying to do here and it's what every company always strives to do, but in some way or another, we always kind of screw it up somewhere. If we miss an execution step, if we miss a specific task in a specific area and we ended up like D-DOSing ourselves as we scale. The service mesh, wasn't really a defined term, I mean, there were things out there, there was Linker D and other components that were out there, but they didn't do everything they were advertised to do and I started kind of looking into this Istio thing, and this was around the 0.60.7 timeframe, the 1.0 came out I think came out a couple of months later and started working with that sort of tinkering with that, looking at that for different components in the Microsoft AI group and I brought that in-house and I brought it up to a GM at the time, and I said, I like what we can do here, however, I think we can do better and I would like to kind of do POC to put the platform into Kubernetes, into microservices. So it was embracing Linux at the time, the dot net core had just been released so we could run our code in Linux and do everything we were trying to do. So I was like, why not? I made the pitch and I'm not going to call out names, but it was too soon. Timing is everything. Folks were not ready to switch away from BM-based infrastructure to microservices and the response I got was no one's ever moved a multi-billion dollar AR business into microservices and he was probably correct, I'm not going to lie. But like I said before, I'm pretty stubborn. So I went over to a group that was like, yes, we're into that kind of stuff. Let's cut our teeth and get this going. And I went over there and I joined that group has what is the dynamics FAX, dynamics 365 group and we were doing like ERP solutions and other components and we started tinkering with Linux containers, doing code checkings and doing things in the Kubernetes infrastructure, and implementing [inaudible 09:02] the Envoy proxy communication paths and it was just fun. It was just good time. We started doing things faster and we just got better and certain components that were building a legacy way to process them because they were on a single VM and we're taking over a day to finish and when we actually introduce those same components into horizontal pod autoscaler, being able to scale out the compute resource and memory resources. We took down the day of processing to get to an output. We took it down to minutes and when it finally came to customers were kind of blown away. So something that used to take them a day to get an output, now it took them minutes to get an output and that's just one example of kind of what it brought and we became code contributors to STL in a pretty big way. Azure started to adopt it. A lot of Azure at the time was actually adopting Linux underneath the hood where everything prior had been nothing but Microsoft. Tzury [10:05]: Windows XP. I'm joking. Chris [10:10]: XP was a good one, we can't knock XP. Tzury [10:15]: XP actually was my favorite. Justin [10:18]: XP was my favorite operating system when I used Windows. Tzury [10:21]: I would say NT4 is my favorite one and that's it. Chris [10:27]: You would, NT4. We could throw away 2000, we can definitely throw away Windows 8 and some of those probably could have been skipped over but XP was pretty good, I have no knocks on that. And server was okay, kind of did its job. Tzury [10:41]: 2003. Chris [10:42]: Yes, Server was good. But when we made the switch over, when we saw how things were going, and it became like a wave and the wave started to build up and as you actually saw the resources that were dedicated at the time to the Microsoft service fabric, which they made open source, the community didn't rally to it. So Microsoft saw it and was like, okay, well, if they're not going to rally to us, we're going to rally to them and then became big contributors and all the resources that were on. Server's fabric were kind of swapped over to different things, a lot of them were reallocated to the work in AKS, the Azure Kubernetes Service and that's kind of when it all started to kind of really shift over there. And it took time and it's still going o from what I understand, I still speak to folks over there occasionally and there's still a whole lot of transition still going on, but it was no more, oh, it's only Microsoft, we can only use Microsoft. It was okay, let's use what works and if it's Microsoft, sure, if it's not Microsoft, okay and make sure that we make sure our customers are the priority and not the technologies we use underneath. It was great, it was great, but then someone came knocking on my door over here in Cisco. Tzury [11:57]: What made you want to leave this rocket ship to go another company? Because I mean, it's got to be to kind of hard because Microsoft in the past decade has done some amazing things. Like Microsoft technologies I use every day is VS code and it's just like, there's such a big force in this industry, it had to be more than money to get you over there. What was it that made you go, yes, I need to go? Chris [12:26]: It really wasn't money and actually, I said no a bunch of times or before I said yes, for full transparency. I liked what I was doing, I liked where things were going, however, an opportunity came over here in Cisco and it was like look, we want to run not just in one cloud, we want to run in every cloud and I knew that if I stayed at Microsoft, I was going to kind of be Azure-centric because you kind of have to eat your own dog food. So Azure-centric was always the way it was going to be. So upon the discussions and the capabilities that folks over here offered me to kind of pivot a lot of the stuff they were doing into a place they were willing to explore and willing to take bets on kind of got me excited. So I did it, I made the move. I left rainy Seattle, which I still love, no knock on Seattle. My kids grew up there and I certainly do appreciate the Washington area, but I was given an option where to go and I chose the place that has the exact opposite weather of Seattle in Austin, nothing but hot and dry all summer versus the soggy Seattle nine months of the year. So I showed up over at Cisco and I was given kind of the keys to the kingdom from a platform perspective for the WebEx contact center. Now this wasn't a market that had huge amounts of competition, but there were some pretty good competitors for sure. However, where they were was they were provisioning in physical data centers. They were building out data centers and geographic expansion would take months to do just to stand up a single data center. So they wanted to start leveraging public cloud. How do we do that? So I went there and you can ask all the teammates, I showed up and while the morale was not the greatest because they kind of had been spinning their wheels, not really getting far but the first day I showed up which was June 24, 2019 and it sticks out in my head every day and I showed up and I said look, I realized that you guys have had some tough times before I arrived and my arrival may or may not help you, but look, I'll make a deal with you all. If you just give me six months to kind of turn some stuff around, we're going to do it but if I can't do it, I will give you all the power to have a vote and I will resign literally six months from this day. Tzury [14:55]: That's some accountability. Chris [14:56]: Yes, I feel everyone should have skin in the game. If you don't have skin in the game, then why trust the person and trust is critical to me and the people I work with. So the first thing I asked was, look, we have too much manual touch in production. By March, I want no hands ever touching production ever again. Second, I want to put us in a public cloud provider. I don't want to be building physical data centers, especially with the VM-based infrastructure, so that's kind of the shift they were trying to get to. However, maybe I was able to be a catalyst to kind of accelerate that particular element. So a couple of months later, we took all hands out of production. We automated and built out data centers in the public cloud, but we wound down from a six, seven months timeframe down to right around 60 minutes. So obviously a huge shift. In hindsight, it looked like a genius move. I'm not sure I would definitely call it a genius move but obviously COVID arrived and we had to start scaling up the contact center in pretty large proportions. So we were actually able to scale up the contact center about 500% in right on eight days. So it was a pretty big shift pretty quickly. It was trial by fire. Tzury [16:15]: Wow. That's a big jump. Chris [16:17]: Yes and we put the right tools in place that we could kind of track our capacity in a better way, which allowed us to do that expansion so quickly and throughout the early days of COVID, Cisco was trying to be there and help however we could. Contact center was going to be a big component, we were trying to help healthcare providers and governments and stuff like that, kind of make sure that when people had issues, they could reach out to individuals that could provide them the right information to guide them. I think the entire team kind of rallied around that and it wasn't even an ask, we all said it's going to take a little bit of extra to do this, but we all understand what's at play here. So I was really proud of the group I was working with because they didn't even bat an eye, it was not even a blink. It was just once we get it done and we did. Tzury [17:11]: Would you classify Cisco as a hybrid architecture or have you just shut down all the data centers inside a hundred percent? Chris [17:20]: No, I don't believe a full public cloud strategy is always the right thing for the right applications. I think Cisco's always going to be a hybrid cloud. I don't see why they wouldn't want to be, especially since Cisco builds most of our network switches and all the components that run in the data centers anyway. So why not leverage what greatness you have at Cisco and leverage the diversability and scalability you can achieve with the public cloud. So I think Cisco is always going to be in that hybrid mode because it just makes the most sense. Work with our partners, work with other cloud service providers and build our data centers because if anyone's going to know how to troubleshoot networking issues it should be Cisco, so let's kind of leverage what we do good. Tzury [18:06]: Yes, I don't think a lot of cloud providers would be too happy if Cisco went into the cloud computing space because it's like they have all got your routers and it would just be terrible. But I liked that approach though of this and I think that's kind of where a lot of shops should definitely look at is the hybrid approach rather than just going all in on one cloud and then they're stuck. Chris [18:32]: Yes, and right before COVID I went over to London, sync up with my security architect over at the contact center and I met this guy with this crazy beard and I'm sitting in the Bedfont Lakes office and Tzury is giving me his sales pitch and you know I love sales and stuff because I'm not exactly the best at bed fluff, very direct and I had a time to look at three buys product and what it did. However, when we sat down with Tzury and he showed me what they did and they had a, this is a compliment, don't take it as a negative, he had a twinkle in his eye is probably the best analogy I could put with you can believe he actually cared about what he was doing. Some folks, they sell you things and it's just, okay, it's a thing to them. But for Tzury, it felt like this is a passion, this is what he does and this is what he wants to do and I was like, look, I like you, I like your technology. However, the implementation of it does not fit what I'm trying to do over here at Cisco. I'm trying to go to, at the time while we were doing all that cloud automation stuff, we were building out a new platform that was from scratch, completely microservice-based, handles horizontal scaling workloads are basically agnostic to cloud so that you can run on any cloud service provider or in our Cisco data centers and I was like, I like your product, but it doesn't fit what I need. So I think it was literally that afternoon, we sat down and drew on a napkin basically, how I could see the [inaudible 20:14] in the stuff that they did over there becoming this thing that kind of would re, kind of invent how people think of web application firewall instead of being an extra hop in the communication path or being kind of hung up in DLB, we were able to; well, we did some crazy stuff to begin with, I'm not going to lie there. We were watching what was going on with Istio, we were seeing what was going on with Envoy and we started an idea and you began iterations of that idea and then I think a couple of months later, I was sitting at the Google campus and I was sitting down with the product managers over at Istio and I was like talking through the roadmap, talking through what we were trying to do and at the time they were trying to get away from a model that added extra complexity to the communication strategy between the proxy and the control plane. So they were removing components from the service mesh that we had originally thought we could depend on to be where we could plug in [inaudible 21:17] and we went through it and then we started to look at proxy filters. We were looking at some of the websites and stuff that they were trying to get towards and Tzury and the folks over there and they were crazy enough to take a chance on it and make that bet. I mean, I'm sure as hell glad that they did. I think it's kind of defined a new frontier. [inaudible 21:41] doesn't have to be this overbearing heavy component that you have to run into your infrastructure that adds extreme amounts of latency, it can become a security filter in the entire communication path. It can so much more lightweight, so much more effective as far as the way your services run internally in the cluster and the way things come into the cluster as well from outside and by having this capability to keep everything within your own trust boundary, and not have to offload to elsewhere and go over the open internet. Even with encrypted traffic, you never know what you're susceptible to across the opener network. We can all admit that it's the wild west out there, so if we can reign it in and clean it up and not have to worry about extenuating circumstances, it just became so valuable and it had some bumps, I won't lie. It had some bumps at the beginning, but it's really kind of evolved and the folks over at [inaudible 22:42], they made themselves available to us to kind of work through these obstacles that just made it that much better and I'll forever be grateful to Tzury and his team over at [inaudible 22:54] for that because they took a chance on some crazy guy from Texas to do something that nobody was even thinking about yet. Tzury [23:04]: If I may share an observation from my end, my listening to the stories you shared with us, I would say you are a man of transformation, if I may. So think about you got in to Microsoft, decades in the industry trying their way in the internet and literally failing with the players, with the browser, with all sort of strategies and it seems like Azure was the platform that took Microsoft, brought them into the game and made him into this frontier and futuristic company. So for me, you were right there in the right place, in the right time to contribute to this transition. Then I'm looking at Cisco, your capital invested in Cisco, when was it like 80s, 70s? When was it, the company who invented their routers right to CPIP was at the beginning. Cisco basically started the internet from an infrastructure perspective and cloud, the more cloud became popular and taking off companies such as Cisco and others who were relying mostly on appliances and hardware really needed to redefine themselves, redefine their strategy and readjust and it seems like Cisco is playing a significant role in the cloud even today, just looking at acquisitions, recent years like duel port shift, and others. Do you mind sharing a little bit about Cisco strategy in the cloud space? Chris [24:32]: So everybody kind of knows the duo stuff and the office that I actually sit here in Boston is actually the original duo office out here in Austin. So I'm very familiar with that group. Cisco's strategy in the cloud has been strategic, that you see certain acquisitions that have been done, they're all very specific purposes. And when I look at WebEx, because that's where I am now, and I have a lot of familiarity with a lot of the recent acquisitions of Babel Labs and other components as they wanted to get into the cloud, as they wanted to enhance the telepresence components, they knew that we can build things and we will build things, but there are other things out there that they do the job and if we get those components and we add a bit more of that muscle that the company has behind it, we can make them that much better and that's kind of what I saw with the acquisition specifically of [inaudible 25:30]. Getting that assistant into the WebEx platform that takes all your meeting notes and sends you an email at the end of the meeting with all your meeting notes, so you didn't need someone to be sitting there typing everything down. That's a huge value, especially during a COVID world where all your meetings are basically happening over a telepresence like this. So that one in particular sits at home because not only did the founder of that company become the GM of the contact center and I have immense amounts of respect for him and the group that helped build [inaudible 26:04] but the product kind of enabled what Cisco's overall vision of WebEx was to become. But now that we have G2 coming over from box and other things, there's a real fire that's going on right now and you can kind of feel it, it's a big shift and it reminds me a bit what happened when Satya came into Microsoft, not when he came, he was always there but when he became CEO of Microsoft, it was okay, we've reached a point, we know what we are. We want to be this now, so let's fire on all cylinders to get that and it's exciting. It's always exciting and does everything go smoothly? Not a chance. I don't think it ever will, but the transformation that is going on daily and the feelings that are behind it are just very, a lot of energy and I appreciate that kind of environment. I love energy and I love fight and I love people that are willing to take chances. Tzury [27:07]: Well, the third transformation I didn't mention was transformation of Reblaze that you played a major role in Chris, and we're grateful for that. Chris [27:17]: I just drew on a napkin and I don't know, I drew on a napkin how I could see it, but you guys, you all really did it. I mean, we had an original idea and we had a second idea, and we had a third idea. I think we ended up with a pretty good one around seven or eight, but the folks over at Reblaze, I'm telling you, when we ever had an issue, literally they would be up to all hours of the night and I didn't want them to do this, but I don't think I could stop them even if I tried, they be up to all hours of the night working through it and the next morning it'd be fixed. I mean, how can you not appreciate that kind of passion and that kind of commitment. So it was pretty easy for me to be okay with that Tzury, even though I would have told you all you guys should sleep, it'd probably be good. You probably want to spend a little bit of time with your families, not with just your code, but they were dedicated to kind of the revolution that needed to take place in this space and I was pretty excited about it. Tzury [28:14]: Definitely, definitely, highly appreciate it, Chris. I believe we will talk a lot about that soon, but you mentioned families, time with families. Before COVID I remember you being traveling probably 70% of the time, wasn't it the case? How did you adjust staying at home in Austin next to famous neighborhood? Not mentioning names, obviously. Chris [28:38]: I love every minute of it. I have little kids so being home with them is always different and being here too with my wife, my wife is the rock that helps me to do these things. I couldn't do any of this stuff without her and she's the key piece to all the good stuff that has happened, she enables me to be able to travel to India, to London, to California, to Boston, to wherever I am. I know she's got it on lock at home and it's just been a blessing. It's been a blessing and a curse, COVID itself is a very sad, horrible thing but being home for the last year has been one of the more, it's the silver lining in the black clouds that I've experienced over the last year. Being at home with the kids and spending time with them more and spending time with my family. Tzury [29:27]: Another question I have for you Chris is I know that you hate management layers, your personality and your style, I think you said it yourself. Now, when we'll look at what we call Cloud Native, Kubernetes, Cisco and it's direct co-system you meet tons of layers of software and configuration management and one on top of another before you get to the actual application code. The application code is what 20% of the code base running in real-time, 30%. But it seems to be take off anyhow, what's your take on it? Chris [30:05]: Yes, like I said, I'm not a fluff guy, so layers, add complexity. So what I typically like to do is strip it down to bare bones and iteratively build, but make sure that we're building in a way that we're looking at the holistic picture while the application layer may only be 23% of the codebase. I want to always be thinking about how does that application work with the hardware? How does that application work with the orchestration behind it? For example, everyone has Android and iOS phones. iOS phones from a hardware perspective are significantly underpowered compared to the specs you see in an Android device but for some reason, iOS destroys Android in those benchmark cases. That's because the software is built for the hardware and that simplification, that unison that's designed into it. If you take that into account and the way you build through those layers, you end up with something pretty remarkable and you can do more with less and coming from the startup world, I didn't grow up in big tech and the billion dollar budgets. I grew up startups and oh, we can't afford to do that so find a cheaper way to make it happen and I still have that frugalness when it comes to how I design and implement. Now, do I take a couple of million-dollar shortcuts sometimes? Sure. There's no doubt, however, if I can avoid it, I'm always certainly willing and Cloud Native has been one of the critical pieces to getting some of the things we have done recently and it has allowed us to do so much more than we ever have before with so much less, it's just been incredible. Open-source community is just so incredible and you can't really think of industries that have this kind of community around it. People just contributing to a single component that is used by all. How often do you actually see that in verticals? You don't right. There's always some business agenda in the way communities work in other verticals, but in this industry, the agenda is just to make it work and make it work better and that has been one of the critical, amazing pieces that I've kind of been blessed with over the last couple of years. Tzury [32:23]: What do you think is the reason actually tech industry is the one who introduced probably to the world the open-source concept sharing and collaborating across even so many cases, our competitors, different companies, right? Why is tech the pioneer of this phenomenon? Chris [32:42]: Because we're crazy. We're crazy to begin with, we're in tech and that was a dumb idea. I always come back to the methodology that someone told me a couple of years back that there are the four knows and I think that the tech industry gets it the best. You know what you know what you don't know, sometimes you don't know what you know, and there are definitely cases where you don't know what you don't know and having that humility and having that understanding kind of enables tech to do these sorts of things. Tzury [33:10]: I would say it also has to do with the gigs that make up the community. Chris [33:17]: And I'm a huge nerd so I'll be perfectly honest. I'm probably at one of the biggest nerds in the world. I'll play video games with my kids, I'll watch Star Wars. That's fine, I'm proud of it. I own it. I'm a geek and I'm good with it. Tzury [33:29]: So back to [inaudible 33:30] Chris, if I may, we're planning to release in a few months, the entire AI base platform, automatic analysis, detection and prevention and so on. We're also looking into integrating with our wings and operations within Cisco, the threat intelligence company. So we're looking into integrating with threads feeds [inaudible 33:50] will be synchronized with those feeds in real time and so on. Are there features, capabilities that you would love to see in [inaudible 34:02] because your demand is our command. Chris [34:06]: So security in Cisco is not a P1 or P2, it is a P0 with an exclamation and it's really important that a company that does networking, that does hardware in certain ways, security is the fundamental base of everything that Cisco does and in the container space, it does get a bad rap sometimes from a security perspective and if you can add that blanket of security that [inaudible 34:37] provides, it definitely helps me sleep at night and I'm sure it definitely helps others that are on call 24/7, not getting phone calls because of a security threat or because of DDOS or something like that. So, yes, security is fundamental here and everything we do is focused around it. Richard [34:58]: And that's it. Thank you so much, Chris, for being on the podcast. Thank you, Justin and Tzury for holding the conversation together and being amazing people. Listeners, I hope you enjoyed this one. Do tune in next time, we're really excited about our lineup of guests. We have super exciting guests next week. Can't wait to share him with you. As well check out the show notes for this podcast at https://podcast.curiefense.io. That's C U R I E F E N S E, podcast.curiefense.io for the community to Cloud Native podcasts. You could also just use Google, but either way thanks again for listening, tune in next week, catch you later. Special Guest: Chris Ferreira.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer | Tzury Bar Yochay Guest Les Jackson Developer Advocate · Author Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the Cloud Native space. Today, we have an exceptional guest, Les Jackson, who is a Developer Advocate at Marketplacer. Find out what Les does at Marketplacer, the story of how his awesome YouTube channel and the Dotnet Playbook started, and the idea behind his book, The Complete ASP.NET Core 3 API Tutorial, developed. Also, Les explains what got him started doing Cloud Data content, and his interest in using Envoy to make microservices. Download this episode now to find out much more! [00:01:53 (https://podcast.curiefense.io/9?t=113)] Les tells us about himself, how he ended up a developer, and how he got to where he is now. [00:04:11 (https://podcast.curiefense.io/9?t=251)] We find out what Les does as a Developer Advocate for Marketplacer and the main frameworks and languages that he uses. [00:06:07 (https://podcast.curiefense.io/9?t=360)] Les tells us the story of how his YouTube channel and the Dotnet Playbook started. [00:07:39 (https://podcast.curiefense.io/9?t=459)] We learn the story behind Les writing a book called, The Complete ASP.NET Core 3 API Tutorial. [00:12:22 (https://podcast.curiefense.io/9?t=742)] Les talks about what got him started doing Cloud Data content. Also, he mentions an API Gateway called Ocelot. [00:16:13 (https://podcast.curiefense.io/9?t=973)] Les explains his creative process and why he was interested in using Envoy to begin with to make microservices. [00:20:29 (https://podcast.curiefense.io/9?t=1202)] Tzury wonders if Les was always around Microsoft tech. [00:25:57 (https://podcast.curiefense.io/9?t=1505)] If Les wrote another book, he tells us what it would be about. [00:29:23 (https://podcast.curiefense.io/9?t=1742)] Richard asks Les if he can speak more about how he feels that open source has influenced his time as a YouTube code reviewer and code maker. [00:32:45 (https://podcast.curiefense.io/9?t=1965)] Richard shares his thoughts about whether providing a credit card to sign up is a barrier to entry to adopting Cloud. [00:35:18 (https://podcast.curiefense.io/9?t=2118)] Tzury asks what efforts Les is taking as Developer Advocates, in making sure what code people are writing and using on your platform, and if they provide guidelines, tutorials, or tools for testing. [00:37:40 (https://podcast.curiefense.io/9?t=2260)] Find out where you can follow Les online. Quotes [00:14:25 (https://podcast.curiefense.io/9?t=865)] “They then pivoted to Envoy, and I thought this looks like a really interesting platform, so that’s actually where the idea for the video came from.” [00:17:03 (https://podcast.curiefense.io/9?t=1023)] “But going back to why do I find microservices interesting, I think cause you kind of mentioned possibly, they’re hard, so it’s a bit of a problem. They’re really not an easy thing to implement.” [00:17:25 (https://podcast.curiefense.io/9?t=1045)] “And I think I did mention that again, I’ve looked at some reference architectures and often what I like to do is pick them apart and kind of reverse engineer them to see if I can get them to work, because often I can’t get them to work out of the box where they maybe give you everything you need, do you try and run them up and there’s just this whole morass of errors.” [00:19:05 (https://podcast.curiefense.io/9?t=1145)] “So, that’s why I picked Envoy, number one, I think it’s going to be around for a while.” [00:23:15 (https://podcast.curiefense.io/9?t=1395)] “What actually got me back into development, one of the things that got me back into it was when Microsoft brought out .NET Core, which is this kind of open source secondary branch.” [00:30:20 (https://podcast.curiefense.io/9?t=1820)] “Open source has completely changed things.” [00:30:55 (https://podcast.curiefense.io/9?t=1855)] “Going back twenty years, everything was kind of locked down. You had to pay for absolutely everything and the quality wasn’t even sometimes that particularly good.” [00:33:10 (https://podcast.curiefense.io/9?t=1990)] “On the flip side, what I think that people who are proponents of Cloud Native things, and I think that’s really interesting technology. So, I thought about why is it good is because it just gives you so much more massive power then you’re able to do at home.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Les Jackson Twitter (https://twitter.com/binarythistle?lang=en) Les Jackson YouTube (https://www.youtube.com/channel/UCIMRGVXufHT69s1uaHHYJIA) Dotnet Playbook (https://dotnetplaybook.com/) Marketplacer (https://marketplacer.com/) Leanpub (https://leanpub.com/) The Complete ASP.NET Core 3 API Tutorial by Les Jackson (https://www.apress.com/gp/book/9781484262542) Ocelot-GitHub (https://github.com/ThreeMammals/Ocelot) The Kubelist Podcast-Episode 13-Curifense with Tzury Bar Yochay and Justin Dorfman of Reblaze (https://www.heavybit.com/library/podcasts/the-kubelist-podcast/ep-13-curiefense-with-tzury-bar-yochay-and-justin-dorfman-of-reblaze/) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript [00:00:01] Intro: A problem that really not an easy thing to implement. And I totally understand why people maintain monoliths for such a long time, because going to microservices architecture is very difficult. And so I think that just appeals to me to try and pick it apart. And I think I did mention that again, I've looked to some reference architectures and often I like to do is pick them apart and kind of reverse engineer them to see if I can get them to work. [00:00:27] Richard Littauer: Hello, and welcome to Committing to Cloud Natives, the podcast where we talk about committing to both open source and the Cloud. How do they go together, how they work to make a better internet world, and how we're here to make sure that we can continue doing awesome stuff with Cloud technology? Today, we have two other panelists besides me. I'm Richard Littauer. Hello everyone. We also have Justin Dorfman. Justin, how you doing? [00:00:52] Justin Dorfman: I'm great, Richard. How are you? [00:00:54] Richard: Doing good. Thanks for being here. And we have Tzury Bar Yochay. Tzury, how are you doing? [00:01:00] Tzury Bar Yochay: Doing great, Richard. Thank you. [00:01:02] Richard: Excellent. And because we don't necessarily like recording talking to just ourselves, we also have a guest on today as is normal. Our guest, however, hopefully, is not normal, but rather an exceptional person. I'm really excited to introduce him. We have Les Jackson. He's calling in today from Melbourne. He is a developer advocate at Marketplacer and has done a ton of really interesting stuff in this space. Les, how are you doing today? [00:01:30] Les Jackson: I am very good. Thank you. I'm very flattered that you would call me exceptional. I'm not sure that's the case, but I will accept your compliment. Thank you very much. [00:01:40] Richard: So Les, you also do a few other things. First off, to just background, to me, it's really obvious that you're not actually Australian. I went to the University of Edinburgh. I lived there for five years and you sound pretty Scottish to me. How did you end up where you are? [00:01:57] Les: That's a good question. So you are very much correct. I'm from Scotland originally. I'm actually from Glasgow, which is like the other big city in Scotland. I moved to Australia about 11 years ago. And when people ask me why I did that, there isn't really a good reason. I had a visa that I had applied for just sort of randomly and it was going to expire. So I thought, well, that was a good time as any. It was really just the change I was after. And yes, it's worked out pretty well. Been here 11 years, so yeah, it's a nice place to live. [00:02:26] Richard: It would be. One of the things I really like about your Twitter handle is that it's Binary Thistle, which shows that you're from Scotland because thistle is the national flower of Scotland, quite thorny, but also binary. Right? So you're interested in code. So you moved to Australia 11 years ago. How did you end up as a developer and how did you get to where you are now? [00:02:45] Les: Sure. Well, I mean the developer journey started way back in Scotland in the UK. So went to university in Glasgow. I did a degree in computer science, which was predominantly maths, which I probably should have researched the course a bit more thoroughly, I always thought I was just going to do a lot of coding, but it was mostly really heavy mathematics, which was okay. Anyway, I joined when I left uni and I was fortunate enough to join the National Telco, which is British Telecoms a big company, lots of opportunities there. And I really started as a developer there and that's really what got me started many years ago, more years ago than I would care to mention. Looking at the young faces around the room, I'm probably the oldest person here and that's what really got me started. And then interestingly, when I moved to Australia, I'd been in the UK working professionally for about 11 years, as well as a developer on various other roles. And when I moved to Australia, I decided to make the change to move away from coding and I moved into business analysis for a while. And that's what I did here for many years. And then more recently I've moved into my role as a developer advocate, which is again a nice blend between almost sort of business analysis and coding and talking and teaching and stuff like that. But yeah, started my developer journey a long time ago, over 20 years ago. [00:04:01] Richard: A lot of developer advocates do start as developers just so that they know how to talk about the person. And they realize, well, actually they like talking to them more than they like necessarily coding all the time or dealing with managers or whatever you want to call it, which makes a lot of sense. What do you do as a developer advocate for Marketplacer? [00:04:16] Les: So Marketplacer I suppose I'll call it a startup. It's probably coming out of that phase now. It's past the startup phase, I would say. About a hundred people, mostly in Australia, but we have opened a US office in San Jose I think is. So we're growing quite rapidly and my role there really is to help people integrate into the platform. So it's probably not, I wouldn't really say it's even the traditional developer advocate role. When I think about developer advocate, I tend to think of people going to conferences and getting people onboard onto someone's API, for example. That's not really what I do. It's more working with our clients who tend to be larger, I don't know, I suppose e-com customers who want to integrate into our platform. So I worked with their development teams to allow them to integrate into our platform, predominantly. Do a lot of documentation writing, which I quite enjoy. A lot of people hate documentation but I actually like writing documentation and working with the onboarding teams, working with the product team to consider new features for the APIs predominantly. So yeah, it's not really like a traditional developer advocate role, but it's one that I really enjoy. It's a good role. [00:05:28] Richard: What are the main frameworks and languages that you're using as part of that job? [00:05:31] Les: Over there at Marketplacer, the application is predominantly written in Ruby. I've done a little bit of Ruby, but if you've watched my YouTube channel you'll know that not really. I focus on other languages, but yeah, Ruby, Tape scripts, Java scripts. We're really lucky there because we do get to experiment with any number of languages that we want to use. So, I [inaudible00:05:50] somebody has written some modules in GOAL. I've written some stuff in C-Sharp, obviously. So yeah, but predominantly it's Ruby on Rails shop and then the front end is React, JavaScript, and Opentype strip. [00:06:04] Richard: So you're not just doing developer advocacy, but you also have the DotnetPlaybook.com and a YouTube channel. Are those the same things? Can you tell us what the YouTube channel is? [00:06:14] Les: It's a bit confusing, to be honest with you. Yeah. So in my kind of brand is, it's actually really confusing. My brand is, or my Twitter handle is Binary Thistle. Yeah. So the YouTube channel started out with that branding and then when I sort of try, when I got into it more, when I kind of looked into running a YouTube channel more, one of the bits of advice that I picked up on was perhaps you should use your actual name. So I kind of switched it to using my actual name. The Dotnet Playbook is just a thing on the side, which is really a blog, to be honest with you. And as the name suggests, it's really, I suppose, quite long articles on taking you through step-by-step guides on how to do a particular thing. The reason that I decided to create a blog. I was on holiday in Vietnam and I got incredibly ill. So I was in the small room, incredibly hot fever, and I didn't have anything to do other than having my phone. So I thought, what can I do to occupy my mind? And I thought I know what I'll do. I'll create a blog. And so I spent half that time just coming up with the domain name that I was going to use for this blog and it went through many iterations. So maybe that's fine. It's a bit disjointed because I was in that kind of strange state of mind when I was coming up with this concept. But yeah, look, the two things are kind of interrelated, interplayed them between each other. So I'll refer to the blog when I'm making my videos and I'll refer to my YouTube channel from the blog. But they're kind of the same thing but branded slightly differently. [00:07:43] Justin: Besides a blogger, you're also an author and you wrote the ASP.NET Core 3 API. Is that right? [00:07:52] Les: That is correct. So the genesis of that was the blog, Dotnet Playbook. And again, I bought a book. I can't remember who it was by, but it was about technical blogging. And one of the things they really suggested you do when you have a technical blog is that you try and build up your mailing list and that sounds like quite an old-school concept. But the idea behind that is you actually own your mailing list. Whereas I've got quite a few subscribers on my YouTube channel, but I don't really own those subscribers. I can sort of contact them, but if YouTube decided to shut me down, I'll lose all those subscribers. I don't have access to their email or anything like that. So I don't really own my subscribers. That's the way I look at it. So that was another reason I decided to create a blog because I thought, well actually, maybe own my audience a bit more. Anyway, going back to my point, I wanted to kind of get people to sign up to my mailing list. And I thought, well, people aren't just going to, well, they might. They might just come to your blog, like, and sign up. I thought, how can I entice people? What can I use as a lure or lure them in to sign up to my blog mailing list? And I thought, well what I'll do is, I'll create a small eBook. Just maybe 20 pages or 50 pages, something really small, call it an eBook and say, look, if you sign up to my blog, you'll get a free copy of my eBook. And that was the original idea. And then what happened was I started writing this book and it just got bigger and bigger and eventually kind of turned into this full book really. And I self-published for a while and that did quite well. I published on a platform called Lean Publishing. Any aspiring authors out there, I really recommend it. Fantastic platform. You get to keep most of your royalties. It's got some really nice offering features. Great platform. Also published on Amazon, of course. You have to kind of do that really to get as big an audience as possible. And the book did okay in the self-publishing kind of domain. But then maybe about after a year, I was approached by Apress. They do a lot of IT books, I guess, you'd call them. And they said, look, would you want to publish through us? And having never published a book before, I thought, well why not. Let's give it a go. And yeah, that was maybe just under a year ago, that kind of was published and it was published last October by Apress. So yeah, been an interesting insight into how actual publishing works as opposed to self-publishing. [00:10:06] Justin: Was that like a bucket list, checkmark? Write a book and get it published. [00:10:12] Les: Well, I think it is for a lot of people. What do you guys think? Is it something that you would consider? [00:10:17] Justin: Oh yeah. [00:10:17] Richard: Definitely. [00:10:18] Justin: Probably not API, API tutorial, but definitely writing a book. Yeah. High up there. [00:10:23] Les: Yeah. I suppose it was a bucket list item and that's actually why I agreed to do it because there was a lot of work involved in the editorial process. When you self-publish, you're basically just you. I mean, it was just me really. I didn't really involve anybody else. And I actually did the editing myself, which is probably the worst. Like a developer testing their own code. It's a big no, no. But I didn't really want to involve anybody else, to be honest with you. When you go with a publisher though, there's a technical reviewer, which is obviously awesome. There was the editorial staff who actually edit, review your English, and use of grammar. So as a Scottish person, my English sometimes isn't that wonderful and people sometimes say, "Is English my first language?" But yes. So they picked up a lot of grammatical errors. Let's put it that way. And then yeah, the actual publishing itself typesetting, all that kind of stuff is very interesting, but very lengthy when compared to self-publishing. So I'm not sure that's something I would do again, to be honest with you. And that's something I would say to you guys, if you have a means to market to an audience and I had my YouTube channel, then I would very strongly consider self-publishing. You have much more control and yeah, you get to keep most of the revenue. Whereas the publisher, I won't mention numbers, they basically keep the vast majority of any money that the book makes and you get what's left, which isn't a lot. And considering all the work that you have to put in to get it to that point. I mean, you have to invest a lot as well. Don't get me wrong. There's a lot of work that goes into it. But I wouldn't probably do it again. I would write a book again, maybe, but I wouldn't actually publish a book again, but I'm glad I did. And if I had to make the same decision again, I would make the same decision again. It wouldn't be something I wouldn't do. But yeah, I probably wouldn't do it again. I'll self-publish them. [00:12:08] Justin: Good to know. So I found you through YouTube and you were doing an Envoy tutorial and I just thought it was very fascinating. And supposedly it's like one of your top videos and what got you to say, you know what? I'm going to do some Cloud data content. [00:12:26] Les: So I'm really interested in microservices generally. And my microservices, generally kind of started back at my last company, which was another large telecoms company in Australia, the big one, the big incumbent here. So they were going through a bit of a transformational phase and they had a lot of big monoliths hanging around and they kind of decided number one to kind of approach software development from an agile delivery perspective. And then they also decided to go down this microservices route, which I thought was really interesting. And those two things, as I'm sure you kind of have crossed, are very intertwined with each other in speed of delivery, all that kind of stuff. So that's kind of what got me started into microservices. But again, as I said, I wasn't actually working in a very technical capacity in that organization. I was actually a team manager of some business analysts. So I was very far removed from the technology, but I was very interested in it. Anyway, so that's why I kind of started a YouTube channel, so I could keep my hand in this technology and actually learn about it myself. So just to be kind of clear, I've not actually implemented microservices in a production environment myself. I haven't ever done that. I only have ever kind of read about them, learned about them. Yes, developed with them, but in a more kind of sandbox-type environment. So that's one of the reasons for the YouTube channel to help me learn as well as teach other people. So Envoy, yeah, it was something I kind of jumped on really through looking at Microsoft, has this microservices reference architecture that they use. I think it's called eShopOnContainers. And I was looking at that and I was trying to actually reverse engineer that. And I was looking at the different patterns that they had adopted there. And one of the things they had that was very clear in that architecture was this concept of an API gateway. And I think they originally used something called ocelots, which I have never used and nobody even looks at, but they then pivoted to Envoy. And I thought this looks like a really interesting platform. So that's really where the idea for the video came from. Because to be honest with you, I found it quite a difficult thing to work with. The conflicting file that you have to work with is really quite, I found it quite monstrous. The documentation is okay, but there was a lot of trial and error and a lot of digging around on Stack Overflow to get what I wanted working, which was a relatively simple API gateway that routed to two different API endpoints. [00:14:58] Justin: I think you bring up a good point. I hear this a lot, especially from senior engineers, that Cloud Native architectures are very difficult to manage. On our previous podcast, we had with Richard Li from Ambassador Labs said upgrading Istio to 1.9 in production is like the worst thing. It's hell. And I think it's great that even someone who has accomplished what you have accomplished can also go, "Hey, you know what? I'm a smart guy, but this is really difficult." And I think that's one thing that the Cloud Native community is just going to have to keep working on and making it simpler, even though not everyone needs microservices. You haven't run microservices because you don't work at a company that really needs that, right? So it's a startup. [00:15:47] Richard: Looking at your YouTube titles and looking at what you've written about .NET, ASPNet, and knowing that you work with clients directly on various different things, but you're mostly a Ruby shop. How do you figure out what you're going to focus on next as far as the next YouTube video you're going to record? You said you know you don't have to do microservices at scale and production. So how do you decide, now here's the thing I want to record? [00:16:12] Justin: Yeah. What's your creative process and why were you interested in using Envoy, to begin with, to make microservices? You say you're really interested in them. What's there to be interested in? [00:16:22] Les: I think it's distributed computing generally is something that I think is an interesting concept. And I think it definitely has its place. I mean, one of the things I would just go back to is for the whole microservices versus monolith. And I wouldn't even say, which is best because I don't think there's an answer to that, as not really. So I think you look at what problems you have and then you adapt and you go with a particular solution, which is often a hybrid. So I mean at the moment where we are predominantly a monolith but we're probably looking at adopting, should I say. I'll even call it microservices, but kind of devolved or decoupled services to some extent and using those where they make sense. But going back to, why do I find microservices interesting? I think because you kind of mentioned possibly too hard. So it's a bit of a problem that's really not an easy thing to implement. And I totally understand why people maintain monoliths for such a long time, because going to a microservices, architecture is very difficult. And so I think that just appeals to me to try and pick it apart. And I think I did mention that again, I've looked at some reference architectures, and often what I like to do is pick them apart and kind of reverse engineer them to see if I can get them to work because often I can't get them to walk out the box or it may be they give you everything you need, you try and run them up and there's just this whole morass of errors. And so I think, yeah, it's just a puzzle or problem that I think I want to solve really. And that's why I find challenging and interesting about them. And then what I would then do, is just focus on a particular part of that architecture. So the API gateway, for example, was one small part of that architecture. And what I was actually thinking and doing was I wanted to develop a course on microservices. So I started to think, okay, how would I structure this course? How would you even go about teaching something like that? And the way I kind of came up with was, okay, let's break this down and focus on individual components that you would teach the audience about that made sense both technically and from a kind of a business perspective, why do you need it to exist. And so, yeah, I picked API gateway. I picked Envoy. But Envoy seems to be something that is used quite widely. It's got quite a vibrant community around it. So I thought that's probably kind of a relatively safe bet to start focusing some video content around. I mean, that's what you want to do as well. One of the things I think in the spaces, there are so many things that come out all the time and you might put your money, you might back your horse on one particular technology and then a few months later it's gone, it's irrelevant. And as a content creator, so you want to make sure you're creating content, that's got some longevity to it. So that's why I picked Envoy. Number one, I think it's going to be around for a while. It's got a vibrant community around it. And yeah, it seems to do the job well in terms of what an API gateway needs to do. So that's why I picked that. And then going back to this concept of developing a course so that I would then look at the different components may be that you would need to employ in our microservices and their architecture, such as the messaging between internal services. Do you go with GRPC? Do you use HTTP? Do you use a message bus? And focusing on those types of things, you know. [00:19:37] Tzury: So Les, looking at your YouTube channel, I see a bunch of, if not, most of the videos around .NET, different versions, different topics, and so on. So my question is someone who needs, I will say the opposite. I went the opposite way, meaning I started with Microsoft technology back in the day, but I left .NET when it was back in 2.1 or 2.2, I believe. I was introduced to Python and Ubuntu started becoming popular. And I never looked back. I did some time look aside and seeing that Microsoft has changed tremendously. And if you would like, I would love to spend some time talking about the evolution of Microsoft in the last few years. Under Satya Nadella, I believe the revolution in the company and transformation the company had gone through. But to you, the first question would be, were you always around the Microsoft tech stock? Was it always your focus? [00:20:34] Les: Yeah, that's a really good question. So going back to my university days that just a total smorgasbord of technologies. In fact, that's when Java actually the first kind of emerged when I was actually still at uni. So that obviously has gone. So that was something I learned about really on the boats of old school C languages that I've never even seen again. Ifil, I think was what we learned object-oriented programming with. Anyway, when I actually left an academic environment and moved into a commercial environment, the company I worked for at the time, British Telecom, the team I worked in, we sold, I suppose you'd call it a telephone switching equipment to other large corporations. And the role of the team was to integrate those switches into things like an agent's desktops or when the phone rang this agent's desktop would spring to life. Now, the reason I'm mentioning all that is the equipment that you sold it predominantly from a company called Nortel, Nortel Networks, a big Canadian telecoms company that no longer exists, unfortunately. But we worked a lot with their switching equipment and their, I suppose application gateway that allows us to integrate into that gateway, we were on a Microsoft-based API. So it started off as something called Tapi which was a, yeah, went actually to a 32 DLL type API. And then they eventually pivoted to the Microsoft.net framework and that's the API application gateway that they provided vendors like ourselves to integrate into. So we were kind of forced in that case to use a Microsoft language. I mean, you could have run that with using similar language, but you would have made life very difficult for yourself. So yeah, that's kind of why I was introduced to this Microsoft development ecosystem. Like a lot of people, I started with Visual Basic 6, which doesn't even exist anymore. [00:22:27] Tzury: I love Visual Basic 6. It was a great language. I was young and I used to build tremendous applications with it in days, literally. [00:22:34] Les: Yeah. Absolutely. Absolutely. I mean, it had a lot of detractors, but you could do a lot with it and it was very nice to use. Then you had moved to the .NET framework and you really had two languages really to pick from C-sharp, Visual Basic, Sharp, GP Sharp. And I think also something called F-sharp, which is still around, although I never looked at that. So yeah, we went with C-Sharp as the language. Actually, I got to pick what language we went with and I chose C-sharp and it just stuck with me, to be honest with you. I liked the language. The .NET framework, the originals sort of framework, is very bloated though. And when I moved to Australia, I kind of left software development behind and kind of went on my merry way. And what actually got me back into development, one of the things that got me back into it was when Microsoft brought out .NET Core, which is this kind of open source as a sort of secondary branch, and everything's based on the .NET standard. But, you know, they had these two frameworks. The original monolithic legacy windows based .NET framework, and then this open source cross-platform version of .NET, which is really what got me interested in it again. And yeah, since then I've been using it and working with. It feels a lot lighter, a lot less bloated. It's multi-platform and I'm running on a Mac at the moment and it works great. Yeah. So I think there have been a lot of changes at Microsoft and I think, for the most part, very positive. They've kind of opened up a lot and they've gone down a lot of open source-type routes, which is nice to see. They've gone cross-platform that feels less, that they're kind of basing the whole entire shop on Windows and Office. Although I mean, they're obviously still very big products for them, but yeah, it feels a lot more open. [00:24:18] Tzury: Yeah. I would say one of the great signs about Microsoft's transformation is the fact that they actually own GitHub. They go a long way with tremendous effort, not to mention it and not to show it off, meaning they want to keep things to the community. And really I was immersed in today's culture of open source and they seem to be doing all the right choices over time. One of the things that I've noticed in the last six to eight months is Microsoft I would say secret, but not so publicly known extreme support for the development of the language for us if you know about it. So people don't know that Microsoft actually pours tons of dollars into promoting and pushing this language, which is a really good sign. They believe in their language and they actually going the full way, which is resourced culture. [00:25:13] Les: I wasn't aware of that, that they were involved in the Rust project and that's interesting. And yet to a point on Github, I remember when it was announced that they bought it. There was quite a large vocal outpouring of anxiety from the developer community that Microsoft has bought Github and they're going to rebadge it and tie it in with Office. And that just hasn't happened, has it? So yes, it's good to see. [00:25:34] Justin: They're not going to screw it up. I'm glad that Microsoft bought Github rather than someone else. And if you told me, I would say that 10 years ago, I would have said, you're crazy. But you know, the way they've come around as Tzury just brought up is just the transformation has been unreal. [00:25:54] Tzury: So Les, if we can stay on this for a moment, what would be your next book about, and don't tell me you never thought about it. [00:26:01] Les: So the next book I actually had kind of off the line was actually microservices. I wanted to do a tutorial book, step by step tutorial book on microservices. But to be honest with you, I mean, to slightly take a step back, I left my previous role in March last year, so almost a year ago, last year. And I took six months out just to focus on YouTube, just to focus on books and all that kind of stuff. So I spent quite a lot of time on this microservices tutorial project/book project. And it's hard. It was really hard and I still haven't fully figured that out. So in answer to your question, I've kind of postponed it. I've not canceled though. I postponed the microservices book because it's going to be pretty hard to write something like that. And instead, I've decided to go with maybe a book, again I like kind of the tutorial format, as you can probably tell. And I've thought about doing like a beginner's guide to like a full stack developer probably within the Microsoft space predominantly, but obviously bringing in things like Docker and possibly things that Envoy, if that makes sense. So yes, I think my next book is going to be for an aspiring full stack developer and taking them all the way through everything you need to know, including things like, yeah, a professional gate, workflow, pool requests, code review, all that kind of stuff, writing requirements, even. So I'm still kind of scoping the book out just to define the parameters of it because many developers might not be where they are interested in requirement rating. But I think it does form part of a developer's role sometimes. So yeah, that's probably what I'm going to focus on. The main problem I'm going to have now though is what do I use as the front end kind of component? And I have been looking at Blazor recently, Web Assembly, which is really interesting. But I'm not sure that's a very marketable technology and it might have to lay on something like React to stand in for the kind of front-end component. That's my thought at the moment. [00:27:55] Tzury: This full stack book sounds like a great idea. You've got to do it. Telling you. [00:27:58] Les: Cool. [00:27:59] Justin: We got our sign up to Reblaze Publications. We have a new venture coming out and we're assigning you to an eight book deal. [00:28:06] Les: Oh, wow. That sounds really good, man. Yeah. Okay. All I have to do now is write the book. [00:28:11] Justin: And you have until next month, no pressure. [00:28:16] Les: Oh that's okay then. That's fine. Yeah. No problem. Next month. Absolutely no issue at all. [00:28:24] Justin: Yeah, yeah. [00:28:25] Richard: So one of the things that I'm wondering, you're someone who uses YouTube as a way of teaching people how to code, and this is increasingly happening and you're like Kent C Dodds out there, he makes great stuff on how to use React. And it's interesting to me because you have this interesting role in the ecosystem where open source projects, mostly live on GitHub at the moment, and you can go and you can write documentation for those projects. You can write PR, sort of add code. You can talk about them on Twitter. But you're kind of just a one-level removed, right? You look at the project as a whole and say, here's something I could do with it. Here's something I could do with it. And so it's contributing back by allowing other people to learn about the open-source stuff, to learn about the open-source projects and your work couldn't exist without open source. It's because it's open-source that you're allowed to go in and learn all these cool technologies and figure out, okay, how would I retell this story? And you said earlier, I'm glad that they do open source things and so you're obviously very interested in it. And I'm just wondering if you could speak more on how you feel that open source has influenced your time as a YouTube code reviewer, a code maker. What do you think? [00:29:32] Les: You're absolutely correct. So I mean to kind of look at the flip side of the coin. I've worked for quite large companies that have been locked into these proprietary vendor technologies. I won't mention any names because I don't want to get litigated against but completely closed off. So even if you just, if you want to learn about this particular platform X, for example, you have to buy into this company's entire document library. They would not release it. In fact, really scarily again, I've got this particular company in mind, you couldn't, if you Googled their documentation or anything about their platform, you couldn't get, I don't know how they managed to keep their stuff so under wraps. It was unbelievable. You had to buy into the, yeah, they had to buy into their developer program and that obviously costs money. And so coming back to your point, yeah, open-source has completely changed things. And when I make videos, one of the things I do is this is the list of things that if you want to follow along with, these are the things you'll need. So you'll need a code editor. You can pick anyone you like, but you know, I choose Visual Studio Code. It's free. You'll need to use this... [00:30:37] Justin: It's the best. [00:30:40] Les: Yeah. You'll need to use this development framework. It's free. You'll need to use Docker. It's free. And basically, it's amazing, you can do this incredible stuff and it's for the [inaudible00:30:51] of community-based developers, all free, open. It's incredible. And you're absolutely right. Going back 20 years, everything was kind of locked down. You had to pay for absolutely everything and the quality wasn't even sometimes that particularly good. But you had these companies that were just locked into technology. They had to use it. To move away from it was just more effort than it was worth and you had these horrible lock-ins. Open source has changed all that. And I even remember when this idea of open source kind of first emerged, that all these large companies are going, "Oh this is such a terrible idea." For obvious reasons, they would say that it's going to be open to security vulnerabilities and it's not going to be as good as we can do it because we put so much money into it. And it was all about the money, obviously. So completely proven to be totally wrong. And even the organizations at Microsoft have pivoted to open source to a larger or greater extent. And yeah, I couldn't do what I do on my channel. I couldn't teach about things. I couldn't learn about things. Going back to the kind of model we had 20 years ago because you had to pay for everything and people are just going to walk away from that or they can't afford it. Whereas now, yeah, it's completely open. The only thing I would call out and I was maybe going to talk about this was Cloud. And one of the things I found in making videos, whenever I do anything on Cloud, I usually use Azure that's kind of my field house. I find they don't tend to do as well and I wondered why do these videos not do quite as well as some of the other stuff? And the only thing I could come up with was that you need to provide a credit card to sign up. That's the only thing I can think of. Now, where they may not charge your credit card and you have these free offerings. I think they bait you all off of free starter so. I think you still need to provide this credit card to kind of sign up for it. And I think that's the only thing I can think of that puts people off playing with Cloud a bit more. Yeah, which is an interesting thing. I thought I'd throw that out there. I don't know what you guys think about that. Do you think that's a barrier to entry to adopting Cloud? [00:32:47] Richard: I think it's a really good point. I mean, one of the main things that I've seen leveled against cloud technologies is that it is some sort of vendor lock-in, right. It's not entirely free in a sense because you're using another person's service and so they could change the service. They could change the terms. They could change the language. Then you have to update yourself or it's gone. And I think the credit card has a way of getting into that service is another barrier to entry. On the flip side, what I think that people who are pro bonos of Cloud native things, and I think that's really interesting technology. So I thought about why is it good? It's because it just gives you so much more massive power than you're able to do at home. I have a really tiny knuck over there on my desk that record stuff every night and it could only really record like a couple of things. The RAM is pathetic. I could update it maybe. But it's never going to be able to do things at scale. But if I want to do stuff at scale, having a credit card and being able to put it in is like the least of my worries. I have much bigger concerns that pretty much Cloud is going to be able to help me with. So I think as far as like an initial hump, yes. But it unlocks more. [00:33:53] Les: I totally agree with you. I mean, I was very reticent to, I don't know why. I was very reticent to take that first step. And I was like, oh, I'm going to end up with some horrific charges here. But yeah, you take that step and you realize what you can do with it and what you have at your disposal exactly as you say. Yeah. It's super powerful. And it's just getting people over that initial hump I think is exactly the point. [00:34:16] Justin: My only thought on that is depending on the demographics of who watches your videos if they're under 18, that could be a big reason why they're not doing, because they don't have access to a credit card or if they live in a country that just doesn't have that luxury. So that's my only thing. But I think Richard hit it on the head. [00:34:36] Les: Yeah, you're right. I think the country demographics. I mean my largest audiences in the US so I don't think it's really an issue for the majority of people in the US. The next largest audience is in India. So I'm not sure of the credit card situation in India, but maybe it's not as prevalent over there. I'm going to imagine, no. I'm happy to be corrected on that. And even in countries like the UK, credit cards are definitely not as, talking about credit cards. But yeah, credit cards are definitely not as prevalent as they are in the US. I'm going to stop talking about credit cards now. [00:35:08] Tzury: I would love to talk about credit cards in terms of you're working developer advocate in that e-commerce company and security, I would say is probably a top concern. I would love to hear about what efforts are you taking as developer advocates, making sure code people are writing and using on your platform is secure. Do you guys provide guidelines, tutorials, some tools for testing? Any plans regarding this? Yeah. [00:35:34] Les: Yeah. I mean, I probably can't talk too much about it, but we've recently gone through, we want to go for, and I can't remember the certification, but there's an internet ISO security certification. So we've been going through that process. [00:35:48] Tzury: 27001, is it? [00:35:48] Les: I think that's probably it. Not actually fully across it, but yeah, we've been going for that. So I've been involved in that to some extent. So that's as we are coming out of the startup phase and we want to kind of grow, bring on bigger clients, yeah, you need this kind of stuff. So we've gone through all that. Thankfully, we've kind of passed, which is good. And I think we're just going through the kind of final stages of that. But to a point, yeah. I mean, part of my role is to help people using an API to understand what they can and cannot do with or without credentials, for example. So we do explore some stuff that's publicly available, such as product listings. There's no reason why we wouldn't allow that to be consumed from the API without really the need to authenticate. But then when you start talking about yes, all those card management, all that kind of stuff, that has to be absolutely 100% secured and tied down. So yeah, part of my role is to help the developers understand what they need to do in order to transact securely. Yeah, absolutely. It's a huge area. And if it goes wrong, well, don't need to tell anybody on this session that yeah, it can be massively damaging to any company's reputation. It can end companies, to be honest with you. [00:36:56] Tzury: Richard, to your point, we're in a Committing to Cloud Native Podcast and Cloud Native the whole idea of Cloud Native is making sure there is a technology, portable technology which you can take to any vendor and move around with freedom. So a vendor may be lucky with other technology, but not with Cloud Native and Committing to Cloud Native is the way to go. [00:37:19] Richard: I agree. Awesome. Les, it is around time to wrap up this episode. So before we do so I want to make sure people know where to find you online just a last time because you said at the very beginning that you're not a marketer and it's been very clear to me this entire conversation, nothing about marketing. That's not true. You're obviously very brilliant at marketing. [00:37:40] Les: Yeah. [00:37:42] Richard: Where do people find you online? [00:37:44] Tzury: Dozens of thousands of YouTube subscribers. [00:37:47] Justin: [Inaudible00:37:47] before we wrap up, I just got to bring this up. Here, this is how great his videos are. He has a video that has 890 up votes and not one down vote. Has anyone ever seen that? Like that's... [00:38:03] Richard: There you go. [00:38:02] Justin: Yeah. I just had to bring that up. [00:38:03] Les: I've probably just jinxed that now because I do check. I think I know the video you're talking about. I did check. Then I go, how is that possible? [00:38:15] Justin: Right. Okay. So you were thinking it too. [00:38:15] Les: Somebody now, is going to go, "I'm going to down vote that now." But hopefully, they don't. [00:38:19] Justin: No, no. Our listeners are awesome. They're mature. And they would never do something like that. [00:38:25] Les: That's good. [00:38:25] Justin: If they do, they can no longer, listen. [00:38:26] Les: Okay. Yeah. I'm just joking. So you can find me, look, yeah, just Google my name. I mean, that sounds a bit pompous, but yeah, that will bring up the YouTube channel. So Les Jackson. From the YouTube channel, that's kind of my central hub but you can kind of find me on Twitter and my blog and all that kind of stuff. So yeah, YouTube's my central kind of launching off points. [00:38:49] Richard: Les that's L E S. [00:38:52] Les: Jackson, yes. [00:38:52] Richard: Twitter is Binary Thistle as we said earlier. Les, this was excellent. Thank you so much. Thank you so much for waking up early. I believe it's 6:50 in the morning where you are, so I'm super pressed. [00:39:04] Justin: Yeah. Thanks, man. [00:39:05] Les: It's my pleasure. I've really enjoyed the talk and yeah any time. It was really good. It was really enjoyable. And so thank you for the opportunity to talk to you guys. It was great. [00:39:13] Richard: Updates this week from Reblaze. Well, we're actually really excited about this one. We get to tell you to go listen to another podcast. I know we shouldn't be doing that, but why not? Cross-linking is the best. So Kubelist.com/podcast. That's K U B E. Obviously from Kubernetes has interviewed Justin and Tzury Bar Yochay. So if you liked hearing them talk about really cool stuff at Reblaze, go listen to kubelist.com/podcast. Otherwise check in next week for another fantastic Committing to Cloud Native experience. Thank you all. Special Guest: Les Jackson.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer | Tzury Bar Yochay Guest Kelsey Hightower Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the cloud native space. If you are ready to be taken on a magical journey look no further. We are super excited to have as our awesome guest today, Kelsey Hightower, who is Principal Engineer and Principal Staff Advocate at Google in the Google Cloud Platform Division. Kelsey shares with us how ended up in the Cloud Native space, joining Google, and his job offer at NASA. We also learn about how Kelsey succeeds with his live demo’s, why he calls himself a Minimalist, and the amazing story behind No Code. Find out why Justin calls Kelsey the “Metaphor King,” and how Kelsey thinks engaging people and telling stories can help a little better at building things. Don’t wait any longer and download this episode now to find out more magical things from the “Principal Storyteller!” [00:02:20 (https://podcast.curiefense.io/8?t=140)] Kelsey talks about how he ended up in the Cloud Native Space and how he joined Google. [00:04:20 (https://podcast.curiefense.io/8?t=260)] Find out about Kelsey’s job offer at NASA. [00:06:33 (https://podcast.curiefense.io/8?t=393)] Richard wonders since open source has longer timelines if it resonates with Kelsey. [00:09:38 (https://podcast.curiefense.io/8?t=578)] Kelsey tells us his role in Google with open source projects. [00:11:59 (https://podcast.curiefense.io/8?t=719)] We hear Kelsey’s thoughts on if Istio is really good at delivering all that documentation, if it could use work, or if that’s the Google way when it comes to open source. [00:17:27 (https://podcast.curiefense.io/8?t=1047)] Tzury wonders how Kelsey’s been doing all these years pitching to the Developers who can be a tough crowd in the industry. [00:19:18 (https://podcast.curiefense.io/8?t=1158)] Justin mentions how “suspenseful” Kelsey’s live demos are and he wonders how he goes in front of thousands of people without freaking out. [00:23:22 (https://podcast.curiefense.io/8?t=1402)] Tzury wonders why Kelsey calls himself a minimalist on his Twitter when he’s so fascinating on stage. [00:24:55 (https://podcast.curiefense.io/8?t=1495)] We learn about Kelsey’s GitHub repo called No Code, which has 46,000 stars. [00:28:15 (https://podcast.curiefense.io/8?t=1695)] Kelsey tells us how he came up with the No Code idea. [00:30:54 (https://podcast.curiefense.io/8?t=1854)] Kelsey shares where he sees Cloud Native going in the future. [00:33:54 (https://podcast.curiefense.io/8?t=2034)] Find out why Kelsey said, “Magicians are cool, but teachers are better!” [00:37:24 (https://podcast.curiefense.io/8?t=2244)] Justin calls Kelsey the “Metaphor King” since he’s so successful at communicating what he sees and his values, and Richard wonders if he has any tips to share on how to do that effectively do that. Kelsey shares some outstanding advice! [00:40:21 (https://podcast.curiefense.io/8?t=2421)] Justin wonders if it’s been a blessing in disguise for Kelsey to have the year off speaking at conferences because of the COVID 19 breakout. [00:43:46 (https://podcast.curiefense.io/8?t=2626)] Find out where you can follow Kelsey online. Quotes [00:03:40 (https://podcast.curiefense.io/8?t=220)] “So, when Kubernetes dropped I kind of looked at it and said you know what, this is probably it!” [00:04:29 (https://podcast.curiefense.io/8?t=269)] “After I left CoreOS I actually signed, you know, my deal to go work for JPL out in Pasadena, and I had done a little bit of work before then on the open source front, and they were using Kubernetes to power their kinda on-site data center there.” [00:05:35 (https://podcast.curiefense.io/8?t=335)] “They explained to me some of the projects you work on at NASA, I think there was one project where, you know, one of the spacecrafts flew past Pluto and took a bunch of pictures, and it’s like that person that worked on that retired two decades ago.” [00:06:36 (https://podcast.curiefense.io/8?t=396)] “It depends on how these projects are launched.” [00:07:22 (https://podcast.curiefense.io/8?t=442)] “And then you have things that are kind of wide open, hey, we’re taking all contributions and then maybe you find out that may or may not be sustainable.” [00:07:37 (https://podcast.curiefense.io/8?t=457)] “You build a product, you gotta make that thing profitable day one. That is the success metric.” [00:08:48 (https://podcast.curiefense.io/8?t=528)] “Because when you say yes or no, you’re signing up for long-term maintenance. Someone can drop by with a contribution and then be gone forever, but it’s on you to maintain.” [00:09:11 (https://podcast.curiefense.io/8?t=551)] “You see people get mad. It’s like, when is this thing going to get updated. I’ve been using it for twenty years for free and this is unacceptable. And you’re like, I’ll give you a refund?” [00:09:57 (https://podcast.curiefense.io/8?t=597)] “I think what people may not understand is Google doesn’t have to try very hard on the open source side.” [00:10:46 (https://podcast.curiefense.io/8?t=646)] “So when it comes to open source and when people say Cloud Native, I think Google has earned appropriate credit for helping spawn this thing. At CoreOS, our mission was GIFEE (Google’s Infrastructure for Everyone Else). [00:12:11 (https://podcast.curiefense.io/8?t=731)] “I mean if you think about it, what does have great documentation? Almost no one likes any documentation anywhere. “ [00:12:22 (https://podcast.curiefense.io/8?t=742)] “What’s the documentation for Bash?” [00:12:46 (https://podcast.curiefense.io/8?t=766)] “I think documentation is kind of a community thing. You can always go write a book to fill in the gaps. You can always create a documentation site all on your own.” [00:13:07 (https://podcast.curiefense.io/8?t=787)] “Google’s approach has been, number one, Envoy is amazing. Let’s contribute to it.” [00:14:58 (https://podcast.curiefense.io/8?t=898)] “But, I also am one of those persons that wrote Kubernetes the hard way because I thought there was a lack of documentation.” [00:17:28 (https://podcast.curiefense.io/8?t=1048)] “So I don’t pitch to them which is the key. The key is I’m just learning in public.” [00:17:33 (https://podcast.curiefense.io/8?t=1053)] “I think when people say, when Kelsey talks about something, I have that empathy that you can see yourself probably typing the same commands that I do in my live demo.” [00:18:54 (https://podcast.curiefense.io/8?t=1134)] “Imagine if every developer had to implement HTTP first, we would all be like this is crazy. But that’s kind of what we’re doing on the infrastructure side and we’re proud of it.” [00:20:17 (https://podcast.curiefense.io/8?t=1217)] “All those micro feelings of joy that I get from making these work, I try to bottle them up and make that part of the talk.” [00:22:40 (https://podcast.curiefense.io/8?t=1360)] “I feel like I’m being the Dungeon Master. I’m telling the story and you can see people leaning in like, this ain’t gonna work!” [00:25:22 (https://podcast.curiefense.io/8?t=1522)] “So No Code, honestly I think the people took it and ran with it, but it told me one thing, that I think people are getting tired of all this complexity.” [00:31:01 (https://podcast.curiefense.io/8?t=1861)] “I just think any technology that’s super successful has to get democratized, that the people who use it in mass shouldn’t know how to actually build it. That shouldn’t be a requirement.” [00:31:13 (https://podcast.curiefense.io/8?t=1873)] “If you want to see any technology, see global adoption, then it can’t require everyone to know how to operate it at the very lowest levels.” [00:33:54 (https://podcast.curiefense.io/8?t=2034)] “I would say magicians are cool, but teachers are better!” [00:36:39 (https://podcast.curiefense.io/8?t=2199)] “If you’re listening to this and you’re struggling, like why language is good for diversity, some people resonate with food better. You go and taste food from another country it’s like, wow, I would have never thought to put those things together! This is delicious! That’s what diversity gets you when people are trying different things based on their own independent culture, so maybe that would help a lot more people understand and maybe appreciate diversity exists.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Kelsey Hightower Twitter (https://twitter.com/kelseyhightower?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Google Cloud (https://cloud.google.com/) “From McDonald’s to Google: How Kelsey Hightower became one of the most respected people in cloud computing” by Tom Krazit (Protocol) (https://www.protocol.com/enterprise/kelsey-hightower-google-cloud) No Code-GitHub (https://github.com/kelseyhightower/nocode) GIFEE (Google’s Infrastructure for Everyone Else)-GitHub (https://github.com/linearregression/GIFEE) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Transcript by Layten Pryce (https://www.fiverr.com/misstranscript)

Transcript Intro: Hello, welcome to Committing To Cloud Native, the podcast where we talk about the interface between open source and Cloud Native. Richard Littauer: We're super excited about our guest today, can't wait to introduce him very briefly before we get to that. I want to introduce the other panelists besides Richard Littauer, that's this guy. We also have Justin Dorfman, Justin how are you? Justin Dorfman: Hey Richard, how are you? Richard Littauer: Doing alright and we have Tzury Bar Yochay. Tzury, how are you doing good? Tzury Bar Yochay: I'm good Richard, thank you. Richard Littauer: Excellent, our guest is awesome, don't think it'll be a surprise to any listeners of this podcast and I can see he's already blushing or trying not to make it clear like that's the thing that I'm doing right now, but we're really excited. So Kelsey Hightower is a principal engineer, principal staff advocate, it's kind of confusing, but he has a lot of terms going on. But wears a lot of hats at Google, he is in the Google Cloud platform division and he's calling today from, I believe his house in Keemas, Washington, which is really exciting across the river from Portland but not in Portland. Kelsey, how are you doing today? Kelsey Hightower: I'm doing fantastic, I woke up my internet was working so I knew everything was probably going to be okay. Richard Littauer: I hope that's the case like 365 days out of the year, is that not always the case for you? Kelsey Hightower: Yeah, there's some days you wake up and it's just like, nope not today. Richard Littauer: Not today, not today. So you've been an engineer now for a long time, you wear a ton of hats, I read a bio of you on this awesome website. I'll put a link of that in the show notes, can you tell us how you ended up in the Cloud Native space? Kelsey Hightower: I think the cloud native space, if you think about the way we talk about today, some people would say it probably started with Kubernetes. But I think my days of configuration management is when I was personally a search of a better abstraction. So I used to work for Puppet Labs in 2011, 2012 timeframe. And back then, I think that's when Cloud was available, right? So you could always launch VMs and there were API for infrastructure and I think people were then trying to figure out what are the missing abstractions from all these kind of virtualized things. Like virtualized networks, virtual machines, but we didn't have anything at the app layer. So I think that was my initial journey into this thing, we call Promise Theory and then we got an app abstraction with containers. So I spent some time at CoreOS and during my time at CoreOS, that's when I really got into Docker but maybe more importantly, I got into things like Kubernetes. And I think that's what most people kind of think of as the Genesis of Cloud Native, even though the practices started much earlier. Justin Dorfman: Was that an acquisition that Google made or that was how you joined Google? Kelsey Hightower: No, actually so when I was at CoreOS, I joined and partnered close with the founders over there. And when Kubernetes came out, we had a different idea of what the future would look like for us, right? We had a tool called Fleet, which was its own container management platform built on system D. And so when Kubernetes dropped, I kind of looked at it and says, you know what? This is probably it, this is the thing I think we've been all trying to build, but we didn't know how and we didn't necessarily have the experience. And so once I was kind of contributing at the core level on the source code level, a lot of people knew me from the workshops and the key notes and the conference talks. But behind the scenes, we were just deep in the code base figuring out how to synergize these two directions that the industry was going in. So after a couple of years, it just made sense, right? I was already into the go programming language, I was already using GCP that was already in the Kubernetes space. It just made sense to transition over to the Google Team. Richard Littauer: I know you also thought about joining NASA for a while, space level is a thing of yours as well? Kelsey Hightower: Yeah, that was the offer I accepted before going to Google. So after I left CoreOS, I actually signed my deal to go work for JPL out in Pasadena and I had done a little bit of work with them before then, right on the open source front and they were using Kubernetes to power their kind of onsite data center there. And their idea was like, look we're all in on these technologies so you've been helping ramp our team up, help us acquire these particular skills. Why not help come get us to Mars and be part of that whole mission. So I went on site a few times and I got a nice little preview of the new experimental Rover that they were building with a laser on the front. I was like, yes, this is the job that I want and so, yeah, I was committed to doing that. Then I got a call from Google and said, hey, give us a shot first and the NASA team was like, hey look, this is going to be 10 years plus before we get a person on Mars, go to Google, you don't like it you can come back here and I promise you we'll still be working on this. Justin Dorfman: Wow! What a cool organization to be like, you know what, go there and then when you're done just come back, we got our seat for you. That's pretty awesome. Kelsey Hightower: They explain to me that some of the projects you work on at NASA, I think there was one project where one of the spacecraft flew past Pluto and took a bunch of pictures. And it's like that person that worked on that retired two decades ago, and you just know that the results of your work, you may not see until way later on. So this is why I think they're grown patients in terms of like working on things. Justin Dorfman: Yeah, I couldn't do that. I need to see like instant results, that's just the way my mind works. Richard Littauer: So you're not only an advocate and a speaker at conferences or keynote of some report, but you're also a developer. You've also been back in the sack, working directly on the code itself. You've been overseeing things that probably never saw the light of day and you also love open source stuff. And open source also has these long timeline issues because you're never totally in control and community projects is kind of, well that's kind of how it's going. Does that resonate with you that open source tends to have a longer timeline. Kelsey Hightower: Depends on how these projects are launched as some projects launch with a very clear vision about what they want to solve. And then what you'll see is that core will stay roughly the same and then you'll get features over time. Especially if you have a strong maintainer with a good plugin system, when you have those two things then the core can stay pretty solid and kind of address it's core mission. Take the Lennox Car Portal for example, if you want more drivers there's like a whole place to go and experiment with drivers. But some people, if you looked at Lennox 20 years ago and he looked at it today, it will roughly feel the same, right? Maybe your user land is different. So I think open source is really about the type of community that spawns up and you can have success with both models, right? So that benevolent dictator model Python, there's some things that don't have foundations and they have a core team. Then you have things that are kind of wide open, okay we're take an all contributions and then maybe you find out that may or may not be sustainable. So I think that's kind of one thing, but I think it's a place where we can innovate without necessarily the pressures you tend to have when you're trying to run a business. When you build a product, you got to make that thing profitable day one, that is the success metric. But with open source, you can just share an idea and if people like the idea, they can take it and run with it even if you can't build a successful business on top. Justin Dorfman: Well, I think you bring up a good point. Curl just passed its 20th birthday; I believe that's a run by Daniel. With that side, I mean, he ran this project and he still runs this project. He dedicated two hours a day after work and just really has dedicated his life to Curl and getting back to the Mars thing that we were going at; his Curl runs in the helicopter. But you know the helicopter that everyone's like, wow! That's really cool. So you're right, it could be a BDFL or a, or an organization, a nonprofit or a foundation like the CNCF. So it's hard to know, it really depends on I think the leaders of the project. Kelsey Hightower: We also get into sustainability issues, right? Because a lot of times, sometimes these code bases become super complex and there's really only a handful of people who understand them and are willing to make the tough decisions of saying no or yes to something. Because when you say yes or no, you're signing up for long-term maintenance, someone can drop by with a contribution and then be gone forever, but it's on you to maintain. So I think this idea of maintainer ship, I don't think that's actually sorted out well in open source. I think the community will talk about community and not realize that you're kind of burdening one or two people with the sustainability promise. And you see people get mad, it's like, what is this thing going to be updated? I've been using it for 20 years for free and this is unacceptable as a maintainer you're like, I'll give you a refund. Justin Dorfman: Yeah, seriously. Kelsey Hightower: And it's like a really hard problems open source and yeah, refunds accepted. We're sending your money back in the mail for the free thing that you downloaded this really great way of dealing with that. Robert: I'm really curious about your role in Google with open source projects. I know Google has an open source program office, there's some excellent people in that office. I don't know how well they interface with the Cloud Native space. Did you have like open source projects that are really well staffed and managed by the [inaudible09:36] there, how does that work? Kelsey Hightower: I think what people may not understand is Google doesn't have to try very hard on the open source side is just part of its natural DNA. Now I work with a lot of companies and they're trying really hard. Like they're doing workshops seminars, learning about inner source, and having guest speakers and good for them, they can go on their trajectory and learn about it. Google has been that way for a very long time so it was very natural. So what you end up with is things like Chrome has been open source forever, so much so that even Microsoft has rebased Edge, their new browser on top of Chrome. So it's just in our DNA, right? And most people can't understand it. It's like, shouldn't you hoard all the great technology for yourself should have more of an open core model. We don't really need to do that; we have kind of SAS like services where we don't always have to worry about selling software to someone else. So when it comes to open source and when people say Cloud Native, I think Google has earned appropriate credit for helping spawn this thing. At CoreOS, our mission was GIFEE, Google's infrastructure for everyone else. That's what we used to talk about when that project was getting off the ground, that's how we used to talk about it. Permitius. A lot of those ideas come from Monarch, which is the system internally at Google. We're doing something similar Kubernetes based on Borg and even before all of the open source products that you can touch, we were producing those white papers, like the MapReduce paper and all of these things. So I think we've got to remind people that this has been normally part of our DNA. And I think a lot of times people are finding tension in the vendor community where they don't know how to interact at that level where it's just a genuine thing that we do. Justin Dorfman: Well, there's one thing that we just did a podcast with Richard Lee of ambassador labs. I think you just did a podcast with them as well and he said, one thing that was really driving him insane was upgrading Istio is very difficult and he's like Googled really doesn't have to try hard with the documentation and obviously this is just his opinion. What are your thoughts about that, do you think that Istio is really good at delivering all the documentation? Could it use work or is that just the Google way when it comes to open source? Kelsey Hightower: I mean, if you think about it, what does have great documentation? Almost no one likes any documentation anywhere, typically this sentiment is really heard louder in the early days. What's the documentation for Bash. Justin Dorfman: That is a good point. Kelsey Hightower: Most people have never seen the documentation or read it, you just get on there and start pounding away through muscle memory, after 20 years you can jump on a shale and know how to use 7,000 different commands. Have you seen the Curl Docs? No, you know what I mean? Like the examples are hard so what do you do, you look at the main page and then you go look for real examples. Richard Littauer: Man, Bash has 48 different lines. Justin Dorfman: Wow! Kelsey Hightower: I think documentation is kind of a community thing, you can always go write a book to fill in the gaps. You can always create a documentation site all on your own and remember that's that maintainer thing again. I gave you something, have hundreds of people working on something for $0 and it's easy to kind of point out some of the faults. On the documentation front so I think Google's approach has been number one, Envoys amazing, let's contribute to it. Istio came from IBM, had a different name before they had a control plain, Google look at that, let's contribute to it. And we brought a lot of our own experience from internal, the way we actually run we have something similar; we call it a different name. But we have a kind of a service mesh that is rooted around identity and policy and we brought a lot of that expertise into that community. So in part, we are contributors to a thing and we stepped up and we'd done the hard maintainer ship. Remember that API went through two or three major refactors and now we've kind of landed on this thing that adopts most of that Kubernetes model and tries to find a little bit of synergy for the rest of the industry, while also working on hard Load of problems like supports that can be custom filters. Just all of these things that you need to have something to be enterprise grade and so do we spend a lot of time on docs? I mean, if you look at the Istio site, I think they've tried their best to give you real world examples and I know people who worked on the kind of Sock Shop App, so you can have a real microservice to test things out. So I think Google did a really good job, now there's some bad blood around like this relationship between foundations and who owns a thing and remember, we get back to that benevolent dictator thing. So some people will say it should be owned by everyone, if you've ever maintained a project owned by everyone is really hard to pull off. I've maintained a few projects where people will come in and say, you should do this totally different thing, I'm out, let me know when you're done. That gets a little tricky, right? So I would say, in the Istio case the fact that you can download it get started add any room for improvement, I see those as opportunities. Justin Dorfman: Totally and I want to be very clear what he said, I'm just resaying it. Kelsey Hightower: No it's valid, everyone should be able to have a valid kind of like this thing could be better, this thing could be better because that's where the features come from, that's where the improvements from. But I also am one of those persons that wrote Kubernetes the hard way, because I thought there was a lack of documentation. Started writing a book on the hole subject because I thought there was a lack of documentation, so again opportunities. Tzury Bar Yochay: Yeah, the hard way is actually the easiest way as we know. And by the way regarding Istio that Kelsey indeed are refactoring the API APIs with probably some incompatibility, may make it a bit harder for people to upgrade from existing platform to new one. But the purpose of the refactoring is actually to make it far easier for newcomers. So new deployments will like a more consistent API and simpler and more intuitive and so on. And by the way, since Richard Lee podcast, we have a podcast with a guy from Cisco, actually one of the co-creators of Curiefense. And he told us about using Istio and to switch from situation where they were deploying a new data center, took them between four to six months, with Istio and Kubernetes it's now a matter of three to four minutes. So imagine that Cisco Scale deploying a new data center. Kelsey Hightower: If you're listening to this, you probably could have done this with Engine Next, just to be fair. So when I hear Istio, I hear decisions being made, Istio already has a framework for thinking about TLS mutual law. Istio already has a framework for describing a config to include those other things. So a lot of times we'll hear like, oh when I adopted Kubernetes or Cloud things got faster, it was more like somebody made a bunch of decisions and then when you make those decisions and we give those decisions, names like Istio. Because when you break down Istio, like when you really break it down, you got Envoy, you got a bunch of little side cars that do things like mint certificates and rotate them. You got a thing like the pilot that goes to Kubernetes and tries to read your config and then match it up and generate an Envoy config and you have a pilot object thing that distributes it out. So when you look at it, you'll say the fundamentals are the same, so I think aloud that time savings come from you don't have to as a team try to make 25 different decisions. You can just say, we're going to do Istio and then boom, there you go. Now you have your kind of network substrate there. Tzury Bar Yochay: I'm looking at it Kelsey as I'm listening to you and thinking what the toughest job you took upon yourself in the industry, which is pitching to developers. This is a tough crowd, man, how have you been doing all these years? Justin Dorfman: So I don't pitch to them, which is the key, the key is I'm just learning in public. And I think when people say, when Kelsey talks about something, I have that empathy that you can see yourself probably typing the same commands that I do in my live demo. I forget all the extras that would just make it unbelievable. I just say, if this is what you're trying to do, here it is, let me show you. This is what I was trying to do, here's where I got stuck, scroll down, here's where I worked around it. This is what we're trying to do, here's the hype, let's peel back the hype a little bit and talk about the fundamentals. And I think what most engineers respect is when you can go below the hype and just say, make it click for me, I think you gained this bit of trust. So once I've had that trust, I've just been careful not to violate it. So if I'm interested in Server less, I try to approach it the same way. Here's why I'm interested in Serverless, you don't have to buy it from me, but here's why I'm interested. This concept, that over time as developers, we tend to serialize our expertise into runtimes, frameworks, and SDKs. So if you're a developer and I have a big ops background, you might be confused why are we still talking about low-level infrastructure 20 and 30 years later? When is someone going to step up and serialize those things into systems like Docker and Kubernetes and we know that we're never going to be finished because as we learn new patterns, we should be creating new SDKs and frameworks. Imagine if every developer had to implement HDP first, we will all be like, this is crazy, but that's kind of what we're doing on the infrastructure side and we're proud of it. Justin Dorfman: Your paragraph, great points. I think one thing about you saying learning in public and if no one has seen one of your live demos, it's probably one of the most suspenseful and it's a great show and you learn stuff. And I mean, how do you go in front of thousands of people and do a live demo without freaking out? I mean, I guess my question. Kelsey Hightower: You know what I do freak out, I freak out. So here's the thing, I write all the code and so let's use the one from like one of the Coop Cons when I was like the path to servers, I was showing the Kubernetes community, the benefits of something like Amazon's Lambda. And I remember I was in my room, first of all I don't really know how I'm going to start these things because I don't have any speaker notes, but I might have a few images to put some people in front of the ideas and the stories. And so when I lead with the people, you then can identify and attach and once we're there, I might give you a very simple use case and say, you see this piece of code, now we're going to have to make this code work in these two different models. So when I'm writing all this code, I'm like, you know what, why am I doing this in go, let's up the ante, let's do it in Fortran. It's like, yeah, does Fortran even work in 2018? We're about to find out, totally works, great and I had to learn some Fortran. So all of those micro feelings of joy that I get from making these work, I try to bottle them up and make that part of the talk. So, all right, now I got the Fortran going and I can put Fortran in Kubernetes, can we put Fortran in Lambda? And when I remember getting this part to work, where I had wrote a little tool that would go to Kubernetes, download a deployment, grab the Docker image, and then extract all the layers and then take out the binary and then wrap it in the Lambda function and then put it in Amazon, I was like, oh I'm a genius. This is cool but it was incredibly boring, I ran the command I'm sitting there like, this is not exciting, of course I could put colors on a terminal. But all of these people in this crowd, I got to do something better and I remember on my Spotify playlist, Diana Ross, I'm coming out was playing and I was like, yeah that's what's happening with the Docker image, right? It's coming out of one thing and going into another. So I use this tool called MP3, MPEG123, it's like an old-school Unix command and you can actually programmatically increment and decrement the volume. So I was like, how about this? I'll write my tool in Go have a separate thread that can deal with the volume. So people thought I scripted it out to be perfect, no I ran a command in one thread, type the standard out to the screen and in another thread, I started the music. And if that command takes a minute, then we will hear a minute of the song, but it takes 30 seconds, we hear 30 seconds of the song and so I could step back. And the weird thing about that, I was on edge because I was doing this on my Chrome book and at the time Chrome didn't have a real audio solution for the Linux VM to play audio through the main system. So I had to install like a sound server and pipe the audio through the sound server. I'm sitting here on stage, like this only works three out of 10 times when I was in my hotel. So it may not play audio right now, so I hit that button and when I heard Dianna Ross come to the house speakers, I was so happy. I was like, this is going to be dope and then it ended and remember I'm unscripted at this point. I'm basically doing live jazz and I said, what? You all don't have music for your command line tools and you said this and it's applaud and then you move on to the next section. So that's how I do it, I'm nervous, I'm doing improv on stage, but I can look at the developers, I know that look we're on the same team. You guys want to see where this goes, so I feel like I'm being the dungeon master. I'm telling the story and you can see people leaning in like this ain't going work and so that's how I do it. Justin Dorfman: Love it, love it, love it. Richard Littauer: I think you'd be an excellent speaker at Bang Con. If you've ever heard of this, it's in New York, it's a conference that's just dedicated towards the joy of computing. I had a friend who made the zip drive, you take mini files and the zip file would go in and out and play the mini file. like, you know the tones of something zip drive. Kelsey Hightower: That's amazing. Richard Littauer: And it sounds like what you're doing, right? Just like, hey, okay this is boring, well how could I have more fun here? Let's put jazz in there, sweet! I love it, I love it so much. Tzury Bar Yochay: Well, we're fascinated Kelsey, obviously then I wonder. I mean, you did all that on stage and then you call yourself a minimalist on your Twitter, where is the minimalism here? Kelsey Hightower: If you think about it, the story was the thing, I just added elements to the story. There's no slide decks, there was no 10 pieces of a microservice, there was no database. There was no algorithms, I was just tell him the story and so I took the simplistic approach of like, how will I connect with humans? I'm trying to ask people at Kub Con to consider something that replaces Kubernetes as a north star for the future. So what is the simplest way to do that? And I figured it was to tell a story so even when you pick something simple, then it becomes complex to execute on it. Because if you have all these things, I could have used numbers and like 50% of people, new workloads or using Lambda and then try to convince you with a bunch of research papers. I would have been all over the place on stage and then you would have to try to pick and choose what was the thing to convince you. So I only had one tool which was story, so I just told the story show don't tell and you saw it in action. And then you're sitting there and asking yourself, what are you actually trying to do, why is this not actually the path to the future? So I think that's where the simplicity comes from. So as a minimalist, I try to get rid of all the extras, all the slides, all the diagrams, all the charts, and then simplify it and then go deep on the simplification. Justin Dorfman: Speaking of minimalism, you have a GitHub repo called No Code. My question to you is how many poll requests do you get on No Code? Kelsey Hightower: Oh, no I don't even look but I remember when I put that thing up, I was getting emails from like engineering managers, like look you need to take this thing down. My whole team is distracted right now, they're in chat talking about this thing and I'm sitting there like this ain't my fault. Like you got other problems and so No Code, honestly, I think the people took it and ran with it, b ut it told me one thing that I think people are getting tired of all this complexity, every new framework has all of these promises that it can solve all your problems. But then that framework becomes a liability because maybe you can't upgrade it, it doesn't keep pace with the features you need. Yeah, so No Code was like a big joke, but I think people took it super serious when you look at some of the comments. Justin Dorfman: If I remember correctly, there was a very colorful hacker news thread, just people talking about exactly what you're saying, dependency and at what point do we just concentrate on just making it simple as opposed to importing a hundred different dependencies. So I thought that was really interesting and so you don't look at any PR you just say, okay. Kelsey Hightower: That thing is a monument for the community. Justin Dorfman: That's cool. Kelsey Hightower: I think it's a checkpoint in our history where we can go look at it and say, there was an inflection point where I think enough was enough and we need to just really take another look. Justin Dorfman: I just can't believe there was managers emailing you or hitting you up, like, hey, can you take this down? Like, who says that? Kelsey Hightower: Oh, you'd be surprised, man, like in this business. I'm also an assessable person, so I'll hear people out and try to respond and be courteous to folks. So I'll hear people out and look, I thought it was funny, I can imagine. I've been a manager before you come in and your whole team is like emojis and look at this comment, look at that comment freaking now and having fun with it, you're like, can you all please write some code and it's like, no we're practicing no code right now, right? Like, you'd be like, you know what? This got to go. Justin Dorfman: Yeah, it's so brilliant and it's got like, how many stars? Like over 60,000? It's like one of the top repositories. Kelsey Hightower: You know what's so funny, GitHub did a thing like top repositories that year you had like the React Framework, TensorFlow and No Code up there in the top repositories because of stars and momentum. I was like, I need to go call a venture capitalist and raise a round, right? Because you have stars or like currency. Justin Dorfman: Yeah, they are. Richard Littauer: For those of you who don't know No Code is a repo, that's basically just, there's no code in it and the read me is not shown in the program, go read it. It's hilarious, awesome. There's 46,000 stars, which is just. Justin Dorfman: It's brilliant. Richard Littauer: Incredible. Justin Dorfman: It was like, oh my God, that's so awesome and it was minimal, that is Kelsey's like emo. Since this isn't a video podcast described that Kelsey is sitting in a room and in the window, there's one plant. Kelsey Hightower: From Ikea. Justin Dorfman: From Ikea. Kelsey Hightower: It's not even real. Justin Dorfman: And it's like, I look at it and I'm like, God that looks so cool. It's just like that little minimal touch, so he practices what he preaches. Tzury Bar Yochay: How did you come up with this idea of no code originally, what was the trigger? Kelsey Hightower: To be honest, for my job, I work at Google so I was in Mountain View and when we're doing product work, you kind of whiteboard with the engineering team. I might go meet a few customers, you might go sit with the cube startups and their engineering team. And then when you're scrolling through hacker news and it's like, everyone has a new solution, new framework, new code, new libraries, and I just want to tweet because I'm waiting for my flight in the airport. And I just wanted to tweet, like we don't need any more code because there's a lot of existing things that work or you hear about all this duplication, even on the same team. Someone write the exact same class that you already have in the code base and you're like, why didn't you at least do a search to see if that was already implemented? And then I said, you know what? I got a little time; you know what would be funny if I just made a GitHub repository called No Code. And then I was like in Kelsey fashion, because I'm all about the docs, I wrote a tutorial, how to get started with no code and the benefits of no code and. Tzury Bar Yochay: In design style guide. Kelsey Hightower: Or add a contribute, right? And the contributing guide says you don't, there's nothing to contribute to it. So then I just posted it, close my laptop, get on the flight and I'm like let me just kind of take a little quick nap. I opened up my laptop and this thing was like product hunt, people had medium posts, Twitter was blowing up, the stars are crazy. And then the coolest thing was someone issued a pull request and the pull request had a blink title, it had a blink commit message and had a blank body, everything was blank and it turned out this was a bug. It's like a GitHub bug, you are not supposed to be able to do that and I think the GitHub team was like, you know what? We'll let it stand for this one, so this issue lives on as this blank issue that does absolutely nothing, changes absolutely nothing and guess what it adhere to the style guide so I merged it. Justin Dorfman: And that was the only merge. Kelsey Hightower: I think that's the only merge, I think I did a Docker file or something. Justin Dorfman: Nice. Kelsey Hightower: I think inside of that Docker file you can go download it, I think there's still a Tar Ball up. If you download it and you catch what's inside, I think it says something like, are you serious? So it's like a little Easter egg in there. I don't know if I still lift it up, but there is something to download. Tzury Bar Yochay: Have you begin releasing new versions. Kelsey Hightower: We are code complete; I mean, I haven't seen a compelling feature to add. I think we nailed it on the first release. Justin Dorfman: I agree, it's perfect. Richard Littauer: So you all heard about the story, No Code is a great story is clear from the very beginning, what it is. You keep talking about going towards the future, or this is the future and I'm curious where you see Cloud Native going because you probably have a very concise story for that lined up and I want to hear what it is? Kelsey Hightower: I just think any technology that's super successful has to get democratized. That the people who use it in mass shouldn't know how to actually build it, that shouldn't be a requirement. Like we want to see any technology see global adoption. Then it can't require everyone to know how to operate at the very lowest levels and so I think that's one challenge that practitioners have and remember this is early days of compute, at least in the style that we're practicing. So we assume that everyone needs to learn all of these low level details in order for them to be able to participate. So if you think long-term like when I was teaching my daughter how to code HTML website on the laptop, 1270.011, website comes up, Blinky tags, cool she's in geo cities. And at this point she's sharing the website where her friend 1270.011 doesn't come up, hey, something's broken. So this is like the inflection point of she's the future. Do I go and say like sit down, pick a Linux, distro, learn some Kubernetes, maybe in a couple of weeks, teach her how to deploy the website or I went to Firebase on her Chromebook. She did Firebase deploy, got a URL, gave it to her friend and it was working. Richard Littauer: Yeah. Justin Dorfman: That's like, as you said, it's like the geo cities of today. That's how I learned how to write HTML, I mean, I just did view source and just edited things. But still like and also another thing that happened for a generation is My Space and them allowing like people that weren't in computing or software engineers, they're just like I want to update my background so this is how I do it. And then some folks said, you know what? This is a great career path and take it. So I Firebase, thank you for keeping the geo city theme alive. Kelsey Hightower: Yeah, if you're a tool builder, don't be discouraged by this we still need people to work on these tools. But we've got to remember that they're tools for creating things and sometimes people want to create things just because you can play the guitar doesn't mean you know how to make a guitar by hand. We just got to understand that's okay and we have to stop thinking that the rite of passage is that you have to build your guitar first before you can make music. So we just got to remove this idea, that's a requirement and just enable people. And then if someone really wants to learn how to build a guitar, then we show them how to do that as well. Richard Littauer: Reminds me of an Arthur C. Clark quote, right? "Any sufficiently advanced technology is indistinguishable from magic." And it's kind of how I imagined the first people sitting around a fire hearing a guitar, what is this music coming out of this thing? Whereas the person who made it, it's like, well you can take a box and you put a hole in it and put strings over it. And so I kind of want to call you a magician now for your work with Kubernetes and building all these things and even for Firebase. Kelsey Hightower: Yeah, I mean I would say magicians are cool, but teachers are better because the person that shows you the trick allows everyone else to be a magician for at least once. And then they can show someone else the thing and then they can graduate to becoming a teacher and I think that's the thing, right? Because the best teachers, sometimes you're right, sometimes you start with the magic show and you catch people's interest, you get them curious and then you say, I'm going to show you how to do the same thing. And I think that's like one of the keys to education in general, like how do you get people engaged? And a lot of times that's the way you do it. Tzury Bar Yochay: It's actually open source showing how the magic happens, opening the source part of that closed source, which is actually showing you just the magic, right? Kelsey Hightower: I wonder though, is that true a hundred percent, you know what I mean? Because I think if you look at the number of people who can really read code and then the smaller number of people who will actually contribute code and you ask yourself, is that enough? Because there's a lot of people who for example, I met a lot of system administrators and I was trying to teach them Go and their like, why do I need to learn Go? I said, look at your at Laptop, Dockers written in Go Terraform is written in Go, your whole Toolkit is written in go. So if something happens then you need to learn. The language is written in and for a lot of people, it really didn't click a hundred percent. So you're right, I think what you're highlighting is that there is a lot of power and seeing how things are made. But you're right I think people have to actually put a lot of effort today to actually capitalize on it, right? It's like imagine like this magical chemistry recipe that you need, but it's written book that's in German, you got to go learn German first, if you want to get the formula or find someone to translate it for you. And I think that's kind of the issue we have with code because when you go around the world, a lot of code is written in English. We kind of forget the fact that some people don't know English so even though we thought, oh they're just keywords, they don't know these words. So I think we still kind of have a high bar in terms of articulating, this is why I love when you see these diagrams about how things work and then like the philosophy behind them and high-level analogies. And say, hey, this code is just one implementation of these ideas and that's where I think we're missing sometimes because I remember I used to work with a bunch of developers and they say, the code is the documentation, I'm like, are you kidding me? This is not the same thing. Justin Dorfman: Seriously. Richard Littauer: Language is really fascinating to me and it's one of the cases of diversity that a lot of people don't think about, especially in our communities forget that speaking English is amazingly privileged. The majority of the world has a lot of issues coding, not just because of lack of access to say technology, but also because maybe their first language is Spanish and they have to learn all these different words or maybe it's something even stranger than that and less common on the internet. Kelsey Hightower: So if you're listening to this and you're struggling, like why language is good for diversity, some people resonate with food better. Like you go and taste food from another country, it's like, wow, I would've never thought to put those things together, this is delicious. That's what diversity gets you when people are trying different things based on their own independent cultures. So maybe that would help a lot more people understand and maybe appreciate that diversity exists. Richard Littauer: I like that metaphor a lot. Justin Dorfman: I was going to just say, he's the metaphor king, like every time it's just like a quote. Richard Littauer: What's amazing is the amount of work I can see behind those metaphors. You spend a lot of time thinking of ways to describe things easier, a lot of time thinking of how to be successful at communicating what you see and what your values are. Do you have any tips for listeners on how to do that effectively? Like how late at night you set up with a white paper coming up with those sorts of metaphors. Kelsey Hightower: So the thing is like never at all, it's usually when I'm just talking to other humans. I'll look at the humanist say I'm talking to you as a person, what experience that I can pull from that we may only have like usually the best analogies are the ones you can relate to and so when you're looking at a person in your mind is doing this puzzle of like, who is this person? And if I want to kind of transfer what I'm thinking to them, what's the best way to do it? So I try to pull from these natural and you have to listen to, right? Like you said something around language and I'm thinking language can be related to food so you kind of pull these things in. So sometimes a lot of the analogies are kind of done on the fly and I have to remember to go listen and try to write them down. But yeah, I think I pivoted towards this when I started asking myself can learn how to tell the stories. When you watch a good movie, you really appreciate the story, the fact that someone came up with the story and held true to the story through the editing process. So during my public speaking, I remember for the very first time, the last Cube Con I was in person, I believe it was Cube Con 2019. Normally I have a clutch when I give presentations and my clutch is the live demo. Even the live demos are hard, they're scary, they became my clutch. So I just used to always have one for safety, as weird as that sounds and then one day I said, you know what? Let the clutches go, that Cube Con no demo, no Laptop, no pictures, no slides, all you get this story. And then you really have to dig deep and start to find those things and the way I practice was just walking around the room and you tell some stories, you tell stories. And for me, the story is good is when you can feel something, when you look at someone else and they laugh and they decide that they want to cry or not, then their mood changes and that emotional roller coaster is dope. So when you're on stage and you see people on the ride and now you know that you can control where they go next, if the laughing you can give a hard pause and say but and then they're back and they can take them somewhere else and at the end of that roller coaster people wake up at the end and the natural human reaction is just a clap. So I'm learning how to tell the stories, that's it. So everything that I'm doing, I'm saying, why am I doing this? Usually that's where the stories come from, why am I writing this app? And then there's a story typically around why you're doing the thing that you're doing and if you learn how to tell stories, I think you actually be a little bit better at building things. Tzury Bar Yochay: Is your next book going to be the art of public speaking second edition. Kelsey Hightower: You know what I think I am getting a little bit better at it because for the first time, in a long time, I'm trying to actually work on it. And I think I'm able to work on it because I'm a bit more patient and I don't have my crutches, so now I'm finally learning how to walk. Justin Dorfman: That's a bit of a blessing in disguise to have the year off pretty much because of the horrible COVID 19 outbreak. Kelsey Hightower: I wish it was but you know one thing I'm looking at all these streams of information and what was that movie, Bruce Almighty and I remember he got this gift that he could hear what everyone was thinking to experience what God feels. And that is torment for most people and we don't realize it like, can humans really absorb all that information? We know our machines can but once you fill up the hard drive, you start losing stuff, things go down, you run out of I knows. You came to allocate the next block for the next piece of information and then what the computers do, they do weird things when the Colonel can't boot, because there's no way to lay out the partition table. You have corrupted storage machines don't work well so we reboot them and we reformat them and you got to do all of these things to them to keep them maintained. We haven't learned how to do that yet, right now, we're just hooked up to the firehose, sucking down everything with no regard to storage and permit in our cash and so I think for this time off, a lot of people almost had a little too much time off because when you have to go somewhere and come back, you're forced to stop and detach from the firehose. That went away, so I think for me, I had to learn like, hey, am I too close to the fire hose? And if I am, I need to have a better filter. So I think there was a little bit of self-reflection, I had a little bit more time to reflect on these conscious decisions that I wasn't making, even though I make a lot of conscious decisions. So I think that firehose, we're not going to understand the ramifications for that until people get back and detached from the fire hose because some people will start having withdrawal. Like that firehose, someone was dictating your emotions this whole time and it goes away and that void you're going to have to look around the real world and say, where am I going to fill it? That's hard for somebody. Justin Dorfman: I'm kind of on the fence, actually no, I want to go back to events. But you have pole, can you get Oz Con back? Come on, you can just go to Tim O'Reilly be like, come on man let's just bring it back for the kids. Kelsey Hightower: What we all forgot about what tech conferences was I think conferences started having way too many tracks. Justin Dorfman: Yeah. Kelsey Hightower: Obviously, they try to cover all the topics and we forgot that half the time all we want to do is hang out with people that we don't typically get to hang out with and learn from them, be patient. So if conferences do come back, I think we should just go back to maybe one or two tracks max, because there's something about shared experience. And at a single track conferences, the magic is the shared experience, because if you have lots of breaks, you can talk about your shared experiences and then we can naturally just make sure we don't need a hundred speakers, right? You can have people do once and then that was your time and open up for everyone else. Richard Littauer: Or maybe the no track conference, a line of no code, right? Justin Dorfman: I love that! Yeah, that'd be a great collaboration, oh my God. Richard Littauer: I'll start a repo. Justin Dorfman: [Cross-Talking43:10] no code and then just do a find a replace, if there's anything to find and replace. Richard Littauer: Unfortunately, it is time for me to submit an empty PR to this talk because I think it was really excellent. And also to clap, thank you so much Kelsey, it was really great to talk to you. This was super great, where can people find you online if they love the words that you had here, where could they hear more? Kelsey Hightower: At Kelsey Hightower on Twitter, my DMS are open. I don't get to all of them, but I do try to make time to comb them and really have conversations with people. So that's where you can find me. Richard Littauer: Thank you so much. I think that's it, any final questions, anything else? Justin Dorfman: I just want to keep going, why can't we just keep going? Kelsey Hightower: This is dope, by the way I like this set. Justin Dorfman: Oh, really. Kelsey Hightower: Yeah. Justin Dorfman: Oh, my God. Kelsey Hightower: In terms of questions, one thing that I'm starting to learn is that having multiple moderators is a really nice pacing technique, right? Like I don't know where the question is going to come from, that keeps me kind of on edge. So I like the variety and then also the questions were super thoughtful and the more thoughtful the questions are typically you get more thoughtful responses, so thanks for taking me there. Justin Dorfman: Oh my God, is this a dream? Richard Littauer: Such a great guest thank you so much. Tzury Bar Yochay: Kelsey I feel like I'm in a dream to be honest, you took us through this journey as you do on stage right here in this podcast and I'm sure listeners will get the true same experience, it's like a magic. Justin Dorfman: Yeah, like magic, full circle. Tzury Bar Yochay: That is Kelsey, we celebrate to have you, and we'll look for the next episode with you, hopefully. Richard Littauer: Thank you. Special Guest: Kelsey Hightower.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer | Tzury Bar Yochay Guest Richard Li Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the cloud native space. Today, we have as a panelist, Tzury Bar Yochay, who is the CTO and Co-Founder of Reblaze. Also, joining us as our guest, we have Richard Li, who is the Co-Founder and CEO of Ambassador Labs, which builds popular open source tools for Kubernetes. Today, Richard explains to us the story behind starting Datawire and the different projects that were built out of that, including Telepresence and Ambassador API Gateway. We also find out about Richard’s experience at Red Hat and why he moved on to becoming an entrepreneur. He shares two areas to focus on in order to build a successful open source company, as well as the process he went through starting his company, and what his first product was. We also learn more about Argo and the things Richard sees Ambassador Labs focusing on in the next few years. Download this episode now to hear many more fascinating things! [00:01:36 (https://podcast.curiefense.io/7?t=96)] Richard explains what Datawire, Ambassador API Gateway, and Emissary- Ingress is. [00:03:58 (https://podcast.curiefense.io/7?t=238)] Tzury gives us his insight into his experience in CNCF licensing. [00:05:11 (https://podcast.curiefense.io/7?t=311)] Speaking about the new project and the donation that Richard did, Justin wonders what his expectations are in getting to graduation and if he’s looking to accelerate getting it to the next level. [00:06:47 (https://podcast.curiefense.io/7?t=407)] We learn how long it took Richard to get to 150 contributors and how closely he works with the Envoy team. [00:08:57 (https://podcast.curiefense.io/7?t=537)] Find out the “golden rules” to build a successful open source company or project, and things you must do from the very beginning to keep the project interesting to the community and to get it out and reach out to those achievements. [00:10:53 (https://podcast.curiefense.io/7?t=653)] Richard explains to us about having a business on top of an open source, the success with open source, where he draws the line between the paid and the free, and all the ecosystems around it. [00:13:32 (https://podcast.curiefense.io/7?t=812)] Justin mentions that overall what Richard is doing for the Cloud Native ecosystem seems to be helping a lot of companies that could turn into monthly recurring revenue or whatever packages he sells and wonders if this is happening. [00:15:30 (https://podcast.curiefense.io/7?t=930)] Richard tells us about his experience at Red Hat and why he moved on to becoming an entrepreneur. He also tells us he is hiring at Ambassador Labs. [00:18:05 (https://podcast.curiefense.io/7?t=1085)] Learn where Richard was in his career that got him into The O’Reilly Velocity Conference back in the days. [00:19:30 (https://podcast.curiefense.io/7?t=1170)] Richard tells us the process he went through starting his company, what his first product was, and the first thing he was working on. He also elaborates on Telepresence and how it has changed the development life cycle. [00:24:47 (https://podcast.curiefense.io/7?t=1487)] With Richard’s awesome education background, Justin wonders what he did at MIT. [00:25:52 (https://podcast.curiefense.io/7?t=1552)] Tzury asks Richard to explain what Argo is. [00:27:47 (https://podcast.curiefense.io/7?t=1667)] Richard tells us the first product he released under Datawire, which is today, Ambassador Labs. Also, he explains what goes into the API Gateway, what he thinks should go into the Proxy, into the Envoy, and what type of functionality he would keep out of the API Gateway. [00:31:51 (https://podcast.curiefense.io/7?t=1911)] Richard gives us his take on how the nature of software development has changed with Kubernetes being in the cloud. [00:34:48 (https://podcast.curiefense.io/7?t=2088)] Learn where Richard sees Ambassador Labs in five years. [00:35:45 (https://podcast.curiefense.io/7?t=2145)] Find out Curiefense updates for the week. Quotes [00:02:00 (https://podcast.curiefense.io/7?t=120)] “So, we ended up basically just using instant domain search and just typing in random things, and we ended up with Datawire, and it was a Black Friday sale, it was five bucks!” [00:02:16 (https://podcast.curiefense.io/7?t=136)] “The reason we called it Ambassador API Gateway was because it was built on Envoy and an Envoy Proxy, an Envoy in the U.S. Department of State Hierarchy actually reports to an Ambassador, so Ambassadors are at a higher-level Envoy.” [00:02:54 (https://podcast.curiefense.io/7?t=174)] “And the CNCF says, well, in order to donate the technology, we need to take your trademark, which is Ambassador, and we said, well, but that’s the name of our company, and they said, well, the only thing you can do is to rename it.” [00:07:04 (https://podcast.curiefense.io/7?t=424)] “And if you’ve ever tried to use Envoy Proxy on its own and then try to use, deploy it on much less deployed on Kubernetes, you would realize, oh, this is actually quite complicated.” [00:09:28 (https://podcast.curiefense.io/7?t=568)] “Especially if you’re a startup, I think the two areas I would focus on is making it easy to install and use, and then two, sort of correlated to that, is to have good documentation.” [00:10:42 (https://podcast.curiefense.io/7?t=642)] “Everyone loves Istio until they try to upgrade it in production at which point they realize it’s actually terrifying.” [00:11:09 (https://podcast.curiefense.io/7?t=669)] “My opinion is my next company is I’m not going to do an open source company because you really have to figure out two different kinds of products and that’s twice as much work as a regular kind of company.” [00:22:08 (https://podcast.curiefense.io/7?t=1328)] “I think one of the things is that what developers discover is that when they start developing a microservices-based application on Kubernetes, life gets very complicated.” [00:26:35 (https://podcast.curiefense.io/7?t=1595)] “And so, we’re big fans of Argo and we’ve actually integrated Argo with our API Gateway so that you can do something like I have a new version two to replace version one of my software." [00:29:48 (https://podcast.curiefense.io/7?t=1788)] “I think the thing that we’ve seen people run into trouble with is when people start to put too much business logic into the API Gateway.” [00:31:59 (https://podcast.curiefense.io/7?t=1919)] “I think with Kubernetes itself, I think the key difference is that the nature of software development has actually changed in a very fundamental way.” [00:32:36 (https://podcast.curiefense.io/7?t=1956)] “So, I like to say a full stack developer now is a full lifecycle developer.” [00:34:23 (https://podcast.curiefense.io/7?t=2063)] “You could argue that Kubernetes has accelerated productivity in some dimension, like the last mile soft delivery, clearly Cloud beats CD ROMs any day of the week.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Community Groups-Curifense (https://community.cncf.io/curiefense/) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Tzury Bar Yochay Twitter (https://twitter.com/tzury?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Li Twitter (https://twitter.com/rdli) Richard Li Linkedin (https://www.linkedin.com/in/richardli/) “CNCF Adopts Ambassador’s API Gateway, Emissary Ingress” by Mike Melanson (The New Stack) (https://thenewstack.io/cncf-adopts-ambassadors-api-gateway-emissary-ingress/) Datawire-GitHub (https://github.com/datawire) Ambassador Labs (https://blog.getambassador.io/) KubeCon 2021 (https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/) “No one wants to manage Kubernetes anymore” by Scott Carey (InfoWorld) (https://www.infoworld.com/article/3614850/no-one-wants-to-manage-kubernetes-anymore.html) Argo (https://argoproj.github.io/argo-workflows/) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Special Guest: Richard Li.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer Guest Mike Sparr Staff Cloud Architect at DoiT International Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the cloud native space. On today’s episode, we are super excited to have as our guest, Mike Sparr, who is a Staff Cloud Architect at DoiT International. Mike shares awesome stories about how he started programming at the age of ten, his journey of getting to where he is today with DoiT International, how DoiT International is different than other companies, and what DoiT does in terms of open source. Find out what Mike’s recipe for success is, what he’s most excited about right now, and the voids in the industry he thinks he could fill. Download this episode now to find out more! [00:01:07 (https://podcast.curiefense.io/6?t=67)] Justin shares Curiefense updates. [00:02:10 (https://podcast.curiefense.io/6?t=130)] Mike shares a story with us about how he started programming at the age of ten on a Tandy TRS-80. [00:03:10 (https://podcast.curiefense.io/6?t=190)] We learn Mike’s journey and how he ended up at DoiT International. [00:10:00 (https://podcast.curiefense.io/6?t=600)] Mike tells us what part of Kubernetes he’s working on, how DoiT is involved, and we learn more about the Founder of DoiT, Vadim Solovey. [00:12:13 (https://podcast.curiefense.io/6?t=733)] Mike explains what ProdOps is. [00:14:51 (https://podcast.curiefense.io/6?t=891)] Richard wonders how Mike optimizes machine learning for his clients and how is DoiT International different than other people in this space in this field. He also tells us about Kaggle. [00:19:30 (https://podcast.curiefense.io/6?t=1170)] Find out what DoiT is doing in terms of open source. Also, Mike tells us about two open source tools, Kube No Trouble and Iris. [00:25:41 (https://podcast.curiefense.io/6?t=1541)] We learn Mike’s recipe for success. [00:27:04 (https://podcast.curiefense.io/6?t=1624)] Mike explains their ‘R&D time’ which is similar to Google’s ‘20% time.’ [00:30:26 (https://podcast.curiefense.io/6?t=1826)] Find out what’s next for Mike and what he’s super excited about right now. We also learn about the “voids” he mentions. [00:34:30 (https://podcast.curiefense.io/6?t=2070)] Mike tells us where you can follow him and learn more about what he’s doing. Quotes [00:19:53 (https://podcast.curiefense.io/6?t=1193)] “So, in order to do that, our founders have had to leverage open source technology and give back to open source.” [00:24:14 (https://podcast.curiefense.io/6?t=1454)] “Well, the need for security in today’s age, especially as people are going more remote, is only going to increase. So, I think you guys are very well positioned for kind of the wave ahead.” [00:25:39 (https://podcast.curiefense.io/6?t=1539)] “I would say, for me what’s always been I guess my recipe for success with working with engineers is being an engineer myself.” [00:28:03 (https://podcast.curiefense.io/6?t=1683)] “Another third is cloud support, and so when customers request help on the cloud, our team essentially acts like stack overflow.” [00:31:59 (https://podcast.curiefense.io/6?t=1919)] “Well, my theory, like I said, my theory is, in the future healthcare and security are going to be the big frontiers. The other, as you guys mentioned is AI and ML.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) 1.4.0 Roadmap Curiefense-GitHub (https://github.com/curiefense/curiefense/milestone/3) Mike Sparr Twitter (https://twitter.com/mikesparr) Mike Sparr Linkedin (https://www.linkedin.com/in/mikesparr/) DoiT International (https://www.doit-intl.com/) DoiT International Linkedin (https://www.linkedin.com/company/doitintl?trk=public_profile_topcard-current-company) DoiT International Blog (https://blog.doit-intl.com/) Cloud Native Ambassadors (CNAs) (https://www.cncf.io/people/ambassadors/) Kaggle (https://www.kaggle.com/) Kube No Trouble-GitHub (https://github.com/doitintl/kube-no-trouble) Iris-GitHub (https://github.com/doitintl/iris) DoiT International-GitHub (https://github.com/doitintl) Vadim Solovey Linkedin (https://www.linkedin.com/in/vadimska) Demystifying Machine Learning by Building an ML Pipeline (Part 1) by Mike Sparr (https://blog.doit-intl.com/demystifying-machine-learning-by-building-an-ml-pipeline-part-1-43fc30eba242) Demystifying Machine Learning by Building an ML Pipeline (Part 2) by Mike Sparr (https://blog.doit-intl.com/demystifying-machine-learning-by-building-an-ml-pipeline-part-2-dfd9ce786088) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Special Guest: Mike Sparr.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer Guest William Morgan Co-creator of Linkerd & CEO Buoyant Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the cloud native space. Today, our special guest is William Morgan, who is the CEO of Buoyant, one of the creators of Linkerd, and a former infrastructure engineer at Twitter. William shares the story of how he went from Twitter to Linkerd, and the company behind Linkerd called Buoyant. We find out what Finagle is and what William does to help make the open source community for Linkerd thrive, and how does Buoyant incentivize user engagement. Also, William shares advice for projects just starting out on how they can best build themselves to be successful, and everything he wants to accomplish with Linkerd. Download this episode to find out much more! [00:02:05] William fills us in on how he went from Twitter to Linkerd, to where he is today. [00:05:24] We learn about Finagle. [00:07:00] Justin wonders if Twitter has moved more to a modern cloud native stack or if it’s the same sort of JVM and all that other stuff. [00:08:26] William tells us what he means by “put things in the proxy.” [00:10:34] We find out what Buoyant does and how William dealt with governance issues when he first started out. [00:14:33] Justin’s been hearing rumors all over the CNCF landscape that William’s “pushing for graduation” and wonders what that means. [00:15:45] Justin asks William how many companies he thinks are using Linkerd in their stack. [00:20:15] Richard asks William what he does to help make the open source community for Linkerd thrive, what does Buoyant do to incentivize user engagement, and how does he make sure that people will go beyond just putting themselves as adopters. [00:23:34] William explains just like choosing what kind of open source project you want, choosing what kind of community you want is also as important. [00:25:36] Find out what’s next for Linkerd, how will William better serve the people in the open source community, and what he is looking forward to. [00:28:27] William shares advice on projects that are just starting out, like Curiefense or CNCF projects, and how they can best build themselves to work in this space as successfully as he has. [00:32:38] Find out where you can follow William online. Hosts: Richard Littauer · Justin Dorfman Guest: William Morgan Quotes [00:03:04] “But the infrastructure was changing when I started, there was this monolithic Ruby on Rails app called the MonoRail.” [00:03:10] “And by the time I left, we had replaced that, or almost replaced it with this massive, what we would now call a Cloud Native architecture, even though it wasn’t built on Docker, it wasn’t built on Kubernetes, it wasn’t even built on Containers.” [00:07:58] “And the modern Linkerd is totally different from, I mean the same heritage, but it’s written in Go for the control plane and Rust it for the data plane, and it’s like a fraction of the CPU and memory cost and much faster and so on.” [00:11:18] “Well, the best way to be successful is to have a commercial backer that is funding this project.” [00:15:01] “And step two is find the corresponding SIG so there’s a CNCF set of SIG’s and have them do an initial review.” [00:16:52] “I mean it’s like bribery. I’m like, just kind of blatant about it. I’m like, Hey, I will send you swag, like I will pay money to buy swag and send it to you if you add yourself to ADOPTERS.md.” [00:18:11] “The vast majority of users are just in the iceberg part that’s not the tip. The butt of the iceberg, that’s under the water, and we just never know about it.” [00:20:57] “There’s a line you have to walk, I think, between being the customer support person where it’s like yes, I will always help everyone with every single thing.” [00:21:32] “Linkerd is just as much their project as it is ours. The moment they use Linkerd they are an official Linkerd community member, or even Read the Docs, you’re officially a Linkerd community member.” [00:23:18] “I was just saying anytime someone gets some recognition, it really just, their self-esteem goes up, more productivity, they become a brand advocate and they talk about Linkerd and how awesome the community is.” [00:23:34] “Just like with kind of choosing what kind of open source project you want, I think choosing what kind of community you want is also something that you have to be intentional about.” [00:24:20] “It’s because the whole point of open source is that you have this self-reinforcing community, and that is bringing Linkerd to the rest of the world. They’re like infecting the world with Linkerd. That’s what I want!” [00:28:40] “So, I think for Curiefense, like Linkerd, I think one thing you really have to get clear about is what’s the relationship between the company behind it and the open source project.” [00:29:53] “Another thing we’ve learned is that adoption is weird. This Cloud Native space, I don’t know if it’s, I’ve got to imagine other spaces are like this too, but there’s a lot of fashion, there’s a lot of fad.” [00:30:28] “And who is writing the blog posts and jumping from one hype wagon to the next.” [00:31:07] “So, that’s my other advice, I guess, is understanding the nature of adoption and being able to pick apart real adoption from buzz adoption.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) William Morgan Twitter (https://twitter.com/wm) Linkerd (https://linkerd.io/) Linkerd Twitter (https://twitter.com/Linkerd?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Buoyant (https://buoyant.io/) Buoyant Twitter (https://twitter.com/BuoyantIO) Finagle (https://twitter.github.io/finagle/) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Special Guest: William Morgan.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer Guest Sergio Méndez Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the cloud native space. On today’s episode, we have a wonderful guest, Sergio Méndez, who is an SRE and Professor of operating systems, software engineering, and AI at San Carlos University of Guatemala. He is currently working at Wizeline as SRE and working on DataOps using Airflow, Kubernetes and Containers. Sergio is the the organizer of the Cloud Native Guatemala Community Group and is working to get students and people from Central America involved into the CNCF ecosystem. Today, we learn how Sergio got started in Cloud Native, what he does at Wizeline, how he got involved in Kubernetes, building Chatbots, getting involved in the Linkerd community, and using his passion to help students in Guatemala. Also, where does Sergio think the future is with the Spanish involvement in Cloud Native, and what projects he’s excited about in the future. Download this episode to find out so much more! [00:00:36] Richard gives Curiefense updates for this week! [00:02:39] Sergio tells us how he started getting involved in Cloud Native, open source, systems engineering, and being a professor at St. Carlos University of Guatemala. [00:03:45] Richard wonders if Sergio connects a lot with other people in his organization of the cloud data group, Guatemala community group, with people from Nicaragua, Columbia, or Mexico. He also tells us how many people are in his community group. [00:05:43] We find out what Sergio does at Wizeline, how he went from being an SRE to really being involved in Cloud Native, and how he started getting involved in Kubernetes. [00:11:18] Sergio did a talk at KubeCon 2020 on how to build Chatbots using Cloud Native based on some work he did with Telefónica. He fills us in on how he got that job and what he was doing there. [00:13:50] Justin wonders how Sergio got involved with the Linkerd community. [00:16:31] Richard wonders how Sergio is building that passion in the community that he’s building in Guatemala and how is he helping people learn about these cool things. Also, what talks go on and what is he offering to people who come to the events, especially during COVID. [00:19:58] Justin brings up a talk he saw that Sergio did in Spanish that gave him chills because he saw his passion, and Sergio shares a story. [00:23:48] Sergio tells us how he judges the open source movement in Central America and where he thinks the future is in terms of Spanish involvement in Cloud Native. [00:27:55] Justin asks Sergio if he thinks that Latin America is going to be like the next hub in terms of outsourcing just like India is. [00:30:50] Sergio informs us on what projects he’s super excited about in the future and what lights the fire in him concerning Cloud Native. [00:33:13] Find out where you can follow Sergio online. Spotlight [00:34:00] Justin’s spotlight is Jekyll. [00:34:13] Richard’s spotlight is two people, Alexis Palmer and Robert Henderson. [00:34:42] Sergio’s spotlight is Alex Ellis. Quotes [00:10:25] “So this guy was starting to move the talk and started to manage the attendees, and that kind of things, telling some jokes, and that kind of things, that’s really funny, really nice guy. So, I said, Oh, that guy is really nice! So, in that moment I said, Wow, maybe this guy doesn’t know anything about these technologies. Can you remind me what guy is this? Kelsey Hightower! Yeah, that was Kelsey Hightower! So, that is why I really admire very human people.” [00:11:57] “So, in that moment I remembered that I didn’t know too much about Kubernetes, so I say well, this is the opportunity to experiment with Kubernetes.” [00:12:27] “Yeah, it was really fun, we play in that moment a lot with Rancher. We create that Chatbots using Containers like in a serverless way.” [00:12:45] “But this one break of one of the CNCF ambassadors, Alex from OpenFaaS, so yeah, we used that break on that moment.” [00:14:05] “I think that I was exploring on the CNCF, let’s say exploring the landscape. I was really curious about that.” [00:17:34] “But the moment dies in the university too, so I said why not start something new with a new topic something more interesting and I started that group, and then I started to ask myself how to get involved with the students too, maybe they can pick this community to be their community, not my community.” [00:18:52] “I met Paulo, that’s another ambassador form Brazil. He’s trying to push Latin America to another level, to be more visible for CNCF and that kind of thing.” [00:19:58] “ I saw one of your talks in Spanish and it was I think really important because it’s showing that you can be inclusive when you’re talking about open source and cloud native stuff, and it just really kind of gave me chills and I’m not lying.” [00:23:01] “The impact that you have on the students and other countries that maybe don’t have too much access to this kind of technology, so it’s very excited and motivated for the students and that kind of things.” [00:24:43] “I think that the philosophy here in Guatemala, because we are like a third world country, they promote, or they see the benefits of just open source because you don’t have to pay a license and that kind of things.” [00:26:20] “But I think that the Latin people is the next force forward of working for something.” [00:27:46] “Mexico is really big to work on technology and they are working for big U.S. companies, so yeah, I think that is something that is growing right now.” [00:29:41] “I think for me, let’s say the that the top countries, as a working force for IT technologies are like, I can say that Mexico is a really huge one, and some South America countries like Brazil, Argentina, Columbia, and that countries are in the top of the working force for IT and working for foreign people.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Sergio Méndez Twitter (https://twitter.com/sergioarmgpl) Sergio Méndez Linkedin (https://www.linkedin.com/in/sergioarmgpl/) Sergio Méndez Website (https://sergiops.xyz/) Wizeline (https://www.wizeline.com/) Istio (https://istio.io/) Bridget Kromhout on Kubernetes-O’Reilly (https://www.oreilly.com/content/bridget-kromhout-on-kubernetes/) Protocol- “From McDonald’s to Google: How Kelsey Hightower became one of the most respected people in cloud computing.” (https://www.protocol.com/enterprise/kelsey-hightower-google-cloud) Ambassador spotlight: Paulo Simoes-Cloud Native Computing Foundation (https://www.cncf.io/blog/2020/10/29/ambassador-spotlight-paulo-simoes/) Jekyll (https://jekyllrb.com/) Alex Ellis Website (https://www.alexellis.io/) Alexis Palmer (https://scholar.google.com/citations?user=NVxAbD8AAAAJ&hl=en) Robert Henderson (https://www.rhenderson.net/) OpenStreetMap: Guatemala (https://www.openstreetmap.org/relation/1521463#map=8/15.736/-90.243) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Special Guest: Sergio Méndez.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer Guest Miles Ward Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the cloud native space. Today, our amazing guest is Miles Ward, who is CTO at SADA, which is Google Cloud’s number one resale and implementation partner. We find what Miles does at SADA, how he helped NASA with the Curiosity Mars Rover landing, a startup he did, his bizarre relationship with Twitter, and he shares some awesome advice for people who are interested in seeing outliers for Cloud Native stuff. Also, find out why Miles tells us how we could be using Kubernetes on the moon one day! Download this episode now to find out so much more! [00:00:59] We start off with some Curiefense updates. [00:02:51] Miles tells us about helping NASA with the Curiosity Mars Rover landing. [00:04:28] We learn what Miles does at SADA and what his experience was with the cloud at AWS. [00:08:44] Miles explains his “bizarre” relationship with Twitter. [00:12:24] Richard wonders why Miles switched from Google to SADA. [00:17:13] Miles tells us why he went to SADA and why wouldn’t Google keep him internal. [00:19:12] Find out what Miles is most excited about and what he’s doing with his extra powers these days. [00:21:23] Richard asks Miles if he has any advice to share for people who are interested in seeing outliers for Cloud Native stuff and how you can plan for that. [00:24:41] We learn if the military uses these data storage sites and diesel generators that Miles was talking about earlier, and if this one of his main clients. He mentions talks with Johnson Space Center using Kubernetes on the moon! [00:27:20] Richard used to work for IPFS and is curious to know what Miles thinks about the decentralized systems and how that’s going to go in the future. [00:30:35] Hear the story how Miles got the gig with Rover on Mars. [00:37:34] Find out where you can follow Miles on the internet. Spotlight [00:38:31] Justin’s spotlight is a CNCF project called Linkerd. [00:39:04] Richard’s spotlight is IPFS. [00:39:18] Miles’s spotlight is NGINX. Quotes [00:03:02] “It was a real trip watching the perseverance landing because it’s like I was in that room, like I remember the last time, let’s see if they don’t screw this up.” [00:07:15] “But Amazon, I spent, you know I built a startup and it was terrible, and we vaporized a bunch of shareholder value. It was outstanding.” [00:07:35] “We read papers about Hadoop and deployed Hadoop at scale, only like four months later to realize we were pronouncing it wrong and we didn’t know what we were doing, and Hadoop is kind of a cool thing now.” [00:08:06] “And I was like you have M1 larges with eight gigs of memory, seven and a half in fact, like not even a whole eight? What did you do with the other half gig?” [00:08:25] “It turns out you can build awesome big stuff with little tiny building blocks.” [00:09:45] “We were the second largest customer of that product behind the CIA.” [00:10:57] “And as a part of that I met Eric Schmidt through the Obama campaign I worked as a part of the 2012 OFA team to help design the data infrastructure for processing all of the campaign ads and analytics and all the rest of the stuff.” [00:11:43] “Google revenue was pretty small and in the five years that I was there we grew to one hundred twelve-fold. “ [00:11:49] “Andy Jassy has this quote at one of the reinvents, I presented I think the first reinvent had sixty sessions and I did seven, so it was a lot of content out there.” [00:11:57] “Jassy had this comment where it was like, “Look, the biggest impediment to AWS growth is the lack of a viable competitor. If only we had a real competition, then companies would have multiple options to select from and they will be able to move to cloud more quickly.” [00:12:31] “So, Google’s incredible. You’re going to build a startup, every one of your little departments and divisions, they’re own little startup, and you now officially have the coolest garage in which to build a startup that there has ever been.” [00:12:43] “They have a fifth of the x86 compute on the planet. They have a network six times the size of this other network you’ve heard of called the internet.” [00:12:52] “They’re putting thirty tons of net new disk in the data centers a day.” [00:13:16] “We were working on the project to work with Twitter to help them understand hey, look, we can handle it.” [00:13:24] “And they go, look, we have 330 petabytes in our Hadoop cluster.” [00:13:46] “Doug Cutting at Yahoo goes, well, this is hot. Let me see if I can sort of reimplement this. He chooses Java because I don’t know, he doesn’t like himself or something.” [00:14:18] “So we run Hadoop tests all the time, and then you can kind of squint back and we’ll performance test logs because they don’t run MapReduce anymore inside Google, but MapReduce is faster than it by a lot.” [00:15:09] “And a petabyte of solid-state disk is a tenth of a percent of the global production of that type of equipment as of that day.” [00:15:45] “You’re like, hold on a minute, ten petabytes of SSD as of 2016 is an absolutely galactic amount of infrastructure.” [00:16:01] “So, I spent a bunch of time with the Pokémon Go folks, they gave us hey, here’s this curve, here’s how much traffic you’re going to use, and here’s this really like worse case right here, and it’s super gnarly, it’s going to be like this and were like okay totally cool, and they used fifty three times that much in a week later.” [00:16:14] “At the peak of that they’re slightly bigger than Gmail.” [00:16:38] “And at the time they’re running on Kubernetes because they’re doing what we told them to do and they’re following best practice.” [00:17:32] “Oh, they love SADA, so they’re stoked.” [00:18:03] “And to be really tactical, like really clear about it, there’s an enormous amount of direct risk when I’m the one who pushes the magic button on your stuff.” [00:18:13] “And if I’m a Google engineer with access to the G3, the core central code base, the multi billion line mono repo that holds the source for surge and the source for MapReduce and every other thing, if I do the wrong commit and push some of the code in my repo into your repo, it’s extra bad.” [00:18:53] “In fact, I think the best of the hyperscalers interaction with open source, but we can really participate in those communities and plug in.” [00:19:19] “One of the big ones that I think is a real opportunity, we spend a lot of cycles thinking about the financial operations. I know it’s not as technical as you may like, but the reality is, totally depend on the financial details.” [00:20:04] “They build the product, they don’t use the product all day, so we use the product all day, and the result of that is I think we’re building a repository of information about its use.” [00:20:26] “It’s a spot where there’s a lot of room because I think your guys’ area in Cloud Native, there’s a bunch of confusion.” [00:20:54] “Like a bunch of noise about this container thing, is that actually a good idea? Does it actually save anybody any money? So that’s a spot where we spend a lot of cycles.” [00:21:34] “Ninety percent of the weird stuff people don’t notice cause they’re not logging.” [00:22:30] “So Google had one where a diesel generator just vaporized a bunch of diesel and just decided to be a fire canon to the sky. It was great!” [00:26:12] “We’re in talking now with the folks at Johnson Space Center about using Kubernetes on the moon, like I joke you not!” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Miles Ward Twitter (https://twitter.com/milesward) Miles Ward Linkedin (https://www.linkedin.com/in/milesward/) SADA (https://sada.com/) Apache Hadoop (https://hadoop.apache.org/) CLOUD N CLEAR Podcast (https://sada.com/insights/cloud-n-clear/) Bobak Ferdowsi (https://en.wikipedia.org/wiki/Bobak_Ferdowsi) Mars Curiosity Rover-NASA (https://mars.nasa.gov/msl/timeline/edl/) Linkerd (https://linkerd.io/) IPFS Documentation-GitHub (https://github.com/ipfs/ipfs-docs) NGINX (https://www.nginx.com/) Credits Executive Produced by Tzury Bar Yochay (https://twitter.com/tzury) Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Special Guest: Miles Ward.

View Details

Sponsored by Reblaze, creators of Curiefense

Panelists Justin Dorfman | Richard Littauer Guest Dmitriy Akulov Show Notes Hello and welcome to Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the cloud native space. Justin makes an announcement about a new release at Curiefense that you don’t want to miss. Today, we have our very first guest, Dmitriy Akulov, Founder of appfleet and jsDelivr. Dmitriy tells us an interesting story of how he came up with the idea of appfleet. We also learn about jsDelivr, what it is, the “bus factor” plan of jsDelivr, domains, his future plans, and his control panel idea. This podcast is about Cloud Native, so if you have a topic or have a speaker, please get in touch with us and let us know. Go ahead and download this episode now to find out much more! [00:00:49] Justin tells us what is currently going on with Curiefense. [00:03:11] Dmitriy tells us about appfleet and how many customers they have acquired since launching. [00:07:15] Justin wonders what kind of feedback Dmitriy got from the Google team. [00:08:12] Dmitriy tells us all about jsDelivr and he explains the ICP license and the restrictions in China. [00:11:48] Justin asks Dmitriy what made him say we need jsDelivr to be able to adopt microservices and architecture and all that stuff. [00:16:03] Dmitriy tells us the “bus factor” plan of jsDelivr and he tells us about who is on his team. [00:17:37] We learn about the domains from Dmitriy. [00:19:13] Richard wonders where Dmitriy is going next and what’s in store. [00:20:46] When talking about protocols, Dmitriy tells us if he sees gRPC or Redis coming down the pipe. [00:22:15] Justin wonders if takes an army of support engineers to manage this or if it’s small right now and Dmitriy explains. [00:23:57] Dmitriy fills us in on who he’s been getting the most interest from and if any have been cloud providers and about his infrastructure. [00:25:55] We find out about a company Dmitriy had previous to appfleet and what’s keeping him from going that route again. [00:30:22] Richard wants to know the downsides and where he would use AWS, also if Dmitriy has any external contributors to his code and if it’s open source. [00:33:24] Richard wonders if Dmitriy has any other ideas besides control panel. [00:34:57] Find out where you can follow Dmitriy. Spotlight [00:35:44] Justin’s spotlight is Katacoda. [00:36:18] Richard’s spotlight is Gulp. [00:36:37] Dmitriy’s spotlight is Envoy. Quotes [00:03:25] “I’ve always had this issue where I wanted to deploy my websites, my services, my API’s to the edge, like closer to users. So, imagine like a CDN which is for study content, but for actual goals, where it can actually run your whole thing on every single location globally. And five years ago, there wasn’t such a solution out there.” [00:04:31] “And we already had all that for the CDN itself because we use a multi CDN, but our API was close to New York and if you’re in Asia you’re stuck with that.” [00:05:34] “I was heavily inspired by Heroku, this idea where developer tools should be easy to use, and they should be beautiful.” [00:07:23] “Well, I guess everyone is always very nice, so it’s tricky to figure out are they nice because you have a good product or are they nice just because they’re nice.” [00:08:39] “So, currently we have I think almost 100 billion queries per month, and we push around 4 petabytes of bandwidth every month.” [00:13:52] “So that’s a much wiser deal living on like six servers in Europe.” [00:15:30] “We get everything we could to optimize for production use and this microservice thinking allows us to do stuff, like we can move this module to a different provider because they sponsored us for three months maybe, or we can move this module to a different hosting service because it’s cheaper there and it really helps us.” [00:16:44] “So, we, it’s basically me and my senior developer, Martin, and it’s basically just the two of us maintaining the whole thing.” [00:16:59] “About the bus factor, we actually have a Wiki page on jsDelivr which explains how everything should work in that case.” [00:19:15] “We keep expanding. I checked the stats a few months ago and we are basically doubling in size every year.” [00:22:36] “I have built another startup in the past. We were always small, but we always built big stuff.” [00:22:46] “So by now I know how to optimize early to make sure that the core of the system doesn’t break in the future no matter what.” [00:26:39] “There’s many differences in how things work in the U.S. and in Europe from the VC and startup perspective.” [00:29:03] “I’m doing everything in bootstrap. I think if you have numbers, the VC’s will sign any kind of contract you give them and not the other way around.” [00:32:15] ”What if there was an open source control panel to basically maintain your own CDN and then you could just hit deploy to put a container on appfleet, and now you have the infrastructure to log balancing, the routing, the bandwidth, and you also have a nice UI which you also get to follow the scan provider.” [00:36:37] “I would say Envoy. We use it on appfleet and it’s really cool because it allows us to do zero-downtime deployments on single instances.” Links Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense) community@curiefense.io (mailto:community@curiefense.io) Reblaze (https://www.reblaze.com/) Justin Dorfman Twitter (https://twitter.com/jdorfman) Richard Littauer Twitter (https://twitter.com/richlitt) Dmitriy Akulov Twitter (https://twitter.com/jimaek) Dmitriy Akulov Linkedin (https://www.linkedin.com/in/dakulov/) appfleet GitHub (https://github.com/appfleetcloud) appfleet (https://appfleet.com/) appfleet Twitter (https://twitter.com/appfleetcloud) jsDelivr (https://www.jsdelivr.com/) Katacoda (https://www.katacoda.com/) Coupon for appfleet CTCN20 (https://appfleet.com/) 20% recurring for 12 months Everyone gets $10 free on registration as a trial, after converting to paying customer the coupon will work. Note: This is not an advertisement, paid promotion, or endorsement Credits Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Special Guest: Dmitriy Akulov.

View Details

Hosts Richard Littauer · Justin Dorfman Hello and welcome to the debut episode of Committing to Cloud Native Podcast! It’s the podcast by Reblaze where we talk about open source maintainers, contributors, sustainers, and their experiences in the cloud native space. You may know Richard Littauer and Justin Dorfman from the Sustain podcast, and now they are starting this new podcast. In this episode, we learn all about Justin, who is the Open Source Program Manager at Reblaze, works on the Curiefense team, and is one of the Co-Founders of SustainOSS.org. Since this is our pilot episode, we will learn more about Reblaze, Curiefense, Envoy Proxy, CNCF, and what this podcast will focus on in future episodes. This podcast is about Cloud Native, so if you have a topic or have a speaker, please get in touch with us and let us know. Go ahead and download this episode now! [00:00:58] Justin tells us who he is, what he does at Reblaze, and what Reblaze and Curiefense are. [00:02:01] Justin explains Envoy Proxy and how it’s made more secure than it already is. [00:03:14] We learn what CNCF is, who runs CNCF, and what a hosting project means. [00:06:44] Justin tells us why getting supporters is a great deal for him. [00:09:13] Richard asks Justin if the CNCF gave him a badge after going through the process and Justin talks about the CII Best Practices Badge. [00:11:50] Find out what this podcast will focus on in the future. [00:13:08] Justin tells us some places you can go to find out more information and to contribute. Links Justin Dorfman Twitter (https://twitter.com/jdorfman) Richard Littauer Twitter (https://twitter.com/richlitt?lang=en) Reblaze (https://www.reblaze.com/) Curiefense (https://www.curiefense.io/) Curiefense Twitter (https://twitter.com/curiefense?lang=en) Cloud Native Computing Foundation (CNCF) (https://www.cncf.io/) Envoy (https://www.envoyproxy.io/) community@curiefense.io (mailto:community@curiefense.io) Sustain podcast (https://podcast.sustainoss.org/) Credits Produced by Justin Dorfman (https://www.justindorfman.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/)