We talk with Oleg Tarasenko and Tze Yiing about crawling the web using Elixir. Oleg created the crawly project to help solve this problem and Tze Yiing joined him as a contributor and maintainer. We cover how Elixir is well suited to orchestrate crawling, how to deal with login pages, understanding the legal concerns, building a codeless scraper and much more!

Show Notes online - https://thinkingelixir.com/podcast-episodes/031-crawling-the-web-using-elixir-with-oleg-tarasenko-and-tze-yiing