The Technium: Recent Episodes
The Technium
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Lightning Network is a network of payment forwarders layered on top of Bitcoin to enable small, fast micropayments with low fees.
Links/Resources:
- https://lightning.network/
- https://www.swanbitcoin.com/a-look-at-the-lightning-network/
- https://web.archive.org/web/20150301190111/http://lightning.network/lightning-network.pdf
- https://lightning.network/lightning-network-technical-summary.pdf
- https://open.spotify.com/episode/1usKtNJonBszrq5lzad8eh?si=RvuAtKbjSqmyfT--FkNogg About routing
- https://www.kevinrooke.com/post/when-sending-money-is-instant-and-free
- https://www.kevinrooke.com/post/the-lightning-networks-next-big-use-case
- https://bitfury.com/content/downloads/whitepaper_flare_an_approach_to_routing_in_lightning_network_7_7_2016.pdf
- https://1ml.com/
- https://explorer.acinq.co/
- https://github.com/davidshares/Lightning-Network
- https://docs.lightning.engineering/
- https://twitter.com/kerooke
- https://blog.coinbase.com/is-the-bitcoin-lightning-network-for-real-26e47029687f
- https://vitalik.ca/general/2021/01/05/rollup.html (good summary on payment channels and how they compare to rollups, even though this is on ethereum network)
- https://www.impervious.ai/
- https://diar.co/volume-2-issue-25/
- https://www.youtube.com/watch?v=qUAyW4pdooA
- https://github.com/bcongdon/awesome-lightning-network
Show notes
- Patio11: https://twitter.com/patio11
- Flare routing: https://bitfury.com/content/downloads/whitepaper_flare_an_approach_to_routing_in_lightning_network_7_7_2016.pdf
- The nature of the firm https://onlinelibrary.wiley.com/doi/full/10.1111/j.1468-0335.1937.tb00002.x
Chapters:
[00:02:21] What is Lightning Network?
[00:08:53] How does it work?
[00:18:51] Payments without Infrastructure
[00:25:34] Routing the Network
[00:36:00] Low value, High volume
[00:39:42] Micropayments for API metering
[00:43:09] Micropayments for Open Source
[00:49:12] Idea Accrediation Network
[01:01:20] Structuring Our Society
[01:15:01] Micropayments Change Relationships of the Firm
[01:21:36] Taking a position
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Hypercore is a set of building blocks to build distributed peer-to-peer applications. It provides a variety of data structures, as well as a discovery mechanism to share data between peers. It is a suite of tools that are the Lego blocks to build peer-to-peer apps.
Chapters:
[00:00:00] Intros
[00:02:16] What is Hypercore?
[00:12:58] HyperDrive: A P2P File System
[00:17:45] IPFS vs Hypercore
[00:21:01] Bitfinex is using HyperBee
[00:23:20] A Web of Data vs. Web of Pages
[00:27:19] P2P Data Structures: Better Than P2P Files
[00:29:52] IOT: a perfect candidate for decentralization?
[00:34:30] P2P Databases for Civic Good
[00:39:41] P2P Databases for Scientific Good
[00:55:08] Impediments to adoption
[01:03:54] Is this too futuristic even for us?
[01:12:59] Outros
Resources:
How the Hypercore Protocol Works
Hypercore Protocol
Dat Ecosystem
On Dat://
What are the differences between IPFS and hyperdrive?
why-hypercore/FAQ.md at master · tradle/why-hypercore
Dripline: Taking the Mauve Pill: Exploring Alternatives to the Centralized Web
Dripline: Finding friends and staying safe: comparing peer discoverability and security in p2p networks
Dripline: Data Fast and Slow: Exploring Data Models and Mutability
Thinking about Filesystems, Databases, and Hypercores
Productizing P2P
https://github.com/automerge/pushpin
https://github.com/kickscondor/duxtape
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Datomic is an immutable database that borrows ideas from functional programming. We discuss how an immutable database changes the architectural possibilities of web apps.
Links/Resources:
- Datomic with Rich Hickey
- Database as Values with Rich Hickey
- Intro To Datomic with Rich Hickey
- KotlinConf 2018 - Datomic: The Most Innovative DB You've Never Heard Of by August Lilleaas
- Love Letter To Clojure: And A Datomic Experience Report - Gene Kim - Turning the database inside out
- Datomic - a scalable, immutable database system by Marek Lipert
- "Real-World Datomic: An Experience Report" by Craig Andera (2013)
- https://tonsky.me/blog/unofficial-guide-to-datomic-internals/
- https://www.infoq.com/articles/Architecture-Datomic/
- https://www.infoq.com/presentations/The-Design-of-Datomic/
- Talking about Datomic, Datalog, GraphQL, APIs
- What Datomic does to REST - Unofficial guide to Datomic Internals
- Datomic Documentation Overview
- The web after tomorrow
- APIs are about policy
Chapters:
0:00 Intros
[00:02:21] What is Datomic?
[00:05:01] The Immutable Database
[00:14:59] The N+1 Problem
[00:20:59] Inference and Logical Programming
[00:26:45] Database in the browser
[00:39:24] Reducing the Impedence Mismatch
[00:42:09] The Change in Perspective
[00:51:14] Data as Social Artifact
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Urbit is an operating system for a global networked computer with a global filesystem. It has its own virtual machine, programming language, OS kernel, and identity layer. It dubs itself as “A clean-slate OS and network for the 21st century.”
Links/Resources:
- https://media.urbit.org/whitepaper.pdf
- https://developers.urbit.org/overview
- https://developers.urbit.org/guides/core/hoon-school
- [https://vimeo.com/75312418
- http://moronlab.blogspot.com/2010/01/urbit-functional-programming-from.html
- https://urbit.org/blog/precepts-discussion
- COMMENTARY
- https://news.ycombinator.com/item?id=6438320
- https://alexkrupp.typepad.com/sensemaking/2013/12/a-brief-introduction-to-urbit.html
- https://thebaffler.com/latest/mouthbreathing-machiavellis
- https://wejn.org/2021/02/urbit-good-bad-insane/#tldr
- https://twitter.com/basileSportif/status/1544963208099909633
- https://www.vanityfair.com/news/2022/04/inside-the-new-right-where-peter-thiel-is-placing-his-biggest-bets
Chapters:
[00:00:00] Intros
[00:01:54] What is Urbit?
[00:06:30] The Functional Kernel Avro
[00:13:46] Application Network Ames
[00:18:07] Code Update Mechanism
[00:23:38] Hoon Lang and Nock VM
[00:33:56] Personal Computing through Digital Sovereignty
[00:38:53] A Personal Computing Experience You Own
[00:43:46] Cypherpunk Values
[00:45:32] A Computer as an Heirloom
[00:51:48] Philosophical Origins and Its Influence on Design
[01:01:09] Capitalism Softens Radical Ideas
[01:05:40] A computer you can't break
[01:12:48] A ghost in the shell for 100+ years
[01:17:05] What if the internet had identity and payments built in from the beginning?
[01:27:29] A Global ID
[01:30:37] A Digital Extension of the Self
[01:43:31] The Takeaway Despite the Roots
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Hyperfiddle is an app builder in Clojure/Clojurescript that uses a compiler to manage the network connections in the app for you. We discuss what makes code easy vs simple, arches for software, home-cooked apps, and the implications of a compiler for everything.
Links/Resources:
- https://www.hyperfiddle.net/
- https://hyperfiddle.notion.site/Reactive-Clojure-You-don-t-need-a-web-framework-you-need-a-web-language-44b5bfa526be4af282863f34fa1cfffc
- https://www.hytradboi.com/2022/uis-are-streaming-dags
- https://hyperfiddle.notion.site/UIs-are-streaming-DAGs-e181461681a8452bb9c7a9f10f507991
- How hyperfiddle changed over time
- https://web.archive.org/web/20180217034740/http://www.hyperfiddle.net/
- First video proof of Hyperfiddle's "distributed dataflow" architecture for server-streamed UI. "UI as an Expression” https://twitter.com/dustingetz/status/1474050461745528839
- https://github.com/hyperfiddle/hyperfiddle-2020
- https://www.youtube.com/watch?v=6888V9YsObM
Show Notes:
- Have you tried rubbing a database on it? https://www.hytradboi.com/
- Self-adjusting Computations https://www.youtube.com/watch?v=G6a5G5i4gQU
- Sloth ecosystems https://www.youtube.com/watch?v=VU9Tn_Qkjb8
- Simple Made Easy https://www.youtube.com/watch?v=SxdOUGdseq4
- Apps can be home-cooked meals https://www.robinsloan.com/notes/home-cooked-app/
- Bundling and Unbundling https://reallifemag.com/bundling-and-unbundling/
- Alan Kays: STEPS http://www.vpri.org/pdf/tr2012001_steps.pdf and the Niles Compiler https://github.com/damelang/nile https://raw.githubusercontent.com/wiki/damelang/nile/socal.pdf
Chapters:
0:00 Intros
[00:02:05] What is Hyperfiddle?
[00:08:53] Managed Network through a Compiler
[00:15:49] Arches for Software
[00:19:21] Beyond Low Code
[00:26:39] Abstractions: Simple vs Easy
[00:29:14] Core of what's hard about apps
[00:33:16] The Eight Fallacies
[00:34:54] Home-cooked apps
[00:40:28] The Piped Piper Dream
[00:45:57] Reports for everyone!
[00:51:35] A compiler to help manage state?
[00:55:12] A Compiler for the Metaverse
[01:01:44] Making Integration Smoother
[01:04:38] OG "bare metal"
[01:12:01] Hypertext as Application State
[01:20:03] Compilers do hard things all the time
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world. Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Generative AI models are quickly reaching the point where their work is indistinguishable from that of humans. This include 3D models, voice, and other media types that were not previously possible to create without significant expertise
Resources
Google's New AI: Flying Through Virtual Worlds! 🕊️
OpenAI Dall-E 2 - AI or Artist? Which is Better? 🧑🎨
NVIDIA's New AI: Next Level Image Editing! 👌
NVIDIA's New AI Draws Images With The Speed of Thought! ⚡
https://gptcrush.com/
Google's Imagen AI: Outrageously Good! 🤖
http://gaugan.org/gaugan2/
DALL·E 2 and The Origin of Vibe Shifts
Chapters:
0:00 Intros
3:42 The amazing progress
7:32 Dall-E 2 and Imagen
11:11 NERF for 3d Scene Generation
17:03 GauGan for Image Editing
18:42 Models can understand the world
21:45 Democratizing Content Creation
24:46 Content to fill the metaverse
28:58 AI = An Army of Interns
31:17 Will Human Skills Atrophy?
34:31 What is the role of humans in creation?
45:03 What is the value of art anymore?
52:43 NFTS, Artists, and "Vibes"
58:59 Feedback Loops and Memeification
1:09:03 Why isn't everyone losing their minds?!
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Zig is a system programming language without the ugliness of C. We discuss its compile time features, its strive towards simplicity, and the potential for resource-efficient software.
Links/Resources:
• https://ziglang.org/learn/overview/
• https://www.youtube.com/watch?v=Gv2I7qTux7g
• https://www.youtube.com/watch?v=Z4oYSByyRak
• https://www.scattered-thoughts.net/writing/assorted-thoughts-on-zig-and-rust/
• https://www.scattered-thoughts.net/writing/how-safe-is-zig/
• https://ziglearn.org/
Chapters:
0:00 Intros
4:54 The Explicitness of Zig
12:16 Compile Time Programming
17:39 A Nice Build System that's Nice to You
19:59 Zig's Integration with C is that it's a C compiler
24:37 Compile time as Metaprogramming
26:23 Compile time for String Interpolation
28:33 Compile time for Queries
31:12 Maybe We'd Get Faster Programs!
33:55 Counter-forces to resource-efficient computing
41:16 System programmers as far as the eye can see
44:11 Zig as Pedagogy
45:48 Cross-compilation and DSL it to the Blockchain!
51:10 Proliferation of Zig Everywhere
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Multimodal and multitask models are machine learning models that can generalize. Multimodal models can generalize to understand different types of input, for example images and text. Multitask models can generalize their knowledge by applying what they’ve learned about one task to solve another task.
Links/Resources:
• MUM: https://blog.google/products/search/introducing-mum/
• Gato: https://www.youtube.com/watch?v=wSQJZHfAg18
• MIA: https://www.youtube.com/watch?v=L9kA8nSJdYw
• Flamingo: https://www.deepmind.com/blog/tackling-multiple-tasks-with-a-single-visual-language-model
• Flamingo explaining a funny photo: https://twitter.com/MelMitchell1/status/1522642194741538817
• Is LaMDA Sentient?: https://cajundiscordian.medium.com/is-lamda-sentient-an-interview-ea64d916d917
Chapters:
0:00 Intros
2:33 Multimodal and Multitasks Models
6:50 Deepmind's Gato: The All-Rounder Athlete
14:43 Google's MUM: The Search Assistant
18:12 Deepmind's Multimodal Interactive Agent: The Domestic Helper
22:31 Deepmind's Flamingo: Reasoning about Pictures
26:45 Why are these mind-blowing?
31:20 Machine Learning has come a looooong way
35:21 Could Flamingo be the real JARVIS?
38:56 Could MIA assist the elderly?
43:05 Multimodal AI for self driving cars
51:15 Multitask = A Shared Brain That Learns Everything
1:00:19 Could these models transcend human knowledge?
1:08:50 Breaking news: AI models are sentient
1:10:37 Is this just a local maximum or a path to AGI?
1:11:50 Outros
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Plan 9 is distributed operating system and it is designed to make a network of disparate computers function as a single system no matter where they are.
Links/Resources:
https://9p.io/wiki/plan9/papers/
Chapters:
0:00 Intros
2:49 What is Plan 9?
6:15 Unique Features
9:23 Everything is a File
16:53 Assembling a Distributed Computer
21:42 Powerful Abstractions Make Writing Powerful Programs Easy
29:01 Was Plan 9 The Original Kubernetes?
34:33 Can Principled Software Succeed in the Marketplace?
40:14 Running WebAssembly on Plan 9
44:16 Running An Intranet on a Distributed OS
45:38 Remote File Systems for Local-first Software
51:27 How do we get this future?
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Matrix is a decentralized chat protocol that is end-to-end encrypted. It lets users communicate with each other using different clients and even
run their own independent chat servers.
Links/Resources:
- https://matrix.org/docs/guides/introduction
- https://salibra.com/p/the-death-of-jabber-8ea395e82f5b- http://jabbermania.blogspot.com/2013/06/how-google-pulled-plug-on-public-jabber.html- https://news.ycombinator.com/item?id=25850832- https://lukesmith.xyz/articles/matrix-vs-xmpp
- Beeper: https://www.beeper.com/
- Moxie's post on decentralization: https://moxie.org/2022/01/07/web3-first-impressions.html
Chapters:
0:00 Intros
2:23 What is Matrix?
3:58 A Brief History of Chat
11:51 Origins of Matrix
15:14 What attracted people to Matrix?
22:35 Motivation for Beeper
25:14 Bridges to other networks32:09 Separation between protocol and client38:42 Chat overload
43:19 Network Fragmentation
47:48 The Chat Future to Look Forward To
52:14 A Future of Specialized Clients
58:01 Chat bots and Chat tools
1:02:57 How do we get this future?
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
The semantic web was an effort to mark up webpages with facts about their content. The dream was to enable computers, to understand the world by reading these webpages. We discuss why the effort failed, how it relates to symbolic AI/knowledge graphs, and how the vision could still be realized today.
Links/Resources:
- Introducing the Knowledge Graph
- https://twobithistory.org/2018/05/27/semantic-web.html
- Metacrap: Putting the torch to seven straw-men of the meta-utopia
- Whatever happened to the Semantic Web? https://news.ycombinator.com/item?id=16806657
- http://ai.stanford.edu/blog/introduction-to-knowledge-graphs/
- Aggregation Theory: https://stratechery.com/concept/aggregation-theory/
Chapters:
0:00 Intros
2:12 What is the Semantic Web?
3:59 The Early 2000's
8:32 The Idealism of the Semantic Web
11:30 How it worked
18:19 Roots in Symbolic AI
22:53 The OG Web 3.0
25:28 Why Did the Semantic Web Fail?
35:46 A Misalignment with the Medium
41:10 Crypto and Semantic Web Collide
43:25 Deep Learning and the Semantic Web Collide
47:09 Vestiges of the Semantic Web
49:44 Path to Intelligent Agents
55:03 Incentive structures
1:02:49 Token Curated Registries and the Semantic Web
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
TLA+ is a language for formally modeling and verifying the correctness of programs, especially concurrent and distributed ones. We discuss how it could be used to improve the practice of software engineering and what it would take for formal methods to gain more popularity.
Links/Resources:
- TLA+ Homepage https://lamport.azurewebsites.net/tla/tla.html
- https://learntla.com/introduction/
- https://lamport.azurewebsites.net/video/videos.html
- https://lamport.azurewebsites.net/tla/tutorial/contents.html
- https://lamport.azurewebsites.net/tla/industrial-use.html
- https://cacm.acm.org/magazines/2015/4/184701-how-amazon-web-services-uses-formal-methods/fulltext
Chapters:
0:00 Intros
2:14 What is TLA+?
5:10 Bugs in Concurrent and Distributed Programming
12:58 What's it like to use?
19:09 Global State Machine
20:34 Testimonials of Industrial Use Cases
24:56 Why is it not more popular?
27:32 How do we currently design our programs?
37:16 Everyone is a distributed systems programmer now
41:29 Thick client app management
49:47 Can haz use for Security?
53:08 TLA+ is for the design, not implementation
59:35 TLA+ all the things
1:02:55 Formal methods in Crypto?
1:08:19 Intrigued with Formal Methods
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Transformers are a building block of Machine Learning systems that have seen great success recently at subsuming all other techniques.
We discuss at a high level, its attention mechanism and its multimodal properties, and the types of applications this can be put to use now and in the future.
Links/Resources:
- Introductions to transformers
- https://www.machinecurve.com/index.php/2020/12/28/introduction-to-transformers-in-machine-learning/
- https://deeplobe.ai/machine-learning-for-transformers-explained-with-language-translation/
- https://www.youtube.com/watch?v=yGTUuEx3GkA
- Attention is All you Need https://arxiv.org/pdf/1706.03762.pdf
- Attention: https://distill.pub/2016/augmented-rnns/
- Transformers replacing CNNs https://becominghuman.ai/transformers-in-vision-e2e87b739feb
- AI models consolidating https://twitter.com/karpathy/status/1468370605229547522?ref_src=twsrc^tfw
- GTP implementation https://github.com/karpathy/minGPT/blob/master/mingpt/model.py
- Google introduces new arch to reduce cost of transformers https://analyticsindiamag.com/google-introduces-new-architecture-to-reduce-cost-of-transformers/
- LaMDA by Google https://gpt3demo.com/apps/lamda-google
Chapters:
0:00 Intros
1:57 What are Transformers?
4:59 How does it work at a high level?
9:27 Self Attention Mechanism
14:03 Input structure agnostic
16:59 Stack it high, pump it with data
25:28 More MultiModal Learning
27:22 The Narrow Waist
34:34 Transformers for Compilation
40:24 Specialized Hardware
43:52 Multimodal Applications
47:19 Generating Media as a Self-sustaining Entity
52:42 The Jobs this Destroys
58:02 Two machines need to talk to each other
1:04:42 A Young Lady's Primer
1:11:19 Try them out!
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
CORBA is an acronym that stands for common object request, broker architecture. It provides interoperability among distributed objects regardless of whether they're remote or local, regardless of whether they're written in different languages or in different locations of the network.
We discuss its influence on distributed systems, and where we are today with these large scale systems
Links/Resources:
Chapters:
0:37 Intros
2:20 What is CORBA?
4:08 90's Throwback
7:25 The Idea of Distributed Objects
9:55 How does it work?
13:16 The Promise of Interoperability
15:35 Influence on Computing Today
18:52 Why did CORBA fail?
24:39 Lots of People Missed the Web
30:09 Distributed Computing
36:06 The REST model of Distributed
40:35 RPC lives on today
47:13 Hard to get this right
52:08 Negotiating Interfaces from Scratch
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Unison is a programming language for distributed systems by leveraging immutable and content addressable code.
We discuss the many implications of a language whose code is content addressable, from fast rename, easy caching, to feature flagging.
Links/Resources:
- Unison Lang https://www.unisonweb.org/
- Unison at StrangeLoop https://www.youtube.com/watch?v=gCWtkvDQ2ZI
- Unison and its Scala-based Runtime https://www.youtube.com/watch?v=knqlWboqf_U
- What color is your function? https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/
- Python Environment XKCD https://xkcd.com/1987/
Chapters:
0:48 Intros
2:32 What is Unison?
6:46 Advantages of Content Addressable Code
15:39 Realizing the Scale before it's Too Late
22:03 Controlled Refactoring Experience
25:54 Abilities or Algebraic Effects
31:08 Programming the Whole Network
34:57 Facilitating the Culture around Coding
50:37 A Library Ecosystem
1:00:50 Reproducibility and Old Computers
1:07:55 Hot Patch Your Server
1:10:37 The Road to Popularity
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Smalltalk was one of the first object oriented programming languages and a software authoring environment that was highly influential in the history of computing.
We discuss its designer Alan Kay, his vision of personal computing, and the lasting influence of Smalltalk despite its limited commercial adoption.
Links/Resources:
- Actor Model - https://www.brianstorti.com/the-actor-model/
- https://dspace.mit.edu/handle/1721.1/6272
- Smalltalk Demo - https://www.youtube.com/watch?v=uknEhXyZgsg
- Smalltalk Build your own Debugger - https://youtu.be/NqKyHEJe9_w?t=355
- Early History of Smalltalk - http://worrydream.com/EarlyHistoryOfSmalltalk/
- “A remarkable number of artists, scientists, philosophers are quite dull outside of their specialty (and one suspects within it as well). The first siren's song we need to be wary of is the one that promises a connection between an interesting pursuit and interesting thoughts. The music is not in the piano, and it is possible to graduate Juilliard without finding or feeling it.”
- Why was smalltalk not a commercial success?
- http://www.wirfs-brock.com/allen/posts/914
- https://gbracha.blogspot.com/2020/05/bits-of-history-words-of-advice.html
- Lively and Caffine
- https://lively-kernel.org/
Chapters:
0:00 Intros
1:57 The Retro Future
2:34 Origin and Motivation of Smalltalk
7:20 Object Orientation is about message passing
11:51 Meta-circular inspiration
14:31 Smalltalk is an entire environment
18:59 Designed with the GUI in mind
24:35 The Music isn't in the Piano
27:21 Taking a bet on the future
31:42 Surprising things about Smalltalk
34:27 Learning to Program By Reading "Literature”
39:56 Did Smalltalk Achieve It's Goals?
44:43 The Failure of Enterprise Smalltalk
52:30 Even Good Ideas Have to Survive the Marketplace
55:23 Big Ideas Meet Reality
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545
Dynamicland is a communal computer where the computer is the room. It's a new dynamic, medium to think about systems with other people.
We discuss taking computation out of the realm of abstraction and putting it in the realm of the physical world, it, and how it breaks down barriers to entry.
Chapters:
0:00 Intros
2:10 What is Dynamicland?
11:17 The Communal Aspect of Computing
18:30 Computing other than Symbol Manipulation
31:29 The Concrete System
38:43 Realtalk is Declarative
43:39 Leverage the Real World for Compute
46:21 Squint beyond the Art Project
49:07 Having your thoughts out in the open
53:55 The Libraries of the Future
1:01:25 Interactive decision making
1:06:59 Throwaway Computing
1:19:46 Dynamicland for Games
1:23:08 A wish for the future
Links/Resources:
- Dynamicland website: https://dynamicland.org/
- Research Agenda: http://worrydream.com/cdg/ResearchAgenda-v0.19-poster.pdf
- Humane representation of thought: https://vimeo.com/115154289
- Seeing Spaces: http://worrydream.com/SeeingSpaces/
- Embodiment and Visualization: https://vimeo.com/114252897
- Notes about the experience
- https://dynamicland.org/research-notes/social-dynamics-of-programming-together/
- https://omar.website/posts/notes-from-dynamicland-geokit/
- http://vitor.io/on-dynamicland**](http://vitor.io/on-dynamicland)
- https://tashian.com/articles/dynamicland/
- https://www.phenomenalworld.org/analysis/the-next-big-thing-is-a-room/
- https://futureofcoding.org/notes/bret-victor/dynamicland.html
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
Non-Fungible Tokens (NFTs) are unique digital collectibles that are represented on a blockchain in the form of a token. NFTs make it possible to own digital media assets in the same way that you can own a digital currency asset, like Bitcoin.
We discuss various applications of NFTs outside of art, and its potential as a building block for future applications.
Links/Resources:
- Girl with House on Fire Meme. https://knowyourmeme.com/memes/disaster-girl
- Ethereum Name Service https://ens.domains/
- Party Bid to Fractionalize NFTs https://www.partybid.app/
- Flash Loans https://coincodecap.com/flash-loan
- Aavegotchi https://aavegotchi.com/
- Uniswap https://uniswap.org/
- 3LAU and NFTs https://decrypt.co/59788/musician-3lau-sells-album-in-nft-form-for-11-million
- Demand Curves https://corporatefinanceinstitute.com/resources/knowledge/economics/demand-curve/
- Davie Bowie on the relationship of media and the internet https://www.youtube.com/watch?v=FiK7s_0tGsg&t=636s
- Skeuomorph https://en.wikipedia.org/wiki/Skeuomorph
- ERC-998 Composable standard https://github.com/ethereum/EIPs/blob/master/EIPS/eip-998.md
- Neolastics https://neolastics.com/
- Latent Space https://stats.stackexchange.com/questions/442352/what-is-a-latent-space
- Jack Dorsey Tweet for $2.9 million https://v.cent.co/tweet/20
- Schelling Point https://medium.com/@willemvandenbergh_85885/on-schelling-points-network-effects-and-lindy-inherent-properties-of-communication-c4eb69b55c60
- Invoice Refactoring https://en.wikipedia.org/wiki/Factoring_(finance)
- Accruing Value in a Value Chain https://stratechery.com/2015/netflix-and-the-conservation-of-attractive-profits/
- The Technium NFT! https://opensea.io/assets/matic/0x2953399124f0cbb46d2cbacd8a89cf0599974963/73071695664951725756103887812533794393262849524101052862455651661423314993153
Chapters:
0:14 Intros
2:00 What are Non-fungible Tokens?
7:21 Ethereum Name Service domains are NFTs
20:05 "Non-fungible" only notable in the digital world
22:54 NFTs are Digital Native Objects
26:21 Fractionalizing NFTs
31:46 NFTs as Collateral
36:02 NFT as Social Proof
42:15 NFT as meme-ified finance
44:36 Permissionless Composability of NFTs
51:15 NFTs for Community Building
55:41 The Curator Economy
1:00:29 Creator Feedback Loop with Fans
1:07:15 Permissionless Remixing
1:10:28 We're still making Skeuomorphs
1:13:43 Content Agnostic and Responsive
1:20:37 Owning the Latent Space
1:24:09 Financialization of Memes
1:30:50 NFTs as boring assets
1:42:27 NFTs Scaled Up and Down
1:49:50 Yet Another Building Block
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
IPFS is a distributed storage network. The content is accessible through peers located anywhere in the world that might relay information store it or both, and IPFS finds data by its content address rather than its locations.
We discuss the main principles behind IPFS, the current use cases, and how it changes the basic unit economics of some businesses, as well as its interplanetary future.
Chapters:
00:00 Intros
02:25 What is IPFS?
13:58 Three Principles of IPFS
15:01 Content-addressable URIs
18:44 Content Linking in a DAG
21:27 Distributed Hash Table for Discovery
22:48 Pinning Content
30:58 Censorship-resistance
36:38 Used for NFTs
39:04 Used for Video and Music Streaming
42:56 Use for Package Manager
49:14 Use for Machine Learning
54:38 Interplanetary Linked Data
56:37 A Key Building Block
59:22 As a Public Good
01:06:36 Developers Tools on top of IPFS
01:10:07 Shifting Operational Burden
01:18:15 The Interplanetary Future
Links/Resources:
- Content Addressing https://simpleaswater.com/ipfs-cids/
- Linked Data https://ontola.io/what-is-linked-data/
- Distributed Hash Tables https://www.cs.cmu.edu/~dga/15-744/S07/lectures/16-dht.pdf
- Napster, Kaaza, Gnutella https://www.slideshare.net/uschmidt/peertopeer-systems/20-Comparison_Napster_Gnutella_KaZaAType_of
- Juan Benet of IPFS https://research.protocol.ai/authors/juan-benet/
- ProtoSchool https://proto.school/
- Marc Andressen’s Blog Archive https://pmarchive.com/
- Left Pad Debacle https://www.davidhaney.io/npm-left-pad-have-we-forgotten-how-to-program/
- NPM as a private company https://www.youtube.com/watch?v=MO8hZlgK5zc&t=46s
- Transfer Learning https://builtin.com/data-science/transfer-learning
- Deno Programming language https://deno.land/
- IPLD https://ipld.io/
- Jack Dorsey Regrets Shutting down API https://www.revyuh.com/news/software/developers/twitters-founder-admits-that-shutting-down-the-api-was-worst-thing-we-did-it-affected-users-and-developers/
- Datomic https://www.datomic.com/
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
End-user programming is a vision of computing that allows people who are not professional software developers to program their own computers.
We discuss some of our favorite end-user programming environments, the qualities that make an end-user programming environment successful and what the world would look like if everyone could program.
Chapters:
0:00: Intros
3:15: What is end-user programming?
6:28: Hypercard: An Early Pioneer
15:22: Quality #1 : Embodiment
17:52: Quality #2: Liveness
19:51: Quality #3: In-Place Toolchain
23:10: Elitism: Programmers vs. Script Kiddies
26:37: Yahoo Pipes: A Web 2.0 Programming Environment
32:25: What makes an environment successful?
40:28: Twitter, Hashtags, and "Situated Software”
47:36: Is programming just inherently hard?
1:04:26: Modern EUP Environments
1:10:39: The World If Everyone Could Program
1:15:01: Programming in the Metaverse
Links/Resources:
[0] https://www.inkandswitch.com/end-user-programming/]
[1] https://www.robinsloan.com/notes/home-cooked-app/
[2] https://www.johndcook.com/blog/2011/07/21/software-exoskeletons/
[3] https://www.gwern.net/docs/technology/2004-03-30-shirky-situatedsoftware.html
===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!
===== Socials =====
This week, we mull on an obscure programming language called Datalog and its outsized impact on the world
In this episode, we muse about the creativity of machine learning models and what it means for the future of content creation/ownership
This week on The Technium, we talk about Zero Knowledge Proofs, a relatively new cryptographic primitive that lets people prove they know a secret without divulging it.
In this week's episode, we ruminate on Webassembly and the dream of writing once and running everywhere.
This week, we talk about the Decentralized Autonomous Organization. How could they bring a new flavor to collective action without a centralized authority?
This week on the Technium, we talk about functional programming invading mainstream programming languages
In this week's episode, we talk about the React Reconciler. What is it? How would it help with building cross-platform frontends? What else could it be applied to?
In this week's episode, we talk about "The Metaverse". What is it, and why it excites technologists about a possible vision of the future.
In this week's episode, we talk about Ink and Switch's manifesto on local-first software, a vision on building software that merges the best parts of both web apps and desktop apps.
In this week's episode, we talk about tools for thinking, note taking apps, and the retro vision for the web.