Scaling Knowledge: Recent Episodes
Moritz Wallawitsch
Scaling Knowledge is a blog and podcast about progress, epistemology, and AI. more at https://scalingknowledge.substack.com/about
Anton on Twitter: https://twitter.com/atroyn
Moritz on Twitter: https://twitter.com/MoritzW42
Scaling Knowledge: https://scalingknowledge.substack.com
Mentioned Resources
- Unlimiformer: Long-Range Transformers with Unlimited Length Input: arxiv.org/abs/2305.01625
- Memorizing Transformers: arxiv.org/abs/2203.08913
- Chroma: https://www.trychroma.com
- You and your Research: https://www.cs.virginia.edu/~robins/YouAndYourResearch.html
- Against Method: https://en.wikipedia.org/wiki/Against_Method
Contents:
(00:00) Preview
(01:14) Introduction
(06:13) Edech
(12:12) Airchips and Square Finances
(24:06) Technical Optimism
(25:09) Convenience vs Enabling Technology
(31:13) Chroma and LLM Landscape
(35:37) Minecraft Bot
(37:05) Knowledge Retrieval
(39:38) New Knowledge Retrieval Research and Techniques: (Direct Retrieval Augmentation of Transformers, Conditional Retrieval)
(42:35) AI Native Platforms
(45:43) Theorem Proofing and Generation
(49:40) Do LLMs Reason
(52:18) Computational Complexity and LessWrong
(53:58) Frameworks and Taking Your Work Seriously
(55:53) Job Displacement and Jobs Supply vs Demand Side
(58:29) The Printing Press
Twitter: Moritz (@moritzW42)
Blog: https://scalingknowledge.substack.com
About Erik: https://en.wikipedia.org/wiki/Erik_J._Larson
Erik's book: https://www.amazon.com/Myth-Artificial-Intelligence-Computers-Think/dp/0674983513
Transcript: https://scalingknowledge.substack.com/i/135360356/transcript-rawapproximation
Content
(00:00) Preview
(01:33) Intro
(06:55) Unpredictability of Invention
(10:09) Limits of Deep learning progress
(12:59) Abduction
(16:45) Creativity and Serendipity
(19:16) Neumann on Self Improving Machines
(24:30) His upcoming book
(26:40) Progress & Innovation decentralisation
(28:21) Neurosymbolic AI
(30:29) AI research progress
(33:16) Automation, Displament, and Alienation
(34:58) Outro
Created with wondercraft.ai (get 50% off with SCALINGKNOWLEDGE50)
Keith Rabois
- Twitter: twitter.com/rabois
- Wikipedia: wikipedia.org/wiki/Keith_Rabois
Mo Golshan
- OG written content version
- Mo's twitter: twitter.com/mogolshan
Mentioned links:
-
Interview with John Sculley on Apple and Steve Jobs: John Sculley On Steve Jobs, The Full Interview Transcript | Cult of Mac
-
Read The Cognitive Style of PowerPoint by Edward Tufte: http://www.edwardtufte.com/tufte/powerpoint
-
Achieving Business Success One Person at a Time* by Peter Engel: The Exceptional Individual: Achieving Business Success One Person at a Time by Engel, Peter
-
Advisory board: Should Your Startup Have an Advisory Board? | by Mark Suster | Both Sides of the Table
-
Essay on hiring great PMs: How to Hire a Product Manager: the Classic Essay by Ken Norton
- What is more important for a start-up: quality of team or size of market? pmarca archive.posterous.com/the pmarca guide to startups part 4
- How to Raise money before launch? How to raise money before launch. by Delian Asparouhov
- Should a student finish college or go work for a startup if given the chance to work for a YC, TechStars alumni? A Student's Guide to Startups
Content:
(00:00): Intro
(01:00): Navigating the Ambitious Problem-Solving Journey: From Start to Success
(28:34): Navigating Startup Operations: From Team Building to Executive Hiring
(28:58): "Lessons on Assembling Your Founding Team: Complimentary Co-founders, Cult Creation, and Problem Solving"
(01:41:31): "Exploring Venture: Lessons on Assembling Founders, Succeeding in Early Stage Investing, and Venture Capital"
(01:41:53): "Assessing Extraordinary Founders: Traits, Strategies, and Unconventional Qualities"
(02:34:31): "Productivity and Life Advice: Strategies for Success in Venture and Beyond"
(02:34:50): Contrarian Thinking and Managing Time: Lessons for Success
(03:06:20): Outro
We discuss qualitative and quantitative moats, midwits, altruism, egalitarianism, regulation, obstructionism and more.
- Brian (@psychosort)
- Moritz (@moritzW42)
Blogs
- Brian: From the New World
- Moritz: Scaling Knowledge
Content:
(00:00) Brian's background
(03:20) On writing
(07:05) AI doomers: steel man and critique
(14:05) Job displacement
(19:20) Signaling jobs
(28:20) Future economics of art
(31:10) Qualitative v. quantitative moats (Google doc leak)
(37:00) Midwits and conformity selection
(40:00) How to fix Midwits phenomenon (Civil service reforms)
(42:00) Why the free market selects against the free market
(46:00) Problems with regulators (FDA and co.)
(52:00) The root of Altruism
(55:00) Origin of envy
(55:25) Conspiracy theories and egalitarianism
(1:00:00) Egalitarian murders
(1:06:45) Polarization and obstructionism
(1:09:05) Obstructionism and regulation
(1:10:25) Issue polarization, affective polarization, and partisanship
(1:12:00) Education system and the Freddie deBoer case
Part 1: Naval and others on Product Design, Smart Glasses, the AI Industry, 10x Designers, and more.
A collection of conversations from Airchat
Find Scaling Knowledge on other platforms
Full transcript
Content
(00:00) Intro
(00:58) Amazon Gentrification
(02:01) Freedom
(04:22) Innovation
(05:25) Consensus
(07:00) Numeracy Skills
(08:19) Mastery and Creativity Intersection
(09:20) Fun
(10:41) Curiosity
(13:48) Nassim Taleb and Deutsch
(18:20) Identity and Epistemology
(23:20) Evolution of Truth
(24:06) Naval on organized religion
(26:42) Curiosity
(26:42) Zone of Genius
(29:53) The great Scientists
(33:00) Nature vs. Nurture
(40:09) Outro
Part 2: Naval and others on Freedom, Consensus, Zone of Genius, Identity, and the Evolution of Truth [Airchat Part 2]
A collection of conversations from Airchat.
Find Scaling Knowledge on other platforms. Scaling Knowledge blog.
Full transcript here.
Content
(00:00) Intro
(00:56) Simplicity
(05:34) Design feeling natural
(06:46) Smart Glasses
(14:00) AirPods
(16:06) Humane (hu.ma.ne)
(24:10) Conversational Dunbar number
(25:10) AI Industry and platforms
(29:04) Disrupting Ebay
(34:00) 10x Engineers/Designers
(40:59) Outro
People
Naval
- Airchat
Brian
- AirChat
Kumar
- AirChat
Keith
- AirChat
Tom
- AirChat
Moritz
- AirChat
Kyle Barber
Patryush Sing
Raghav Gulati
Petri Kajander
Some AirChat Links
Conversational Dunbar number
product design simplicity (Naval)
product design simplicityu (Brian)
natural design
The Knowledge Illusion: Why We Never Think Alone: https://www.amazon.com/Knowledge-Illusion-Never-Think-Alone/dp/0399184368
Stevens Bio: https://vivo.brown.edu/display/ssloman
Steven on Google Scholar: https://scholar.google.com/citations?user=YL5RTcUAAAAJ
Follow Moritz on Twitter: https://twitter.com/MoritzW42
I had fun chatting with my friend Logan Chipkin about various topics and misconceptions related to the growth of knowledge and wealth including:
- Austrian Economics and Critical Rationalism
- Black Box View of Government
- Economic Creationism
- Time Imperialists
- Why Equality is undesirable
- Moralizing
- Anarcho Capitalism
- Austrian Economics
Subscribe to Scaling Knowledge: https://scalingknowledge.substack.com
Follow Logan on Twitter: https://twitter.com/ChipkinLogan
Follow Moritz on Twitter: https://twitter.com/MoritzW42
Follow Kenneth on Twitter: https://twitter.com/kenneth0stanley and check out his website!
Subscribe to Scaling Knowledge: https://scalingknowledge.substack.com
Follow James on Twitter: https://twitter.com/experilearning
Follow Moritz on Twitter: https://twitter.com/MoritzW42
0:00 - Intro
1:02 - Why are objectives flawed? What does it mean for the education system?
14:48 - What feedback have you gotten from students/parents/teachers/administrators?
21:37 - How can students implement non-objective driven search book while in school?
31:21 - What is the role of curiosity in non-objective driven search?
39:55 - Is our nose for the interesting innate? Is it fixed ability, or can it improve/get worse over time? Does the education system worsen it?
49:25 - How does novelty search accumulate complexity?
59:36 - Can following your interests just cause you to spin your wheels, not making any significant progress? What is the risk-reward tradeoff in novelty search
1:11:27 - Startups and non-objective search.
1:19:40 - Homeschooling - balancing non-objective driven learning with objective-driven learning.
1:28:11 - Knowledge accumulation in society and open-endedness.
1:35:11 - Creating networks/communities to facilitate open-ended search in education and beyond.
1:48:51 - How to adapt to the mindset of non-objective search and taking risks?
1:55:15 - Would privatization of the education system help? How can we measure whether an open-ended system is making progress?
2:04:37 - Minimum criteria and open-ended systems. Observing rather than measuring open-ended systems.
2:09:16 - Closing comments
Subscribe to Scaling Knowledge: https://scalingknowledge.substack.com
Follow Moritz on Twitter: https://twitter.com/MoritzW42
Follow James on Twitter: https://twitter.com/experilearning