The Lindahl Letter

View Details

Thank you for tuning in to week 220 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Welcome to 2026 and beyond.”

This last week has been about being a reflective practitioner and thinking about where we have been throughout the last year of the Lindahl Letter. This last year we covered research notes numbered from week 175 to 219. Back in June I did acknowledge a 56 day posting break in 2025 which is interesting to look back on now as an opportunity to reflect and build something substantial going forward. Toward the end of the year we got back into the groove of quality weekly missives which is good and something to continue. My focus on quantum, robotics, and AI seems to hold true to my roots of being generally interested in technology.

Overall, my general interest in technology is what drives my interest in lifelong continuous learning. With that context being set it is probably easy enough to set the expectation that in 2026 and beyond the Lindahl Letter will be targeted toward the production of weekly research notes that are accessible, targeted, and focused. These missives will require less than 10 minutes of a reader’s time and should be a clear value add in terms of gaining knowledge, understanding, and context for complex technical content.

Let’s establish the theoretical home base of this writing enterprise for 2026 which will be set on the foundation of digging into the edge of realized technology. That topic might sound familiar from week 212 of the Lindahl Letter. During that writing project we took a look at what technology is likely to be realized in the next 30 years. That coverage included looking at the metaverse, robotics, climate tech, space economy, biotech, synthetic biology, neurotech, and even fusion. I do believe that we will see quantum, robotics, and some AI mixed into that soup of potentially realized technology.

All of that technology will see advancement and it will certainly be moving toward the edge of becoming realized technology. That is fundamental where it goes from being exploratory and research driven to being in production out in the wild where it will eventually become commoditized unless a clear winner breaks away and can hold onto a real advantage. I’m pretty skeptical about any of these technologies having a clear moat that allows that advantage. For the most part once a group of people know how to do these things the technology will be realized and break out into wider use.

My primary weekly writing focus will be the Lindahl Letter and this is the place you will be able to find out what topics grab my attention and I consider to be worth sharing. My focus in the last 90 days has been heavily on quantum computing which is understandable due to how close it is getting to be a realized technology. We are on the edge of people figuring out how to demonstrate quantum supremacy for use cases and building these things into data centers as a clear value add for corporate customers and research labs that can afford to be a part of the journey. Outside of that, most of the major quantum computers that will be part of the early wave demonstrating the technology will be tied to either a research lab or corporate R&D group.

Those early systems are starting to really scale up focus on specific advances in the quantum space. My research project in that space helped me to focus on open-access nanofabs, national laboratories, commercial foundry services, and captive industrial fab sites. Each of those groups has different advantages and research interests. We will see where the ultimate breakthroughs end up coming from as the story unfolds toward realized quantum technology.

That is where we are heading throughout 2026. Thank you for being here for the journey and I look forward to learning more about technology and digging into the frontier of what will be realized this year. Overall the state of the Lindahl Letter is strong and we should be able to continue moving forward on our weekly journey of exploration into technology.

What’s next for the Lindahl Letter? New editions arrive every Friday. If you are still listening at this point and enjoyed this content, then please take a moment and share it with a friend. If you are new to the Lindahl Letter, then please consider subscribing. Make sure to stay curious, stay informed, and enjoy the week ahead!

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 219 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “2025 End of Year Recap.”

Thank you for being here! The Lindahl Letter this week started out as a Merry Christmas and Happy Holidays post and ended up just being an end of year recap. As the year comes to a close, I am taking a brief pause from publishing this week to spend time with family, recharge, and reflect on the remarkable conversations and ideas we have explored together throughout the year. If you are reading this one, then you certainly learned about AI/ML/AGI, robotics, and quantum computing this year. The Lindahl Letter will return to its regular schedule next year, and I am grateful for your continued readership, curiosity, and engagement. I wish you and yours a happy holiday season and a thoughtful, restorative start to the new year.

My top 5 posts of 2025 included:

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 218 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Nested learning and the illusion of depth.”

Just for fun with this nested learning paper we are evaluating today, I downloaded the 52 page PDF and uploaded it to my Google Drive to have Gemini create an audio overview of the paper. That is just a one button request these days. We have reached a point where we can easily listen to a paper recap with very little friction. It’s actually harder to get a complete reading of the PDF as an audio file. I had tried the Adobe Acrobat read aloud feature and I don’t really like the robotic output. Sometimes, I would rather listen to a paper than read it when I am trying to really think deeply about something. The 5 minutes of podcast audio Gemini spit out about the paper are embedded below. It’s interesting to say the least how quickly Gemini turned that paper into a short podcast. It’s entirely possible that my analysis might be less entertaining than the podcast Gemini created on the fly. You will be the judge of that one.

This is a paper I actually printed out 2 pages per page using the double sided setting. That is how I used to read papers during graduate school. This paper had a few color elements that is something my graduate school papers never really had. They were all monochromatic. I had to put on my reading glasses and hold the paper a little closer than I used to with the 2 pages per page printing. I’ll have to remember to just print using single page spacing next time around. I really only print out papers I want to keep in my stack of stuff. This one certainly fits that criteria.

Trying to make content that is accessible is one of the reasons that I have been recording audio for the Lindahl Letter. Sometimes listening to something is a great unlock. Other times due to complexity and the diagrams included you just have to read academic papers. I try to bring things forward without complex charts in a highly consumable way. My take on research notes is that they need to be generally understandable and communicate a clear take on whatever topic is being covered. The content has to be condensed into something that can be considered in 5-10 minutes. To that end I’m going to do my best to bring this paper on nested learning to life today.

This paper matters, it really does, because the research presented undermines one of the core assumptions driving modern AI investment and the endless LLM building and training that has been occurring, namely that stacking more layers reliably produces qualitatively better intelligence [1]. The mantra to just keep scaling maybe will fade away. If many so-called deep models collapse into shallow equivalents during training, then reported gains attributed to architectural depth may instead be artifacts of data scale, regularization, or optimization heuristics rather than true representational progress.

This has direct implications for benchmarking, since comparisons that reward parameter count or depth risk overstating advances that do not translate into more robust reasoning or generalization. It also affects hardware and infrastructure strategy, because enormous resources are being allocated to support depth that may not deliver proportional returns. At a deeper level, the result forces a reconsideration of what meaningful learning progress actually looks like, shifting attention from surface complexity toward mechanisms that introduce genuinely new inductive structure and adaptive behavior.

Maybe the long term impact of this call out is likely to be gradual rather than abrupt, but it meaningfully shifts the intellectual ground beneath current AI narratives [1]. The paper in question provides a formal vocabulary for a concern many researchers have held intuitively, that architectural depth has become a proxy metric for progress rather than a principled design choice. Over time, this reframing may influence how serious research groups evaluate models, placing more weight on identifiably distinct learning mechanisms, training dynamics, and robustness properties instead of raw scale.

It is unlikely to immediately change the minds of investors or vendors whose incentives favor larger systems, but it can shape academic norms, reviewer expectations, and eventually benchmark construction. Historically, results like this matter most not because they halt a paradigm, but because they constrain it, narrowing the space of credible claims and forcing future advances to justify themselves on grounds other than appearance and size.

This argument intersects directly with my broader concerns about interpretability and generalization. I am still curious about creating a combiner model, but this might change the mechanics of how that might ultimately work. If performance gains arise primarily from optimization dynamics rather than architectural expressivity, then claims about learned representations should be treated with caution. Apparent abstraction may not correspond to stable semantic structure but to transient equilibria shaped by training order, learning rates, and implicit regularization. This aligns with growing skepticism about whether large models truly learn hierarchical concepts or merely approximate them through iterative adjustment [2].

The implications extend beyond theory. Nested learning reframes debates about model scaling, architectural novelty, and transfer learning. It suggests that progress may come less from ever deeper networks and more from better understanding and controlling learning dynamics. This has practical consequences for reproducibility, safety, and deployment, since nested optimization can introduce path dependence and sensitivity to training regimes that are difficult to observe or audit.

In the broader context of the AI marketplace, this work reinforces a recurring theme. Fluency and performance do not necessarily imply understanding. As with recent neuroscience critiques of language models, nested learning highlights how impressive outputs can emerge from mechanisms that lack stable, interpretable internal structure [3]. That gap matters when systems are deployed in high stakes environments where reliability, robustness, and reasoning are essential.

We will see how this plays out in 2026 and what new research will ultimately shift the landscape.

Footnotes:

[1] Behrouz, A., Razaviyayn, M., Zhong, P., & Mirrokni, V. “Nested learning: The illusion of deep learning architectures.” Advances in Neural Information Processing Systems 39 (2025). https://abehrouz.github.io/files/NL.pdf

[2] Raghu, M., Poole, B., Kleinberg, J., Ganguli, S., & Dickstein, J. “On the expressive power of deep neural networks.” Proceedings of the 34th International Conference on Machine Learning (2017). https://arxiv.org/abs/1606.05336

[3] Riley, B. “Large language mistake: Cutting edge research shows language is not the same as intelligence.” The Verge (2025). https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 217 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “The great 2025 LLM vibe shift.”

Vibe shifts came and went. People are certainly adding the word vibe to all sorts of things as the initial meaning has ironically faded. Casey Newton in the industry standard setting Platformer newsletter wrote about a big silicon valley vibe shift in 2022 [1]. It was a big thing; until it wasn’t. The really big completely surreal LLM shift has happened toward the tail end of 2025. We went from extreme AI bubble talk to very clear, rational, and thoughtful perspectives on how LLMs won’t realize the promises that have been made. Keep in mind the market fears of an AI bubble are different from the understanding that LLMs might be the technology that ultimately wins. All of the spending in the marketplace and the academic argument may get reconciled at some point, but we have not seen that happen in 2025.

The backward linkages of how potential technological progress regressed may not have been felt just yet, but the overall sentiment has shifted. The ship has indeed sailed. Let that sink in for a moment and think about just how big a shift in sentiment that really happens to be and how it just sort of happened. As OpenAI and Anthropic move toward inevitable IPO, that shift will certainly change things. Maybe the single best written explanation of this is from Benjamin Riley who wrote a piece for The Verge called, “Large language mistake: Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it” [2]. I owe a hat tip to Nilay Patel for recommending and helping surface that piece of writing.

I was skeptical at first, but then realized it was a really interesting and well reasoned read. I’ll admit at the same time, I was also reading a 52 paper from the Google Research team, “Nested Learning: The Illusion of Deep Learning Architecture” around the same time which was interesting as a paired reading assignment [3]. More to come on that paper and what it means in a later post. I’m still digesting the deeper implications of that paper.

Maybe to really sell the shift you could take a moment and listen to some of the recent words from OpenAI cofounder Ilya Sutskever. I’m still a little shocked about the casual way Ilaya described how we moved from research and the great AI winter, to the age of scaling, and finally back to the age of research again. The idea that scaling based on compute or size of corpse won’t win the LLM race is a very big shift and Ilya makes it pretty casually during this video.

You will notice I have set the video to play about 1882 seconds into the conversation:

Maybe a video with a really sharp looking classic linux Red Hat fedora in the background featuring a conversation between Nilay Patel and IBM CEO Arvind Krishna can help explain things. Don’t panic when you realize that the CEO of IBM very clearly argues with some back of the envelope math that all the data center investment has no real way to pay off in practical terms or an actual return on investment. Try not to flinch when it is described that within 3-5 years the same data centers could be built at a fraction of the current cost. Technology does just keep getting better. The argument makes sense. It is no less shocking based on the billions being spent.

I set the video to start playing 502 seconds into the conversation.

The argument that I probably prefer in the long run is how quantum computing is going to change the entire scaling and compute landscape [4]. The long-term argument that may end up mattering the most suggests that quantum computing will transform the economics of scale and ultimately reset expectations about what is computationally feasible. Former Intel CEO Pat Gelsinger recently framed quantum as the force likely to deflate the AI bubble by altering the fundamental relationship between compute and capability, a claim that is gaining analytical support across the research community. We may see it be an effective counter to the billions being spent on data centers for a late mover willing to make a prominent investment in the space or it could just end up being Alphabet who is highly invested in both TPU and quantum chips [5].

Footnotes:

[1] Newton, C. (2022). The vibe shift in Silicon Valley. Platformer. https://www.platformer.news/the-vibe-shift-in-silicon-valley/

[2] Riley, B. (2025). Large language mistake: Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it. The Verge. https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems

[3] Behrouz, A., Razaviyayn, M., Zhong, P., & Mirrokni, V. (2025). Nested learning: The illusion of deep learning architectures. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. https://abehrouz.github.io/files/NL.pdf

[4] Shrivastava, H. (2025). Quantum computing will pop the AI bubble, claims ex-Intel CEO Pat Gelsinger. Wccftech. https://wccftech.com/quantum-computing-will-pop-the-ai-bubble-claims-ex-intel-ceo-pat-gelsinger/

[5] Yahoo Finance, “Alphabet CEO just said quantum computing could be close to a breakthrough,” https://finance.yahoo.com/news/alphabet-ceo-just-said-quantum-155229893.html

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 216 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “The biggest unsolved problems in quantum computing.”

The field of quantum computing has accelerated rapidly during the last decade, yet its most important breakthroughs remain incomplete. The core research challenges that stand between today’s prototypes and large scale, industrially relevant systems are now visible with unusual clarity. I think we are on the path to seeing this technology realized. These challenges are increasingly framed not as incremental milestones but as structural bottlenecks that shape the entire trajectory of the field. This week’s analysis focuses on the five most critical problems that must be solved for quantum computing to reach fault tolerant, economically meaningful operation. These gaps define where research investment, national strategy, and competitive advantage will be determined in the coming decade.

A fully fault tolerant logical qubit with logical error rates below threshold

The first and most fundamental problem is the absence of a fully fault tolerant logical qubit. I know, I know, people are getting close, but this technology is not fully realized just yet. Theoretical thresholds for fault tolerance are well studied, and progress has been reported through surface codes, low density parity check codes, and recent advances in magic state distillation. Several groups have demonstrated logical qubits whose performance exceeds their underlying physical qubits, and some trapped-ion experiments now show better than break-even behavior under repeated rounds of error correction. However, no team has yet realized a logical qubit that maintains below-threshold logical error rates in a fully integrated setting that combines encoding, stabilizer measurement, real time decoding, and continuous correction across arbitrarily deep circuits. Experiments such as the University of Osaka’s zero level magic state distillation results and Quantinuum’s recent logical circuit demonstrations illustrate meaningful progress, yet a complete fault tolerant logical qubit build rolling off the assembly line has not been achieved [1]. This missing element prevents reliable execution of deep circuits and stands as the central research challenge of the field. I am also tracking a leaderboard of efforts aimed at increasing the number and stability of logical qubits as new systems emerge [2].

A scalable and manufacturable quantum architecture that supports thousands of high fidelity qubits

The second unsolved problem is the absence of a scalable, manufacturable quantum architecture capable of supporting thousands of high fidelity qubits. Superconducting platforms continue to face wiring congestion, cross talk, and fabrication variability across large wafers, which limits reproducibility at scale. Trapped-ion systems achieve some of the highest gate fidelities reported, but their physical footprint, control volume, and relatively slow gate speeds constrain system growth. Neutral atom arrays offer large qubit counts, yet they have not demonstrated uniform, high fidelity two qubit gates across arrays large enough to support fault tolerant codes. Photonic and spin qubits continue to advance but remain earlier in their development for universal, gate based architectures. Across all platforms, the transition from laboratory systems to repeatable, wafer scale manufacturing has not occurred. Most resource estimates indicate that tens of thousands of physical qubits will be required for practically useful, error corrected applications, and no architecture is yet positioned to deliver this scale with consistent fidelity. I am tracking universal gate based physical qubit leaders closely, and I expect to see significant shifts in 2026 as fabrication strategies evolve [3].

Integrated cryogenic classical control systems capable of real time decoding at scale

The third unsolved problem concerns the integration of classical control systems capable of operating efficiently at cryogenic temperatures. Quantum processors rely on classical electronics to generate precise control pulses, read measurement outcomes, and perform real time decoding. As devices grow, these classical requirements become a dominant engineering bottleneck. Current systems depend on extensive room temperature hardware and thousands of coaxial lines, an approach that is not viable for scaling beyond a few hundred qubits. Research into cryogenic CMOS, multiplexed readout architectures, and fast low noise routing has shown meaningful progress, and prototype decoders have demonstrated sub microsecond performance. However, the field still lacks a fully integrated classical to quantum control stack that can operate near the device, support large scale decoding throughput, and eliminate the wiring overhead required for million channel systems. Solving this challenge is as essential as improving qubit fidelity, because fault tolerant computation will require tightly coupled classical and quantum subsystems functioning in real time at cryogenic depths.

A modular, networked quantum architecture with reliable chip to chip entanglement

The fourth major unsolved problem involves modularity and quantum networking. Large scale quantum computers will not be monolithic systems. They will require distributed architectures in which multiple chips or modules exchange entanglement to support error corrected computation across larger systems. Research groups have demonstrated chip to chip photonic links, heralded entanglement generation, and short range coupling between trapped-ion and superconducting devices, but these demonstrations remain small scale and experimental. No team has yet produced a modular architecture capable of sustaining reliable inter module entanglement rates, routing operations, and error corrected logical circuits across networked components. A practical quantum interconnect, whether photonic or microwave based, would redefine system design by enabling large logical qubit counts without relying on a single monolithic wafer. Developing these networked architectures is now seen as one of the highest value targets for national research programs, because modularity is likely the only viable path to systems with millions of physical qubits.

A verified quantum advantage tied to a real scientific or industrial workload

The fifth unsolved problem is the absence of a widely accepted, independently verified quantum advantage tied to a real scientific or industrial workload. Quantum supremacy experiments have demonstrated that certain random circuit sampling tasks are exceptionally difficult for classical systems to simulate, but these tasks do not translate into chemistry, materials, optimization, or cryptography workloads. Several vendors have recently reported domain specific quantum advantages, including applications in quantum navigation and narrow optimization tasks, but these demonstrations have not yet achieved broad community validation or independent replication under strict verification and resource accounting. A robust demonstration of advantage requires a computation that is infeasible for classical systems within realistic time and energy constraints, produces an output that can be meaningfully verified, and operates using real hardware error rates rather than idealized gates. Achieving this milestone would mark a decisive shift in the strategic landscape of the field and would accelerate commercial investment into fault tolerant platforms.

Together, these five problems outline the most important questions I’m tracking that are facing quantum computing today. This is based on my research interests. Please feel free to let me know if something else jumps out when you read this list. Each topic represents an opportunity for technical leadership, research investment, and industrial strategy. That does not mean my list is complete. It’s directionally accurate for late 2025, but things in the quantum computing space are changing rapidly. These elements called out also define the hurdles that stand between early laboratory demonstrations and the large-scale quantum platforms required for transformative scientific progress.

Links I’m sharing this week!

You may not have watched Linus Torvalds build a computer on your watch list for 2025, but I’m sharing that link anyway. I truly enjoyed watching this video.

This video made me chuckle several times and was delightful.

Footnotes:

[1] Itogawa, T., Takada, Y., Hirano, Y., & Fujii, K. (2024). Even more efficient magic state distillation by zero-level distillation. arXiv preprint arXiv:2403.03991. http://arxiv.org/pdf/2403.03991

[2] Top quantum computers by logical qubit

[3] Updating my top 10 quantum computer leaderboard

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 215 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Process capture and the future of knowledge management.”

The history of knowledge management has been shaped by repeated attempts to store, retrieve, and reuse organizational insight. So much institutional knowledge gets lost and discarded as organizations change and people shift roles or exit. People within organizations learn through the every day practice of getting things done. It’s only recently that systems are augmenting and sometimes automating those processes. Early systems focused on document repositories, and later platforms emphasized collaboration, tagging, and collective intelligence. We now find ourselves in a period where knowledge management converges with automated workflows and computational assistants that can observe, extract, and generalize decision patterns. We are seeing a major change in the ability to observe and capture processes. Systems are able to capture and catalog what is happening. This creates an interesting inflection point where the system may store the knowledge, but the users of that knowledge are dependent on the system. That does not mean the process is understood in terms of the big why question. Scholars have noted that the operational layer of organizational memory is often lost because it resides in informal practices rather than formal documentation. The shift toward embedded and automated capture offers a remedy to that problem.

The rise of agentic AI and workflow-integrated assistants alters the knowledge landscape by making it possible to synthesize procedural knowledge in real time. Instead of relying on teams to manually update wikis or define operating procedures, modern systems can extract key steps from repeated actions, identify dependencies, and flag anomalies that deviate from observed patterns. This transforms knowledge management from a static library into a dynamic computational environment. What exactly happens to this store of knowledge over time is something to consider going forward. Supervising the repository will require deep knowledge of the systems which are now being maintained systematically. Maintaining and refining it will be the difference between sustained institutional knowledge or temporary model advantages that drop with the next update. Recent studies on digital trace data argue that high fidelity observational streams can significantly improve the accuracy of organizational models. When this data flows into agents capable of modeling tasks, predicting outcomes, and recommending actions, the role of knowledge management shifts from storage to orchestration.

Process capture also introduces new opportunities for long-horizon learning systems. This is the part I’m really interested in understanding. The orchestration layer has to have some background learning and storage that runs periodically. When workflows are automatically translated into structured representations, organizations can run simulations, perform optimization, and enable higher levels of task autonomy. These capabilities begin to resemble continuous improvement environments that merge human judgment with machine-refined operational insight. Researchers have observed that structured process models can improve downstream automation and decision support, particularly in complex enterprise settings where procedures evolve rapidly. This suggests that the next phase of knowledge management will involve systems that not only store information but also refine it through computational analysis and real world feedback. It’s in that refinement that the magic might happen in terms of real knowledge management.

Links I’m sharing this week!

https://www.computerworld.com/article/4094557/the-world-is-split-between-ai-sloppers-and-stoppers.html

This video is a super interesting look at a number we don’t normally question on a daily basis. The delivery style is a bit bombastic, but the fact check on the argument is interesting. You know I enjoy numbers and was really curious how this was calculated.

That video referenced this widely shared analysis from Michael W. Green on Substack.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 214 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “The great manufacturing reset.”

Boston Dynamics captured public imagination when they introduced Spot the dog-like robot back in 2016. Things have changed. Robots that walk around are beginning to enter the commercial landscape, and new entrants continue to appear. A humanoid robot product from Russia built by the company Idol surfaced last week [1]. Other companies such as Agility Robotics (USA), Figure AI (USA), Boston Dynamics (USA), UBTECH (China), and 1X Technologies (Norway/USA) are all working toward delivering humanoid robots. Optimus, the Tesla bot introduced conceptually in 2021 and now in its third-generation prototype which remains part of an internal program and has not yet reached commercial deployment is also being talked about.

The stage is now set, and we are at a point where robotics, autonomous fabrication systems, and advanced materials are converging into a new industrial baseline. The last decade brought low-cost filament printers into hobbyist and commercial spaces at massive scale, and the next decade is poised to move far beyond that early wave. Industrial additive manufacturing has already expanded into metals, composites, and high-performance polymers, with global revenue expected to accelerate over the coming years. At the same time, the field is absorbing rapid advancements in AI-enabled calibration, defect detection, and real-time optimization, allowing machinery to tune production parameters autonomously. That capability shifts what it means to operate a modern fabrication workflow. Things are changing rapidly.

Alongside these developments, humanoid and semi-autonomous industrial robots are transitioning from research demonstrations to contract manufacturing deployments. Several builders are scaling up pilot programs in which general-purpose robots support assembly, materials handling, and repetitive manufacturing tasks. These systems benefit from advances in reinforcement learning, enhanced sensors, and cloud-based model updates. Industrial robotics shipments are increasing rapidly, driven by global demand for flexible production lines and labor-augmentation strategies. The supply side of robotics is not only expanding but also becoming modular and more interoperable across fabrication environments.

The most significant shift may come from the emergence of machines that build machines. That is a topic I’m focused on understanding. Historically, tooling design required long lead times, significant manual labor, and specialized expertise. Today, automated CAM pipelines, printable tooling, adaptive CNC systems, and robotically tended fabrication cells allow factories to generate and regenerate their own production processes. Some aerospace and automotive facilities already deploy these closed-loop systems to create fixtures, jigs, and replacement components internally. This form of self-manufacturing reduces dependency on external suppliers and removes friction from engineering iteration cycles. We are moving toward a world where design, testing, and tooling are all integrated within an AI-guided, robotics-driven feedback loop. That integration is the foundation of the great manufacturing reset.

For the United States, these technologies open a realistic path to reshoring custom and small-batch manufacturing in ways that were not economically viable during the offshoring wave of the late twentieth century. Rising labor costs in traditional manufacturing hubs, geopolitical risk, and supply chain disruptions have already encouraged firms to reconsider where they build things. Additive manufacturing and flexible robotics change the cost structure by reducing reliance on large minimum-order quantities, expensive hard tooling, and long logistics chains. A factory that can print tooling on demand, deploy modular robots, and run AI-optimized production scheduling can serve shorter runs and more specialized designs while remaining geographically close to end customers. In effect, the United States can replace scale-driven arbitrage with speed, customization, and resilience. That is why we are at the inflection point for the great manufacturing reset.

Policy and infrastructure are beginning to support this transition. Federal programs such as Manufacturing USA and its associated network of advanced manufacturing institutes are working to diffuse next-generation production technologies across domestic firms and regions [2]. Investments in semiconductor fabrication, battery plants, and clean-energy hardware have already catalyzed billions of dollars in new onshore manufacturing commitments. The same capabilities that support large facilities can extend to mid-market and smaller manufacturers through shared tooling libraries, regional robotics integrators, and standardized digital design pipelines. Universities and community colleges can align curricula with this reset by emphasizing mechatronics, robotics programming, and design-for-additive principles that translate directly to a modern factory floor.

If the United States leans into this transition, the great manufacturing reset will not simply re-create legacy industrial capacity. It will establish a distributed network of automated, digitally coordinated micro-factories specializing in custom work, rapid prototyping, and short-run production. The strategic advantage will be the ability to move from concept to physical part in days instead of months, while retaining critical capabilities within domestic borders. The risk is that other regions may scale faster and capture the integrator role that coordinates robots, additive systems, and AI platforms across global supply chains. The next few years will determine whether the United States treats these technologies as incremental enhancements or as foundational infrastructure for a new manufacturing baseline. Ideally, this reset will create conditions for a new wave of startups delivering smaller manufacturing runs, bespoke development cycles, and entirely new product categories.

Things to consider:

The economics of reshoring depend as much on automation and design speed as on wage differentials.
Policy support for advanced manufacturing will matter most where it connects directly to tooling, robotics, and workforce upskilling.
Custom, short-run production could become a core competitive advantage for regions that adopt additive and robotics early.
The integrators that connect robots, printers, and AI software may end up more powerful than any single hardware vendor.
Manufacturing resilience will increasingly be measured by how quickly domestic systems can reconfigure to new designs and shocks.

Links I’m sharing this week!

Footnotes:

[1] Mesa, J. (2025, November 11). Russia ‘human’ robot falls on stage during debut. Newsweek. https://www.newsweek.com/russia-human-robot-falls-stage-during-debut-11031104

[2] Manufacturing USA. (n.d.). Home. https://www.manufacturingusa.com/

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 213 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Why a “combiner model” might someday work.”

Open models abound. Every week, new open-weight large language models appear on Hugging Face, adding to a massive archive of fine-tuned variants and experimental checkpoints. Together, they form a kind of digital wasteland of stranded intelligence. These models aren’t all obsolete; they’re simply sidelined because the community lacks effective open source tools to combine their specialized insights efficiently. The concept of a “combiner model” offers one powerful path to reclaim this lost potential. Millions of hours of training, billions of dollars in compute, and so much electricity have been spent. Sure you can work by distillation to capture outputs from one model into another, but a combiner model would be different as it overlays instead of extracts.

A combiner model represents a critical shift away from the assumption that AI progress requires ever-larger single systems. Instead of training another trillion-parameter monolith, we can learn to combine many smaller, specialized models into a coherent whole. The central challenge lies in making these models truly interoperable. The challenges form from questions around how to merge or align their parameters, embeddings, or reasoning traces without degrading performance. The combiner model would act as a meta-learner, adapting, weighting, and reconciling information across independently trained systems, unlocking the latent knowledge already encoded in thousands of open weights. Somebody at some point is going to make an agent that works on this problem and grows stronger by essentially eating other modals.

This vision can be realized through at least three technical routes. The first involves weight-space merging. Techniques such as Model Soups and Mergekit show that when models share a common base, their weights can be effectively averaged or blended. More advanced methods, like TIES-Merging, learn adaptive coefficients that vary across layers, turning model blending into a trainable optimization process rather than a static recipe. In this view, the combiner model becomes a universal optimizer for reuse, synthesizing the gradients of many past experiments into a single, functioning network.

The second approach focuses on latent-space alignment. When models differ in architecture or tokenizer, their internal representations diverge. Even so, a smaller alignment bridge can learn to translate between their embedding spaces, creating a shared semantic layer, or semantic superposition. This allows, for example, a legal-domain model and a biomedical model to exchange information while their original knowledge weights remain frozen. The combiner learns the translation rules, effectively building a common interlingua for neural representations that connects thousands of isolated domain experts.

The third approach treats the combiner not as a merger but as a controller or orchestrator. In this design, the combiner dynamically decides which expert model to invoke, evaluates their outputs, and fuses the results through its own learned inference layer. This idea already appears in robust multi-agent frameworks. A true combiner model or maybe combiner agent would internalize this orchestration as a core part of its reasoning process. Instead of running one model at a time, it would simultaneously select and synthesize outputs from many experts, producing complex, context-aware intelligence assembled on demand. This approach is the most immediately viable and is already being used in sophisticated production systems today.

If such systems mature, the economics of AI will fundamentally change. Rather than concentrating resources on a few massive, proprietary models, research will shift toward modular ecosystems built from reusable parts. Each fine-tuned checkpoint on Hugging Face will become a potential building block, not an obsolete artifact. The combiner would turn the open-weight landscape into an evolving lattice of knowledge, where specialization and reuse replace the endless cycle of frontier retraining. This vision is demanding, but the promise remains compelling: a world where intelligence is assembled, not hoarded; where the fragments of past experiments contribute directly to future understanding. The combiner model might not exist yet, but its underlying logic already dictates the future of open source AI.

Links I’m sharing this week!

This is the episode with Sam Altman that everybody was talking about.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 212 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “The edge of realized technology.”

Welcome to the start of season 5. Don’t panic, we are still covering advancing technology including quantum, robotics, and artificial intelligence within the Lindahl Letter. I’ll be writing about the intersection of technology and modernity until the singularity. For better or worse, modernity’s shadow will continue to be the edge of realized technology. We are on the path to seeing a bunch of different technologies end up being realized in the not so distant future. That is why I’m so focused on the path toward realizing robotics, quantum, and agentic. That is where season 5 of the Lindahl Letter is going to pick up and start to dig into those topics at the edge of realized technology. To that end, I started to make a graphic of the timeline of major financial bubbles and extended it out to emerging technologies expected to deliver before 2045 [1]. You can modify the Python visualization code for this one if you want, I shared an executable version of it on GitHub.

Within that visualization I started to sketch out the next 10 most likely technologies we will see realized. Within each path toward realization is where private investment and ultimately retail investors will crowd into the market before it gets commoditized to the point where the initial leaders in the space have no first mover advantage and some type of bubble ensues. That does not mean these things won’t be game changing. I’m just expecting some type of financial crowding followed by pressure against expected profits that won’t be realized. Resulting from that would be some type of financial bubble which might very well be led by a huge windfall of some sort. People made money on tulips and pepper before those markets crashed out.

2026, “AI Bubble”, “Tech”
2028, “Metaverse and XR Bubble”, “Tech/Speculative”
2029, “Robotics Bubble”, “Tech”
2031, “Climate Tech Bubble”, “Climate Tech”
2032, “Space Economy Bubble”, “Space Economy”
2033, “Biotech and Longevity Bubble”, “Biotech/Longevity”
2034, “Synthetic Biology and Food Tech Bubble”, “Synthetic Bio/Food Tech”
2035, “Quantum Bubble”, “Tech”
2035, “Neurotech and BCI Bubble”, “Neurotech/BCI”
2040, “Fusion Energy Bubble”, “Energy”

These edges of technology realization might not be in the right order or tied exactly to the right year, but I do think that directionally this list will prove to be an accurate prediction of when technology will be achieved and we will see meaningful changes to modernity. Futurist considerations abound for what might end up happening. This was my swing at predicting what’s next. Only time will tell if it was an accurate swing or it will be disrupted by some other emerging technology.

Going forward you are going to see my weekly writing efforts get split into 4 distinct buckets. My general weekly think pieces will stay here within the relative safety of the standard Lindahl Letter publication, writing about civics, civility, and civil society will be over on the Civic Honors domain, blogging will be done within the Functional Journal, and my hope is to resume daily posting back over on the nels.ai domain. Ideally, enough content will be generated in the major domains that only a small amount of blogging will occur. Going forward it is far better to produce meaningful work than to complete passages of extended navel gazing. Sure being a reflective practitioner and blogging has its place, but sometimes all that writing about the process ends up being more circular than forward looking.

Links I’m sharing this week!

Footnotes:

[1] https://github.com/nelslindahlx/Data-Analysis/blob/master/TimelineofMajorFinancialBubbles.ipynb

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Happy Halloween everybody! Thank you for tuning in to week 211 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “When Satoshi-Era Wallets Wake Up.”

Seriously, Bitcoin is weird. It has an enigmatic and anonymous founder. The origin story of how this cryptocurrency came to be is pretty much ineffable. Roughly a third of all bitcoin has never moved [1].These dormant or maybe abandoned coins shape both the scarcity and the psychology of the network. Now, some of those early wallets are coming alive again, and their reawakening reveals a deeper story about profit, security, and the bleeding edge of quantum cryptography. Maybe some of these cutting edge massive quantum computers are being used to run Shor’s algorithm and factor some of these older wallet keys [2]. That seems more likely to me than somebody remembering they had some old bitcoin after a decade and moving it around. We could write a really spooky short story about people waking up to old bitcoin wallets getting cracked by quantum computers running Shor’s algorithm. That is the type of short story that could move from fiction to non-fiction with one scientific breakthrough. It’s even possible it has already started to happen. By possible, I think it probably already is happening.

Speculation aside, it’s true that an estimated thirty percent of all mined bitcoin has been untouched for more than five years [3]. That is shocking. About seventeen percent of bitcoins have not moved in a decade [4]. Those figures mean that even as mining nears completion, a huge fraction of the network’s supply remains functionally absent or potentially abandoned. This long-term dormancy amplifies Bitcoin’s scarcity, turning lost or forgotten coins into a silent deflationary force. Yet in 2025, something shifted. Several ancient wallets, first active during Bitcoin’s infancy, have begun to stir after twelve to fourteen years of silence. Their movements are rare, deliberate, and full of meaning.

Some of these wallets trace back to 2010 and 2011, a time when bitcoin traded for less than a dollar. In July, eight early addresses moved roughly eighty thousand bitcoin in a coordinated set of transfers [5]. That is wealth that once totaled a few thousand dollars but is now worth billions. Somebody made some shocking profits. Later, a miner-era wallet from 2010 moved four hundred bitcoin after twelve years of dormancy [6]. In October, an early 2011 wallet that had accumulated four thousand bitcoin sent a small test transaction of 150 coins before going quiet again [7]. None of these events caused market disruption, but each drew immediate attention. Every time an ancient wallet moves, it feels like a fragment of Bitcoin’s early history is stepping into the present.

Why are these early coins moving now? The first reason is straightforward economics. With bitcoin surpassing one hundred thousand dollars, even small transfers yield generational wealth. Another reason is technological maturity. Over the past decade, wallet recovery methods have improved, and holders who once misplaced keys or old software backups can now retrieve them. Security has also evolved. Many early wallets were built with primitive address types that expose their public keys, leaving them theoretically vulnerable to a future cryptographic breakthrough. This leads to the third and most forward-looking motivation: the quantum threat. That is the part I’m super curious about. Some of the larger quantum systems that I shared in my leaderboard could be active here, but we don’t really know.

Quantum computing is still developing, but progress is steady. Bitcoin relies on elliptic-curve digital signatures that would be mathematically vulnerable to sufficiently powerful quantum machines. The earliest wallets used formats that make this risk more immediate, because they reveal public keys on-chain once a transaction occurs. If quantum computing advances far enough, those exposed keys could allow attackers to derive private keys and spend the coins. Experts estimate that a quarter of all existing bitcoin resides in such legacy formats. That reality has not escaped early holders. Some of the recent awakenings may reflect quiet migrations of classic wallet cold coins being moved to SegWit, multi-signature, or even post-quantum-resistant wallets to protect them from future compromise. These reactivations might not be about profit at all. They could be acts of defensive foresight from people who understand how close technology may be to challenging the foundations of digital security.

There are also practical motivations. Estate planning, custodial audits, and consolidation are all normal parts of managing large digital holdings. After more than a decade, early miners are updating their records, creating inheritance plans, and transferring assets to institutional custodians. The act of moving coins from an old address is sometimes less a financial maneuver and more a generational effort to ensure those digital fortunes survive their original owners.

Each time these wallets awaken, the community reacts with fascination and unease. The first question is always the same: could this be Satoshi Nakamoto? So far, none of the reactivated wallets match known Satoshi mining patterns, but the mythology persists. Beyond the curiosity, there’s the market anxiety that large moves might signal selling pressure. Yet most transfers have not flowed into exchanges. They seem measured, intentional, and quiet. In a sense, this is the opposite of panic: the calm movement of old wealth into modern systems that have better security.

What we are witnessing is also a potential generational handoff. The early experimenters who mined coins on laptops are now confronting questions of succession and security that mirror those of traditional wealth. Their coins, once symbols of rebellion against institutions, are being integrated into structured estates, custodial frameworks, and long-term trusts. As these coins move, they pass through new layers of infrastructure and oversight, becoming part of a global financial fabric that looks very different from the anarchic beginnings of Bitcoin.

As quantum computing advances and Bitcoin’s price continues to rise, more early wallets are likely to move. Some of those transfers will be tests or migrations; others may represent the quiet liquidation of immense fortunes. Watching these awakenings provides a rare link between the network’s origin story and its future. The early holders are not gone, some of them may have abandoned some bitcoins, or lost access, but some of them are simply preparing for the next phase of digital permanence, one where Bitcoin must prove its resilience against both time and the very real likelihood of advancements in quantum based technology.

Summary of the key points on the quantum threat to bitcoin:

Quantum computing is advancing, and although no system has yet cracked the relevant cryptography used by Bitcoin, experts estimate that within the next 5-10 years some wallets using legacy address types could become vulnerable.
A significant fraction of Bitcoin’s supply (in some estimates around 25 %) is held in addresses whose public key has been exposed (or in older formats such as pay-to-public-key) and thus are considered more at risk from a “Q-day” style attack.
The network is already responding: developers have floated proposals (e.g., a draft BIP‑360) to freeze coins in vulnerable legacy addresses and force migration to quantum-resistant formats, with multi-phase transition plans.
The fact that early “Satoshi-era” wallets are waking up now may reflect not just profit or estate planning motives but also pre-emptive security behaviour by holders who recognise the quantum risk and wish to migrate coins to safer custody.
From a scarcity and supply-dynamics perspective, the quantum threat adds another layer of complexity: dormant coins may not just be inert, they may be targeted or moved due to security fears, altering how one thinks about long-term supply, holder behaviour and concentration risk.

Links I’m sharing this week!

Footnotes:

[1] Van Straten, J. (2023, November 15). Record 70% of Bitcoin supply lies dormant for a year or more. CryptoSlate. https://cryptoslate.com/insights/record-70-of-bitcoin-supply-lies-dormant-for-a-year-or-more/

[2] Chojecki, P. (2025, May 20). Quantum computers threat to Bitcoin: Q-Day, post-quantum cryptography and Bitcoin. Medium. https://pchojecki.medium.com/quantum-computers-threat-to-bitcoin-e1b57b0da2aa

[3] AInvest. (2025, July 5). 30.4% of Bitcoin supply dormant for over five years. https://www.ainvest.com/news/30-4-bitcoin-supply-dormant-years-2507/

[4] Crypto News. (2025, June 18). Over 3.4 million BTC, more than 17% of the total supply, have not moved in at least a decade. https://crypto.news/bitcoin-dormant-supply-growth-outpaces-issuance-2025/

[5] Malwa, S. (2025, July 5). Eight Bitcoin wallets move 80,000 BTC in largest ever ‘Satoshi-era’ transfers. CoinDesk. https://www.coindesk.com/markets/2025/07/05/eight-bitcoin-wallets-move-80000-btc-in-largest-ever-satoshi-era-transfers

[6] Kumari, I. (2025, September 29). Bitcoin address from miner era reactivates to shift 400 BTC – Report. AMBCrypto. https://ambcrypto.com/bitcoin-address-from-miner-era-reactivates-to-shift-400-btc-report/

[7] Van Straten, J. (2025, October 24). Dormant Bitcoin Whale With $442 M Awakens for First Time in 14 Years Amid Quantum Fears. CoinDesk. https://www.coindesk.com/markets/2025/10/24/dormant-bitcoin-whale-with-usd442m-awakens-for-first-time-in-14-years-amid-quantum-fears

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 210 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “AI Is Burning Through Graphics Cards.”

Generational wealth is being invested into data centers for AI. It’s so prevalent that you hear about it on the nightly news and municipalities are dealing with the power demands. The clock is ticking on graphics cards being used for AI inference. The current generation of GPUs was never designed to run around the clock under inference loads. These chips were originally built for bursts of rendering, not continuous model execution at scale. What we are seeing now is an industry trying to stretch gaming hardware into a role it was never meant to fill. The result is heat, power consumption, and a ticking clock based on the inevitable wear.

Each graphics card has a limited operational lifespan. These are not like bricks being used to build a house; they are just expensive computer hardware. The more intensive the workloads, the shorter that lifespan becomes. Fans fail, thermal paste dries out, and the silicon itself begins to degrade. Inference tasks, particularly when stacked across large fleets of GPUs, magnify this effect. The relentless pace of AI workloads accelerates the failure curve, turning once-premium cards into temporary consumables. I’m actually really curious what is going to happen to all of them at the end of this cycle. A secondary market does exist for these used devices and companies like Iron Mountain will help data centers with secure disposal.

By most reasonable estimates, there are now between 3.5 and 4.5 million NVIDIA data-center GPUs actively deployed in production environments. Hyperscalers such as Meta, Microsoft, and Google each operate hundreds of thousands of units, while smaller data centers fill out the rest of the global total. Each GPU represents a remarkable amount of compute density, but also a constant thermal and economic liability. Even with optimized cooling, sustained inference loads drive high thermal stress and power draw that shorten component life. These systems were never meant to run 24 hours a day, 365 days a year.

Under heavy duty cycles, many GPUs experience significant degradation within one to three years of continuous operation. The warranties often match that window, which reflects a design expectation rather than coincidence. Silicon aging and persistent thermal cycling all take their toll. Even when the hardware technically survives longer, it becomes economically obsolete as new architectures quickly double efficiency and throughput. The pace of improvement ensures that by 2027 or 2028, most of today’s fleet will either be retired, resold, or relegated to low-priority inference tasks. Right now TSMC would have to make the chips to replenish this fleet of GPUs which would be outrageously expensive. Both NVIDIA and TSMC manufacturing teams could be looking at a huge impending need for production or a shift to a new type of technology.

That replacement cycle has massive implications. The cost of refreshing millions of GPUs every few years is enormous, and the environmental impact of manufacturing and disposing of that much silicon is even harder to ignore. As AI inference continues to scale, this churn becomes unsustainable. Companies are already exploring purpose-built accelerators, ASICs, and FPGAs that can deliver better efficiency and longer service life. These designs aim to handle continuous inference without the same thermal or aging limitations that plague graphics cards.

Sustainability will define the next phase of AI infrastructure. The transition away from general-purpose GPUs is underway, but what comes after silicon remains uncertain. Research into photonic computing, quantum processors, and neuromorphic architectures offers glimpses of what a post-GPU world might look like. Each of these alternatives seeks to break free from the limits of traditional chips while extending useful lifespans. The next leap in AI hardware will not be measured by sheer speed, but by how well it can endure the relentless demands of inference at scale.

Links I’m sharing this week!

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 209 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Social media stopped being social.”

Before we get going this week. I need to provide an update about last week’s post. I take full responsibility, as the principal writer here, that last week my writing efforts were just not up to par within the 208th Lindahl Letter publication. You have come to expect better from me and last week I just delivered a dud of a post. It’s the first post in a long time that actively drove people to leave the Lindahl Letter. It’s pretty easy to see the signal within the noise when something was bad enough to drive people away and I take responsibility for delivering that subpar effort.

That being noted, let’s pivot back to the main topic at hand related to social media.

I’m not sure if social media was ever really about togetherness and being social. Those are things after the fact that I want to ascribe to it. Let’s blame it on nostalgia. Communities tend to align with place, interest, or circumstance. Certainly online communities that are highly focused and targeted on a distinct community probably work. Later in a different essay it might be worth digging into the pocks of working online communities. That side of the coin however is not the focus of this missive.

Things were different back when Twitter arrived in 2006 and ultimately became popular during South by Southwest in 2007. During the initial development and discovery of these applications for social media sharing things were different and maybe that newness is now something to be nostalgic about. Social media now is fragmented and stopped being social the moment algorithms learned how to predict the things that would hold our attention better than we could possibly direct it.

What started the social media ball rolling as a digital gathering of friends slowly transformed into a system of engineered consumption. The feed no longer reflects our relationships. It reflects what the platform believes will keep us scrolling. In the process, the human layer was optimized out of existence. I am hoping the Substack experience ends up being different. Right now Substack is really my only active social media platform. It’s full of actual readers and writers for the most part. I’m trying to get into the swing of using Substack Notes, but that just seems to be an ongoing process of trying to figure it out. Previously, I tried to get into posting on Bluesky and I’ll admit that during Colorado Avalanche games it did feel like some level of community existed. Outside of gametime I just never really got much out of the Bluesky experience.

Let’s take a step back from where we are now to consider history for a moment. Things were different for the first wave adopters. The first generation of social networks were built around connection. You followed people you knew, saw what they were doing, and commented because you cared. The platforms of today are not built for connection, but instead of being factored around community they are built for amplification. The more content flows, the more data moves, and the more ads get served. The mechanics of community were replaced by the logic of engagement.

That shift changed the culture. Ultimately, it spawned the influencer movement. Maybe it’s a moment or it could be a watershed change away from public intellectuals to something else more product centric. People began curating identities instead of sharing moments. Every post became a performance. Every response was an opportunity for algorithmic reinforcement. What once felt like a conversation now feels like an audition. Social validation metrics turned communication into competition. The ultimate winners being the people who ended up making a career within this new flow of attention online.

As that dynamic took hold, the real social behavior moved into the shadows. Private group chats, invite-only communities, and niche networks quietly took over the role that public timelines once held. The visible web is now dominated by content farms and brand influencers. The meaningful conversations happen elsewhere, often out of reach of recommendation systems. What used to feel like a town square has become a noisy digital strip mall.

Social networks have become media networks. In some ways they are just the next generation of broadcast television or radio. It’s just more targeted and in some ways a lot more divisive. They are not spaces for dialogue but for distribution. Every interaction is mediated through a system that values attention over authenticity. That is why the average user now feels less connected than ever, even as they scroll through an endless feed of “content.” The core function of social media has inverted. It no longer connects people directly instead connecting people to platforms.

We may be entering a post-social era online. Connection is returning to smaller spaces: group chats, email lists, federated platforms, and direct exchanges. The large-scale, public-facing feed is collapsing under the weight of its own incentives. Maybe that’s the natural end of a system built on attention rather than empathy. What comes next may not look like social media at all. It might look more like correspondence. The strange part is that most users know this. We can feel the shift. We see fewer updates from friends, fewer real conversations, and more noise disguised as engagement. The feedback loop is obvious, but breaking free from it is hard. Every design choice keeps us tethered to the cycle. The system runs on our participation, but not our connection.

Links I’m sharing this week!

White, M. (2025, October 17). Anatomy of a crypto meltdown. Citation Needed. Retrieved from https://www.citationneeded.news/anatomy-of-a-crypto-meltdown/

The Vergecast. (2024, October 17). AI can’t even turn on the lights | The Vergecast [Video].

SearchParty. (2024, April 12). The big flaw in Trump’s AI plan [Video]. YouTube.

Nathan Labenz & Erik Torenberg. (2024, March 8). Is AI slowing down? Nathan Labenz on GPT-5, progress and predictions [Video]. YouTube.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Day after release update: I guess it was the 208th post where we hit the proverbial wall with a dud of a post. This post in retrospect turned out to be one of my weaker efforts. I thought it was a strong take about dealing with the rate of change in model development, but it was just not focused and targeted based on delivering quality and insights.

Thank you for tuning in to week 208 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Building with constant model churn.”

Developers have spent a lot of time in the past patching software. That happens based on vulnerabilities, edge cases, and performance issues. All this vibe-coded content and things built on models are not getting any patches to make them better going forward. You may get a new release or a new model, but that patch to save you from vulnerabilities is not being developed and is not on the way. It is the nature of modern development. The ecosystem of dependencies is real. However, the pace of model development has created an unusual environment for anyone trying to build durable systems.

You cannot really hot swap models within production systems. That just does not work. In the last five years, we have seen large language model releases from OpenAI, Anthropic, Google, Meta, Mistral, Cohere, and several open-source groups. Each iteration has been faster, larger, and sometimes more efficient than the one before. What has not been stable is the interface between models and the systems people build around them. Even seemingly small changes in context window size, output quality, or API availability ripple outward and cause redesigns, migrations, and sudden pivots. Sometimes these changes happen with no warning whatsoever.

For builders, this creates a paradox. The potential upside of adopting a newer model is undeniable: better reasoning, lower costs, and expanded capabilities. At the same time, the risk of betting on an API or framework that may be deprecated in months is a constant concern. Some developers chase every release, weaving the newest model into their applications as quickly as possible. Others step back, building abstractions and wrappers that allow for switching models without disrupting core workflows. Neither path offers complete insulation from this wave of almost continuous churn.

The history of technology offers parallels. Software engineers have long had to deal with shifting operating systems, frameworks, and libraries. What makes this moment different is the velocity of change and the sheer dependency of emerging applications on model behavior. The model is not just another dependency, it is the foundation of the system. When that foundation shifts, everything built on top of it must be reconsidered.

There is also a deeper strategic question. Should builders lean into constant change and accept churn as a feature of the landscape? Or should they try to design in ways that minimize dependency, focusing more on proprietary data pipelines, unique integrations, and distinctive user experiences? Both strategies reflect an awareness that stability is not guaranteed in this ecosystem. The companies that endure will be the ones that treat churn not as an annoyance but as a design constraint.

Things to consider:

The lack of patching for AI models makes long-term maintenance difficult.
Model churn introduces structural instability into modern systems.
Abstraction layers help, but they cannot prevent cascading change.
Treating churn as a core design constraint is a pragmatic approach.
Builders must balance innovation speed with long-term stability.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 207 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Enforcing AI standards without exception.”

Standards are something we need to spend more time talking about. That is a general statement and not a special argument. Years ago, I actually witnessed a physical desk sign at an office that said, “We either have standards or we don’t.” It’s not a great mystery how that particular leader felt about standards. That type of adherence to standards is not all that common. In our LLM sponsored chat by prompt first and ask questions later world; people just keep prompting. Allowing models to just keep generating without standards is how we ended up where we are right now. Those tokens are being burnt at prodigious rates. All of those burnt tokens yield nothing reusable or even effectively carried forward. Mostly they are highly siloed outputs to an audience of one. They are all spent and the electricity and compute used will never be recovered. They are just an expense on somebody else’s balance sheet.

Everything about the open web is pretty much in rapid decline. I would argue that enforcing standards without exception is the only way the end user can truly control the agenda or hope to manage the ultimate outcome when working with AI. It might even help us save the internet. That cause however might have already been lost. One of the great ironies of generative AI is that it demands more discipline from the human interacting with it to get quality outputs, not less. Sure prompt engineering has become a hands on the keyboard type of sport, but my best guess is everything ends up being more conversational in the end. You would expect a machine to be the enforcer of rules, to deliver outputs with mechanical precision. Instead, the responsibility ultimately falls back on the end user to enforce standards at every turn. The system will generate endlessly, but unless you control the agenda, it will wander away from the very standards that define your work. A lot of people are also just creating AI slop and potentially worse AI generated workslop.

This is not a trivial annoyance. It is the defining challenge of using AI effectively. You might tell a system: no em dashes, strict numeric citations, Substack-compatible footnotes. And for a moment, it will comply. Then, in the next draft, it slips back into its defaults. Suddenly the citations are misplaced, the formatting is broken, or the output is square when you clearly require 14:10. It doesn’t matter how many times you’ve said it for some reason the system’s memory for discipline is shallow. If you do not enforce the standard without exception, the drift takes over. For an organization, that can mean tens or thousands of drifting lines of argument and fragmented results.

That is why the end user must step into a role that looks less like automation’s promise and more like quality assurance. You are not simply a writer or a collaborator. You are the auditor, the rule enforcer, the one who stops the drift. We either have standards or we don’t. Allow one exception, and you have taught the system that exceptions are acceptable. Enforce the standard every time, and you create a boundary strong enough to shape consistent results.

This relentless enforcement becomes the core of collaboration. Without it, the system defaults to “plausible” instead of “correct,” “close enough” instead of “aligned.” You cannot rely on the machine to protect the integrity of your work or really even to have solid consistent outputs. That responsibility is yours. The human must guard the agenda with vigilance and insistence. Outside of ruthlessly enforcing standards without exception the path forward is just full of slop.

Over time, this process builds more than consistency. It builds identity. A body of work that holds together across hundreds of posts or thousands of outputs does so because the user enforced the standards that give it coherence. We may very well look at the internet archives before all the LLM training as untainted and everything after that point with skepticism. I’m not arguing that everything in that first tranche of content was high quality or even accurate, but it was before the models. Without that enforcement, the work would fracture into a mix of styles, structures, and shortcuts. Enforcing standards without exception is exhausting, but it is also the only way to produce work that reflects your agenda rather than the system’s defaults.

Things to consider:

AI will always drift back toward its defaults unless the user enforces rules consistently.
The promise of automation is inverted: the human enforces discipline, not the machine.
Exceptions teach the system the wrong lesson and erode consistency.
Vigilant enforcement is what turns scattered outputs into a coherent body of work.
Control of the agenda belongs to the end user, or it is lost altogether.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 206 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration is “The Great Tokenapocalypse.”

As large language models reach deeper into consumer devices, the cost of running them becomes the real bottleneck. So many tokens get burned with no ROI or use case for the company burning them; it's really out of control. Almost as out of control as the sunk cost of data centers that will probably be regretted at some point in the next 5 years. It’s sort of the unspoken reality of an arms race where building data centers that just depreciate and spending compute resources without any plan for recovering the cost is happening. This week explores how token economics is silently shaping the deployment strategies of Google and Apple.

You may have noticed something strange about the rollout of generative AI: despite Google’s global reach and technical infrastructure, Gemini is not yet present on every device. It isn’t quietly running in the background on your Nest Hub, it doesn’t summarize content on your Pixel Watch, and it hasn’t taken over the always-on interactions that dominate the smart home experience. On paper, Gemini could power all of this: but in practice, it doesn’t. The reasons are not technical, but economic.

It’s the tokens. Each time a large language model like Gemini processes a prompt or generates a response, it consumes tokens which are effectively a unit of computation that translate directly into cost. This cost is not abstract. It is real-time, metered, and at scale becomes wildly continuous with enough uses. When you ask Gemini to summarize an email or rewrite a paragraph, you’re triggering a live cloud inference cycle that draws directly on Google’s TPU infrastructure. At a small scale, these requests are manageable. But when deployed across millions of devices, in billions of micro-interactions, the financial and infrastructure burden becomes extreme. What looks like product restraint is actually cost containment. Google is avoiding what could become a tokenapocalypse which would be a runaway escalation of inference demand that outpaces both compute supply and operating budget.

Gemini was designed for centralized, high-performance environments. It was not optimized for low-power edge devices or offline operation. Its rollout has been concentrated in strategic, high-leverage use cases: Workspace productivity, Pixel exclusives, and experimental features inside Search Labs. These are high-value zones where the cost per token can be justified. Gemini has not been deployed ambiently in the wild on smart speakers, in Android Auto, or on lightweight wearables mostly because those endpoints offer little to no margin against token cost. The model cannot run constantly without triggering exponential cloud expenditure. Until inference becomes drastically cheaper or edge-native Gemini variants emerge, Google is likely to continue rationing its deployment to protect against economic overextension.

Apple, by contrast, has chosen an entirely different path forward. They elected a path that avoids the token problem from the outset. Its 2024 rollout of “Apple Intelligence” emphasized a local-first architecture built around on-device models. Instead of sending every prompt to the cloud, Apple routes the vast majority of inference through its A-series and M-series silicon. This strategy means that users can rewrite notes, summarize messages, or interact with Siri entirely offline, with zero token cost to Apple. When tasks exceed the capability of local models, they are sent to Apple’s “Private Cloud Compute” system, but this fallback is used selectively, with strict privacy and latency guarantees.

Apple’s approach isn’t just a branding play. It reflects a fundamental architectural decision to avoid the economics of inference altogether. Apple doesn’t operate a hyperscale public cloud business, so it has no incentive to absorb or monetize cloud-based generative AI usage. Its profits come from hardware margins and platform services. This gives Apple the freedom to constrain usage, limit interaction complexity, and push AI to the edge. A strategy they can get away with, ultimately without incurring the compounding costs that Google faces. It’s a token-avoidant strategy, and it may prove to be the more sustainable one.

Where Google builds outward from a full-stack cloud foundation, Apple builds inward from a controlled edge. Google’s strategy scales across models and modalities, but each expansion amplifies cost. Apple’s strategy constrains functionality but keeps economics stable. Both are reacting to the same underlying pressure: token costs are rising faster than monetization models can support. The more embedded the model becomes, the more tokens flow. A stark reality comes into existence where it becomes more urgent to rethink deployment patterns. This isn’t just a question of technical feasibility. It’s a matter of financial survivability.

The race to deploy generative AI at scale is quickly becoming a race to control token exposure. Inference cost and not model quality may be the key determinant of which platforms can sustainably integrate AI across the stack. If cloud economics don’t shift, and if token optimization doesn’t advance, then ambient LLMs may remain a luxury reserved for premium endpoints and enterprise tasks. The real future of ubiquitous AI may depend less on how powerful models become, and more on how efficiently they run in the wild.

Things to consider:

Google’s restraint in deploying Gemini across its device ecosystem likely reflects real-time token cost constraints rather than technical limits.
Every cloud-based Gemini interaction consumes metered compute, making global deployment economically unstable without stronger monetization.
Apple avoids these problems by designing for on-device inference and constraining AI functionality to remain token-light.
Token economics are now shaping the strategic posture of every major platform, defining where and how AI appears in consumer workflows.
Sustained deployment of generative models may depend less on breakthrough architecture and more on advances in inference efficiency and local compute.

As the tokenapocalypse looms, we’ll be watching how companies respond. That response could be through model compression, edge acceleration, hybrid routing, and new monetization strategies. In the coming weeks, we’ll explore how these constraints are shaping research priorities, ecosystem fragmentation, and what it means to run AI sustainably across global networks. If you see an AI endpoint that should exist but doesn’t, it may be because someone, somewhere, did the token math.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 205 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Apple’s Hidden AI Strategy: Waiting it out with token avoidance as a first principle.”

Apple’s relative restraint in deploying large-scale generative AI isn’t just about privacy posturing or design philosophy. Maybe it is just the latest supply chain management initiative in terms of managing tokens. It may reflect a deliberate avoidance of token-expensive cloud inference which is an infrastructural and financial commitment that Apple has historically chosen not to make. This choice is akin to keeping supply chain costs down; this type of effort fits with the general operating model. Right now engaging token usage would just eat profits.

Apple’s approach to “Apple Intelligence,” announced in 2024, hinges on three pillars:

On-device first: Apple designed its models (small language models and transformer variants) to run locally on A17+ and M-series chips. This dramatically reduces reliance on cloud GPUs and token accounting. If you generate 200 tokens on your phone, there’s no inference cost to Apple. This method avoids cloud costs, but makes the hardware the tipping point.
Private Cloud Compute: For tasks that exceed the capabilities of on-device models, Apple routes requests to its proprietary cloud using Secure Enclaves. But this only happens for high-value or infrequent tasks. That would include things like summarizing a document, generating email replies, or rewriting notes. This keeps cloud token loads minimal and predictable.
Selective rollout: Apple isn’t putting generative models everywhere. The system isn’t always listening, and “AI” is offered as an opt-in assistant across Mail, Notes, Safari, and Siri. There’s no ChatGPT clone embedded system-wide, and certainly nothing ambient like Gemini could theoretically become.

You can see based on the bottom line and balance sheet concerns why Apple’s caution probably makes financial sense in the long run. Apple sells hardware, not compute. Even some of the cloud forward vendors might operate at a loss at Apple scale. Unlike Google or Microsoft, it doesn’t have an economic engine tied to cloud usage. If it gave every iPhone user unlimited generative AI access via the cloud, it would have to subsidize trillions of tokens per year without monetization return. Nothing in the workflow has any ROI for Apple where the hardware is a sunk cost and they have not offered a standalone monthly AI service. They let everybody else spend billions on hardware, data centers, and electricity.

Instead, Apple wants:

Efficiency over scale.
Local inference over cloud latency.
Sporadic usage over daily token floods.

In short: Apple is playing defense against the tokenapocalypse before it ever hits. That token apocalypse will happen when billions of devices become token hungry.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 204 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Context window garbage collection.”

Here we are this week contemplating how to clean up the mess from all these celebrated LLM chat sessions. It’s a disjointed mess that lacks federation or dare we say portability. We are at the point where we need to think about how context window garbage collection explores the deeper process based idea of how large language models might manage overflowing histories, selectively pruning, discarding, or compressing tokens to maintain efficiency without losing coherence. Certainly my thoughts on this is that we would all benefit from portable knowledge sharding, but that is just one way to look at the potential set of solutions that need to be built. Another way to slice the apple up and put just the best parts back together again would be to build out some context window garbage collection.

When you open a long conversation with a large language model, the context window eventually fills with tokens from your prompts and the model’s replies. These windows have strict limits, whether 128k tokens, 200k tokens, or more, and yet our usage tends to grow indefinitely. As prompts expand, sessions become inefficient and costly, sometimes degrading in coherence as irrelevant or outdated details pile up. We have all run into hallucinations and just weird output from models. At this point in the experience the next best question then becomes very clear. We have to evaluate how models should manage their overflowing context windows at the end or during a chat session.

In programming, garbage collection has long been the answer to similar problems. Long garbage collection problems have literally kept me up at night. Computer systems with finite memory must constantly decide what to keep and what to discard. Techniques such as reference counting, mark-and-sweep, and generational garbage collection have been developed to handle this challenge. The analogy we can build out here is very straightforward: in a world where context is the working memory of LLMs, garbage collection could provide the rules and processes for pruning, summarizing, or discarding tokens without breaking continuity.

Several strategies already hint at how this could work. Some systems automatically prune less relevant history, while others compress sections of text into summaries or embeddings that can be retrieved later. I would run a knowledge reduce function based on my previously shared research, but I always think that is the answer. User-directed pinning, where important content is marked as permanent, is another possible feature. In longer interactions, models could run background “cleanup passes,” automatically condensing earlier exchanges into portable knowledge shards. Each of these approaches mirrors classic computing strategies while being adapted to the new problem space of language models.

The risks are obvious. Things could go sideways. We face a direct computing time cost associated with this effort. Poorly designed garbage collection could lead to subtle context loss, missing small but crucial details. Summarization may introduce semantic drift or hallucinations. I would argue that properly structured context will actually reduce drift or hallucinations. Users may also resist invisible pruning, questioning whether they can trust a model that silently discards information. The challenge lies in balancing efficiency, fidelity, and transparency, ensuring that garbage collection makes interactions smoother rather than introducing new points of failure.

Looking forward, context window garbage collection could become a fundamental layer of model architecture. Standardized processes and even some APIs might emerge to expose garbage collection logs to users or allow customization of pruning strategies. Entire ecosystems of agents could share compressed or pruned context shards across models, creating interoperability where today there is only fragmentation. Just as garbage collection enabled more scalable and reliable programming environments, context window garbage collection may become the invisible backbone of scalable AI interaction.

Things to consider:

Should context garbage collection be visible and user-controllable?
Can models balance efficiency with fidelity when pruning?
What lessons from programming garbage collection apply directly to LLMs?
Does context GC make interoperability between models easier or harder?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to week 203 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Portable knowledge sharding.”

The shards of knowledge that we need are everywhere. They are just not portable and all packaged up. You open an interaction with an LLM based on a prompt, but you don’t really close it out by receiving a prompt or a packaged transferable output. This week’s focus is on how knowledge can be broken into modular, transferable units and moved across systems, sessions, or users. At its core, this concept involves fragmentation by design, creating smaller, self-contained pieces that retain meaning independently while becoming more useful when recombined. These modular shards offer a practical method for bridging gaps between disconnected tools, memory systems, and AI agents. You could just ask the model at the end of your session to package up the results for you. That type of effort makes you your own data broker. You are then responsible for putting the right data in all the right places.

A true system of portable knowledge sharding that is easily transferable addresses the growing problem of fragmentation in digital workflows. Isolated AI memory systems, disconnected application ecosystems, and session-based interactions that fail to persist information have made continuity more difficult to maintain. In this context, a knowledge shard can be understood as a compact, self-contained packet of insight that includes metadata and minimal context. This idea draws from concepts such as database sharding, microservices architecture, Zettelkasten-style note-taking, and linked data formats like JSON-LD. The defining characteristic of a portable shard is that it contains just enough information to be interpreted outside its original environment.

Fragmentation is increasing across nearly every dimension. Large language models operate in isolation. Memory is not shared between models, or even across sessions within the same system. Users frequently move between unconnected platforms and tools. The result is a scattered intellectual landscape. Portable knowledge sharding provides a way to restore structure, making it easier to preserve, transport, and reassemble valuable insights.

Several key principles support the creation of effective knowledge shards. These include atomicity, where each shard captures a single coherent idea; context tagging, where metadata includes origin, date, and relationships; minimal dependency, ensuring each shard is understandable on its own; mergeability, allowing recombination into larger ideas; and transportability, which enables movement across systems without loss of meaning. Together, these principles provide a foundation for more resilient and flexible knowledge systems.

Real-world applications of portable knowledge sharding are already emerging. Tools like Manus and Rewind.ai offer memory replay capabilities that hint at this modular future. As workflows become more complex, it will be necessary to repackage experiences and decisions into transferable learning units. Research systems like the nels.ai KnowledgeReduce project are grounded in this very concept. Portable shards could also improve task handoffs between AI agents, support modular scientific publishing, or serve as components within platform-spanning knowledge graphs. A side-by-side comparison of traditional notes and knowledge shards would help illustrate these differences more clearly.

This approach is not without challenges. Shards can lose critical context or become misleading when separated from their origin. Interoperability suffers without standardized formats. Version control becomes more difficult. Excessive sharding may also reduce clarity instead of enhancing it. Even with these limitations, portable knowledge sharding remains a promising strategy for managing complexity in highly fragmented environments.

Consider whether your current workflows support modular knowledge reuse. Think about how agents might benefit from receiving portable shards as part of their input. Reflect on whether we are moving toward an ecosystem of shard-native tools and practices.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning into the podcast. This is week 202 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Personalized context bubbles.”

Last week we took a deeper look into content window fragmentation. That is just the right amount of foundation to start to consider personalized content bubbles. Don’t panic; no foundation model pun intended. I even talked about it on YouTube during a recent Nelscast episode. It has been years since I actively livestreamed on YouTube. Live streaming is apparently a lot like riding a bicycle and you pick it back up pretty quickly.

The context bubbles we are focusing on today are model-specific and ultimately memory-driven. Functionally they are hidden from the outside world and only interacted with by the user. The whole experience is highly gated and mostly hidden by design. Unlike the algorithmic filter bubbles of the social media era, highly personalized context bubbles in LLMs emerge from user-specific interaction histories and model memory. They are shaped by prompts, preferences, and usage over time. Some people have ridiculously deep context in memory and have radically changed how the model even interacts with them on an ongoing basis. Some people even call this a type of modal rot where things worked better initially and then over time degraded.

This type of both hidden and blatantly obvious fragmentation is increasing across AI ecosystems. There is no interoperability between different models’ memory systems. A user’s context in GPT-4o does not translate to Claude, Gemini, or Mistral, leading to siloed experiences that fragment continuity and collaboration. It means the only point of continuity is individual to the user and fundamentally disjointed to any external view.

Ultimately what we are talking about is that private AI interactions are creating isolated knowledge spaces. As more users rely on fine-tuned personal agents and persistent memory features, the result is a proliferation of parallel digital realities, each uniquely shaped by the individual’s bubble of past interactions. It takes everything that was creating conflict within our broader social fabric and exacerbates it both in terms of isolation and from an observability consideration it remains completely invisible.

The risk of invisible epistemic bias is growing. Personalized bubbles can limit intellectual perspective and reinforce confirmation bias, particularly when LLMs refine outputs based solely on a user’s prior behavior and inputs. You can basically create a walled garden of very optimistic reinforcement that just celebrates whatever perspective, the bubble has fostered for the end user.

I would argue that this is happening because no standards for portability of data exist. Truly at this point in the ecosystem there is a lack of standards for context portability. Without a framework to export or synchronize context across tools or agents, users remain locked into proprietary silos, impeding collaborative and transparent knowledge generation.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for being a part of the adventure. This is week 201 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Context window fragmentation.”

Exactly. You are either aware of or have felt the effects of context window fragmentation. I thought this topic would be a really sharp angle for consideration. Something that would stand out and be interesting as a point of conjecture. Instead of just digging into the technical fragmentation inside one model’s context window, I’m more interested in evaluating ecosystem-level fragmentation. Fragmentation confounds and conflates concepts between environments, users, and ultimately sessions.

Users move between multiple models (say, Claude, Gemini, GPT, Mistral, etc.), each with its own maximum context length and its own memory handling. Nothing of that experience is shared between them and no effective way of packaging and sharing context windows even exists. Effectively that means that no shared continuity across those systems exists and what one model “remembers” has no carryover to another. A lot of the time it has no carryover between sessions either based on how the memory is managed. We effectively run tabula rasa into whatever training that particular model had received. It’s a new dance every time, but the dance partners are unclear.

This creates fragmented work products: research threads, writing drafts, or code bases become scattered across different LLM silos.

Even within one provider, different versions of the same model can handle context differently, further increasing the fragmentation. Sometimes you even end up with the model changing in the middle of things from say the more comfortable GPT-4o to the underwhelming GPT-5 [1].

Let’s highlight two layers of “context window fragmentation”:

Within-model limits: losing track of earlier tokens inside one long prompt.
Across-model silos: knowledge, drafts, and reasoning don’t travel with the user between systems and sessions.

This double fragmentation means users are left stitching things back together manually, or trying to impose their own save-points, knowledge graphs, or workflows to keep continuity.

Things to consider:

Expanding context windows does not solve fragmentation across providers.
Users are building their own ad hoc systems to bridge silos, often with limited success.
True continuity may require hybrid architectures that integrate structured memory with LLMs.
The absence of interoperability standards ensures this fragmentation will persist for now.

Footnotes:

[1] https://www.theverge.com/openai/759755/gpt-5-failed-the-hype-test-sam-altman-openai

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in this week. Not only will this not be an audio only presentation this week and you will have to check YouTube at some point in the not so distant future for more details on that, but also I have tweaked the noise gate in GarageBand to improve the overall audio quality. This is week 200 of the Lindahl Letter publication. A new edition arrives every Friday. This week, the topic under consideration for the Lindahl Letter is, “My 200th Lindahl Letter Explained.”

My very first Substack post was published on January 26, 2021, at 5:44 p.m. We can safely say that was about 236 weeks and 3 days ago at the point I started this draft. You probably have figured out that 36 weeks in that window did not receive any Lindahl Letter. That inaction does reflect a series of gaps in my publishing schedule. It happened. It happens. It might very well happen again at some point. Sometimes that is the way things go during an ongoing writing project. Today, however, you are getting the 200th cumulative week of my writing efforts for the Lindahl Letter publication. We have arrived at this major milestone in my writing efforts, and we should take a moment and celebrate. It is pretty exciting. I had compiled the previous years into manuscripts you can find online. At this point, it would probably make sense to assemble the year 4 edition of that writing project.

For this 200th milestone post, it really does make sense to spotlight five topics that capture both the breadth of coverage and the depth of research that have defined the Lindahl Letter. Those weekly research notes don’t self-generate.

The first topic would be from our recent shift to understanding quantum computing. It was a big shift from covering AI/ML to talking about quantum computing. That series included posts such as Magic state distillation explained and Quantum computing near Denver, which showcased detailed explorations of cutting-edge hardware breakthroughs and the regional tech ecosystem. I really do think that quantum computing is getting near a key breakthrough point where it will be more accessible. I’m not talking about it being widespread or used for everyday compute, but we are getting very close to the edge of possibility where quantum workloads are going to be a part of daily processes.

The second topic relates to machines that build machines, a series examining advanced robotics, automation, and manufacturing systems that has been a defining theme of the current season. As we move forward, understanding prototyping and building the means of manufacturing is important to the next phase of building. We are going to see a movement from 3D printing to small-scale prototyping. Generally, I do not provide the same talk over and over again, but I think my elevator pitch on this topic needs some work.

Third, we spent a lot of time discussing AI governance and ethics. That effort has been among the most engaged-with posts, offering insights into regulation, societal impact, and the responsible development of large language models. This topic just seems to be a known thing that we need, but it just does not end up being at the forefront of considerations.

The fourth topic would have to be my all-time favorite topic to consider. That would be the intersection of technology and modernity, a recurring theme that connects technology trends to broader cultural and societal contexts. At some point, I plan on finishing my magnum opus on that topic. It should be a good read.

Finally, the fifth topic would be worth rewinding back to my earliest foundational topics that were all just adapted talks, including my very first post in January 2021, which ultimately would provide you with a clear sense of how far the publication has evolved over 200 weeks. That key foundational topic was all about ROI and how to ensure you are aligning your priorities with actual dollars from the budget.

As we move forward, the Lindahl Letter will continue along this new trajectory of research based on focusing on 3 topics: quantum computing, robotics, and enabled agents. I’m not sure if I will end up writing another 200 Lindahl Letters, but it is an interesting moment to consider having reached 200 of them and still be considering what’s next. I have considered moving to a monthly cadence where a paper is produced vs. a weekly research note. We will see where that ends up going during the next few weeks.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to this audio only podcast presentation. This is week 198 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Machines that build machines.”

As we embarked on this new season of the Lindahl Letter, I signaled that we would focus on three content themes. These themes included quantum computing, machines building machines, and agents taking action. We are now shifting from quantum related topics toward innovations in advanced robotics, which I group under a series of research notes titled “machines that build machines.” The future of manufacturing depends on more than automation. It hinges on the ability to rapidly prototype, iterate, and deploy the machines that make everything else possible.

Modern manufacturing at scale nearly always involves some form of robotics. These robotic systems range from industrial arms performing repetitive tasks to highly customized modular assemblies tailored to the needs of specific products. A few weeks ago, during our trip back to Kansas City, I listened to the audiobook Apple in China by Patrick McGee [1]. That narrative presented a vivid exploration of Apple’s entanglement with China’s manufacturing infrastructure. Beneath McGee’s primary story is a powerful subtext about the foundation of China’s manufacturing prowess. The critical enabler in that story is the set of machines that build other machines. These tools not only support prototyping and product assembly but also underpin supply chain resilience and adaptability. A nation’s or company’s ability to design, iterate, and build these enabling machines directly influences how quickly it can scale production, respond to demand, and recover from disruption.

To refine this research note further, I want to focus on the prerequisites for developing these prototyping machines. What infrastructure, talent, and technological components are essential to make all of this work going forward. Unlocking these prerequisites will be key. It likely starts with a feedback loop between design software, materials science, and precision engineering. You need high-accuracy CNC tools, industrial-grade 3D printers, flexible robotic arms, and a digital design plus rapid testing environment that allows for fast iteration. Integration with simulation software enables virtual testing before physical builds. On top of that, you need skilled operators with interdisciplinary knowledge across mechanical design, embedded systems, control theory, and software development. Building machines that build machines is not just about automation. It's about compressing the distance between what can be imagined and what can be executed.

The engineering talent capable of achieving this is increasingly interdisciplinary. My thought here is that clusters of skilled workers in this space have a distinct advantage. Based on my initial research you can find clusters in five primary regions in the United States. Boston and Cambridge are anchored by MIT and home to legacy firms like Boston Dynamics [2]. Silicon Valley remains a stronghold with a deep pool of venture-backed robotics startups. Pittsburgh leverages Carnegie Mellon University to drive robotic innovation, while Austin, Texas, is rising fast with Tesla’s Gigafactory and a strong embedded systems culture. Here in Colorado, the Denver–Boulder–Fort Collins corridor is building momentum. The University of Colorado Boulder contributes robotics talent, and local companies like AMP Robotics, Ball Aerospace, and Intrinsic (a Google X spinout) are growing engineering teams focused on automation and scalable machine design [3][4]. It’s not the largest cluster, but it’s one with real promise and momentum. It’s the region where I plan on making contributions going forward.

Outside formal clusters, much of the talent exchange is happening in online communities. Hackaday.io is one of the most active hubs for open-source hardware builders. Reddit forums like r/robotics and r/functionalprint allow engineers to share designs and feedback loops. GitHub is where firmware, control systems, and design files live. Especially for foundational projects like GRBL, Klipper, and Marlin [5][6][7]. This is an ecosystem I want to investigate further: what enables it, where it thrives, and how it might be scaled to bring the next generation of prototyping and manufacturing capability into reality.

Things to consider:

Manufacturing capability now depends on how fast you can build and reconfigure the machines behind production lines.
Engineering talent that enables machine-building clusters around universities, megafactories, and open-source communities.
Denver and Boulder are emerging as credible nodes in this ecosystem with strong robotics and aerospace footholds.
Online platforms like Hackaday, GitHub, and ROS Discourse are core to knowledge sharing and prototyping workflows.
The real unlock may come from compressing time between idea, simulation, prototype, and deployment.

Footnotes:

[1] Patrick McGee, “Apple in China: The Capture of the World’s Greatest Company”, Scribner, 2025. https://www.simonandschuster.com/books/Apple-in-China/Patrick-McGee/9781668053379[2] https://bostondynamics.com/industry/manufacturing/[3] https://ampsortation.com/[4] https://x.company/projects/intrinsic/[5] https://github.com/grbl/grbl[6] https://github.com/Klipper3d/klipper[7] https://github.com/MarlinFirmware/Marlin

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to this audio-only podcast presentation. This is week 197 of the Lindahl Letter publication. A new edition arrives every Friday. This week, the topic under consideration for the Lindahl Letter is, “Magic state distillation explained.”

We have spent the last 3 weeks digging into quantum computing. That journey involved looking at the top 10 quantum computer leaderboard, annealing vs. gate-based systems, and the reality of enterprise plays. Trying to figure out where the edge of what is possible for quantum computing actually exists is a tricky proposition. A lot of roadmaps and promises exist in this space. People have plans, and they seem reasonable. It however, is hard to figure out what parts of them are actually real and delivering. We have seen some major movement in announcements for quantum error-reduction, which is a major step or part of a lot of roadmaps. News is going to keep breaking as we get closer to fault-tolerance. One of those breakthroughs is explained in a paper about magic state distillation that was submitted back in 2024, but just officially was published this month. The good people at the University of Osaka published a paper called, “Efficient Magic State Distillation by Zero-Level Distillation” [1]. The full citation for that 12-page paper happens to be:

Tomohiro Itogawa, Yugo Takada, Yutaka Hirano, Keisuke Fujii. Efficient Magic State Distillation by Zero-Level Distillation. PRX Quantum, 2025; 6 (2) DOI: 10.1103/thxx-njr6

Sure, improving how we use magic states is a key element of unlocking one of the top bottlenecks in quantum hardware design. The more base hardware elements that can be incorporated the lower the ceiling falls for practical implementation of quantum systems. You can read the PDF online, and the paper is readable if you are willing to look up a few terms that are commonly used in the quantum computing space [2]. You are probably well aware by now that I’m super duper interested in better understanding where gate-based quantum computing is heading in the next couple of years. This paper happens to dig into a subset of gate-based quantum computing called the Clifford operations. This is where a lot of things start as it is a well defined space. A Clifford operation is a quantum gate operation or circuit that maps Pauli operators to other Pauli operators under conjugation and can be composed of Hadamard, Phase, and CNOT gates. Think base actions or building blocks that need to be taken as part of a quantum system. A Pauli operator is one of the four fundamental 2×2 matrices (I, X, Y, Z) used to represent quantum bit-flip, phase-flip, and identity operations, forming the core building blocks of quantum error correction and circuit analysis.

This paper introduces a new technique called zero-level distillation, which dramatically simplifies how quantum computers prepare the special “magic” states needed for universal computation. Traditionally, this process required error-corrected logical qubits, making it slow and resource-intensive. The team at the University of Osaka figured out how to do this more efficiently at the physical qubit level, verify the state using error-detecting circuits, and then teleport the result into a fully protected logical qubit. This method reduces both error rates and resource costs, bringing us one step closer to practical, large-scale quantum computers. It will be interesting to see how this advancement gets built into practical hardware implementations. I was digging into an advance shared by the Microsoft Quantum team related to a new four-dimensional geometric code method trying to figure out if this used a hardware-based method or something post-hardware [3]. That paper is 40 pages long and goes into a degree of depth that is interesting, but could have benefited from a brief summary beyond the provided abstract.

Aasen, D., Hastings, M. B., Kliuchnikov, V., Bello-Rivas, J. M., Paetznick, A., Chao, R., ... & Svore, K. M. (2025). A Topologically Fault-Tolerant Quantum Computer with Four-Dimensional Geometric Codes. arXiv preprint arXiv:2506.15130.

Still wanting to learn more? You can pretty easily do a Google Scholar search for “efficient magic state distillation” and you will get a bunch of different papers you can read [4]. It did not hold my interest enough for me to pull out key papers for you, but it was a thread that almost got pulled.

This week I want to include a bonus topic from a video I watched on YouTube, “Quantum Complexity: Scott Aaronson on P vs NP and the Future.”

Aaronson explains that while P ≠ NP is widely believed, quantum computing does not resolve this distinction or solve NP-complete problems efficiently. He introduces BQP or Bounded-Error Quantum Polynomial Time as the class of problems solvable by quantum computers, noting that quantum speedups like those from Grover’s and Shor’s algorithms can apply only to problems with specific structure. Aaronson concludes that quantum computing offers significant but limited advantages, and that future breakthroughs will depend on understanding the deep complexity boundaries that define its capabilities. You could dig into the very large paper Aaronson released about computational complexity [5]. It’s 59 pages and I downloaded it to give it a read later this week.

Footnotes:

[1] The University of Osaka. (2025, June 26). Quantum breakthrough: ‘Magic states’ now easier, faster, and way less noisy. ScienceDaily. Retrieved July 12, 2025 from www.sciencedaily.com/releases/2025/06/250621233816.htm

[2] Tomohiro Itogawa, Yugo Takada, Yutaka Hirano, Keisuke Fujii. Efficient Magic State Distillation by Zero-Level Distillation. PRX Quantum, 2025; 6 (2) DOI: 10.1103/thxx-njr6 https://journals.aps.org/prxquantum/abstract/10.1103/thxx-njr6

https://journals.aps.org/prxquantum/pdf/10.1103/thxx-njr6

[3] https://azure.microsoft.com/en-us/blog/quantum/2025/06/19/microsoft-advances-quantum-error-correction-with-a-family-of-novel-four-dimensional-codes/ you can read the paper here https://arxiv.org/abs/2506.15130

[4] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=Efficient+Magic+State+Distillation&btnG=

[5] Scott Aaronson, “Why Philosophers Should Care About Computational Complexity” https://www.scottaaronson.com/papers/philos.pdf

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to this audio-only podcast presentation. This is week 196 of the Lindahl Letter publication. A new edition arrives every Friday. This week, the topic under consideration for the Lindahl Letter is, “Is quantum computing becoming an establishment play?”

You probably have heard of IBM, Google, and Microsoft. They are a pretty big deal in the technology world. IBM has a really involved and well-defined quantum computing roadmap [1]. They pretty much tell everybody who will listen about it. That roadmap includes details about error correction, fault tolerance, and the road to 10,000 gates. We also have a roadmap from Google Quantum AI which details 6 milestones and notes that they have achieved 2 of the 6 noted milestones [2]. We also have a fun quantum roadmap from the Microsoft team that notes 3 levels: foundational, resilient, and scale [3]. All those roadmaps make me wonder if quantum computing will end up becoming, in the end, a pure establishment play. On a side note, building a matrix that compares all 3 roadmaps might be interesting for a future research note and has been added to the backlog.

Some of the companies we mentioned earlier in Lindahl Letter research notes like Rigetti Computing, D‑Wave Quantum, IonQ, and Quantum Computing Inc. are doing both pure research into quantum computing to drive the technology forward and applied applications of that research, allowing people to actually access hardware. The Amazon Braket functionality platforms quantum hardware and will sell you actual access to IonQ, Rigetti, QuEra, and IQM for a reservation rate of under $7,000.00 per hour [4]. Making AWS positioned to deliver as long as the hardware is available for sale in the quantum space. That AWS hardware-as-a-service model spreads out risk. If Google or Microsoft ends up being the winner, then that strategy might involve having to buy services by API from them as they might not distribute hardware.

Apple as a company has a lot of available cash and could make a defensive patent play here by acquiring a potentially emerging technology leader in the quantum computing space. Given the workloads on Apple devices are highly repeatable and pattern-specific, maybe an annealing play while potentially limited in the end could work in the near horizon. Apple engineers are incorporating post-quantum cryptography (PQC) into their messaging and technology stacks, so they are working ahead of the game, but not directly in the quantum hardware space at least in an observable way [5].

My concern here and the reason for this research note is that no matter what innovation ends up happening in the quantum computing space, the establishment technology companies may end up winning. They may move a little bit slower getting things to market, but will end up winning the space at scale and delivery. As we all know by now, the elephant does not dance all that quickly. However, those three companies (IBM, Google, and Microsoft) do have a history of delivering enterprise scale. My guess here is if one of the quantum computing companies listed above ends up getting a key technology patented and gains a distinct advantage, the bidding war to complete an acquisition will be intense. It’s also possible that IBM may end up building the winning quantum computer by 2028, which is what they are currently shouting from the rooftops [6]. They are and have been delivering on a detailed, very well-publicized roadmap.

Footnotes:

[1] https://www.ibm.com/roadmaps/quantum/[2] https://quantumai.google/roadmap[3] https://quantum.microsoft.com/en-us/vision/quantum-roadmap[4] https://aws.amazon.com/braket/pricing/[5] https://security.apple.com/blog/imessage-pq3/[6] https://www.technologyreview.com/2025/06/10/1118297/ibm-large-scale-error-corrected-quantum-computer-by-2028/

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to this audio only podcast presentation. This is week 195 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “Annealing vs. gate based quantum computing.”

You may have picked up from the last edition of the Lindahl Letter that I’m more focused on gate based quantum computing than I am concerned about the current advances in annealing based systems. This week I went back over and dug out some gems on Google Scholar related to annealing quantum computing [1]. Some of these academic articles have a few hundred citations, but none of them seem to be breakout articles with thousands of citations. Within a small academic discipline you will see a paper pick up citations under 100 and that is probably a well read paper. Some of the mega papers in the AI space have 100,000 citations and those are foundational and very well read academic papers. Honestly, the only things that might be better read are books that break into the public mind and become bestsellers. I don’t think anything that I have found in the annealing based quantum computing space would qualify as breakout or bestseller.

Here are 5 papers that are highly cited that I thought were interesting this week:

Das, A., & Chakrabarti, B. K. (2008). Colloquium: Quantum annealing and analog quantum computation. Reviews of Modern Physics, 80(3), 1061-1081. https://arxiv.org/pdf/0801.2193

Pudenz, K. L., Albash, T., & Lidar, D. A. (2014). Error-corrected quantum annealing with hundreds of qubits. Nature communications, 5(1), 3243. https://www.nature.com/articles/ncomms4243.pdf

Hauke, P., Katzgraber, H. G., Lechner, W., Nishimori, H., & Oliver, W. D. (2020). Perspectives of quantum annealing: Methods and implementations. Reports on Progress in Physics, 83(5), 054401. https://arxiv.org/pdf/1903.06559

Morita, S., & Nishimori, H. (2008). Mathematical foundation of quantum annealing. Journal of Mathematical Physics, 49(12). https://arxiv.org/pdf/0806.1859

Yarkoni, S., Raponi, E., Bäck, T., & Schmitt, S. (2022). Quantum annealing for industry applications: Introduction and review. Reports on Progress in Physics, 85(10), 104001. https://arxiv.org/pdf/2112.07491

What exactly is annealing quantum computing?

Let’s answer this question the hard way by first stating that universal gate based quantum computing is built out to a set of qubits where any type of computation could be worked using the available qubits. If you have quantum computing work, then you are good to go within the universal gate-based system. Now let’s say you were a company like D-Wave Systems and you wanted to take a different direction than universal gate-based quantum computing and lean into the annealing quantum computing world. You would start to build a system that works toward being optimized for special use cases and you might write a nice presentation about it which you could read [2]. Based on what D-Wave Systems is sharing, annealing quantum computing system uses a method that solves optimization problems by gradually evolving a quantum system toward its lowest energy state, or ground state. It leverages the quantum adiabatic theorem, which ensures that a system will remain in its ground state if changes to its energy landscape are made slowly enough. Instead of using logic gates, annealing encodes a problem into a Hamiltonian (which is a mathematical function that describes the total energy of a quantum system), and the solution emerges as the system relaxes. This model excels at solving complex combinatorial optimization problems. Unlike gate-based systems, annealers are not universal quantum computers but offer practical advantages for certain narrow tasks. It’s like special quantum computing vs. general quantum computing.

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=annealing+quantum+computing&oq=Annealing+qu

[2] https://s201.q4cdn.com/339170267/files/doc_presentations/2025/Mar/31/20250331_D-Wave-Technology-and-the-Competitive-Landscape.pdf

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to this audio only podcast presentation. This is week 194 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “The top 10 quantum computer leaderboard.”

…and we are back writing extra fresh weekly research notes for your inbox. My current writing process involves working during some weekend morning writing sessions to produce a one week forward product. At some point, it is entirely possible that we will get back to the 5 week production process that I used for the last few years, but that is not on the current production roadmap for the foreseeable future. You can think of this like a weekly research sprint with a solid weekly written narrative based retrospective. We are going to stay timely, focused, and consistent in terms of delivery and quality.

During this fine Saturday morning with some absolutely beautiful weather in Denver, Colorado, I’m working to pull together a list of the biggest verifiable quantum computers based on the qubit count and their country of origin. A lot of different ways exist to assemble this potential list. The list is going to effectively be a top 10 leaderboard snapshot that I check in on a quarterly basis or when a new major announcement happens to just figure out how much change is happening in the space. I think a separate research question exists related to some of the investment returns people have seen in the quantum computing space. Spoiler alert: some of the returns have been huge like IonQ, D-Wave Quantum, and Rigetti Computing, but that is a topic I’ll dig into during another research note. This topic has been added to the brand new research topic backlog for future consideration.

This particular research note is focused on cataloging the best quantum computing systems so we can compare them to what is going to be released going forward to better understand the rate of change. My thesis here is that the rate of change in terms of qubits is about to radically increase. We are going to see the number of qubits increase, unlocking some use cases that were not practical before the increase. Players like Amazon are getting into the quantum computing space with products like the Ocelot chip [1]. A lot of roadmaps like the one from IBM currently exist with new builds of major flagship quantum computers that incorporate error-correction [2]. Keep in mind that right now both IBM and AWS will sell you quantum computing time [3][4]. I made another backlog reminder to run some quantum code on both of those services to see the services in action and provide feedback.

Top 10 quantum computers by universal gate-based physical qubits:

Atom Computing - 1,180 qubits (October 24, 2023) United States
IBM Condor - 1,121 qubits (December 4, 2023) United States
CAS Xiaohong - 504 qubits (December 6, 2024) China
IBM Osprey - 433 qubits (November 9, 2022) United States
Fujitsu & RIKEN - 256 qubits (April 22, 2025) Japan
Xanadu Borealis - 216 qubits (June 1, 2022) Canada
IBM Heron R2 - 156 qubits (November 13, 2024) United States
IBM Eagle - 127 qubits (November 16, 2021) United States
Google Willow - 105 qubits (December 9, 2024) United States
USTC Zuchongzhi 3.0 - 105 qubits (March 3, 2025) China

For the most part the race is between the United States (6) and China (2). It’s worth noting that two of the freshest entries to this list are from China and the latest one (April, 2025) is from Japan. We are going to see some major changes to this list either in late 2025 or 2026 as a number of companies (IBM, Fujitsu & RIKEN, and Microsoft) are targeting releases of quantum computers that would make this list. I’m specifically tracking universal gate-based physical qubit quantum computers vs. annealing systems like D-Wave Quantum as those are the ones that I think have the best shot of actually implementing and scaling Shor’s algorithm which will be the most destabilizing news headline from any of this as it will render anything outside of quantum resistant encryption obsolete [5]. That won’t be universally true on day one of this technology, but it would be true in practice as nobody is going to just use one of these quantum computers to sit around and break basic encryption all day for sport. Maybe for specific use cases or high value targets, but it won’t be a universal shift all at one time.

Projected future top 10 quantum computers by universal gate-based physical qubits:

Pasqal 10k Neutral‑Atom - 10,000 qubits (Projected 2026) France
QuEra 10k Neutral‑Atom - 10,000 qubits (Projected 2026) United States
Atom Computing - 1,180 qubits (October 24, 2023) United States
IBM Condor - 1,121 qubits (December 4, 2023) United States
CAS Xiaohong - 504 qubits (December 6, 2024) China
Fujitsu & RIKEN - 1,000 qubits (Projected 2026) Japan
IBM Osprey - 433 qubits (November 9, 2022) United States
Fujitsu & RIKEN - 256 qubits (April 22, 2025) Japan
Xanadu Borealis - 216 qubits (June 1, 2022) Canada
IBM Heron R2 - 156 qubits (November 13, 2024) United States
Pasqal Orion Gamma - ~140 qubits (Projected 2025) France
IBM Heron R1 - 133 qubits (December 4, 2023) United States
IBM Eagle - 127 qubits (November 16, 2021) United States
IBM Loon - ~120 qubits (Projected 2025) United States
IBM Nighthawk - ~120 qubits (Projected 2025) United States
Microsoft Majorana 1 - 8 qubits (Projected 2025+) United States [6]

Footnotes:

[1] https://www.aboutamazon.com/news/aws/quantum-computing-aws-ocelot-chip[2] https://www.ibm.com/quantum/technology#roadmap[3] https://www.ibm.com/quantum/pricing[4] https://aws.amazon.com/braket/pricing[5] Realization of a scalable Shor algorithm paper https://arxiv.org/pdf/1507.08852[6] I know the Microsoft entry is projected to be really tiny in terms of qubits, but I included it anyway given the importance of Microsoft as a company entering the quantum chip market.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to this audio only podcast presentation. This is week 193 of the Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for the Lindahl Letter is, “A 56 day posting break.”

Over the last 4 years, the Lindahl Letter has taken two pretty decent breaks in posting content. This last pause in posting happened to be the most recent 56-day posting break. Maybe this (right here, right now) is a good point in the process to refocus, reconsider, and maybe reboot. Before all that happens, let me answer the question that you all have outstanding. Yes, I read that paper from Machine Learning Research at Apple called, “The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity” [1]. This paper was foundation shaking. Those researchers from Apple ask some very serious questions about what is actually happening with these reasoning models. I have told people during conversations for years that I fervently believe that what we will see is machine learning methods and the new class of large language models being used to augment workflows and deliver specific value based use cases. That is a rational expectation for what is going to be delivered. I, for better or worse, have argued that we would see a lot of technology get built into products with enterprise scale and scope. People are going to use what gets delivered to them and is easy to enable.

Several trends are on my radar that I’m curious about researching, and that is in the end what yields the spark for these Lindahl Letter research notes. A lot of companies are doing some really great work at the edge of making actual quantum computers that work. My method of measuring that is in how the latest releases are explaining the great race to have the biggest number of qubits. I think a leaderboard could be maintained with the largest qubit-based quantum computers in the world. Let’s call that race to have the best quantum computer that can accomplish real things, the first trend I’m interested in following. Second, I’m curious about the potential for agents to take action on your behalf and what that will mean for society in general. Right now, as I mentioned above, we have a lot of augmentation, but not as much action being built. I think that is a trend that will change as Google, Microsoft, and Apple get more engaged in the game. It’s also possible that Meta figures that one out, but their surface for action is more limited than the other platform companies. Third, I'm really curious about what is going to happen with machines that build machines. We had the 3D printing revolution where these things almost got commoditized to the point where most people could afford one. Moving from making things with 3D printing to potentially making machines or components that make other machines, I think will be the next major trend in manufacturing enablement.

Those three trends on my radar are the great quantum computing race, agents taking action, and machines making machines. My goal here is to continue to produce research notes on a weekly basis, really diving into various parts of these trends that are totally and wholesale organically written, researched, and ultimately published. Bespoke and hand-curated content brought to your inbox every Friday. At this point, I’m taking my backlog that includes around a hundred topics for this Lindahl Letter writing effort and setting it aside to pivot to the trends listed above as an attempt to get closer to the edge of what is possible and further away from pure research of things that have already happened. As a true pracademic, my interests are really in what is becoming possible. That is not a futurist question, but a practical edge of possibility question that I believe deserves in-depth consideration.

Footnotes:

[1] https://machinelearning.apple.com/research/illusion-of-thinking

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for being a part of the journey. This is week 179 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “Designed to Distract: How Technology Gets Your Attention.”

Your attention is a battlefield, and modern technology is armed with automated and now AI powered weapons of mass distraction. Every ping, notification, and infinite scroll is designed to keep you engaged, often longer than you intend. This isn’t a coincidence—it’s a calculated business model. The longer you stay on a platform, the more data it collects and the more revenue it generates through ads. This system thrives on capturing and exploiting your focus, turning your attention into a commodity. My bookshelf includes a physical copy of Tim Wu’s 2016 book, “The Attention Merchants” [1]. A lot of things have been published that dial into things related to how attention is changing.

The tactics used to divert attention are subtle yet powerful. One of the most pervasive is infinite scroll, a feature introduced to eliminate natural stopping points. Instead of deciding when to stop, you’re continuously pulled into the next post or article. Similarly, autoplay videos take advantage of your inertia, playing the next episode or clip before you even have a chance to close the app. Then there are push notifications, which interrupt your focus with alerts that feel urgent but rarely are. These tools aren’t neutral—they’re designed to create a sense of compulsion.

At the heart of these tactics is personalized algorithms, powered by artificial intelligence. These algorithms study your behavior, preferences, and even vulnerabilities to predict and serve content that will keep you engaged the longest. While they often provide convenience, they also create feedback loops, reinforcing behaviors that keep you tethered to a platform. For example, social media thrives on social validation loops, where likes, shares, and comments trigger dopamine hits that make you crave more engagement.

This constant assault on your focus has real consequences. On a personal level, it leads to fragmented attention—the inability to concentrate deeply on tasks. Every time a notification interrupts your work, it takes an average of 23 minutes to fully refocus [2]. Multiply that by the dozens of interruptions you experience daily, and the productivity cost becomes staggering. Emotionally, the effects are just as damaging. Platforms often prioritize sensational or negative content because it generates more engagement, leading to heightened anxiety, outrage, and even depression. Relationships suffer as well; when your attention is split between your phone and the people around you, trust and connection erode.

But perhaps the most insidious effect is the erosion of your ability to think deeply. Focused, uninterrupted time is essential for problem-solving, creativity, and self-reflection. Yet, in a world of constant distractions, these opportunities become increasingly rare. Instead of engaging in deep work, many of us find ourselves trapped in cycles of shallow tasks, like checking email or scrolling social media.

The good news is that you can take back control. Start by turning off non-essential notifications to reduce interruptions. Most apps don’t need to buzz or flash for your attention—set boundaries so you decide when to engage. Limit your screen time with tools like app blockers or by scheduling specific periods for digital use. Another effective strategy is to introduce stopping cues to counteract infinite scroll and autoplay. For example, commit to watching one episode or reading for a set amount of time, then stop deliberately.

Curating your digital environment can also help. Unfollow accounts or unsubscribe from feeds that don’t add value to your life. Replace them with content that inspires or educates you. When you use technology, do so intentionally. Ask yourself, “Why am I opening this app? What do I hope to achieve?” This small pause can prevent mindless scrolling and keep your focus aligned with your goals.

The battle for your attention is ongoing, but it’s one you can win. By understanding how your focus is being diverted and taking deliberate steps to protect it, you regain the power to direct your attention where it truly matters. The next chapter will show you how to shift from reacting to distractions to prioritizing what’s most important, laying the foundation for a more intentional and focused life.

Footnotes:

[1] https://www.penguinrandomhouse.com/books/234876/the-attention-merchants-by-tim-wu/

[2] https://ics.uci.edu/~gmark/chi08-mark.pdf

What’s next for The Lindahl Letter?

Week 180: The Focus Formula: Prioritize What Truly Matters
Week 181: Your Attention Fortress: Building a Distraction-Free Life
Week 182: Deep Work, Rare Results: The Art of Uninterrupted Focus
Week 183: Connection in the Chaos: Restoring Presence in Relationships
Week 184: Recharge to Refocus: The Power of Rest and Renewal

If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. Make sure to stay curious, stay informed, and enjoy the week ahead!

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for being a part of the journey. This is week 178 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “Inside the Mind: The Science of Focus and Distraction.”

Focus is a skill, but to master it, you need to understand the mechanisms driving it. At its core, focus is the ability to direct attention to a specific task, thought, or sensation while filtering out, blocking, or generally ignoring distractions. It’s not a static state but a dynamic process, constantly influenced by biology, psychology, and the environment. This interplay determines whether you can sustain deep concentration or get pulled into the whirlwind of modern distractions that are a part of the digital age.

Adding complexity to this equation is dopamine, the brain’s “reward” forward chemical. Dopamine motivates you by creating a sense of pleasure and satisfaction when you complete tasks or encounter something new. However, modern technology exploits this system. Every notification, like, or email provides a small dopamine hit, training your brain to seek instant gratification. This cycle rewires your focus, making it harder to engage deeply in tasks that don’t offer immediate rewards. Understanding this chemical dynamic is key to reclaiming your ability to concentrate.

Focus also operates in cycles, influenced by your body’s natural rhythms. The ultradian rhythm reflecting some fraction of an hour cycles of peak energy followed by dips plays a significant role in your ability to sustain attention [1]. Aligning your work with these cycles can maximize productivity. Equally important is sleep. Quality rest doesn’t just restore your energy; it consolidates memories, clears mental clutter, and primes your brain for focus the next day. Neglecting sleep, on the other hand, leads to brain fog, reduced cognitive function, and an increased susceptibility to distractions. My sleep is tracked every day by my Oura ring and it really does correlate with readiness [2].

Many myths about focus further complicate the path to mastering it. For instance, multitasking is often celebrated as a valuable skill, but research shows it splits attention and decreases productivity. Similarly, the belief that some people are naturally better at focusing overlooks the fact that focus is a skill that can be developed. And while eliminating all distractions might seem like the ultimate solution, it’s neither practical nor entirely beneficial. Instead, the goal should be to manage distractions and strengthen your ability to return to your chosen task.

Despite these barriers, focus can be cultivated with the right strategies. Start by setting clear priorities for your day. A short list of three key tasks can help reduce decision fatigue and keep your attention directed. I always keep a list of things to stop doing as well. Next, design a distraction-free workspace. Declutter your environment, silence notifications, and use tools like website blockers during periods of deep work. Incorporating brief, intentional breaks is another powerful way to sustain focus. Techniques like the Pomodoro Method—25 minutes of work followed by 5 minutes of rest—can refresh your mind and prevent burnout [3].

Focus is also strengthened through consistent training. Practices like mindfulness meditation improve your ability to resist distractions by teaching your brain to sustain attention on a single thought or sensation [4]. Single-tasking, where you commit to completing one task before moving to the next, is another effective exercise. Over time, these practices build your focus muscle, making it easier to engage deeply with challenging work.

Understanding how focus works isn’t just an academic exercise—it’s the foundation for living intentionally in a world filled with distractions. By aligning your habits with the science of attention, you can reclaim control over your focus, direct it toward meaningful goals, and unlock your full potential. The next step is to recognize how your attention is being deliberately diverted by external forces—and to learn how to defend it.

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=ultradian+rhythm+productivity&oq=ultradian+rhythm

[2] https://ouraring.com/blog/how-does-the-oura-ring-track-my-sleep/

[3] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=pomodoro+method+effectiveness&oq=Pomodoro+Method

[4] https://mindful.usc.edu/resources/

What’s next for The Lindahl Letter?

Week 179: Designed to Distract: How Technology Grabs Your Attention
Week 180: The Focus Formula: Prioritize What Truly Matters
Week 181: Your Attention Fortress: Building a Distraction-Free Life
Week 182: Deep Work, Rare Results: The Art of Uninterrupted Focus
Week 183: Connection in the Chaos: Restoring Presence in Relationships

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for being a part of the journey. This is week 177 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “Your valuable attention: Why Your Focus Is Under Siege.”

In a world where your attention is more valuable than ever, every scroll, click, and swipe is part of an invisible economy. This “attention economy” drives social media platforms, streaming services, and even productivity tools. It’s not your time they want—it’s your focus. The cost of lost attention is both personal and societal. On an individual level, fragmented focus lowers productivity, weakens relationships, and diminishes a sense of purpose. On a societal scale, the effects ripple outward, creating polarization, misinformation, and a culture that values busyness over depth. Occupied time is not always productive. We have to move to strengthen the fabric of civil society. It’s our general civility that has become unsettled.

The statistics are startling. The average person now spends over seven hours daily consuming digital media. We are focused on digital driving through a forever updating sea of digital content. Notifications, pop-ups, and infinite scrolls have rewired our brains and expectations to crave constant stimulation, sadly leaving little room for deep thought or creativity. The attention span of the modern human is estimated at just 8.25 seconds—shorter than that of a goldfish [1]. This isn’t an accident; it’s by design. Technology companies have mastered the art of capturing your focus. Every feature on your favorite app, from autoplay videos to personalized algorithms, is crafted to keep you engaged for as long as possible. The longer you stay, the more data they collect and the more ads they show. Attention has become the currency of the 21st century, and you’re the commodity. People have been saying that attention is the new oil for about 7 years [2][3].

Your attention is the gateway to everything you value—learning, relationships, civility, and achieving your goals. Without the ability to focus, time slips away unnoticed. Productivity declines, creativity dwindles, and even happiness suffers. The constant pull of distractions chips away at your ability to live intentionally. Yet, understanding the problem is the first step to regaining control. When you recognize that your attention is being diverted, you can begin to take deliberate steps to reclaim it.

The attention economy thrives on a simple premise: the longer you stay engaged, the more valuable you are. Algorithms study your habits, preferences, and vulnerabilities, ensuring that the content you see is optimized to keep you scrolling. But the effects go beyond wasted time. In the workplace, frequent interruptions reduce productivity and lead to decision fatigue, costing billions in lost output annually. In personal relationships, divided attention weakens connections, leaving friends, partners, and colleagues feeling undervalued. On a mental health level, the endless cycle of notifications and comparisons fosters anxiety, burnout, and a distorted sense of self-worth. It feels good to feel busy, but that does not translate to actual outcomes.

The good news is that you can fight back. Reclaiming your attention starts with awareness. Recognize when and where your focus is being pulled, then take actionable steps to protect it. Turn off non-essential notifications; your phone doesn’t need to buzz for every like, comment, or update. Set digital boundaries using tools like screen time trackers or app blockers to create intentional limits. Schedule time for focused, uninterrupted work on meaningful tasks. Most importantly, reconnect with presence during conversations and relationships. Put away your devices and engage fully.

Your attention isn’t infinite, but it is powerful. By reclaiming control, you can transform your relationship with technology, your work, and the people in your life. The battle for your attention isn’t just a personal challenge—it’s a societal one. As individuals, we must learn to resist the pull of distractions. As a society, we must demand ethical technology that respects our focus rather than exploits it. Your focus is your greatest asset. Don’t let it be stolen. One of the big changes that I made was shifting to a fitness ring instead of allowing alerts on my wrist from a watch. For me those wrist alerts shattered my efforts to achieve deep work and sustain focus. Sometimes you just need to focus and those alerts, notifications, or messages just need to wait a little bit in the attention priority queue.

Footnotes:

[1] I’m not entirely sure this citation is the best source for this metric, but it does seem to be commonly cited and is from 2015 Time magazine https://time.com/3858309/attention-spans-goldfish/

[2] https://www.google.com/search?q=%22attention+is+the+new+oil%22

[3] https://medium.com/@setsutao/attention-is-the-new-oil-not-data-bf54c64d3279

What’s next for The Lindahl Letter?

Week 178: Inside the Mind: The Science of Focus and Distraction
Week 179: Designed to Distract: How Technology Grabs Your Attention
Week 180: The Focus Formula: Prioritize What Truly Matters
Week 181: Your Attention Fortress: Building a Distraction-Free Life
Week 182: Deep Work, Rare Results: The Art of Uninterrupted Focus

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Quantum computing continues to captivate the imagination of scientists, technologists, and futurists alike, offering the promise of solving problems intractable for classical machines. Amidst the steady stream of breakthroughs, one concept has emerged with both scientific intrigue and practical potential: time crystals. These exotic states of matter, once considered the stuff of theoretical musings, are now taking shape in laboratories and, intriguingly, hold promise for quantum computing applications.

At their core, time crystals are a new phase of matter, one that breaks time-translation symmetry. In classical physics, symmetry breaking usually refers to spatial phenomena—such as ice forming from water, where the uniformity of liquid water transitions to the structured lattice of solid ice. Time crystals, however, add a temporal twist: they exhibit periodic motion that persists indefinitely without energy input, defying classical expectations. Discovered in 2012 by Nobel laureate Frank Wilczek as a theoretical construct and experimentally realized in 2016, time crystals are not perpetual motion machines but rather quantum systems that oscillate in a stable, repeating pattern under the influence of an external driver.

For quantum computing, time crystals offer a tantalizing prospect. They provide a platform where quantum states can be maintained with high coherence—essential for reliable quantum computation. Time crystals are inherently non-equilibrium systems, making them robust against many types of environmental noise. This resilience could address one of the major hurdles in quantum computing: error correction and qubit stability. A significant step forward was the recent use of time crystals in trapped-ion quantum computers, where researchers demonstrated their potential for executing quantum gates. By leveraging the stable periodicity of time crystals, quantum systems can operate in an environment that naturally mitigates decoherence, effectively improving the reliability of computations.

Recent advances have seen time crystals moving from theoretical oddities to functional components in experimental setups. For instance, researchers using Google’s Sycamore processor observed time-crystal behavior, showing how these systems can be integrated into existing quantum hardware. Similarly, trapped-ion systems have demonstrated the potential of time crystals to enhance the coherence of qubits, making them candidates for long-term storage and high-fidelity operations. Additionally, their unique oscillatory states could play a role in synchronizing quantum systems across distributed networks, paving the way for scalable quantum communication.

Despite these exciting prospects, integrating time crystals into practical quantum computing remains a challenge. Their behavior, while stable, is highly sensitive to precise conditions and external drivers. Scaling these systems to handle complex quantum algorithms will require significant advancements in both hardware and theoretical understanding. Furthermore, the interplay between time crystals and other emerging quantum technologies, such as topological qubits and error-correcting codes, remains an open field of inquiry. Bridging these domains could unlock entirely new architectures for quantum computation.

The journey of time crystals from a theoretical prediction to an experimental reality is a testament to the rapid pace of quantum innovation. As we continue to explore their potential, these shimmering oscillations in the fabric of time may serve as a cornerstone for the next generation of quantum computers. In the ever-evolving narrative of quantum technology, time crystals represent both a scientific triumph and a beacon for what lies ahead—a fusion of curiosity, creativity, and the relentless pursuit of the unknown.

Thank you for joining me for this week’s edition of The Lindahl Letter. Stay curious, and see you next week as we delve deeper into the quantum frontier.

What’s next for The Lindahl Letter?

Week 177: The Attention Economy: Why Your Focus Is Under Siege
Week 178: Inside the Mind: The Science of Focus and Distraction
Week 179: Designed to Distract: How Technology Hijacks Your Attention
Week 180: The Focus Formula: Prioritize What Truly Matters
Week 181: Your Attention Fortress: Building a Distraction-Free Life

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Welcome back to another edition of The Lindahl Letter. It’s week 175, and we’re diving into the fascinating topic of universal quantum computation. This is an area where the boundaries of theory and practical application intersect, offering both incredible promise and immense challenges. If you’re tuning in to this podcast for the first time, welcome aboard. For regular readers and listeners, you already know this is a space where we examine complex topics with an eye on clarity and relevance.

At its core, the concept of a universal quantum computer is as ambitious as it sounds. It’s the quantum computing equivalent of a general-purpose classical computer—think of it as a machine that can perform any quantum operation, given enough time and resources. The analogy to the classical Turing machine is apt, but the quantum realm is a different beast altogether. Where classical systems rely on bits flipping between 0 and 1, quantum systems leverage qubits, which exist in superpositions and can be entangled in ways that fundamentally alter how computations unfold.

Achieving universality in quantum computation boils down to the idea that we can simulate any quantum process using a combination of quantum gates. These gates are the building blocks of quantum circuits, manipulating qubits in ways that enable properties like superposition, entanglement, and interference. In practice, a small set of gates—such as the CNOT gate combined with single-qubit operations like the Hadamard and Pauli gates—forms what’s known as a universal set. With these, any quantum operation can theoretically be approximated to arbitrary precision.

Of course, theory and practice are rarely perfect companions. The current landscape of quantum computing is dominated by what’s known as Noisy Intermediate-Scale Quantum (NISQ) devices. These systems are powerful but imperfect, constrained by issues like qubit fidelity, error rates, and limited coherence times. The leap to universal quantum computation requires addressing two major challenges: error correction and scalability. Quantum error correction is a monumental task in itself, demanding additional qubits to safeguard against the natural noise and decoherence that plague quantum systems. Scalability, meanwhile, demands not just more qubits but better qubits—ones that can operate with higher fidelity and stronger connectivity.

Despite these hurdles, progress is being made. Theoretical frameworks, like the Church-Turing-Deutsch principle, assert that any physical process can be simulated by a universal quantum computer. That idea has fueled decades of research and development. On the practical side, companies like IBM, Google, and IonQ are racing to push the limits of what quantum systems can achieve. IBM’s ambitious roadmap to a million-qubit machine is a bold declaration of intent, and the algorithms already developed for quantum systems—like Shor’s algorithm for factoring large numbers—hint at the transformative potential waiting to be unlocked.

It’s easy to see why universal quantum computation captures the imagination. The implications stretch far beyond the confines of academia or industry, touching fields as diverse as cryptography, materials science, and optimization. Yet, the path forward is long and uncertain. It’s not a matter of if we get there but when—and how the journey reshapes the landscape of computing along the way.

Thank you for taking the time to explore this frontier with me. If you’ve made it this far, I appreciate your curiosity and engagement. As always, stay curious, stay informed, and I’ll see you next week for another deep dive.

What’s next for The Lindahl Letter?

Week 176: Quantum Computing and Advances in Time Crystals
Week 177: The Attention Economy: Why Your Focus Is Under Siege
Week 178: Inside the Mind: The Science of Focus and Distraction
Week 179: Designed to Distract: How Technology Grabs Your Attention
Week 180: The Focus Formula: Prioritize What Truly Matters

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Quantum computing has long been hailed as a transformative technology with the potential to revolutionize fields such as cryptography, optimization, material science, and beyond [1]. However, quantum computing faces a fundamental challenge: the fragility of quantum states. Quantum bits, or qubits, are extraordinarily sensitive to errors caused by environmental noise, decoherence, and operational inaccuracies. Without robust error correction, this fragility undermines the reliability of quantum computations and makes it nearly impossible to scale quantum systems for practical use. Solving this problem is not just important—it is essential. Overcoming the challenge of error correction is the key to unlocking the transformative potential of quantum computing.

The most cited relevant reference here has over 900 citations. It’s 46 pages and rather math heavy in parts.

Gottesman, D. (2010, April). An introduction to quantum error correction and fault-tolerant quantum computation. In Quantum information science and its contributions to mathematics, Proceedings of Symposia in Applied Mathematics (Vol. 68, pp. 13-58). https://arxiv.org/pdf/0904.2557

Historically, quantum error correction has been viewed as a critical but demanding overhead. Detecting and correcting errors in quantum systems requires an extraordinary number of physical qubits to encode logical qubits, with some estimates suggesting hundreds to thousands of physical qubits are needed for just one logical qubit. This sheer overhead has presented a formidable barrier to scaling quantum systems. Recent advances, however, are changing the narrative. The concept of error correction tolerant quantum computing represents a new paradigm: rather than simply adding layers of error correction, these systems aim to minimize the resources and performance penalties associated with error correction. They incorporate innovations in fault-tolerant architectures, error-resilient algorithms, and hardware designs that lower baseline error rates, making error correction more efficient and less resource-intensive.

The significance of this shift cannot be overstated. Quantum computers operate using qubits that harness the principles of superposition and entanglement, which enable powerful computational possibilities but also make qubits susceptible to errors. Errors can take the form of bit flips, phase flips, or decoherence, any of which can disrupt calculations. Without a solution to these challenges, quantum computing will remain a theoretical possibility rather than a practical tool. Error correction tolerance offers a pathway forward, reducing the burden on physical qubits and accelerating the timeline for practical quantum systems.

The promise of error correction tolerant quantum computing lies in its ability to make quantum computing scalable, efficient, and cost-effective. With reduced error correction overhead, more logical qubits can be supported without requiring exponential increases in physical qubits. This enhances scalability while making quantum systems more efficient and affordable for research and industrial applications. Furthermore, error correction tolerance paves the way for faster execution of quantum algorithms, ensuring that quantum computers are not only reliable but also competitive with classical systems in terms of speed.

Major players in the quantum space, including IBM, Google, and Rigetti, are actively pursuing this critical area of research. Recent breakthroughs include adaptive error correction that dynamically adjusts protocols to system performance, noise-aware algorithms that tolerate specific noise patterns, and hybrid quantum-classical approaches that use classical computation to support quantum error correction. These developments demonstrate both the complexity of the problem and the progress being made to address it. Looking ahead, future directions will likely include the integration of machine learning techniques to optimize error correction strategies and the development of materials and designs that are inherently resistant to errors.

Ultimately, solving the challenge of error correction is essential for quantum computing to achieve its full potential. Without it, the field will remain limited to small-scale, experimental systems. With it, quantum computing can scale to tackle some of the most complex problems in science, industry, and beyond. Error correction tolerance represents a critical step toward this future, making the dream of practical quantum computing not just possible, but inevitable.

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=Error+correction+tolerant+quantum+computing&btnG=

What’s next for The Lindahl Letter?

Week 175: universal quantum computation
Week 176: Quantum Computing and Advances in Time Crystals
Week 177: The Attention Economy: Why Your Focus Is Under Siege
Week 178: Inside the Mind: The Science of Focus and Distraction
Week 179: Designed to Distract: How Technology Hijacks Your Attention

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

We are going to spend some time digging into quantum computing over the next few weeks. Things are starting to move forward in that space which is exciting [1]. Let’s not waste a second and just go ahead and jump right into the deep end of this magical quantum puzzle. Here we go!

Nondeterministic gates present a fascinating challenge within the evolving landscape of quantum computing. At their core, these gates function probabilistically, meaning their outcomes are not guaranteed in the deterministic sense familiar to classical computation. This intrinsic uncertainty aligns with the broader principles of quantum mechanics but complicates the goal of building reliable and scalable quantum systems. Understanding how to integrate nondeterministic gates into fault-tolerant architectures is an essential step in moving quantum computing from the lab to practical applications. On a side note we may very well dig into the brilliantly intriguing world of creating time crystals again soon during week 176 where some of the ambiguity of being probabilistic disappears.

Fault-tolerant quantum computation relies on carefully crafted error-correction techniques to manage the delicate states of qubits, which are highly susceptible to noise and decoherence. The introduction of nondeterministic gates adds another layer of complexity to this already intricate problem. These gates often succeed probabilistically, necessitating either multiple attempts or supplementary operations to ensure the desired outcome. While this characteristic can simplify certain hardware requirements—especially in photonic systems where nondeterministic interactions are a natural fit—it also demands more sophisticated error management strategies to maintain computational fidelity.

The key to making nondeterministic gates viable lies in adaptive computation strategies. Measurement-based quantum computing (MBQC) exemplifies this approach, using entangled resource states and measurements to drive computation. In MBQC, the probabilistic nature of certain operations is counterbalanced by flexible correction protocols, which adjust subsequent steps based on observed outcomes. It’s basically overhead from error checking and dropping the results of failed gates. This adaptability creates a robust framework for handling nondeterminism but comes at the cost of increased resource requirements, including additional qubits and computational overhead. Balancing these trade-offs is critical for the success of practical quantum systems.

Nondeterministic gates challenge the quantum community to rethink what fault tolerance means in this new paradigm. Traditional error-correction methods like the surface code were designed with deterministic operations in mind, and they must evolve to address the probabilistic errors introduced by these gates. This evolution involves tighter integration of classical and quantum systems, allowing for real-time error detection and response. It also calls for a deeper understanding of how to optimize quantum resources to handle the additional uncertainty without sacrificing scalability.

Here are three articles to check out:

Li, Y., Barrett, S. D., Stace, T. M., & Benjamin, S. C. (2010). Fault tolerant quantum computation with nondeterministic gates. Physical review letters, 105(25), 250502. https://arxiv.org/pdf/1008.1369

Kieling, K., Rudolph, T., & Eisert, J. (2007). Percolation, renormalization, and quantum computing with nondeterministic gates. Physical Review Letters, 99(13), 130501. https://arxiv.org/pdf/quant-ph/0611140

Nielsen, M. A., & Dawson, C. M. (2005). Fault-tolerant quantum computation with cluster states. Physical Review A—Atomic, Molecular, and Optical Physics, 71(4), 042323. https://arxiv.org/pdf/quant-ph/0405134

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=Nondeterministic+gates+tolerant+quantum+computation&btnG=

What’s next for The Lindahl Letter?

Week 174: error correction tolerant quantum computing
Week 175: universal quantum computation
Week 176: Quantum Computing and Advances in Time Crystals
Week 177: The Attention Economy: Why Your Focus Is Under Siege
Week 178: Inside the Mind: The Science of Focus and Distraction

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Transfer learning has proven to be an invaluable tool in machine learning, enabling us to take advantage of pre-trained models to boost performance on new tasks, even with limited data. Instead of training a model from scratch, we can repurpose one trained on a large, diverse dataset to extract features—essential characteristics of the data—that are often applicable across various problems. For instance, in image recognition, these features might be edges, textures, or patterns that the model has learned to detect. Transfer learning allows us to reuse these learned features and apply them to new tasks, saving both time and computational resources.

The key idea behind transfer learning for features is that many of the low-level features learned by a model are transferable to new domains. Models like ResNet in computer vision or BERT in natural language processing learn generalizable features from large datasets, which can be applied to a variety of new tasks. By transferring the feature extraction layers from these models, we can fine-tune them for specific tasks with far less data. This significantly reduces the amount of time and effort needed to train a model, since the lower-level features have already been learned, allowing us to focus on task-specific learning.

Take medical imaging, for example. A model trained on a vast dataset of general images can be fine-tuned for tasks like detecting tumors in X-rays or MRIs by leveraging the features it already knows how to extract. Similarly, in natural language processing, models like GPT or BERT can be adapted to perform sentiment analysis or text classification tasks with minimal additional data. In voice recognition, a pre-trained model could be adapted to identify speakers or recognize commands in a noisy environment, utilizing previously learned features from a broader speech dataset.

While transfer learning offers numerous benefits, it’s not without its challenges. One potential issue is domain shift, where the source and target datasets are too dissimilar, making the transferred features less useful. Fine-tuning is often required to ensure the model performs well on the new task, and this can be tricky if the new data is too sparse. Additionally, there’s the risk of overfitting when working with limited data, which could compromise the model’s generalization ability. Despite these hurdles, transfer learning remains a powerful tool, allowing us to adapt pre-trained models to new challenges quickly and efficiently.

Looking ahead, the growing availability of pre-trained models and powerful transfer learning techniques is likely to drive even more innovations in fields like healthcare, finance, and beyond. As the models become more specialized and the datasets even larger, the opportunities for transfer learning will expand, enabling more complex tasks to be tackled with fewer resources. By enabling machines to generalize features across tasks, transfer learning is not only enhancing efficiency but also making machine learning more accessible to a wider range of applications, from startup projects to large-scale enterprise solutions.

Things to consider this week:

Footnotes:

[1]

What’s next for The Lindahl Letter?

Week 173: nondeterministic gates tolerant quantum computation
Week 174: error correction tolerant quantum computing
Week 175: universal quantum computation
Week 176: resilient quantum computation
Week 177: quantum computation with higher dimensional systems

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

As machine learning evolves, traditional approaches to feature engineering are being transformed by the power of graph data structures. Graphs—representing entities as nodes and relationships as edges—provide a rich framework to model complex, non-linear connections that go beyond what’s possible with tabular data. It’s an area of focus I keep going back to better represent knowledge. By embracing graph-based feature engineering, we can uncover deeper insights and create more effective predictive models. I spent some time looking around Google Scholar results trying to find a really interesting deep dive on this subject and was somewhat disappointed [1].

Graphs are highly versatile and have applications in diverse domains. In social networks, for example, users (nodes) interact through actions like likes, shares, or friendships (edges). Graph-based features such as centrality measures can reveal influential users or detect communities. In e-commerce, graphs model user-product interactions, capturing relationships that enhance recommendation systems. For instance, understanding the co-purchase network helps predict new product recommendations. Similarly, in bioinformatics, graphs representing protein-protein interactions or gene relationships enable predictions about biological functions or disease pathways. Knowledge graphs, which structure information in interconnected formats, help machines reason over relationships, such as identifying entity connections for natural language processing tasks.

To leverage the full potential of graphs, several advanced techniques are employed. Centrality measures, for instance, quantify the importance of nodes in a graph. Degree centrality counts direct connections, while betweenness centrality identifies nodes bridging clusters. These measures are critical for tasks like identifying influencers or analyzing communication networks. Graph embeddings, such as Node2Vec or DeepWalk, map graph structures into continuous vector spaces, making them compatible with machine learning models [2][3]. Additionally, Graph Neural Networks (GNNs), like Graph Convolutional Networks (GCNs), aggregate information from neighboring nodes. These networks excel in tasks such as node classification, where labels are assigned to nodes (e.g., identifying spam accounts), and link prediction, which predicts relationships between nodes, such as friendships in social networks.

Despite their advantages, graph-based feature engineering comes with challenges. Large-scale graphs can be resource-intensive, requiring efficient algorithms like graph sampling or distributed computing frameworks to manage their computational costs. Sparse graphs with limited connections can also hinder meaningful feature extraction, making advanced techniques like graph regularization essential. Addressing these challenges is critical to fully harness the potential of graph-based methods and create robust machine learning models.

Graph-based feature engineering is revolutionizing machine learning by enabling us to capture relationships and dependencies within data. From refining recommendation systems to advancing healthcare predictions, graph-based approaches pave the way for deeper, more accurate insights in an interconnected world. As machine learning continues to evolve, the potential of graph-based methods will only grow, offering exciting opportunities for innovation. I’m ultimately interested in how knowledge ends up getting stored and represented moving forward within the context of exceedingly large language models.

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=Graph-Based+Feature+Engineering&btnG=

[2] https://arxiv.org/pdf/1607.00653

[3] https://arxiv.org/pdf/1609.02907

What’s next for The Lindahl Letter?

Week 172: Transfer Learning for Features
Week 173: nondeterministic gates tolerant quantum computation
Week 174: error correction tolerant quantum computing
Week 175: universal quantum computation
Week 176: resilient quantum computation

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to this audio only podcast presentation. This is week 170 of The Lindahl Letter publication. A new edition arrives hopefully every Friday. This week the topic under consideration for The Lindahl Letter is, “Are 8K Blu-ray a thing?”

The short answer is no—there’s currently no official 8K Blu-ray format on the market. The highest-resolution Blu-ray available right now is 4K Ultra HD. Despite the emergence of 8K TVs, the development of an 8K physical media standard has been slow to nonexistent. You can generate content at 8K using 65/70mm IMAX film scans or recording the content using native 8K RED cameras. But there’s more to this story, because what’s holding back 8K Blu-ray isn’t just a lack of demand for higher resolutions. It’s about the larger shift in how we consume media, the infrastructure needed to support it, and even questions of accessibility and ownership.

The dominance of streaming has completely changed the landscape of home entertainment. Most people today reach for their remote or phone, pull up a streaming app, and press play, accessing a vast library of content without needing physical discs. And while it’s convenient, streaming isn’t the perfect solution for everyone, and it raises some interesting challenges for high-resolution content. The reality is, even today, reliable 4K streaming requires a fast and stable internet connection—something many regions in the world, including parts of the United States, still struggle with.

For people in areas with slower or less reliable internet, streaming high-definition content, let alone 4K or 8K, isn’t an option. This digital divide is often overlooked in the rush to adopt the newest formats and streaming platforms. A physical 8K Blu-ray option, although niche, would offer these users a way to access ultra-high-definition content without relying on the vagaries of internet service. Physical media doesn’t buffer or depend on bandwidth. It’s a permanent, reliable way to enjoy high-quality media.

Another issue that streaming raises is the matter of ownership. When you buy a Blu-ray disc, you own a copy of that film or show—something tangible that you can keep, loan, or sell. With streaming, you’re essentially renting access to content. Licensing agreements and platform decisions dictate what’s available, and content can disappear from a service overnight due to contract disputes or shifting corporate strategies. Even if you purchase a digital copy, the platform still controls your access to it, and it could be removed or rendered inaccessible if the platform decides to remove it or goes under. We’ve already seen titles vanish from digital libraries, leaving consumers who thought they “owned” these digital copies with no recourse.

For film enthusiasts, collectors, or anyone who values the security of owning their media outright, physical Blu-rays still hold a lot of appeal. An 8K Blu-ray, in particular, would give these users a chance to own ultra-high-definition content at its absolute best quality. Streaming platforms, while convenient, can’t match the fidelity of a physical disc, especially when it comes to uncompressed audio and video quality. And for those who value the archival aspect of physical media, 8K Blu-ray would represent a way to preserve the best possible version of their favorite films and shows.

Yet, despite these potential advantages, the market for physical media has become niche. Blu-ray players are harder to come by, with fewer manufacturers making them each year, and studios are releasing fewer physical editions. Streaming is simply more profitable and cost-effective for companies, and it aligns with current consumer habits. There’s also the fact that creating a new standard for 8K Blu-ray would require a significant investment in technology, from new players to new discs, and that investment likely wouldn’t be recouped given current market conditions.

In the meantime, tech companies are focusing on improving streaming infrastructure to support 8K content. Compression algorithms are advancing, and AI-powered upscaling technologies are making it possible for 4K content to look sharper on 8K screens, even if it’s not natively 8K. This makes it unlikely that we’ll see a mass-market push for 8K Blu-ray anytime soon [1][2]. It’s possible that high-quality 8K streaming will fill that void, but it’s a solution that still doesn’t serve everyone equally.

So, are 8K Blu-rays a thing? Not at this point, and they may never become mainstream. But as we move toward a fully digital media landscape, we should keep in mind what’s lost when physical formats disappear: ownership, access for all, and the assurance that our favorite content won’t vanish overnight.

Thank you for joining me for this week’s discussion. Until next week, let’s keep asking what the future of media really means for us all.

Things to consider this week:

“Monster 4,400-qubit quantum processor is '25,000 times faster' than its predecessor”https://www.livescience.com/technology/computing/monster-4-400-qubit-quantum-processor-is-25-000-times-faster-than-its-predecessor

TechCrunch: “Microsoft and Atom Computing will launch a commercial quantum computer in 2025” https://techcrunch.com/2024/11/19/microsoft-and-atom-computing-will-launch-a-commercial-quantum-computer-in-2025/

“Physicists Transformed a Quantum Computer Into a Time Crystal” https://www.sciencealert.com/physicists-transformed-a-quantum-computer-into-a-time-crystal

Footnotes:

[1] https://www.homecinemachoice.com/content/no-8k-upgrade-blu-ray-admits-8k-association

[2] https://8kassociation.com/

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

Thank you for tuning in to this audio only podcast presentation. Here we are, at week 169 of The Lindahl Letter, reflecting on a different kind of science—one rooted in a boldness that prioritized discovery over deadlines, adventure over immediate outcomes. This week’s topic, “Pure science from the last millennium,” brings us to a fundamental question: What do we want scientific investment to be going forward?

Decades ago, space exploration wasn’t about quarterly returns or brand endorsements. It was the “final frontier”—a true and meaningful challenge that demanded our collective curiosity and belief in something larger than ourselves. When we look back at Voyager 1, launched in September of 1977 and still sending back data from beyond the solar system, we’re reminded of an era when we were willing to invest in the unknown and pure science [1]. We, the taxpayers, poured resources into pure science, trusting that whatever Voyager discovered would expand our horizons, even if it took generations. Those investments bank intergenerational equality in ways that pay forward with unlocked potential.

That previous era of government driven budgets is evolving. Today, we’re witnessing a new chapter in space exploration, one driven not only by government agencies but by private space companies backed by some very rich individuals. Companies like SpaceX, Blue Origin, and others are racing to develop technologies that can propel humanity forward, not just in the pursuit of knowledge but with a practical eye on commercial possibilities. These companies have reignited public interest in space exploration, capturing imaginations with promises of lunar bases, Mars colonies, and low-cost satellites. But they bring a shift in perspective too—a focus on efficiency, profitability, and measurable results.

Private companies are undeniably accelerating technological progress. They’re launching rockets at a pace governments could never match and making space travel more accessible. In many ways, they’re pushing us into the future faster than traditional models of science funding would allow. But this pace has implications: private companies often operate on a very different timeline and set of incentives than the public missions of the past. The long, open-ended pure science based inquiries that characterized projects like Voyager or Hubble might not fit as seamlessly into the bottom-line-driven model of private enterprise.

What does this mean for the future of pure science? There’s a risk that in our rush to commercialize space, we could lose sight of the kind of exploration that doesn’t pay off right away, the kind that asks questions not because they’re immediately useful but because they might change everything someday. Voyager, Hubble, and the Mars rovers were funded with a faith that curiosity itself was valuable. They didn’t need to deliver a profit; they only needed to expand our knowledge.

Investment in pure science has, over the years, shifted in response to economic pressures, political priorities, and the rise of private industry. In the mid-20th century, there was a golden age of public funding for fundamental research, driven by a sense of national pride and urgency, especially during the Space Race. Governments around the world poured money into science for the sake of knowledge itself—driven by the belief that scientific exploration, even with uncertain outcomes, would ultimately benefit society. This mindset fueled projects like the Apollo missions, the Voyager probes, and the Hubble Space Telescope, all examples of pure scientific research where the primary goal was exploration, not commercial gain.

But over the last few decades, the focus has gradually shifted toward more immediate, application-driven science. Public budgets have tightened, and government funding has increasingly emphasized practical and commercial outcomes. Today, many funding bodies expect quick, measurable results—preferably ones that contribute to the economy, healthcare, or national security. This shift means that pure scientific research, with its inherently uncertain timeline and lack of immediate commercial payoff, often struggles to secure the same level of investment it once enjoyed.

Thank you for joining me this week, and here’s to staying curious, even in a world that asks us to measure every journey in miles and profits.

Things to consider this week:

TechCrunch: “Bluesky raises $15M Series A, plans to launch subscriptions”https://techcrunch.com/2024/10/24/bluesky-raises-15m-series-a-plans-to-launch-subscriptions/

Reuters: “New Nvidia AI chips overheating in servers, the Information reports”https://finance.yahoo.com/news/nvidia-ai-chips-face-issue-141200900.html

[Must watch] Gary Marcus: OpenAI could be the next WeWork https://www.foxbusiness.com/video/6364719527112

Footnotes:

[1] https://www.cnn.com/2024/11/01/science/voyager-1-transmitter-issue/index.html

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.nelsx.com

View Details

We all thought you would be able to easily ask the house to turn on all the lights or command your television by voice to do things by walking into the room. Some of us thought the entire wall would be a television screen by now and that has not happened either. However, some of the new 100 inch TVs on the market are really large. Enabling automated actions is what is happening within the latest development kits related to the companies making and contributing to LLMs. We have seen Google teams introduce low stakes automated actions like making dinner reservations or screening calls. Those types of training activities help them build and reinforce automations without really doing anything particularly risky. It’s all about the ecosystems and when Google teams start to really deeply allow an assistant based agent to do things that are deeply integrated at that point things are going to rapidly change. That is when your agent will be empowered to the point of being able to really automate some things that will be impactful. Sam Altman of OpenAI has said that 2025 will be the year that we will see agents working effectively [1]. I’m guessing that Sam has spent some time thinking deeply about what these agents are going to be capable of doing.

We will probably start to see Google calendar automations where meetings with unfulfilled action items automatically get scheduled or task follow ups by chat can come from the agent. This type of recursive review of things that happened where a transcript is recorded and checked against a project plan or calendar is certainly on the roadmap. It’s going to be about bringing the next set of low stakes actions to the business world and calling it revolutionary. A lot of hype is going to occur. Sure systems with robotic process automation or coded workflows have been able to automate things for people willing to invest in those automations. With the advent of agents that are able to schedule automated actions it changes the barrier to entry by fundamentally lowering it. People are probably going to be more willing to trust one of the known major brands with this technology considering that most smartphones have banking information saved and are logged into a myriad of other consequential accounts. Having practical limits on what agents are able to enable in terms of automation remains probably the most important process enablement gate to be considered. Apparently the teams at Google don’t expect to deploy any useful agents until 2025 at the earliest [2].

The push toward true agent-based automation is an ongoing journey. While current tools may seem like small, incremental advances—like handling calendar follow-ups or screening calls—they represent foundational steps toward a more integrated, intuitive digital ecosystem. As AI agents begin to bridge the gap between simple command-driven functions and context-aware actions, we’re stepping into an era where automation isn't just a convenience but a fundamental part of daily life. This gradual transformation will bring more impactful applications, positioning agents not as isolated tools but as active partners in our productivity.

Footnotes:

[1] https://www.tomsguide.com/ai/chatgpt/the-agents-are-coming-openai-confirms-ai-will-work-without-humans-in-2025

[2] https://techcrunch.com/2024/10/29/google-says-its-next-gen-ai-agents-wont-launch-until-2025-at-the-earliest/

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

This post is all Dr. Flint Dibble’s fault. That fault is wholesale based on all the #realarchaeology posts that have been flying around the internet [1]. If you missed all that online content about archaeology, then consider diving into all that informationally rich real archeology content. This week Flint got me thinking about how artificial intelligence and machine learning fit within the broader academic domain of archaeology. Technology is always approaching the intersection with modernity. Our technology now is ultimately becoming very different based on what the fields of AI, ML, and robotics are able to accomplish. I should have thrown quantum computing on that list, but I’m still a little skeptical about scalability.

A lot of modern archaeology on television is about the discovery part of the process. Finding things like buried treasure, missing cities, or maybe a significant shipwreck yields pure excitement. Just this week I watched an episode of the ongoing television show Expedition Unknown with Josh Gates digging at Petra [2]. You may be aware that I’m generally interested in all things Indiana Jones related and this adventure certainly was. Josh Gates joined Dr. Pearce Paul Creasman onsite for the discovery and the American Center of Research is the group facilitating the actual archaeology.

Generally speaking, finding new things is hard, but the process of trying to understand them is where the work of archaeology happens. Applying some type of scientific rigor to the process of figuring things out brings forward quality and makes the process definable and repeatable. We have done a lot of exploring and studying the world with satellites every day. A lot of laboratory, office, or digging work happens that is more hands-on and is about the academic parts of archaeology. That is where I was curious about how both AI and ML fit into the actual practice of archaeology. I wondered what people are doing with advanced technology. I could easily imagine people trying a machine learning model to evaluate satellite images to try to find structures in a jungle or desert. You could use a machine learning model to match images of text fragments or match a partial text to other larger texts.

I’m going to share my top 10 thoughts about how AI or ML could be impactful within the field of archeology.

Automated Site Detection: Using AI to analyze satellite images and locate hidden archaeological sites
Predictive Modeling: Guided site discovery by predicting likely artifact locations from geological and historical data
Excavation Data Analysis: Speeding up artifact categorization and soil dating during digs
3D Reconstruction: Rebuilding artifacts or sites digitally to visualize original structures
Text Decipherment: Using AI to decode ancient texts and connect languages or symbols
Remote Sensing Interpretation: Processing LiDAR and radar data to reveal hidden structures
Artifact Classification: Identifying and classifying artifacts using computer vision
Preservation Monitoring: Predicting and preventing environmental damage to sites
Cross-dataset Analysis: Finding patterns across separate data sources for deeper historical connections
Virtual Archaeology and Immersive Experiences: AI can create virtual reality (VR) and augmented reality (AR) experiences, allowing researchers and the public to explore reconstructed sites and artifacts interactively

Beyond considering that list, you know I went out to Google Scholar and started to look for highly cited papers within this space [3]. I pulled together 5 papers you can read about AI and ML within the academic space of archaeology. None of these papers have very high citation numbers so they are not widely read like those papers I recently shared from Dr. Geoffrey Hinton that bridged 100,000 citations. These papers were in the sub 100 citation range and that does mean people are reading them, but not at the same prolific rates of core AI or ML papers.

Bickler, S. H. (2021). Machine learning arrives in archaeology. Advances in Archaeological Practice, 9(2), 186-191. https://www.researchgate.net/profile/Simon-Bickler/publication/351713328_Machine_Learning_Arrives_in_Archaeology/links/60a62e36a6fdcc731d3ea200/Machine-Learning-Arrives-in-Archaeology.pdf

Barceló, J. A. (2007). Automatic archaeology: Bridging the gap between virtual reality, artificial intelligence, and archaeology. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=9f03a88221022b93d07f271037776a2c01099fce

Mantovan, L., & Nanni, L. (2020). The computerization of archaeology: Survey on artificial intelligence techniques. SN Computer Science, 1(5), 267. https://arxiv.org/pdf/2005.02863

Casini, L., Marchetti, N., Montanucci, A., Orrù, V., & Roccetti, M. (2023). A human–AI collaboration workflow for archaeological sites detection. Scientific Reports, 13(1), 8699. https://www.nature.com/articles/s41598-023-36015-5

Argyrou, A., & Agapiou, A. (2022). A review of artificial intelligence and remote sensing for archaeological research. Remote Sensing, 14(23), 6000. https://www.mdpi.com/2072-4292/14/23/6000

The broader implications of how AI is reshaping archaeology’s future are currently unfolding. What we’re seeing now is that technology is actively transforming archaeology. As AI helps detect patterns and analyze data in ways humans simply can’t, it’s redefining the possibilities within this field. AI isn’t here to replace the archaeologist’s trowel but rather to enhance their insights. It’s an assistive tool, an enabler, expanding the reach and depth of what we can know about our past. And that’s exactly why this conversation about AI in archaeology matters—it’s the next step in understanding not just what we’ve found but, ultimately, what it means.

Footnotes:

[1] https://real-archaeology.com/

[2] https://www.cnn.com/2024/10/12/science/petra-tomb-indiana-jones-discovery/index.html

[3] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=artificial+intelligence+archeology&btnG=

What’s next for The Lindahl Letter? New editions arrive every Friday. If you are still listening at this point and enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. Make sure to stay curious, stay informed, and enjoy the week ahead!

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Thank you for tuning in to this audio only podcast presentation. This is week 166 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “Those recent Nobel Prizes.”

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

For those of you listening to the audio version this week you may have noticed that we moved from the natural vocal capture to the narration vocal capture. I’m still recording the podcast audio for this adventure using a Blue Microphones Yeti X and my MacBook Air with the included GarageBand software. To further enhance your listening experience I’m still working to dial in the best possible recording technique to deliver superior podcast audio.

Last week we really dug deep into the major corporate players that released quantum computing programing languages for general use. This week unfortunately needs to start with an epic spoiler alert. Please know that you should be aware that it does not appear quantum computing is practical at this time or really very scalable. I have read a fair number of jokes throughout the last week about neither the blockchain or quantum computer being scalable. With that spoiler delivered upfront this week it is time to move from the breadth of coverage to the depth of understanding related to what people are actually doing with practical use cases within the quantum computing space. One of the places I went to search around and learn a little bit more about use cases was the NASA Quantum Artificial Intelligence Laboratory [1]. The good folks over at NASA shared a paper in June of 2024 that was about “Assessing and Advancing the Potential of Quantum Computing: A NASA Case Study” [2]. For those of you wanting to read more quantum computing papers this is pretty easy to read and digest. It’s 27 pages and is very well cited throughout.

You can join the quantum computing reddit community and it seems to be pretty active with academic discussion amongst the 50k+ members [3]. Beyond that reddit community the next place I ended up spending some time looking around was the Quantum Open Source Foundation which had a lot of content in terms of links [4]. They have a lot of curated links on the GitHub page that they maintain [5]. You could spend hours and hours of time just clicking around and looking at all of those projects. Eventually I ran into another GitHub repository called Awesome Quantum Computing that is another collection of curated links [6]. These collections of links will send you all over the place to see some interesting projects people are developing.

Somehow during the hunt for the best projects using quantum computing to accomplish things I ended up back looking at the IBM Quantum Learning pages to see what things they were encouraging people to code as they learn to program [7]. The Azure Quantum team had a whole section devoted to trying to explain what solutions they are offering [8]. A lot of that seems to be focused on physics, chemistry, and ultimately material discovery and other applied applications to understand some type of complex interaction. Modeling really complex things seems to be a core use case that quantum computing has centered on based on the available evidence. I really think at this point I’m going to invest some time into completing a couple of these courses to get more hands on in the quantum computing space. I’ll share one last note about the fastest quantum computer that now has 1,180 qubits from Atom Computing [9]. It was a sizable leap from the IBM’s Osprey that was capable of 433 qubits. Hopefully we will see a bunch of fastest quantum computer records broken in the coming years. That will be a good sign that things have forward momentum in the quantum computing space.

Things to consider this week:

You might want to read the 15,000 word October 2024 essay from Anthropic CEO Dario Amodei called “Machines of loving grace.” https://darioamodei.com/machines-of-loving-grace
The Anthropic team also released an updated responsible scaling policy that is a related read with the Dario Amodei essay https://www.anthropic.com/news/announcing-our-updated-responsible-scaling-policy
I enjoyed listening to Yannic Kilcher talk about and question a paper this week “GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models” https://arxiv.org/abs/2410.05229

Footnotes:

[1] https://www.nasa.gov/intelligent-systems-division/discovery-and-systems-health/nasa-quail/

[2] https://arxiv.org/abs/2406.15601

[3] https://www.reddit.com/r/QuantumComputing/

[4] https://qosf.org/project_list/

[5] https://github.com/qosf/awesome-quantum-software

[6] https://github.com/desireevl/awesome-quantum-computing

[7]

https://learning.quantum.ibm.com/

[8] https://quantum.microsoft.com/en-us/solutions/azure-quantum-solutions

[9] https://www.newscientist.com/article/2399246-record-breaking-quantum-computer-has-more-than-1000-qubits/

What’s next for The Lindahl Letter? At some point this series will move back to being planned out 5 weeks ahead of publication.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Over the last week I started digging into quantum computer programming to see what is currently going on and what people are doing in that space [1][2]. General availability of actual quantum computing systems is the major barrier to using one to write some code [3]. You can buy time on a quantum computer or you can use a simulator. The simulators are sort of weird in general as the computing power you are using to simulate the quantum computer has a very small fractional power equivalent in terms of computing power. The biggest thing to really consider in terms of understanding where we are in terms of quantum computing is not cost or availability, but instead its a true question about the computing method itself related to having fully fault-tolerant quantum computing. Not only do you have to have methods for error correction within your quantum computing setup, but also you need a design that has proper fault-tolerant gates to avoid introducing increasing error levels within your computing. You are probably wondering if I went out to Google Scholar to find papers about fault-tolerance in quantum computing [4]. Of course that was where I went to look for papers. Here are 3 academic papers you can read with over 300 citations to consider:

Steane, A. M. (1999). Efficient fault-tolerant quantum computing. Nature, 399(6732), 124-126. https://arxiv.org/pdf/quant-ph/9809054

Chow, J. M., Gambetta, J. M., Magesan, E., Abraham, D. W., Cross, A. W., Johnson, B. R., ... & Steffen, M. (2014). Implementing a strand of a scalable fault-tolerant quantum computing fabric. Nature communications, 5(1), 4015. https://www.nature.com/articles/ncomms5015

Preskill, J. (1998). Fault-tolerant quantum computation. In Introduction to quantum computation and information (pp. 213-269). https://arxiv.org/pdf/quant-ph/9712048

Let’s set aside those questions about making quantum computing sustainable and consider the code part of the equation. Major programming languages for quantum computing do exist from the players you would expect. They tend to have a lot of documentation and GitHub profiles to share the code. You could spend whole days looking at some of the languages made by major players. Here are 3 examples of major players making quantum computing language contributions. First, Qiskit was introduced as a quantum computing programming language by IBM teams back in 2017 [5][6]. Second, teams at Microsoft introduced Q# back in 2017 [7][8]. Third, Cirq is from Google AI and was introduced in 2018 [9][10]. You can start to dig into those code bases and I could find a lot of content related to the languages and a lot of hype about quantum computing.

You can see that a lot of evidence exists related to quantum programming languages. The next logical questions would be how do they execute that code in practice and maybe what exactly are they doing with this quantum code. The teams at IBM have been pretty good about sharing plans to build faster and faster quantum computers [11]. You could watch the hype film about the IBM Quantum System Two that they shared to YouTube back on December 4, 2023. It’s always interesting to look at quantum computer builds; they are not the sort of thing that is going to sit under my desk or on my desk at this point within the technology curve.

After all that foundational digging into the current state of quantum computing my thoughts are still conflicted about it. I know some questions exist about exactly how to deploy large scale efforts that work without propagating errors and the complexity of fault-tolerant gates. Probably the next step in my research process will be to find some examples of solid quantum computing code being used or actually deployed. That is maybe the best way to get a sense of what is being done within the quantum computing space. That research note about practical quantum computing and getting things done will probably shed some light on where things are trending. Beyond people setting records for the fastest machine or sharing hype videos the real deeper question here is what use cases are going to end up defining the technology.

Footnotes:

[1] https://towardsdatascience.com/an-introduction-to-quantum-computers-and-quantum-coding-e5954f5a0415

[2] https://www.mckinsey.com/featured-insights/mckinsey-explainers/what-is-quantum-computing

[3] https://github.com/qosf/awesome-quantum-software

[4] A Google Scholar search for “fully fault-tolerant quantum computing” https://scholar.google.com/scholar?q=fully+fault-tolerant+quantum+computing&hl=en&as_sdt=0&as_vis=1&oi=scholart

[5] https://www.ibm.com/quantum/qiskit

[6] https://github.com/qiskit

[7] https://learn.microsoft.com/en-us/azure/quantum/qsharp-overview

[8] https://github.com/microsoft/qsharp

[9] https://quantumai.google/cirq

[10] https://github.com/quantumlib/Cirq

[11] https://www.fastcompany.com/90992708/ibm-quantum-system-two

What’s next for The Lindahl Letter? At some point this series will move back to being planned out 5 weeks ahead of publication. That point has not arrived just yet so you can expect a more fluid selection of topic coverage.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

We are facing a new reality where the tipping point has been passed and a majority of content being generated online within the internet is now synthetic. Organic content generation simply cannot keep up with the flood of synthetic content. Those bot farms creating content never sleep. They just churn out content and pretend that it remains evergreen. My consideration on this topic started back on Wednesday, July 31, 2024, when I was invited to access SearchGPT from OpenAI [1]. Using that platform made me think a lot about how we access information and the ways that will change going forward. People are now getting summaries and completing searches that go beyond Googling something. You are probably well aware by now that I’m deeply concerned about how facts and knowledge are going to be stored and curated going forward.

Going forward whoever owns the stores of facts or knowledge will effectively own history and how it is presented which is truly a watershed change in our shared understanding of the world. Individual voices and publications will be overshadowed by these collections. Owning the datastores that provide definitive facts or knowledge will be the cornerstone of whatever emerges going forward and should not be underestimated in terms of future value. I don’t think the ownership of facts will become commoditized and open sourced. I really do think it will be privatized and tightly controlled. Somebody who wanted to pivot our understanding on a particular point of inquiry could just start serving up that alternative perspective. Instead of people funding think tanks to ultimately change the messaging the next step will be funding content farms to just flood the message delivery. Keep in mind that people generally are not really reading books anymore [2]. That means that reasoning during the course of interpreting information may be a diminishing skillset.

Organically written original content exists online. Synthetically generated content has been on the rise. A lot of bots are scraping content for model training and trying to figure out what is organic and what is synthetic has become increasingly difficult. One of the hallmarks of my writing efforts has been the originality or maybe novelty of my research efforts. Within the broader context of the academy of academic thought, original contributions are what build that content and strengthen it overall. Diluting, derivative, and otherwise mediocre publications just flood the overall academic community. It’s perfectly fine to write a publication and decide it was not a significant contribution. Instead of maybe holding back those lesser works they are now freely shared in online archives and unfortunately a new generation of journals. The increase in AI related publications has been astonishing from 2010 to 2022 the number of publications nearly tripled [3].

Now we have a mix of the poorly written articles mixing with the synthetically generated to create a truly problematic future of consuming content. I’m considering web traffic at the moment, but the overall storage of facts and knowledge is certainly in scope. We reached the tipping point around 2016 where more traffic is mobile traffic than from a desktop browser [4]. OpenAI has now launched SearchGPT and beyond the dichotomy between mobile and desktop traffic we are about to see the rise of LLM interpreted results where people may never actually leave the landing page or interface of the search engine. It’s possible that dichotomy will fade away and the majority of traffic will be from bots scraping things to share within the newly powered search interfaces. People may very well interact with the grand volume of online information from applications using APIs to respond that are completely disconnected from what was the open internet people surfed and experienced. From reading the thoughts of a single writer to interpreting the output of the largest language models ever created. Things are changing at an incredibly rapid pace.

Now it’s time for a brief editorial note. Please note that my writing output over the last few years became over indexed on artificial intelligence and machine learning. Going down that rabbit hole was good at first and it was an effort truly focused on depth and breadth within the subject. Unfortunately, my focus lingered and instead of writing research notes about technological innovation, civil society, and the intersection of technology and modernity that pesky over indexing occurred. Now thanks to a moment of reflective practitioning I’m breaking out of that pattern and returning to what I consider a better balance of writing topics. Thank you for being along for that journey and the upcoming course correction.

Footnotes:

[1] https://chatgpt.com/search

[2] https://www.pewresearch.org/short-reads/2021/09/21/who-doesnt-read-books-in-america/

[3] https://arxiv.org/abs/2405.19522 or https://aiindex.stanford.edu/wp-content/uploads/2024/04/HAI_AI-Index-Report-2024_Chapter1.pdf

[4] “>50% of web traffic comes from mobile.” Google Analytics Data, U.S., Q1 2016. https://www.thinkwithgoogle.com/marketing-strategies/app-and-mobile/mobile-web-traffic-statistics/

What’s next for The Lindahl Letter? I’m just going to sit down and write weekly or more likely bi-weekly so the topic won’t be planned out 5 weeks ahead of publication.

If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. Stay curious, stay informed, and enjoy the week ahead!

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

In the digital age, the methods we use to organize and comprehend information are continually evolving. Two significant approaches stand out: indexing facts and graphing knowledge. Both play essential roles in how we structure, retrieve, and understand data, but they serve distinct purposes and offer different advantages.

Let’s start out by looking a little deeper into indexing facts. Indexing is a traditional and straightforward method. It involves categorizing and listing information in a manner that allows for easy retrieval. Think of it as a library catalog, where every book has a unique identifier and is placed in a specific location based on its subject. This system is incredibly efficient for finding discrete pieces of information quickly. For example, a keyword search in a database relies heavily on indexing.

Indexes are foundational to databases and search engines. They allow us to locate specific data points without having to sift through every piece of information manually. This method is highly effective for tasks that require precision and speed. However, indexing has its limitations. It often lacks context and relational understanding between different pieces of data. An index can tell you where something is but not necessarily how it connects to other information.

Graphing knowledge, on the other hand, is about mapping relationships between data points. This approach is exemplified by knowledge graphs, which visually represent the connections between different concepts. A knowledge graph is more than a mere collection of facts; it is an interconnected web that shows how different pieces of information relate to one another.

In a knowledge graph, nodes represent entities (such as people, places, or concepts), and edges represent the relationships between these entities. This structure allows for a more holistic understanding of information. For instance, a knowledge graph can illustrate how historical events are connected, how scientific concepts overlap, or how social networks operate.

The advantages of graphing knowledge are manifold. It provides context, reveals patterns, and helps in discovering new insights that might not be apparent through traditional indexing. Knowledge graphs are particularly useful in fields that require deep understanding and analysis, such as artificial intelligence, semantic web technologies, and complex decision-making processes.

The intersection of indexing facts and graphing knowledge represents the future of information management. By combining the precision of indexing with the relational depth of knowledge graphs, we can create systems that are both efficient and insightful. This hybrid approach can enhance our ability to process and understand vast amounts of data, making it possible to derive meaningful insights quickly.

For example, search engines are evolving to incorporate elements of both indexing and graphing. Google's Knowledge Graph is a prime example, aiming to understand the context behind search queries to provide more relevant results. This system not only indexes web pages but also understands the relationships between different pieces of information, delivering a more nuanced response to users' queries.

Indexing facts and graphing knowledge are not mutually exclusive; they are complementary methods that, when combined, can significantly enhance our understanding and management of information. By leveraging the strengths of both approaches, we can build more robust systems that not only store and retrieve data efficiently but also provide deeper insights and understanding. As we continue to advance in the digital age, the fusion of these methods will undoubtedly play a pivotal role in shaping the future of information technology and knowledge management.

What’s next for The Lindahl Letter?

Week 163: Self-Supervised Learning
Week 164: Graph-Based Feature Engineering
Week 165: Federated Feature Engineering
Week 166: Explainable Feature Engineering
Week 167: Adaptive Feature Engineering

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

You never really have to worry about how you store knowledge. All that knowledge just gets accumulated day by day and how it is stored, structured, and even retrieved happens without any intervention. It’s just something that you do autonomously. That is very different when you have to manage all that data and store it. Navigating the intricate maze of data science, structuring really large knowledge graphs presents itself as both an art and a science. This endeavor, crucial for deepening our understanding and utilization of complex datasets, entails a series of pivotal steps and considerations. Today, we'll explore the foundational principles and practical strategies for effectively structuring these expansive networks of interconnected information, while drawing on the organizational wisdom of traditional knowledge structures like the Dewey Decimal System (DDS).

Understanding the basics seems to take more and more time these days. Knowledge graphs are more than mere data structures; they represent information through a web of entities and their interrelations. They offer a robust framework for integrating data from a multitude of sources, enhancing our ability to derive richer insights and make more informed decisions. As the size of the knowledge graph expands, so too does the complexity of structuring it effectively.

Key Components of Knowledge Graphs

Nodes and Edges: At the heart of any knowledge graph are nodes (entities) and edges (relationships). Nodes can embody concepts, objects, or events, while edges illustrate the interconnections among these nodes. The quality and comprehensiveness of your knowledge graph hinge on the precise definition and linking of these elements. You deal with these types of relationships every day without even an afterthought. The types of things you manage passively are more complex when they have to be handled in a planful way.
Ontology: This serves as the schema or structural framework that delineates the types of entities and relationships within the graph. A well-crafted ontology ensures consistency and coherence, enabling more effective querying and analysis. I’m actually a fan of declaring things as a fact or not a fact and then storing those facts in buckets that are easy to retrieve.
Data Ingestion and Integration: Large knowledge graphs often amalgamate data from various sources. Efficiently integrating this data while preserving its integrity and relevance is a critical challenge, involving data cleaning, normalization, and transformation.

Knowledge Structures and the Dewey Decimal System

Drawing inspiration from the Dewey Decimal System can provide valuable insights into structuring knowledge graphs. The DDS organizes information into a hierarchical, decimal-based classification system, which can serve as a model for categorizing and indexing data within a knowledge graph.

Hierarchical Classification: Like the DDS, hierarchical classification in a knowledge graph helps organize information into broad categories and narrower subcategories. This ensures that related entities are grouped together, facilitating easier navigation and retrieval.
Decimal Notation: Utilizing a decimal notation system to categorize entities and relationships can add a layer of precision and order to a knowledge graph. Each node and edge can be assigned a unique identifier, akin to how books are classified in libraries.
Subject Headings: Implementing subject headings, similar to those in the DDS, can aid in tagging and describing nodes with relevant keywords. This enhances the searchability and contextual understanding of the graph.

Strategies for Structuring Large Knowledge Graphs

Scalability: Ensure your infrastructure can handle the increasing volume of data. This often involves distributed computing and storage solutions, such as cloud-based platforms that can scale horizontally.
Data Modeling: Design your data model with future growth in mind. Anticipate new types of entities and relationships, ensuring that the graph can evolve without significant restructuring.
Indexing and Partitioning: Use indexing to speed up queries and improve performance. Partitioning the graph into manageable sub-graphs can also enhance efficiency, especially when dealing with very large datasets.
Query Optimization: Develop efficient query strategies to handle complex searches. This might involve using specialized query languages like SPARQL or leveraging graph database technologies that support high-performance querying.
Visualization and Interaction: For large knowledge graphs, visualization tools are invaluable. They help in understanding the structure and relationships within the graph, making it easier to navigate and extract insights.

Tools and Technologies

Several technologies and tools are pivotal in constructing and managing large knowledge graphs:

Graph Databases: Neo4j, Amazon Neptune, and ArangoDB are designed to handle large-scale knowledge graphs, offering robust querying capabilities and efficient management of complex relationships.
Data Integration Tools: Tools like Apache NiFi and Talend facilitate seamless ingestion and integration of data from various sources into your knowledge graph.
Ontology Management: Protégé and TopBraid Composer are popular tools for creating and managing ontologies, ensuring your knowledge graph has a solid structural foundation [1].

Conclusion

Structuring really large knowledge graphs is a multifaceted challenge that requires careful planning, robust technology, and a strategic approach to data integration and management. By focusing on scalability, data modeling, and efficient querying, and drawing inspiration from traditional knowledge structures like the Dewey Decimal System, we can harness the full potential of knowledge graphs to drive innovation and insight across various fields. As we continue to refine these processes, the power and utility of knowledge graphs will only grow, unlocking new opportunities for understanding and leveraging the vast sea of data that defines our world.

In the spirit of continuous improvement and adaptation, our journey with knowledge graphs reflects the age-old pursuit of knowledge itself—ever-evolving, always striving for greater understanding.

Footnotes:

[1] https://protege.stanford.edu/

What’s next for The Lindahl Letter?

Week 162: Indexing facts vs. graphing knowledge
Week 163: Self-Supervised Learning
Week 164: Graph-Based Feature Engineering
Week 165: Federated Feature Engineering
Week 166: Explainable Feature Engineering

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Over the years the team over at Google has made a really big knowledge graph that you can access via an API and that they use as an informational backbone [1]. In some ways it is the best of what the old web had to offer. They note that it is a database of billions of facts. We are starting to see the creation of just a ton of middling, mediocre, or otherwise terribly written content online [2][3]. Now imagine you had built a knowledge graph of billions of facts. You can’t stop updating that large of a knowledge graph. It would grow stale so quickly with how fast the intersection of technology and modernity is occurring. Let me say that another way you now face a situation where a great flooding of bad content is going to overwhelm your knowledge graph. Yeah a tsunami of imagined information and otherwise hallucinated content is going to destabilize the integration of that knowledge graph. Even the notion that it would be built on facts begins to fade away as a sea of LLMs spit out confusion in the form of very confidently written fabrication.

I’m now going to dig into the world of thought related to combining or using in concert LLMs and knowledge graphs. Probably the most interesting breakdown for the future of knowledge graphs will be proprietary locked in ones vs. the decentralized knowledge graphs that could even be powered by a blockchain [4]. We are going to see a huge battle between decentralized knowledge graphs that maybe use even a federated approach to stay fresh and the near monolithic large knowledge graphs that individual corporations are trying to keep perpetually fresh. One paper dealing with the combination of LLMs and knowledge graphs would be, “Large Language Models and Knowledge Graphs: Opportunities and Challenges,” that was published back in 2023 [5]. Another paper from 2023 would be, “Connecting AI: Merging Large Language Models and Knowledge Graph,” that is generally covering the same conceptual landscape [6].

Some people are starting to make arguments that maybe the internet is really starting to break. The internet, once hailed as a beacon of boundless opportunity, now finds itself at a precarious crossroads clouded by mounting concerns. As behemoth tech entities tighten their grip, questions of control and influence darken the digital horizon. Privacy breaches and the insidious spread of misinformation cast a long shadow over its once-promising landscape. Algorithms meant to connect have inadvertently fueled division, while the exploitation of personal data raises profound ethical quandaries. Meanwhile, the internet's infrastructure strains under the weight of cyber threats and a persistent digital divide, where access remains unequal and opportunities unevenly distributed. Yet amidst these challenges, a sense of loss pervades—the fading promise of an open, inclusive digital future. Navigating this uncertain terrain demands a collective effort to reclaim the internet's original ideals of empowerment and connectivity, ensuring it remains a force for good amid mounting challenges.

Footnotes:

[1] https://support.google.com/knowledgepanel/answer/9787176?hl=en

[2] https://www.niemanlab.org/2022/12/the-ai-content-flood/

[3] https://www.thealgorithmicbridge.com/p/how-the-great-ai-flood-could-kill

[4] https://medicpro.london/decentralised-knowledge-graphs/

[5] https://arxiv.org/pdf/2308.06374

[6] https://www.computer.org/csdl/magazine/co/2023/11/10286238/1RimWA0RzFK

What’s next for The Lindahl Letter?

Week 161: Structuring really large knowledge graphs
Week 162: Indexing facts vs. graphing knowledge
Week 163: Self-Supervised Learning
Week 164: Graph-Based Feature Engineering
Week 165: Federated Feature Engineering

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Things are starting to align within this renewed writing project as my content creation process gets back into some semblance of a proper routine. We are getting pretty close to a place based on the current state of technology where my weekly podcast audio could be produced using a model based on my voice in a matter of seconds. That is not really something that I am considering. I have recorded the last two podcast episodes using my newly acquired MacBook Air using the freely supplied GarageBand software instead of using Audacity on my Windows powered desktop computer. I’m still using the Yeti X microphone and a Marantz Sound Shield Live professional vocal reflection filter, but the operating system and software being used for recording the audio is very different. For scientific purposes, you are welcome to go back and listen to a few of the previous recordings and then check out any episode from 157 forward to see if the audio quality is different. I think the overall quality of the recording is higher with the new setup.

We are going to jump into the deep end of featurization for machine learning this week. To achieve that effort in practice a series of potential next level featurization techniques will be evaluated. Yes – you guessed it, a new series is forming. Within 7 upcoming editions of this Substack newsletter, including upcoming weeks 163 to 169, I’m going to try to pull together some solid coverage and include some academic articles to read related to these topics. You know that I strive to find the best open research papers to share. Things that reside behind a paywall where practitioners and pracademics cannot easily read them I tend to exclude from these missives. That is a choice that is being made on purpose to favor open research. I’ll be really digging into each of these topics in more detail during some future missives. On a side note, It’s about time to refresh my open source intro to machine learning syllabus as well [1].

Here are some concepts that to me are highly promising strategies in feature engineering that represent good places to focus understanding as we move toward the future of the field:

Self-Supervised Learning: Leveraging large amounts of unlabeled data to automatically learn feature representations.
Graph-Based Feature Engineering: Utilizing graph neural networks to capture relationships and dependencies in graph-structured data.
Federated Feature Engineering: Creating features in a decentralized manner to enhance privacy and security by keeping data distributed.
Explainable Feature Engineering: Developing features that improve model interpretability and explainability.
Adaptive Feature Engineering: Using dynamic techniques that evolve features based on real-time data and model feedback.
Synthetic Data Generation: Generating synthetic datasets to create new features and augment training data.
Transfer Learning for Features: Reusing feature representations learned from one domain or task to another, reducing the need for extensive feature engineering in new tasks.

These strategies power feature engineering by providing more advanced, adaptive, and interpretable features for cutting-edge machine learning models. Feature engineering is crucial to machine learning for several reasons:

Improves Model Performance and Efficiency: Well-engineered features enhance the predictive power and efficiency of machine learning models, leading to better accuracy and faster convergence during training.
Simplifies Complexity and Enhances Interpretability: Effective feature engineering simplifies the problem space, making models easier to understand and interpret, thereby increasing stakeholder trust in the model's predictions.
Incorporates Domain Knowledge and Handles Diverse Data: Integrating domain-specific knowledge and transforming diverse data types into a consistent format ensures models can process information effectively and produce relevant results.
Addresses Data Quality and Robustness: Feature engineering helps clean and normalize data, handle missing values and outliers, and improves the model's robustness to changes in data distribution and external conditions.

Now that the foundation has been set for considering featurization within the machine learning space you can sit back and relax as these topics receive even more evaluation within future editions of this newsletter.

Footnotes:

[1] https://www.researchgate.net/publication/362679091_An_independent_study_based_introduction_to_machine_learning_syllabus_for_2022

What’s next for The Lindahl Letter?

Week 160: Increasingly problematic knowledge graph updates
Week 161: Structuring really large knowledge graphs
Week 162: Indexing facts vs. graphing knowledge
Week 163: Self-Supervised Learning
Week 164: Graph-Based Feature Engineering

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Longtime readers of my work know that within my normative bias I tend to break things down into form, function, assumptions, and structure (FFAS). Instead of taking that path with this Substack based letter format, each week my commentaries are going to drift into a more ongoing narrative about the patterns, traditions, and concerns that rise to the forefront of my thoughts. All right, now is the time, let’s jump into that narrative at the deepest end of the things being considered. Maybe the theme of today is about thinking globally and building action locally. That trope remains popular and will continue to be popular moving forward. Zooming out to the global view of things a number of ongoing narratives abound these days with a sea of digital content being created. A great flood of information has been intensifying. I would actually begin to build up an argument that even the best collections of knowledge are going to start breaking down as the flood intensifies. We are now seeing the script get flipped and things are going from macro flooding to incredibly local models that are unique to individual computer operating systems. We are probably going to have to see some sort of defense against actors trying to federate all the local models into a larger system of trading.

Apple executives are reasoned and measured in the deployment of products. Like many of you, I was seriously curious to see what they would do at Apple as AI hype reached a crescendo. Earnings calls and forecasts seem to be triangulated on what AI will do for a company. Apparently, Apple Intelligence is going to do a lot of things [1]. It’s going to do so many things within the Apple ecosystem that endless hours have speculated about it. Google had the opportunity to really bring all the data within their ecosystem together in a very local way, but for some reason they just did not deliver on that potential. We are seeing Microsoft teams bringing forward a feature called recall which uses continuous screenshots with a local model. We are also starting to see the arrival of Copilot machines [2]. That means both Apple and Microsoft are going to provide very local personalized AI experiences. It’s unlikely that Apple executives will try to capture local user data and use it for federated LLM training. However, our friends at Microsoft will probably call this anonymized continuous learning a feature that enhances the model.

Over the last couple of years I have had numerous conversations with people about understanding ROI related to technology projects. During those chats I try to explain that AI or ML is not really the product of the future for most companies. Telling people that their company is probably not the one that will become infinitely rich off of AI is always dicey. I think the underlying technology, models, and methods will become commoditized. Whatever company emerges at the top may have a brief advantage, but it will fade quickly as no moat exists for a repeatable idea. Generally speaking, most companies will end up using AI/ML to augment, automate, or add features to products they already have or are considering building. We are starting to see major players like Apple, Google, and others explaining that AI will power features or delivery in core products. It’s happening now in terms of announcements and we are now waiting for all the future AI features to launch. Maybe at some point along the way during the true intersection of technology and modernity we will see an AGI event or something on that level.

All the future AI features are about to get a lot more coverage as people realize just how much has been spent to get to where we are right now. Nvidia as a company had a stock split and has the largest market cap after recently beating out both Apple and Microsoft. A mind boggling amount of money has been spent training models in both the cloud and in terms of hardware. We are starting to see some more specialized hardware showing up to the party. For some companies the GPU was the coin of the realm. Over at Hugging Face you can take a look at the open LLM leaderboard to see what model is currently king of the hill [3]. You will see from that sightseeing tour of the Hugging Face leaderboards that a lot of different LLM models exist right now and a lot of them are bunched up in terms of the rankings. What is interesting about that is that you can download models. Some of them even run locally and are pretty accessible in terms of deployment and usability.

Footnotes:

[1] https://www.apple.com/apple-intelligence/

[2] https://blogs.microsoft.com/blog/2024/05/20/introducing-copilot-pcs/

[3] https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard

What’s next for The Lindahl Letter?

Week 159: The next level of featurization
Week 160: Increasingly problematic knowledge graph updates
Week 161: Structuring really large knowledge graphs
Week 162: Indexing facts vs. graphing knowledge

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

It’s time to get ruthless about actually managing my epic writing backlog. You can rewind to a previous Substack newsletter with a search or for you subscribers you can check your previous email for the epic 2024 predictions post on January 19, 2024. That was according to my records the post for week 156. Instead of starting up a season two of the podcast I’m just going to lean into the signal of things and publish this missive as week 157 of the Lindahl Letter. For those of you who are new to this ongoing chautauqua of learning and consideration, welcome to the journey. Let’s refocus on working to fix that problematic backlog. Right now it is a Google Doc stored backlog with 147 line items or topics (years worth) that were cataloged for future coverage. At this point in that writing journey, I’m not entirely sure that during that backlog acceptance process a degree of good judgment was used. A lot of things piled up and were not ruthlessly screened for quality or the best possible adventure.

You can’t stop the signal. We as a society have opened the door to a never ending, always growing, or perpetually flooding stream of content. Even experts in their respective fields of study are facing more content being created than can be consumed. At that point, even the experts are having to gate, limit, or constrain the universe of possible material to consider. We may have hit that weird tipping point where no matter what the amount of content that exists it is greater than what can be consumed. Not only are we at the edge of technology intersecting with modernity, but also we have crossed the maximum of human consumable knowledge. No matter what as we go forward even the best specialized experts will have a limited view of the possible universe of knowledge. You have to pick the best possible window of understanding. That means that really only the researchers at the edge of what is possible will be able to define what’s next, but only for a certain window of time which will quickly be reframed by new windows.

One way to look at the flooding of academic articles is to evaluate how reviewers (functionally the gatekeepers) are being impacted by the flooding of content [1][2]. Maybe just maybe the review system itself will break down and something else will need to be created. I actually favor a system where each university willing to do the work as a department would be the home of a journal and the system is generally more open for people to be able to read academic research. I think this will focus and push clear research trajectories forward. That system might just help push things along toward a system where the answer is to conduct more research and publish it. All that research will beget more writing at the edge of knowledge. Questions will be answered. New questions will appear and that cycle will continue going forward. A lot of academic articles should be sorted by contribution level to the academy. That might help researchers limit the universe of articles that need to be reviewed during any given research project.

Footnotes:

[1] Hanson, M. A., Barreiro, P. G., Crosetto, P., & Brockington, D. (2023). The strain on scientific publishing. arXiv preprint arXiv:2309.15884. https://arxiv.org/abs/2309.15884

[2] Thelwall, M., & Sud, P. (2022). Scopus 1900–2020: Growth in articles, abstracts, countries, fields, and journals. Quantitative Science Studies, 3(1), 37-50. https://direct.mit.edu/qss/article/3/1/37/109076/Scopus-1900-2020-Growth-in-articles-abstracts

What’s next for The Lindahl Letter?

Week 158: All the future AI features
Week 159: The next level of featurization
Week 160: Increasingly problematic knowledge graph updates
Week 161: Structuring really large knowledge graphs
Week 162: Indexing facts vs. graphing knowledge

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Greetings, readers and avid listeners and technology enthusiasts! You're either reading or tuned in to the audio-only podcast of The Lindahl Letter, now in its 154th week. Remember, an extra fresh and original edition lands in your inbox every Friday. Now certified with a three year proven track record. Today, we delve into an intriguing theme: “My 2024 Predictions.” Let's explore the future together.

Generative AI's Expanding Horizons: We're on the brink of witnessing a generative AI leap forward into agency and actions. The upcoming wave is set to introduce more nuanced language models and sophisticated image generators. Imagine a world where content creation and design are revolutionized, and software development is seamlessly intuitive. A standout prediction? Micro-targeting will become a staple, with chat agents offering highly personalized experiences, finely attuned to individual preferences and interests. I think it's actually going to get uncomfortable with how far people are going to take targeting in 2024.
The Evolution of AI-Powered Automation: AI's influence in automation is deepening its roots across various sectors. We'll analyze potential milestones in logistics, retail, and online services. Could these innovations redefine job roles and reshape business workflows? Let's ponder the possibilities. I think people are going to jump in and start automating all sorts of things that might deserve automation and others that maybe should have waited for a more mature point in the technology development curve.
AI Ethics and Legislative Landscape: As AI entwines more with our daily lives, the drumbeat for ethical standards and regulations grows louder. We'll reflect on how various nations might navigate the governance of AI and its impact on global AI development and cooperation. Don’t worry this won’t be a purely legislative capture point of view or a comparative political analysis.
AI policy may abound: AI's Role in Government and Public Sector is going to increase. AI's potential in enhancing public administration, policy formulation, and citizen services is immense. The discussion will spotlight AI's applications in public safety, urban planning, and social welfare initiatives, marking a significant shift in governmental functions. I think things on this front are going to get moving at a rapid speed in 2024.
Quantum Computing Breakthroughs: Turning our gaze to quantum computing, I’ll speculate on its role in tackling problems that are currently beyond classical computing's reach. From cryptography to material science, the implications are vast and profound. Maybe 2024 is the year people use some of that IBM quantum computing and share the results.
AI in Healthcare - Next Frontiers: The healthcare sector is ripe for AI-driven innovation. We'll dive into expected advancements in personalized medicine, advanced diagnostics, and AI's emerging role in streamlining healthcare administration and enhancing patient care.

What’s next for The Lindahl Letter?

Week 155: Generative AI's Expanding Horizons
Week 156: The Evolution of AI-Powered Automation
Week 157: AI Ethics and Legislative Landscape
Week 158: AI policy may abound
Week 159: Quantum Computing Breakthroughs
Week 160: AI in Healthcare

If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you for joining us this week. Stay curious, stay informed, and enjoy the week ahead!

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Please accept this note of gratitude for being a part of this journey. We made it. Some of you have been a part of the entire journey. Three years ago on January 26, 2021, this very Substack started rocking and rolling along a weekly journey to share research notes. My backlog of weekly items to cover is still pretty darn larger. It contains well over 100 blocks of potential writing content. A few larger projects are lurking within that backlog that could be stacked up block by block into future books, manuscripts, or larger articles. I’m feeling reflective today about the nature of and future of Stubstack as a publishing platform. A lot of writers flocked to Substack and it as a platform has certainly helped nurture independent writing. Newsletters have come a long way over the years, but for the most part they are fundamentally the same asymmetric writer to audience communication method. Within the situation we are experiencing at the moment it appears to be Substack as a platform that has changed.

I believe and consider it to be true that sunlight has always been the best disinfectant. Understanding begins the path to knowledge. Substack as a platform is at a crossroads. It’s my guess that the platform that is Substack will radically change in 2024. To be honest about that change, I’d have to say I’m not entirely sure what will happen [1]. Generally, I’m going to keep writing and publishing until a move to my core WordPress domain is required [2]. Everything is all set up over at that domain just in case things have to be moved, but I’m legitimately hoping that the Substack community survives the year. My corpus of writing is well over 5 million words and while none of them are particularly spicy or super eventful they were written to be shared.

You can tell here now that you made it to the third paragraph that this missive has gone back to my previous writing strategy and is not reduced into highly curated bullet points. That was something that I tried out to see if that modern communication strategy would work for my research notes. I think my preference going forward will be to write in a more long form communication structure. Bullet points have a place and are great for reducing large amounts of information into something more palatable. My writing generally has been more about being a self contained research note that brings forward a degree of understanding about something complex. That will most certainly be the standard going forward. Things that catch my attention are going to receive coverage and that will be the ongoing basis of each weekly Lindahl Letter.

Links and thoughts:

I listed to Hard Fork this week “The Times Sues OpenAI + A Debate Over iMessage + Our New Year’s Tech Resolutions”
I read this paper “Mixtral of Experts” https://arxiv.org/pdf/2401.04088.pdf

Footnotes:

[1] I read this article from Platformer:

[2] https://www.nelslindahl.com/ or here https://www.nelslindahl.com/weblog/

What’s next for The Lindahl Letter?

Week 154: My 2024 predictions

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Maybe a longer title for this post could be, “Bayesian Models and Elections: A Dive into the Dance of Uncertainty.” This is the 150th transmission of the Lindahl Letter.

In the vast and often unpredictable theater of electoral forecasting, the quest for precision is a relentless pursuit. The choreography of voter behavior is a complex ballet, orchestrated by a myriad of factors—societal tremors, economic tides, the charisma of candidates, and the machinations of campaign strategies. Amidst this swirling cauldron of variables, the call for a more nuanced forecasting method is loud and clear. And what answers the call with a finesse born of probabilistic reasoning is the realm of Bayesian models. These statistical marvels stand at the confluence of data and uncertainty, offering a refined lens to dissect the electoral enigma.

The essence of Bayesian statistics, a legacy of Thomas Bayes, is a narrative of evolving beliefs in the face of emerging evidence. It's a realm where estimates aren't static, but dynamic, continually reshaped by the rhythm of new data—a narrative that resonates with the pulsating heart of electoral dynamics.

In the Bayesian narrative, the tale begins with initial beliefs, our prior probabilities. As the story unfolds with new data—the likelihood—our beliefs morph, culminating in updated beliefs or posterior probabilities. This dance of iterative learning is akin to the dynamism of electoral scenarios, where a single debate, policy announcement, or campaign rally could tilt the scales of public sentiment.

A compelling act in the Bayesian play is its ability to weave historical election data into the forecasting fabric. It’s not just about the now, but a dialogue with the past, understanding how the ghost of incumbency, the whisper of economic indicators, or the shout of demographic shifts have choreographed electoral outcomes before.

And then, there’s the magnum opus of Bayesian models—the articulation of uncertainty. Unlike the static snapshot often rendered by traditional polling, Bayesian models compose a symphony of probability distributions. They unveil a spectrum of possible electoral outcomes, each with its associated probability, painting a picture of electoral reality that's as rich as it is realistic.

The spotlight often falls on case studies like the 2012 and 2016 U.S. Presidential Elections, where the Bayesian choreography, as orchestrated by platforms like Nate Silver’s FiveThirtyEight, navigated the electoral tumult with a commendable degree of accuracy. By embracing uncertainties and dancing with historical context, Bayesian models orchestrate a forecast that traditional polling methods seldom match.

Yet, the narrative isn’t without its share of cliffhangers. The hurdles of data scarcity, model misspecification, and computational intricacies are challenges that beckon solutions. Despite these, the Bayesian voyage into electoral forecasting holds a promise—of rendering narratives that are not only statistically sound but resonate with intuitive clarity.

As the electoral saga continues to unfold, the allure for better forecasting tools is a relentless whisper. Bayesian models, with their eloquence in narrating the dance of uncertainty, emerge as potent companions for pollsters and policymakers. They underline an electoral truism—in a realm replete with uncertainties, understanding and embracing these uncertainties isn’t just the hallmark of wisdom, but a cornerstone of robust electoral forecasting.

A few scholarly articles I found interesting this week:

Linzer, D. A. (2013). Dynamic Bayesian forecasting of presidential elections in the states. Journal of the American Statistical Association, 108(501), 124-134. https://www.ocf.berkeley.edu/~vsheu/Midterm%202%20Project%20Files/Linzer-prespoll-May12.pdf

Lock, K., & Gelman, A. (2010). Bayesian combination of state polls and election forecasts. Political Analysis, 18(3), 337-348. https://academiccommons.columbia.edu/doi/10.7916/D88K7GV1/download

Heidemanns, M., Gelman, A., & Morris, G. E. (2020). An updated dynamic Bayesian forecasting model for the US presidential election. Harvard Data Science Review, 2(4), 10-1162. https://assets.pubpub.org/wbec6d9k/9dfc3335-6d48-4f8e-bf5d-0011c7817a09.pdf

Olsson, H., Bruine de Bruin, W., Galesic, M., & Prelec, D. (2021). Election polling is not dead: a Bayesian bootstrap method yields accurate forecasts. Preprint at https://osf.io/nqcgs/

If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. Thank you and enjoy the week ahead.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Title: The Confluence of Agent Systems and Expert Opinion in Election Simulations

In the ever-evolving landscape of political science and technology, curiosity often paves the way for innovative approaches and fresh perspectives. Recently, a wave of curiosity has washed over me, primarily centered around the exploration of diverse agent systems to simulate elections. This intersection of technology and electoral processes opens up new realms of possibilities, allowing us to mimic, analyze, and potentially enhance our understanding of elections in a simulated environment.

Agent systems provide a powerful tool for creating algorithmic or model-based simulations. These systems can be meticulously crafted, incorporating synthetic focus groups and panels that emulate real-world election scenarios. The meticulous design allows for the creation of adversarial agents that can engage in debates or various activities that accurately mirror the complexities and dynamics of an actual election.

However, my curiosity doesn’t end here. I’m also deeply intrigued by the fusion of election simulation with expert opinion systems. By appending the term ‘systems’ to ‘expert opinion’, the concept transcends beyond individual viewpoints, fostering an environment where aggregated expert opinions are diligently worked upon and analyzed. These collected data become a potent resource, providing invaluable insights that can be seamlessly integrated into the simulated election models.

Imagine the immense potential unlocked by the combination of these two realms. The simulated agents, fortified with synthesized expert opinions, could operate in a nuanced manner that echoes the depth and diversity of actual election contenders and voters. These enhanced agents could engage in debates, make decisions, and navigate the election simulation with a level of sophistication that brings us closer to understanding the myriad factors influencing election outcomes.

Through this amalgamation, the simulation becomes a crucible where technological prowess meets the wisdom of expertise. The interplay between algorithmically driven simulations and the rich reservoir of expert opinions can unveil unprecedented avenues for exploring electoral processes. It can deepen our comprehension, offering a clearer lens through which we may view the multifaceted realms of elections.

In conclusion, the fusion of various agent systems with expert opinion systems presents a promising frontier in the world of election simulations. By harnessing the collective wisdom of experts and embedding this knowledge within algorithmic agents, we stand on the brink of developing more nuanced, realistic, and insightful election simulation models. The curiosity driving this exploration is not merely a personal quest, but rather a shared journey towards enriching our understanding and approaches to simulating and analyzing electoral processes.

What’s next for The Lindahl Letter?

Week 147: Bayesian Models
Week 148: Running Auto-GPT on election models
Week 149: Sentiment Analysis
Week 150: Voter Models

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Let’s get to work unpacking the Delphi Method & Door-to-Door Canvassing. This is the Substack deep dive you don’t want to miss.

Hey there, dear weekly Substack readers!

Today, let's embark on a journey through two fascinating potentially election related realms: the Delphi Method and Door-to-Door Canvassing. While these two concepts might seem worlds apart, there is a curious intersection between the two of them that's worth exploring. So, grab your favorite beverage, and let's dive in! I had two shots of espresso and they were delightful.

Delphi Method: More Than Just Ancient Greece

First up, the Delphi Method. No, we're not time-traveling back to ancient Greece using the world’s finest Delorean, but we are diving into a method inspired by the oracle of Delphi. It's a structured communication technique designed for interactive forecasting. Picture this: a group of experts, multiple rounds of questionnaires, and a quest for consensus. The beauty of this method? It taps into collective intelligence while ensuring every expert voice gets its moment in the sun, minus the overshadowing by dominant personalities. Generally this method is unlikely to occur naturally today due in part to the overwhelming decay in civility that has occurred. People just don’t cross over into different parisian camps these days. Something has distinctly changed in our politics.

Prokesch, T., Von der Gracht, H. A., & Wohlenberg, H. (2015). Integrating prediction market and Delphi methodology into a foresight support system—Insights from an online game. Technological Forecasting and Social Change, 97, 47-64 [1].

Dalkey, N. C., Brown, B. B., & Cochran, S. (1969). The Delphi method: An experimental study of group opinion (Vol. 3, p. 107). Santa Monica, CA: Rand Corporation [2].

Door-to-Door Canvassing: Old School politics, But otherwise pure Gold

Switching gears, let's talk about the age-old art of door-to-door canvassing. It's personal, it's direct, and it's all about that face-to-face interaction. Whether it's political volunteers rallying support or grassroots movements gathering opinions, this method has stood the test of time. Why? Because nothing beats the authenticity of a real conversation. Sure people are less likely to want to answer the door or talk politics at the front door, but this method does still show signs of working.

Green, D. P., Gerber, A. S., & Nickerson, D. W. (2003). Getting out the vote in local elections: Results from six door-to-door canvassing experiments. The Journal of Politics, 65(4), 1083-1096 [3].

The Unexpected Crossover

Now, for the fun part. How do these two methods intertwine?

Imagine harnessing the Delphi Method's expert-driven insights to supercharge a door-to-door canvassing campaign. Before our canvassers even lace up their shoes, we could have a panel of experts—from veteran canvassers to communication gurus—forecasting the best strategies, pinpointing challenges, and highlighting golden opportunities.

That type of Magic could happen

Marrying the Delphi Method's structured insights with the grassroots authenticity of door-to-door canvassing could:

Elevate the Message: Crafting narratives that truly resonate.
Stay Two Steps Ahead: Predicting and preparing for potential challenges.
Strategize Like a Pro: Directing efforts where they count the most.

Wrapping Up

Merging the old with the new, the traditional with the innovative, can lead to some unexpected and powerful synergies. And isn't that what we're all about here on Substack? Exploring, questioning, and connecting the dots in unexpected ways.

Stay curious, and until next time!Dr. Nels Lindahl

Footnotes:

[1] https://www.researchgate.net/profile/Heiko-Von-Der-Gracht/publication/260755254_Integrating_prediction_market_and_Delphi_methodology_into_a_foresight_support_system_-_Insights_from_an_online_game/links/5c339e35299bf12be3b5592a/Integrating-prediction-market-and-Delphi-methodology-into-a-foresight-support-system-Insights-from-an-online-game.pdf

[2] https://apps.dtic.mil/sti/trecms/pdf/AD0690498.pdf

[3] https://d1wqtxts1xzle7.cloudfront.net/45996527/Getting_Out_the_Vote_in_Local_Elections_20160527-16511-1wf5rrd-libre.pdf?1464362918=&response-content-disposition=inline%3B+filename%3DGetting_Out_the_Vote_in_Local_Elections.pdf&Expires=1696684199&Signature=S6S9UNmWYRMepopnbBWQlGkCn4q4C889yqi3aoE~-47Z~DL2Hpw5TWKDz6Cq4IF9~gp-sfEPaVehWkrW7YiYQLLL0f6XEsNNtlU3WUl4NSee2JH1B2CNTWcy9glqPjVo6KBfe6oKUYr4YlCatCXgDgJEL~HtRsIiwswn4XxGpWAv~7sLT-X5M8Zc13wVlYl8MEzNF32WpOM5JaJUtUA8Z5k8G2cMgHWzRRYyB6GXf1Pr2MWovSCameHEHC~G44wcCYoK-54jdYdnP605msL6gifKpj0dp58ETNOcFnBvCVwU5hjfUcIgjCJLpDTlXV7OLokXfL3uydrisDoN~H3QZA__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Don’t panic, the Google Scholar searches are coming in fast and furious on this one [1]. We had a footnote in the first sentence today. Megan Tomlin writing over at neo4j had probably the best one line definition of the difference by noting that knowledge graphs are going to be in the human readable data camp and vector databases are more of a black box [2]. I actually think that eventually one super large knowledge graph will emerge and be the underpinning of all of this, but that has not happened yet given that the largest one in existence Google holds will always remain proprietary.

Combining two LLMs… right now you could call them one after another, but I’m not finding an easy way to pool them into a single model. I wanted to just say to my computer, “use Baysian pooling to combine the most popular LLMs from Hugging Face,” but yeah that is not an available command at the moment. A lot of incompatible content is being generated in the vector database space. People are stacking LLMs and working in sequence or making parallel calls to multiple-models. What I was very curious about was how to go about the process of merging LLMs, combining LLMs, actual model merges, ingestion of models, or even a method to merge transformers. I know that is a tall order, but it is one that would take so much already spent computing cost and move it from sunk to additive in terms of value.

A few papers exist on this, but they are not exactly solutions to this problem.

Jiang, D., Ren, X., & Lin, B. Y. (2023). LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion. arXiv preprint arXiv:2306.02561. https://arxiv.org/pdf/2306.02561.pdf you can see more content related to this one here https://yuchenlin.xyz/LLM-Blender/

Wu, Q., Bansal, G., Zhang, J., Wu, Y., Zhang, S., Zhu, E., ... & Wang, C. (2023). AutoGen: Enabling next-gen LLM applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155. https://arxiv.org/pdf/2308.08155.pdf

Chan, C. M., Chen, W., Su, Y., Yu, J., Xue, W., Zhang, S., ... & Liu, Z. (2023). Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201. https://arxiv.org/pdf/2308.07201.pdf

Most of the academic discussions and even the cutting edge papers like AutoGen are about orchestration of models instead of merging, combining, or ingestion of many models into one. I did find a discussion on Reddit from earlier this year about how to merge the weights of transformers [3]. It’s interesting what things end up on reddit. Sadly that subreddit is closed due to a dispute over 3rd party plugins.

Exploration into merging and combining Large Language Models (LLMs) is indeed at the frontier of machine learning research. While academic papers like "LLM-Blender" and "AutoGen" offer different perspectives, they primarily focus on ensembling and orchestration rather than true model merging or ingestion. The challenge lies in the inherent complexities and potential incompatibilities when attempting to merge these highly sophisticated models.

The quest for effectively pooling LLMs into a single model or merging transformers is a journey intertwined with both theoretical and practical challenges. Bridging the gap between the human-readable data realm of knowledge graphs and the more opaque vector database space, as outlined in the beginning of this podcast, highlights the broader context in which these challenges reside. It also underscores the necessity for a multidisciplinary approach, engaging both academic researchers and the online tech community, to advance the state of the art in this domain.

In the upcoming weeks, we will delve deeper into the community-driven solutions, and explore the potential of open-source projects in advancing the model merging discourse. Stay tuned to The Lindahl Letter for a thorough exploration of these engaging topics.

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=knowledge+graph+vector+database&btnG=

[2] https://neo4j.com/blog/knowledge-graph-vs-vectordb-for-retrieval-augmented-generation/

[3] https://www.reddit.com/r/MachineLearning/comments/122fj05/is_it_possible_to_merge_transformers_d/

What’s next for The Lindahl Letter?

Week 145: Delphi method & Door-to-door canvassing
Week 146: Election simulations & Expert opinions
Week 147: Bayesian Models
Week 148: Running Auto-GPT on election models
Week 149: Modern Sentiment Analysis

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

After the adventures of last week, I started this writing adventure wanting to try to figure out what people were doing with LangChain and social media. People are both generating content for social media using LLMs and oddly enough repurposing content as well. We have to zoom out for just a second and consider the broader ecosystem of content. In the before-times, people who wanted to astroturf or content farm had some work to do within the content creation space. Now ChatGPT has opened the door and let the power of synthetic content creation loose. You can create personas and just have them generate endless streams of content. People can even download and run models trained for this purpose. It’s something I’m legitimately worried about for this next election cycle. Sometimes I wonder how much content within the modern social media spaces is created artificially. Measuring that is actually pretty difficult. It’s not like organically created content gets a special badge or recognition.

For those of you who were interested in finding out insights on any topic with a plugin that works with the OpenAI ChatGPT system then you could take a moment and install “The Yabble ChatGPT Plugin” [1]. Fair warning on this one I had to reduce my 3 plugins down to just Yabble and be pretty explicit in the prompts within ChatGPT to make it do some work. Sadly, I could not just login to Yabble and had to book a demo with them to get access. Stay tuned on that one to get more information on how that system works. I had started by searching out plugins to have ChatGPT analyze social media. This has become easier now with the announcements that OpenAI can openly use Bing search [2].

Outside of searching using any OpenAI tooling like ChatGPT, Google was pretty clear on the reality that what I was really looking for happened to actually be marketing tools. Yeah, I went down the SEO Assistant rabbit hole and it was shocking. So much content exists in this space that is like watching a very full ant farm for the most part. Figuring out where to jump in without getting scammed is probably a questionable decision framework. Whole websites and ecosystems could be synthetically generated pretty quickly. It’s not exactly one click turn key deployments, but it is getting close to that level of content farming.

I was willing to make the assumption that people who were going to the trouble of making actual plugins for ChatGPT within the OpenAI platform are probably going to be more interesting and maybe are building actual tooling. For those of you who are using ChatGPT with OpenAI and have the plus subscription you just have to open a new chat, expand the plugin area, and scroll down to the plugin store to search for new ones…

I also did some searches for marketing tools. I’m still struck with the possibility that a lot of content is being created and marketed to people. It’s not the potential flooding of content that becomes so overwhelming that nobody is able to navigate the internet anymore. We are getting very close to the point where it would be entirely possible for the flooding of new content to occur in ways that simply overwhelm everybody and everything with new content. This would be like the explosion of ML/AI papers over the last 5 years, but maybe 10x or 100x even that digital content boom [3].

Footnotes:

[1] https://www.yabble.com/chatgpt-plugin

[2] https://www.reuters.com/technology/openai-says-chatgpt-can-now-browse-internet-2023-09-27/

[3] https://towardsdatascience.com/neurips-conference-historical-data-analysis-e45f7641d232

What’s next for The Lindahl Letter?

Week 144: Knowledge graphs vs. vector databases
Week 145: Delphi method & Door-to-door canvassing
Week 146: Election simulations & Expert opinions
Week 147: Bayesian Models
Week 148: Running Auto-GPT on election models

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

And now it’s time to pivot toward the “Learning LangChain” topic…

The most fun introduction to LangChain seems to be from DeepLearning.ai with Andrew Ng and Harrison Chase [1]. You can expect to spend a couple of hours to complete the process of watching the videos and absorbing the content. Make sure you use a browser window large enough to support both the jupyter notebook and the video. You are probably going to want these items to run side by side. This course covers models, prompts, parsers, memory, chains, and agents. The part of this learning package that I was the most interested in learning more about was how people are using agents and of course what sort of plugins could that yield as use cases in the generative AI space. Going forward I think agency will be the defining characteristic of the great generative AI adventure. These applications are going to do things for you and some of those use cases are going to be extremely powerful.

After that course I wanted to dig in more and decided to go ahead and learn everything I could from the LangChain AI Handbook [2]. This handbook has 6 or 7 chapters depending on how you count things. My favorite part about this learning build is that they are using Colab notebooks for hands-on development during the course of the learning adventure. That is awesome and really lets you get going quickly. A side quest spawned out of that handbook learning which involved starting to use Pinecone in general which was interesting. You can do a lot with the Pinecone including building AI agents and chatbots.

I’m going to spend some time working on the udemy course “Develop LLM powered applications with LangChain” later this weekend [3]. You can also find a ton of useful information within the documentation for LangChain including a lot of content about agents [4].

You might now be wondering what alternatives to LangChain exist… I started looking around at AutoChain [5], Auto-GPT [6], AgentGPT [7], BabyAGI [8], LangDock [9], GradientJ [10], Flowise AI [11], and LlamaIndex [12]. Maybe you could also consider TensorFlow to be an alternative. You can tell from the combination of companies and frameworks being built out here a lot of attention is on the space between LLMs and taking action. Getting to the point of agency or taking action is where these spaces are gaining and maintaining value.

Footnotes:

[1] https://learn.deeplearning.ai/langchain/lesson/1/introduction

[2] https://www.pinecone.io/learn/series/langchain/

[3] https://www.udemy.com/course/langchain/

[4] https://python.langchain.com/docs/modules/agents/

[5] https://github.com/Forethought-Technologies/AutoChain

[6] https://github.com/Significant-Gravitas/Auto-GPT

[7] https://github.com/reworkd/AgentGPT

[8] https://github.com/miurla/babyagi-ui

[9] https://www.langdock.com/

[10] https://gradientj.com/

[11] https://flowiseai.com/

[12] https://www.llamaindex.ai/

What’s next for The Lindahl Letter?

Week 143: Social media analysis
Week 144: Knowledge graphs vs. vector databases
Week 145: Delphi method & Door-to-door canvassing
Week 146: Election simulations & Expert opinions
Week 147: Bayesian Models

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

You can feel the winds of change blowing and the potential of people building out election expert opinion chatbots. Maybe you want to know what they are probably going to use to underpin that sort of effort. If you were going out to build some generative AI chatbots for you might very well use one of the 5 systems we are going to dig into today.

Voiceflow - This system may very well be the most prominent of the quick to market AI agent building platforms [1]. I have chatbots deployed to both Civic Honors and my main weblog powered by Voiceflow.
LangFlow - You are going to need to join the waitlist for this one to get going [2]. I’m still on the waitlist for this one…
Botpress - Like Voiceflow this system lets you pretty quickly jump into the building process of actual chatbot workflows [3]. To be fair with this one I was not able to build and deploy something into production within minutes, but you could do it pretty darn quickly if you had a sense of what you were trying to accomplish. I built something on Botpress and it was pretty easy to use. After login I clicked answer questions from websites to create a bot. I added both Civic Honors and my main Nels Lindahl domain. They just jumped in and advised me that the knowledge upload was complete. Publishing the bot is not as low friction as the Voiceflow embedding launch point, but it was not super hard to work with after you find the share button.
FloWiseAI - You will find this is the first system on the list that will require you to get out of your web browser, stretch a bit, and open the command line to get this one installed with a rather simple “npm install -g flowise” command [4]. I watched some YouTube videos on how to install this one and it almost got me to flip over into Ubuntu Studio. Instead of switching operating systems I elected to just follow the regular Windows installation steps.
Stack AI - With this one you are right back into the browser and you are going to see a lot of options to start building new projects [5].

All of these chatbots built using a variety of generative AI models are generally working within the same theory of building. The conversation is being crafted with a user and some type of exchange with a knowledge base. For the most part the underlying LLM is being used to facilitate the conversational part of the equation while some type of knowledge base is being used to gate, control, and drive the conversation based on something deeper than what the LLM would output alone. It’s an interesting building technique and one that would not have been possible just a couple of years ago, but the times have changed and here we are in this brave new world where people can build, deploy, and be running a generative AI chatbot in a few minutes. It requires some planning about what is being built, you need some type of knowledgebase, and the willingness to learn the building parameters. None of that is a very high bar to pass. This is a low friction and somewhat high reward space for creating conversational interactions.

Messing around with all these different chatbot development systems made me think a little bit more about how LangChain is being used and what the underlying technology is ultimately capable of facilitating [6]. To that end I signed up for the LangSmith beta they are building [7]. Sadly enough “LangSmith is still in closed beta” so I’m waiting on access to that one as well.

During the course of this last week I have been learning more and more about how to build and deploy chatbots that take advantage of LLMs and other generative AI technologies. I’m pretty sure that the development of agency to machine learn models is going to strap rocket boosters to the next stage of technological deployment. Maybe you are thinking that is hyperbole… don’t worry or panic, but you are very soon going to be able to ask these agents to do something and they will be able to execute more and more complex actions. That is the essence of agency within the deployment of these chatbots. It’s a very big deal in terms of people doing basic task automation and it may very well introduce a distinct change to how business is conducted by radically increasing productivity.

Footnotes:

[1] https://www.voiceflow.com/

[2] https://www.langflow.org/

[3] https://botpress.com/

[4] https://flowiseai.com/

[5] https://www.stack-ai.com/

[6] https://www.langchain.com/

[7] https://www.langchain.com/langsmith

What’s next for The Lindahl Letter?

Week 142: Learning LangChain
Week 143: Social media analysis
Week 144: Knowledge graphs vs. vector databases
Week 145: Delphi method & door to door canvasing
Week 146: Election simulations

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Sometimes a simplified model of something is easier to work with. We dug into econometric models recently during week 136 and they can introduce a high degree of complexity. Even within the world of econometrics you can find information about proxy models. In this case today we are digging into proxy models for elections. My search was rather direct. I was looking for a list of proxy models being used for elections [1]. I was trying to dig into election forecasting proxy models or maybe even some basic two step models. I even zoomed in a bit to see if I could get targeted on machine learning election proxy models [2].

After a little bit of searching around it seemed like a good idea to maybe consider what it takes to generate a proxy model equation to represent something. Earlier I had considered what the chalk model of election prediction would look like with using a simplified proxy of voter registration as an analog for voting prediction. I had really thought that would end up being a highly workable proxy, but it was not wholesale accurate.

Here are 3 papers I looked at this week:

Hare, C., & Kutsuris, M. (2022). Measuring swing voters with a supervised machine learning ensemble. Political Analysis, 1-17. https://www.cambridge.org/core/services/aop-cambridge-core/content/view/145B1D6B0B2877FC454FBF446F9F1032/S1047198722000249a.pdf/measuring_swing_voters_with_a_supervised_machine_learning_ensemble.pdf

Zhou, Z., Serafino, M., Cohan, L., Caldarelli, G., & Makse, H. A. (2021). Why polls fail to predict elections. Journal of Big Data, 8(1), 1-28. https://link.springer.com/article/10.1186/s40537-021-00525-8

Jaidka, K., Ahmed, S., Skoric, M., & Hilbert, M. (2019). Predicting elections from social media: a three-country, three-method comparative study. Asian Journal of Communication, 29(3), 252-273. http://www.cse.griet.ac.in/pdfs/journals20-21/SC17.pdf

I spent some time messing around with OpenAI’s GPT-4 on this topic. That effort drove down to a few proxy models that are typically used. The top 10 seemed to be the following: social media analysis, google trends, economic indicators, fundraising data, endorsement counts, voter registration data, early voting data, historical voting patterns, event-driven, and environmental factors. Combining all 10 proxy models into a single equation would result in a complex, multi-variable model. Here's a simplified representation of such a model:

E=α1(S)+α2(G)+α3(Ec)+α4(F)+α5(En)+α6(VR)+α7(EV)+α8(H)+α9(Ed)+α10(Ef)+β

Where:

E is the predicted election outcome.
α1, α2,...α10 are coefficients that determine the weight or importance of each proxy model. These coefficients would be determined through regression analysis or other statistical methods based on historical data.
S represents social media analysis.
G represents Google Trends data.
Ec represents economic indicators.
F represents fundraising data.
En represents endorsement count.
VR represents voter registration data.
EV represents early voting data.
H represents historical voting patterns.
Ed represents event-driven models.
Ef represents environmental factors.
β is a constant term.

This equation is a linear combination of the proxy models, but in reality, the relationship might be non-linear, interactive, or hierarchical. The coefficients would need to be determined empirically, and the model would need to be validated with out-of-sample data to ensure its predictive accuracy. Additionally, the model might need to be adjusted for specific elections, regions, or time periods. It would be interesting to try to pull together the data to test that type of complex multivariable model. Maybe later on we can create a model with some agency designed to complete that task.

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=election+proxy+models&btnG=

[2] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=election+proxy+models+machine+learning&btnG=

What’s next for The Lindahl Letter?

Week 141: Building generative AI chatbots
Week 142: Learning LangChain
Week 143: Social media analysis
Week 144: Knowledge graphs vs. vector databases
Week 145: Delphi method

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

This might be the year that I finally finish that book about the intersection of technology and modernity. During the course of this post we will look at the intersection of machine learning and election models. That could very well be a thin slice of the intersection of technology and modernity at large, but that is the set of questions that brought us here today. It’s one of things we have been chasing along this journey. Oh yes, a bunch of papers exist related to the topic this week of machine learning and election models [1]. None of them are highly cited. A few of them are in the 20’s in terms of citation count, but that means the academic community surrounding this topic is rather limited. Maybe the papers are written, but have just not arrived yet out in the world of publication. Given that machine learning has an active preprint landscape that is unlikely.

That darth of literature is not going to stop me from looking at them and sharing a few that stood out during the search. None of these papers is approaching the subject from a generative AI model side of things they are using machine learning without any degree of agency. Obviously, I was engaging in this literature review to see if I could find examples of the deployment of models with some type of agency doing analysis within this space of election prediction models. My searching over the last few weeks has not yielded anything super interesting. I was looking for somebody in the academic space doing some type of work within generative AI constitutions and election models or maybe even some work in the space of rolling sentiment analysis for targeted campaign understanding. That is probably an open area for research that will be filled at some point.

Here are 4 articles:

Grimmer, J., Roberts, M. E., & Stewart, B. M. (2021). Machine learning for social science: An agnostic approach. Annual Review of Political Science, 24, 395-419. https://www.annualreviews.org/doi/pdf/10.1146/annurev-polisci-053119-015921

Sucharitha, Y., Vijayalata, Y., & Prasad, V. K. (2021). Predicting election results from twitter using machine learning algorithms. Recent Advances in Computer Science and Communications (Formerly: Recent Patents on Computer Science), 14(1), 246-256. www.cse.griet.ac.in/pdfs/journals20-21/SC17.pdf

Miranda, E., Aryuni, M., Hariyanto, R., & Surya, E. S. (2019, August). Sentiment Analysis using Sentiwordnet and Machine Learning Approach (Indonesia general election opinion from the twitter content). In 2019 International conference on information management and technology (ICIMTech) (Vol. 1, pp. 62-67). IEEE. https://www.researchgate.net/publication/335945861_Sentiment_Analysis_using_Sentiwordnet_and_Machine_Learning_Approach_Indonesia_general_election_opinion_from_the_twitter_content

Zhang, M., Alvarez, R. M., & Levin, I. (2019). Election forensics: Using machine learning and synthetic data for possible election anomaly detection. PloS one, 14(10), e0223950. https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0223950&type=printable

My guess is that we are going to see a wave of ChatGPT related articles about elections post the 2024 presidential cycle. It will probably be one of those waves of articles without any of them really standing out or making any serious contribution to the academy.

The door is opening to a new world of election prediction and understanding efforts thanks to the recent changes in both model agency and generative AI models that help evaluate and summarize very complex things. It’s really about how they are applied to something going forward that will make the biggest difference in how the use cases play out. These use cases by the way are going to become very visible as the 2024 election comes into focus. The interesting part of the whole equation will be when people are bringing custom knowledge bases to the process to help fuel interactions with machine learning algorithms and generative AI.

It's amazing to think how rapidly things can be built. The older models of software engineering are now more of a history lesson than a primer on building things with prompt-based AI. Andrew Ng illustrated in a recent lecture the rapidly changing build times. You have to really decide what you want to build and deploy and make it happen. Ferris Bueller once said, "Life moves pretty fast." Now code generation is starting to move even faster! You need to stop and look around at what is possible, or you just might miss out on the generative AI revolution.

You can see Andrew's full video here:

Links and thoughts:

I really enjoyed this talk from Bill Gurly, “All-In Summit: Bill Gurley presents 2,851 Miles”

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=Machine+learning+election+models&btnG=

What’s next for The Lindahl Letter?

Week 140: Proxy models for elections
Week 141: Building generative AI chatbots
Week 142: Learning LangChain
Week 143: Social media analysis
Week 144: Knowledge graphs vs. vector databases

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

We have been going down the door of digging into considering elections for a few weeks now. You knew this topic was going to show up. People love prediction markets. They are really a pooled reflection of sentiment about the likelihood of something occuring. Right now the scuttlebut of the internet is about LK-99, a potential, maybe debunked, maybe possible room temperature superconductor that people are predicting whether or not it will be replicated before 2025 [1]. You can read the 22 page preprint about LK-99 on ArXiv [2]. My favorite article about why this would be a big deal if it lands was from Dylan Matthews over at Vox [3]. Being able to advance the transmission power of electrical lines alone would make this a breakthrough.

That brief example being set aside, now people can really dial into the betting markets for elections where right now are not getting nearly the same level of attention as LK-99 which is probably accurate in terms of general scale of possible impact. You can pretty quickly get to all posts that the team over at 538 have tagged for “betting markets” and that is an interesting thing to scroll through [4]. Beyond that look you could start to dig into an article from The New York Times talking about forecasting what will happen to prediction markets in the future [5].

You know it was only a matter of time before we moved from popular culture coverage to the depths of Google Scholar [6].

Snowberg, E., Wolfers, J., & Zitzewitz, E. (2007). Partisan impacts on the economy: evidence from prediction markets and close elections. The Quarterly Journal of Economics, 122(2), 807-829. https://www.nber.org/system/files/working_papers/w12073/w12073.pdf

Arrow, K. J., Forsythe, R., Gorham, M., Hahn, R., Hanson, R., Ledyard, J. O., ... & Zitzewitz, E. (2008). The promise of prediction markets. Science, 320(5878), 877-878. https://users.nber.org/~jwolfers/policy/StatementonPredictionMarkets.pdf

Berg, J. E., Nelson, F. D., & Rietz, T. A. (2008). Prediction market accuracy in the long run. International Journal of Forecasting, 24(2), 285-300. https://www.biz.uiowa.edu/faculty/trietz/papers/long%20run%20accuracy.pdf

Wolfers, J., & Zitzewitz, E. (2004). Prediction markets. Journal of economic perspectives, 18(2), 107-126. https://pubs.aeaweb.org/doi/pdf/10.1257/0895330041371321

Yeah, you could tell by the title that a little bit of content related to time-series analysis was coming your way. The papers being tracked within Google Scholar related election time series analysis were not highly cited and to my extreme disappointment are not openly shared as PDF documents [7]. For those of you who are regular readers you know that I try really hard to only share links to open access documents and resources that anybody can consume along their lifelong learning journey. Sharing links to paywalls and articles inside a gated academic community is not really productive for general learning.

Footnotes:

[1]

[2] https://arxiv.org/ftp/arxiv/papers/2307/2307.12008.pdf

[3] https://www.vox.com/future-perfect/23816753/superconductor-room-temperature-lk99-quantum-fusion

[4] https://fivethirtyeight.com/tag/betting-markets/

[5] https://www.nytimes.com/2022/11/04/business/election-prediction-markets-midterms.html

[6] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=election+prediction+markets&btnG=

[7] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=election+time+series+analysis&oq=election+time+series+an

What’s next for The Lindahl Letter?

Week 139: Machine learning election models
Week 140: Proxy models for elections
Week 141: Election expert opinions
Week 142: Door-to-door canvassing

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Trying to figure out how many republicans, democrats, and independents are registered in each state is actually really hard. It’s not a trivial task. Even with all our modern technology and the extreme power of the internet providing outsized connectedness between things and making content accessible to searches. Even GPT-4 from OpenAI with some decent plugins turned on will struggle to complete this task.Your best searches to get a full list by state are probably going to land you into the world of projections and surveys. One that will show up very quickly are some results from the Pew Research which contacted people (300 to 4,000 of them) from each state to find out more data about political affiliation [1]. They evaluated responses into three buckets with no lean, lean republication, or lean democrat. That allowed the results to evaluate based on sampling to get a feel for general political intentions. However, that type of intention based evaluation does not give you a sense of the number of voters within each state.

It opened the door to me considering if political registration is even a good indicator of election outcomes. Sports tournaments rarely play out based on the seeding. That is the element of it that makes it exciting and puts the sport into the tournament. To that end back during week 134 I shared the chalk model to help explore a hypothesis related to registration being predictive. At the moment, I’m more interested to see how proxy models for predicting sporting events are working. Getting actual data to track changes in political registrations is an interesting process. ChatGPT, Bard, and Bing Chat are capable of providing some numbers if you prompt them properly. The OpenAI model GPT-3.5 has some older data from September 2021 and will tell you registered voters by state [2]. I started with a basic prompt, “make a table of voter registration by state.” I had to add a few encouraging prompts at some points, but overall the models all 3 spit out results [3]. The Bing Chat model really tried to direct you back to the United States Census Bureau website [4].

This is an area where setting up some type of model with a bit of agency to go out to the relevant secretary of states websites for the 30 states that provide some data might be a way to go to build a decent dataset. That would probably be the only way to really track the official data coming out by state to show the changes in registration over time. Charting that change data might be interesting as a directional view of how voters view themselves in terms of voter registration in a longitudinal way. People who participate in Kaggle have run into challenges where election result prediction is actually a competition [5]. It’s interesting and thinking about what features are most impactful during election prediction is a big part of that competition. Other teams are using linear regression and classification models to help predict election winners as well [6]. I was reading a working paper from Ebanks, Katz, and King published in May 2023 that shared an in depth discussion about picking the right models and the problems of picking the wrong ones [7][8].

To close things out here I did end up reading this Center for Politics article from 2018 that was interesting as a look back at where things were [9]. Circling back to the main question this week, I spent some time working within the OpenAI ChatGPT with plugins trying to get GPT-4 to search out and voter registration by state. I have been wondering why with a little bit of agency one of these models could not do that type of searching. Right now the models are not set up with a framework that could complete this type of tasking.

Footnotes:

[1] https://www.pewresearch.org/religion/religious-landscape-study/compare/party-affiliation/by/state/

[2] https://chat.openai.com/share/8a6ea5e7-6e42-4743-bc23-9e8e7c4f79c5

[3] https://g.co/bard/share/96b6f8d02e8e

[4] https://www.census.gov/topics/public-sector/voting/data/tables.html

[5] https://towardsdatascience.com/feature-engineering-for-election-result-prediction-python-943589d89414

[6] https://medium.com/hamoye-blogs/u-s-presidential-election-prediction-using-machine-learning-88f93e7f6f2a

[7] https://news.harvard.edu/gazette/story/2023/03/researchers-come-up-with-a-better-way-to-forecast-election-results/

[8] https://gking.harvard.edu/files/gking/files/10k.pdf

[9] https://centerforpolitics.org/crystalball/articles/registering-by-party-where-the-democrats-and-republicans-are-ahead/

What’s next for The Lindahl Letter?

Week 138: Election prediction markets & Time-series analysis
Week 139: Machine learning election models
Week 140: Proxy models for elections
Week 141: Election expert opinions
Week 142: Door-to-door canvassing

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

It has been a few weeks here since we started by digging into a good Google Scholar search and you know this topic would be just the thing to help open that door [1]. My searches for academic articles are always about finding accessible literature that sits outside paywalls that is intended to be read and shared beyond strictly academic use. Sometimes that is easier than others when the topics lend themselves to active use cases instead of purely theoretical research. Most of the time these searches to find out what is happening at the edge of what is possible involve applied research. Yes, that type of reasoning would place me squarely in the pracademic camp of intellectual inquiry.

That brief chautauqua aside, my curiosity here is how do we build out econometric election models or other model inputs to feed into large language model chat systems as prompt engineering for the purposes of training them to help either predict elections or interpret and execute the models. This could be a method for introducing extensibility or at least the application of targeted model effect to seed a potential future methodology within the prompt engineering space. As reasoning engines go it’s possible that an econometric frame could be an interesting proxy model within generative AI prompting. It’s a space worth understanding a little bit more for sure as we approach the 2024 presidential election cycle.

I’m working on that type of effort here as we dig into econometric election models. My hypothesis here is that you can write out what you want to explain in a longer form as a potential input prompt to train a large language model. Maybe a more direct way of saying that is we are building a constitution for the model based on models and potentially proxy models then working toward extensibility and agency from introducing those models together. For me that is a very interesting space to begin to open up and kick the tires on in the next 6 months.

Here are 6 papers from that Google Scholar search that I thought were interesting:

Mullainathan, S., & Spiess, J. (2017). Machine learning: an applied econometric approach. Journal of Economic Perspectives, 31(2), 87-106. https://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.31.2.87

Fair, R. C. (1996). Econometrics and presidential elections. Journal of Economic Perspectives, 10(3), 89-102. https://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.10.3.89

Armstrong, J. S., & Graefe, A. (2011). Predicting elections from biographical information about candidates: A test of the index method. Journal of Business Research, 64(7), 699-706. https://faculty.wharton.upenn.edu/wp-content/uploads/2012/04/PollyBio58.pdf

Graefe, A., Green, K. C., & Armstrong, J. S. (2019). Accuracy gains from conservative forecasting: Tests using variations of 19 econometric models to predict 154 elections in 10 countries. Plos one, 14(1), e0209850. https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0209850&type=printable

Leigh, A., & Wolfers, J. (2006). Competing approaches to forecasting elections: Economic models, opinion polling and prediction markets. Economic Record, 82(258), 325-340. https://www.nber.org/system/files/working_papers/w12053/w12053.pdf

Benjamin, D. J., & Shapiro, J. M. (2009). Thin-slice forecasts of gubernatorial elections. The review of economics and statistics, 91(3), 523-536. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2860970/pdf/nihms190094.pdf

Beyond those papers, I read some slides from Hal Varian on “Machine Learning and Econometrics” from January of 2014 [2]. The focus of the slide was applied to modeling human choices. Some time was spent on trying to understand the premise that the field of machine learning could benefit from econometrics. To be fair since that 2014 set of slides you don’t hear people in the machine learning space mention econometrics that often. Most people talk about Bayesian related arguments.

On a totally separate note for this week I was really into running some of the Meta AI Llama models on my desktop locally [3]. You could go out and read about the new Code Llama which is an interesting model trained and focused on coding [4]. A ton of researchers got together and wrote a paper about this new model called, “Code Llama: Open Foundation Models for Code” [5]. That 47 page missive was shared back on August 24, 2023, and people have already started to build alternative models. It’s an interesting world in the wild wild west of generative AI these days. I really did install LM Studio on my Windows workstation and run the 7 billion parameter version of Code Llama to kick the tires [6]. It’s amazing that a model like that can run locally and that you can interact with it locally using your own high end graphics card.

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=econometric+election+prediction+models&btnG=

[2] https://web.stanford.edu/class/ee380/Abstracts/140129-slides-Machine-Learning-and-Econometrics.pdf

[3] https://ai.meta.com/llama/

[4] https://about.fb.com/news/2023/08/code-llama-ai-for-coding/

[5] https://ai.meta.com/research/publications/code-llama-open-foundation-models-for-code/

[6] https://lmstudio.ai/

What’s next for The Lindahl Letter?

Week 137: Tracking political registrations
Week 138: Prediction markets & Time-series analysis
Week 139: Machine learning election models
Week 140: Proxy models
Week 141: Expert opinions

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

I read and really enjoyed the book by Nate Silver from 2012 about predictions. It’s still on my bookshelf. Strangely enough the cover has faded more than any other book on the shelf.

Silver, N. (2012). The signal and the noise: Why so many predictions fail-but some don't. Penguin.

That book from Nate is sitting just a few books over from Armstrong’s principles of forecasting. A book that I have referenced a number of times before. It will probably be referenced more as we move ahead as well. It’s a resource that just keeps on giving. Math it’s funny like that.

Armstrong, J. S. (Ed.). (2001). Principles of forecasting: a handbook for researchers and practitioners (Vol. 30). Boston, MA: Kluwer Academic.

My podcast feed for years has included the 538 podcast where I listened to Nate and Galen talk about good and bad uses of polling [1]. Sadly, it does not currently feature Nate after the recent changes over at 538. They reported on and ranked a lot of polling within the 538 ecosystem of content. Model talk and the good or bad use of polling were staples in the weekly pod journey. I really thought at some point they would take all of that knowledge about reviewing, rating, and offering critiques of polling to do some actual polling. Instead they mostly offered polling aggregation which is what we are going to talk about today. On the website they did it really well and the infographics they built are very compelling.

Today setting up and running a polling organization is different from before. A single person could run a large amount of it thanks to the automation that now exists. An organization with funding to set up automation and run the polling using an IVR and some type of dialogue flow [2]. Seriously, you could build a bot setup that placed calls to people and completed a survey in a very conversational way. That still runs into the same problem that phone survey methods are going to face. I screen out all non-contact phone calls and I’m not the only person doing that. Cold calls are just not effective for business or polling in 2023 and the rise of phone assistants that can effectively block out noise are going to make the phone methodology even harder to effectively utilize.

It’s hard to make a hype based drum roll on the written page. You are going to have to imagine it for me to get ready for this next sentence. Now that you are imagining that drum roll… Get ready for a year of people talking about AI and the 2024 election. It probably won’t get crypto bad in terms of the hype trane showing up to nowhere, but it will get loud. I’m going to contribute to that dialogue, but hopefully in the softest possible way. Yeah, I’m walking right into that by reflecting on the outcome of my actions while simultaneously writing about them during this missive.

You can see an article from way back in November 2020 talking about how AI does show some potential to gauge voter sentiment [3]. That was before all of the generative AI and agent hype started. Things are changing rapidly in that space and I’m super curious about what can actually be accomplished in that space. I’m spending time every day learning about this and working on figuring out ways to implement this before the next major presidential election in 2024. An article from The Atlantic caught my attention as it talked about how nobody responds to polls anymore and started to dig into what AI could possibly do in that space, microtargeting, and Kennedy (1960) campaign references [4]. That was an interesting read for sure but you could veer over to VentureBeat to read about how AI fared against regular pollsters in the 2020 election [5]. That article offered a few names to watch out for and dig into a little more including KCore Analytics, expert.ai, and Polly.

We will see massive numbers of groups purporting to use AI in the next election cycle. Even The Brooking Institute has started to share some thoughts on how AI will transform the next presidential election [6]. Sure you could read something from Scientific American where people are predicting that AI could take over and undermine democracy [7]. Dire predictions abound and those will probably also accelerate as the AI hype train pulls up to election station during the 2024 election cycle [8][9]. Some of that new technology is even being deployed into nonprofits to help track voters at the polls [10].

Footnotes:

[1] https://projects.fivethirtyeight.com/polls/

[2] https://cloud.google.com/contact-center/ccai-platform/docs/Surveys

[3] https://www.wsj.com/articles/artificial-intelligence-shows-potential-to-gauge-voter-sentiment-11604704009

[4] https://www.theatlantic.com/technology/archive/2023/04/polls-data-ai-chatbots-us-politics/673610/

[5] https://venturebeat.com/ai/how-ai-predictions-fared-against-pollsters-in-the-2020-u-s-election/

[6] https://www.brookings.edu/articles/how-ai-will-transform-the-2024-elections/

[7] https://www.scientificamerican.com/article/how-ai-could-take-over-elections-and-undermine-democracy/

[8] https://www.govtech.com/elections/ais-election-impact-could-be-huge-for-those-in-the-know

[9] https://apnews.com/article/artificial-intelligence-misinformation-deepfakes-2024-election-trump-59fb51002661ac5290089060b3ae39a0

What’s next for The Lindahl Letter?

Week 136: Econometric election models
Week 137: Tracking political registrations
Week 138: Prediction markets & Time-series analysis
Week 139: Machine learning election models
Week 140: Proxy models

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Last week we started to mess around with some methods of doing sentiment analysis and setting up some frameworks to work on that type of effort. This week we take a little different approach and are going to look at an election model. I’m actively working on election focused prompt based training for large language models for better predictions. Right now I have access to Bard, ChatGPT, and Llama 2 to complete that training. Completing that type of training requires feeding election models in written form as a prompt for replication. I have been including the source data and written out logic as a part of the prompt as well.

Party registration drives the signal. Everything else is noise. That is what I expected to see within this model. It was the headline that could have been, but sadly could not be written. It turns out that this hypothesis could be tested. You can pretty easily try to view the results as a March Madness college basketball style bracket. Accepting that chalk happens or to be put more bluntly the higher ranked seeds normally win. Within the NCAA tournament things are more sporting and sometimes major upsets occur. Brackets are always getting busted. That is probably why they have ended up branding it as March Madness. Partisan politics are very different in terms of the chalk being a lot more consistent. Sentiment can change over time and sometimes voter registration does not accurately predict the outcome.

We are going to move into the hypothesis testing part of the process. This model accepts a bi-model two party representation of political parties with an assumption that generally the other parties are irrelevant to predicting the outcome. The chalk model for predicting elections based on registration reads like this, the predicted winner = max{D,R} where D = registered democrats and R = registered republicans at the time of election. For example, the State of Colorado in December of 2020 that would equate to the max{1127654,1025921} where registered Democrats outnumber registered Republicans [1]. This equation accurately predicted the results of the State of Colorado during the 2020 presidential election. 30 states report voter statistics by party with accessible 2020 archives. Using the power of hindsight we can test the chalk model for predicting elections against the results of the 2020 presidential elections.

Several internet searches were performed using Google with the search, “(state name) voter registration by party 2020.” Links to the referenced data are provided for replication and or verification of the data. Be prepared to spend a little time completing a verification effort as searching out the registered voter metric for each of the states took about 3 hours of total effort. It will go much faster if you use the links compared to redoing the search from scratch. Data from November of 2020 was selected when possible. Outside of that the best fit of the data being offered was used.

Alaska max{78664,142266}, predicted R victory accurately [2]
Arizona max{1378324,1508778}, predicted R victory in error [3]
California max{10170317,5334323}, predicted D victory accurately [5]
Colorado max{1127654,1025921}, predicted D victory accurately [6]
Connecticut max{850083,480033}, predicted D victory accurately [7]
Delaware max{353659,206526}, predicted D victory accurately [8]
Florida max{5315954,5218739}, predicted D victory in error [9] * The data here might have been lagging to actual by 2021 it would have been accurate at max{5080697,5123799}, predicting R victory
Idaho max{141842,532049}, predicted R victory accurately [10]
Iowa max{699001,719591}, predicted R victory accurately [11]
Kansas max{523317,883988}, predicted R victory accurately [12]
Kentucky max{1670574,1578612}, predicted D victory in error [13] * The data here might have been lagging to actual voter sentiment. The June 2023 numbers flipped max{1529360,1593476}
Louisiana max{1257863,1020085}, predicted D victory in error [14,15]
Maine max{405087,321935}, predicted D victory accurately [16]
Maryland max{2294757,1033832}, predicted D victory accurately [17]
Massachusetts max{1534549,476480}, predicted D victory accurately [18]
Nebraska max{370494,606759}, predicted R victory accurately [19]
Nevada max{689025,448083}, predicted D victory accurately [20]
New Hampshire max{347828,333165}, predicted D victory accurately [21]
New Jersey max{2524164,1445074}, predicted D victory accurately [22]
New Mexico max{611464,425616}, predicted D victory accurately [23]
New York max{6811659,2965451}, predicted D victory accurately [24]
North Carolina max{2627171,2237936}, predicted D victory in error [25,26]
Oklahoma max{750669,1129771}, predicted R victory accurately [27]
Oregon max{1043175,750718}, predicted D victory accurately [28]
Pennsylvania max{4228888,3543070}, predicted D victory accurately [29]
Rhode Island max{327791,105780}, predicted D victory accurately [30]
South Dakota max{158829,277788}, predicted R victory accurately [31]
Utah max{250757,882172}, predicted R victory accurately [32]
West Virginia max{480786,415357}, predicted D victory in error [33]
Wyoming max{48067,184698}, predicted R victory accurately [34]

This model predicting a winner with the max(D,R) ended up with incorrect prediction outcomes in 6 states during the 2020 presidential election cycle including Arizona, Florida, Kentucky, Louisiana, North Carolina, and West Virginia. 5 of these states based on voter registration data should have yielded D victory, but did not perform that way in practice. Arizona worked the other direction. Some of these states clearly have shifted voter registration and I have added some notes to show those changes in Kentucky and Florida. It is possible that in both of those states voter registration was a lagging indicator compared to the sentiment of votes cast. The chalk model for predicting elections ended up being 24/30 or 80% accurate.

You can imagine that I was expecting to see a much more accurate prediction of elections out of this chalk model. Again, calling back to that March Madness and thinking about what it means to have a clear path to victory for registered voters, but it not working out that way. So, that is why we tested this hypothesis of the chalk model. You can obviously see here that it is accurate most of the time, but not all the time. It’s something that we will continue to dig into as I look at some other models and I do some other tests with voter data while we are looking at elections and how they intersect with AI/ML. The next step here would be to see if a model can be developed with enough agency through plugins to be able to conduct this effort without intervention in an automated way based on a single prompt.

Footnotes:

[1] https://www.sos.state.co.us/pubs/elections/VoterRegNumbers/2020/December/VotersByPartyStatus.pdf or https://www.sos.state.co.us/pubs/elections/VoterRegNumbers/2020VoterRegNumbers.html

[2] https://www.elections.alaska.gov/statistics/2020/SEP/VOTERS%20BY%20PARTY%20AND%20PRECINCT.htm#STATEWIDE

[3] https://azsos.gov/sites/default/files/State_Voter_Registration_2020_General.pdf

[4] https://azsos.gov/elections/results-data/voter-registration-statistics

[5] https://elections.cdn.sos.ca.gov/ror/15day-gen-2020/county.pdf

[6] https://www.sos.state.co.us/pubs/elections/VoterRegNumbers/2020/December/VotersByPartyStatus.pdf

[7] https://portal.ct.gov/-/media/SOTS/ElectionServices/Registration_and_Enrollment_Stats/2020-Voter-Registration-Statistics.pdf

[8] https://elections.delaware.gov/reports/e70r2601pty_20201101.shtml

[9] https://dos.myflorida.com/elections/data-statistics/voter-registration-statistics/voter-registration-reports/voter-registration-by-party-affiliation/

[10] https://sos.idaho.gov/elections-division/voter-registration-totals/

[11] https://sos.iowa.gov/elections/pdf/VRStatsArchive/2020/CoNov20.pdf

[12] https://sos.ks.gov/elections/22elec/2022-11-01-Voter-Registration-Numbers-by-County.pdf

[13] https://elect.ky.gov/Resources/Pages/Registration-Statistics.aspx

[14] https://www.sos.la.gov/ElectionsAndVoting/Pages/RegistrationStatisticsStatewide.aspx

[15] https://electionstatistics.sos.la.gov/Data/Registration_Statistics/statewide/2020_1101_sta_comb.pdf

[16] https://www.maine.gov/sos/cec/elec/data/data-pdf/r-e-active1120.pdf

[17] https://elections.maryland.gov/pdf/vrar/2020_11.pdf

[18] https://www.sec.state.ma.us/divisions/elections/download/registration/enrollment_count_20201024.pdf

[19] https://sos.nebraska.gov/sites/sos.nebraska.gov/files/doc/elections/vrstats/2020vr/Statewide-November-2020.pdf

[20] https://www.nvsos.gov/sos/elections/voters/2020-statistics

[21] https://www.sos.nh.gov/sites/g/files/ehbemt561/files/documents/2020%20GE%20Election%20Tallies/2020-ge-names-on-checklist.pdf

[22] https://www.state.nj.us/state/elections/assets/pdf/svrs-reports/2020/2020-11-voter-registration-by-county.pdf

[23] https://klvg4oyd4j.execute-api.us-west-2.amazonaws.com/prod/PublicFiles/ee3072ab0d43456cb15a51f7d82c77a2/aa948e4c-2887-4e39-96b1-f6ac4c8ff8bd/Statewide%2011-30-2020.pdf

[24] https://www.elections.ny.gov/EnrollmentCounty.html

[25] https://vt.ncsbe.gov/RegStat/

[26] https://vt.ncsbe.gov/RegStat/Results/?date=11%2F14%2F2020

[27] https://oklahoma.gov/content/dam/ok/en/elections/voter-registration-statistics/2020-vr-statistics/vrstatsbycounty-11012020.pdf

[28] https://sos.oregon.gov/elections/Documents/registration/2020-september.pdf

[29] https://www.dos.pa.gov/VotingElections/OtherServicesEvents/VotingElectionStatistics/Documents/2020%20Election%20VR%20Stats%20%20FINAL%20REVIEWED.pdf

[30] https://datahub.sos.ri.gov/RegisteredVoter.aspx

[31] https://sdsos.gov/elections-voting/upcoming-elections/voter-registration-totals/voter-registration-comparison-table.aspx

[32] https://vote.utah.gov/current-voter-registration-statistics/

[33] https://sos.wv.gov/elections/Documents/VoterRegistrationTotals/2020/Feb2020.pdf

[34] https://sos.wyo.gov/Elections/Docs/VRStats/2020VR_stats.pdf

What’s next for The Lindahl Letter?

Week 135: Polling aggregation
Week 136: Econometric models
Week 137: Time-series analysis
Week 138: Prediction markets
Week 139: Machine learning election models

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

I’m still spending some time digging into notebooks. This time around the topic as you might have guessed for that inquiry is figuring out how to automate a survey or more pointedly some sentiment analysis. People are building automated phone surveys with interactive voice response (IVR) systems. The next wave of this technology will be hard to tell if it is a person or a bot. Seriously, those systems are going to keep getting better at a rapid pace. The new wave of generative large language models are going to make outbound call surveys better and probably more plentiful. When the outbound call survey plugins roll in for ChatGPT, Bard, and anybody can build one for Llama 2 if they are willing to serve up a custom model. At the same time, I’m entirely sure (and hopeful) that people will be using more advanced technology to block those phone calls as well.

All right, let’s shift away from considering phone calls and start to dig around into some of the automated sentiment analysis techniques that exist. We are starting to see frameworks where you can ask these new series of ChatGPT type services to act as an agent for you and complete some type of tasking. One of the things that would be interesting to ask that type of agent to complete would be to evaluate sentiment about something. I’m sure brands would like to have some automated brand evaluation methods. This will inevitably be used for politics as well. Right now we are not to the point where everybody has plugins that allow agency for ChatGPT or other toolings at the moment. That really is coming very soon as far as I can tell. Between lower energy costs and the solid platforms being built, those changes together may enable the compute for this type of interaction to happen in very conversational ways with a computer in the next 5 years.

Right now you could start by messing around with Google Colab and use the forms options they have [1]. Completing some really solid sentiment analysis may require more than just focusing on the Google Colab environment. You may want to go out to somewhere like Hugging Face to get some information on how to do this with some python code [2]. A nice place to go along this journey to get some sentiment analysis done would be to venture out to the world of Kaggle and access one of their notebooks for sentiment analysis [3].Another notebook that I liked was from notebook dot community and it shares some of the natural language processing basic of sentiment analysis in really good chunks that make it easy to understand the mechanics of how things are happening [4]. At this point in the process you are probably ready to start to do some work on your own to complete some sentiment analysis and I found the right Google Colab notebook for you to start work on designing your own sentiment analysis tool [5].

I was talking to somebody recently about the future of AI. My explanation may have not been what they expected to hear. Within the next couple of years I expect to see a lot of companies spin up and a lot of different creativity happening in the space. All of that will end up settling out into a commodified built in series of advancements. A lot of new features for applications and tooling will spin out off the great wave of AI builds that are happening now, but it will end up feeling more commonplace and build into technologies that exist now. These technologies will mostly supplement or augment things as we move forward. You will have to know how to interact with and work with the generative models that exist, but they are going to be built into the platforms and systems that end up winning out within the business world in the next couple of years.

Content consumed this week:

“The Impact of chatGPT talks (2023) - Keynote address by Prof. Yann LeCun (NYU/Meta)”

“Llama 2: Open Foundation and Fine-Tuned Chat Models” https://arxiv.org/pdf/2307.09288.pdf

“Stanford CS229 Machine Learning I Naive Bayes, Laplace Smoothing I 2022 I Lecture 6”

“MASTER Auto-GPT in under 60 MINUTES | Ultimate Guide”

Footnotes:

[1] https://colab.research.google.com/notebooks/forms.ipynb

[2] https://huggingface.co/blog/sentiment-analysis-python

[3] https://www.kaggle.com/code/omarhassan1406/notebook-for-sentiment-analysis

[4] https://notebook.community/n-kostadinov/sentiment-analysis/SentimentAnalysis

[5] https://colab.research.google.com/github/littlecolumns/ds4j-notebooks/blob/master/investigating-sentiment-analysis/notebooks/Designing%20your%20own%20sentiment%20analysis%20tool.ipynb

What’s next for The Lindahl Letter?

Week 134: The chalk model for predicting elections
Week 135: Polling aggregation
Week 136: Econometric models
Week 137: Time-series analysis
Week 138: Prediction markets

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Thank you for tuning in to this audio only podcast presentation. This is week 132 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “Synthetic data notebooks.”

People are totally working on this one actively which I thought was pretty interesting. I have a general interest in how to create synthetic data using notebooks as it helps to provide people with lessons on how to do it from an educational based perspective. Really solid automated testing process may include some of this in the development process. It makes automation even more amazing as a part of the process. It looks like the folks over at Towards AI released a nice guide to synthetic data that is geared at beginners in March of 2023 [1]. That guide walked through some of the concepts and a few pieces of information like how some report from a researcher at Gartner showed that half of future AI data will end up being synthetic data. Don’t worry I went out and found sourcing for that from Gartner and Alexander Linden who estimated by 2030 that synthetic data would outpace the regular data [2].

Those future considerations aside, the reason that is happening is that most people are going to be expanding their datasets with synthetic data to help them do training and work with models [3]. We are pretty far into the second paragraph and you might be wanting to access a couple of Google Colab notebooks to be able to do some of this yourself. Don’t worry that is about to happen for you. The team over at gretel AI shared a couple of notebooks that you can use for this type of effort:

https://colab.research.google.com/github/gretelai/gretel-synthetics/blob/master/examples/synthetic_records.ipynb

The first notebook had all sorts of errors and would not work.

https://colab.research.google.com/github/gretelai/gretel-blueprints/blob/main/docs/notebooks/create_synthetic_data_from_a_dataframe_or_csv.ipynb

The second one required a Gretel API key to get going which was a lot less fun than it could have been without that part of the equation. I went out to the website over at

https://gretel.ai/

and they have some free elements. I got into the dashboard they have pretty easily and started to look around to see what they are offering [4]. I went out to YouTube and found a 12 minute video from one of the co-founders Alex Watson showing how to do this effort. They did quickly show how to get the API key for the above notebook.

I really did follow those instructions to get that magic API key which totally worked in the 2nd Google Colab notebook link from above. I stepped through the entire notebook in about 15 minutes and was able to see the process of synthetic data generation from a dataframe or CSV which was exciting to watch and learn about in a notebook. The main model training took 7 minutes so don’t expect that it will happen in just a click.

Maybe you wanted to see somebody else do some generation of synthetic data in Google Colab on YouTube. You can see YData work with a fabric environment and work in a notebook. They had 44 subscribers and the video had 28 views before I shared this link for your enjoyment.

You could also check out this other YouTube video from The Next Phase team that shows more information about “Synthetic data generation with CTGAN” in a Google Colab notebook as well [5].

Maybe you wanted to switch gears a bit and learn a little bit about how to create 8-bit audio samples [6]. I’m going to share one more article here that walks through how to generate datasets as I thought it was actually pretty good [7]. I’m going to close this one out with a zoom out to what some people think is the future of these synthetic data driven creations which is in fact the politician of the open internet and eventual model collapse [8]. People have even recently gone as far as to say copies of the internet before all this generated content are worth more for training than the derivatives. We will see what happens soon. Oftentimes in the machine learning spaces people have used randomness, chaos, or other techniques of shifting things around to overcome blocks. I’ll be curious to see if something is developed to overcome these potential model collapse elements.

Footnotes:

[1] https://towardsai.net/p/machine-learning/a-beginners-guide-to-synthetic-data

[2] https://www.gartner.com/en/newsroom/press-releases/2022-06-22-is-synthetic-data-the-future-of-ai

[3] https://towardsdatascience.com/generating-expanding-your-datasets-with-synthetic-data-4e27716be218

[4] https://console.gretel.ai/use_cases/cards/use-case-synthetic/projects

[5] https://colab.research.google.com/drive/18vavq2Kt8HqhSnZvvFxJUc-70NCPeDU_?usp=sharing

[6] https://medium.com/mlearning-ai/python-machine-learning-gans-synthetic-data-and-google-colab-5bb43491a8c7

[7] https://medium.com/nerd-for-tech/synthetically-generate-datasets-using-deep-learning-c1f6ee7a0990

[8] https://arxiv.org/pdf/2305.17493.pdf

What’s next for The Lindahl Letter?

Week 133: Automated survey methods
Week 134: Make a link based news report automatically
Week 135: Saving some notebooks every day
Week 136: What if July was startup month? 31 days for 31 ideas
Week 137: The battle is about having the idea

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

We are not to the point where some AI agent is going to be able to do this type of bulk image improvement by a verbal command. I really wanted to be able to just say, “ok computer, edit the photos in this folder to improve them and ping me when you are done.” This use case will probably show up at some point, but now is not that point in the timeline. The future of this type of computer usage is close, but not here just yet.

Over the last couple of weeks, I have spent some time messing around with ways to complete some bulk image improvement via either scripting or applications. It turns out a lot of people use Adobe Lightroom to edit images. Adobe is really spending a lot of time and effort to share the AI powered innovations they have built into the product. Fundamentally, it is not that surprising that Adobe would try to use adoption and the major amount of brand equity they have as a moat to deflect away from the entry of new products into the photo editing process. A lot of room exists for a product to come in and be disruptive in this space.

My use case here is really simple. I want to just have a folder of photos and via either scripting or sharing the folder to an application have those images improved via all these brand new AI tools we keep hearing about. I’m asking for a low friction solution to just have a tool or some tooling do a bit of work to improve the photos being taken. You might be thinking, but I thought Google Photos would basically do this for you if you use that service. It does not really work that way. You can go in and individually update photos with a single click, but it won’t just do the work on the photos in the background. Actually getting Google Photos to work in the background would essentially involve writing scripting to bounce each photo against a vision based API for improvement. That sounds tedious and is not a low friction solution.

Within the marketplace generally a number of things have popped up to compete with the Adobe products including some interesting ones:

Fotor (100 employees, started 2012)
Skylum (150 employees, started 2008)

Searching for Fotor in the Google Play store ended up directing me to something called “AI Photo Editor, Collage-Fotor” which was not what I expected [1]. The installation was clearly a Fotor owned application and it is apparently used by more than 10 million people with a 4.3 out of 5 star review rating. Before using the product I ended up backing out and just searching for “AI photo editor” in Google Play which was interesting [2]. That search produced a ton of options. Way more options that I was expecting to see. I know that a lot of people use smartphones or tablets and don’t really even open a computer to do editor or general computing work. Right now I’m writing this missive on a double monitor desktop setup going in the total other direction of things.

The application based editing route is one thing, but I wanted to see if some other method existed. I had hoped that something in the Google ecosystem would make this task a little bit easier. During the course of a few more searches and looking around I did end up watching this video from Josiah Blizzard who used the Batch AI Adobe Lightroom plugin to edit 1,000 photos per minute. This seemed promising. It was a way more entertaining video that I expected.

I thought it might be interesting to check out a few more videos related to Batch AI on YouTube and was able to find a few different options.

Footnotes:

[1] https://play.google.com/store/apps/details?id=com.everimaging.photoeffectstudio&hl=en_US≷=US

[2] https://play.google.com/store/search?q=AI%20photo%20editor&c=apps&hl=en_US≷=US

What’s next for The Lindahl Letter?

Week 132: Synthetic data notebooks
Week 133: Automated survey methods
Week 134: Make a link based news report automatically
Week 135: Saving some notebooks every day
Week 136: What if July was startup month? 31 days for 31 ideas

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

We are getting closer and closer to the world envisioned in Star Trek and countless other science fiction universes where you could just ask the computer to complete some sort of task or build for you. We are starting to see what would be like an AutoGPT or a plugin style version of ChatGPT with agents that work to complete some sort of request. It’s not like they have a full built in chron system with scheduling and routines. Really actionable virtual assistants are going to show up here very quickly. I’m pretty sure Google is not very far off from being able to allow their voice assistant to activate more things within routines and schedules with a real degree of agency. I’m actually fairly surprised they have not just launched that feature already. That ecosystem has the connectivity to be able to turn off and on lights or other home connected devices and connect to so many other things. What I am saying here is that you very well could have an agent do a variety of things very soon.

You can imagine that it would be fun to just ask the ChatGPT agent to write and do something with fractals. To that end I spent some time asking by prompt both Bard and ChatGPT to produce code for a Colab notebook. Strangely enough Bard never produced any code that was executable within a Colab notebook. When advised I was wanting code that would execute in a Colab environment the ChatGPT model did spit out code that would execute properly [1]. I’m adding notebooks that work to my GitHub repository on an ongoing basis.

Beyond messing around with the generative models it seemed like a fun idea to look around and see what people were doing with fractals in Google Colab notebooks. The notebooks I found were produced a while ago, but that made them no less fun to play with.

Fractal Art With Python by Dr. Mike Lam of James Madison University: https://colab.research.google.com/drive/1tSBkON1Uj0NCfYEgELdXxC-PS9eOaUQG?usp=sharing#scrollTo=p02VqSeizpUNFractal Generation with L-Systems by Paul Butlerhttps://colab.research.google.com/github/paulgb/notebooks/blob/master/source/l-systems/Fractal%20Generation%20with%20L-Systems.ipynb

I ended up wanting to just look at Python code for fractal building and manipulation. That is where I’m working with things right now in the process [2]. My base test for these new generative models relates to how well they can actively generate and manipulate fractals.

Footnotes:

[1] https://github.com/nelslindahlx

[2] https://towardsdatascience.com/creating-fractals-with-python-d2b663786da6

What’s next for The Lindahl Letter?

Week 131: Bulk imagine improvement scripting
Week 132: Synthetic data notebooks
Week 133: Automated survey methods
Week 134: Make a link based news report automatically
Week 135: Saving some notebooks every day

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Working out of notebooks is an easy and lightweight way to just mess around with code. One of my all time favorite ways to go about doing that is in the Colab research space provided by the team over at Google [1]. They have an offering for a pro subscription, but you can get in and use things without a payment plan. Sure the promise of faster GPUs, more memory, and longer runtimes is enticing. If you really get deeply into the process of training models, then it might make sense for you to make arrangements to pay for some compute. A lot of people have shared notebooks and you can quickly start to work on things and see code in action. The part of it that is so appealing to me is that you can look at a very distinct block of action and run it in the notebook environment without having to worry about anything else in the stack.

Breaking things into smaller understandable pieces is absolutely fantastic for learning about things. It’s also nice to just jump in and be able to execute the code. Here very soon according to the Google Blog page for developers we are going to see AI-powered coding free of charge within the Colab space [2]. You will be able to open the notebook and instead of adding code or text you will have access to another option to generate. That is where the magic will be located and you will be able to very rapidly use Colab in a generative way. I think it is going to bridge the gap between interacting with generative models to accomplish things and the current sort of limitations. Inside that argument would be the backward linkages to the Google ecosystem where instead of plugins or managing extensions the system will just have extensibility into a massive and mature ecosystem.

I have been waiting since that May 17, 2023 post for the feature to show up. Literally, I just keep checking on an almost daily basis to see when we can get to the point of rock and rolling the generative creation of code from the PaLM 2 family of code models [3]. At this point, I’m going to admit that my favorite part of this whole thing is that they have called it Codey. Maybe the controversial Clippy was just ahead of its time, but I think Codey will be just right as a part of a larger ecosystem of generative coding efforts that are about to take flight. You will see that very little traffic occurred on Twitter about this news [4]. In terms of education and getting people going with generative code generation this is going to be epic. You will see my contributions to GitHub of notebooks go from sparse to daily [5]. I have started dropping some code to make fractals and do some various types of math into a new repository [6].

All of the security for Google Colab notebooks is managed by those teams. You certainly have control over your own private notebook being saved to your Google drive. Access to the data and notebooks is controllable and the team over at Google is spinning up and then spinning down virtual machines to do this work. Most of the process is entirely ephemeral and to the end user is fundamentally just a browser. The FAQ page does not really spend much time addressing security [7]. It’s a place to very quickly be able to execute code from a browser. Using it for educational purposes makes total sense. It’s a great place to share code and learn about how that code works. It has enough power to do some basic machine learning and to that end it is a pretty awesome experience. You are not going to end up using this as a production runtime for any sort of tasking. One of the things I’m actually curious about in the future is what happens when people start using ChatGPT plugins or agents that go out and farm compute from environments like this type of browser based engagement.

Footnotes:

[1] https://colab.research.google.com/

[2] https://blog.google/technology/developers/google-colab-ai-coding-features/

[3] https://9to5google.com/2023/05/17/google-colab-codey/

[4] https://twitter.com/search?q=colab%20codey&src=typed_query

[5] https://github.com/nelslindahlx

[6] https://github.com/nelslindahlx/fractals

[7] https://research.google.com/colaboratory/faq.html

What’s next for The Lindahl Letter?

Week 130: Build captain fractal using Colab
Week 131: Bulk imagine improvement scripting
Week 132: Synthetic data notebooks
Week 133: Automated survey methods
Week 134: Make a link based news report automatically

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

The topic for this week was a weighty one for sure. It’s one that I think will functionally happen, but I do not believe will be real in terms of actual practice. We are seeing a lot of emphasis on using these new AI systems that are popping up everywhere. People are not able to build these systems from scratch anymore. Using code from somebody else is becoming more and more a part of the process. When you look at how to democratize AI systems it is certainly about spreading usage and making these systems omnipresent. Security is structurally something that is important in this situation, but it remains something that generally happened before you got involved. In that scenario, really deeply considering how your AI system security posture is going to be invoked you have to consider both forward looking actions and the backward linkages within these products both in terms of the build and design.

Within the calling of an API in this space it certainly is possible to consider what data is going out and what is coming back. You can pretty clearly understand what happens with the data after it is sent over and what is going to happen with it after the fact. My searches on Google Scholar for articles related to AI security were not super exciting [1]. That search with the brackets around the search term yielded about 2,900 items. Taking the brackets off opened up the results to over 3 million [2]. This may very well be a space where the academic content has not caught up with where the technology happens to be at the moment. A lot of academic research in the space is occurring, but since the papers are not really about or from production implementations the topics of how to secure, manage, and deploy are distinctly lacking.

At this point in the story, I got super interested in reading about comparisons of protocols for distributed social networking. Things like the AT protocol vs activitypub [3]. Oddly, one of the best lists that I encountered was on Wikipedia where somebody is clearly keeping track of open projects related to this one [4]. You might be wondering if that list includes 30+ projects with tacking and information and you will find out that it totally does. However, being able to take a look at such a large list actually made me a little bit concerned about any of these protocols actually becoming dominant and taking over in practice.

Let’s take this post in a different, more meta direction. Ok, here are all the spilled beans on that one, I ended up making an executive decision on how to manage my writing backlog. It was a super disruptive and spur of the moment decision at this point in the process. I went ahead and at the week 128 issue (which is this post) pasted into the backlog the new list of 49 items from a Google Keep note. That happened as a result of a really productive day that happened last week. During one of the vacation days at the beach recently, I started making a list of topics I would like to spend some time either researching or writing about.

It turns out that list ended up including 49 total items. With the addition of those 49 items, my backlog is now tracking out to week 218. Switching things up like that means I’m going to struggle with updating the next five weeks of forward looking items on a bunch of blocks of content, but that is a solvable problem in terms of editing and writing. It’s just a bit annoying and somewhat time consuming vs. creating any really ongoing problematic situation. That does mean that starting in July you will start to get the benefit of this new backlog and strictly speaking you are going to end up waiting about a year to return to the previously structured program content.

While we are zoomed out and considering some of the more meta implications of things I’ll note that I have kept the podcast part of this effort going. Readership numbers indicate that it is a smaller portion of things compared to the standard text based link clicking. About 20% of the ongoing traffic for this writing project seems to be related to the audio based podcast. The recording process is now pretty streamlined and I have it as a part of my overall creative weekend routine. These weekly blocks of writing won’t grow long enough to make recording the audio for them problematic in terms of a time commitment.

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=%22ai+security%22&btnG=

[2] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=ai+security&btnG=

[3] https://www.theverge.com/2023/4/20/23689570/activitypub-protocol-standard-social-network

[4] https://en.wikipedia.org/wiki/Comparison_of_software_and_protocols_for_distributed_social_networking

What’s next for The Lindahl Letter?

Week 129: How do you use Colab in a generative way?
Week 130: Build captain fractal using Colab
Week 131: Bulk imagine improvement scripting
Week 132: Synthetic data notebooks
Week 133: Automated survey methods

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Things have been happening with Google DeepMind. A lot of news has been dropping since I started crafting this post. I don’t want to write about the news of the moment on this one. The context and analysis within this post is going to try to be deeper than a base reaction to that recent news. The Verge (and a ton of other people) reported that, “Google’s big AI push will combine Brain and DeepMind into one team” [1]. Apparently, some drama exists around those two organizations being combined. We will almost certainly see some books about this from the principal players. Let’s hope they turn out as good as the 2020 memoir “Uncanny Valley” by Anna Wiener. Some of them won’t be that dynamic, but they should be.

I spent some time reading content from DeepMind. They have a solid research page [2]. It highlights some interesting things including AlphaFold, WaveNet, and AlphaGo [3][4][5]. They have a lot of content out on GitHub in a variety of repositories [6]. They have over 190 repositories with a variety of code you could go take a look at. You may even notice that they have signed the “Lethal Autonomous Weapons Pledge” [7][8].

Some good conversation is occurring about what happens when AI hallucinates, bias bounties, new bug bounties, and how red teams work [9]. You could also find out a little more about how the GitLab AI-based security feature helps to scan codebases for vulnerabilities [10]. You could go check out the Google Cloud Security Podcast EP52 Securing AI with DeepMind CISO Vijay Bolina from February 14, 2022 [11].

You really have to consider that even the best pair programming with assistive technology could introduce security vulnerabilities [12]. We could probably spend an entire week looking at the medical artificial intelligence system or more clearly named Med-PaLM 2 that was released [13]. They also shared an interesting method to discover faster matrix multiplication algorithms which I thought was rather interesting [14].

Don’t worry I did go out and grab a couple (really 4) scholarly papers to share with you this week:

Beattie, C., Leibo, J. Z., Teplyashin, D., Ward, T., Wainwright, M., Küttler, H., ... & Petersen, S. (2016). Deepmind lab. arXiv preprint arXiv:1612.03801. https://arxiv.org/pdf/1612.03801.pdf

Powles, J., & Hodson, H. (2017). Google DeepMind and healthcare in an age of algorithms. Health and technology, 7(4), 351-367. https://link.springer.com/article/10.1007/s12553-017-0179-1

Tassa, Y., Doron, Y., Muldal, A., Erez, T., Li, Y., Casas, D. D. L., ... & Riedmiller, M. (2018). Deepmind control suite. arXiv preprint arXiv:1801.00690. https://arxiv.org/pdf/1801.00690

Evans, R., & Gao, J. (2016). Deepmind ai reduces google data centre cooling bill by 40%. DeepMind blog, 20, 158. https://www.deepmind.com/blog/deepmind-ai-reduces-google-data-centre-cooling-bill-by-40

Footnotes:

[1] https://www.theverge.com/2023/4/20/23691468/google-ai-deepmind-brain-merger

[2] https://www.deepmind.com/research

[3] https://www.deepmind.com/research/highlighted-research/alphafold

[4] https://www.deepmind.com/research/highlighted-research/wavenet

[5] https://www.deepmind.com/research/highlighted-research/alphago

[6] https://github.com/deepmind

[7] https://www.deepmind.com/safety-and-ethics

[8] https://futureoflife.org/open-letter/lethal-autonomous-weapons-pledge/?cn-reloaded=1

[9] https://www.theregister.com/2023/04/26/is_your_ai_hallucinating/

[10] https://www.spiceworks.com/tech/artificial-intelligence/news/gitlab-launches-ai-security-feature/

[11] https://cloud.withgoogle.com/cloudsecurity/podcast/ep52-securing-ai-with-deepmind-ciso/

[12] https://www.techtarget.com/searchsoftwarequality/news/252523049/Developers-beware-AI-pair-programming-comes-with-pitfalls

[13] https://www.verdict.co.uk/google-lauches-new-medtech-ai-system-amid-calls-for-greater-data-security/

[14] https://venturebeat.com/ai/top-5-stories-of-the-week-deepmind-and-openai-advancements-intels-plan-for-gpus-microsofts-zero-day-flaws/

What’s next for The Lindahl Letter?

Week 128: Democratizing AI system security
Week 129: Snapchat Security
Week 130: Generative model security
Week 131: Profiling Microsoft Azure security
Week 132: Engineering discipline security

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Something is going to have to give here pretty soon for Hugging Face or they are going to get left behind by the extended model, plugin driven, and now internet connected GPT models that some of the biggest platforms are working to take mainstream. We are seeing the real time development creating extensibility into actions from companies like OpenAI and their partner Microsoft that are going to be a foundational groundwork for how connectivity works within these models. This ecosystem is going to be something so highly proprietary and interconnected between a set of foundational companies that no ability to directly open source a competitor is going to exist. Part of that will be due to the payment models that are going to get setup as a foundational services layer starts to get setup.

People have been really excited about Hugging Face for some time now. Since 2016 the private company Hugging Face has worked to share machine learning content with the world. You can easily get to some of the training courses they have set up on NLP and Deep RL [1]. They are pretty good training courses. People have really dug into models and spaces from Hugging Face to help democratize NLP models. At one point, everybody was checking in with what Hugging Face was up to and they made a very large space in the AI and ML space. You can easily go out to Google Scholar and see a bunch of academic papers that reference Hugging Face either as two words or sometimes one word [2]. I spent some time looking for papers with a decent number of citations. Here are 5 that were selected:

Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., ... & Rush, A. M. (2019). Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771. https://arxiv.org/pdf/1910.03771

Jiang, W., Synovic, N., Hyatt, M., Schorlemmer, T. R., Sethi, R., Lu, Y. H., ... & Davis, J. C. (2023). An empirical study of pre-trained model reuse in the hugging face deep learning model registry. arXiv preprint arXiv:2303.02552. https://arxiv.org/pdf/2303.02552

Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., ... & Rush, A. M. (2020, October). Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations (pp. 38-45). https://aclanthology.org/2020.emnlp-demos.6.pdf

Pfeiffer, J., Rücklé, A., Poth, C., Kamath, A., Vulić, I., Ruder, S., ... & Gurevych, I. (2020). Adapterhub: A framework for adapting transformers. arXiv preprint arXiv:2007.07779. https://arxiv.org/pdf/2007.07779

Zhang, Y., Sun, S., Galley, M., Chen, Y. C., Brockett, C., Gao, X., ... & Dolan, B. (2019). Dialogpt: Large-scale generative pre-training for conversational response generation. arXiv preprint arXiv:1911.00536. https://arxiv.org/pdf/1911.00536.pdf%7D.

You can get a feel for what the Hugging Face community has been working on by taking a look at their GitHub repository [3]. They have transformers, datasets, diffusers, and tools for a variety of things that people find useful. I would describe Hugging Face as a very hands-on community where you can dig in and be a part of what is going on within the space. That is the distinctive difference from some of the other larger corporations that do open source things. Most of the corporate AI labs will produce and distribute things when they are ready. Iterative tool releases and a vibrant community full of contributions are popping up all over. We are also seeing a newer trend where companies like OpenAI are releasing models and other technology via API without open sourcing the content. This pay to play API model shields the intellectual property better, but in this space people are rapidly learning from innovations and working to build alternatives.

Here are a couple of YouTube videos including one from Hugging Face:

You can dig into the security features offered by Hugging Face [4]. It says clearly on the security page that they are SOC2 Type 1 certified with a link to the AICPA page [5]. With all sources where you might download code and use it you are going to want to understand the code, be ready to keep it current, and of course be prepared to take rapid action. They are even trying to compete with ChatGPT and share more open models for that type of effort [6]. I spent some time looking around to try to find some deep assessments of the security profile for Hugging Face and have not really found what I was looking for yet.

Footnotes:

[1] https://huggingface.co/learn/nlp-course/chapter1/1

[2] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=%22hugging+face%22&btnG=

[3] https://github.com/huggingface

[4] https://huggingface.co/docs/hub/security

[5] https://us.aicpa.org/interestareas/frc/assuranceadvisoryservices/aicpasoc1report.html

[6] https://venturebeat.com/ai/hugging-face-launches-open-source-version-of-chatgpt-in-bid-to-battle-openai/

What’s next for The Lindahl Letter?

Week 127: Profiling DeepMind Security
Week 128: Democratizing AI system security
Week 129: Snapchat Security
Week 130: Generative model security
Week 131: Profiling Microsoft Azure security

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Security professionals the world over are concerned about what people like some of Samsung’s engineers might be doing with OpenAI’s ChatGPT services [1]. Let’s back up just a bit from that very real breaking news security reality and make sure to set the stage. Instead of starting with a complete corporate history about OpenAI, you knew that I would go out and search Google Scholar to find some of the key OpenAI related scholarly works [2]. Alternatively, you could surf over to the OpenAI website and find the section they have for over 100 papers and just download the PDFs from the source [3]. You could download papers like the much maligned “GPT-4 Technical Report” that was lengthy at over 100 pages, but did not go into the model mechanics people were interested in understanding [4]. Other papers are hosted on that page like the Dall-E-2 paper “Hierarchical Text-Conditional Image Generation with CLIP Latents” [5]. It is a lot of great content and you will get a sense that at one point OpenAI was sharing and building a great legacy of published content that helped people really understand where artificial intelligence was going. They also did some really awesome work building agents that competed in pretty complex video games against world class players. That part of the equation is what caught my attention and really pulled me into trying to understand OpenAI.

Here are 5 interesting papers with a solid number of citations:

Eloundou, T., Manning, S., Mishkin, P., & Rock, D. (2023). Gpts are gpts: An early look at the labor market impact potential of large language models. arXiv preprint arXiv:2303.10130. https://arxiv.org/pdf/2303.10130.pdf

Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., ... & Zhang, S. (2019). Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680. https://arxiv.org/pdf/1912.06680.pdf

Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016). Openai gym. arXiv preprint arXiv:1606.01540. https://arxiv.org/pdf/1606.01540.pdf

Gawłowicz, P., & Zubow, A. (2019, November). Ns-3 meets openai gym: The playground for machine learning in networking research. In Proceedings of the 22nd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems (pp. 113-120). https://www2.informatik.hu-berlin.de/~zubow/gawlowicz19_mswim.pdf

Zamora, I., Lopez, N. G., Vilches, V. M., & Cordero, A. H. (2016). Extending the openai gym for robotics: a toolkit for reinforcement learning using ros and gazebo. arXiv preprint arXiv:1608.05742. https://arxiv.org/pdf/1608.05742.pdf

Now that we have covered some of the decently cited scholarly content it is probably good to zoom out from that space into the broader media coverage of OpenAI. Most of it is about the risk, reward, and fear that this type of generative AI fosters. You will see that security is not at the forefront of these discussions.

From TheVerge “OpenAI co-founder on company’s past approach to openly sharing research: ‘We were wrong’”. Ok. Where do you go from there…https://www.theverge.com/2023/3/15/23640180/openai-gpt-4-launch-closed-research-ilya-sutskever-interview

You can hear Greg Brockman who was an OpenAI cofounder talk about the astonishing potential during a 30 minutes TED talk.

The CEO of OpenAI Sam Altman sat down with ABC News for 20 minutes here:

You can see a long form interview with Lex Fridman and Sam Altman here:

They have an OpenAI YouTube channel that gets a lot of views. They used to share some pretty interesting videos.

Ok at this point in our journey here, we have covered OpenAI in general and looked at what is being created. Now it’s probably best to try to explain the drama that surrounds the company as quickly as possible. Oddly, it’s not a drama focused on the technology being created that certainly has entered the public mind. Elon Musk was an initial board member of OpenAI back in 2015 [6]. It used to be a non-profit and things have changed. That is where the real drama exists about the company. Microsoft now has a multi-billion dollar investment in OpenAI [7]. You can read the Microsoft official blog where they detail that they have invested in OpenAI three times including 2019, 2021, and 2023 and is now the exclusive cloud provider [7]. You might be thinking how did Elon Musk invest or more accurately donate 100 million to OpenAI which now is clearly partnered with Microsoft [9]. It’s a confusing plot twist for sure and a lot of media coverage exists trying to explain how that exactly happened [10]. I’m sure we will learn more about what exactly happened in the next couple of years.

Let’s circle back to the security realities of using things like OpenAI’s ChatGPT system. People in this case are going to use the system to input things to the prompt. Some of that input might very well be intellectual property owned distinctly by somebody. You could ask a model to create more content in an author’s style or to extend the story of characters. It’s also entirely possible that somebody might feed in code for review or ask the model to produce code.

Footnotes:

[1] https://www.securitymagazine.com/articles/99220-cybersecurity-leaders-reflect-on-samsung-chatgpt-incidents

[2] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=openai&btnG=

[3] https://openai.com/research

[4] https://cdn.openai.com/papers/gpt-4.pdf

[5] https://cdn.openai.com/papers/dall-e-2.pdf

[6] https://finance.yahoo.com/news/openai-cofounder-elon-musk-said-142815403.html

[7] https://www.theverge.com/2023/1/23/23567448/microsoft-openai-partnership-extension-ai

[8] https://blogs.microsoft.com/blog/2023/01/23/microsoftandopenaiextendpartnership/

[9] https://fortune.com/2023/03/16/elon-musk-openai-non-profit-switch-30b-market-cap-for-profit-after-donation-chatgpt/

[10] https://www.businessinsider.com/elon-musk-defends-role-in-openai-chat-gpt-microsoft-2023-2

What’s next for The Lindahl Letter?

Week 126: Profiling Hugging Face Security
Week 127: Profiling DeepMind Security
Week 128: Democratizing AI system security
Week 129: Snapchat Security
Week 130: Generative model security

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

It's all happening. I'm pivoting to AI + Security as the central topic for The Lindahl Letter. The next weekly modules I'm going to create are all going to be pivoting to a clear security based focus each week. Content for this year of The Lindahl Letter was packaged into posts encompassing weeks 105 to 156 or 52 total blocks of content. Think year divisible by week with production spread evenly. This is the 3rd year of that type of writing effort. This is week 124 in that sequence and I have yelled pivot and brought in 11 new topics that will follow the profiles of OpenAI, Hugging Face, and DeepMind. That content pivot pretty much involved highlighting a section of the backlog and just moving it beyond week 156. I’m going to keep on working to balance the topics together to bring forward the best possible set of content for this year. Getting the best possible blocks of content brought together as a research project is essential to my work moving forward.

It’s possible that I will go back and rework some of the content from weeks 105 to 123 to include more of a security focus. Obviously, that won’t change the previously released podcast audio or published content, but it would update the content that gets put into a manuscript at the end of the year. That content gets edited and revised during the course of the publishing process so it is never exactly the same as the content that was published throughout the year in weekly installments. While I certainly strive to produce high quality prose my ability to edit to perfection on a weekly basis remains suspect. Editing for continuity and general cohesiveness is a different element within that large of a manuscript compared to producing a weekly installment. Putting content into an eBook and ultimately a hardback or paperback format means that it needs to be free of grammarian enraging distractions.

During my initial cut at introducing security related topics into the backlog 11 topics jumped out at me and got included. My backlog editing will be pretty extreme for the next couple of weeks to really get the most out of this epic content based pivot. Right now we are at the precipice of some very serious questions about how AI will impact society. Seriously, AI and society are very alive within the public mind at the moment. Understanding how we secure, interact over time, and ultimately establish security within the watershed event of modern AI development will be more important than my rather casual walk into AI in general. You will find that this will be a good pivot that ends up providing more depth and context to what is happening currently. Each of the new installments will be written with a degree of perspective that helps prevent them from being passing observation installments. Writing pure reaction content is not where I’m trying as that provides less of an advisor type function and more of an ephemeral of the moment observation.

You may well be aware that I’m still struggling with the idea that all the content that gets produced as a part of my weekly research notes could be replicated by ChatGPT in minutes. This is the 124th installment and OpenAI’s ChatGPT could easily provide alternative versions of the content I have created. Understanding how to bring larger themes together and diligently searching for the best academic articles is not something that ChatGPT does well at the moment. It’s entirely possible that the combination of content selection and prompt engineering could change the potential output. I’m pretty sure somebody will work to better adapt ChatGPT to the review of complex academic work. Having the right data sources selected for a model will be a key element of how things work moving forward. You could take the papers from my independent study ML syllabus and get a decent set of inputs for a model. However, would you add that as a layer to a current model like ChatGPT 3.5 or 4.0 or will you have focused layers for the models to consider in sequence or as a refinement step along the way. All of those things are up in the air and we will learn more about where they will ultimately end up.

As a general editorial decision here and now I may consult and work with ChatGPT, but the output in terms of writing posts for the rest of the year will just include my words. For me the novelty of including some examples of the differences between what I would write and what OpenAI’s ChatGPT would create has lost a certain degree of interest for me. Maybe the novelty or the thrill of it is gone. It took a couple months for that to happen, but now I’m going to focus on providing the best research note I can each week. Even this note focusing my content going forward signals an important change in my research trajectory and was worth devoting a week of content creation to building and sharing.

Links and thoughts:

What’s next for The Lindahl Letter?

Week 125: Profiling OpenAI Security
Week 126: Profiling Hugging Face security
Week 127: Profiling DeepMind security
Week 128: Democratizing AI systems security
Week 129: Snapchat Security

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Substack Week 123: We are wholesale oversubscribed on AI related content

Thank you for tuning in to this audio only podcast presentation. This is week 123 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “We are wholesale oversubscribed on AI related content.”

You may have noticed that the topic for the post this week changed. I called an audible and wrote a different piece of content.

All that AI content has hit a real saturation point. We are wholesale oversubscribed on AI related content and we have not even really crossed into the situation I’m concerned about where people are using the content to flood the open internet with synthetic content. Generally given the current state of the AI excitement level and some of the fear, apprehension, and concern a lot of people are writing or commenting. All that commenting has built into a very real saturation point where AI content is literally everywhere. Sam Altman of OpenAI recently noted in a Wired article that, “the Age of Giant AI Models Is Already Over” [1]. The initial release date of ChatGPT was November 30, 2022 so the cycle on this one was no more than a 6 month explosion of hype content. You can go out to Google Trends and pretty quickly get a sense of just how fast ChatGPT spun up in December of 2022 [2]. If you went out and added a comparison term within Google Trends of auto-gpt or AutoGPT, then you would get a sense of just how much more popular ChatGPT happens to be in the wild [3]. Keep in mind that since April 11, 2023 the searches for AutoGPT have taken off exponentially [4].

Major things are happening in terms of ChatGPT, AutoGPT, and people expanding how agents are able to cooperate. A paper from Park et al. that was just published on April 7, 2023 shows some very interesting use cases for simulating behavior using generative agents.

Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023). Generative Agents: Interactive Simulacra of Human Behavior. arXiv preprint arXiv:2304.03442. https://arxiv.org/pdf/2304.03442.pdf

Rewind a little bit and I thought the key Stanford University paper would be that 214 page multi author one on foundation models.

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., ... & Liang, P. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. https://arxiv.org/pdf/2108.07258.pdf

You can get a sense from the demonstration of capability and use that occurred in that paper on generative agents just a couple years after scholars circled the wagons on foundation models just how fast the field is changing. Things are changing so fast that Sam Altman above noted that the age of giant models might have arrived and be complete between the publication of those two papers. It can make conducting research a very interesting thing to contemplate. Content at the bleeding edge of AI technology today could very well encounter a context, vibeshift, or even meaning change within months. Taking that into consideration I’m going to recognize that we are approaching the halfway point on the production of this year’s Substack content. To that end, I’m starting to ponder how to improve things and really make things better in terms of the content that is being produced.

Links and thoughts:

Footnotes:

[1] https://www.wired.com/story/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over/

[2] https://trends.google.com/trends/explore?geo=US&q=chatgpt&hl=en

[3] https://trends.google.com/trends/explore?date=now%207-d&geo=US&q=chatgpt,autogpt&hl=en

[4] https://trends.google.com/trends/explore?date=2023-03-18%202023-04-18&geo=US&q=autogpt&hl=en

What’s next for The Lindahl Letter?

Week 124: Profiling OpenAI
Week 125: Profiling Hugging Face (open and collaborative machine learning)
Week 126: Profiling Deep Mind
Week 127: Democratizing AI systems
Week 128: Building dreaming into AI systems

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

I spent some time over the last couple of weeks looking for scholarly articles that reference AI as a service (AIaaS) [1]. Then I realized that things are changing so quickly in the landscape of AI these days that none of those articles even consider Auto-GPT [2]. Unless you have been on vacation and hiding from AI news recently you have been hearing about the advent of this plugin, multiagent, and task targeted implementation of a GPT model called Auto-GPT. People are using it to do things. You can see that that GitHub repository already has 50,000 stars and the demo was from March 30, 2023. In terms of open source code projects this one has taken off with rocket boosters. Right now the project has been forked almost 8,000 times. I’m trying to find a version that can be run in a jupyter notebook on Colab. This space is rapidly changing and will probably be changing rapidly over the next 6 months.

What did the OpenAI ChatGPT 4.0 model spit out on this one when prompted?

AI can be both a platform and a service, depending on how it is implemented and used. The distinction between the two is not mutually exclusive, as AI applications often span across various domains and industries. Let's explore each concept:

AI as a Platform: An AI platform provides the infrastructure, tools, and libraries necessary for developing and deploying AI applications. These platforms enable developers and data scientists to build, train, and manage AI models without having to start from scratch. Examples of AI platforms include TensorFlow, PyTorch, and OpenAI's GPT series. In this context, AI serves as a foundational technology upon which various solutions can be built.
AI as a Service (AIaaS): AIaaS refers to the delivery of AI capabilities as a service, usually through APIs or web services. Companies that offer AIaaS provide access to pre-built AI models, which clients can use without having to develop their own AI capabilities. This allows organizations to leverage AI technology without investing in infrastructure or expertise. Examples of AIaaS include Google Cloud AI, IBM Watson, and Amazon Web Services' machine learning services.

In conclusion, AI can function both as a platform and a service, depending on its implementation and usage. Many companies and developers use AI platforms to create AI services, which can then be integrated into various applications and business processes.

I thought it might be fun to follow up and prompt to see who was the best at AIaaS…

It is difficult to definitively say who the "best" AI-as-a-Service (AIaaS) provider is, as different providers excel in different areas and offer varying services that cater to specific use cases. However, some of the most prominent and reputable AIaaS providers include:

Google Cloud AI: Google offers a comprehensive suite of AI services, including natural language processing, computer vision, and machine learning. Its offerings, such as Google Cloud Vision API, Dialogflow, and AutoML, are widely used and respected for their accuracy and ease of use.
Amazon Web Services (AWS): AWS provides a range of AI services, including machine learning, computer vision, and natural language processing. Some popular offerings are Amazon Rekognition, Amazon Lex, and Amazon SageMaker.
IBM Watson: IBM's AIaaS platform, Watson, offers various services, including natural language processing, machine learning, and chatbot development. Watson Assistant and Watson Discovery are among its popular services.
Microsoft Azure: Azure offers a suite of AI services, including machine learning, natural language processing, and computer vision. Key offerings include Azure Cognitive Services, Azure Machine Learning, and Azure Bot Service.

The "best" AIaaS provider for your specific needs will depend on several factors, such as the required AI capabilities, pricing, ease of integration, and the level of customization needed. It's essential to research and evaluate each provider based on your unique requirements to make an informed decision.

Links and thoughts:

Top 4 Tweets of the week:

https://twitter.com/ericschmidt/status/1659619610063249409

https://twitter.com/CaseyNewton/status/1659579798857871360

https://twitter.com/robmay/status/1659610763005399042

https://twitter.com/DynamicWebPaige/status/1659075284891738112

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=AIaaS&btnG=

[2] https://github.com/Torantulino/Auto-GPT

What’s next for The Lindahl Letter?

Week 123: Considering open source AI
Week 124: Profiling OpenAI
Week 125: Profiling Hugging Face (open and collaborative machine learning)
Week 126: Profiling Deep Mind
Week 127: Democratizing AI systems

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

My initial take on writing an independent study based syllabus for applied AI was to find the best collection of freely available scholarly papers that somebody could read as an onramp to beginning to understand the field. That I think is a solid approach to helping somebody get going within a space that is very complex and full of content. It’s a space that is perpetually adding more content than any one person could possibly read or consume. Before you take that approach it is important to understand that one definitive textbook does exist. You certainly could go out and read it.

Russell, S. J. (2010). Artificial intelligence a modern approach. Pearson Education, Inc..

You could find the first edition, second edition, or third edition for sale on eBay or somewhere else if you wanted a physical copy of the book. The book is currently in a 4th edition run, but I don’t have a copy of that edition yet. It’s used by over 1,500 schools so a lot of copies exist out in the wild [1]. The authors Stewart Russell and Peter Norvig have shared a PDF of the bibliography for that weighty tome of AI insights as well [2]. Even with 35 pages of bibliography nobody with the name Lindahl made the cut. On a side note you can find the name Schmidhuber included twice if that sort of thing is important to you.

Let’s reset for a second here. If you are brand new to the field of AI or want to read a textbook based introduction, then you should seriously consider buying a copy of the aforementioned textbook. That is a really great way to start which has worked for tens of thousands of people. My approach here is going to be a little bit unorthodox, but it works for me. My last run at this type of effort was, “An independent study based introduction to machine learning syllabus for 2022” and you can find it out on Google Scholar [3]. This outline will be the basis of a similar type of work that will end up getting crafted in Overleaf and shared out to the world.

Searching for just pure introductions to artificial intelligence is really hit or miss. A lot of different introductions to various fields exist. In this case, I’m trying to zoom out a little more into a larger evaluation of content instead of focusing on any one field. Nothing I ran into during my search had the number of citations or impact of the Russel and Norvig textbook. I’m going to endeavor to structure and organize 70+ articles into a syllabus. To give you an idea of the kind of things that are going to get pulled together here are 5 different papers.

Oke, S. A. (2008). A literature review on artificial intelligence. International journal of information and management sciences, 19(4), 535-570. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=f4dfcfe3f132b1079d054e2db29adc063fab0007

Contreras, I., & Vehi, J. (2018). Artificial intelligence for diabetes management and decision support: literature review. Journal of medical Internet research, 20(5), e10775. https://www.sciencedirect.com/science/article/pii/S0004370218305988/pdfft?md5=ec6948d3f66efe5e57d1336a54d1604d&pid=1-s2.0-S0004370218305988-main.pdf

Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. (2018). Artificial intelligence in radiology. Nature Reviews Cancer, 18(8), 500-510. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6268174/

Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature, 521(7553), 452-459. https://www.repository.cam.ac.uk/bitstream/handle/1810/248538/Ghahramani%25202015%2520Nature.pdf?sequence=1

Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., ... & Wang, Y. (2017). Artificial intelligence in healthcare: past, present and future. Stroke and vascular neurology, 2(4). https://svn.bmj.com/content/svnbmj/2/4/230.full.pdf

Obviously, working out of a textbook format will help provide a better structured introduction and that is why I plan on mentioning that in the paper and I have noted it here. On a scholarly basis I understand the need to dive directly into source papers and really get a sense of what is happening within the applied side of AI research in the last 10 years. That is why I want to spend some time preparing the independent study based syllabus for others as well as it being a good solid effort for me to learn more rapidly with an extreme degree of focus.

What did ChatGPT 4.0 model come up with?

An independent study applied AI syllabus will provide you with the necessary knowledge and skills to develop practical AI solutions. Below is a suggested 14-week syllabus to guide you through the essential topics and resources.

Week 1: Introduction to AI and Machine Learning
History of AI
Overview of machine learning, deep learning, and reinforcement learning
Common machine learning algorithms
Week 2: Python for AI and Machine Learning
Basic Python programming
NumPy, pandas, and matplotlib
Setting up a Python environment for machine learning
Week 3: Data Preprocessing and Exploration
Data cleaning and transformation
Handling missing data and outliers
Feature engineering and selection
Week 4: Supervised Learning - Regression
Linear regression
Ridge and Lasso regression
Decision trees and random forests for regression
Week 5: Supervised Learning - Classification
Logistic regression
k-Nearest Neighbors (k-NN)
Support vector machines (SVM)
Week 6: Unsupervised Learning
k-Means clustering
Hierarchical clustering
Principal component analysis (PCA)
Week 7: Neural Networks and Deep Learning
Introduction to neural networks
Activation functions and backpropagation
Convolutional neural networks (CNNs)
Week 8: Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM)
Introduction to RNNs and LSTMs
Time series forecasting
Natural language processing (NLP) applications
Week 9: Reinforcement Learning
Introduction to reinforcement learning
Q-learning and Deep Q-Networks (DQNs)
Policy gradients and actor-critic methods
Week 10: Transfer Learning and Pretrained Models
Introduction to transfer learning
Fine-tuning pretrained models
Pretrained models for NLP and computer vision
Week 11: Natural Language Processing
Word embeddings and Word2Vec
Text classification and sentiment analysis
Text summarization and translation
Week 12: Computer Vision
Object detection and recognition
Semantic segmentation
Image-to-image translation
Week 13: Generative Models
Introduction to generative models
Variational autoencoders (VAEs)
Generative adversarial networks (GANs)
Week 14: Ethics and AI Deployment
Ethical considerations in AI
Bias and fairness in machine learning
Deploying AI models

Key resources to study these topics include popular textbooks, online courses, blogs, and tutorials. Some notable resources include:

"Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow" by Aurélien Géron
"Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
"Reinforcement Learning: An Introduction" by Richard S. Sutton and Andrew G. Barto
Online courses like Coursera's "Machine Learning" by Andrew Ng or fast.ai's "Practical Deep Learning for Coders"
AI research papers from sources like arXiv.org, NeurIPS, and ICML
By following this syllabus, you will gain a solid foundation in applied AI and be well-equipped to pursue more advanced topics or specific application domains.

Links and thoughts:

Top Tweets of the week:

https://twitter.com/ijustine/status/1657505930563448833

https://twitter.com/CaseyNewton/status/1657062682229633024

Footnotes:

[1] https://aima.cs.berkeley.edu/adoptions.html

[2] https://aima.cs.berkeley.edu/Bibliography.pdf

[3] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=An+independent+study+based+introduction+to+machine+learning+syllabus+for+2022&btnG=

What’s next for The Lindahl Letter?

Week 122: Will AI be a platform or a service?
Week 123: Considering open source AI
Week 124: Profiling OpenAI
Week 125: Profiling Hugging Face (open and collaborative machine learning)
Week 126: Profiling Deep Mind

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Right now at the start of 2023, I would probably highlight 3 AI trends: generative models, automation, and legislation. Before we get into those specific topics let’s zoom out for just a second and look at two different reports you could read to get a sense of what is going on right now. One of the great places to start would be with the recently released 2023 AI Index report from the Institute for Human Centered AI.

Nestor Maslej, Loredana Fattorini, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Helen Ngo, Juan Carlos Niebles, Vanessa Parli, Yoav Shoham, Russell Wald, Jack Clark, and Raymond Perrault, “The AI Index 2023 Annual Report,” AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, CA, April 2023. https://aiindex.stanford.edu/wp-content/uploads/2023/04/HAI_AI-Index-Report_2023.pdf

You can look around the website they have setup here:

https://hai.stanford.edu/research/ai-index-2023

If the 386 page PDF seems like a lot of content to consume, then feel free to read the slimed down 2023 state of AI in 14 charts that they also produced

https://hai.stanford.edu/news/2023-state-ai-14-charts

The other interesting report that I read recently was the Google Cloud 2023 Data and AI Trends Report which you can download by giving them your email. That step seemed annoying so I’m going to leave the link to the report below, but it might default back to the information page

https://services.google.com/fh/files/misc/data_and_ai_trends.pdf

Those two reports aside, let’s jump into the 3 topics that will comprise the bulk of this missive.

Generative models - generative models are probably the biggest thing happening at the start of 2023. We are setting chat based and image based models like GPT-4 and DALL-E 2 from OpenAI [1]. One of the things spinning out of the generative models are the low code, no code, and copilot efforts. This is one of the areas where you will find a lot of coverage in the media as the generative models are being used as search engines and the image generating efforts are so easy to use people have gone thumbnail happy using them recently [2].

Automation - We are going to see a tremendous amount of better automation out of the new technology that is available [3]. I’m tempted to mention that I consider a lot of the copilot efforts to be a part of early automation. Those copilot efforts will move from being inline assistive to probably a more prompt based delivery. That type of request and receiving copiloting is what I would consider a form of automation.

Legislation - Countries are wholesale banning ChatGPT [4]. A lot of legislation will probably come into being related to trying to ensure safety [5]. I’m not going to spend a ton of time on this trend as it's going to unfold in a pretty predictable way.

You might find that this week the large language model from OpenAI has produced a better post than what I crafted. To that end, let's find out now what the ChatGPT model GPT-4 produces after being prompted about this one…

As of April 2023, the top trends in AI include the following:

Federated Learning: This approach allows AI models to be trained on decentralized data while maintaining privacy. It has gained prominence as a way to harness the potential of data from various sources without compromising user privacy.
AI Ethics and Regulation: As AI systems become more integrated into everyday life, there is an increasing focus on the ethical implications and regulatory measures required to ensure AI is used responsibly.
Explainable AI (XAI): With the growing complexity of AI models, there is a rising demand for models that provide interpretable and understandable explanations for their predictions or decisions.
AI in Healthcare: AI applications in healthcare are expanding rapidly, from diagnosis and treatment recommendations to drug discovery and personalized medicine.
AI for Climate Change: AI techniques are being used to tackle climate change by optimizing energy consumption, forecasting extreme weather events, and modeling climate patterns.
Edge AI: As IoT devices proliferate, there is a growing interest in running AI models on-device (at the edge) rather than relying on cloud-based processing. This approach can improve latency, privacy, and energy efficiency.
AI for Cybersecurity: AI is being deployed to enhance cybersecurity measures by detecting and preventing potential threats and attacks using advanced pattern recognition and anomaly detection.
Reinforcement Learning (RL): RL, a type of machine learning that focuses on decision-making, continues to gain traction as researchers explore its potential applications in various fields, including robotics, finance, and gaming.
Natural Language Processing (NLP): NLP techniques continue to advance, enabling more sophisticated AI applications in areas like language translation, sentiment analysis, and chatbot development.
AI and Robotics: AI is playing a key role in the development of advanced robotics systems, allowing for improved autonomy, decision-making, and adaptability in various domains, including manufacturing, logistics, and personal assistance.

These trends are expected to evolve as AI research and development continues to advance, addressing the opportunities and challenges associated with the integration of AI into various aspects of human life.

Links and thoughts:

Footnotes:

[1] https://openai.com/

[2] https://www.forbes.com/sites/ganeskesari/2022/12/22/ai-trends-for-2023-industry-experts-and-chatgpt-ai-make-their-predictions/

[3] https://venturebeat.com/programming-development/3-key-trends-for-2023-low-code-no-code-generative-ai-and-the-evolution-of-programming/

[4] https://www.cnbc.com/2023/04/04/italy-has-banned-chatgpt-heres-what-other-countries-are-doing.html

[5] https://apnews.com/article/joe-biden-artificial-intelligence-science-technology-chatgpt-6948df344041ef1e794d199595bf69e9

What’s next for The Lindahl Letter?

Week 121: Considering an independent study applied AI syllabus
Week 122: Will AI be a platform or a service?
Week 123: Considering open source AI
Week 124: Profiling OpenAI
Week 125: Profiling Hugging Face (open and collaborative machine learning)

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Flooding and astroturfing abound at the moment. Both of those things were happening before the advent of large language models (LLMs), but they have increased in frequency now that bad actors are able to just open the floodgates for content. Making large swaths of the internet that is just designed for search engine placement and self-referencial boosting has become so much easier recently. Sure all that bad data was abounding before this shift in what is now happening with OpenAI, Google, Microsoft, and Facebook recently sharing out chat services.

It’s one of those things where it is hard to put words on a page about it. Working with one of the chat systems to make content seems to trivialize the writing process. My day starts with an hour of focused academic work. That time is the fulfilled promise of decades of training that included a lot of hard work to get to this point. I can focus on a topic and work toward understanding it. All of that requires my focus and attention on something for that hour. Sometimes on the weekends I spend a couple of hours doing the same thing on a very focused topic. Those chat models with their large language model backends (LLM) produce content within seconds. It’s literally like a 1:60 ratio for output. It takes me an hour to produce what it creates within that minute including the time for the user to enter the prompt.

Maybe I did not expect this type of interaction to really affect me in this way. Everything has been questioned in terms of my writing output and what exactly is going to happen now. The door has been flung open to the creation of content. Central to that problem is the reality that the careful curation of content within academics and the publish first curation of the media are going to get flooded. Both systems are going to get absolutely overloaded with submissions. Something has to give based on the amount of attention that exists. They are not minting any new capacity for attention and the channels for grabbing that attention are relatively limited. The next couple of years are going to be a mad scrabble toward some sort of equilibrium between the competing forces of content curation and flooding.

This really is something that I’m concerned about on an onboarding basis. Do all the books, photos, articles, and paintings in the before times just end up with a higher value weighting going forward? Will this AI revolution have cheapened the next generation of information delivery in ways we will not fully get to appreciate until the wave has passed us and we can see the aftermath of that scenario? Those questions are at the heart of what I’m concerned about. Selfishly they are questions about the value and purpose of my own current writing efforts. More broadly they are questions about the value of writing within our civil society as we work toward the curation of sharable knowledge. We all work toward that perfect possible future either with purpose or without it. Knowledge is built on the shoulders of the giants that came before us adding to collective understanding of the world around us. Anyone with access and an adventurous spirit can pick up the advancement of some very complex efforts to enhance the academy's knowledge on a topic.

Maybe I’m worried that the degree of flooding with flatten information so much that the ability to move things forward will diminish. Sorting, seeking, and trying to distill value from an oversupply of newly minted information may well create that diminishing effect. We will move from intellectual overcrowding in the academy to just an overwhelming sea of derivative content marching along beyond any ability to constrain or consume. I’m going to stop with that last argument as it may be the best way to sum this up.

Links and thoughts:

Top 5 Tweets of the week:

https://twitter.com/CaseyNewton/status/1651967468322320387

https://twitter.com/Benioff/status/1651605849310130178

https://twitter.com/AndrewYNg/status/1651605660382134274

https://twitter.com/verge/status/1651355922751422464

https://twitter.com/NateSilver538/status/1650899579234140168

What’s next for The Lindahl Letter?

Week 120: That one with an obligatory AI trend’s post
Week 121: Considering an independent study applied AI syllabus
Week 122: Will AI be a platform or a service?
Week 123: Considering open source AI
Week 124: Profiling OpenAI

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

This is going to be an interesting edition of The Lindahl Letter. Please pardon the self-indulgence of working with ChatGPT to write a paper and ultimately expand that into a full book using prompts and the model. Sit back and relax as this will be a long one.

I was able to give OpenAI’s ChatGPT a prompt of, “A paper on political debt as a concept vs. technical debt.” That created the following output:

Political debt and technical debt are two distinct concepts that have different implications and applications in their respective fields. While technical debt refers to the long-term costs associated with the trade-offs made in the software development process, political debt refers to the exchange of favors, resources, or benefits for political influence or gain.

Technical debt arises when developers make decisions that favor short-term gains over long-term benefits, such as prioritizing speed over quality or failing to maintain code cleanliness. While these decisions may be necessary in the short-term, they can result in higher maintenance costs and increased risk in the long-term.

Political debt, on the other hand, arises when political actors trade favors, resources, or benefits in exchange for political influence or gain. This can include actions such as accepting campaign contributions from specific interest groups, providing special treatment to certain individuals or corporations, or ignoring public opinion in favor of personal interests.

While technical debt is primarily a concern for software developers and companies, political debt has broader implications for democratic governance and the public interest. Political debt can undermine the principles of transparency, accountability, and ethical leadership in government, and can result in policies that do not fully address the needs or preferences of the broader public.

Managing and reducing political debt requires a sustained commitment to transparency, accountability, and ethical leadership in political systems. This can include measures such as campaign finance reform, strengthening democratic institutions such as the judiciary and civil society organizations, and promoting public participation and engagement in the political process.

In conclusion, while political debt and technical debt are two distinct concepts, they both demonstrate the challenges associated with short-term thinking and decision-making in their respective fields. By promoting the values of transparency, accountability, and ethical leadership, we can build stronger, more effective, and more representative political and technical systems that serve the interests of all stakeholders.

That was not good enough so I thought it would be a good idea to write a full book working with ChatGPT on this one. Don’t begin a state of panic on this one, but the remainder of this effort is about 45 pages that include 21 chapters of content. I’m not entirely sure I’ll be able to read this output in one session for the audio version of this effort. To create this series of chapters I just kept prompting the model to produce more chapters. This is the result of a couple hours of my working with the interface to get this output.

Chapter 1: Introduction

Political debt and technical debt are two distinct but related concepts that have significant implications for their respective fields. Political debt refers to the exchange of favors, resources, or benefits for political influence or gain, while technical debt refers to the long-term costs associated with the trade-offs made in the software development process.

Political debt is a complex and multifaceted challenge that can undermine the principles of transparency, accountability, and ethical leadership in government. When political actors are indebted to specific interest groups or individuals, they may be less likely to consider policies that conflict with the interests of these groups. This can result in policies that do not fully address the needs or preferences of the broader public, and can undermine the democratic legitimacy of government.

Technical debt, on the other hand, arises when developers make decisions that favor short-term gains over long-term benefits. These decisions may be necessary in the short-term to meet project deadlines or to address urgent technical issues, but they can result in higher maintenance costs and increased risk in the long-term.

Despite their different origins and applications, political debt and technical debt share important similarities. Both concepts are associated with short-term thinking and decision-making, and both can result in long-term costs and negative consequences. Additionally, strategies for managing and reducing political debt and technical debt require a sustained commitment to transparency, accountability, and ethical leadership.

This book will compare and contrast political debt and technical debt, examining the causes, consequences, and impact of these concepts on their respective fields. We will also explore strategies for managing and reducing political debt and technical debt, and examine case studies illustrating successful approaches to addressing these challenges.

In the following chapters, we will examine the causes and consequences of political debt and technical debt, compare and contrast the two concepts, and explore strategies for managing and reducing these forms of debt. By promoting the values of transparency, accountability, and ethical leadership, we can build stronger, more effective, and more representative political and technical systems that serve the interests of all stakeholders.

Chapter 2: Political Debt: Causes, Consequences, and Impact on Governance

Political debt is a pervasive and complex challenge that can have significant implications for governance and democratic institutions. In this chapter, we will explore the causes, consequences, and impact of political debt on governance.

Causes of Political Debt

Political debt can arise from a variety of factors, including the influence of money in politics, the power dynamics between political actors and interest groups, and the personal interests and motivations of individual elected officials. When political actors are indebted to specific interest groups or individuals, they may be less likely to consider policies that conflict with the interests of these groups, creating a system of privilege and inequality that undermines the democratic process.

Consequences of Political Debt

The consequences of political debt can be far-reaching, impacting the quality and legitimacy of governance in a variety of ways. Political debt can undermine public trust in government, reduce transparency and accountability in the decision-making process, and limit the range of policy options available to elected officials. Additionally, political debt can create conflicts of interest for elected officials, leading them to prioritize the interests of their debtors over the interests of the broader public.

Impact of Political Debt on Governance

The impact of political debt on governance can be significant, with the potential to undermine the principles of democratic accountability and transparency. When political actors are indebted to specific interest groups or individuals, they may be less likely to prioritize the public interest over the interests of these groups. This can result in policies that do not fully address the needs or preferences of the broader public, and can lead to a lack of trust in democratic institutions.

Furthermore, political debt can also create a system of privilege and inequality that undermines the democratic process, as elected officials may be more likely to act in the interests of their debtors rather than the interests of the broader public. This can result in a lack of representation for certain groups or individuals, further eroding the principles of democratic governance.

In conclusion, political debt is a pervasive and complex challenge that can have significant implications for governance and democratic institutions. By understanding the causes and consequences of political debt, we can develop strategies for managing and reducing its negative impact and promoting transparency, accountability, and ethical leadership in political systems.

Chapter 3: Technical Debt: Causes, Consequences, and Impact on Software Development

Technical debt is a common challenge in software development, with important implications for project management and long-term success. In this chapter, we will explore the causes, consequences, and impact of technical debt on software development.

Causes of Technical Debt

Technical debt can arise from a variety of factors, including the pressure to meet project deadlines, the need to address urgent technical issues, and the lack of resources or time available for high-quality software development. When developers make decisions that favor short-term gains over long-term benefits, such as prioritizing speed over quality or failing to maintain code cleanliness, technical debt can accumulate.

Consequences of Technical Debt

The consequences of technical debt can be significant, impacting the quality, efficiency, and maintainability of software development projects. Technical debt can increase the cost and time required for future software development, increase the risk of software failures, and reduce the quality of software products. Additionally, technical debt can create a culture of “quick fixes” and short-term thinking, which can undermine the overall success and sustainability of software development projects.

Impact of Technical Debt on Software Development

The impact of technical debt on software development can be significant, with the potential to undermine the success and sustainability of software products and projects. Technical debt can result in software products that are difficult to maintain and update, leading to increased costs and decreased efficiency over time. Additionally, technical debt can increase the risk of software failures and security breaches, resulting in reputational damage and financial loss for software development companies.

Furthermore, technical debt can create a culture of “quick fixes” and short-term thinking, which can limit the creativity and innovation that is necessary for long-term success in software development. Technical debt can also create a sense of frustration and burnout among developers, who may feel that they are not given the time or resources necessary to develop high-quality software products.

In conclusion, technical debt is a common challenge in software development that can have significant implications for project management and long-term success. By understanding the causes and consequences of technical debt, we can develop strategies for managing and reducing its negative impact and promoting long-term success and sustainability in software development.

Chapter 4: Comparing and Contrasting Political Debt and Technical Debt

While political debt and technical debt are two distinct concepts, they share important similarities in terms of their causes, consequences, and impact on their respective fields. In this chapter, we will compare and contrast political debt and technical debt, exploring the similarities and differences between these two concepts.

Causes of Political Debt and Technical Debt

Both political debt and technical debt arise from short-term thinking and decision-making. In the case of political debt, elected officials may prioritize the interests of their debtors over the interests of the broader public in order to secure political power or resources. In the case of technical debt, developers may prioritize speed or functionality over the long-term maintainability and sustainability of their code.

Consequences of Political Debt and Technical Debt

Both political debt and technical debt can result in long-term costs and negative consequences. In the case of political debt, policies that do not fully address the needs or preferences of the broader public can undermine the democratic process and reduce public trust in government. In the case of technical debt, software products that are difficult to maintain and update can result in increased costs and decreased efficiency over time.

Impact of Political Debt and Technical Debt

Both political debt and technical debt can have significant impacts on their respective fields. In the case of political debt, a lack of transparency and accountability in government can reduce public trust in democratic institutions and lead to a lack of representation for certain groups or individuals. In the case of technical debt, a culture of “quick fixes” and short-term thinking can undermine the overall success and sustainability of software development projects and limit creativity and innovation in the field.

Despite these similarities, there are also important differences between political debt and technical debt. Political debt is primarily a concern for democratic governance, while technical debt is primarily a concern for software development. Additionally, the strategies for managing and reducing political debt and technical debt may differ significantly, with political debt requiring a sustained commitment to transparency, accountability, and ethical leadership in government, while technical debt requires a focus on maintainability, scalability, and sustainability in software development.

In conclusion, while political debt and technical debt are distinct concepts, they share important similarities in terms of their causes, consequences, and impact on their respective fields. By understanding the similarities and differences between these two concepts, we can develop strategies for managing and reducing these forms of debt and promoting transparency, accountability, and ethical leadership in government and software development.

Chapter 5: Managing and Reducing Political Debt

Political debt is a complex and pervasive challenge that can undermine the principles of transparency, accountability, and ethical leadership in government. In this chapter, we will explore strategies for managing and reducing political debt, promoting democratic accountability, and ensuring that government policies serve the broader public interest.

Campaign Finance Reform

One of the primary causes of political debt is the influence of money in politics. Campaign finance reform can help to reduce the influence of special interest groups and individuals on the political process, promoting transparency and accountability in government. Measures such as limiting the amount of money that individuals or corporations can contribute to political campaigns, and requiring greater disclosure of campaign contributions, can help to reduce the influence of money in politics.

Transparency and Accountability Measures

Transparency and accountability measures can help to promote greater transparency in government decision-making, reducing the likelihood of political debt. This can include measures such as open meetings, public record requests, and whistleblower protections, which can help to ensure that government officials are held accountable for their actions and that the public has access to information about government decision-making processes.

Promoting Ethical Leadership

Promoting ethical leadership is an important strategy for reducing political debt and ensuring that government policies serve the broader public interest. This can include measures such as developing and enforcing ethical codes of conduct for government officials, providing training and education on ethical leadership, and encouraging greater public participation and engagement in the political process.

Strengthening Democratic Institutions

Strengthening democratic institutions such as the judiciary and civil society organizations can help to reduce political debt and promote democratic accountability. By providing checks and balances on government power, these institutions can help to ensure that government policies are transparent, accountable, and serve the broader public interest.

Public Participation and Engagement

Promoting greater public participation and engagement in the political process can help to reduce political debt and promote transparency and accountability in government. This can include measures such as town hall meetings, public hearings, and citizen advisory boards, which can provide opportunities for the public to voice their opinions and concerns about government policies.

In conclusion, managing and reducing political debt requires a sustained commitment to transparency, accountability, and ethical leadership in political systems. Strategies such as campaign finance reform, transparency and accountability measures, promoting ethical leadership, strengthening democratic institutions, and promoting public participation and engagement can help to reduce the influence of special interest groups and individuals on the political process and promote democratic accountability.

Chapter 6: Managing and Reducing Technical Debt

Technical debt is a common challenge in software development that can have significant implications for project management and long-term success. In this chapter, we will explore strategies for managing and reducing technical debt, promoting sustainable software development, and ensuring the long-term success of software products.

Refactoring

Refactoring is the process of restructuring existing code without changing its external behavior. This can help to reduce technical debt by improving the quality and maintainability of code over time. By regularly refactoring code, developers can reduce the risk of software failures, improve the efficiency and scalability of software products, and ensure that the software is maintainable over the long-term.

Prioritization

Prioritization is the process of identifying and prioritizing technical debt in software development projects. By prioritizing technical debt, developers can focus their efforts on the most critical issues, reducing the overall risk of software failures and ensuring that software products are maintainable over the long-term.

Technical Debt Tracking

Tracking technical debt is an important strategy for managing and reducing technical debt in software development. By identifying and tracking technical debt, developers can monitor the progress of technical debt reduction efforts, measure the impact of technical debt on software development projects, and ensure that technical debt is managed effectively over time.

Automation

Automation is the process of using software tools to automate repetitive tasks in software development. Automation can help to reduce technical debt by improving the efficiency and accuracy of software development tasks, reducing the risk of errors and software failures, and freeing up developers to focus on more complex tasks.

Training and Education

Training and education are important strategies for managing and reducing technical debt in software development. By providing developers with the training and education necessary to develop high-quality code, organizations can reduce the overall risk of technical debt and ensure that software products are maintainable over the long-term.

In conclusion, managing and reducing technical debt requires a sustained commitment to sustainable software development practices. Strategies such as refactoring, prioritization, technical debt tracking, automation, and training and education can help to reduce technical debt, improve the quality and maintainability of software products, and ensure the long-term success of software development projects.

Chapter 7: Case Studies

In this chapter, we will examine case studies of political debt and technical debt, exploring real-world examples of these challenges and the strategies used to manage and reduce them.

Case Study: Political Debt in Brazil

In Brazil, political debt has been a significant challenge, with elected officials often indebted to specific interest groups or individuals. This has led to policies that do not fully address the needs or preferences of the broader public, reducing trust in democratic institutions and undermining the democratic process.

To address this challenge, Brazil has implemented a number of reforms, including greater transparency and accountability measures, campaign finance reform, and strengthening democratic institutions such as the judiciary. These efforts have helped to reduce the influence of special interest groups and individuals on the political process and promote greater democratic accountability.

Case Study: Technical Debt at Google

At Google, technical debt has been a common challenge in software development, with developers often prioritizing short-term gains over long-term benefits. This has led to software products that are difficult to maintain and update, increasing the risk of software failures and reducing the overall efficiency of software development projects.

To address this challenge, Google has implemented a number of strategies, including prioritizing technical debt reduction efforts, tracking technical debt, and providing developers with the training and education necessary to develop high-quality code. These efforts have helped to reduce the overall risk of technical debt and ensure the long-term success of software development projects.

Case Study: Political Debt in Kenya

In Kenya, political debt has been a significant challenge, with elected officials often indebted to specific interest groups or individuals. This has led to policies that do not fully address the needs or preferences of the broader public, reducing trust in democratic institutions and undermining the democratic process.

To address this challenge, Kenya has implemented a number of reforms, including greater transparency and accountability measures, campaign finance reform, and strengthening democratic institutions such as the judiciary. Additionally, Kenya has prioritized public participation and engagement, providing opportunities for the public to voice their opinions and concerns about government policies. These efforts have helped to reduce the influence of special interest groups and individuals on the political process and promote greater democratic accountability.

In conclusion, case studies of political debt and technical debt illustrate the real-world impact of these challenges and the strategies used to manage and reduce them. By examining these case studies, we can learn valuable lessons about the importance of transparency, accountability, and ethical leadership in government and sustainable software development practices.

Chapter 8: Conclusion and Future Directions

Political debt and technical debt are two distinct challenges that share important similarities in terms of their causes, consequences, and impact on their respective fields. In this book, we have explored these challenges in depth, examining the causes and consequences of political debt and technical debt, the impact of these challenges on their respective fields, and strategies for managing and reducing political debt and technical debt.

Looking to the future, it is clear that political debt and technical debt will continue to be important challenges for government and software development, respectively. However, there are opportunities for innovation and progress in both fields. For political debt, greater transparency and accountability measures, campaign finance reform, and strengthening democratic institutions can help to reduce the influence of special interest groups and individuals on the political process and promote greater democratic accountability. For technical debt, sustainable software development practices such as refactoring, prioritization, technical debt tracking, automation, and training and education can help to reduce technical debt, improve the quality and maintainability of software products, and ensure the long-term success of software development projects.

Moreover, there is potential for cross-disciplinary collaboration and learning between the fields of government and software development. By examining the similarities and differences between political debt and technical debt, and exploring strategies for managing and reducing these challenges, we can develop innovative solutions that promote sustainable software development practices and strengthen democratic governance.

In conclusion, political debt and technical debt are complex and pervasive challenges that require sustained commitment and innovative thinking to manage and reduce. By developing strategies that promote transparency, accountability, and ethical leadership in government and software development, we can ensure the long-term success and sustainability of these fields, and promote a more just and equitable society for all.

Chapter 9: The Future of Political and Technical Debt

As the fields of government and software development continue to evolve, it is important to consider the future of political and technical debt. In this chapter, we will explore the future of these challenges and the potential for innovation and progress in both fields.

The Future of Political Debt

The future of political debt is uncertain, with ongoing challenges such as the influence of money in politics, rising inequality, and social fragmentation contributing to the persistence of this challenge. However, there are opportunities for innovation and progress in this field, including greater use of technology to promote transparency and accountability in government, expanding public participation and engagement in the political process, and strengthening democratic institutions to promote greater democratic accountability.

The Future of Technical Debt

The future of technical debt is also uncertain, with ongoing challenges such as the pressure to deliver software products quickly and the rapid pace of technological change contributing to the persistence of this challenge. However, there are opportunities for innovation and progress in this field, including the use of emerging technologies such as artificial intelligence and machine learning to automate software development tasks, the adoption of sustainable software development practices such as DevOps, and the development of tools and techniques to identify and prioritize technical debt in software development projects.

Cross-disciplinary Collaboration

Cross-disciplinary collaboration between the fields of government and software development can play a key role in addressing both political debt and technical debt. By exploring the similarities and differences between these challenges, and developing innovative solutions that draw on the expertise of both fields, we can promote sustainable software development practices and strengthen democratic governance.

Ethical Considerations

As political and technical debt continue to evolve, it is important to consider the ethical implications of these challenges. In government, ethical considerations may include issues such as the representation of marginalized communities and the distribution of resources, while in software development, ethical considerations may include issues such as privacy, security, and bias in algorithms.

In conclusion, the future of political and technical debt is uncertain, but there are opportunities for innovation and progress in both fields. By developing strategies that promote transparency, accountability, and ethical leadership in government and software development, we can ensure the long-term success and sustainability of these fields and promote a more just and equitable society for all.

Chapter 10: Final Thoughts

Political debt and technical debt are complex and pervasive challenges that have significant implications for government and software development, respectively. In this book, we have explored these challenges in depth, examining their causes and consequences, the impact of these challenges on their respective fields, and strategies for managing and reducing political debt and technical debt.

Looking back on the book, it is clear that political debt and technical debt share important similarities in terms of their causes, consequences, and impact on their respective fields. Both challenges can undermine the principles of transparency, accountability, and ethical leadership in government and software development, respectively. However, there are opportunities for innovation and progress in both fields, including the development of sustainable software development practices and the strengthening of democratic institutions to promote greater democratic accountability.

Chapter 11: Key Takeaways

In this final chapter, we will summarize the key takeaways from this book and highlight the most important lessons that can be applied to managing and reducing political debt and technical debt.

Transparency and Accountability

Transparency and accountability are crucial for managing and reducing political debt and technical debt. By promoting greater transparency in government decision-making processes and software development practices, we can reduce the influence of special interest groups and individuals and promote democratic accountability.

Prioritization

Prioritization is key to managing and reducing political debt and technical debt. By identifying and prioritizing the most critical issues, we can focus our efforts on addressing the root causes of these challenges, reducing the overall risk of political and technical debt.

Sustainable Development Practices

Sustainable development practices are important for managing and reducing technical debt in software development. By using practices such as refactoring, automation, and technical debt tracking, we can ensure that software products are maintainable over the long-term, reducing the overall risk of technical debt.

Strengthening Democratic Institutions

Strengthening democratic institutions is important for managing and reducing political debt. By providing checks and balances on government power, democratic institutions can help to ensure that government policies are transparent, accountable, and serve the broader public interest.

Public Participation and Engagement

Ethical Leadership

Promoting ethical leadership is crucial for managing and reducing political debt and technical debt. By developing and enforcing ethical codes of conduct for government officials and software developers, we can ensure that government policies and software products serve the broader public interest.

In conclusion, managing and reducing political debt and technical debt requires a sustained commitment to transparency, accountability, and ethical leadership in government and software development. By prioritizing these values and implementing sustainable development practices, we can promote greater democratic accountability and ensure the long-term success of government policies and software development projects.

Chapter 12: Recommendations for Future Research

As we have explored throughout this book, political debt and technical debt are complex and pervasive challenges that have significant implications for government and software development, respectively. While we have discussed strategies for managing and reducing these challenges, there is still much that we do not know about the causes, consequences, and impact of political debt and technical debt. In this chapter, we will outline recommendations for future research to better understand these challenges and develop more effective strategies for managing and reducing them.

Understanding the Causes of Political Debt

There is a need for more research to better understand the causes of political debt, including the influence of special interest groups and individuals on government decision-making processes, the role of campaign finance in political debt, and the impact of political debt on democratic accountability and representation.

Assessing the Impact of Political Debt

There is a need for more research to assess the impact of political debt on government policies and democratic institutions, including the effectiveness of transparency and accountability measures in reducing the influence of special interest groups and individuals on government decision-making processes.

Identifying the Causes of Technical Debt

There is a need for more research to better understand the causes of technical debt, including the impact of short-term thinking, the pressure to deliver software products quickly, and the rapid pace of technological change on software development practices.

Assessing the Impact of Technical Debt

There is a need for more research to assess the impact of technical debt on software development projects, including the impact on software quality and maintainability, the risk of software failures, and the overall efficiency of software development projects.

Developing Effective Strategies for Managing and Reducing Political Debt and Technical Debt

There is a need for more research to develop more effective strategies for managing and reducing political debt and technical debt, including the use of emerging technologies and sustainable development practices in software development, the development of ethical leadership in government and software development, and the promotion of public participation and engagement in the political process.

In conclusion, political debt and technical debt are complex and pervasive challenges that require ongoing research and innovation to manage and reduce effectively. By developing a better understanding of these challenges, and implementing effective strategies for managing and reducing them, we can promote greater transparency, accountability, and ethical leadership in government and software development, respectively.

Chapter 13: Final Remarks

In this book, we have explored the challenges of political debt and technical debt, examining their causes and consequences, the impact of these challenges on their respective fields, and strategies for managing and reducing political debt and technical debt. While these challenges are complex and pervasive, there are opportunities for innovation and progress in both government and software development.

By promoting transparency and accountability, prioritizing sustainable development practices, strengthening democratic institutions, promoting public participation and engagement, and developing ethical leadership in government and software development, we can reduce the influence of special interest groups and individuals, ensure the long-term success of software development projects, and promote greater democratic accountability in government.

Moreover, by exploring the similarities and differences between political debt and technical debt, and developing cross-disciplinary collaborations and learning opportunities between the fields of government and software development, we can develop innovative solutions that draw on the expertise of both fields to promote sustainable software development practices and strengthen democratic governance.

In conclusion, the challenges of political debt and technical debt require sustained commitment and innovative thinking to manage and reduce. However, by developing strategies that promote transparency, accountability, and ethical leadership in government and software development, we can ensure the long-term success and sustainability of these fields, and promote a more just and equitable society for all.

Chapter 14: Call to Action

The challenges of political debt and technical debt are complex and pervasive, and require sustained commitment and innovative thinking to manage and reduce effectively. However, the implications of these challenges are too important to ignore, and require action from all stakeholders involved in government and software development.

In this chapter, we will outline a call to action for stakeholders in government and software development to address the challenges of political debt and technical debt.

Government Officials

Government officials have a crucial role to play in addressing the challenges of political debt. They can promote greater transparency and accountability in government decision-making processes, prioritize the interests of the broader public over special interest groups and individuals, and strengthen democratic institutions to promote greater democratic accountability.

Software Developers

Software developers have a crucial role to play in addressing the challenges of technical debt. They can prioritize sustainable software development practices, such as refactoring, automation, and technical debt tracking, to ensure that software products are maintainable over the long-term, reducing the overall risk of technical debt.

Educators

Educators have a crucial role to play in preparing the next generation of government officials and software developers to address the challenges of political debt and technical debt. They can incorporate ethics and sustainability into their curriculum, promote critical thinking and problem-solving skills, and provide opportunities for cross-disciplinary collaboration and learning between government and software development.

Civil Society Organizations

Civil society organizations have a crucial role to play in holding government officials and software developers accountable and promoting greater transparency and democratic accountability. They can provide opportunities for public participation and engagement in the political process, promote advocacy and lobbying efforts on behalf of marginalized communities, and hold government officials and software developers accountable to ethical standards.

In conclusion, the challenges of political debt and technical debt require action from all stakeholders involved in government and software development. By promoting transparency, accountability, and ethical leadership, prioritizing sustainable development practices, and strengthening democratic institutions, we can ensure the long-term success and sustainability of government policies and software development projects, and promote a more just and equitable society for all.

Chapter 15: Conclusion

Throughout the book, we have emphasized the importance of promoting transparency, accountability, and ethical leadership in government and software development, prioritizing sustainable development practices, strengthening democratic institutions, and promoting public participation and engagement in the political process.

While these challenges are complex and pervasive, there are opportunities for innovation and progress in both fields. By developing cross-disciplinary collaborations and learning opportunities between the fields of government and software development, we can develop innovative solutions that draw on the expertise of both fields to promote sustainable software development practices and strengthen democratic governance.

In conclusion, the challenges of political debt and technical debt require sustained commitment and innovative thinking to manage and reduce effectively. However, by developing strategies that promote transparency, accountability, and ethical leadership in government and software development, we can ensure the long-term success and sustainability of these fields, and promote a more just and equitable society for all.

Chapter 16: Glossary

In this final chapter, we provide a glossary of key terms and concepts discussed throughout this book to provide readers with a quick reference guide.

Political debt: The accumulation of obligations, favors, and commitments owed by elected officials to special interest groups and individuals, which can undermine the principles of transparency, accountability, and democratic accountability in government.
Technical debt: The accumulation of technical problems and inefficiencies in software development projects, which can lead to higher costs, reduced quality, and increased risk of software failures.
Transparency: The degree to which government decision-making processes and software development practices are open and accessible to the public.
Accountability: The degree to which government officials and software developers are responsible for their actions and can be held accountable for their decisions.
Democratic accountability: The degree to which government officials and policies serve the broader public interest and are accountable to the public.
Special interest groups: Groups or individuals who seek to influence government policies and decisions to serve their own interests.
Sustainable development practices: Software development practices that prioritize the long-term maintainability and quality of software products, such as refactoring, automation, and technical debt tracking.
Ethical leadership: Leadership that prioritizes ethical conduct, transparency, and accountability in government and software development.
Public participation and engagement: Opportunities for the public to voice their opinions and concerns about government policies and software development projects.
Cross-disciplinary collaboration: Collaboration between the fields of government and software development to develop innovative solutions that draw on the expertise of both fields.

In conclusion, this glossary provides readers with a quick reference guide to key terms and concepts discussed throughout this book. By understanding these concepts, we can better understand the challenges of political debt and technical debt and develop effective strategies for managing and reducing these challenges.

Chapter 17: Further Reading

In this final chapter, we provide a list of recommended books, articles, and resources for readers who are interested in further exploring the topics of political debt and technical debt.

The Concept of Political Debt by Benoît Hamon and Yves Sintomer

This book provides a comprehensive analysis of the concept of political debt, exploring its causes and consequences, and offering strategies for managing and reducing political debt.

Technical Debt: A Framework for Analysis by Steven Fraser and Joanne F. Ho

This article provides a framework for analyzing technical debt in software development, and offers strategies for managing and reducing technical debt.

Democracy in America by Alexis de Tocqueville

This classic book explores the principles of democracy and democratic governance in the United States, and provides insights into the challenges of democratic accountability and representation.

The Agile Manifesto

This influential document outlines the principles of agile software development, which prioritize sustainable development practices and prioritize customer satisfaction.

The Center for Democracy and Technology

The Center for Democracy and Technology is a nonprofit organization that advocates for policies that promote transparency, accountability, and democratic governance in government and technology.

The National Democratic Institute

The National Democratic Institute is a nonprofit organization that promotes democratic governance and accountability around the world, providing technical assistance and training to government officials and civil society organizations.

In conclusion, these resources provide readers with a variety of perspectives and insights into the challenges of political debt and technical debt, and offer strategies for managing and reducing these challenges. By engaging with these resources, readers can deepen their understanding of these important issues and contribute to ongoing efforts to promote transparency, accountability, and ethical leadership in government and software development.

Chapter 18: Acknowledgements

In writing this book, we have benefited from the insights and support of many individuals and organizations, and we would like to take this opportunity to express our gratitude.

We would like to thank our colleagues in government and software development who have shared their experiences and insights with us, providing valuable perspectives on the challenges of political debt and technical debt.

We would also like to thank our academic colleagues who have provided feedback and guidance throughout the writing process, helping us to refine our ideas and arguments.

Finally, we would like to thank our families and loved ones for their unwavering support and encouragement throughout this project.

In conclusion, we would like to express our gratitude to all those who have contributed to this book, and we hope that it will serve as a valuable resource for readers interested in understanding and addressing the challenges of political debt and technical debt.

Chapter 19: About the Authors

[Insert the names of the authors and a brief bio for each author, including their areas of expertise and any relevant publications.]

[Insert a professional headshot or photo of each author.]

In conclusion, we hope that readers have found this book informative and engaging. We believe that the challenges of political debt and technical debt are significant, but not insurmountable, and that through sustained commitment and innovative thinking, we can develop effective strategies for managing and reducing these challenges. We encourage readers to continue to explore these important issues, and to contribute to ongoing efforts to promote transparency, accountability, and ethical leadership in government and software development.

Chapter 20: Book Summary

We began by defining political debt and technical debt, exploring their similarities and differences and examining the impact of these challenges on government and software development, respectively. We then examined the causes and consequences of political debt, including the influence of special interest groups and individuals on government decision-making processes and the impact of political debt on democratic accountability and representation. We also explored the causes and consequences of technical debt, including the impact of short-term thinking, the pressure to deliver software products quickly, and the rapid pace of technological change on software development practices.

We then turned to strategies for managing and reducing political debt and technical debt. For political debt, we discussed the importance of promoting transparency, accountability, and ethical leadership in government, prioritizing sustainable development practices, and promoting public participation and engagement in the political process. For technical debt, we discussed the importance of prioritizing sustainable software development practices, such as refactoring, automation, and technical debt tracking, to ensure that software products are maintainable over the long-term, reducing the overall risk of technical debt.

Throughout the book, we emphasized the importance of cross-disciplinary collaboration and learning opportunities between the fields of government and software development, and the need for sustained commitment and innovative thinking to manage and reduce the challenges of political debt and technical debt effectively.

In conclusion, this book provides readers with a comprehensive overview of the challenges of political debt and technical debt, and offers strategies for managing and reducing these challenges. We hope that readers will find this book informative and engaging, and that it will serve as a valuable resource for those interested in understanding and addressing the challenges of political debt and technical debt.

Chapter 21: Future Directions

In this final chapter, we will explore potential future directions for research and action on political debt and technical debt.

Examining the intersection of political debt and technical debt

While we have examined political debt and technical debt separately in this book, there is a need to explore the intersection of these challenges. This would involve examining the impact of political debt on software development practices and the impact of technical debt on government policies and decision-making processes.

Promoting ethical leadership in government and software development

There is a need to promote ethical leadership in both government and software development to address the challenges of political debt and technical debt. This would involve developing training programs and resources that prioritize ethical conduct, transparency, and accountability.

Strengthening democratic institutions

Strengthening democratic institutions is crucial to promoting democratic accountability and reducing the influence of special interest groups and individuals. This would involve promoting public participation and engagement in the political process, developing effective oversight mechanisms, and addressing the challenges of money in politics.

Incorporating sustainability into software development practices

Incorporating sustainability into software development practices is crucial to reducing the risk of technical debt and ensuring the long-term success of software products. This would involve prioritizing sustainable development practices, such as refactoring, automation, and technical debt tracking, and developing metrics to assess the sustainability of software products.

Examining the impact of emerging technologies on political debt and technical debt

The rapid pace of technological change is likely to have significant implications for political debt and technical debt. There is a need to examine the impact of emerging technologies, such as artificial intelligence, blockchain, and the internet of things, on government policies and software development practices.

In conclusion, there are many potential future directions for research and action on political debt and technical debt. By continuing to explore these challenges and developing innovative solutions that draw on the expertise of both government and software development, we can promote sustainable development practices, strengthen democratic governance, and ensure the long-term success and sustainability of government policies and software development projects.

Links and thoughts:

Top 5 Tweets of the week:

https://twitter.com/svpino/status/1649744866174459907

https://twitter.com/JoannaStern/status/1649782429563904002

https://twitter.com/fchollet/status/1649523626067267584

https://twitter.com/DeepMind/status/1649097822338449409

https://twitter.com/paulg/status/1648283472807559169

What’s next for The Lindahl Letter?

Week 119: All that bad data abounds
Week 120: That one with an obligatory AI trend’s post
Week 121: Considering an independent study applied AI syllabus
Week 122: Will AI be a platform or a service?
Week 123: Considering open source AI

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

This week I ended up trying to find some solid academic articles to read. This time around I found a few that have publicly available downloads to share. Digging around literature related to non-mail polling methods opens the door to a lot of different directions. You will get a sense from the journals listed below that a lot of literature exists and research is benign conducted about public opinion. My interests are more in the automated methods that would help collect sentiment to complete polling without active respondents. Data abounds and some of it holds the key to understanding how sentiment is being expressed. Being able to catalog and share that sentiment is an interesting thing to automate.

Voss, S., Gelman, A., & King, G. (1995). The polls—A review: Preelection survey methodology: Details from eight polling organizations, 1988 and 1992. Public Opinion Quarterly, 59(1), 98-132. http://www.stat.columbia.edu/~gelman/research/published/voss.pdf

Prosser, C., & Mellon, J. (2018). The twilight of the polls? A review of trends in polling accuracy and the causes of polling misses. Government and Opposition, 53(4), 757-790. https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID3184232_code2420517.pdf?abstractid=3184232&mirid=1

Martin, E. A., Traugott, M. W., & Kennedy, C. (2005). A review and proposal for a new measure of poll accuracy. Public Opinion Quarterly, 69(3), 342-369. http://www.asasrms.org/Proceedings/y2003/Files/JSM2003-000913.pdf

Price, V., & Neijens, P. (1998). Deliberative polls: Toward improved measures of “informed” public opinion?. International Journal of Public Opinion Research, 10(2), 145-176. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=3922d1362956bb55df25dbaa8aae8d4706067002

Let’s see what ChatGPT model 3.5 would generate related to this one.

While mail polling remains a valuable tool for collecting data, advances in technology have led to the development of a variety of non-mail polling methods. In this literature study, we will explore the latest research on non-mail polling methodology.

Firstly, online polling has become increasingly popular, particularly for collecting data quickly and cost-effectively. A study by the Pew Research Center found that online polling can be just as accurate as traditional phone or mail polling, but response rates may be lower, particularly among certain populations such as older adults or those with lower levels of education.

Secondly, phone polling remains a popular method for collecting data, particularly for political polling. However, the rise of mobile phones has presented new challenges for phone polling. A study by the European Social Survey found that mobile phone users are more likely to be younger and have higher levels of education, leading to a potential bias in the results. Additionally, increasing rates of spam and robo-calls may lead to declining response rates.

Thirdly, social media has presented new opportunities for pollsters, particularly for collecting data on younger populations. However, concerns have been raised about the representativeness of social media polls, as they may attract self-selected respondents and may not be representative of the broader population.

Fourthly, researchers have explored the use of text message polling, particularly for collecting data quickly and with high response rates. A study by the National Opinion Research Center (NORC) found that text message polling had higher response rates than email or online polling, particularly among younger populations.

Finally, researchers have also explored the use of location-based polling, which involves collecting data from individuals based on their geographic location. This method can be particularly useful for collecting data on local issues, but concerns have been raised about the potential for privacy violations.

In conclusion, non-mail polling methods have become increasingly popular in recent years, particularly for collecting data quickly and cost-effectively. While online, phone, social media, text message, and location-based polling all have their advantages and disadvantages, researchers have found that no single method is universally superior. The choice of polling method should depend on the research question, the population being surveyed, and the resources available.

Links and thoughts:

Top Tweets of the week:

https://twitter.com/CaseyNewton/status/1646886386031788032

https://twitter.com/robmay/status/1646862194775130114

What’s next for The Lindahl Letter?

Week 118: A paper on political debt as a concept vs. technical debt
Week 119: All that bad data abounds
Week 120: That one with an obligatory AI trend’s post
Week 121: Considering an independent study applied AI syllabus
Week 122: Will AI be a platform or a service?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Searching around on Google Scholar is something that I’m pretty adept at by now. This happens to be the 116th Substack post and almost all of them have involved some type of research. Trying to get a set of the best articles on mail polling methodology was actually really challenging [1]. I know that modern polling has moved away from mail polling in general. It’s expensive. The response rate has gotten worse over time. Probably the worst part about it is that compared to phone or internet based polling it is very slow. My interest here is to really understand polling methodologies. Getting to a high level of understanding about these methodologies will help me evaluate that current literature against automated sentiment analysis. It’s the combination of those two literatures that really has my attention at the moment.

Google Scholar was actually recommending a related search, “design method mail and internet surveys” [2]. I’ll admit that the results on that search were a little more interesting, but none of it was really targeted at what I wanted to learn about. I ended up broadening my searches to include content related to mail surveys. That yielded a few highly cited sources. One of the problems with this area of research compared to AI or ML is that all the research is not easily accessible for download.

Brown, M. L. (1965). Use of a postcard query in mail surveys. The Public Opinion Quarterly, 29(4), 635-637.

Sudman, S., Sirken, M. G., & Cowan, C. D. (1988). Sampling rare and elusive populations. Science, 240(4855), 991-996.

Dillman, D. A. (1991). The design and administration of mail surveys. Annual review of sociology, 17(1), 225-249.

Scott, C. (1961). Research on mail surveys. Journal of the Royal Statistical Society: Series A (General), 124(2), 143-195.

Kanuk, L., & Berenson, C. (1975). Mail surveys and response rates: A literature review. Journal of marketing research, 12(4), 440-453.

Maybe you were wondering what ChatGPT model 3.5 would come up with today. Using the prompt, “write a substack post called, ‘A literature study of mail polling methodology’”, a post was created.

A literature study of mail polling methodology

In an era where online and mobile polling have become more popular, it can be easy to overlook the traditional method of mail polling. However, mail polling remains a valuable tool for collecting data, particularly for certain populations. In this literature study, we will explore the latest research on mail polling methodology.

Firstly, studies have shown that mail polling can be particularly effective for reaching older populations, who may be less likely to use online or mobile polling methods. A study by the Pew Research Center found that mail polling had higher response rates among adults aged 65 and older than online or phone polling.

Secondly, mail polling can also be useful for collecting data on sensitive or controversial topics. A study by the National Opinion Research Center (NORC) found that mail polling resulted in higher rates of disclosure on topics such as drug use and sexual behavior, possibly because respondents feel more comfortable disclosing sensitive information in writing rather than in person or over the phone.

Thirdly, mail polling can be useful for collecting data from hard-to-reach populations, such as those without internet or phone access. A study by the American Association for Public Opinion Research (AAPOR) found that mail polling resulted in higher response rates among rural populations than online or phone polling.

Fourthly, researchers have explored the use of incentives to increase response rates in mail polling. Studies have shown that offering incentives such as cash or gift cards can lead to higher response rates, although the effectiveness of incentives may vary depending on the population being surveyed.

Finally, researchers have also explored the use of mixed-mode polling, which combines mail polling with other methods such as online or phone polling. This approach can help reach a wider range of respondents and increase response rates.

In conclusion, mail polling remains a valuable tool for collecting data, particularly for certain populations. While online and mobile polling methods have become more popular, mail polling should not be overlooked, particularly for collecting data on sensitive topics or hard-to-reach populations. The use of incentives and mixed-mode polling can also help increase response rates and improve the accuracy of results.

Links and thoughts:

Top 5 Tweets of the week:

It’s possible that these tweets will not show up as embeds and will just be links. This is due to a change in how Twitter is handling things and not a Substack issue.

https://twitter.com/sama/status/1644878154870427649

https://twitter.com/jdevalk/status/1644439739645194279

https://twitter.com/MKBHD/status/1644562346843467776

https://twitter.com/AndrewYNg/status/1644418518056861696

https://twitter.com/CaseyNewton/status/1644358645084459013

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=mail+polling+methodology&oq=mail+polling+met

[2] https://scholar.google.com/scholar?hl=en&as_sdt=0,6&qsp=1&q=%22design+method%22+mail+and+internet+surveys&qst=br

What’s next for The Lindahl Letter?

Week 117: A literature study of non-mail polling methodology
Week 118: A paper on political debt as a concept vs. technical debt
Week 119: All that bad data abounds
Week 120: That one with an obligatory AI trend’s post
Week 121: Considering an independent study applied AI syllabus

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Earlier today I did end up getting ChatGPT Plus for a cost of USD $20 a month. That apparently will provide me better availability during high demand, faster response speeds, and priority access to new features [1]. I’m not sure if this subscription will be worth keeping. I’m about to open up the latest model GPT-4 and see what it is able to accomplish in terms of advanced reasoning, complex instructions, and more creativity. I have spent the last 6 weeks thinking about writing these polling related papers. A lot of that time and effort was spent trying to figure out what the trajectory of the current literature was and what that meant for the future of understanding and evaluating sentiment. To me polling is a sampling of sentiment on something. It’s an evaluation of respondent preference or attitude. I’m writing that explanation without using the word opinion. That word choice happened on purpose.

A lot of opportunity exists to study breakdowns in modern polling techniques. This is the start of my series of inquiries into that potential area of study. To be really clear here upfront in this analysis I believe that respondent fatigue, general unavailability, and systemic methodology breakdowns have made modern polling problematic. You can see commentary in very public news sources about people being frustrated with polling [2][3]. I’m surprised so far that more of the literature did not openly discuss the breakdown in previously solid methodologies for polling the public.

We are here working on these series of literature evaluations, because I’m principally interested in opinion polling and sentiment analysis. You could go out to the Pew Research Center to learn about polling basics [4]. They present research and polling in some pretty easy to understand ways. That visit to Pew might even send you in the direction of the American Association for Public Opinion Research (AAPOR) [5]. The outcome of that digging might point you in the direction of some methods of administering polling.

Traditional mail survey questionnaire
Phone based delivery of questionnaire
Email survey questionnaire
Web based questionnaires

I have spent some time listening to Nate Silver on podcasts over the years. One of the interesting things Nate shared was that a recent batch of polls was more accurate than expected [6]. That made me wonder if the metrology being used had improved or if they were just tuning properly based on some type of modeled expectation. I will caveat here that I read Nate’s book:

Silver, Nate. The signal and the noise: Why so many predictions fail-but some don't. Penguin, 2012.

It was interesting, but I wanted to know more about the future of polling. One of the main questions that I ended up having was about the differences between polling and modeling and what method was better and would end up being more popular over time [7].

Public attitude extraction
Indicator based tracking
Modeled behavior
Simulated response analysis
Persona based economic modeling
Sentiment interviewing

Where did I end up at the end of this analysis? Two things came into focus. First, it was pretty clear to me after digging around that a clear framework for auditing polling methodologies needs to be developed. Second, a sea change in polling is about to occur where modeling as a method or some type of supplemental grounding activity will have to augment traditional polling methodologies. We have already seen extremely convoluted weighting and error ranges become forward reaching into the public mind.

You could search Google Scholar for online polling best practices [8]. You could also take a look at direct search for “modern polling methodology” and see what shows up [9]. Only one result on the first page of that search had more than 100 citations and that was a handbook. It came in at 172 citations.

Donsbach, W., & Traugott, M. W. (Eds.). (2007). The SAGE handbook of public opinion research. Sage.

The PDF was not available over at ResearchGate or on Google Scholar. You can buy the hardcover from Sage for $215 dollars which seems a little bit out of hand. You can rent or buy the eBook from Sage as well which for a lifetime copy would set you back $117 [10].

I did prompt ChatGPT from OpenAI using the GPT-4 model to generate a paper called, "A literature review of modern polling methodology" that turned out to be interesting. My exact prompt on this one was, “write an academic paper called, ‘A literature review of modern polling methodology’”.

Oddly enough on the first attempt it produced a ton of content then had a “network error” with a more specific message of, “There was an error generating a response.” My only option to move forward was to click the “Regenerate response” box and hope for the best. It took 4 regeneration attempts to get a complete output.

Title: A Literature Review of Modern Polling Methodology

Abstract

The present study aims to provide a comprehensive review of the modern polling methodologies, analyzing their strengths and weaknesses, as well as the ongoing challenges faced by the polling industry. This literature review discusses traditional and emerging methodologies, such as random-digit dialing, online surveys, and mixed-mode approaches. Moreover, the impact of social media on public opinion polling is considered, and the potential of data mining and machine learning for future advancements in the field is explored.

Introduction

Public opinion polling plays a significant role in modern democracies, shaping political strategies, and informing policy-making processes. Over the years, several polling methodologies have been developed and refined to collect accurate and reliable data on public opinion. However, recent events, such as the 2016 United States Presidential Election and the Brexit referendum, have brought the accuracy and reliability of polling methods into question. This literature review aims to examine the current state of polling methodologies, evaluate their strengths and weaknesses, and discuss the future directions of public opinion polling.

Traditional Polling Methodologies

1.1. Random-Digit Dialing (RDD)

RDD has been the gold standard of telephone survey sampling for decades. By generating random phone numbers, researchers ensure that each household has an equal chance of being included in the survey. However, the rapid decline in landline use and the increasing preference for mobile phones has challenged RDD's efficacy (Pew Research Center, 2019).

1.2. Area Probability Sampling

Area probability sampling involves selecting a sample of respondents from a predefined geographical area. This method offers an alternative to RDD and has been proven useful for face-to-face surveys (Lynn & Jäckle, 2020). However, it can be expensive and time-consuming compared to other sampling techniques.

Online Polling Methodologies

2.1. Online Panels

Online panels consist of pre-recruited participants who have agreed to take part in surveys. These panels provide a cost-effective and faster alternative to traditional methodologies. Nevertheless, concerns about sample representativeness and potential biases remain (Baker et al., 2013).

2.2. River Sampling

River sampling is an online methodology where respondents are recruited as they visit websites, often through ads or pop-ups. This method can provide a large number of respondents quickly. However, the lack of control over the selection process raises concerns about the quality and representativeness of the sample (Mercer, 2018).

Mixed-Mode Approaches

Mixed-mode approaches combine multiple data collection methods, such as telephone and online surveys, to compensate for the shortcomings of individual methods (Vavreck & Rivers, 2008). This approach has gained popularity as a means to improve representativeness and reduce nonresponse bias. However, challenges related to mode effects and data integration persist (De Leeuw, 2021).

The Impact of Social Media on Public Opinion Polling

Social media platforms have transformed how people communicate and consume information. As such, there is growing interest in harnessing social media data to understand public opinion (Jungherr et al., 2021). Sentiment analysis and network analysis have emerged as promising techniques for analyzing social media data, but concerns about data quality, privacy, and representativeness need to be addressed (Stier et al., 2020).

The Future of Polling: Data Mining and Machine Learning

Data mining and machine learning techniques offer potential for improving the accuracy and efficiency of public opinion polling. For instance, these techniques can be employed to correct biases, enhance sample representativeness, and identify patterns in survey responses (Wang et al., 2019). However, the ethical implications of these methods, particularly with

You may have noticed that the model output stopped in the middle of a sentence. It seemed like a better idea to change up the prompt for GPT-4 and request it to outline a paper instead of writing it. That seemed like a better way to get an actual complete set of output from the model. The previous 4 network errors were actually rather frustrating. OpenAI is only allowing 30 prompts per 3 hours and having to use them up to overcome network errors was frustrating.

Title: A Literature Review of Modern Polling Methodology

Abstract:

This paper provides a comprehensive review of the modern polling methodology, focusing on the evolution of survey techniques, data collection methods, and analysis tools in the field of public opinion research. By examining the strengths and weaknesses of these methodologies, we aim to identify best practices and guide future research in the field of political and social polling.

Introduction

1.1 Background and purpose of the study

1.2 Scope and limitations of the review

Evolution of Polling Methodology

2.1 Early beginnings: Straw polls and face-to-face interviews

2.2 Telephone surveys: The rise and decline

2.3 The advent of the internet: Online surveys and panels

2.4 Mobile-based polling: SMS and mobile app surveys

2.5 Social media as a data source

Data Collection Techniques

3.1 Probability sampling methods

3.1.1 Simple random sampling

3.1.2 Systematic sampling

3.1.3 Stratified sampling

3.1.4 Cluster sampling

3.2 Non-probability sampling methods

3.2.1 Convenience sampling

3.2.2 Quota sampling

3.2.3 Snowball sampling

3.2.4 Purposive sampling

3.3 Comparing probability and non-probability sampling techniques

Questionnaire Design and Implementation

4.1 Principles of effective questionnaire design

4.2 Types of survey questions

4.2.1 Open-ended questions

4.2.2 Closed

Links and thoughts:

Top 5 Tweets of the week:

Footnotes:

[1]

https://chat.openai.com/chat

[2] https://www.nytimes.com/interactive/2022/10/24/opinion/frustrated-with-polling-pollsters-are-too.html

[3] https://www.nationalaffairs.com/publications/detail/the-trouble-with-polling

[4] https://www.pewresearch.org/course/public-opinion-polling-basics/

[5]

https://aapor.org/

[6] https://fivethirtyeight.com/videos/polls-havent-been-this-accurate-since-at-least-1998/

[7] https://www.majoritystrategies.com/blog/polling-or-modeling-which-do-you-need/

[8] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C10&q=best+practices+online+polling&oq=online+polling

[9] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C10&q=modern+polling+methodology&btnG=

[10] https://us.sagepub.com/en-us/nam/the-sage-handbook-of-public-opinion-research/book228115

What’s next for The Lindahl Letter?

Week 116: A literature study of mail polling methodology
Week 117: A literature study of non-mail polling methodology
Week 118: A paper on political debt as a concept vs. technical debt
Week 119: All that bad data abounds
Week 120: That one with an obligatory AI trend’s post

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Imagine you move part of your workload to an environment that you don’t fully own. That used to be a datacenter and more recently it is probably a cloud environment. Sooner or later you might think about how to secure that environment in terms of hardware-based security or other ways to secure things. Most of the promotional pages on the internet covering this one try to bring forward the idea of securing code and data [1]. Seriously, the idea of confidential computing is everywhere on these cloud pages, but not something you hear about all the time generally [2][3][4]. It is a topic that receives a lot of industry specific coverage related to managing and operating cloud environments.

The low number of highly cited “confidential computing” articles over at Google Scholar was probably indicative of the relative interest in this topic [5]. Very few of the available academic works are highly cited in this area. It’s actually an area where some research opportunity exists related to a survey of security and the outcomes of confidential computing in practice.

A few of the more interesting ones with a higher level of citation ended up being:

Mulligan, D. P., Petri, G., Spinale, N., Stockwell, G., & Vincent, H. J. (2021, September). Confidential Computing—a brave new world. In 2021 international symposium on secure and private execution environment design (SEED) (pp. 132-138). IEEE. https://dominicpm.github.io/publications/mulligan-seed2021.pdf

Rashid, F. Y. (2020). The rise of confidential computing: Big tech companies are adopting a new security model to protect data while it's in use-[news]. IEEE Spectrum, 57(6), 8-9. https://ieeexplore.ieee.org/iel7/6/9099910/09099920.pdf

Zhu, J., Hou, R., Wang, X., Wang, W., Cao, J., Zhao, B., ... & Meng, D. (2020, May). Enabling rack-scale confidential computing using heterogeneous trusted execution environment. In 2020 IEEE Symposium on Security and Privacy (SP) (pp. 1450-1465). IEEE. https://heartever.github.io/files/hetee.pdf

The central question here seems to be if you are going to have your data and run your workloads in the cloud how do you really make sure that effort is confidential. At this point in my analysis, I thought it might be interesting to see what ChatGPT would produce on this topic.

OpenAI ChatGPT Prompt: “scrap that and rewrite it with ethics included”

Sure, here's a rewritten version of the substack post about AI and confidential computing with a focus on ethics:

The intersection of AI and confidential computing has become increasingly important as we realize the importance of privacy and security in computing. Confidential computing is a field that seeks to protect sensitive data and computations by ensuring that they remain encrypted and secure throughout the computing process. Meanwhile, AI relies heavily on large amounts of data to make accurate predictions and decisions, but this data is often sensitive and needs to be kept confidential to protect individuals’ privacy. This is where confidential computing can play an important role, as it can ensure that sensitive information remains private, even when being processed by AI algorithms.

The integration of confidential computing into AI has several benefits. First, it ensures that personal data is kept secure and private, which is crucial for industries such as healthcare and finance. Second, it enables collaboration between organizations without risking the leak of sensitive information. Lastly, it can improve the accuracy and effectiveness of AI models as more data can be used for training without compromising privacy.

However, there are also ethical considerations when it comes to the intersection of AI and confidential computing. For example, there is a risk that confidential computing could be used to conceal unethical or discriminatory practices. It is important to consider how confidential computing is used and to ensure that it is not used to violate ethical principles. Additionally, there is a need to ensure that confidential computing is not used to perpetuate biases that already exist in datasets.

To address these ethical concerns, it is important to ensure that the development and deployment of confidential computing technologies are guided by ethical principles. Transparency and accountability should be central to the development of these technologies, and there should be a focus on ensuring that they are used in a way that promotes fairness, accountability, and transparency. This requires collaboration between computer scientists, ethicists, and policymakers to ensure that these principles are integrated into the design and use of confidential computing technologies.

In conclusion, the intersection of AI and confidential computing has promising implications for privacy and security. However, it is important to ensure that these technologies are developed and used ethically. By integrating ethical considerations into the design and use of confidential computing technologies, we can ensure that these technologies are used in a way that promotes fairness, accountability, and transparency.

Alright, so you can get a sense of what I produced related to confidential computing, then you can get a sense of what the ChatGPT model produced and it’s interesting. It’s an interesting thing here. I spent a couple hours researching and doing work on what exactly people are doing with confidential computing trying to figure out what information was available from scholarly sources on how people were doing confidential computing and how that methodology was occurring and then wrote a Substack post. It took ChatGPT a few seconds to spit out what it produced. I think the depth of the reference to specific sources I think the difference in looking at what happens from the scholarly output compared to how it’s being used, none of that occurred within the ChatGPT. It definitely is very surface level in terms of how it made the argument, but it did produce it very quickly.

Links and thoughts:

Top 7 Tweets of the week:

Footnotes:

[1] https://www.intel.com/content/www/us/en/security/confidential-computing.html

[2] https://azure.microsoft.com/en-us/solutions/confidential-compute/

[3] https://cloud.google.com/confidential-computing

[4] https://www.ibm.com/topics/confidential-computing

[5] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=confidential+computing&btnG=

What’s next for The Lindahl Letter?

Week 115: A literature review of modern polling methodology
Week 116: A literature study of mail polling methodology
Week 117: A literature study of non-mail polling methodology
Week 118: A paper on political debt as a concept vs. technical debt
Week 119: All that bad data abounds

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

You may remember that I listened to an audiobook during a road trip back to Kansas about AI.

Kissinger, H. A., Schmidt, E., & Huttenlocher, D. (2021). The age of AI: and our human future. Hachette UK.

Former CEO of Google Eric Schmidt has had a lot to say about the future of AI and a lot of that has been grounded in the implications of AI on society. It’s important to set the foundation of any introduction to AI ethics on the implications to people and society.

Crowdsourcing a really solid introduction to AI ethics seems like something that would be easy enough to achieve. People have a lot of somewhat divergent thoughts on what exactly AI ethics includes. This effort is my attempt at structuring an introduction to AI ethics. I’ll take this and produce an independent study version of this using Overleaf LaTex to make it extra awesome. My last run at this was about 6 months ago during week 86 that was primarily about ML ethics centering on fairness, bias, and privacy as core topics [1]. It might be a good start to dig deeper this time around and look for more topics to include with a larger focus on AI and not staying limited to the world of ML. I cloned my previous project in Overleaf and then removed all the extra content. Not only should it be pretty easy to take this structure or outline and expand it to a longer form for publication, but also it is probably time to revisit using LaTeX before that rather academic skill set fades.

What are other major topics to consider besides fairness, bias, and privacy?

Laws and legislation
Intellectual property
Academic citation
Human-centered design
Examples (applied or theoretical)
Data sourcing
Labor practices
Frameworks
Operationalizing
Misinformation

You could go out and read “The Ethics of Artificial Intelligence” by Nick Bostrom and Eliezer Yudkowsky [2]. Or maybe go out to the “Everyday Ethics for Artificial Intelligence” from the team over at IBM [3].

As a little bonus for you here, instead of closing out this Substack post I’m going to provide you with a few chapters from a session of content generation I had with OpenAI’s ChatGPT. The session was seeded with a request to write an introduction to AI ethics. You will find that my prompts are very direct and highly word economic. The output here is really just based on getting the ChatGPT large language model to keep generating as the session output is limited per interaction. You will see evidence of this as it sometimes just sometimes appears to stop mid chapter or closes out abruptly.

OpenAI ChatGPT Prompt: “write a summary for the book”

The book is about the ethical considerations of artificial intelligence (AI) and its impact on society. It covers a wide range of topics including bias and fairness, transparency and explainability, privacy and security, the impact of AI on employment and the economy, the use of AI in sensitive domains, and the future of AI and its potential impact on society. The book emphasizes that ethical considerations surrounding AI are complex and multifaceted, and that it is crucial to stay informed about the latest developments in the field. It also highlights the importance of developing strategies to ensure that the benefits of AI are shared by all members of society, including policies such as retraining programs, universal basic income, and government oversight. The book also emphasizes that AI ethics is an interdisciplinary field that brings together experts from various domains such as computer science, philosophy, law, sociology, psychology, and policy.

OpenAI ChatGPT Prompt: “why should i read this book”

You should read this book because it provides a comprehensive overview of the ethical considerations surrounding AI and its impact on society. The book covers a wide range of topics that are important for understanding the current and future implications of AI on society. It highlights the potential benefits and risks of AI and provides strategies for addressing these challenges. Additionally, the book emphasizes the importance of staying informed about the latest developments in AI and the importance of developing policies to ensure that the benefits of AI are shared by all members of society.

As AI is a rapidly advancing technology that has the potential to shape the future of humanity in ways that are difficult to predict, it is important to be aware of the ethical considerations surrounding AI. This book is a valuable resource for anyone who wants to understand the impact of AI on society and the ethical considerations that need to be taken into account as the technology continues to advance.

OpenAI ChatGPT Prompt: “what would you title book”

"AI Ethics: Navigating the Challenges and Opportunities of Artificial Intelligence"

OpenAI ChatGPT Prompt: “tell me the chapters again”

The book has the following chapters:

Chapter 1: Introduction to AI Ethics

Chapter 2: Bias and Fairness in AI

Chapter 3: Transparency and Explainability in AI

Chapter 4: Privacy and Security in AI

Chapter 5: The Impact of AI on Employment and the Economy

Chapter 6: The Use of AI in Sensitive Domains

Chapter 7: The Future of AI and its Potential Impact on Society

Chapter 8: Conclusion

Chapter 9: References

OpenAI ChatGPT Prompt: “write a dedication for the book”

"Dedicated to all those who strive to make AI a force for good, and to the future generations who will be impacted by its development."

OpenAI ChatGPT Prompt: “who is the author”

As an AI, I do not have a physical presence, so I cannot be an author of a book. But I can provide you with the information and the content of the book.

OpenAI ChatGPT Prompt: “ok. write the first chapter”

Chapter 1: Introduction to AI Ethics

Artificial intelligence (AI) is rapidly advancing and has the potential to transform society in countless ways. As AI becomes more prevalent in our lives, it is crucial to consider the ethical implications of its development and use. AI ethics is the study of the ethical considerations surrounding the design, development, deployment, and use of AI systems.

This chapter will introduce the key ethical issues associated with AI and provide an overview of the field of AI ethics. We will begin by defining AI and discussing its various forms, including narrow AI and general AI. We will then explore the potential benefits and risks of AI, highlighting the ways in which it can be used to improve people's lives and the potential for AI to cause harm.

Next, we will discuss the importance of considering ethical considerations in the development and use of AI. We will explore the role of stakeholders in AI, including developers, users, and society as a whole, and the responsibilities of each of these groups in ensuring that AI is developed and used ethically.

Finally, we will introduce the main ethical frameworks used in AI ethics, including consequentialism, deontology, and virtue ethics. We will also discuss the concept of "machine ethics" and the potential for AI systems to have their own moral principles.

The goal of this chapter is to provide a foundation for the ethical considerations surrounding AI and set the stage for the rest of the book. As we explore the various ethical issues in more depth in the chapters to come, it is important to keep in mind that the ethical considerations surrounding AI are complex and multifaceted. The field of AI ethics is constantly evolving, and it is important to approach these issues with an open mind and a willingness to consider multiple perspectives.

OpenAI ChatGPT Prompt: “ok. write the second chapter”

Chapter 2: Bias and Fairness in AI

One of the most important ethical considerations surrounding AI is the potential for AI systems to perpetuate or exacerbate societal biases. Bias in AI can occur at various stages of the AI development process, including in the data used to train models, in the algorithms used to develop models, and in the deployment of models in real-world settings.

This chapter will explore the concept of bias in AI and discuss strategies for addressing bias in AI systems. We will begin by defining bias and discussing the different types of bias that can occur in AI systems. We will then explore the ways in which bias can occur in the data used to train models, including issues such as underrepresentation and overrepresentation of certain groups.

Next, we will discuss the ways in which bias can be introduced in the algorithms used to develop models, including issues such as fairness and discrimination. We will also explore the potential for AI to perpetuate or exacerbate societal biases, such as discrimination against marginalized groups.

To mitigate these issues, we will discuss different strategies for addressing bias in AI systems. These include techniques for detecting and correcting bias in data and algorithms, as well as the use of fairness metrics to evaluate the performance of AI systems. We will also discuss the importance of transparency and explainability in AI, which can help to ensure that AI systems are fair and unbiased.

Finally, we will discuss the role of regulation and policy in addressing bias in AI, including the importance of government oversight and the need for industry-wide standards and guidelines.

It's important to note that addressing bias in AI is an ongoing process that requires constant monitoring and improvement. It requires a combination of technical, legal and ethical efforts to make sure that AI systems are fair and unbiased towards all individuals and groups.

OpenAI ChatGPT Prompt: “ok. write chapter three”

Chapter 3: Transparency and Explainability in AI

As AI systems become increasingly prevalent in society, it is important to ensure that they are transparent and explainable to humans. Transparency refers to the ability to understand how an AI system makes decisions, and explainability refers to the ability to understand the reasoning behind those decisions.

This chapter will explore the challenges of transparency and explainability in AI and discuss strategies for addressing these challenges. We will begin by defining transparency and explainability in AI and discussing the importance of these concepts in ensuring that AI systems are trustworthy and accountable.

Next, we will discuss the challenges of achieving transparency and explainability in AI, including issues such as the complexity of AI systems, the use of "black box" models, and the limitations of current explainability techniques. We will also explore the potential for AI to make decisions that are difficult or impossible for humans to understand or explain.

To mitigate these challenges, we will discuss different strategies for achieving transparency and explainability in AI. These include techniques for making AI systems more interpretable, such as the use of decision trees and rule-based models, and the use of explainability methods such as feature attribution and saliency maps. We will also discuss the importance of human-centered design in AI, which can help to ensure that AI systems are understandable and usable by humans.

Finally, we will discuss the role of regulation and policy in promoting transparency and explainability in AI, including the importance of government oversight and the need for industry-wide standards and guidelines.

It's important to note that transparency and explainability are important not only for ensuring the accountability of AI systems, but also for building trust with the public. As AI systems are increasingly integrated into society, it becomes essential to develop methods to ensure that AI systems are understandable and explainable to the people who will be impacted by them.

OpenAI ChatGPT Prompt: “ok. write chapter four”

Chapter 4: Privacy and Security in AI

As AI systems become more prevalent in society, it is crucial to consider the privacy and security implications of their development and use. Privacy refers to the ability to control the collection, use, and dissemination of personal data, while security refers to the protection of personal data and AI systems from unauthorized access and malicious attacks.

This chapter will explore the privacy and security challenges associated with AI and discuss strategies for addressing these challenges. We will begin by defining privacy and security in the context of AI and discussing the importance of these concepts in ensuring that AI systems are trustworthy and secure.

Next, we will discuss the privacy and security challenges associated with AI, including issues such as data breaches, data misuse, and the potential for AI-enabled surveillance. We will also explore the potential for AI systems to be used for malicious purposes, such as cyber attacks and identity theft.

To mitigate these challenges, we will discuss different strategies for ensuring privacy and security in AI. These include techniques for protecting personal data, such as encryption and anonymization, and the use of security measures to protect AI systems from unauthorized access and malicious attacks. We will also discuss the importance of responsible data management practices, such as data minimization and data governance, which can help to ensure that personal data is handled in a secure and ethical manner.

Finally, we will discuss the role of regulation and policy in promoting privacy and security in AI, including the importance of government oversight and the need for industry-wide standards and guidelines.

It's important to note that privacy and security are essential to the responsible development and use of AI. As AI systems are increasingly integrated into society, it becomes essential to develop methods to ensure that personal data is protected and AI systems are secure from malicious attacks.

OpenAI ChatGPT Prompt: “ok. write chapter five”

Chapter 5: The Impact of AI on Employment and the Economy

As AI continues to advance, it has the potential to transform the way we work and the structure of the economy. While AI has the potential to create new jobs and improve productivity, it also has the potential to displace existing jobs and contribute to income inequality.

This chapter will explore the impact of AI on employment and the economy and discuss strategies for addressing these challenges. We will begin by discussing the potential benefits and risks of AI in the workplace, including the potential for AI to improve productivity and create new jobs, and the potential for AI to displace existing jobs and contribute to income inequality.

Next, we will discuss the potential impact of AI on different sectors of the economy, including manufacturing, healthcare, and finance. We will also explore the potential for AI to change the way we work, including the rise of the gig economy and the potential for remote work to become more prevalent.

To mitigate these challenges, we will discuss different strategies for addressing the impact of AI on employment and the economy. These include policies such as retraining programs and universal basic income, as well as the importance of creating an economic and social safety net for those who may be displaced by AI. We will also discuss the importance of investing in education and training to ensure that workers have the skills needed to succeed in the new economy.

Finally, we will discuss the role of regulation and policy in addressing the impact of AI on employment and the economy, including the importance of government oversight and the need for industry-wide standards and guidelines.

It's important to note that the impact of AI on employment and the economy is a complex issue that requires a multifaceted approach. As AI continues to advance, it is crucial to consider the potential impact on workers and the economy and to develop strategies to ensure that the benefits of AI are shared by all members of society.

OpenAI ChatGPT Prompt: “ok. write chapter six”

Chapter 6: The Use of AI in Sensitive Domains

As AI systems become more advanced and prevalent in society, they are increasingly being used in sensitive domains such as healthcare, criminal justice, and military applications. The use of AI in these domains raises important ethical considerations, including issues of accountability, fairness, and transparency.

This chapter will explore the ethical considerations associated with the use of AI in sensitive domains. We will begin by discussing the different types of sensitive domains in which AI is being used, including healthcare, criminal justice, and military applications. We will then explore the potential benefits and risks of AI in these domains, highlighting the ways in which AI can be used to improve outcomes and the potential for AI to cause harm.

Next, we will discuss the ethical considerations associated with the use of AI in sensitive domains, including issues of accountability, fairness, and transparency. We will also explore the potential for AI to perpetuate or exacerbate societal biases, such as discrimination against marginalized groups.

To mitigate these issues, we will discuss different strategies for addressing the ethical considerations associated with the use of AI in sensitive domains. These include techniques for detecting and correcting bias in data and algorithms, as well as the use of fairness metrics to evaluate the performance of AI systems. We will also discuss the importance of transparency and explainability in AI, which can help to ensure that AI systems are fair and unbiased.

Finally, we will discuss the role of regulation and policy in addressing the ethical considerations associated with the use of AI in sensitive domains, including the importance of government oversight and the need for industry-wide standards and guidelines.

It's important to note that the use of AI in sensitive domains is a complex and nuanced issue that requires a thorough understanding of the ethical considerations involved. As AI systems are increasingly used in these domains, it is crucial to consider the potential impact on society and to develop strategies to ensure that the use of AI is safe, fair, and transparent.

OpenAI ChatGPT Prompt: “ok. write chapter seven”

Chapter 7: The Future of AI and its Potential Impact on Society

As AI continues to advance, it has the potential to shape the future of humanity in ways that are difficult to predict. While AI has the potential to bring about significant benefits, such as improved healthcare and increased productivity, it also has the potential to cause harm, such as job displacement and the erosion of privacy.

This chapter will explore the potential impact of AI on society in the future and discuss strategies for addressing these challenges. We will begin by discussing the potential benefits and risks of AI in the future, including the potential for AI to improve our lives and the potential for AI to cause harm.

Next, we will discuss the concept of "superintelligent AI" and the potential impact of AI on society if machines surpass human intelligence. We will also explore the potential impact of AI on issues such as privacy, security, and democracy.

To mitigate these challenges, we will discuss different strategies for addressing the future impact of AI on society. These include policies such as retraining programs and universal basic income, as well as the importance of creating an economic and social safety net for those who may be displaced by AI. We will also discuss the importance of investing in education and training to ensure that workers have the skills needed to succeed in the new economy.

Finally, we will discuss the role of regulation and policy in addressing the future impact of AI on society, including the importance of government oversight and the need for industry-wide standards and guidelines.

It's important to note that the future of AI is uncertain and that it is difficult to predict the long-term impact of AI on society. However, it is crucial to consider the potential impact of AI on society and to develop strategies to ensure that the benefits of AI are shared by all members of society. This includes both short-term and long-term considerations, as well as the development of regulation and policy that can guide the responsible development of AI for the future.

OpenAI ChatGPT Prompt: “ok. write chapter eight”

Chapter 8: Conclusion

Artificial intelligence (AI) is a rapidly advancing technology with the potential to transform society in countless ways. The ethical considerations surrounding AI are complex and multifaceted, and it is important to approach these issues with an open mind and a willingness to consider multiple perspectives.

This book has covered various ethical issues associated with AI, including bias and fairness, transparency and explainability, privacy and security, the impact of AI on employment and the economy, the use of AI in sensitive domains, and the future of AI and its potential impact on society.

In conclusion, it is important to keep in mind that the ethical considerations surrounding AI are constantly evolving, and it is crucial to stay informed about the latest developments in the field. It is also important to consider the potential impact of AI on society and to develop strategies to ensure that the benefits of AI are shared by all members of society. This includes policies such as retraining programs, universal basic income, and government oversight. Additionally, it is important to invest in education and training to ensure that workers have the skills needed to succeed in the new economy.

AI ethics is an interdisciplinary field that brings together experts from various domains such as computer science, philosophy, law, sociology, psychology, and policy. As AI continues to advance, it is crucial to continue the conversation about the ethical considerations surrounding AI and to work together to ensure that AI is developed and used in a responsible and ethical manner.

OpenAI ChatGPT Prompt: “ok. write a references section”

References:

Bostrom, N. (2014). Superintelligence: Paths, Dangers, and Strategies. Oxford University Press.

Domingos, P. (2015). The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. Basic Books.

Floridi, L. (2019). The Ethics of Artificial Intelligence. Oxford University Press.

Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning (Adaptive Computation and Machine Learning series). MIT Press.

Gudivada, V. N., Raghavan, V. V., & Riedl, J. (Eds.). (2019). Artificial Intelligence Ethics: A Primer. Springer.

Kember, S., & Loke, L. (2019). Artificial Intelligence: A Guide for Thinking Humans. Farrar, Straus and Giroux.

Russell, S. J., & Norvig, P. (2016). Artificial Intelligence: A Modern Approach (3rd ed.). Prentice Hall.

Zeng, X., & Liu, Y. (Eds.). (2018). Handbook of Research on Artificial Intelligence Ethics and Society. IGI Global.

Note: These are a sample of books and references about AI ethics, there are more recent and thorough resources available.

Footnotes:

[1]

[2] https://intelligence.org/files/EthicsofAI.pdf

[3] https://www.ibm.com/watson/assets/duo/pdf/everydayethics.pdf

What’s next for The Lindahl Letter?

Week 114: How does confidential computing work?
Week 115: A literature review of modern polling methodology
Week 116: A literature study of mail polling methodology
Week 117: A literature study of non-mail polling methodology
Week 118: A paper on political debt as a concept vs. technical debt

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Lectures related to this topic are mostly about convolutional neural networks and visual recognition systems. My searches were explicitly for autonomous vehicles and did not directly include any of the methods being deployed. That in part was due to my interest in going into higher levels of analysis this time around. My searches of Google Scholar on this topic of autonomous vehicles returned results with a lot fewer citations than I was expecting to find [1]. Three of the papers in the first 30 results had more than 1,000 citations.

Fagnant, D. J., & Kockelman, K. (2015). Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendations. Transportation Research Part A: Policy and Practice, 77, 167-181. https://www.caee.utexas.edu/prof/kockelman/public_html/TRB14EnoAVs.pdf

Shah, S., Dey, D., Lovett, C., & Kapoor, A. (2018). Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Field and Service Robotics: Results of the 11th International Conference (pp. 621-635). Springer International Publishing. https://arxiv.org/pdf/1705.05065.pdf

Bonnefon, J. F., Shariff, A., & Rahwan, I. (2016). The social dilemma of autonomous vehicles. Science, 352(6293), 1573-1576. https://arxiv.org/pdf/1510.03346

The rest of the scholarly articles seemed to have around a couple hundred citations. Maybe the collection of scholars interested in autonomous vehicle research is about that size. In practical terms it is probably a fairly expensive area of study. Even building simulated models would be computationally expensive. Collecting real world model data would be cost prohibitive for most researchers.

Faisal, A., Kamruzzaman, M., Yigitcanlar, T., & Currie, G. (2019). Understanding autonomous vehicles. Journal of transport and land use, 12(1), 45-72. https://conservancy.umn.edu/bitstream/handle/11299/209218/JTLU_vol-12_pp45-72.pdf?sequence=1

Schwarting, W., Alonso-Mora, J., & Rus, D. (2018). Planning and decision-making for autonomous vehicles. Annual Review of Control, Robotics, and Autonomous Systems, 1, 187-210. http://alonsomora.com/docs/18-schwarting-AR.pdf

Kato, S., Takeuchi, E., Ishiguro, Y., Ninomiya, Y., Takeda, K., & Hamada, T. (2015). An open approach to autonomous vehicles. IEEE Micro, 35(6), 60-68. http://cs.furman.edu/~tallen/csc271/source/openAppr.pdf

You will notice pretty quickly that all of those scholarly works are aging a bit compared to where we are now at the doorstep of modernity. I’ll be curious about digging into things a bit more over time to find more current state reviews of autonomous vehicles.

Links and thoughts:

Top 5 Tweets of the week:

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=autonomous+vehicles&btnG=&oq=autonomous+v

What’s next for The Lindahl Letter?

Week 113: Structuring an introduction to AI ethics
Week 114: How does confidential computing work?
Week 115: A literature review of modern polling methodology
Week 116: A literature study of mail polling methodology
Week 117: A literature study of non-mail polling methodology

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

We have reached the 7th installment of the new series for 2023 related to AI. In just a couple of weeks we are going to venture out of these framing topics into the heart of my journey back into polling and sentiment analysis. Toward the end of that journey it will be very clear on the relationships between AGI, AI, and sentiment analysis. Processing to the point of understanding then adding a degree of directionality in terms of sentiment may be beyond the current state of things at this moment. A lot of debate will go into what is understanding, the occurrence of it, and does it have to be sustained to be relevant. Commonly people try to sum that debate up within the context of next best alternative consideration and that is interesting. However, that debate could be summed up within the analogy of trying to argue that a really well drawn map could provide understanding. It certainly could provide the pathing and the next best routing assuming you have a general idea of where you are going and that path is covered on the map.

Let’s zoom back out for a minute and look at a relatively current webinar. A recent one hour webinar from Stanford online professor Christopher Potts related to GTP-3 & Beyond was pretty interesting. It’s easy to find on YouTube and you can watch it faster than 1x if you enjoy that sort of thing.

Outside of that consideration, I’m wondering if we are going to end up moving from people writing books and articles to a world where people are certifying books and articles for correctness. Some sort of model is used to produce content and then a person edits it to or just verifies its correctness. People have spent a lot of time trying to figure out how to handle natural language processing (NLP). As editors of that content people could act as the ultimate gatekeeper of knowledge and correctness within what is spit out of the systems. We are after all capable of processing natural language ourselves. It’s just a lot harder to explain or rationalize the rule systems that each individual person uses to achieve that objective.

Here are 5 scholarly works related to NLP:

Manning, C., & Schutze, H. (1999). Foundations of statistical natural language processing. MIT press. http://www.cs.cornell.edu/home/llee/papers/topost.pdf

Bird, S., Klein, E., & Loper, E. (2009). Natural language processing with Python: analyzing text with the natural language toolkit. " O'Reilly Media, Inc.".

Chowdhary, K., & Chowdhary, K. R. (2020). Natural language processing. Fundamentals of artificial intelligence, 603-649. https://strathprints.strath.ac.uk/2611/1/strathprints002611.pdf

Hirschberg, J., & Manning, C. D. (2015). Advances in natural language processing. Science, 349(6245), 261-266. https://nlp.stanford.edu/~manning/xyzzy/Hirschberg-Manning-Science-2015.pdf

Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S., & McClosky, D. (2014, June). The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations (pp. 55-60). https://aclanthology.org/P14-5010.pdf

You could go out and watch Christopher Manning’s “Natural language processing with deep learning CS224N/Ling284” 23 video course on YouTube [1]. This one contains a lot of content.

Maybe you were looking for a more media rich introduction to NLP. The team over at TensorFlow also has a collection of 6 videos on the topic that are going to be a lot more visually involved [2]. They are also considerably shorter than the 23 video course from Christopher Manning.

Rounding this one out will be a video from the IBM Cloud team which was pretty decent.

Links and thoughts:

Top 4 Tweets of the week:

Footnotes:

[1]

[2]

What’s next for The Lindahl Letter?

Week 112: Autonomous vehicles
Week 113: Structuring an introduction to AI ethics
Week 114: How does confidential computing work?
Week 115: A literature review of modern polling methodology
Week 116: A literature study of mail polling methodology

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

In total the largest knowledge graph ever created on the shoulders of giants happens to be the college academy of knowledge compiled as a part of our higher education traditions. That knowledge graph is inherently built on connections and the learning patterns of academics. It would have to be considered to be a very human based aggregate not a computer based one. It’s possible that could change over time, but I’m not entirely convinced that is going to happen in the short term. The totality of academics is a much more complex knowledge collection than any model. It also contains a lot of conflicting arguments and disagreement.

Here is a video from IBM Technology that explains, “What is a knowledge graph?”

If you wanted to really go deeper, then you could check out this video from Yanic.

Wang, C., Liu, X., & Song, D. (2020). Language models are open knowledge graphs. arXiv preprint arXiv:2010.11967. https://arxiv.org/pdf/2010.11967.pdf

Here are 3 academic papers that are highly cited within the context of knowledge graphs.

Nickel, M., Murphy, K., Tresp, V., & Gabrilovich, E. (2015). A review of relational machine learning for knowledge graphs. Proceedings of the IEEE, 104(1), 11-33. https://ieeexplore.ieee.org/ielaam/5/7360840/7358050-aam.pdf

Ji, S., Pan, S., Cambria, E., Marttinen, P., & Philip, S. Y. (2021). A survey on knowledge graphs: Representation, acquisition, and applications. IEEE transactions on neural networks and learning systems, 33(2), 494-514. https://arxiv.org/pdf/2002.00388.pdf%E2%80%8Barxiv.org

Wang, Z., Zhang, J., Feng, J., & Chen, Z. (2014, June). Knowledge graph embedding by translating on hyperplanes. In Proceedings of the AAAI conference on artificial intelligence (Vol. 28, No. 1). https://ojs.aaai.org/index.php/AAAI/article/download/8870/8729

Academic papers are really just a series of single meals within the greater academy of academic work. You have to really be committed to bringing together your overall thoughts on the field, the most relevant publications, and ultimately the totality of things happening in your field of academic study. You cannot for the most part hang your hat on any one academic paper being able to explain it all. Together with a multitude of other papers the content has more context. Sometimes people stop eating single serving meals to write introductions, textbooks, or other onramp materials to help people catch up. My fear has been that within the ML and AI spaces the flooding of content has made catching up almost impossible for people wanting to find the on ramp to getting up to speed on everything that is happening. This problem exists in different ways for researchers and scholars that are just trying to keep up with the endless stream of new content along the way. Every day, month, and year new papers are being published at rates that are mind boggling. Now that synthetic written content has gone mainstream thanks to ChatGPT kickstarting that marketplace. My honest guess here is that a lot of researchers will kickstart their writing endeavors with the rocket booster for written content creation that is ChatGPT. They can just edit it down and move along to publish more papers.

That preamble is super important to trying to contextualize why understanding the knowledge graph is so important. An important factor within the larger aggregate knowledge graph we all share itself is changing. It’s really like somebody showed up to the party and invited way more people than could possibly attend. Yeah, I’m about to talk about the effects of content flooding on future knowledge graph management. When the intake protocols for the knowledge graph cannot tell the difference between synthetically created input data and the standard product that has been on the market for thousands of years. For the most part knowledge graph based algorithms have had a hard time dealing with satire and sarcasm content where the author is intentionally spinning a false narrative. You end up with two conflicting next best answers where one should have a very low confidence score. However, this new effort within ChatGPT is whole-sale different. You have content that looks and seems like it should be included in the knowledge graph, but it is not tested or curated in any way shape or form. It’s almost like a point in time has to be considered where things before ChatGPT and things after ChatGPT have different weights within the overall knowledge graph. That is a troubling idea to consider.

Top 5 Tweets of the week:

What’s next for The Lindahl Letter?

Week 111: Natural language processing
Week 112: Autonomous vehicles
Week 113: Structuring an introduction to AI ethics
Week 114: How does confidential computing work?
Week 115: A literature review of modern polling methodology

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Right before sitting down to work on this post I recorded the audio for week 108. My voice has almost recovered all the way from catching that pandemic thing in December. It’s interesting that sustaining vocal quality would be one of the last things to stabilize from that distinctly unwanted experience. You have probably been able to hear that within the last few recorded podcasts. I just did not have that totality of vocal control. I might go back and record that audio segment again, but at this point it is entirely possible that won’t happen based on time and the weight of other commitments this week. I was also a little concerned when Audacity, the software I use for this recording, required an update before the recording started, but it was apparently a small one and I was able to start the recording process within a couple of minutes. It’s very possible that this missive would go out without an audio edition. You however know that is not the case.

It turned out that I was able to sit down and record the audio version of this Substack post. Right now I’m working without my backlog due to some unavoidable things that disrupted my writing routine a bit. I’m hopeful that given a little bit of time a few weeks of backlog will build back up, but right now each post is as fresh as it will ever be as things are being written directly before being recorded as part of my weekend writing routine. Typically it is better to have the posts created within the 5 week planning and review cycle. At this point, that is not possible. Welcome to the now and being in the moment as the very freshest words are brought your way within this missive of consideration for Substack.

For the most part households adopted some machines pretty quickly and in a sustained way. Depending on where you are, microwaves, refrigerators, freezers, dishwashers, washing machines, and dryers are a part of the technology footprint in households. None of those devices need to be connected to either WiFi or Bluetooth to be operational. They existed for many years without that type of connectivity. This essay happens to be about robots in the house and the aforementioned appliances are not really what people talk about in terms of modern robotics in the household. It’s something more mobile that gets a lot of attention. To be fair a lot of robot vacuum brands and companies now exist. They roam and clean, get stuck, and have to be rescued and maintained.

I went out to find some papers that referenced the Roomba and they seem to have peaked between 2006 to 2007. Which I thought was a very interesting element to see within this literature review. They just sort of stopped in frequency. Here are three of them that were well referenced.

Forlizzi, J., & DiSalvo, C. (2006, March). Service robots in the domestic environment: a study of the roomba vacuum in the home. In Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction (pp. 258-265). https://www.cs.cmu.edu/~kiesler/publications/2006pdfs/2006_service-robots-roomba.pdf

Tribelhorn, B., & Dodds, Z. (2007, April). Evaluating the Roomba: A low-cost, ubiquitous platform for robotics research and education. In Proceedings 2007 IEEE International Conference on Robotics and Automation (pp. 1393-1399). IEEE. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=15cb2263caf26ac68906858093ae8d7749ad7827

Jones, J. L. (2006). Robots at the tipping point: the road to iRobot Roomba. IEEE Robotics & Automation Magazine, 13(1), 76-78. https://www.researchgate.net/profile/Joseph-Jones-14/publication/3344755_Robots_at_the_tipping_point_the_road_to_iRobot_Roomba/links/5728aa7308ae2efbfdb7dce8/Robots-at-the-tipping-point-the-road-to-iRobot-Roomba.pdf

None of these robots in the house are equipped with any conversational subroutines. Within the worlds created by science fiction writers it is not uncommon for the robots in the house to talk back and demonstrate some degree of personality. We currently have no legitimate AGI that would facilitate that type of exchange. It’s probably up next at some point. Now that both Microsoft (powered by OpenAI) and Google are trying to create chat-like interactions I’m guessing that chatting with the robots in the house will arrive as a feature. Right now in Google Scholar searches you can find almost 5,000 results for ChatGPT [1].

Links and thoughts:

Top 6 Tweets of the week:

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=chatgpt&btnG=

What’s next for The Lindahl Letter?

Week 110: Understanding knowledge graphs
Week 111: Natural language processing
Week 112: Autonomous vehicles
Week 113: Structuring an introduction to AI ethics
Week 114: How does confidential computing work?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Part of what made Twitter so interesting is the diversity of argument and the townhall nature of it being the first place things show up in the feed. I’m not sure any other company or social platform could attract the same amount of hyperactive content creation users geared at news and coverage of the moment. People are arguing and I’m sure papers will soon be arriving to describe the end of social media. This weekend I’m going to spend a bit of time reading a book by Robert Putnam of “Bowling Alone” fame called “The Upswing”.

Putnam, R. D. (2000). Bowling alone: The collapse and revival of American community. Simon and schuster.

Putnam, R. D. (2020). The upswing: How America came together a century ago and how we can do it again. Simon and Schuster.

It’s probably somewhere in the meta analysis between social capital and social media that a compelling story about why Twitter as a company would not happen today exists. During the course of this analysis you are going to receive two different lines of inquiry. First, I’ll consider the nature of Twitter and a few books related to it and silicon valley in general. Second, we will dig into some of the AI and sentiment analysis scholarly work related to that field of study to help keep the writing trajectory for the year on track.

Books have arrived to tell the stories of what happened in Silicon Valley. A lot of unlikely things happened within the borders of the space described as silicon valley. Some of them will be a part of business courses for decades to come. It truly is an interesting thing that happened where so much creativity and output happen in such a relatively small area.

Three of the books that I have enjoyed are listed below.

Bilton, N. (2014). Hatching Twitter: A true story of money, power, friendship, and betrayal. Penguin.

Frier, S. (2021). No filter: The inside story of Instagram. Simon and Schuster.

Wiener, A. (2020). Uncanny valley: A memoir. MCD.

You can zoom out a bit and grab some classic silicon valley reading like:

Isaacson, W. (2014). The innovators: How a group of inventors, hackers, geniuses and geeks created the digital revolution. Simon and Schuster.

A lot of scholars over the years have focused their attention on Twitter for a variety of purposes. You can imagine that my interest and the interest of those scholars overlap around the ideas of AI and sentiment analysis. Digital agents abound within the Twitter space and some of them are doing some type of sentiment analysis with what scholars are identifying as artificial intelligence. That second part of the equation makes me a little bit skeptical about the totality of the claims being made. We will jump right into the deep end of Google Scholar on this one anyway [1].

Papers from a search for “Sentiment analysis Twitter artificial intelligence” [2]

Kouloumpis, E., Wilson, T., & Moore, J. (2011). Twitter sentiment analysis: The good the bad and the omg!. In Proceedings of the international AAAI conference on web and social media (Vol. 5, No. 1, pp. 538-541). https://ojs.aaai.org/index.php/ICWSM/article/download/14185/14034

Ghiassi, M., Skinner, J., & Zimbra, D. (2013). Twitter brand sentiment analysis: A hybrid system using n-gram analysis and dynamic artificial neural network. Expert Systems with applications, 40(16), 6266-6282.

Giachanou, A., & Crestani, F. (2016). Like it or not: A survey of twitter sentiment analysis methods. ACM Computing Surveys (CSUR), 49(2), 1-41. https://arxiv.org/pdf/1601.06971.pdf

Alsaeedi, A., & Khan, M. Z. (2019). A study on sentiment analysis techniques of Twitter data. International Journal of Advanced Computer Science and Applications, 10(2). https://www.researchgate.net/profile/Abdullah-Alsaeedi/publication/331411860_A_Study_on_Sentiment_Analysis_Techniques_of_Twitter_Data/links/5c78175ba6fdcc4715a3d664/A-Study-on-Sentiment-Analysis-Techniques-of-Twitter-Data.pdf

I had considered some evaluation of searches for both “opinion mining Twitter artificial intelligence” and “artificial intelligence analysis of public attitudes” [3][4]. It’s possible some papers from both of those searches show up later. Generally, all of that argument and content could be broken down into two camps of intelligence gathering related to advertising and general opinion mining geared at understanding sentiment. One divergent thread of research from those two would be some of the efforts to identify fake or astroturf content. You can imagine that flooding either fake or astroturf content could change the dynamic for advertising or sentiment analysis. Advertising to a community of bots is a rather poor use of scarce resources.

Links and thoughts:

Top 5 Tweets of the week:

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=twitter+artificial+intelligence&oq=twitter+artif

[2] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=Sentiment+analysis+Twitter+artificial+intelligence&btnG=

[3] https://scholar.google.com/scholar?q=Opinion+mining+Twitter+artificial+intelligence&hl=en&as_sdt=0&as_vis=1&oi=scholart

[4] https://scholar.google.com/scholar?hl=en&as_sdt=0,6&qsp=4&q=artificial+intelligence+%22analysis+of+public+attitudes%22&qst=ir

What’s next for The Lindahl Letter?

Week 109: Robots in the house
Week 110: Understanding knowledge graphs
Week 111: Natural language processing
Week 112: Autonomous vehicles
Week 113: Structuring an introduction to AI ethics

Lindahl, N. (2023). The Lindahl letter: 104 Machine Learning Posts. Lulu Press, Inc. https://www.lulu.com/shop/nels-lindahl/the-lindahl-letter-104-machine-learning-posts/ebook/product-y244ep.html

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

This week it seemed like a good idea to take a look at some of the most highly cited AI papers. Back during week 81 of this journey into writing on Substack, I took a look at some of the most highly cited ML papers [1]. I was expecting a lot more overlap, but was pleasantly surprised at the differences. One of the papers really stood out based on the total number of citations and it’s up first. Intellectually I can accept that a paper has more than 100,000 citations, but in practice that is an awful lot of references for an academic paper to have and a representation of a degree of asynchronous interaction between researchers that helps bring the intellectual space called the academy to life.

Papers with over 100,000 citations:

Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. https://arxiv.org/pdf/1412.6980.pdf

Papers with over 50,000 citations:

Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28. https://proceedings.neurips.cc/paper/2015/file/14bfa6bb14875e45bba028a21ed38046-Paper.pdf
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30. https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

Papers with over 20,000 citations:

Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., ... & Hassabis, D. (2015). Human-level control through deep reinforcement learning. nature, 518(7540), 529-533. https://daiwk.github.io/assets/dqn.pdf
Ioffe, S., & Szegedy, C. (2015, June). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning (pp. 448-456). PMLR. http://proceedings.mlr.press/v37/ioffe15.pdf

Papers with over 10,00 citations:

Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., ... & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. nature, 529(7587), 484-489.
Arjovsky, M., Chintala, S., & Bottou, L. (2017, July). Wasserstein generative adversarial networks. In International conference on machine learning (pp. 214-223). PMLR. http://proceedings.mlr.press/v70/arjovsky17a/arjovsky17a.pdf
Kipf, T. N., & Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907. https://arxiv.org/pdf/1609.02907.pdf

Without question these levels of citation are an indication that the works are being read and actively referenced within the scholarly community. I’m referencing the citation numbers here to give you a sense of scale when it comes to considering the AI community and how many people are researching and considering the things happening in this space. This is a very crowded and vibrant place in the academy where a lot of time and effort are going into moving things along toward building very real and deployable technology in this space. Given the sheer volume of people working in this space it’s only a matter of time before somebody will shout “Eureka!” and we see practical deployments in production which will influence our daily lives.

What would ChatGPT create?

If you were wondering what ChatGPT from OpenAI would have generated with the same prompt, then you are in luck. I had that output generated over at https://chat.openai.com/chat by issuing a prompt.

Links and thoughts:

Top 5 Tweets of the week:

Footnotes:

[1] Week 81 of The Lindahl Letter:

What’s next for The Lindahl Letter?

Week 108: Twitter as a company probably would not happen today
Week 109: Robots in the house
Week 110: Understanding knowledge graphs
Week 111: Natural language processing
Week 112: Autonomous vehicles

Lindahl, N. (2023). The Lindahl letter: 104 Machine Learning Posts. Lulu Press, Inc. https://www.lulu.com/shop/nels-lindahl/the-lindahl-letter-104-machine-learning-posts/ebook/product-y244ep.html

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

You probably were wondering how long into the new year before a bunch of focus and attention were placed at the efforts of Hugging Face. You won’t have to wait any longer as this missive will dig into BigCode and some other efforts to use AI to build out code generating systems [1]. I was reading a TechCrunch article from Kyle Wiggers and wondering about how many different systems existed [2]. That article references 5 code generating systems that you could in practice elect to go evaluate. For completeness I’m listing Codex and Copilot separately in this last given that the interfaces are holistically different.

BigCode - Hugging Face & ServiceNow’s R&D division [3]
AlphaCode - DeepMind [4]
CodeWhisperer - Amazon [5]
Codex - OpenAI [6]
Copilot - GitHub (Codex based) [7]

One of the things you might be interested in learning about at this point would be a dataset called “The Stack” which happens to be a collection of 6 terabytes of permissive code data that covers 300 programming languages [8]. The permissive code part of the dataset is interesting. The GitHub archive was roughly 69 terabytes of data that they filtered by licensing which they considered permissive and ended up with that 6 terabyte collection. Understanding how the dataset that feeds the code generating system was built is very important. All my contributions on GitHub are intended to be MIT license which I think should be permissive [9]. You have to deeply consider that a lot of propriety code writers and corporations employing said coders would not have given permission to use their code in a code generation system.

Generative coding systems will abound shortly and are in an early and developing state at the moment. We are getting to the point where you can instruct Codex to build something code related in terms of creating an application and you might get a great result. It’s not a universal code generation engine at this point. However, we are getting closer and closer to conversational code generation or some flavor of that outcome which I would classify as a generative coding system. It will be a seismic shift in code generation based on democratizing the creation of applications.

What would ChatGPT create?

If you were wondering what ChatGPT from OpenAI would have generated with the same prompt, then you are in luck. I had that output generated over at https://chat.openai.com/chat by issuing a prompt.

Links and thoughts:

Top 5 Tweets of the week:

Footnotes:

[1] https://huggingface.co/bigcode

[2] https://techcrunch.com/2022/09/26/hugging-face-and-servicenow-launch-bigcode-a-project-to-open-source-code-generating-ai-systems/

[3] https://www.bigcode-project.org/

[4] https://alphacode.deepmind.com/

[5] https://aws.amazon.com/codewhisperer/

[6] https://openai.com/blog/openai-codex/

[7] https://github.com/features/copilot

[8] https://huggingface.co/datasets/bigcode/the-stack

[9] https://github.com/nelslindahlx

What’s next for The Lindahl Letter?

Week 107: Highly cited AI papers.
Week 108: Twitter as a company probably would not happen today
Week 109: Robots in the house
Week 110: Understanding knowledge graphs
Week 111: Natural language processing

Lindahl, N. (2023). The Lindahl letter: 104 Machine Learning Posts. Lulu Press, Inc. https://www.lulu.com/shop/nels-lindahl/the-lindahl-letter-104-machine-learning-posts/ebook/product-y244ep.html

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

We are now starting the third year journey of The Lindahl Letter on Substack. You can find the first two years worth of that effort combined into one manuscript right here or at other online retailer:

Lindahl, N. (2023). The Lindahl letter: 104 Machine Learning Posts. Lulu Press, Inc. https://www.lulu.com/shop/nels-lindahl/the-lindahl-letter-104-machine-learning-posts/ebook/product-y244ep.html

During the course of this next year I’m probably going to write a book about how I would set up a modern polling company or to put it more bluntly, a book about how to monetize the sentiment measuring process in modern America. Part of that book writing effort could end up being a part of this series or it could end up being a stand alone set of content. Yes, that would mean 2023 would yield at least two books of exceptionally enjoyable reading content.

Overall, this new writing direction is one of those things that I’m going to really dig into throughout 2023. This would be inline with my research trajectory and current five year writing plan. My thoughts have just become a little more focused on what exactly I would produce within that writing window. Naturally, that is a good thing to start forming up before the writing project begins. I’m going to need to sit down and write up the topics to include within my literature review on this one. That part of this analysis will be key and has to be pretty darn good at this point.

Within that work effort I’m considering a few article titles to include in the writing plan:

“Sentiment pooling: Applied multimedia polling”
“Beyond the phone: A study of multimedia polling methods”
“When nobody answers: Evaluating the efficacy of phone based polling”
“Automated sentiment analysis compared to respondent based sentiment”
“Beyond the paywalls of academic journals: how information is being shared more broadly.”

We will see which of those ends up getting produced as either book chapters or true stand alone articles. This last week of planning and review was actually pretty productive. I have sketched out a writing plan for about the next 25 weeks. This new Substack series now has a backlog to work against as posts get put into a five week planning and review cycle. As we move from 2022 to 2023, my primary weekly Substack based content creation is going to shift a bit from ML related coverage to a stronger AI focus mixed with more content on modern polling methodologies.

You might be wondering why the combination of AI and sentiment analysis (polling methodologies) happens to be important to me as something to consider. As the intersection of technology and modernity occurs certain things will happen. Part of understanding civil society and figuring out what happens to civility during that intersection will require better active sentiment analysis. Critical thought and analysis into understanding the nature of measuring modern sentiment will be essential on a go forward basis based on the automation that AI will introduce throughout business.

I was looking at the proposed content from weeks 105 to 132 and realized that ethics as a part of the AI journey and process were left out as an early topic. I’m going to try to work in some evaluation of and references to academic works that consider ethics and responsible AI each week as we move forward.

As we close out here at the start of this series it would be only fitting to share what I do consider to be a foundational AI book to consider as key reference material. Probably the best book about AI you could grab is a textbook called, “Artificial Intelligence: A Modern Approach,” by Stuart Russell and Peter Norvig [1]. The current version appears to be the 4th edition [2]. You can get a copy on eBay or some other used book proprietary for a lot less than the academic asking price.

Some people prefer the findings of a certain alternative deep learning historian. I don’t want to deprive those readers of a link to what they might be looking for as fresh reading material. I am aware that Jürgen Schmidhuber has started writing some histories. It does present some alternative findings and different points of view. You can now access and read 75 pages of that content that have been recently published and are pretty easy to download from arXiv.

Schmidhuber, J. (2022). Annotated History of Modern AI and Deep Learning. arXiv preprint arXiv:2212.11279. https://arxiv.org/ftp/arxiv/papers/2212/2212.11279.pdf

Outside of those two works a book that would be more conversational and easier overall to absorb about where AI is at current would be this one:

Kissinger, H. A., Schmidt, E., & Huttenlocher, D. (2021). The age of AI: and our human future. Hachette UK.

This is the start of a series that will last 52 weeks. It should be an interesting journey.

What would ChatGPT create?

If you were wondering what ChatGPT from OpenAI would have generated with the same prompt, then you are in luck. I had that output generated over at https://chat.openai.com/chat by issuing a prompt.

Links and thoughts:

All-In Podcast “E110: 2023 Bestie Predictions!”

Top 5 Tweets of the week:

Footnotes:

[1] http://aima.cs.berkeley.edu/

[2] https://www.pearson.com/store/p/artificial-intelligence-a-modern-approach/P100000291856/9780137505135

What’s next for The Lindahl Letter?

Week 106: Code generating systems
Week 107: Highly cited AI papers
Week 108: Twitter as a company probably would not happen today
Week 109: Robots in the house
Week 110: Understanding knowledge graphs

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

My appreciation for you reading this sentence right now is very real. That holds true in the moment of writing and within my intentions moving forward. At the start of this project a bit of writing on Substack happened two years ago now. During the course of starting that effort, I had not considered the amount of effort sustaining it would require. Probably the single most sustained writing project I have ever completed was my doctoral dissertation [1]. That writing process took less time than producing the 104 posts that are currently a part of The Lindahl Letter. I know some future writing projects could involve more effort, but that seems unlikely given the amount of time that was devoted to this one. 

You can take a moment and reflect on the realization that we have reached the point in the program where 104 weeks of content creation has occurred. Over time the writing process ended up following a planful approach to getting things done. The structure ended up being a block of writing, links/thoughts, top five tweets, and footnotes. 

Weekly topic coverage. This is the heart of what is happening. Five weeks in planning or review is what makes the magic happen. For the most part, I ended up with a backlog with a bunch of planned posts. You should always keep a writing backlog of topics that deserve attention but might not have been advanced before. This method of keeping a writing topic backlog let me work on a couple of different weeks of content at any one time. Believe it or not, this is really important to keep things moving along. Momentum in writing is a real and present element of the creative process. If you run into a bit of writer’s block, or the inspiration is not showing up for a given topic, then you can move ahead and shift to a topic that allows the creation to continue. I’m not one to sit back and brew some word tea, but switching topics and trying to mix things up with a bit of stream of consciousness is certainly an option. 
Links and thoughts. During the course of the week, I end up selecting some links and from research. Depending on where I’m at during the planning and review process, these links are either very timely or could have been pulled a week or two before.
Top five tweets. This section is really just for my own amusement. Based on the data from the last two years, nobody ever clicks on any of these tweets. I’m getting so close to having published 10,000 tweets. It will probably happen toward the end in the next couple of months. 
Footnotes. Most writers in the academic space are comfortable with footnotes. Sometimes it takes people a bit of reading to get used to the links, references, and footnotes that I drop into the things being published. It is something that I plan on continuing with to help provide the context that we stand on the shoulders of giants in terms of our intellectual library of thoughts and considerations. The academy lives based on the interaction of scholars. That is something that cannot be forgotten or set aside to write prose without acknowledgement of the contributions of others. 
Podcast. Adding the podcast element changed my posture to working to stay a week or two ahead. A part of that change was that I had to now plan ahead for vacation windows. That five-week planning and review cycle was key to keeping the publishing streak alive for what is approaching two years.

After Post 104, I’m going to switch things up and focus more on sharing the content I find interesting in the AI and technology space [2]. Part of that is planned and was noted out in my five-year writing plan. In case you were wondering, I have included my five-year writing plan as of March 3, 2022:

Year 1: Keep a heavy machine learning focus for the rest of 2022. Finish writing a collected series of machine learning/AI essays on Substack and combine them into a manuscript, The Lindahl Letter: 104 Machine Learning Posts. This manuscript should include both Years 1 and 2 of the Substack series.

Keep writing weekly Substack posts.
Take time for the manuscript generation process at the end of the year.
That manuscript will need to be edited by a professional before the print edition goes live.
Rework last year’s speaking engagement talks into academic papers. This could be one combined paper or potentially five different papers depending on how the initial effort shapes up.
“What Is Machine Learning Scale? The Where and the When of Machine Learning Usage”
“The Machine Learning Scale Problem: Thinking About Where and When to Use Machine Learning, ROI Models, Synthetic Data, Repeatable Frameworks, and Teams”
“Applied Machine Learning ROI: Understanding Machine Learning ROI From Different Approaches at Scale”
“Demystifying Applied Machine Learning: Building Frameworks and Teams to Operationalize Machine Learning at Scale”
“Figuring Out Applied Machine Learning: Building Frameworks and Teams to Operationalize Machine Learning at Scale, V3”
Rerun the MLOps GitHub research and turn that content into a paper.

Year 2: For 2023, I want to pivot into studying sentiment analysis and modern polling methodologies. At this point, I will have written 104 essays on machine learning/AI and should probably refocus on a specific topic that is material to machine learning/AI, but adjacent to it as an area of research. It’s possible by 2023 that quantum computing will be a huge topic for research and will end up getting some attention as well.

Complete work on an automated sentiment analysis paper.
Write some sentiment analysis and machine learning essays for Substack.
Work on publishing modern polling methods essays for Substack.
Finish up writing the breakdown of modern polling paper.

Year 3: 2024 will include a return to writing about local government administration and technology. It will be 20 years since earning my master of public administration degree. By this time, my writing should be as crisp and focused as it will ever be, and my perspective on technology will be well considered from my previous work on machine learning/AI. A few topics will be considered:

Technology and local government administration
The intersection of public administration and technology
How technology influences the practice of governing 
How government uses machine learning/AI technology

Year 4: 2025 will probably be the year where quantum computing has broken down modern encryption frameworks. A few topics will be covered here:

Changes and uses in encryption technology
Encryption and society
Quantum encryption

Year 5: 2026 is going to be a year where my backlog should be highly full. The previous four years of this writing plan should have created a ton of leftover writing works. A few topics that I plan on highlighting include these:

A reflective work on machine learning/AL
Did open source MLOps technology survive?
Did the serverless trend pan out in the cloud?

Pretty much everything that was a part of Year 1 of my five-year writing plan was worked on and completed. You can find a lot of that content contained in this 104-week writing project on machine learning. The future of writing things is wide open, and I’ll be starting work on Year 2 of that writing plan shortly. The last thing I’m about to work on will be moving the content from all 104 Substack posts over to a manuscript to send off to be edited. 

Footnotes:

[1] Lindahl, N. P. (2010). Responsive e-government: A study of local government e-feedback methodology (Publication No. 3426428) [Doctoral dissertation, Walden University]. ProQuest Dissertations and Theses Global. https://www.proquest.com/openview/5fd815b75e8efa09a64814d0e290389b/1.pdf?pq-origsite=gscholar&cbl=18750 

[2] https://trends.google.com/trends/explore?date=today%205-y&geo=US&q=technology,artificial%20intelligence,machine%20learning,AI,ML

What’s next for The Lindahl Letter? A totally new writing format…

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

You probably have gotten a sense of the tremendous and overwhelming flood of publishing that is happening in the machine learning space. So much content is being created right now that nobody could possibly consume it all. The flood of content is real and overwhelming. That is one of the reasons that I work really hard to distill the complex topics I select into a readable format for people to consume. I really try to provide a pathway to people who want to be a part of this journey. Maybe those two observations are inherently in conflict, but I think helping people navigate the deluge of information to focus on key things helps break down the conflict. 

As I sit down to rethink the future of machine learning, my thoughts are circling back to some of the original things I was writing about use cases and how people select what they want to work on based on ROI. Right now, machine learning is being built into everything, and we are seeing a creative explosion of people using machine learning to generate things on the fly that otherwise would have never been possible. Some of the models within the image- and video-creation spaces are really changing how we interact with the world. My honest guess about the future of machine learning is that it will become seamless within the background of our daily lives. It will be ever present and constantly just on the edge of how we perceive and interact with the world around us. 

I grabbed five papers where the future of machine learning was discussed from 2001 to 2018.

Zhou, Z. H. (2016). Learnware: On the future of machine learning. Frontiers in Computer Science, 10(4), 589–590. https://www.lamda.nju.edu.cn/publication/fcs16learnware.pdf

Mjolsness, E., & DeCoste, D. (2001). Machine learning for science: State of the art and future prospects. Science, 293(5537), 2051–2055. https://www.researchgate.net/profile/Eric-Mjolsness/publication/11789794_Machine_Learning_for_Science_State_of_the_Art_and_Future_Prospects/links/09e415147695a42e12000000/Machine-Learning-for-Science-State-of-the-Art-and-Future-Prospects.pdf

Choy, G., Khalilzadeh, O., Michalski, M., Do, S., Samir, A. E., Pianykh, O. S., Geis, J. R., Pandharipande, P., Brink, J. A., & Dreyer, K. J. (2018). Current applications and future impact of machine learning in radiology. Radiology, 288(2), 318–328. https://doi.org/10.1148/radiol.2018171820 

Obermeyer, Z., & Emanuel, E. J. (2016). Predicting the future—Big data, machine learning, and clinical medicine. The New England Journal of Medicine, 375(13), 1216–1219. https://doi.org/10.1056/NEJMp1606181

Handelman, G. S., Kok, H. K., Chandra, R. V., Razavi, A. H., Lee, M. J., & Asadi, H. (2018). eDoctor: Machine learning and the future of medicine. Journal of Internal Medicine, 284(6), 603–619. https://doi.org/10.1111/joim.12822 

What’s next for The Lindahl Letter?

Week 104: That 2nd year of posting recap

I’ll try to keep the what’s next list forward looking with at least five weeks of posts in planning or review. If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

You end up with people who are working in a field and people who study the field of inquiry academically. Sometimes and especially within the machine learning space you end up with people who are actively working as a practitioner. Those very same practitioners of the craft of machine learning are publishing academic articles at a rate never before seen within any field of study at large within the academy. The best way to describe that effort would be to call them the pracademics of the machine learning space. Intellectually it’s probably good to have people write about things who really understand how they are occurring. You certainly get solid firsthand accounts of what people are creating. 

Research is the process of digging into things, investigating, studying, or maybe just seeking to understand the things better. Original research is often brought out to describe novel inquiry. Some of these pracademics are pushing things into a new frontier for machine learning. The DALL-E-2 AI system was introduced and changed the way people think about how an AI system would create realistic images and art based on a prompt [1].

During the course of working on this Substack post, you can imagine I was surprised to find a webpage called “AI Brain Drain” [2]. That site opens up with a clear, “Welcome to the AI Brain Drain Index.” They pretty much are tracking the number of AI faculty that have left academic areas to go work within industry. You can see charts and other figures that sort of run from 2004 to apparently 2018. The researchers must have stopped making charts in the last few years, but the ones they made are pretty nice. All of that research seems to have yielded a paper you can read [3]. According to the SSRN website where I downloaded the paper, 928 downloads have occurred on this one. 

Gofman, M., & Zhao, J. (2022, July 31). Artificial intelligence, education, and entrepreneurship. Journal of Finance [Forthcoming]. https://doi.org/10.2139/ssrn.3449440 or at SSRN: https://ssrn.com/abstract=3449440

An article from Inside Higher Education called “AI Academy Under Siege” from author Oren Etzioni took a look at AI experts leaving institutions of higher education and what might be some potential solutions to that situation [4]. You could find more from the work of Professor Michael Gofman [5]. You could check out an editorial published in Springer by Lars Kunze called, “Can We Stop the Academic AI Brain Drain?” [6]. Outside of those sources, one of the articles that I really liked was from Ben Dickson titled, “What Is the AI Brain Drain?” [7]. You could pivot to an article that I enjoyed less called, “Brain Drain of AI Researchers: Academia vs Industry” [8]. 

Let’s close this one out with an interesting look at, “AI Brain Drain to Google and Pals Threatens Public Sector’s Ability to Moderate Machine-Learning Bias” [9]. That article links back out to a paper that was interesting.

Jurowetzki, R., Hain, D. S., Mateos-Garcia, J., & Stathoulopoulos, K. (2021). The privatization of AI research(-ers): Causes and potential consequences. https://regmedia.co.uk/2021/02/04/drain.pdf 

Top 5 Tweets of the week:

Footnotes:

[1] https://openai.com/dall-e-2/

[2] http://www.aibraindrain.org/

[3] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3449440

[4] https://www.insidehighered.com/views/2019/11/20/how-stop-brain-drain-artificial-intelligence-experts-out-academia-opinion

[5] https://simon.rochester.edu/blog/deans-corner/brain-drain

[6] https://link.springer.com/article/10.1007/s13218-019-00577-2

[7] https://bdtechtalks.com/2019/09/26/artificial-intelligence-brain-drain/

[8] https://medium.com/codex/brain-drain-of-ai-researchers-academia-vs-industry-8e385e8fd517

[9] https://www.theregister.com/2021/02/04/ai_brain_drain/

What’s next for The Lindahl Letter?

Week 103: Rethinking the future of ML
Week 104: That 2nd year of posting recap

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

One of the core topics within the machine learning space that I have studied happens to be ROI or, more to the point, an examination of just how well spent money would be in the space. You can certainly spend money on machine learning and AI related efforts as a part of a think tank, an independent lab, or a pure research institution. Within corporate spaces, pure research and development is one thing, but generally to expend the precious capital resources of an institution, organization, or group, you want to know that some type of return will be accumulated from those activities. It’s inherent to the nature of the venture into running a corporation compared to running some other type of organization.

ROI for machine learning is a topic that deserves consideration. You can evaluate a variety of potential machine learning use cases to solve problems. Going from being a special product to an operationalized business process that utilizes technology to get things done will be dependent on the ROI associated with the use case. If it costs more to do it, then you could reasonably expect that use case is going to make most responsible actors pump the brakes on the project. However, it appears that a lot of use cases went forward anyway. 

I spent some time trying to find a good accounting of how much money has been spent on machine learning projects overall and how many of them have actually yielded solid ROI. You probably will not be all that surprised to learn that very few rigorous studies exist of successful ROI in the machine learning space. If some of those types of studies exist and I just missed them during my search, then by all means feel free to share them in the comments and let me know [1]. It won’t hurt my feelings or anything, and I’d actually be a little bit relieved that research on the subject exists. 

Before we conclude here, I do want to share one paper that does seem to be directly addressing this question. It has only been cited by three other papers since 2020. I had hoped it would lead me to a cluster of academic research. That was not the case. The search for solid academic research on machine learning ROI is still ongoing.

Mizgajski, J., Szymczak, A., Morzy, M., Augustyniak, Ł., Szymański, P., & Żelasko, P. (2020). Return on investment in machine learning: Crossing the chasm between academia and business. Foundations of Computing and Decision Sciences, 45(4), 281–304. https://sciendo.com/article/10.2478/fcds-2020-0015

Links and thoughts:

“We Talked To A VP At Microsoft - WAN Show December 23, 2022”

Top 5 Tweets of the week:

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=%22return+on+investment%22+%22machine+learning%22&btnG=

What’s next for The Lindahl Letter?

Week 102: ML pracademics
Week 103: Rethinking the future of ML
Week 104: That 2nd year of posting recap

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

You might remember back to Week 75 when I tried to explain the nature of overcrowding within the machine learning space and what exactly that is doing to engineering colleges. Back in 2020, Igor Susmelj wrote a piece in Towards Data Science, “How to Keep up With the Latest Research and Trends in ML” [1]. Within that paper, Igor raised awareness of a chart from Jeffrey Dean that showed machine learning arXiv papers versus Moore’s law growth rate [2]. A quick Google Scholar search of that paper will show that it is cited by 72 papers at the moment [3]. I had hoped that a batch of papers would help me find more academic work about overcrowding and machine learning, but that did not really turn out to be the case. I have been trying to figure out the right words to describe intellectual overcrowding and machine learning, but for the most part I have not been able to figure out the secret decoder ring settings to locate a bunch of papers on the subject. 

I noticed a chart on page 10 of a paper called, “An Overview on Applications of Machine Learning in Petroleum Engineering,” which showed the number of publications with AI/machine learning within the SPE OnePetro digital library [4]. Just like the chart from Jeffrey Dean, it showed a clear takeoff point after 2010, where the terms just skyrocket like a hockey stick. That is probably a fairly consistent trend, and things will shake out over time to a key set of academic articles that get referenced a lot and a core set of topics that are covered within the AI/machine learning academic community. At the moment, however, we are at the peak of the inflection point, where intellectually everybody rushed to be first in the pool and kept on swimming. 

With a rush of academic focus in the area of machine learning, we will see both a great deal of progress and a potentially calamitous fallout from intellectual overcrowding. Only so much progress and ultimately only so many faculty positions are going to exist within engineering programs. I do think we will see a pretty significant oversupply in the number of people seeking those faculty positions at the more prestigious set of academic institutions. Right now it is a lot more lucrative for people who are able to get jobs within industry to do that and to enjoy some pretty solid compensation. That is probably the element that has made the overcrowding less problematic during the rise of machine learning implementations. People being able to work in the private section and people working within academic spaces have been able to find work. You can find a lot of articles about brain drain within academic institutions related to both AI and machine learning. Some of those pieces make some interesting arguments. A lot of private organizations have run labs that are doing things of a more academic nature than perhaps applied use case development. 

Links and thoughts:

Top 4 Tweets of the week:

Footnotes:

[1] https://towardsdatascience.com/how-to-keep-up-with-the-latest-research-and-trends-in-ml-a45a356b1001

[2] https://arxiv.org/ftp/arxiv/papers/1911/1911.05289.pdf

[3] https://scholar.google.com/scholar?cites=3848695121612936760&as_sdt=4005&sciodt=0,6&hl=en

[4] https://www.researchgate.net/publication/339952951_An_Overview_on_Applications_of_Machine_learning_in_petroleum_Engineering

What’s next for The Lindahl Letter?

Week 101: Back to the ROI for ML
Week 102: ML pracademics
Week 103: Rethinking the future of ML
Week 104: That 2nd year of posting recap

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Perhaps you were looking for a bit more of a deep dive about deep generative models than will be contained in this relatively short missive. You could go check out Stanford University’s CS236 from Fall 2021 course [1]. That would help you begin to figure out just how unsupervised learning could be used to figure out the data distribution well enough to generate predicted other data elements. The content is broken up into 5 sections and you could contribute to the class GitHub if you wanted to provide feedback or improvement suggestions. Alternatively, you could learn more about this one from Prakash Pandey over at Towards Data Science from back in 2018 [2]. It's a faster read and pretty easy to digest compared to taking on a college level course. You could go the academic paper route for an introduction and dig into a work from Ruthotto & Haber from 2021:

Ruthotto, L., & Haber, E. (2021). An introduction to deep generative modeling. GAMM‐Mitteilungen, 44(2), e202100008. https://arxiv.org/pdf/2103.05180.pdf

My initial run at digging into deep generative models opened the door to a bunch of different topics within the space. Right at the start I ran into semi-supervised learning, graphs, urban mobility, and molecular science. You can imagine the urban mobility one made me a little curious what people were doing to model that with deep generation. Apparently, Google Scholar had enough data to offer up three paths including migration, mobility, and morphology. To get started I dug in with a quick search on "Deep generative models" with urban mobility [3]. None of the articles within this search space have a lot of references. It might be a relatively small area of academic inquiry at the moment. You could read most of the relevant academic content related to deep generative models in an afternoon. Trying to dig into using them for some type of use case will take a bit more effort in terms of setup, technology, and selection of that use case.

Here are 3 papers that showed up with the urban mobility search:

Eigenschink, P., Vamosi, S., Vamosi, R., Sun, C., Reutterer, T., & Kalcher, K. (2021). Deep Generative Models for Synthetic Data. ACM Computing Surveys. https://epub.wu.ac.at/8394/1/Deep_Generative_Models_for_Sequential_Data__WU_ePub_.pdf

Anda, C., & Ordonez Medina, S. A. (2019). Privacy-by-design generative models of urban mobility. Arbeitsberichte Verkehrs-und Raumplanung, 1454. https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/357034/3/ab1454.pdf

Johnsen, M., Brandt, O., Garrido, S., & Pereira, F. (2022). Population synthesis for urban resident modeling using deep generative models. Neural Computing and Applications, 34(6), 4677-4692. https://arxiv.org/ftp/arxiv/papers/2011/2011.06851.pdf

3 decently cited academic papers:

Salakhutdinov, R. (2015). Learning deep generative models. Annual Review of Statistics and Its Application, 2, 361-385. https://www.utstat.toronto.edu/~rsalakhu/papers/Russ_thesis.pdf

Kingma, D. P., Mohamed, S., Jimenez Rezende, D., & Welling, M. (2014). Semi-supervised learning with deep generative models. Advances in neural information processing systems, 27. https://proceedings.neurips.cc/paper/2014/file/d523773c6b194f37b938d340d5d02232-Paper.pdf

Maaløe, L., Sønderby, C. K., Sønderby, S. K., & Winther, O. (2016, June). Auxiliary deep generative models. In International conference on machine learning (pp. 1445-1453). PMLR. http://proceedings.mlr.press/v48/maaloe16.pdf

Links and thoughts:

“Teaching MLOps at scale with GitHub - Universe 2022”

“#83 Dr. ANDREW LAMPINEN (Deepmind) - Natural Language, Symbols and Grounding [NEURIPS2022 UNPLUGGED]”

Top 5 Tweets of the week:

Footnotes:

[1] https://deepgenerativemodels.github.io/

[2] https://towardsdatascience.com/deep-generative-models-25ab2821afd3

[3] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=%22Deep+generative+models%22+urban+mobility&btnG=

What’s next for The Lindahl Letter?

Week 100: Overcrowding and ML
Week 101: Back to the ROI for ML
Week 102: ML pracademics
Week 103: Rethinking the future of ML
Week 104: That 2nd year of posting recap

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

We are going to break out of the planned programming and go where everybody else involved in the machine learning space is going to go this week. Something new arrived this week and has captured the attention of the public mind. Seriously, I thought stable diffusion would be the big thing for 2022, but something else arrived in the intellectual space that might be more influential in the long run. The contents of this post were written right after the release, and I have been tinkering with that content throughout the week.

Staying current and staying well-grounded in the field of machine learning has been increasingly difficult. Those two things are ultimately very hard to achieve at the same time. This is an area with a great deal of breath and depth. I say that after writing 98 consecutively published weekly installments of a Substack machine learning newsletter. The main example this week would be of the new interactive session based chat framework (bot) that OpenAI released this week. Like most of the people actively curious about how bot’s have improved these days, I went out to https://chat.openai.com/chat which the team over at OpenAI blogged about here https://openai.com/blog/chatgpt/. All you need to do is create an account and you can sign in and chat with the bot. Each new session is tabula rasa as a reset to the model without the additional layer of your previous interactions. This is the interesting part of the equation as building a model and then having the model keep context within a conversation or a series of conversations. That ability to keep conversational context across multiple conversations is not a part of the current deployment.

The research preview they are sharing right now does not have access to the internet. It was also trained on data from about a year ago. Given the size of the language model they are invoking I would think it has a very large knowledge graph included , but that does not really appear to be the case. I gave it the following series of prompts to see what would happen in terms of how it generates content. The answers are in screenshot to help identify that the content was not created by me as a part of my normal writing output.

My prompt: “write 10 words about machine learning”

My prompt: “write 25 words about machine learning”

My prompt: “write 50 words about machine learning”

My prompt: “write 100 words about machine learning”

The last prompt I used was to ask it to “write about paper about machine learning with citations” which caused it to spit out 5 paragraphs that were pretty good.

This topic is way too early for academic articles [1]. A lot of news articles have been written about OpenAI’s chatbot they recently shared called ChatGPT. Here are 4 of them that came out this week:

“OpenAI’s new chatbot can explain code and write sitcom scripts but is still easily tricked”

https://www.theverge.com/23488017/openai-chatbot-chatgpt-ai-examples-web-demo

“OpenAI’s new ChatGPT is scary-good, crazy-fun, and—so far—not particularly evil.” https://slate.com/technology/2022/12/chatgpt-openai-artificial-intelligence-chatbot-whoa.html

“OpenAI invites everyone to test new AI-powered chatbot—with amusing results”

https://arstechnica.com/information-technology/2022/12/openai-invites-everyone-to-test-new-ai-powered-chatbot-with-amusing-results/

“OpenAI’s ChatGPT shows why implementation is key with generative AI”

https://techcrunch.com/2022/12/02/openais-chatgpt-shows-why-implementation-is-key-with-generative-ai/

Links and thoughts:

“E106: SBF's media strategy, FTX culpability, ChatGPT, SaaS slowdown & more”

“Why Do I Keep Getting Called Out - WAN Show December 2, 2022”

Top 6 “ChatGPT” Tweets of the week:

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=chatgpt&btnG=

What’s next for The Lindahl Letter?

Week 99: Deep generative models
Week 100: Overcrowding and ML
Week 101: Back to the ROI for ML
Week 102: ML pracademics
Week 103: Rethinking the future of ML
Week 104: That 2nd year of posting recap

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Recently, I started spending a bit more time writing about quantum machine learning and quantum computing in general. One of the things I became curious about was related to a thread of thoughts about how they code something for a quantum computer. One of the first things that I came across while trying to learn more about how people were coding with Twist was an article in IEEE Spectrum called, “Meet Twist: MIT’s Quantum Programming Language: Keeping tabs on data entanglement keeps reins on buggy quantum code” [1]. This article referenced out to CSAIL or the Computer Science and Artificial Intelligence Laboratory which happens to be located at the Massachusetts Institute for Technology. You can find their delightful website right here: https://www.csail.mit.edu/ and it does pretty easily direct you toward Twist with a few searches [2].

Within that website you will find that the researchers Charles Yuan, Christopher McNally, and Michael Carbin shared a paper at POPL 2022 [3].

Proceedings of the ACM on Programming LanguagesVolume 6Issue POPL January 2022 Article No.: 30pp 1–32 https://doi.org/10.1145/3498691

Or you could go out to the paper on arXiv and download the PDF.

Yuan, C., McNally, C., & Carbin, M. (2022). Twist: sound reasoning for purity and entanglement in Quantum programs. Proceedings of the ACM on Programming Languages, 6(POPL), 1-32. https://arxiv.org/abs/2205.02287

You might be curious how many papers cite that paper and the answer at this very moment from Google Scholar happens to be 6 [4]. None of this content seems to be highly citated at this point in terms of a network of academic coverage.

However, if you were wondering about an MIT course you could take related to this, then you are in luck. For only $2,249.00 you could take the 4-week course that starts on January 23, 2023: https://learn-xpro.mit.edu/quantum-computing.

At this point in the process, I started looking around for coding examples or notebooks with something to try to absorb. You are going to end up with the paper reading pages 46-53 of that paper linked above from arXiv.

I took a look at this artifact up on GitHub here: https://github.com/psg-mit/twist-popl22

Over the next year I’ll be looking for more practical coding examples or maybe a tutorial that really explains how to use the Twist quantum programing language. I could not find an emulator or a code dojo to test out things either. That is problematic given that I’m probably not going to pay for time on a quantum computer to learn how to write Twist code or more to the point you are paying to engage in the activity of coding.

Links and thoughts:

“A Hard Fork in the Road: FTX’s Unraveling and Elon’s Loyalty Oath”

“#81 JULIAN TOGELIUS, Prof. KEN STANLEY - AGI, Games, Diversity & Creativity [UNPLUGGED]”

“Galactica: A Large Language Model for Science (Drama & Paper Review)”

“A Sports Card Documentary IN THEATERS?! 👀 (Behind The Card)”

Top 5 Tweets of the week:

Footnotes:

[1] https://spectrum.ieee.org/quantum-programming-language-twist[2] https://www.csail.mit.edu/news/language-quantum-computing[3] https://popl22.sigplan.org/details/POPL-2022-popl-research-papers/30/Twist-Sound-Reasoning-for-Purity-and-Entanglement-in-Quantum-Programs[4] https://scholar.google.com/scholar?cites=6310617750987639534&as_sdt=4005&sciodt=0,6&hl=en

What’s next for The Lindahl Letter?

Week 98: Deep generative models
Week 99: Overcrowding and ML
Week 100: Back to the ROI for ML
Week 101: Revisiting my MLOps paper
Week 102: ML pracademics
Week 103: Rethinking the future of ML
Week 104: That 2nd year of posting recap

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Within the broader generative AI space, the part I tend to focus on is related to the written word. Right now, all of the visual generation parts of generative AI in terms of images and videos are wholesale living in the public mind [1]. Creative people are generating thumbnails and playing with all sorts of plausible image generation technology. A few teams are rapidly working on how to make video from that same type of generative AI and that is going to be interesting. We are probably going to see generative AI shows that people create very soon. I have previously written (Week 78) that I think all public trust in imagines is going to erode and that we are going to hit a zero-trust wall when it comes to being able to believe what we see [2]. This missive will be about the future of where large language models are going and a bit of a reflection on what has happened in the last couple of years.

Back on October 26, 2021, the folks over on Hugging Face shared out a post called, “Large Language Models: A New Moore's Law? [3]. This post starts out with a very familiar graphic of the models in terms of billions of parameters over time. This is a relatively recent phenomenon with a start during 2018 and massive acceleration after 2020. You may well have heard about the Megatron-Turing natural language generation model (MT-NLG) [4][5][6]. You can imagine that people were thinking they should make larger models. After all what is better than a billion-parameter model? It obviously has to be a trillion-parameter model. I would argue that the reality of having unique parameters within that large of a search space is probably something that is being disregarded at this point, but that has not stopped the march for more and more parameters. You might be thinking that nobody has really done that type of effort in practice.

The M6 model happens to be 10 trillion parameters [7]. Yeah, a 10 trillion parameter model exists. That is mind bogglingly large.

One of the longest papers with the most authors was published by a lot of people from Stanford was “On the opportunities and risks of foundation models” [8]. You probably could have guessed that I was going to throw the title of that paper into Google Scholar to see what other people were adding with that citation to the scholarly world aka the academy [9]. That query offered up about 565 results to consider.

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., ... & Liang, P. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. https://arxiv.org/pdf/2108.07258.pdf

This link will take you right to the 412 papers that Google Scholar believes have a direct citation to that massive 212 page paper [10].

One of the real concerns when that paper got published was that it covered so much ground and had so many coauthors that it would very quickly become an anchor citation that was heavily cited. Some people were worried it would just become a stock or default citation for people in literature reviews. I’m pretty sure that the number of scholars that came together on that work will pretty much guarantee that it gets cited a ton going forward. The other element that will help with that is that the paper is highly readable for people wanting to learn and understand large language models. Together those two elements of it being useful to read and known by a large number of scholars from the start pretty much guarantee that people will hear about it for years to come.

Links and thoughts:

“[ML News] Multiplayer Stable Diffusion | OpenAI needs more funding | Text-to-Video models incoming”

“We've Made Some Big Mistakes - WAN Show November 18, 2022”

Top 5 Tweets of the week:

Footnotes:

[1] https://venturebeat.com/ai/how-2022-became-the-year-of-generative-ai/

[2]

[3] https://huggingface.co/blog/large-language-models

[4] https://developer.nvidia.com/megatron-turing-natural-language-generation

[5] https://arxiv.org/abs/1909.08053

[6] https://developer.nvidia.com/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/

[7] https://towardsdatascience.com/meet-m6-10-trillion-parameters-at-1-gpt-3s-energy-cost-997092cbe5e8

[8] https://arxiv.org/pdf/2108.07258.pdf

[9] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=%22On+the+Opportunities+and+Risks+of+Foundation+Models%22&btnG=

[10] https://scholar.google.com/scholar?cites=9595110325981705564&as_sdt=4005&sciodt=0,6&hl=en

What’s next for The Lindahl Letter?

Week 97: MIT’s Twist Quantum programming language
Week 98: Deep generative models
Week 99: Overcrowding and ML
Week 100: Back to the ROI for ML
Week 101: Revisiting my MLOps paper
Week 102: ML pracademics
Week 103: Rethinking the future of ML
Week 104: That 2nd year of posting recap

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

We are living on the edge of meeting the weekly Friday publishing deadline at this point. As we quickly approach the 104th post and the major two-year milestone I’m still working on the same Saturday and Sunday schedule of early morning writing. I’m just having to be extra mindful of making sure I don’t get sidetracked into working on other things. This week’s topic could be an entire book full of insight. People are certainly going to fill the shelves with quantum machine learning books in the coming years. It is certainly starting to turn into whole conferences and other gatherings of people interested in telling people all about and sharing stories of quantum machine learning.

Nobody is really bringing a quantum computer to any of these events. These are not something that is going to fit in your car and be ready to head out to a conference event. Seriously, to the best of my knowledge no laptop or portable quantum computer exists at this time and if it did nobody is using it for machine learning. They would probably be taking it to conferences or other gathering to show people how delightfully wonderful it is to be able to carry around such a power computing device. I imagine that it will be like Steve Wozniak showing up and assembling an early homebrew computer club kit. Those were moments of endless possibility, delight, and wonder. Bringing back the chance at some type of epic moment like that is certainly something that could very well happen within this space. My money is on the team over at IBM making that happen at some point.

Let’s rewind the coverage here for a moment and reflect on two of my previous Substack posts:

Week 42: Time crystals and machine learning (this one was epic)

Week 77: Is quantum machine learning gaining momentum?

You may have forgotten from the first paragraph that this is week 95 of The Lindahl Letter and quantum computing has received 3 different weeks of coverage. You can tell from that degree of focus that I believe it is a topic that will eventually change the nature of compute.

All right let’s jump right into the best academic articles about quantum machine learning.

Biamonte, J., Wittek, P., Pancotti, N., Rebentrost, P., Wiebe, N., & Lloyd, S. (2017). Quantum machine learning. Nature, 549(7671), 195-202. https://arxiv.org/pdf/1611.09347.pdf

Schuld, M., Sinayskiy, I., & Petruccione, F. (2015). An introduction to quantum machine learning. Contemporary Physics, 56(2), 172-185. https://arxiv.org/pdf/1409.3097.pdf

I did enjoy this 35-minute YouTube video about quantum computing from TechTechPotato which is hosted by Dr. Ian Cutress, “Quantum Computing: Now Widely Available!”

Being able to see some of these videos with the crew from IBM make me feel better about this technology actually existing. This would be much easier to accept as science fiction. Within that video the part of the coverage that caught my attention the most was that really it was academics and some startups that were coding for practical use cases to do something with quantum computing. For the most part, the type of machine learning efforts that would be easily transition over into this type of compute are not called out super clearly.

One of the things I’m considering for next year is maybe coding something simple up and running it on one of these IBM systems. To really get a handle on what is happening within this quantum computing system it feels like only some hands work will close the gap for me here.

Links and thoughts:

“The Uses of IBM's Next Generation 433 Qubit Chip”

“#036 - Max Welling: Quantum, Manifolds & Symmetries in ML”

This is a podcast from the folks who are in the room with Twitter… “E104: FTX collapse with Coinbase CEO Brian Armstrong + election results, macro update & more”

Top 5 Tweets of the week:

Footnotes:

None.

What’s next for The Lindahl Letter?

Week 96: Where are large language models going?
Week 97: MIT’s Twist Quantum programming language
Week 98: Deep generative models
Week 99: Overcrowding and ML
Week 100: Back to the ROI for ML

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

I must have missed it when Samuel K. Moore covered this topic back on February 24, 2022 in an article titled, “RISC-V AI Chips Will Be Everywhere Esperanto Technology’s chip heralds new era in open-source architecture; Intel set to cash in,” [1]. I’m going to admit that I did fix the spelling on the word technology when I cut and pasted that title over to this Substack post. Fast forward just a couple of months and in April an article in Forbes talked about people actually sampling the 1,000 core hardware [2]. Within the broader context of things 1,000 cores is a lot of computing cores for a CPU chip. This computer for example is running an Intel i7 series chip and has 6 cores. You can see the difference in the number of cores is very large.

You can pretty easily get to the website for Esperanto Technologies

https://www.esperanto.ai/

and it has some information on it. They clearly believe that RISC-V is the future of computing innovation. For those of you who might be chuckling that “RISC architecture is going to change everything” yes that line from the 1995 feature film Hackers certainly continues to resonate with people. The reduced instruction set computer architecture certainly has had a strong run. I’m pretty sure that the RISC-V design is about 12 years old (2010) and it really is an open standard instruction set architecture.

You can read about at the Berkeley Architecture Research site here: https://bar.eecs.berkeley.edu/projects/riscv.html

Some of you may have already guessed that I was going to search Google Scholar for anything on this topic that might be interesting [3]. You can pretty quickly dig into the 18 results that came back for that search. Here are a sampling of those results:

Imran, H. A., Mujahid, U., Wazir, S., Latif, U., & Mehmood, K. (2020). Embedded development boards for edge-AI: A comprehensive report. arXiv preprint arXiv:2009.00803. https://arxiv.org/ftp/arxiv/papers/2009/2009.00803.pdf

Reuther, A., Michaleas, P., Jones, M., Gadepally, V., Samsi, S., & Kepner, J. (2022). AI and ML Accelerator Survey and Trends. arXiv preprint arXiv:2210.04055. https://arxiv.org/pdf/2210.04055.pdf

Dokic, K., Mikolcevic, H., & Radisic, B. (2021). Inference speed comparison using convolutions in neural networks on various SoC hardware platforms using MicroPython. In RTA-CSIT (pp. 67-73). http://ceur-ws.org/Vol-2872/paper10.pdf

Overall, I read this article in Interesting Engineering that talked about RISC-V having shipped more than 10 billion cores already [4]. The only reference to RISC-V AI in that article does reference Esperanto and its in the last paragraph. I was really hoping to find more content about these AI hardware chips. My guess here is that some other phrase is being used to describe the technology. I had spent some time looking around at searches related to, “Esperanto Technologies competitors.” That did not really yield anything major to share here. This is one I will need to circle back to at some point to see how AI specific hardware is changing. My efforts going down that road ended up looking at the IBM, “AI Hardware Center” [5]. I feel like a lot more companies are working in this space and more coverage could help highlight that at some later point.

Links and thoughts:

“Why Signal won’t compromise on encryption, with president Meredith Whittaker”

“Stanford Seminar - ML Explainability Part 1 I Overview and Motivation for Explainability”

Top 6 Tweets of the week:

Footnotes:

[1] https://spectrum.ieee.org/risc-v-ai

[2] https://www.forbes.com/sites/karlfreund/2022/04/20/risc-v-startup-esperanto-technologies-samples-first-ai-silicon/?sh=37506f9c773d

[3] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=%22risc-v+ai%22&btnG=

[4] https://interestingengineering.com/innovation/rise-of-risc-v-the-computer-chip

[5] https://research.ibm.com/collaborate/ai-hardware-center

What’s next for The Lindahl Letter?

Week 95: Quantum machine learning
Week 96: Generative AI: Where are large language models going?
Week 97: MIT’s Twist Quantum programming language
Week 98: Deep generative models
Week 99: Overcrowding and ML

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

We are going to get to the 104th Substack post before you know it here for The Lindahl Letter publication. Things are moving along, and we are on the very tail end of that journey. Don’t panic about this post not being full of links. (Spoiler alert) You will have plenty of perspectives to read this week that are linked to for your reading pleasure.

This is one of the topics that deserves a lot of attention. I circle back to asking people to always consider ethics within the context of both ML and AI. One of the considerations within that argument would be to really try to understand the outcomes of what the technology is being used to achieve or the negative externalities that would be possible from it’s use. You have heard me say it before and you will certainly hear it again, “Just because you can do a thing, does not mean you should.” It’s a real consideration within the AI/ML space. A lot of the things that can be done with both ML and AI are unconscionable and should be avoided. That is why ethics should be a core part of the AI/ML journey without question. Full stop.

Beyond that consideration I tried to gather up a bunch of papers critical of ML in general. It was actually much harder in practice to find written criticism of ML than I expected in published article forms. You are certainly welcome to out and subscribe to the Substack of Gary Marcus who publishes, “The Road to AI We Can Trust,” [1]. A good scroll across those posts will give you a real sense of criticism and questions about the ML space. It has some really solid engagement as well from people who care enough to deeply question things. I wholesale consider that to be a healthy part of the process.

Marcus, G. (2018). Deep learning: A critical appraisal. arXiv preprint arXiv:1801.00631. https://arxiv.org/ftp/arxiv/papers/1801/1801.00631.pdf

Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B., & Sutskever, I. (2021). Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment, 2021(12), 124003. https://arxiv.org/pdf/1912.02292.pdf

Lake, B., & Baroni, M. (2018). Still not systematic after all these years: On the compositional skills of sequence-to-sequence recurrent networks. https://openreview.net/pdf?id=H18WqugAb

Mitchell, M. (2021). Why AI is harder than we think. arXiv preprint arXiv:2104.12871. https://arxiv.org/pdf/2104.12871.pdf

Biderman, S., & Scheirer, W. J. (2020). Pitfalls in machine learning research: Reexamining the development cycle. http://proceedings.mlr.press/v137/biderman20a/biderman20a.pdf

Henderson, P., & Brunskill, E. (2018). Distilling information from a flood: A possibility for the use of meta-analysis and systematic review in machine learning research. arXiv preprint arXiv:1812.01074. https://arxiv.org/pdf/1812.01074.pdf

Vinny, P. W., Garg, R., Padma Srivastava, M. V., Lal, V., & Vishnu, V. Y. (2021). Critical Appraisal of a Machine Learning Paper: A Guide for the Neurologist. Annals of Indian Academy of Neurology, 24(4), 481–489. https://doi.org/10.4103/aian.AIAN_1120_20 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8513942/

You can get to a bit more content outside of scholarly articles when it comes to finding critics of machine learning. I’m going to share a handful of links to different things that I found interesting along the way to researching this Substack post.

“5 myths about learning and innateness”

https://open.substack.com/pub/garymarcus/p/5-myths-about-learning-and-innateness?r=8oh0m&utm_campaign=post&utm_medium=web

“The Limitations of Machine Learning”

https://towardsdatascience.com/the-limitations-of-machine-learning-a00e0c3040c6

“When Machine Learning Goes Off the Rails”

https://hbr.org/2021/01/when-machine-learning-goes-off-the-rails

“The way we train AI is fundamentally flawed”

https://www.technologyreview.com/2020/11/18/1012234/training-machine-learning-broken-real-world-heath-nlp-computer-vision/

“Why deep-learning AIs are so easy to fool”

https://www.nature.com/articles/d41586-019-03013-5

“How a Pioneer of Machine Learning Became One of Its Sharpest Critics”

https://www.theatlantic.com/technology/archive/2018/05/machine-learning-is-stuck-on-asking-why/560675/

“AI researchers allege that machine learning is alchemy”

https://www.science.org/content/article/ai-researchers-allege-machine-learning-alchemy

Links and thoughts:

“Generative AI is Here. Who Should Control It?”

“Twitter is now an Elon Musk company”

“Apple's new App Store tax, Microsoft Surface reviews, and Meta's earnings”

“Emergency Pod: Elon Musk Owns Twitter”

Top 5 Tweets of the week:

Footnotes:

[1] Gary Marcus’s Substack “The Road to AI We Can Trust”

What’s next for The Lindahl Letter?

Week 94: AI hardware (RISC-V AI Chips)
Week 95: Quantum machine learning
Week 96: Generative AI: Where are large language models going?
Week 97: MIT’s Twist Quantum programming language
Week 98: Deep generative models

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

It seems like having a national artificial intelligence initiative is popular these days. Back on February 18, 2022, I shared my week 56 Substack post, “Comparative analysis of national AI strategies.” That missive continues to get a good bit of traffic so I thought now would be a good time to go ahead and revisit national AI strategies, advisory committees, institutes, legislation, and the myriad of research institutes or think tanks that are jumping into this area of consideration. This is an area that I think could be a good place for some solid academic contributions. Instead of digging into all of those areas my attention really got focused on one advisory committee. That will become clear here in the next couple of sections of content.

This week I have considered shifting The Lindahl Letter over to being an AI strategy advisory committee after spending a bunch of time reading about them this week. I’m not going to do that as it would limit my creative output to just one area and that sounds intellectually exhausting. One of them you can read about would be the National AI Advisory Committee (NAIAC) [1]. The next committee meeting was about to happen before writing this post. I had plans to listen live and I was totally signed up for everything [2]. Go forward I’m fully registered and signed up for alerts from the NAIAC. I would be happy to provide them guidance on effective national AI strategies from a comparative perspective, but that has not happened so far. This topic is an interesting space to consider at length. We are seeing a huge amount of academic work and companies like Hugging Face democratizing AI through community. Consider for a moment just how fast stable diffusion showed up and then was actively built into things and deployed. We are seeing massive changes within the ML/AI space and the deployment cycle is super-fast based on how interconnected the community happens to be worldwide. That has huge ramifications for any advisory committee considering the national level of AI strategy. Adapting to the rate of change and decentralized nature of things requires a different type of national AI strategy. I’ll be listening to the NAIAC in October to see how things are going. You can find the sessions on YouTube by searching for “NAIAC” pretty easily.

“National Artificial Intelligence Advisory Committee (NAIAC) Meeting”

“National Artificial Intelligence Advisory Committee (NAIAC) Field Hearing”

You could read the meeting minutes from May 4, 2022.

https://www.ai.gov/wp-content/uploads/2022/07/NAIAC-Minutes-05042022.pdf

I went out to Google Scholar and took a look to see if anybody had published or shared anything with this advisory committee referenced [3]. Nothing really came up except the above-mentioned meeting minutes from May 4, 2022. Nothing really showed up during a search of arXiv either [4]. It’s possible in about 6 months more content will show up reacting to the hours of meetings that are linked above. Right now, we appear to be a little bit ahead of things in terms of reactions to the work being done by this advisory committee. I’m going to keep an eye out for more content related to NAIAC. It’s possible sometime next year it will be the right time to dig back into this one.

Links and thoughts:

“5 Practical Machine Learning Lessons You’re NOT Taught in School”

“This Has Never Happened Before - WAN Show October 14, 2022”

“Stanford CS330: Deep Multi-Task & Meta Learning I 2021 I Lecture 4”

“Confusing new Apple products, Netflix password sharing, and NFT cults”

Top 5 Tweets of the week:

Footnotes:

[1] https://www.ai.gov/naiac/

[2] https://events.nist.gov/profile/form/index.cfm?PKformID=0x17861abcd

[3] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=National+Artificial+Intelligence+Advisory+Committee&btnG=

[4]

https://search.arxiv.org/?in=&query=%22National%20Artificial%20Intelligence%20Advisory%20Committee%22

What’s next for The Lindahl Letter?

Week 93: Papers critical of ML
Week 94: AI hardware (RISC-V AI Chips)
Week 95: Quantum machine learning
Week 96: Where are large language models going?
Week 97: MIT’s Twist Quantum programming language

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

For those of you that keep track of these types of things we are now in real time based on my publishing schedule. Over the course of the next few weeks no backlog exists as we make the run to 104 consecutive Substack posts spanning 2 years of content creation on this platform. It’s week 91 right now in the publishing schedule which means that only 13 blocks of super exciting writing about machine learning stand between you and the completion of that penultimate tasking. To that end I’ll be working without a sizeable backlog that would prevent procrastination or a loss of focus from breaking the streak. At this very moment, I’m probably writing about that consideration to help refocus my efforts on completing this last stretch. You may recall that after the 2-year mark I’m going to mix things up a bit and switch focus from machine learning to artificial intelligence in general. The format might change a bit as well, but you will have to stay tuned to see what ends up showing up every Friday.

My physical notebook where I write things down by hand with a Parker Sonnet fountain pen includes a few different sketches related to building a universal request handler. A lot of the voice assistants receive queries that need to be given to some type of ML model to solve. Making a decision about which model to apply and then how to manage and sort those results back into the general knowledge graph is an interesting problem to solve. Generally, one solution is to just fire off the request to a series of API channels and the one that reports back a probable answer in the shortest time is what gets served up by the voice assistant. All of that happens so very quickly that you don’t really notice a long delay. We as people handle super complex reasoning tasks and work on things with an extreme depth without even questioning the solution selection process.

Perhaps as a subset of that general selection of what model to use when questioning something else props up from researchers and practitioners. One of the things people who are involved with work in the ML models space ask from time to time is about why you cannot just combine all the ML models together and make a super model. One of the ways people are working to bring models together involves the ensemble method for machine learning models. This methodology involves making a few models and then combining them to improve results. This is not a method to just stack random models and try to make it work. The ensemble method is a technique that is based on working with the same dataset and maybe combining for example a bunch of favorable random forests or some other set of similar models to form an ensemble. From what I have been able to tell from reading articles in this space its not a super solution to just bring all machine learning models together in one unified model theory.

Dietterich, T. G. (2000, June). Ensemble methods in machine learning. In International workshop on multiple classifier systems (pp. 1-15). Springer, Berlin, Heidelberg. https://web.engr.oregonstate.edu/~tgd/publications/mcs-ensembles.pdf

Dietterich, T. G. (2002). Ensemble learning. The handbook of brain theory and neural networks, 2(1), 110-125. https://courses.cs.washington.edu/courses/cse446/12wi/tgd-ensembles.pdf

This one is out and in use in the wild. You can actually utilize ensemble ML models from some of the systems like scikit-learn [1]. You can also pretty quickly implement ensemble models with the “The Functional API” as a part of TensorFlow core [2]. You can pretty quickly get up to speed and use this one in notebooks or other places.

Links and thoughts:

Lex Fridman Podcast “#324 – Daniel Negreanu: Poker”

Lex Fridman Podcast “#315 – Magnus Carlsen: Greatest Chess Player of All Time”

“Microsoft's Surface event, Pixel 7 and Pixel Watch reviews, and Meta Connect 2022”

“Mark Zuckerberg on the Quest Pro, future of the metaverse, and more”

Top 5 Tweets of the week:

Footnotes:

[1] https://scikit-learn.org/stable/modules/ensemble.html[2] https://www.tensorflow.org/guide/keras/functional

What’s next for The Lindahl Letter?

● Week 92: National AI strategies revisited

● Week 93: Papers critical of ML

● Week 94: AI hardware (RISC-V AI Chips)

● Week 95: Quantum machine learning

● Week 96: Where are large language models going?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

The post this week is going to be on the shorter side of things. I think that is in part due to the very straightforward nature of the topic under consideration. It really could have just been a link to a single book on the subject with a polite note that reading it would help you understand pretty much everything you need to know. To that end, it looks like the book on probabilistic machine learning from Kevin Patrick Murphy has been downloaded 168 thousand times [1]. That is pretty darn good for something in the machine learning space where the inflection point is generally over or under around 10,000 points of interest on a topic. It appears that Kevin really surpassed that ceiling by a ton of downloads. The book is very easy to get to and the search engines really seem to algorithmically love it as well. Given that this topic has a lot of refences to Bayesian decision theory you probably could predict that it would get my full attention. The topics that generally grounds all of my efforts in the machine learning space is that background in statistics and my enjoyment of working with Bayesian pooling. Let’s begin to breakdown the idea of probabilistic machine leaning involves understanding two general steps. First, you must accept that you want to explain observed data with your machine learning models. Second, those explanations are going to need to come from inferring plausible models to aid you in that explanation. Together those two steps help you begin to evaluate data in a probabilistic way which means that you are aided by the power of statistical probability grounding you to a rational approach. To me this sort of spells out an approach that is not based on randomness or anything particularly chaotic.

Murphy, K. P. (2012). Machine learning: a probabilistic perspective. MIT press. https://research.google/pubs/pub38136.pdf

Probabilistic machine learning papers

Rain, C. (2013). Sentiment analysis in amazon reviews using probabilistic machine learning. Swarthmore College. https://www.sccs.swarthmore.edu/users/15/crain1/files/NLP_Final_Project.pdf

Probabilistic deep learning papers

Nie, S., Zheng, M., & Ji, Q. (2018). The deep regression bayesian network and its applications: Probabilistic deep learning for computer vision. IEEE Signal Processing Magazine, 35(1), 101-111. https://sites.ecse.rpi.edu/~cvrl/Publication/pdf/Nie2018.pdf

Peharz, R., Vergari, A., Stelzner, K., Molina, A., Shao, X., Trapp, M., ... & Ghahramani, Z. (2020, August). Random sum-product networks: A simple and effective approach to probabilistic deep learning. In Uncertainty in Artificial Intelligence (pp. 334-344). PMLR. http://proceedings.mlr.press/v115/peharz20a/peharz20a.pdf

Andersson, T. R., Hosking, J. S., Pérez-Ortiz, M., Paige, B., Elliott, A., Russell, C., ... & Shuckburgh, E. (2021). Seasonal Arctic sea ice forecasting with probabilistic deep learning. Nature communications, 12(1), 1-12. https://www.nature.com/articles/s41467-021-25257-4?tpcc=nleyeonai

Links and thoughts:

“How Arm conquered the chip market without making a single chip, with CEO Rene Haas”

Top 5 Tweets of the week:

Footnotes:

[1] https://probml.github.io/pml-book/book1.html

What’s next for The Lindahl Letter?

Week 91: What are ensemble ML models?
Week 92: National AI strategies revisited
Week 93: Papers critical of ML
Week 94: AI hardware (RISC-V AI Chips)
Week 95: Quantum machine learning

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

A lot of people talk about deploying AI in the business world and almost all that conjecture is entirely based on deploying a machine learning model into a production environment or some interesting POC. When those same people deploy an actual AI product into production, they will hopefully see the difference. They are not the same. A lot of the AI hype is underpinned by advances in machine learning. Artificial general intelligence or more commonly abbreviated as AGI represents an interesting summation of possibility contained in a name. You have seen representations of AGIs in books, movies, comics, and all sorts of works of fictions. At the moment, machine learning models are generally trained to do one thing well and cannot generally pick up and learn tasking like a person would.

That is why most of those fiction writers do not bother to include a machine learning model as the antagonist in stories. The expectation is that a person (or villain for that matter) would be able to generally pick up and learn tasking for a wide variety or purposes. That exception gets rolled up into what an AGI would be expected to achieve in practice. Generally, the expectation would be that the AGI could complete a mix of tasking just like a person would be able to handle. You would need a large number of machine learning models to complete the tasking that a person does in a single day. You could test this as a practical exercise with a sheet of paper and a pen throughout the day. As you built up the list of machine learning models you would need throughout the day to accomplish all the various tasking it would become very obvious that your ML model is not an AGI as your list would be much greater than a single model. Or even a small collection of models being sorted out by a piece of software upfront. To be fair to the idea contained within that point, we don’t even have a good method to switch between a collection of ML models to deploy a collection to complete a variety of tasks.

Artificial general intelligence - Let’s begin by digging into a few books and papers related to AGI before introducing the ML part of the equation. This will help create a foundation for the concept and scholarly evaluation of AGI without spending as much time on ML. You will find a theme in the literature here were Goertzel is prominently featured.

Goertzel, B., Orseau, L., & Snaider, J. (2015). Artificial general intelligence. Scholarpedia, 10(11), 31847. http://var.scholarpedia.org/article/Artificial_General_Intelligence

Goertzel, B. (2007). Artificial general intelligence (Vol. 2). C. Pennachin (Ed.). New York: Springer. https://www.researchgate.net/profile/Prof_Dr_Hugo_De_GARIS/publication/226000160_Artificial_Brains/links/55d1e55308ae2496ee658634/Artificial-Brains.pdf

Goertzel, B. (2014). Artificial general intelligence: concept, state of the art, and future prospects. Journal of Artificial General Intelligence, 5(1), 1. https://sciendo.com/abstract/journals/jagi/5/1/article-p1.xml

I did discover along the way that Dr. Ben Goertzel who has papers referenced above has made a lot of content on YouTube. You may remember some of the Sophia the robot content (Hanson Robotics) from 2016 to 2018 as it was fairly prevalent in the media. You can read and article from The Verge about this one [1].If you wanted to dig into a more video based set of content, then feel free to check out this 7 video playlist on the general theory of general intelligence:

https://www.youtube.com/playlist?list=PLAJnaovHtaFTK9E1xHnBWZeKtAOhonqH5

Machine learning – This next set of research will consider both ML and AGI together.

Pei, J., Deng, L., Song, S., Zhao, M., Zhang, Y., Wu, S., ... & Shi, L. (2019). Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature, 572(7767), 106-111. https://aiichironakano.github.io/cs653/Pei-ArtificialGeneralIntelligenceChip-Nature19.pdf

Silver, D. L. (2011, August). Machine lifelong learning: Challenges and benefits for artificial general intelligence. In International conference on artificial general intelligence (pp. 370-375). Springer, Berlin, Heidelberg. https://www.researchgate.net/profile/Daniel-Silver-3/publication/221328970_Machine_Lifelong_Learning_Challenges_and_Benefits_for_Artificial_General_Intelligence/links/00463515d5bc70ed5c000000/Machine-Lifelong-Learning-Challenges-and-Benefits-for-Artificial-General-Intelligence.pdf

Conclusion – Back during week 62, I started to question how close we were to touching the singularity and that question aligns somewhat to when we will see a true AGI. A well referenced paper was mentioned titled, “Future Progress in Artificial Intelligence: A Survey of Expert Opinion,” published in 2016 by Vincent C. Müller and Nick Bostrom [2].

Müller, V. C., & Bostrom, N. (2016). Future progress in artificial intelligence: A survey of expert opinion. In Fundamental issues of artificial intelligence (pp. 555-572). Springer, Cham. https://philpapers.org/rec/MLLFPI

Within that paper they note that expert opinions found that there is a 50/50 chance between 2040 and 2050 that a general artificial intelligence or AGI would spring into existence or be created. Keep in mind that debating when it will happen does not judge the ethics of creating it and what purpose it would have. Arguments can be made and are being made about if the singularity is inherently good or bad for civil society and civility in general. That is not a consideration I’m working with at the moment. My consideration of this is as an event or more to the point right before the event occurs. I did go back and read an article from a 2015 issue of the New Yorker magazine online called, “The Doomsday Invention: Will artificial intelligence bring us utopia or destruction?” [3]. That article is principally about Nick Bostrom and does consider utopia and destruction if you want to go give that a read.

Links and thoughts:

This was a really solid conversation between Kara and Chris. The discussions of journalistic ethics and making choices is what caught my attention. “Chris Cuomo’s Comeback”

Top 5 Tweets of the week:

Footnotes:

[1] https://www.theverge.com/2017/11/10/16617092/sophia-the-robot-citizen-ai-hanson-robotics-ben-goertzel

[2] fhttps://philpapers.org/rec/MLLFPI

[3] https://www.newyorker.com/magazine/2015/11/23/doomsday-invention-artificial-intelligence-nick-bostrom

What’s next for The Lindahl Letter?

Week 90: What is probabilistic machine learning?
Week 91: What are ensemble ML models?
Week 92: National AI strategies revisited
Week 93: Papers critical of ML
Week 94: AI hardware (RISC-V AI Chips)

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Brief aside: A bunch of shuffling has occurred in the forward-looking topics as we approach two years of The Lindahl Letter. Reworking the content for weeks 89 to 104 had to happen after the syllabus project. My focus and interest shifted a bit and due to that it made sense to go ahead and rework the pathing toward that extra special two-year anniversary of writing posts on Substack.

That brief aside is now complete. Some congratulations are in order, you made it to the post where I am going to write about academic paper mills and the future of synthetic papers flooding the academy. Using GPT-2 and a million-word corpus of my own words I trained a model to mimic my writing style. Unlike some of the newer iterations of those models you could tell it was not ready to pass as human generated. Initially, I had thought this post was going to be about the nature of how the peer review academic system of gatekeeping was failing compared to writing about the problematic possibility of synthetic writing being able to approach something that could pass the gatekeeping. Instead of going in that direction it was during this research that I realized the flooding problem of endless content creation was far worse than the breakdown of the academy-based gatekeeping system. Academic gatekeeping is a function of the quality of the gatekeepers and the rules they apply. That is inherent within the academy system, but it is being tested in a way that it has not had to endure before. Extreme oversupply of content is not going to slow down any time soon. To that end, I have spent a lot of time wondering about the future of publishing.

Large language models have created a scenario where a bit of prompt engineering can help generate blocks of prose. Previously I discussed a bit of the automation that is occurring within the instant news and financial reporting sections of the media. Using some type of model-based generation they take a bit of news and generate a story related to it and that can go out almost immediately. I have wondered about how many papers in the academic space get created in this way [1]. You can find examples of academics submitting papers to see if they can fool the reviewers into allowing them into journals [2]. Some scholars have taken this maybe a step too far and initially tried to publish fake papers [3]. I’m worried that flooding might occur within the world of academic publishing with fake journals and fake papers creating chaos.

Any field of academic study where a key journal exists and the academics within that field have a strong network and focus on the work in that journal or maybe a handful of key journals the system of academic publishing is probably still working well enough to unify the field. Within the field of machine learning things have broken down to the point where a lot of the content that I read is not from peer reviewed academic journals or prestigious conferences. I read a lot of preprints and things that people have shared. You could go through my entire independent introduction to machine learning syllabus and only really consume open access academic works [4]. The number of academic journals focused on machine learning is really (really) large and appears to be growing. That is one of the reasons that I really focused on citations to see trends and papers that are bubbling up to the top of active consideration. While you cannot totally trust citation counts as a metric of authority of ideas it is a solid way that can be used to gain single out of the noise that a paper might be worth reading.

If I was given a vote about things, then I would convene a regular conference cadence and associate a conference journal with it where submitted papers could be aggregated based on some peer review system of the conference attendees served as the gatekeeping system. That conference to journal system is probably my preferred method of journal aggregation as it is becoming community standard based. The people who want to be a part of it and read the journal are working together to uphold standards on the work they contribute to the academy. Right now, the opposite of that is occurring where people are defaulting back to reading preprints of papers and sometimes those preprints have more citations than the final location where the work is published. I’m pretty sure that based on the paywalls for some of the journals its entirely possible that the preprint reading rate is an order of magnitude larger. My preference here is keyed to building community vs. the totality of the contribution to the academy. I believe both elements are important and should be considered.

Links and thoughts:

“Everyone knows what YouTube is. Few know how it really works.”

“GTA leaks, TikTok search, and Apple reviews hotline”

Top 5 Tweets of the week:

Footnotes:

[1] https://www.nature.com/articles/d41586-021-00733-5

[2] https://undark.org/2020/11/26/fake-paper-predatory-journal/

[3] https://www.theatlantic.com/ideas/archive/2018/10/new-sokal-hoax/572212/

[4] https://github.com/nelslindahlx/Introduction-to-machine-learning-syllabus-2022

What’s next for The Lindahl Letter?

Week 89: That ML model is not an AGI
Week 90: What is probabilistic machine learning?
Week 91: What are ensemble ML models?
Week 92: National AI strategies revisited
Week 93: Papers critical of ML

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Research Note: You made it all the way to week 8 of 8 for the ML syllabus. You can find the files from the syllabus being built on GitHub. The latest version of the draft in PDF form can be found here.

This lecture is going to be provided in two parts. First, I’m going to provide you with a few scholarly articles that dig into what MLOps involves and how researchers are addressing the topic. Second, I’ll provide you my insights on the topic of MLOps which I have been presenting for the last few years. When you get to the point of applying ML techniques in production you will end up needing MLOps.

MLOps research papers

Alla, S., & Adari, S. K. (2021). What is mlops?. In Beginning MLOps with MLFlow (pp. 79-124). Apress, Berkeley, CA. https://arxiv.org/pdf/2103.08942.pdf[Zugriffam09.09.2021

Zhou, Y., Yu, Y., & Ding, B. (2020, October). Towards mlops: A case study of ml pipeline platform. In 2020 International conference on artificial intelligence and computer engineering (ICAICE) (pp. 494-500). IEEE. https://www.researchgate.net/profile/Yue-Yu-126/publication/349802712_Towards_MLOps_A_Case_Study_of_ML_Pipeline_Platform/links/61dd00575c0a257a6fdd62f3/Towards-MLOps-A-Case-Study-of-ML-Pipeline-Platform.pdf

Renggli, C., Rimanic, L., Gürel, N. M., Karlaš, B., Wu, W., & Zhang, C. (2021). A data quality-driven view of mlops. arXiv preprint arXiv:2102.07750. https://arxiv.org/pdf/2102.07750.pdf

Ruf, P., Madan, M., Reich, C., & Ould-Abdeslam, D. (2021). Demystifying mlops and presenting a recipe for the selection of open-source tools. Applied Sciences, 11(19), 8861. https://www.mdpi.com/2076-3417/11/19/8861/pdf

My insights about MLOps

Conceptually, I have been breaking down the categories of applied ML deployment based use cases into three buckets:

Bucket 1: “Things you can call” e.g. external API services
Bucket 2: “Places you can be” e.g. ecosystems where you can build out your footprint (AWS, GCP, Azure, and many others that are springing up for MLOps delivery)
Bucket 3: “Building something yourself” e.g. open source and self tooled solutions

These buckets will impact your ability to run MLOps and how much control you have over the frameworks and underlying data pipes. Bucket one is the easiest to implement because all you have to do is go out and consume it. You just need to connect to it, send some information out to it, get some information back, and you're ready to go. Bucket two is really about places where you can be totally within an ecosystem where you can build out your footprint for the endeavor. AWS, Azure, and GCP and many others that are springing up for MLOps delivery. I do mean many others are ready to provide you an ecosystem. You should be starting to see other ecosystems become available besides the major three. They are popping up and they're going to provide a different workflow in a different place where you can serve up your ML models and to be able to get going in this space. Now the third category or bucket three is where you will be building something yourself. These are the open source and self-tooled solutions. A few years ago, this space was the primary place people were building and now we are seeing a shift. We're seeing that movement into other buckets. Those API based solutions are so readily available and you can get into these ecosystems where you can get going so quickly. Things are moving around and changing. That categorization of the three buckets helps me think about where things are for use cases and where things are gonna happen. It's a very tactical question versus strategic one.

Some of the major players within the information technology space are trying to break into the machine learning operations (or MLOps) space. Like anything else, picking the right tools to get things done is about matching the right technology and use case to achieve the best possible results. We are really starting to see some solid maturity in the MLOps space. The next stage will be either a round of purchasing where established players buy up the upstart players building MLOps or the established players will build out the necessary elements to move past the newer players in the enterprise level market.

Let's look at the first technology in Table 1 which happens to be TensorFlow. You should not be surprised to see that TensorFlow has by far the largest influence at 154,162 stars. Getting a star requires a GitHub user to click the star function. People have really placed a lot of attention on TensorFlow. It has 2,933 contributors that means that almost 3,000 people are contributing to TensorFlow. From that point you can see that PyTorch drops off considerably. It's going from around 154k stars to just 47k stars. The number of contributors drops off significantly as well. Now, you're down to around 1,785. Now on the PyTorch example, they do have 4,620 branches which honestly I don't know why you would want to look at that many branches. No human wants to manage that many branches of anything. That is unmanageable in terms of iteration. You can see that scikit-learn has roughly 44,000 stars and has 1,936 contributors. So you can kind of see here that the three major projects that are out there for machine learning are definitely adopted. People are using them and they're making forks of it, they're making versions of it, and they're starting to really dig into it out in the wild of software development right now.

So now if we take it to the next level and look a little deeper in terms of what's happening with the MLOps part of it. You're gonna see a major drop-off. Remember TensorFlow had 154,162 stars. Now you're starting to see the number of stars drop off considerably. You're starting to see that number of stars at 10,000 or less. You are starting to see kubeflow, mlflow, and some of these things that you know are complex stuff like metaflow from Netflix and you're only gonna see 4,000 stars and each of these things is gonna have sub 500 contributors. We haven't seen everyone trying to implement MLOps swarm in and start using these things. One of the reasons for that rapid decline in interest has to be the previously described bucket 1 where you can just connect to an API and functionally someone else is running part of the day to day MLOps.

You probably noticed that the previous set of analysis was looking at data from 2021. I’m sure you wanted to see some updated data to see if things had changed significantly. I went back and reran the same table build to create a 2022 version. A few of the repositories changed order in terms of total stars, but for the most part things are relatively the same.

Links and thoughts:

“The Biggest Tech Divorce - WAN Show September 16, 2022”

“Everyone knows what YouTube is. Few know how it really works.”

“Websites are back: inside The Verge's redesign” (This is pure Nilay meta content)

Top 5 Tweets of the week:

What’s next for The Lindahl Letter?

Week 88: The future of publishing
Week 89: your ML model is not an AGI
Week 90: What is probabilistic machine learning?
Week 91: What are ensemble ML models?
Week 92: National AI strategies revisited
Week 93: Papers critical of ML
Week 94: AI hardware (RISC-V AI Chips)
Week 95: Quantum machine learning

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

This set of topics was either going to be the foundation to start this series or it was going to be collected as a set of thoughts at the end. You can tell that obviously I demurred from starting with ethics, fairness, bias, and privacy in machine learning until the full foundation was set for the topics under consideration. These topics are not assembled as an afterthought and are very important to any journey within the machine learning space. This technology in terms of machine learning and artificial intelligence has the potential to be near omnipresent in day to day life and certainly within anything where decision making or anything digital persists. Each of these topics is going to receive a solid overview followed by a series of scholarly articles like the previous lectures. You are now well aware from seeing dozens of other scholarly articles that these topics do not appear in each and every work and while they are conceptually foundational as intellectual guardrails they are not consistently presented that way in literature reviews or considerations for the practical work occurring within the machine learning space. I would clearly argue and have for years that just because you can do a thing does not mean that you should. You have to consider the consequences and realities of bringing that thing forward in a world where models and methods are so readily shared on GitHub and other platforms.

Overlap certainly occurs between the topics of ethics, fairness, bias, and privacy within the machine learning academic space. I have tried to sort the articles to help enhance readability within the different categories, but you will see some overlap.

Ethics - This topic got covered back in week 65. I’m going to rework part of that content here so if it feels familiar that is consistent with it appearing before about 20 weeks ago. Anybody preparing machine learning content should be comfortable with presenting ethics as a topic of consideration. I firmly believe and hope you would support that effort after coming along for this journey so far into this independent study syllabus. Ethics should be covered as a part of every machine learning course. Perhaps the best way to sum it up as an imperative would be to say, “Just because you can do a thing does not mean you should.” Machine learning opens the door to some incredibly advanced possibilities for drug discovery, medical image screening, or just spam detection to protect your inbox. The choices people make with machine learning use cases is where the technology and ethics have to be aligned.

No one really solid essay or set of essays on AI/ML ethics jumped out and caught my attention this week during my search. Part of my search involved digging into results from Google Scholar that yielded a ton of different options to read about “ethics in machine learning” [1]. A lot of those articles cover how to introduce ethics to machine learning courses and about the need to consider ethics when building machine learning implementations. Given that those two calls to action are the first things that come up and they are certainly adjacent to the primary machine learning content being shared it might make you take a moment to pause and consider how much the field of machine learning should deeply consider the idea that just because it can do something does not mean you should. Some use cases are pretty basic and the ethics of what is happening is fairly settled. Other use cases walk right up to the edge of what is reasonable in terms of fairness and equity.

Lo Piano, S. (2020). Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward. Humanities and Social Sciences Communications, 7(1), 1-7. https://www.nature.com/articles/s41599-020-0501-9.pdf

Greene, D., Hoffmann, A. L., & Stark, L. (2019). Better, nicer, clearer, fairer: A critical assessment of the movement for ethical artificial intelligence and machine learning. https://scholarspace.manoa.hawaii.edu/server/api/core/bitstreams/849782a6-06bf-4ce8-9144-a93de4455d1c/content

Fairness and Bias - Implementing machine learning algorithms generally involves working with imperfect datasets that have different biases that have to be accounted for and ultimately corrected.

Corbett-Davies, S., & Goel, S. (2018). The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023. https://arxiv.org/pdf/1808.00023.pdf

Chouldechova, A., & Roth, A. (2018). The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810. https://arxiv.org/pdf/1810.08810.pdf

Barocas, S., Hardt, M., & Narayanan, A. (2017). Fairness in machine learning. Nips tutorial, 1, 2. https://fairmlbook.org/pdf/fairmlbook.pdf

Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6), 1-35. https://arxiv.org/pdf/1908.09635.pdf

Yapo, A., & Weiss, J. (2018). Ethical implications of bias in machine learning. https://scholarspace.manoa.hawaii.edu/server/api/core/bitstreams/d062bd2a-df54-48d4-b27e-76d903b9caaa/content

Privacy - No conversation about machine learning would be complete without a consideration of privacy. A part of the ethical considerations surrounding the use of machine learning algorithms is inherently privacy of data and privacy of the outputs.

Ji, Z., Lipton, Z. C., & Elkan, C. (2014). Differential privacy and machine learning: a survey and review. arXiv preprint arXiv:1412.7584. https://arxiv.org/pdf/1412.7584.pdf

Rigaki, M., & Garcia, S. (2020). A survey of privacy attacks in machine learning. arXiv preprint arXiv:2007.07646. https://arxiv.org/pdf/2007.07646.pdf

Conclusion - I wanted to refocus my efforts on the macro considerations related to ethics in machine learning at this point. I remembered that Rob May shared a weekend commentary as a part of the Inside AI newsletter recently about the dark side of reducing friction in taking action with advanced technology [2]. Rob even went as far as sharing an article from one of my favorite technology related sources “The Verge” about just how easy and low friction it was to use machine learning to suggest new chemical weapon builds [3]. That is a very real example of where reducing friction to doing a thing opens the door to very problematic actions that illustrate the need for a foundational set of ethics.

Links and thoughts:

“How to design algorithms with fairness in mind”

“Why our Screwdriver took 3 YEARS”

Top 5 Tweets of the week:

Footnotes:

[1] https://scholar.google.com/scholar?q=ethics+in+machine+learning&hl=en&as_sdt=0&as_vis=1&oi=scholart

[2] https://inside.com/campaigns/inside-ai-31781/sections/inside-ai-commentary-by-robmay-275419

[3] https://www.theverge.com/2022/3/17/22983197/ai-new-possible-chemical-weapons-generative-models-vx

Research Note:

You can find the files from the syllabus being built on GitHub. The latest version of the draft is being shared by exports when changes are being made. https://github.com/nelslindahlx/Introduction-to-machine-learning-syllabus-2022

What’s next for The Lindahl Letter?

Week 87: MLOps (ML syllabus edition 8/8)
Week 88: The future of publishing
Week 89: your ML model is not an AGI
Week 90: What is probabilistic machine learning?
Week 91: What are ensemble ML models?
Week 92: National AI strategies revisited
Week 93: Papers critical of ML
Week 94: AI hardware (RISC-V AI Chips)
Week 95: Quantum machine learning

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Neuroscience is a complex topic to dig into in general. Studying the nervous system is a complex thing to do before you add in the concept of machine learning or artificial intelligence. Within the context of machine learning it gets even more interesting for academic researchers, practitioners, and anybody building neural networks. Understanding that context of complexity within any inquiry into neuroscience, it will make sense here to focus on 5 scholarly articles that could help provide a solid context here for the relationship between neuroscience and machine learning. Within this section of inquiry the articles are really going to bring forward the complexity of the issue. The scholarly articles selected to cover neuroscience include a lot of focus on how the two subjects work together and the future of that collaboration.

Savage, N. (2019). How AI and neuroscience drive each other forwards. Nature, 571(7766), S15-S15. https://www.nature.com/articles/d41586-019-02212-4

Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., ... & Kording, K. P. (2019). A deep learning framework for neuroscience. Nature neuroscience, 22(11), 1761-1770. https://www.nature.com/articles/s41593-019-0520-2

Marblestone, A. H., Wayne, G., & Kording, K. P. (2016). Toward an integration of deep learning and neuroscience. Frontiers in computational neuroscience, 94. https://www.frontiersin.org/articles/10.3389/fncom.2016.00094/pdf

Richiardi, J., Achard, S., Bunke, H., & Van De Ville, D. (2013). Machine learning with brain graphs: predictive modeling approaches for functional imaging in systems neuroscience. IEEE Signal processing magazine, 30(3), 58-70. https://archive-ouverte.unige.ch/unige:33936/ATTACHMENT01

Vu, M. A. T., Adalı, T., Ba, D., Buzsáki, G., Carlson, D., Heller, K., ... & Dzirasa, K. (2018). A shared vision for machine learning in neuroscience. Journal of Neuroscience, 38(7), 1601-1607. https://www.jneurosci.org/content/jneuro/38/7/1601.full.pdf

Bonus Papers

This section includes a few additional papers that I have enjoyed and thought you might as well. They are not sorted in any particular order. This section may see the most updates between first publication and any updates of this syllabus. I’m sure that papers will get recommended to be included and if they don’t naturally fit into the main structure without overloading the reader, then they will end up here in the bonus papers section.

Marcus, G. (2018). Deep learning: A critical appraisal. arXiv preprint arXiv:1801.00631. https://arxiv.org/ftp/arxiv/papers/1801/1801.00631.pdf

Lake, B., & Baroni, M. (2018). Still not systematic after all these years: On the compositional skills of sequence-to-sequence recurrent networks. https://openreview.net/pdf?id=H18WqugAb

Mitchell, M. (2021). Why AI is harder than we think. arXiv preprint arXiv:2104.12871. https://arxiv.org/pdf/2104.12871.pdf

Biderman, S., & Scheirer, W. J. (2020). Pitfalls in machine learning research: Reexamining the development cycle. http://proceedings.mlr.press/v137/biderman20a/biderman20a.pdf

Links and thoughts:

“The Future of AI is Self-Organizing and Self-Assembling (w/ Prof. Sebastian Risi)”

“The Man behind Stable Diffusion”

“Lab Naming Controversy - WAN Show August 26, 2022”

Top 6 Tweets of the week:

Research Note:

What’s next for The Lindahl Letter?

Week 86: Ethics, fairness, bias, and privacy (ML syllabus edition 7/8)
Week 87: MLOps (ML syllabus edition 8/8)
Week 88: The future of publishing
Week 89: your ML model is not an AGI
Week 90: What is probabilistic machine learning?
Week 91: What are ensemble ML models?
Week 92: National AI strategies revisited
Week 93: Papers critical of ML
Week 94: AI hardware (RISC-V AI Chips)
Week 95: Quantum machine learning

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

You may find in the literature that this topic of neural networks is sometimes called the zoo or more specifically, “the neural network zoo.” Corresponding to the articles that make this reference is a wonderful included graphic that shows a ton of different neural networks and can really give you a sense of how they work at the most fundamental level. Two papers that make this reference and include that wonderful graphic are from researchers at the Asimov institute that had papers published in 2016 and 2022. Both of those papers are great places to start learning about neural networks.

Van Veen, F., & Leijnen, S. (2016). The neural network zoo. The Asimov Institute. https://www.asimovinstitute.org/neural-network-zoo/

Leijnen, S., & Veen, F. V. (2020). The neural network zoo. Multidisciplinary Digital Publishing Institute Proceedings, 47(1), 9. https://www.mdpi.com/2504-3900/47/1/9

That brief introduction aside. We are now going to focus on specific types of neural networks and next week our focus will shift to the topic of neuroscience. I have separated the two topics on purpose. Briefly, I had considered trying to combine the two topics as one set of content, but I think it would have become unwieldy in terms of trying to present a distinct point of view on both topics. Digging into neural networks is really about digging into deep learning and trying to understand it as a subfield of machine learning. Keep in mind that while machine learning is exciting it's just a small part of the broader grouping of artificial intelligence as a field of study. I’m going to provide a brief introduction and some links to scholarly articles for 9 types of neural networks that you might run into. This list is in no way comprehensive and is built and ordered based on my interests as a researcher. A lot of speciality models and methods exist. One of them could end up displacing something on the list if it proves highly effective. I’m open to suggestions of course for different models or even orders of explanation.

Artificial Neural Networks (ANN)

Simulated Neural Networks (SNN)

Recurrent Neural Networks (RNN)

Generative Adversarial Network (GAN)

Convolutional Neural Network (CNN)

Deep Belief Networks (DBN)

Self Organizing Neural Network (SONN)

Deeply Quantized Neural Networks (DQNN)

Modular Neural Network (MNN)

Artificial Neural Networks (ANN) - This is the model that is generally shortened to just neural networks and it is a very literal title. An ANN is really an attempt or more accurately a computational model designed to either mimic or create a neural network akin to what is used within a biological brain using hardware or software. You can assume this model to be fundamental to any consideration of neural networks, but you are going to quickly want to dig into other more targeted models based on your specific use case. What you are trying to accomplish will certainly help you focus on a model or method that best meets the needs of that course of action. However, in the abstract people will consider how to build ANNs and what they could be used for as the technology progresses.

Jain, A. K., Mao, J., & Mohiuddin, K. M. (1996). Artificial neural networks: A tutorial. Computer, 29(3), 31-44. https://www.cse.msu.edu/~jain/ArtificialNeuralNetworksATutorial.pdf

Hassoun, M. H. (1995). Fundamentals of artificial neural networks. MIT press. https://www.researchgate.net/profile/Terrence-Fine/publication/3078997_Fundamentals_of_Artificial_Neural_Networks-Book_Reviews/links/56ebf73a08aee4707a3849a6/Fundamentals-of-Artificial-Neural-Networks-Book-Reviews.pdf

Simulated Neural Networks (SNN) - As you work along your journey in the deep learning space and really start to dig into neural networks you will run into those ANNs and very quickly a subset of machine learning adjacent to that type of model called the simulated neural networks. Creating a neural network that truly mimics the depth and capacity of the brain is something to strive for right now and with that constraint it makes sense that work is being done to simulate the best possible representation we can achieve currently or a very special use case that limits the simulation. Using models that generate a simulation based on some complex sets of mathematics, these SNNs are being created to challenge certain use cases. One of the papers shared below is associated with figuring out the shelf life of processed cheese for example.

Kudela, P., Franaszczuk, P. J., & Bergey, G. K. (2003). Changing excitation and inhibition in simulated neural networks: effects on induced bursting behavior. Biological cybernetics, 88(4), 276-285. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.57.9281&rep=rep1&type=pdf

Goyal, S., & Goyal, G. K. (2012). Application of simulated neural networks as non-linear modular modeling method for predicting shelf life of processed cheese. Jurnal Intelek, 7(2), 48-54. https://ir.uitm.edu.my/id/eprint/34381/1/34381.pdf

Recurrent Neural Networks (RNN) - At some point you will want to move from simulating and modeling to accomplishing the hard work of applied machine learning for a specific use case. One of the models you will see being used actively are variations and direct implementations of recurrent neural networks. Within this type of model patterns are going to be identified within the data and the modeling will be based on those patterns to engage in a prediction of the most likely next scenario. This is a useful approach for speech recognition or handwriting analysis. You probably have run into an RNN at some point today with your smartphone or a connected home speaker. A lot of very interesting applied use cases exist for RNNs.

Lipton, Z. C., Berkowitz, J., & Elkan, C. (2015). A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019. https://arxiv.org/pdf/1506.00019.pdf

Yin, C., Zhu, Y., Fei, J., & He, X. (2017). A deep learning approach for intrusion detection using recurrent neural networks. Ieee Access, 5, 21954-21961. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8066291

Generative Adversarial Network (GAN) - For me personally, this is where things get interesting. Instead of looking at one neural network this GAN model creates the possibility of gamification or more to the point direct competition between models in an adversarial way. Two generative models or potentially more can be compared to figure out an optimal approach. I think this is a very interesting methodology and one that could yield very interesting futur results. You can read a lot about this and see the early code published about 8 years ago from Ian Goodfellow over on GitHub [1].

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27. https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf

Yi, X., Walia, E., & Babyn, P. (2019). Generative adversarial network in medical imaging: A review. Medical image analysis, 58, 101552. https://arxiv.org/pdf/1809.07294.pdf

Aggarwal, A., Mittal, M., & Battineni, G. (2021). Generative adversarial network: An overview of theory and applications. International Journal of Information Management Data Insights, 1(1), 100004. https://www.sciencedirect.com/science/article/pii/S2667096820300045

Convolutional Neural Network (CNN) - You will run into use cases where you want to dig into visual imagery and that is where CNNs will probably pop up very quickly. You are building a model or algorithm that based on weights and biases can evaluate a series of images or potentially other content. The process of how layers are made and what exactly fuels a CNN is a very interesting process of abstraction.

Albawi, S., Mohammed, T. A., & Al-Zawi, S. (2017, August). Understanding of a convolutional neural network. In 2017 international conference on engineering and technology (ICET) (pp. 1-6). Ieee. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6197001/pdf/CIN2018-6973103.pdf

O'Shea, K., & Nash, R. (2015). An introduction to convolutional neural networks. arXiv preprint arXiv:1511.08458. https://arxiv.org/pdf/1511.08458

Deep Belief Networks (DBN) - You may have run into this one in the news recently with all the coverage related to drug discovery. DBNs are frequently described as graphical in nature and generative. The reason it works for something as influential and interesting as drug discovery is that you can produce all the possible values for potential new drugs in a use case and evaluate those results. This is an area where I think the things being produced will be extremely beneficial assuming the methodology is used in positive ways.

Salakhutdinov, R., & Murray, I. (2008, July). On the quantitative analysis of deep belief networks. In Proceedings of the 25th international conference on Machine learning (pp. 872-879). https://era.ed.ac.uk/bitstream/handle/1842/4588/MurrayI_On%20the%20Quantitative%20Analysis.pdf?sequence=1&isAllowed=y

Hinton, G. E. (2009). Deep belief networks. Scholarpedia, 4(5), 5947. http://scholarpedia.org/article/Deep_belief_networks

Self Organizing Neural Network (SONN) - Imagine a neural network model based on feature maps or Kohonen maps that is unsupervised and self-organizing. Within that explanation you are going to get a self organizing neural network model. This could be used for adaptive pattern recognition or just regular pattern recognition. The two references shared below will spell out how this works in more detail if you are interested.

Carpenter, G. A., & Grossberg, S. (1988). The ART of adaptive pattern recognition by a self-organizing neural network. Computer, 21(3), 77-88. https://search.iczhiku.com/paper/bELWExDU1wAMpDkP.pdf

Carpenter, G. A., & Grossberg, S. (Eds.). (1991). Pattern recognition by self-organizing neural networks. MIT Press. https://books.google.com/books?id=2u1fH0mxfz0C&lpg=PP19&ots=d_sdwFOQk3&dq=%22Self%20Organizing%20Neural%20Network%22%20machine%20learning&lr&pg=PP19#v=onepage&q=%22Self%20Organizing%20Neural%20Network%22%20machine%20learning&f=false

Deeply Quantized Neural Networks (DQNN) - Within a neural network model when you are creating weights you could elect to use only very small ones from 1 to 8 bits and to that end you would be on your way to a deeply quantized neural network. Development tools exist for this type of effort like the Google team’s qKeras [2] and Larq [3]. Getting open access to papers on this topic is a little harder than some of the others, but you can pretty quickly get to the code on how to implement this type of neural network.

F. Loro, D. Pau and V. Tomaselli, "A QKeras Neural Network Zoo for Deeply Quantized Imaging," 2021 IEEE 6th International Forum on Research and Technology for Society and Industry (RTSI), 2021, pp. 165-170, doi: 10.1109/RTSI50628.2021.9597341.

Dogaru, R., & Dogaru, I. (2021). LB-CNN: An Open Source Framework for Fast Training of Light Binary Convolutional Neural Networks using Chainer and Cupy. arXiv preprint arXiv:2106.15350. https://arxiv.org/ftp/arxiv/papers/2106/2106.15350.pdf

Modular Neural Network (MNN) - Within this model you are going to want to create independent neural networks and moderate them. Within this framework each independent neural network is a module of the whole. This one always makes me think of building blocks for some reason, but that is a simplistic representation given the ability for moderation required to make this work.

Devin, C., Gupta, A., Darrell, T., Abbeel, P., & Levine, S. (2017, May). Learning modular neural network policies for multi-task and multi-robot transfer. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 2169-2176). IEEE. https://arxiv.org/pdf/1609.07088.pdf

Happel, B. L., & Murre, J. M. (1994). Design and evolution of modular neural network architectures. Neural networks, 7(6-7), 985-1004. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.54.8248&rep=rep1&type=pdf

Conclusion - This is an intense way of starting to dig into neural networks and you will very quickly see that the use cases outside of pure machine learning or artificial intelligence are driving this field forward. A lot of these use cases are within the medical field or health care in general and are super interesting and somewhat related to neuroscience. That is where the next lecture will head in this series. Discussion will move from specific types of neural networks and the research associated with them to the broader topic of neuroscience and how it relates to machine learning.

Links and thoughts:

“Types of Neural Network Architectures”

“[ML News] AI models that write code (Copilot, CodeWhisperer, Pangu-Coder, etc.)”

“Trust Me Bro - WAN Show August 12, 2022”

Top 5 Tweets of the week:

Footnotes:

[1] https://github.com/goodfeli/adversarial

[2] https://github.com/google/qkeras

[3] https://github.com/larq/larq

Research Note:

What’s next for The Lindahl Letter?

Week 85: Neuroscience (ML syllabus edition 6/8)

Week 86: Ethics, fairness, bias, and privacy (ML syllabus edition 7/8)

Week 87: MLOps (ML syllabus edition 8/8)

Week 88: The future of publishing

Week 89: Understanding data quality

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

During the last lecture we jumped in and looked at 10 machine learning algorithms. This week the content contained within this lecture will cover from a machine learning perspective reinforcement learning and 3 types of supervised learning. Those types of supervised learning will include the general use case of supervised learning, unsupervised learning, and the super interesting semi-supervised learning. Like the model for consideration used in the last lecture I’ll cover the topics in general and provide links to papers covering the topic to allow people looking for a higher degree of depth to dive deeper into academic papers to achieve that goal. My general preference here is to find academic papers that are both readable and are generally available for you to actually read with very low friction. Within the machine learning and artificial intelligence space a lot of papers are generally available and that is great for literature reviews and generally for scholarly work and practitioners working to implement the technology. My perspective is a mix between those two worlds which could be defined as a pracademic view of things. All right; here we go.

Reinforcement learning - Welcome to the world of machine learning. This is probably the first approach you are going to learn about in your journey. That’s right, it's time to consider for a brief moment the world of reinforcement learning. You are probably going to need to start to create some intelligent agents and you will want to figure out how to maximize the reward those agents could get. One method of achieving that result is called reinforcement learning. A lot of really great tutorials exist trying to explain this concept and one that I enjoyed was from Towards Data Science way back in 2018 [1]. The nuts and bolts of this one involve trial and error with an intelligent agent trying to learn from mistakes using a maximization of reward function to avoid going down paths that don’t offer greater reward. The key takeaway here is that during the course of executing a model or algorithm a maximization function based on reward has to be in place to literally reinforce maximization during learning. I’m sharing references and links to 4 academic papers about this topic to help you dig into reinforcement learning with a bit of depth if you feel so inclined.

Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of artificial intelligence research, 4, 237-285. https://www.jair.org/index.php/jair/article/view/10166/24110

Sutton, R. S., & Barto, A. G. (1998). Introduction to reinforcement learning. https://login.cs.utexas.edu/sites/default/files/legacy_files/research/documents/1%20intro%20up%20to%20RL%3ATD.pdf

Szepesvári, C. (2010). Algorithms for reinforcement learning. Synthesis lectures on artificial intelligence and machine learning, 4(1), 1-103. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.308.549&rep=rep1&type=pdf

Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602. https://arxiv.org/pdf/1312.5602.pdf

Supervised learning - You knew it would only be a matter of time before we went out to some content from our friends over at IBM [2]. They note that within a world where you have some labeled datasets and are training an algorithm to engage in classification or perhaps regression, but probably classification. In some ways the supervised element here is the labeling and guiding of the classification. Outside of somebody or a lot of people sitting and labeling training data the supervision is not from somebody outright sitting and watching the machine learning model run step by step. Some ethical considerations need to be taken into account at this point. A lot of people have worked to engage in data labeling. A ton of services exist to help bring people together to help do this type of work. Back in 2018 Maximilian Gahntz published a piece in Towards Data Science that talked about the invisible workers that are doing all that labeling in large curated datasets [3]. Within the world of supervised learning being able to get high quality labeled data really impacts the ability to make solid models. It’s our ethical duty as researchers to consider what that work involves and who is doing that work. Another article in the MIT Technology Review back in 2020 covered the idea of how gig workers are powering a lot of this labeling [4]. The first academic article linked below with Saiph Savage as a co-author will cover the same topic and you should consider giving it a read to better understand how machine learning is built from dataset to model. After that article, the next two are general academic articles about predicting good probabilities and empirical comparisons to help ground your understanding of supervised learning.

Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P. (2018, April). A data-driven analysis of workers' earnings on Amazon Mechanical Turk. In Proceedings of the 2018 CHI conference on human factors in computing systems (pp. 1-14). https://arxiv.org/pdf/1712.05796.pdf

Niculescu-Mizil, A., & Caruana, R. (2005, August). Predicting good probabilities with supervised learning. In Proceedings of the 22nd international conference on Machine learning (pp. 625-632). https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.60.7135&rep=rep1&type=pdf

Caruana, R., & Niculescu-Mizil, A. (2006, June). An empirical comparison of supervised learning algorithms. In Proceedings of the 23rd international conference on Machine learning (pp. 161-168). http://www.cs.cornell.edu/~alexn/papers/empirical.icml06.pdf

Unsupervised learning - It’s a good thing that you were paying very close attention to the explanation of supervised learning above. Imagine that the humans or in some cases the vast collectives of humans labeling training sets just stopped doing that. Within the unsupervised learning world the classification within the machine learning problem space is going to be handed differently. Labeling and the creation of classification has to be a part of the modeling methodology. This topic always makes me think of the wonderful time capsule of a technology show about startups called Silicon Valley (2014 to 2019) that was broadcast by HBO. They had an algorithm explained at one point as being able to principally identify food as hot dog or not hot dog. That’s it the model only could do the one task. It was not capable of correctly identifying all food as that is a really complex task. Trying to use unsupervised learning for example, based on tags and other information identifying different types of food in photographs is something that people have certainly done with unsupervised learning approaches. I’m only sharing one paper about this approach and its from 2001.

Hofmann, T. (2001). Unsupervised learning by probabilistic latent semantic analysis. Machine learning, 42(1), 177-196. https://link.springer.com/content/pdf/10.1023/A:1007617005950.pdf

Semi-supervised learning - All 3 of these different types of learning supervised, unsupervised, and semi-supervised are related. They are different methods of attacking a problem space related to learning as part of the border landscape of machine learning. You can imagine that people wanted to try to create a hybrid model when a limited set of labeled data is used to help begin the modeling process. That is the essence of the process of building out a semi-supervised learning approach [5]. I’m sharing 3 different academic papers related to this topic that cover a literature review, a book about it, and the more advanced topic of pseudo labeling.

Zhu, X. J. (2005). Semi-supervised learning literature survey. https://minds.wisconsin.edu/bitstream/handle/1793/60444/TR1530.pdf?sequence=1

Chapelle, O., Scholkopf, B., & Zien, A. (2009). Semi-supervised learning (chapelle, o. et al., eds.; 2006)[book reviews]. IEEE Transactions on Neural Networks, 20(3), 542-542. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=4787647

Lee, D. H. (2013, June). Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML (Vol. 3, No. 2, p. 896). https://www.kaggle.com/blobs/download/forum-message-attachment-files/746/pseudo_label_final.pdf

Conclusion - This lecture covered reinforcement learning and 3 types of supervised learning. You could spend a lot of time digging into academic articles and books related to these topics. Generally, I believe you will start to want to look at use cases and direct your attention to highly specific examples of applied machine learning at this point. Fortunately, a lot of those papers exist and you won’t be disappointed.

Links and thoughts:

“[ML News] This AI completes Wikipedia! Meta AI Sphere | Google Minerva | GPT-3 writes a paper”

Top 4 Tweets of the week:

Footnotes:

[1] https://towardsdatascience.com/reinforcement-learning-101-e24b50e1d292

[2] https://www.ibm.com/cloud/learn/supervised-learning#toc-unsupervis-Fo3jDcmY

[3] https://towardsdatascience.com/the-invisible-workers-of-the-ai-era-c83735481ba

[4] https://www.technologyreview.com/2020/12/11/1014081/ai-machine-learning-crowd-gig-worker-problem-amazon-mechanical-turk/

[5] https://towardsdatascience.com/supervised-learning-but-a-lot-better-semi-supervised-learning-a42dff534781

Research Note:

What’s next for The Lindahl Letter?

Week 84: Neural networks (ML syllabus edition 5/8)

Week 85: Neuroscience (ML syllabus edition 6/8)

Week 86: Ethics, fairness, bias, and privacy (ML syllabus edition 7/8)

Week 87: MLOps (ML syllabus edition 8/8)

Week 88: The future of publishing

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Welcome to the lecture on ML algorithms. This topic was held until the 3rd installment of this series to allow a foundation for the concept of machine learning to develop. At some point, you are going to want to operationalize your knowledge of machine learning to do some things. For the vast majority of you one of these ML algorithms will be that something. Please take a step back and consider this very real scenario. Within the general scientific community getting different results every time you run the same experiment makes publishing difficult. That does not stop authors in the ML space. Replication and the process of verifying scientific results is often difficult or impossible without similar setups and the same datasets. Within the machine learning space where a variety of different ML algorithms exist that is a very normal outcome. Researchers certainly seem to have gotten very used to getting a variety of results. I’m not talking about using post theory science to publish based on allowing the findings to build knowledge instead of the other way around. You may very well get slightly different results every time one of these ML algorithms is invoked. You have been warned. Now let the adventure begin.

One of the few Tweets that really made me think about the quality of ML research papers and the research patterns impacting quality was from Yaroslav Bulatov who works on the PyTorch team back on January 22, 2022. That tweet referenced a paper on ArXiv called, “Descending through a Crowded Valley — Benchmarking Deep Learning Optimizers,” from 2021 [1].

That paper digs into the state of things where hundreds of optimization methods exist. It pulls together a really impressive list. The list itself was striking just in the volume of options available. My next thought was about just how many people are contributing to this highly overcrowded field of machine learning. That paper about deep learning optimizers covered a lot of ground and would be a good place to start digging around. We are going to approach this a little differently based on a look at the most common ones.

Here are some (10) very common ML algorithms (this is not intended to be an exhaustive list):

XGBoost

Naive Bayes algorithm

Linear regression

Logistic regression

Decision tree

Support Vector Machine (SVM) algorithm

K-nearest neighbors (KNN) algorithm

K-means

Random forest algorithm

Diffusion

I’m going to talk about each of these algorithms briefly or this would be a very long lecture. We certainly could go all hands and spend several hours all in together in a state of irregular operations covering these topics, but that is not going to happen today. To make this a more detailed syllabus version of the lecture I’m going to include a few references to relevant papers you can get access to and read after each general introduction. My selected papers might not be the key paper or the most cited. Feel free to make suggestions if you feel a paper better represents the algorithm. I’m open to suggestions.

XGBoost - Some people would argue with a great deal of passion that we could probably be one and done after introducing this ML algorithm. You can freely download the package for this one [2]. It has over 20,000 stars on GitHub and has been forked over 8,000 times [3]. People really seem to like this one and have used it to win competitions and generally get great results. Seriously, you will find references to XGBoost all over these days. It has gained a ton of attention and popularity. Not exactly to the level of being a pop culture reference, but within the machine learning community it is well known. The package is based on gradient boosting and provides parallel tree boating (GBDT, GBM). This package generally creates a series of models that boost the trees and help create overfitting in sequential efforts. You can read a paper from 2016 about it on arXiv called, “XGBoost: A Scalable Tree Boosting System” [4]. The bottom line on this one is that you get a lot of benefits from gradient boosting built into a software package that can get you moving quickly toward your goal of success.

Chen, T., & Guestrin, C. (2016, August). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining (pp. 785-794). https://dl.acm.org/doi/pdf/10.1145/2939672.2939785

Chen, T., He, T., Benesty, M., Khotilovich, V., Tang, Y., Cho, H., & Chen, K. (2015). Xgboost: extreme gradient boosting. R package version 0.4-2, 1(4), 1-4. https://cran.microsoft.com/snapshot/2017-12-11/web/packages/xgboost/vignettes/xgboost.pdf

Naive Bayes algorithm - You knew I would have to have something Bayes related near the top of this list. This one is a type of classifier that helps evaluate the probability or relationship between classes. One of the classes with the highest probability will be considered the most likely class. It also assumes that those features are independent. I found a paper on this one that was cited about 4,146 times called, “An empirical study of the naive Bayes classifier” [5].

Rish, I. (2001, August). An empirical study of the naive Bayes classifier. In IJCAI 2001 workshop on empirical methods in artificial intelligence (Vol. 3, No. 22, pp. 41-46). https://www.researchgate.net/profile/Irina-Rish/publication/228845263_An_Empirical_Study_of_the_Naive_Bayes_Classifier/links/00b7d52dc3ccd8d692000000/An-Empirical-Study-of-the-Naive-Bayes-Classifier.pdf

Linear regression - This is the most basic algorithm and statistical technique in use here where based on a line (linear) a relationship can be charted for prediction between two things. A lot of the graphics you will see where a lot of content is mapped on a chart with a line dividing the general middle of the distribution would potentially be using some form of linear regression.

Forkuor, G., Hounkpatin, O. K., Welp, G., & Thiel, M. (2017). High resolution mapping of soil properties using remote sensing variables in south-western Burkina Faso: a comparison of machine learning and multiple linear regression models. PloS one, 12(1), e0170478. https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0170478&type=printable

Maulud, D., & Abdulazeez, A. M. (2020). A review on linear regression comprehensive in machine learning. Journal of Applied Science and Technology Trends, 1(4), 140-147. https://jastt.org/index.php/jasttpath/article/view/57/20

Logistic regression - This type of statistical model allows an algorithmic analysis of the probability of success or failure. You could model other binary type questions. The good folks over at IBM have an entire set of pages set up to run through how logistic regression could be a tool to help with decision making [6]. This model is everywhere in simple analysis of things when people are trying to work toward a single decision.

Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019). A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. Journal of clinical epidemiology, 110, 12-22. https://www.researchgate.net/profile/Ewout-Steyerberg/publication/331028284_A_systematic_review_shows_no_performance_benefit_of_machine_learning_over_logistic_regression_for_clinical_prediction_models/links/5c66bed192851c1c9de3251b/A-systematic-review-shows-no-performance-benefit-of-machine-learning-over-logistic-regression-for-clinical-prediction-models.pdf

Dreiseitl, S., & Ohno-Machado, L. (2002). Logistic regression and artificial neural network classification models: a methodology review. Journal of biomedical informatics, 35(5-6), 352-359. https://core.ac.uk/download/pdf/82131402.pdf

Decision tree - Imagine diagramming decisions and coming to a fork where you have to decide to go one way or the other. That is how decision trees work based on inputs and corresponding outputs. Normally you will have a bunch of interconnected forks in the road and together they form up a decision tree. A lot of really great explanations of this exist online. One of my favorite ones is from Towards Data Science and was published way back in 2017 [7].

Dietterich, T. G., & Kong, E. B. (1995). Machine learning bias, statistical bias, and statistical variance of decision tree algorithms (pp. 0-13). Technical report, Department of Computer Science, Oregon State University. https://citeseerx.ist.psu.edu/viewdoc/download?rep=rep1&type=pdf&doi=10.1.1.38.2702

Support Vector Machine (SVM) algorithm - You are going to need to imagine graphing out a bunch of data points then trying to come up with a line that separates them with a maximum margin [8].

Noble, W. S. (2006). What is a support vector machine?. Nature biotechnology, 24(12), 1565-1567. https://www.ifi.uzh.ch/dam/jcr:00000000-7f84-9c3b-ffff-ffffc550ec57/what_is_a_support_vector_machine.pdf

Wang, L. (Ed.). (2005). Support vector machines: theory and applications (Vol. 177). Springer Science & Business Media. https://personal.ntu.edu.sg/elpwang/PDF_web/05_SVM_basic.pdf

Hearst, M. A., Dumais, S. T., Osuna, E., Platt, J., & Scholkopf, B. (1998). Support vector machines. IEEE Intelligent Systems and their applications, 13(4), 18-28. https://www.ifi.uzh.ch/dam/jcr:00000000-7f84-9c3b-ffff-ffffbdb9a74e/SVM.pdf

K-nearest neighbors (KNN) algorithm - Our friends over at IBM are sharing all sorts of knowledge online including a bit about the KNN algorithm [9]. Apparently, the best commentary explaining this one comes from Sebastian Raschka back in the fall of 2018 [10]. This one is pretty much what you would expect from a technique that looks at distance between neighboring points.

Peterson, L. E. (2009). K-nearest neighbor. Scholarpedia, 4(2), 1883. http://scholarpedia.org/article/K-nearest_neighbor

Zhang, M. L., & Zhou, Z. H. (2005, July). A k-nearest neighbor based algorithm for multi-label classification. In 2005 IEEE international conference on granular computing (Vol. 2, pp. 718-721). IEEE. https://www.researchgate.net/profile/Min-Ling-Zhang-2/publication/4196695_A_k-nearest_neighbor_based_algorithm_for_multi-label_classification/links/565d98f408ae1ef92982f866/A-k-nearest-neighbor-based-algorithm-for-multi-label-classification.pdf

K-means - Some algorithms work to evaluate clusters and K-means is one of those. You can use this to try to help classify unlabeled data into clusters which can be helpful.

Sinaga, K. P., & Yang, M. S. (2020). Unsupervised K-means clustering algorithm. IEEE access, 8, 80716-80727. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9072123

Random forest algorithm - Most of the jokes that have been told within the machine learning space often relate to decision trees. The field is not full of a lot of jokes, but trees falling in a random forest are often included in that branch. People really liked the random forest algorithm for a time. You can imagine that a bunch of trees are created to engage in the prediction of classification. The random tree in the forest with the best classification production becomes the winner. This is great as it could find something that was noval or unexpected result based on the randomness.

Biau, G., & Scornet, E. (2016). A random forest guided tour. Test, 25(2), 197-227. https://arxiv.org/pdf/1511.05741.pdf

Diffusion - Previously I covered diffusion back in week 79 to try to figure out why it is becoming so popular. It is in no way as popular as XGBoost, but it has been gaining popularity. Over in the field of thermodynamics you could study gas molecules. Maybe you want to learn about how those gas molecules would diffuse from a high density to a low density area and you would also want to know how those gas molecules would reverse course. That is the basic theoretical part of the equation you need to absorb at the moment. Within the field of machine learning people have been building models that learn how based on degree of noise to diffuse the data and then reverse that process. That is basically the diffusion process in a nutshell. You can imagine that the cost to do this is computationally expensive.

Wei, Q., Jiang, Y., & Chen, J. Z. (2018). Machine-learning solver for modified diffusion equations. Physical Review E, 98(5), 053304. https://arxiv.org/pdf/1808.04519.pdf

Dhariwal, P., & Nichol, A. (2021). Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34, 8780-8794. https://proceedings.neurips.cc/paper/2021/file/49ad23d1ec9fa4bd8d77d02681df5cfa-Paper.pdf

Wrapping this lecture up should be pretty straightforward. Feel free to dig into some of those papers if anything grabbed your attention this week. A lot of algorithms exist in the machine learning space. I tried to grab algorithms that are timeless and will always be relevant when considering where machine learning as a field is going.

Links and thoughts:

“[ML News] BLOOM: 176B Open-Source | Chinese Brain-Scale Computer | Meta AI: No Language Left Behind”

“Is Intel ARC REALLY Canceled? - WAN Show July 29, 2022”

Top 5 Tweets of the week:

Footnotes:

[1] https://arxiv.org/pdf/2007.01547.pdf

[2] https://xgboost.ai/

[3] https://github.com/dmlc/xgboost

[4] https://arxiv.org/pdf/1603.02754.pdf

[5] https://www.cc.gatech.edu/home/isbell/classes/reading/papers/Rish.pdf

[6] https://www.ibm.com/topics/logistic-regression

[7] https://towardsdatascience.com/decision-trees-in-machine-learning-641b9c4e8052

[8] https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47

[9] https://www.ibm.com/topics/knn

[10] https://sebastianraschka.com/pdf/lecture-notes/stat479fs18/02_knn_notes.pdf

Research Note:

What’s next for The Lindahl Letter?

Week 83: Machine learning Approaches (ML syllabus edition 4/8)

Week 84: Neural networks (ML syllabus edition 5/8)

Week 85: Neuroscience (ML syllabus edition 6/8)

Week 86: Ethics, fairness, bias, and privacy (ML syllabus edition 7/8)

Week 87: MLOps (ML syllabus edition 8/8)

I'll try to keep the what’s next list forward looking with at least five weeks of posts in planning or review. If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

You can find a lot of quality explanations of the differences between the various flavors of machine learning [1]. This second lecture in the introduction to ML syllabus series should open with a series of the best literature reviews I could find and pull together to share. That will be the second part of this lecture. The third part will cover the intersection of programming languages. Some rather high quality textbooks and manuscripts exist within the field of machine learning. You can even find ones for free on GitHub and other places. Instead of starting with the obvious way to go by digging into some weighty tomes. I’m going to spend some time sharing readouts of some of the most highly cited machine learning papers. For a lot of people jumping into the field they are working on something in a different field of study and find a use case or a business related adventure that could benefit from machine learning. Typically at this point they are going to start digging into software and can get going very rapidly. That part of the journey requires no real deep dive into the relevant literature. It’s great that people can just jump in and find machine learning accessible. However, (you knew that was coming) the next phase in the journey is when people start wondering about the why and how of what is happening or they dig deep enough that they may want to know about the foundations of the technology or techniques they are using. At that point, depending on what is being done people will see a massive number of papers published and shared online. The vast majority are available to freely download and read.

Part 1: Highly cited machine learning papers

Within this section I’m going to try to build out a collection of 10 things you could read to start getting a sense of what papers within the machine learning space are highly cited. That is not a measure of readability or how solid of a literature review for machine learning they provide. You will find that most of them do not have really lengthy literature sections. The authors make the citations they need to make for related work and jump into the main subject pretty quickly. I’m guessing that is a key part of why they are highly cited publications. To begin with; from what I can tell, the most highly cited and widely shared paper of all time in the machine learning or deep learning space has over 125,285 citations that Google Scholar is aware of and can index. That is the first paper in the list below.

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778). https://arxiv.org/abs/1512.03385?context=cs

This paper is cited a ton of times and has a pretty solid references section. If you read it after seeing the link above, then you would run into a bit of introduction on deep convolutional neural networks and then it would jump into some related work sections on residual representations, shortcut connections, and finally deep residual learning. While this paper is cited well over one hundred thousand times it is not designed to be an introduction to machine learning. It’s 12 pages and it provides a solid explanation of using deep residual learning for doing image recognition. To that end, this paper is highly on point and easy to read which is probably why so many people have cited it from 2016 to now.

Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255-260. https://www.science.org/doi/abs/10.1126/science.aaa8415

Within the start of this review you are going to get a lot more of an introduction to what machine learning involves and I’m not surprised this work is highly cited.

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015). https://doi.org/10.1038/nature14539

This one is a very readable paper. It was certainly written to be widely read and is very consumable. It has 103 citations as well which is an intense number.

Ioffe, S., & Szegedy, C. (2015, June). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning (pp. 448-456). PMLR. https://arxiv.org/pdf/1502.03167.pdf
Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28. https://proceedings.neurips.cc/paper/2015/file/14bfa6bb14875e45bba028a21ed38046-Paper.pdf
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30. https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. https://arxiv.org/pdf/1409.0473.pdf
Mnih, V., Kavukcuoglu, K., Silver, D. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015). https://doi.org/10.1038/nature14236
Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998, doi: 10.1109/5.726791. http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. https://arxiv.org/pdf/1412.6980.pdf

During part one of this lecture I covered 10 different machine learning papers that are highly cited. My top 10 list might very well not be your top 10 list. If you have a different one, then feel free to share it as well. I'm open to criticism and alternative methods. You can work the paper and reference journey to start to get a solid understanding of machine learning. That is one way to go about getting an introduction to the field. That method involves reading key pieces of literature and as you see footnotes and references that would help fill in your knowledge you take the time to work your way backward from anchor to anchor completing a highly personalized literature review. For academics or people highly focused on a special area within the academic space this is a tried and true method for learning. People are doing it all the time in business and in graduate schools all over the world. Another method exists as well and we will explore that more next.

Part 2: General literature reviews, text books, and manuscripts about machine learning

Sometimes you just want to have all the content packaged up and provided to you as a single serving introduction to machine learning. I’m aware that within this lecture I did not elect to take that single serving path. This field of study is large enough and includes a diverse enough set of knowledge that I think you need to approach it in a variety of different ways based on your specific learning needs. To that end I broke my machine learning literature review into two distinct parts. This second part is about where you could pick up one source and get started, but hopefully it won’t be the final destination in the lifeline learning journey that is understanding the ever changing field of machine learning. For those of you who have been reading this series for sometime you know that my go to introductory text is from the field of artificial intelligence and would be Stuart Russel and Peter Norvig’s classic “Artificial Intelligence: A Modern Approach” which is in its 4th edition based on the Berkeley website [2]. I have the 3rd edition on my bookshelf that I picked up on eBay. The 4th edition has a whole section devoted to machine learning including: learning from examples, probabilistic models, deep learning, and reinforcement learning. That is certainly a popular place to start for people who are starting to dig into machine learning and probably more importantly want a solid foundation in artificial intelligence as well.

You could go with a classic from 1997 and start with a book literally called “Machine Learning” by Tom Mitchell. Recently shared as a PDF by the author.

Mitchell, T. M. (1997). Machine learning. New York: McGraw-hill.
http://www.cs.cmu.edu/~tom/files/MachineLearningTomMitchell.pdf

Maybe you were looking for something a little newer than 1997. You could jump over to the freely available Deep Learning book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville that was published back in 2016.

Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press. https://www.deeplearningbook.org/

A lot more books exist that could help give you an introduction to machine learning, but I’m going to close out with the three that I happen to like the best. That does not mean they are the only way to go about learning machine learning.

Part 3: All the code based introduction to machine learning efforts

I’m going to let my TensorFlow bias run wild here for a moment and say that on my bookshelf right now are a few different works from Valliappa Lakshmanan. Within the TensorFlow community you will find a ton of well written and interesting sets of videos, courses, and other content that will help you dig into the field of machine learning. Outside of the TensorFlow content and the myriad of works by Lak I have a few other books on my bookshelf worth mentioning. I’m not sure why they are all published by O’Reilly, but that appears to be a theme of what made it to my bookshelf in terms of coding books. I know buying and subsequently keeping physical books is something that I do when I’m first learning something. I find it comforting to see them sitting next to me on my bookshelf in my office.

Grus, J. (2019). Data science from scratch: first principles with python. O'Reilly Media. https://www.oreilly.com/library/view/data-science-from/9781492041122/
Hope, T., Resheff, Y. S., & Lieder, I. (2017). Learning tensorflow: A guide to building deep learning systems. O'Reilly Media. https://www.oreilly.com/library/view/learning-tensorflow/9781491978504/
Graesser, L., & Keng, W. L. (2019). Foundations of deep reinforcement learning: theory and practice in Python. Addison-Wesley Professional. https://www.oreilly.com/library/view/foundations-of-deep/9780135172490/

Part 4: Super brief conclusion

Within this brief introduction to machine learning literature review we covered the top 10 articles I think you should start out reading and then we dug into the top 3 textbooks that stood out to me. During the first lecture you might also remember that in terms of forecasting and statistics another book was recommended. It had nothing to do with machine learning, but it's a solid foundational textbook for people interested in understanding the statistics of forecasting.

Armstrong, J. S. (Ed.). (2001). Principles of forecasting: a handbook for researchers and practitioners (Vol. 30). Boston, MA: Kluwer Academic.

Other introduction to statistical methods books exist and one of them might be right for you if you need to brush up on some of the mathematics that you will encounter within the machine learning space. Beyond that, hopefully this lecture has given you a brief introduction to the treasure trove of literature available to give you an introduction to machine learning.

Part 5: Links and thoughts

You can spend hours and just scratch the surface of what they have posted in the Machine Learning Street Talk channel over on YouTube. Generally, I listen to the podcast version of this vs. watching it based on how I tend to consume things. Most of these videos have around 10,000 views (some a bit more and some a bit less) which is reflective of the general sea of humanity that consumes machine learning content.

https://www.youtube.com/c/MachineLearningStreetTalk

“MIT OpenCourseware: Introduction to Computational Thinking and Data Science, Fall 2016, Lecture 11: Introduction to Machine Learning”

“Deep Learning Basics: Introduction and Overview”

“Intro to Machine Learning (ML Zero to Hero - Part 1)”

Top 5 Tweets of the week:

Footnotes:

[1] https://www.ibm.com/cloud/blog/ai-vs-machine-learning-vs-deep-learning-vs-neural-networks

[2] http://aima.cs.berkeley.edu/

Research Note:

What’s next for The Lindahl Letter?

Week 82: ML algorithms (ML syllabus edition 3/8)

Week 83: Machine learning Approaches (ML syllabus edition 4/8)

Week 84: Neural networks (ML syllabus edition 5/8)

Week 85: Neuroscience (ML syllabus edition 6/8)

Week 86: Ethics, fairness, bias, and privacy (ML syllabus edition 7/8)

Week 87: MLOps (ML syllabus edition 8/8)

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

You might remember the Substack post from week 57 titled, “How would I compose an ML syllabus?” We have now reached the point in the program where you are going to receive 8 straight Substack posts that would combine together to compose what I would provide somebody as an introduction to machine learning syllabus. We are going to begin to address the breadth and depth of the field of machine learning. Please do consider that machine learning is widely considered just a small slice of the totality of artificial intelligence research. As a spoken analogy, you could say that machine learning is just one slice of bread in the loaf that is artificial intelligence. I did seriously entertain the idea of organizing the previous 79 posts into a syllabus based format for maximum delivery efficiency. That idea gave way quickly as it would be visually and topically overwhelming and that is the opposite of how this content needs to be presented. Let’s take this in the direction it was originally intended to take. To that end, let’s consider the framework that back during the week 57 writing process I thought was important. My very high level introduction to the creation of a machine learning syllabus from back in week 57 on February 25, 2022, would center on 8 core topics:

Week 80: Bayesian optimization (ML syllabus edition 1/8)

Week 81: A machine learning literature review (ML syllabus edition 2/8)

Week 82: ML algorithms (ML syllabus edition 3/8)

Week 83: Neural networks (ML syllabus edition 4/8)

Week 84: Reinforcement learning (ML syllabus edition 5/8)

Week 85: Graph neural networks (ML syllabus edition 6/8)

Week 86: Neuroscience (ML syllabus edition 7/8)

Week 87: Ethics (fairness, bias, privacy) (ML syllabus edition 8/8)

That is what we are going to cover. At the end of the process, I’ll have a first glance at an introduction to machine learning syllabus. My efforts are annotated and include some narrative compared to a pure outline based syllabus. Bringing content together that is foundational is an important part about building this collection. At this point, just describing the edge of where things are in the field of machine learning would create something that would only be current for a moment and would fade away as the technology frontier curve advances. Instead of going that route it will be better to build a strong foundation for people to consume that will support the groundwork necessary to move from introductory to advanced machine learning. Yes, you might have caught from that last sentence that at some point I’ll need to write the next syllabus as a companion to this one. Stay tuned for a future advanced machine learning syllabus to go along with this introductory to machine learning edition. Enough overview has now occurred. It’s time to get started…

Introduction to ML: Bayesian optimization (Lecture 1 of 8)

I remember digging into Armstrong’s “Principles of forecasting” book which was published back in 2001 [1]. You can get a paper copy or find it online for a lot less than the $429 dollars Springer wants for the eBook. I thought the price was a typo at first, but I don’t think it actually is a typo. It’s just another example of how publishers are confused about how much academic work should cost for students to be able to read. Within that weighty tome of knowledge you can find coverage of the concept of Bayesian pooling which people have used for, “Forecasting analogous time series.” That bit of mathematics is always where my thoughts wander when considering Bayesian optimization. I have spent a lot of time researching machine learning and I really do believe most of the statistical foundations you would need to understand the field could be found in the book, “Principles of forecasting: A handbook for researchers and practitioners.”

I do not think you should pay $429 dollars for it, but it is a wonderful book. Keep in mind that the book does not mention machine learning at all. It is from 2001 and does not really consider how forecasting tools would be extended within the field of machine learning. A lot of machine learning use cases are based on observation and the prediction of things. That is pretty much at the heart of the mathematics of forecasting. You need to understand the foundations of the statistical paradigm that Thomas Bayes introduced a couple hundred years ago in the 1700’s. The outcome of that journey will be the simple aside that we are about to work toward inferring some things. Yes, at this point in the journey we are about to work on inference.

You could move directly to the point and examine Peter Frazier’s 2018 “A Tutorial on Bayesian Optimization” paper [2]. You may want to extend that analysis to figure out all the connected papers [3]. Instead of wandering off into the vast collection of papers that are connected to that one I started to wonder about a very different set of questions. You may have wondered as well if Bayesian optimization is an equation. Within the field of machine learning it is treated more like an algorithm and people typically invoke or call it from previously coded efforts. It does not appear that generally within the field of machine learning people really do the math themselves. You are going to see a whole lot of extending things that are developed as part of a package or framework. Applied Bayesian optimization is going to fall into that format of delivery and application without question.

The rest of this lecture on Bayesian optimization consists of three parts. First, 3 different videos you could watch. Second, 3 papers you could read to really dig into the subject and start to flush out your own research path. Third, an introduction to where you would find this type of effort expressed in code. Between those 3 different areas of consideration you can take your understanding of Bayesian optimization to the next level.

3 solid video explanations:

“Bayesian Optimization - Math and Algorithm Explained”

“Bayesian Optimization (Bayes Opt): Easy explanation of popular hyperparameter tuning method”

“Machine learning - Bayesian optimization and multi-armed bandits”

3 highly cited papers for background:

Pelikan, M., Goldberg, D. E., & Cantú-Paz, E. (1999, July). BOA: The Bayesian optimization algorithm. In Proceedings of the genetic and evolutionary computation conference GECCO-99 (Vol. 1, pp. 525-532).

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.467.8687&rep=rep1&type=pdf

Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & De Freitas, N. (2015). Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1), 148-175.

https://ieeexplore.ieee.org/abstract/document/7352306

Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical Bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25.

https://proceedings.neurips.cc/paper/2012/file/05311655a15b75fab86956663e1819cd-Paper.pdf

Where would you find the code for this?

Tensorflow:

https://blog.tensorflow.org/2020/01/hyperparameter-tuning-with-keras-tuner.html

Keras:

https://github.com/keras-team/keras-tuner

Scikit-learn:

https://scikit-optimize.github.io/stable/

A Google Colab notebook:

https://colab.research.google.com/github/krasserm/bayesian-machine-learning/blob/master/bayesian_optimization.ipynb

The base Github for the above Google Colab notebook:

https://github.com/krasserm/

Closing out this lecture on Bayesian optimization has to end with a general bit of caution about the mathematics of machine learning. A lot of very complex mathematics including statistical devices are available to you within the machine learning space. Working toward a solid general understanding of what the underlying methods (especially the statistical methods) are doing is really important as a foundation for your future work. It is easy to allow the software to pick up the slack and to report outputs. Moving purely toward this type of effort allows the potential for problematic internal breakdowns of the mathematics to occur. You may very well get the outcome you wanted, but it is not explainable or repeatable in any way shape or form. Yes, I’m willing to accept that the majority of people working within the machine learning space could not take a step back and express their work in a pure mathematical way by abstracting away the code to a pure equation based form. That type of pure mathematical explanation by equation is not generally required in papers or read outs. Most of the time it comes down to the simple truth of working in production.

Footnotes:

[1] https://link.springer.com/book/10.1007/978-0-306-47630-3

[2] https://arxiv.org/pdf/1807.02811.pdf

[3] https://www.connectedpapers.com/main/c27078d60737ea10e8ca4f05acd114fef29c8276/graph

What’s next for The Lindahl Letter?

Week 81: Deep learning (ML syllabus edition 2/8)

Week 82: ML algorithms (ML syllabus edition 3/8)

Week 83: Neural networks (ML syllabus edition 4/8)

Week 84: Reinforcement learning (ML syllabus edition 5/8)

Week 85: Graph neural networks (ML syllabus edition 6/8)

Week 86: Neuroscience (ML syllabus edition 7/8)

Week 87: Ethics (fairness, bias, privacy) (ML syllabus edition 8/8)

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Transformers were the thing. They were a big thing in the machine learning field. It was glorious. People talked about them a lot and papers were published. Oh so many papers were published. Now it feels like diffusion might be the thing. You will find that the thing of the moment in the field of machine learning shifts rapidly. I was looking at a GitHub repository based on, “high-Resolution Image Synthesis with Latent Diffusion Models,” and it has over 2,000 stars and has been forked 242 times [1]. I started reading this Tweet from Sebastian Raschka back on January 30, 2022 that asked the question, “Has anyone tried diffusion-based models, yet? Heard that they produce better results than GA” [2]. That Tweet linked out to a paper on ArXiv called, “Diffusion Models Beat GANs on Image Synthesis” [3]. It was published back during May of 2021 by OpenAI researchers and has seen 4 revisions so far. The paper loaded very slowly for me which was surprising. Rarely do I ever watch an update bar slowly creep across the screen waiting for a file to load up. It was 44 pages and 38 megabytes of data. That file should have arrived a lot faster. I took a look at another GitHub repository on guided diffusion from OpenAI that had 1,700 stars [4]. The audience for these diffusion code sets seems to be about 2,000 people which is interesting. Machine learning in general gets roughly 10,000 people focusing on things making this a subset within that slightly larger universe of attention.

Given that we have moved into the 2nd paragraph it might be a good time to talk about what exactly diffusion might be in the context of machine learning. Over in the field of thermodynamics you could study gas molecules. Maybe you want to learn about how those gas molecules would diffuse from a high density to a low density area and you would also want to know how those gas molecules would reverse course. That is the basic theoretical part of the equation you need to absorb at the moment. Within the field of machine learning people have been building models that learn how based on degree of noise to diffuse the data and then reverse that process. That is basically the diffusion process in a nutshell. You can imagine that the cost to do this is computationally expensive. Let’s jump from OpenAI over to the Google AI team who wrote about, “High Fidelity Image Generation Using Diffusion Models” [5]. If you get to read that last link from the Google AI team then you will get to see a bunch of examples of how this works in practice. Imagine a lower quality image that is smaller, being increased in both quality and size. Now you need to imagine that happening again which is what makes the model they are using seem really novel and exciting. I ended up going back to a 2015 paper, “Deep Unsupervised Learning using Nonequilibrium Thermodynamics,” and trying to get a little bit more detail on how the noise process works to create and reverse diffusion [6].

Links and thoughts:

Top 5 Tweets of the week:

Footnotes:

[1] https://github.com/CompVis/latent-diffusion

[2]

[3] https://arxiv.org/abs/2105.05233

[4] https://github.com/openai/guided-diffusion

[5] https://ai.googleblog.com/2021/07/high-fidelity-image-generation-using.html

[6] https://arxiv.org/abs/1503.03585

What’s next for The Lindahl Letter?

Week 80: Bayesian optimization (ML syllabus edition 1/8)

Week 81: Deep learning (ML syllabus edition 2/8)

Week 82: ML algorithms (ML syllabus edition 3/8)

Week 83: Neural networks (ML syllabus edition 4/8)

Week 84: Reinforcement learning (ML syllabus edition 5/8)

Week 85: Graph neural networks (ML syllabus edition 6/8)

Week 86: Neuroscience (ML syllabus edition 7/8)

Week 87: Ethics (fairness, bias, privacy) (ML syllabus edition 8/8)

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

This week based on the backlog, I should be covering the topic of Bayesian optimization. During the course of sitting down to write this week something different happened. Apparently, I was a highly misbehaven backlog prompt this morning. Instead of digging into that topic I’m going to spend some time talking about a more pressing philosophical question related to the future of trust and digital images. This missive is more about expressing and recognizing concern than delivering information. Starting in April of 2022, OpenAi shared the DALL-E 2 model which the researchers noted, “DALL·E 2 is a new AI system that can create realistic images and art from a description in natural language” [1]. Strangely enough the word ethics does not really appear on the homepage for the DALL-E 2 system. It was apparently more important to spend time setting up an Instagram account for art created by the model [2]. You can pretty easily go see how photorealistic some of these images are. With this release from OpenAI, I never requested to join the waitlist to kick the tires on this one. I’m generally more interested in natural language processing than visual image processing.

Let’s set the stage as clearly as possible on this one. People are used to being able to go to the photo finish. Races have been decided by photos and ultimately video for years. We trust video in replay for sports and it remains the visual record of our times. Things changed. Full stop. It used to require a lot of effort to make a deep fake or to alter photographs. It required software and spending time to accomplish that task. Right now a host of new models and other ML implementations are creating the possibility of asking a prompt for an image and within a few seconds getting a reasonable approximation. Some of these images are really high quality. They are photorealistic renderings. You can call up pictures of people who never existed doing things that never happened.

All of this raises really interesting ethical questions about the use and propagation of such ML technology. Keep in mind that these models are just openly being used and served up online. No degree of concern for the potential social impact stopped the distribution and ultimately additional model development. Right now I’m going to say that you cannot trust the future of digital photography. Don’t believe what your eyes report at this point within the digital space. Right now these models very quickly change and make digital images. Soon enough people will develop that technology into a series of images and realistic video will be produced based on a prompt. Essentially they just have to extend the model to the context of a few frames in series and short videos will spring into existence. The evening news could galvanize popular opinion with a story and photograph. At this point, I’m not sure we can trust that type of evidence anymore. Lingering implementations for how civil society is going to change in the face of a zero trust image paradigm. I’m not sure people even understand the alternate realities that could be created and presented as fact. Somebody could bring forward the presentation of a very news forward YouTube channel powered by DALL-E 2 created images. For example, you could introduce a new continent and talk about the discovery of Atlantis and potentially go on for years presenting an alternate reality as truth. Somebody will probably make a living doing that or something equivalent to it. That is where the ethical considerations of this technology and the impacts on society as a whole took a backseat to race to share and demonstrate effectiveness.

Take a moment and consider that just because a technology can do a thing does not mean it should be used to do those things. We make choices. You have to have ethical considerations at the forefront of that type of decision making. We are getting to a point where we have a zero trust image paradigm that will effectively make it a necessity to question everything you see in terms of digital photography and ultimately video. That realization and reality will reverberate across interactions in daily life. At this point, based on the evidence we have, I’m going to declare we have to embrace and ultimately enforce a zero trust image paradigm. How do we even label actual historical documents accurately at this point? Historians will have to be very careful going forward in the analysis of the times about to happen. This may very well be a watershed moment about how we evaluate the truth in front of us and how we verify and validate that narrative. My argument here is not intended to be hyperbolic or presented with any sarcasm whatsoever. A very real situation is developing within our ability to trust the visual world being presented to us. We have to consider the possibilities in front of us and begin to evaluate a path forward. I’m assuming that the path forward is zero trust. That should be clear within the argument being presented. You will have to decide what to do with the world being brought to live by models and systems like DALL-E by OpenAI.

Other prompt based text to image generating models exist as well: DALL-E mini, GLID-3, CLIP, RuDALL.E, and X-LXMERT. My focus here is on the DALL-E model from OpenAI as it has seemed to capture a higher degree of interest from the public mind [3]. You can go run the DALL-E mini model from Hugging Face spaces online for free [4]. However, that site is apparently migrating to craiyon and you find a link to that shared within the footnotes [5]. You can check it out for yourself and see if you share my concern. At the very least, you need to be prepared to openly question any images that are presented go forward. They could very well be synthetically generated.

Links and thoughts:

“Intel Messed Up - WAN Show June 24, 2022”

“Vergecast: M2 MacBook Pro review, Solana’s crypto phone, and this week’s tech news”
https://megaphone.link/VMP9448497842

Top 7 Tweets of the week:

Footnotes:

[1] https://openai.com/dall-e-2/

[2] https://www.instagram.com/openaidalle/

[3] https://techcrunch.com/2022/04/06/openais-new-dall-e-model-draws-anything-but-bigger-better-and-faster-than-before/

[4] https://huggingface.co/spaces/dalle-mini/dalle-mini

[5] https://www.craiyon.com/

What’s next for The Lindahl Letter?

Week 79: Why is diffusion so popular?

Backlog: What is GPT-NeoX-20B? Bonus topic: What is XGBoost?

Week 80: Deep learning Bonus topic: Bayesian optimization

Week 81: Classic ML algorithms

Week 82: Classic neural networks

Week 83: Neuroscience

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

One of the things I have seen trending around the internet places I visit is related to quantum machine learning. I went over to Google Trends and took a look at the last 12 months and could see a decent volume of people generating related queries. If you were to categorize interest by state, then the top 8 would look like: New Hampshire, Washington, California, Pennsylvania, New York, Indiana, Texas, and Florida [1]. I’m a little surprised that my research efforts within the last 12 months did not get Colorado on that list of interest by subregion. Apparently, I’m going to need to step up my quantum computing game.

After getting a sense of where in the United States people are interested in quantum machine learning I started to consider what topics are slightly related. That effort is intended to help me understand the edges of the topic and be able to look 360 degrees around the idea being evaluated today.

The Top 10 related queries would be:

quantum machine learning

open source quantum machine learning

quantum machine learning solutions

quantum machine learning tools

quantum machine learning system

quantum machine learning software

quantum machine learning services

quantum machine learning applications

free quantum machine learning solutions

free quantum machine learning tools

Within the Google trends frameworks they offer another layer of insight beyond the Top 10 related queries. Let’s take a look at that next layer.

The Top 10 rising queries would be:

free cloud based quantum machine learning tools

free cloud based quantum machine learning solutions

free quantum machine learning tools

free quantum machine learning applications

free open source quantum machine learning applications

free quantum machine learning solutions

open source quantum machine learning software

free cloud based quantum machine learning software

free cloud based quantum machine learning services

free open source quantum machine learning tools

To help drill down to some context of how often quantum machine learning really comes up as a topic I ended up comparing machine learning, artificial intelligence, and quantum machine learning [2]. Searches for artificial intelligence are an order of magnitude bigger than searches for quantum machine learning. Overall the searches for machine learning are much larger than quantum machine learning, but still like 20% of the artificial intelligence related volume.

The interest quantified as searches for artificial intelligence have jumped up incredibly high. That is wholesale due to an employee from Google who is an engineer that works with the technology noting publicly a concern that the AI had become sentient [3]. That disclosure has caused a stir online and in the media. I’m not going to cover the nature of what sentient AI describes in this newsletter. It will certainly be a topic that gets covered in more detail after the dust settles on this one. You can check out the first video in the links and thoughts section below for 20 minutes of coverage on this one from the one and only Yannic Kilcher. I’ll share two links to the Medium platform here, but I’m not going to cover them by naming the engineer [4][5]. I think it is entirely possible that the content from those two links will get pulled down at some point in the not so distant future. Unless all of this was just some sort of weird publicity play from Google to raise awareness about LaMDA.

Links and thoughts:

“Did Google's LaMDA chatbot just become sentient?”

From CNET, “Why I’m More Excited About the Next UPS Truck Than the Next Tesla”

“The Download: GitHub Achievements, LTT to the Rescue, and Goodbye to an Old Friend”

Top 6 Tweets of the week:

Footnotes:

[1] https://trends.google.com/trends/explore?geo=US&q=%2Fm%2F012c1btp

[2] https://trends.google.com/trends/explore?geo=US&q=machine%20learning,%2Fm%2F0mkz,%2Fm%2F012c1btp

[3] https://www.npr.org/2022/06/16/1105552435/google-ai-sentient

[4] https://cajundiscordian.medium.com/is-lamda-sentient-an-interview-ea64d916d917

[5] https://cajundiscordian.medium.com/what-is-lamda-and-what-does-it-want-688632134489

What’s next for The Lindahl Letter?

Week 77: Is quantum machine learning gaining momentum?

Week 78: Why is diffusion so popular?

Week 79: What is GPT-NeoX-20B? Bonus topic: What is XGBoost?

Week 80: Deep learning Bonus topic: Bayesian optimization

Week 81: Classic ML algorithms

I’ll try to keep the what’s next list forward looking with at least five weeks of posts in planning or review. If you enjoyed reading this content, then please take a moment and share it with a friend.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Last week some difficult questions were asked about overcrowding within the field of machine learning and the effect of that on engineering colleges [0]. This week things are getting even deeper into the hard philosophical questions we are starting to face. Working to answer the question, “What is post theory science?” will require a lot of consideration and a good bit of digging around. A lot of the ML things that are bubbling up right now in terms of ethical conundrums and rapidly changing ML delivery use cases are just ahead of the next wave of publications and methodology research articles, manuscripts, and textbook updates. This is another topic that I think is going to end up getting a lot of coverage at some point. Just like the potential effects of overcrowding within the machine learning space limiting other research within engineering colleges. The compounding problem from that is that the overcrowded research happens to be very shallow within the ML space resulting in papers without real and lasting contributions to the field.

Let’s focus on the question at hand, “What is post theory science?” This is an interesting scenario to have in existence. Machine learning models can be built to seek out solutions without any theoretical methodology being applied to the potential solution. That type of possibility created a situation where Laura Spinney of The Guardian asked the question, “Are we witnessing the dawn of post-theory science?” [1]. Within that analysis Laura called back to an article from 2008 by Chris Andeson in Wired magazine titled, “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete” [2]. In this scenario without using any theoretical basis to create a testable hypothesis or any classic research method a machine learning use case could derive defendable answers. You could do a similar task with big data or just data in general. Things can be observed and built into a postulate or observation that is not directly based on scientific theory. We can learn things that are seen as being objectively true, but are not derived from theory. In this case that type of effort could very well be called a post theory scientific research methodology. You could build an AI model that aims to understand elements of the universe and it could just be allowed to run and work on that effort. Within that proposition the AI model could test and work with model settings that yield results, but are not based on theory. It could just be using random configurations or working down a path that was independently derived.

A search within Google for “post theory science” does not yield a ton of results. At the time this post was written only about 1,740 results existed within the knowledge graph. I’m truly curious when research method books will contain post theory methods. For post theory science to really pick up steam as a method of research in academic institutions it will need to start showing up in research methods textbooks. I have a couple of them on the bookshelf next to me and while they cover mixed methods and a variety of approaches, nothing within that very large book presumes the idea of post theory science. I’m going to guess at some point that will change here in the next couple of years. We are going to see findings and research coming out of the ML space at record levels and a portion of it will be delivering results, conclusions, and data that are not derived from the traditional scientific method.

Links and thoughts:

I watched this entire video, “This Beat Up, Non-Running Omega Seamaster Has Big Potential! Vintage Watch Restoration”

“What unions could mean for Apple with Zoe Schiffer”

I’m in and out on Lex’s content, but sometimes I just enjoy listening to a good conversation. “Jonathan Haidt: The Case Against Social Media | Lex Fridman Podcast #291”

“The Lab is a Disaster - WAN Show June 3, 2022”

Top 5 Tweets of the week:

Footnotes:

[0]

“Is ML destroying engineering colleges?”

[1] https://www.theguardian.com/technology/2022/jan/09/are-we-witnessing-the-dawn-of-post-theory-science

[2] https://www.wired.com/2008/06/pb-theory/

What’s next for The Lindahl Letter?

Week 77: What is GPT-NeoX-20B?

Week 78: A history of machine learning acquisitions

Week 79: Bayesian optimization

Week 80: Deep learning

Week 81: Classic ML algorithms

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Welcome to a more investigative journalism based issue of The Lindahl Letter this week. This one really made me think. It's a provocative question for sure. Emotional reactions to the premise of destruction aside the question of overcrowding within the field of ML has really caught my attention this week. One piece of prose stood out on this topic and I’m not the only one to react to it. Published back on January 22, 2022 on LinkedIn of all places was a post titled, “How a False Love for AI/ML is Destroying our Engineering Colleges,” [1]. This post seems like an article in waiting or a research note of some kind that was intended to be widely shared by Smruti Sarangi who is the Usha Hasteer Chair Professor at IIT Delhi. It has caused a lot of discussion for sure with over 500 comments and around 10,000 interactions.

Obviously, at this point, I needed to do a bit more research on this one beyond just reading a post shared by a professor on LinkedIn. I ended up looking all over and found a few Tweets that were adjacent to the topic. Yes, I fell deeply into the rabbit hole of reading exchanges on Twitter that were long stale and otherwise left online like the remnants of previous sunrises. Sure they happened and outside of photograph evidence or in this case Tweets everyone has generally moved on to whatever is next on the agenda.

One of them was from Yaroslav Bulatov who is on the PyTorch team over at Meta/Facebook.

That Tweet included some arguments toward trying to illustrate what was wrong with ML research by putting papers with a theorem expressed or mentioned, but no significant improvement demonstrated. That paper referenced was from Schmidt, Schneider, and Hennig back in 2021 called, “Descending through a Crowded Valley – Benchmarking Deep Learning Optimizers,” [2]. It’s about selecting an optimizer in the deep learning space and it is an interesting read. They are essentially trying to empirically test optimizers and share the results. It’s an interesting thing to go out and work on in terms of answering questions about what optimizers to actually use. Within the conclusion the authors shared a truly interesting observation that, "Perhaps the most important takeaway from our study is hidden in plain sight: the field is in danger of being drowned by noise." I don’t normally include quotes in my weekly research notes, but this one just stood out and needed to be read aloud for the true impact to be appreciated. My interest here really is about a deeper question about if overcrowding within the ML space is causing negative externalities.

Generally I avoid doing research by reading Twitter posts, but I thought sharing this thread from Tom Goldstein focusing on a history of AI winters. It's from a National Science Foundation town hall talk, “ML Needs Science,” but distilled on Twitter to be easily consumed.

A counter argument was shared by Melanie Mitchell on Twitter as well. Melanie has also written a paper about AI Winters that is easy to read called, “Why AI is Harder Than We Think” [3].

One of the things in all of that back and forth that caught my attention was a commencement address from 1974 delivered at the California Institute of Technology by Richard Feynman [4].

I could not find enough content to really complete a full post on this topic so not only did I reach out to Gary Marcus on Twitter, but also I sent a note over to Smruti Sarangi on LinkedIn to see if anything additional had been written or published. It turns out that Smruti was just sharing a few thoughts on LinkedIn as a post back on January 22, 2022 and was not preparing any academic papers on the core question of this Substack post. That question was really about, “Is ML destroying engineering colleges?” I ended up wondering about an overcrowding effect within engineering colleges where ML is sucking up all the oxygen and research focus of a generation. That could be reframed into a testable hypothesis by grabbing the last 10 years of publications in some of the top engineering journals to see if publications related to ML are crowding out other academic contributions. I spent a bit of time trying to figure out if somebody had done some academic work on ML creating overcrowding within engineering fields, but I have not found anything directly addressing the elements that caught my attention.

My Twitter interactions this week have been interesting and lively. Normally my Tweets and other research links shared during the weeks don’t get such a high level of interaction. This is a topic that people seem to be passionate about and a lot of different points of view exist on this one. Research is certainly happening and will continue to happen. My honest guess here is that all the efforts to publish within the general scope of ML will reach a peak and people will branch off into other things. My take is that veering off into new areas of academic exploration for most researchers will be a healthy thing to happen for the academy after a bit of intellectual overcrowding occurred.

Footnotes:

[1] https://www.linkedin.com/pulse/how-false-love-aiml-destroying-our-engineering-colleges-sarangi/

[2] https://arxiv.org/pdf/2007.01547.pdf

[3] https://arxiv.org/pdf/2104.12871.pdf

[4] https://calteches.library.caltech.edu/51/2/CargoCult.htm

[5] https://www.barnesandnoble.com/w/surely-youre-joking-mr-feynman-richard-phillips-feynman/1112142471

What’s next for The Lindahl Letter?

Week 76: What is post theory science?

Week 77: What is GPT-NeoX-20B?

Week 78: A history of machine learning acquisitions

Week 79: Bayesian optimization

Week 80: Deep learning

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Thank you for tuning in to this audio only podcast presentation. This is week 74 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “ML content automation.”

At this point, I started to ask myself if I’m the prompt of an exceedingly large language model. Perhaps a better trained and curated model than any foundational model. Prompt engineering is a wonderfully interesting part of ML content automation in terms of generative models. You have to sort of know how to prime the pump or in this case the prompt to get the right content to start to flow from something like GPT-3 or one of the larger foundational models that are starting to be floated around. Previously I have warned about content flooding and the potential for the entire internet to just be astroturfed with nearly endless content if the wrong Web3 comes into being.

Machine learning elements can certainly be used for content generation and extending that functionality to the practice of content automation. You can set up a workflow that just automatically generates content. It could be a bot implementation use case that generates content in response to people. That is an easy method of feeding the model prompts as the only method to interact with a chatbot is to engage in the prompt base activity of sending something from the user to the chatbot. At that point, the prompt has been opened and something is being exchanged to the model which will cause the generation of content. The use case could be extended nearly indefinitely at that point.

Let’s go beyond just a chatbot use case and jump into the complexity of “automated journalism” or more generally the use of machine learning or artificial intelligence to generate articles for the purpose of reporting news. This is where I worry about the use case being expanded from news rooms to a general effort of flooding or astroturfing topics. Bad actors could step in and create such a flood of content that figuring out what was real and what was synthetic could become the greatest challenge facing the internet. Truth could be swept away into a totality of coverage that covers all potential prompt lines given that synthetically generated content may have no association with reality whatsoever. My concern related to this path of course started with the advent of GPT-3 and the potential for the synthetic creation of prose that is believable like the article that was published in The Guardian in 2020 [1]. A lot of the content I run into during the course of reading news during the day could very well be synthetically created from a large language or foundational model. We are seeing things like the Microsoft corporation reducing news staff at MSN to replace them with automation [2].

We are now starting to see models like DALL-E 2 from places like OpenAI that are able to make realistic images from a prompt [3]. That takes a prose based use case for the creation of content to a much wider range of use cases. I’m sure the model will go from images to videos at some point and that type of model would be a legitimate game changer for content creation. Automating the ability for a machine learning model to create video from a prompt would be a huge advancement within the world of content automation. You could open and stage an art gallery with live prompt based installations. I think it would actually be an interesting use case for OpenAI to demonstrate the potential of the model. Bad actors within this space could also decide to create endless images and videos to flood the online world with content dedicated to a specific topic or general theme.

I have spent a lot of time worrying about how to deal with or manage the problematic elements content flooding could create for society in general. The very fabric that binds civil society together might already have seen the breakdown of a curated common thread based on shared experiences. We may have created such a curated content bubble that any shared experience might be limited to commercials and knowledge of products. Being willing to make that assumption might explain a lot of the things happening within society in general at the moment as we face the reality of the intersection of technology and modernity.

Links and thoughts:

The WAN Show was full of wild Linus stories about home automation today. “Story Time! - WAN Show May 27, 2022”

“Stanford Seminar - Leveraging Human Input to Enable Robust AI Systems”

“Ask the Experts: Scaling responsible MLOps with Azure Machine Learning | CATE21”

Top 5 Tweets of the week:

Footnotes:

[1] https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3

[2] https://www.seattletimes.com/business/local-business/microsoft-is-cutting-dozens-of-msn-news-production-workers-and-replacing-them-with-artificial-intelligence/

[3] https://openai.com/dall-e-2/

What’s next for The Lindahl Letter?

Week 75: Is ML destroying engineering colleges?

Week 76: What is post theory science?

Week 77: What is GPT-NeoX-20B?

Week 78: A history of machine learning acquisitions

Week 79: Bayesian optimization

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Within my staging Google document for Substack posts I reached the end of the originally planned out posts for this series of content. Earlier this morning I expanded the staging shell post outlines to week 104 which as you can imagine is a significant point in the publication lifecycle. 2 years of writing Substack posts will be here before you know it. I have enough content in the backlog for this Substack series to get to week 120. At the two year mark I’m planning on moving away from machine learning posts into just generally covering artificial intelligence and producing research notes related to a planned set of academic articles.

That means that it is possible that weeks of ongoing coverage of a topic being worked on as a future academic article could be distributed during year 3 of this Substack series. That is probably a good method to really dig deep into a few topics along the way. One of the things I have worked pretty hard to avoid is producing coverage of the same topic over and over again. One of the things I have noticed in the last few months is that I may have reached conceptual exhaustion within the machine learning topics at around one hundred different concepts. At this point, I should probably go look at all the general conceptual models of the machine learning space and see how close I am to reaching comprehensive coverage. I jumped over into Google Trends and took a look at what topics are bubbling to the surface [0].

That very meta aside about the future of The Lindahl Letter being complete; let’s jump into the topic at hand for today. Most of the time you will see people calling out Symbolic AI vs. symbolic machine learning. If you are interested in trying to build an artificial intelligence system that works similar to the human brain in terms of learning, then you are going to run into the idea of Symbolic AI. Think of things like deep learning, Bayesian networks, or evolutionary algorithms. What I was curious about this week was how many times people try to evaluate symbolic machine learning as a concept. Explicitly searching on Google Scholar for “symbolic machine learning” will yield just over 2,000 results [1]. Some of the academic coverage on this topic goes back to the 1990’s which was obviously where my reading started. Typically I try to rewind back to where articles were sparse and the content was more focused. Recently the volume of content has exploded, but a good portion of it is derivative. I ended up reading an article from Harries and Horn called, “Detecting Concept Drift in Financial Time Series Prediction using Symbolic Machine Learning,” that was published back in 1995 [2]. Sometimes I just enjoy reading about forecasting related concepts as it is grounded in a field of study that has always just made sense to me. Within that space of consideration a copy of Armstrong’s Principles of Forecasting (2001) is sitting on my bookshelf just a couple of feet away. I don’t plan on letting go of that weighty tome any time soon. Oddly enough this article seemed to focus on the potential promise of symbolic machine learning in the future. The phrase only occurs 4 times in the article and that includes the title and abstract. I’m wondering if maybe it was added after the article was written.

I was reading a few academic articles and wondering what exactly people are doing within the practical applications of symbolic machine learning. Google Scholar indicates that 6 related searches stand out. Those searches include remote sensing, algorithms, neural networks, classifiers, reactive control systems, and European settlement maps. Obviously, I was super curious what symbolic machine learning had to do with settlement maps. I found a letter from IEEE Xplore called, “Application of the Symbolic Machine Learning to Copernicus VHR Imagery: The European Settlement Map,” [3]. It’s pretty much exactly what you might think it would be about in terms of a literal mapping of settlements. The scale of the data being processed on this one seems really interesting. This letter did mention symbolic machine learning within the body of the work related to model innovations. It’s a pretty dense publication in terms of concepts being blended together without a lot of explanation. That is probably a byproduct of the authors trying to keep this to 5 pages vs. around 20 pages where that additional commentary would be flushed out.

Links and thoughts:

“Computex 2022 laptops 💻 Elon vs Twitter bots 🤖 Apple ‘testing’ foldable E Ink display 📱”

“See Where Your Electric Car’s Battery Will Go One Day”

“My Investment Pays Off - WAN Show May 20, 2022”

Top 5 Tweets of the week:

Footnotes:

[0] https://trends.google.com/trends/explore?q=machine%20learning

[1] https://scholar.google.com/scholar?q=%22symbolic+machine+learning%22&hl=en&as_sdt=0&as_vis=1&oi=scholart

[2] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.51.5260&rep=rep1&type=pdf

[3] https://ieeexplore.ieee.org/abstract/document/8941071

What’s next for The Lindahl Letter?

Week 74: ML content automation

Week 75: Is ML destroying engineering colleges?

Week 76: What is post theory science?

Week 77: What is GPT-NeoX-20B?

Week 78: A history of machine learning acquisitions

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Security is the element of open source software that has to always be considered. Depending on the size of the associated developer community participating and the rate of development the number of security vulnerabilities is going to rise and fall. It will be a constant battle between those people trying to take advantage of vulnerabilities and the people who fight the good fight of software security. I have taken a serious look at this topic before. Realities of risk associated with software security are a problem for both open source and proprietary software. Arguments have been made that bringing together more and more people who are using a piece of software via the open source model will create a scenario where risk is reduced via transparency and contribution from a multitude of sources.

Back in January Kent Walker who is the President of Global Affairs for Google shared a blog post about, “Making Open Source software safer and more secure” [1]. That missive talked about log4j, a recent open source vulnerability that was impactful to a variety of industries. A reminder was included about a $100 million dollar donation to the Open Source Security Foundation [2]. Kent reduced the question down to figuring out the critical projects instead of trying to boil the ocean, being clear about security testing baselines, and figuring out methods for increased support from both public and private sources.

Let’s zoom out for a second and look at public policy and regulation related to open source software security. Back on May 12, 2021 Executive Order 14028 was issued about “Improving the Nation’s Cybersecurity” [3]. The whole order is 15 pages long and may take you about 20 minutes to read. You can pivot from that to the update from May 11, 2022 to the National Institute of Standards and Technology (NIST) guidance on “Software Security in Supply Chains” [4]. That collection of online pages will take you a lot longer to read. It has a pretty high density of content. A lot of supply chains are now using machine learning and a mix of open source software elements to make things work along the path from production to delivery. As you can imagine a lot of policy makers are legitimately concerned about risks to supply chains.

Now that we have considered concerns related to the developers, companies, and governments looking at open source software security you can see the scope of risk involved. I’m not sure I see any easy solutions on the horizon for this one. It is going to be something that has to be mitigated in real time and a lot of people are going to have to work together to make that happen on an ongoing basis.

Does this week include some bonus edition content? Yes, it does. We are about to cover a bonus topic related to, “Machine learning and surveillance.”

Welcome to the bonus topic this week. My backlog of topics has grown a bit out of control. This is week 72 for example and the backlog has 120 topics. Moving forward I’m going to grab a few of the topics and work on making a few double issues of The Lindahl Letter.

Making sense of and working with mind boggling amounts of data is something that machine learning can help with based on anomaly detection and computer vision elements. You can quickly work through hours of security video footage from cameras at a building and only work with the footage where motion or some type of change occurs. In terms of overnight security and monitoring this means that a large portion of the effort can be almost immediately cleared away. No review is required. You can then move from anomaly detection to the more complicated elements of computer vision to tag elements in the video and flag things for manual review or intervention by alarming or notification. I jumped into a quick Google Scholar search for all of the academic papers that might include or be related to, “computer vision machine learning surveillance” [5]. This is an area where you can find some really solid and well understood use cases.

Back during week 37 coverage one of the links referenced out to the CLIP technology from OpenAI [6]. You can grab an implementation of that from Johan Modin over on GitHub that will help you do contrastive language to image searches [7]. When you see people in movies just searching hours and hours of video for the needle in the haystack and coming back with a quick response of all the examples of “The Man with One Red Shoe” it would be based on a technology like this making that magic happen. If you have not seen the 1985 Tom Hanks comedy thriller by the same name, then you might be missing out on the rich comedic depth of that reference. With the right amount of investment and computing power you can do amazing things in the surveillance space with machine learning. Some of them are shockingly advanced compared to where we were before.

The part of this topic that I really want to cover, but is again a deeper topic for conversation involves the various methods people stitch data together for internet tracking. Some of these tracking methods make the surveillance methods mentioned above seem primitive. I’ll try to figure out a solid way to explain how machine learning is being used within internet tracking frameworks and work that content into a weekly post in the not so distant future.

Links and thoughts:

“[ML News] DeepMind's Flamingo Image-Text model | Locked-Image Tuning | Jurassic X & MRKL”

“UiPath CEO Daniel Dines thinks automation can fight the great resignation”

“Vergecast: Google CEO Sundar Pichai on Google I/O 2022”

“The Download: Markdoc, VS Code Updates, Optimus Prime LEGO and More!”

Top 5 Tweets of the week:

Footnotes:

[1] https://blog.google/technology/safety-security/making-open-source-software-safer-and-more-secure/

[2] https://openssf.org/

[3] https://www.federalregister.gov/documents/2021/05/17/2021-10460/improving-the-nations-cybersecurity or in PDF here https://www.govinfo.gov/content/pkg/FR-2021-05-17/pdf/2021-10460.pdf

[4] https://www.nist.gov/itl/executive-order-14028-improving-nations-cybersecurity/software-security-supply-chains

[5] https://scholar.google.com/scholar?q=computer+vision+machine+learning+surveillance&hl=en&as_sdt=0&as_vis=1&oi=scholart

What’s next for The Lindahl Letter?

Week 73: Symbolic machine learning

Week 74: ML content automation

Week 75: Is ML destroying engineering colleges?

Week 76: What is post theory science?

Week 77: What is GPT-NeoX-20B?

I’ll try to keep the what’s next list for The Lindahl Letter forward looking with at least five weeks of posts in planning or review. If you enjoyed this content, then please take a moment and share it with a friend. Thank you and enjoy the week ahead.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

This week we are going to dig into content about ML newsletters. I’ll do my best to bring that content to life via the spoken word, but you may want to check out the actual written text this week if you are looking for hyperlinks to the actual newsletter locations. Generally, if you want to live dangerously, then you can find these newsletters with a quick Google search or you can visit this post for a more direct approach.

Probably the newsletter that I have subscribed to the longest would be the Inside AI newsletter that has featured weekend commentary from Rob May [1]. Recently the entire Inside newsletter ecosystem has changed it up a little bit with a new platform where you can login and interact with posts. The Inside newsletter ecosystem includes tons of different topics thanks to the founder and CEO Jason Calacanis mixing things up. Way back on July 22, 2018 I did get to write a weekend commentary for that newsletter. I’ll leave that missive below at the end of this post.

A lot of smaller newsletters exist and a few of them are wrapped around brands or larger institutions. Let’s talk about a few Substack newsletters: Last Week in AI which provides a weekly free edition or you can subscribe for more issues [2]. Subscribing to that Substack newsletter will suggest you subscribe to 3 other newsletters. I always think it is interesting to see what other people are suggesting in terms of subscriptions.

The AI Ethics Brief: Democratizing AI ethics literacy [3]
The Gradient: Overviews on cutting-edge AI research and perspectives on the direction of the field [4]
AI Weirdness: The weirdness of artificial intelligence [5]. Interestingly enough when I subscribed to the Substack of this one I got an email from the author advising me the newsletter had moved to Ghost and I should shift my subscription.

I have read a lot of content from The Next Web (TNW) including the Neural newsletter [6]. You can find newsletters from O’Reilly and the MIT Technology review and a slew of other businesses that share AI/ML related content primarily associated with their interest. Over the years I have gotten subscribed to a ton of these and some of them I’m not entirely clear how those subscriptions occurred. A lot of them came about around the time I was working on my AIOps/MLOps research which makes sense.

Here is my weekend commentary post from back in 2018, “Everyday AI: From open source tools with a growing library of free training to being accessible in the business world.”

Sitting down to write a commentary on AI that will match Rob’s high standards made me pause and think deeply. These are amazing times and anybody that is actively seeking to take training on artificial intelligence or use the multitude of tools and languages being deployed within the open source community can open the door to truly interesting possibilities. That is the part of all of this that really caught my attention.

Translating the newly accessible tools and training into action has been the big challenge facing practitioners of artificial intelligence, machine learning, and deep learning. People throughout business are trying to gain insights from data to make decisions. Some of those efforts help build out highly complex dashboards and compendiums of KPIs that help provide insights and drive decisions. Some of the AI use cases are more plumbing related vs. being a scoreboard by replacing a step in a workflow that drives productivity or introducing something new that brings tremendous value to a product. Between informing strategic decisions and driving value, people are hearing about AI and they want to use it in the workplace. They want to use it to be successful. Nobody wants to miss out on this wave of opportunity to describe how things are happening and make predictions about the future.

A new host of tools and techniques related to deep learning exist to really perform complex analysis. The increase in data availability for training models has also risen exponentially. Within our everyday lives models have been deployed to improve the pictures we take and handle other tasks at the edge. Very few of those new tools have started to really be deployed within the day to day workflow of business. That is where I see the greatest opportunity for practitioners. My gut tells me that 2019 will be the year that AI becomes accessible to common workplace reporting, tasking, and IT deployments.

During the demo of Google’s Duplex at Google IO a technology was delivered that will change more than a few games. Both call centers and personal digital assistants come to mind. Both of those advancements will help push engagements with consultant companies and business partners delivering new technology implementations. Just about everybody that has called a major corporation over the years has felt the frustration with poorly setup phone systems. Some of them are interactive and some of them were just a mix of old technologies that made it hard to reach customer service. The technology demoed as part of Google Duplex at IO will provide something different, “...the Assistant can understand complex sentences, fast speech, and long remarks, so it can respond naturally in a phone conversation.”

Part of what makes advances in assistive technology that incorporate artificial intelligence into the solution they deliver so interesting is how accessible they are becoming. A host of free, low cost, or professional training resources are making the technology more accessible within the workplace. Anybody that works on reports or a definable and repeatable tasking could see artificial intelligence, machine learning, or deep learning as a possible door to pushing things forward. Tools like Microsoft Excel that have helped people complete data analysis for years are starting to get backward linkages to very powerful artificial intelligence plumbing. Those are the seeds that are going to grow the use of artificial intelligence from the ground up in business. Beyond major initiatives with consultants and key business partners the building blocks of artificial intelligence techniques will begin to simply be a part of normal business routines.

A growing combination of open source software and free training are helping to grow the number of advocates and enthusiasts in the workplace. That enthusiasm mixed with an increasing level of accessibility are the basis of what I expect to be a banner year in 2019 for the adoption and implementation of complex machine learning and deep learning models in the business world. A lot of that change will be driven from the bottom up based on enthusiasts using technology that has now become accessible.

Links and thoughts:

“How the mechanical keyboard went mainstream again”

“A Billionaire Hedge Fund Manager Predicts the Future — and What He Sees Is Concerning”

“[ML News] Meta's OPT 175B language model | DALL-E Mega is training | TorToiSe TTS fakes my voice”

Top 5 Tweets of the week:

Footnotes:

[1] https://inside.com/ai

[2]

[3]

[4]

[5] https://www.aiweirdness.com/

[6] https://thenextweb.com/neural

What’s next for The Lindahl Letter?

Week 72: Open source machine learning security (Machine learning and surveillance bonus issue)

Week 73: Symbolic machine learning

Week 74: ML content automation

Week 75: Is ML destroying engineering colleges?

Week 76: What is post theory science?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

A lot of enthusiasm and money are flowing into cryptocurrency these days. That wave of 25 billion in venture back financing started back in 2021 [0]. Most of that technology is based on blockchains. The co-founder of Ethereum Gavin Wood as far back as 2014 started talking about a blockchain technology being used to fundamentally change the internet [1]. Gavin is a big proponent of fixing the modern internet and very proud of the idea of coining the term “Web 3.0” certainly before the term became a part of the general lexicon of technology terms. Certainly fixing the modern internet is a noble quest to undertake. At this point, the fundamental technology allowing a free and open internet still exists, but we are seeing islands and speed traps popping up at a much larger pace. It is still an open question about if the metaverse will be pay to play or free and open with financial consequences after you enter the front door.

At this point, it might be a good time to talk about what each of the different versions of the webs really mean and why that is important. You can learn about it from various cryptocurrency forms including Etherum [2]. They are very loud and eager to sell you on a vision of the future. A lot of people are betting venture capital on this version of the future. One of the writers over at Geeks for Geeks broke it down in a pretty decent way back on January 27, 2022 [3].

We need to go back to the 1990’s and consider how Web1 came to be and was used and consumed by a lot of people. People started to stand up servers and have web pages with rather static content. In a lot of ways it was the billboards of the internet built for the online highway. It was amazing and this method of communication was based on servers with content being delivered in a mostly one way direction. Some people called this the mostly “read-only web” [4]. That description was and still remains a pretty solid way of describing Web1.

Fast forward to 2000’s and you start to see Web2 begin to emerge. This is where the “read-only web” shifted to a way more collaborative web. You can think back to examples like Flickr for photo sharing, digg which mostly gave way to Reddit for links and things, Twitter, Facebook, and MySpace. I had to drop a MySpace reference into this post for those of you who miss your first online friend Tom. This is where network platforms started to gain steam and walled ecosystems began to flourish from collecting data and selling advertisements. Sometimes people call this the surveillance web and reference the immense infrastructure deployed to track people online using a combination of cookies and stitching data together. That is a topic that could end up being a future post. Nothing really stops me from weaving in bonus topics for future editions.

While Gavin Wood might be very vocal about coining the term Web3 you can rewind back to San Francisco, California on November 12, 2006 and see that term shared by John Markoff in the New York Times [5]. That article however does not talk about or have any context for blockchain or Bitcoin. Given those considerations I can see how Gavin makes the argument to have coined the version of Web3 that Etherum or other blockchain based technologies would power. Leave it to John Bogna of PC Magazine to dig into “What Is Web3 and How Will it Work?” [6]. It basically comes down to having a decentralized digital infrastructure where your personal information is not the foundation of the interaction. I would describe it as a version of the internet powered in some way by a blockchain based technology. I’m very curious to see an implementation working in practice.

You can go out to the Web3 foundation and learn about what they are doing [7]. On their about page it clearly talks about the Web 3.0 technology stack. Entire communities have developed to track and learn about Web3 jobs including one that tracks the job counts at the top 100 Web3 companies [8]. The top five with several hundred jobs each are Binance, Crypto.com, Coinbase, Ripple, and Consensys. I scrolled past the rest of the list and some of them were names I know something about, but the vast majority of them don’t have any meaning to me. One day they might occupy space in the public mind. It is also possible that most of them will never have the name recognition of Google, Facebook, Microsoft, or Apple.

Before we wrap up this post I will acknowledge that machine learning and artificial intelligence are mentioned all over the Web3 landscape in a variety of ways. However, primarily the technology core of how the stacks are being built seems to be around some type of blockchain based method to be decentralized. Generally speaking both machine learning and artificial intelligence are centralizing technologies. Certainly distributed machine learning model delivery or decentralized artificial intelligence would be interesting topics to consider, but neither of those things are commonly in the lexicon of discussion currently ongoing.

Links and thoughts:

This week I’m sharing a video from Dr. Rachel Thomas who is a researcher in residence at the Queensland AI Hub. The video is titled “You CAN and SHOULD get involved with AI” which I thought was an interesting pitch about getting involved with AI.

Top 5 Tweets of the week:

Footnotes:

[0] https://www.nytimes.com/2021/12/01/business/dealbook/crypto-venture-capital.html

[1] https://gavofyork.medium.com/why-we-need-web-3-0-5da4f2bf95ab

[2] https://ethereum.org/en/developers/docs/web2-vs-web3/

[3] https://www.geeksforgeeks.org/web-1-0-web-2-0-and-web-3-0-with-their-difference/

[4] https://www.web3.lu/divide-the-web-timeline-in-nine-epochs/

[5] https://www.nytimes.com/2006/11/12/business/12web.html

[6] https://www.pcmag.com/how-to/what-is-web3-and-how-will-it-work#:~:text=Proponents%20envision%20Web3%20as%20an,public%20ledger%20of%20the%20blockchain

[7] https://web3.foundation/ or https://web3.foundation/about/

[8] https://web3.career/web3-companies/

What’s next for The Lindahl Letter?

Week 71: What are the best ML newsletters?

Week 72: Open source machine learning security

Week 73: Symbolic machine learning

Week 74: ML content automation

Week 75: Is ML destroying engineering colleges?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Don’t panic! We are going to take this topic in two different directions this week. First, the obvious path will be taken and we will talk about programming, code, and python cookbooks. Second, for those of you that were hoping this week will be a deep dive into artificial intelligence and machine learning derived recipes compiled into a cookbook you will not be disappointed either. Another way to talk about what we are covering today would be that not only are we talking python cookbooks, but also we are covering the epicurean adventures that could be created.

Beginning with the first topic under consideration; coding related answers to the machine learning cookbook question are real. What exactly is a Python Cookbook? If you are a fan of the work by David Beazley and the 3rd edition of the Python Cookbook from the publisher O’Reilly, then you might already have an idea [1]. I personally did not have a copy of this book on my shelf or any of the other python cookbook compilations. These are compilations of code brought together like recipes to help people learn how to use and deploy Python as a computer programming language. You can get to all the code examples that David Beazley has shared on GitHub [2]. For those of you looking to dig into some Python code and learn about how it works, a collection of recipes compiled into a cookbook is a decent place to start. You might get lucky enough to find some recipes on the specific thing you are trying to solve. The other book that stood out to me was the Modern Python Cookbook from Steven F. Lott [3]. The code from that book is also published on GitHub for easy access [4].

At this point we are pivoting to the second question about actual recipes. You might remember back to 2017 when Eric Schmidt shared an AI created cookie recipe built out by engineers at Google as part of a real world challenge [5]. You can find the recipe for the “chocolate chip and cardamom cookie” via the link above. Getting to that point apparently took 59 recipe iterations which is interesting. Oddly enough that article also brought in some of the warning elements that Eric Schmidt has shared about the potential misuse of AI. Cookie related AI examples have improved since 2017 and you could check out the Nestle Toll House cookie expert or alternatively the “cookie coach” AI now to help answer your baking questions [6]. It got announced on Twitter and they had a fun video about working with Ruth the cookie coach.

It does appear that Ruth the Nestle Toll House cookie coach really is still online and working over at https://cookiecoach.tollhouse.com. From Eric Schimdit sharing recipes to interactive chat experiences people have really opened the door to recipe driving artificial intelligence when it comes to cookies. At this point it might be a good idea to move beyond cookies and look at recipes in general. You could go out to the Google Cloud blog and learn about “Baking recipes made by AI” from December 15, 2020 [7]. The embedded video digs into making recipes with ML and is a 7 minute journey of fun featuring Sara Robinson. You may already know that during the pandemic Sara Robinson collected a ton of recipes and used a TensorFlow model to make predictions on future iterations [8]. Between the code shared on the Google Cloud blog and by Sara Robinson you can actually work on a similar cookie effort to make your own machine learning model to work on recipes.

Our friends at Google are not the only ones that have started to wonder about using machine learning for the practical activity of creating recipes. The team over at Towards Data Science have walked through the good and the bad of the process as well within an article titled, “Using machine learning to generate recipes that actually work” [9]. This article really walks you through the process and it is interesting. People really do seem to be having fun with using ML to create recipes. Another way to take a look at the process would be with an article from KDNuggets called, “Generating cooking recipes using TensorFlow and LSTM Recurrent Neural Network: A step-by-step guide” [10]. As you can tell from the number of sources of examples on how to start making your own recipes with machine learning it is a topic that people really seem to be passionate about. Some of that passion is translating further into action. Consider the article, “Forage: Optimizing Food Use With Machine Learning Generated Recipes” [11]. Within that article Angelica, Elbert, and Brian take a real look at reducing potential food waste with machine learning. It is an interesting way to apply machine learning to a practical real world problem.

My efforts to find a real cookbook for sale that is made up exclusively of AI/ML made recipes were not successful this week. You can find websites like https://cookbook.ai/ that help you search recipes and build meal plans, but I did not find a cookbook you could buy from a bookstore.

Links and thoughts:

“Sparse Expert Models (Switch Transformers, GLAM, and more... w/ the Authors)”

Christina Warren this week reminds us that the Download has moved over to https://www.youtube.com/GitHub

“More WordPress Add-on Trouble”

Top 5 Tweets of the week:

Footnotes:

[1] https://www.oreilly.com/library/view/python-cookbook-3rd/9781449357337/

[2] https://github.com/dabeaz

[3] https://www.packtpub.com/product/modern-python-cookbook/9781786469250

[4] https://github.com/packtpublishing/modern-python-cookbook

[5] https://www.cnbc.com/2017/12/05/eric-schmidt-google-used-ai-to-create-the-perfect-cookie-recipe.html

[6] https://www.mashed.com/348141/nestle-toll-house-just-released-a-cookie-expert-ai-for-all-of-your-baking-questions/

[7] https://cloud.google.com/blog/topics/developers-practitioners/baking-recipes-made-ai

[8] https://sararobinson.dev/2020/04/30/baking-machine-learning.html

[9] https://towardsdatascience.com/using-machine-learning-to-generate-recipes-that-actually-works-b2331c85ab72

[10] https://www.kdnuggets.com/2020/07/generating-cooking-recipes-using-tensorflow.html

[11] https://cs229.stanford.edu/proj2017/final-reports/5244233.pdf

What’s next for The Lindahl Letter?

Week 70: ML and Web3 (decentralized internet)

Week 71: What are the best ML newsletters?

Week 72: Open source machine learning security

Week 73: Symbolic machine learning

Week 74: ML content automation

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Both OpenAI and Hugging Face have teams doing great things with respect to machine learning models. Their delivery models are very different. You can visit OpenAI at https://openai.com/api/ and look around at the machine learning models being sold to consumers like IBM, Salesforce, Intel, and Cisco. Delivering machine learning models via an API is one way to go about publishing and sharing your work. Alternatively, people are publishing models like what is happening over at Hugging Face https://huggingface.co/models where if I’m reading the page correctly you can search for various models from a list of 36,028. Both of these organizations are delivering excellent machine learning content to a world of people looking to operationalize machine learning within their corporate strategies. Deciding to publish a model or to sell it via an API is a major decision to make. Selfishly, I much prefer the open source models that I can play with and download.

Before we move on to the rest of this analysis please consider my full disclosure that I participated in the OpenAI private beta for both Codex and GPT-3. Both of those sets of beta analysis provided API access and not full downloads of the models in question. That participation may have given me a good idea of how the system works and let me kick the tires, but it did not cloud my judgment or make me want to give OpenAI favorable treatment. I do agree with the original assessment by the OpenAI team that the GTP-3 and GTP-2 models open the door to misuse [1]. Back in 2019 The Verge team noted that, “OpenAI has published the text-generating AI it said was too dangerous to share,” [2].

We face a very real possibility that the models could be misused to flood our information streams and that it would become almost impossible for communication to function. Some people already believe that bots and other flooding techniques to AstroTurf and falsely drive news cycles are already breaking a problematic news ecosystem. A truly asymmetric delivery problem exists when the amount of content being produced is massively larger than what can be consumed by an individual. Traditional media has transformed from the highly curated view newspapers both national and local mixed with nightly news broadcasts provided to near real time broadcasting. The level of curation within a 24 hour broadcasting channel is even fundamentally different from the single serving real time publishing cycle that happens online. While the topic of information flooding deserves an entire post of consideration especially related to the mechanics of how it works I’m going to move on to the ethics part of the question.

Ethicists have been debating the potential release of dangerous machine learning models for some time [3]. It is a serious debate that needs to be had probably at a governmental and ultimately international consensus level given the potential influences on civil society as a whole from a dangerous intersection of technology and modernity. You can easily provide a model like GPT-3 a prompt for a topic and it will very quickly spit out content. If you elected to do that over and over again for a nefarious purpose, then you could flood comments, posts, news, and other points of information. It is a truly great tragedy of the public information commons that takes the power of sharing information online and tips it to an extreme.

Outside of the ethical considerations of these large language or foundational models. We are probably going to see the heavily used and curated machine learning models deployed via the API method of selling and providing accessibility. Reducing the friction to be able to access and use an API which is generally going to be curated by an organization that is handling all the maintenance and training has a certain value proposition going forward. You almost get to set it and forget about the ongoing cost of training, enhancing, and maintaining the machine learning model. Your machine learning return on investment model may very well allow for some additional cost per transaction within an externally sold API to get the benefits of speed to access and ongoing scalability. That creates an advantage for the biggest companies that can provide proven uptime and reliable service. My attention turned to looking at Google Scholar for “machine learning API marketplace” to see what publications surfaced [4]. A lot of the articles felt like pitches or introductions to specific technology. They were describing parts of the landscape, but were missing the bigger picture of what would happen in the overall marketplace.

Links and thoughts:

I watched Linus and Luke during the WAN show episode from April 8, 2022. They were super excited about launching a screwdriver, backpack, and maybe getting a second giant product testing studio. It's good that Linus is getting into the product testing part of the review space as that is an area where we need more focus and professional attention for the people to consume in general. Our ability to get independent reviews of a high quality seems to be shrinking over the last few years. We have seen a lot more focus on unboxing and reviews from people on YouTube as people are branching out to try to get unbiased and unfiltered insights into products before buying them increasingly online.
This week in ML News Yannic Kilcher talked to people on the street and we learned about Google's 540B PaLM Language Model and the OpenAI DALL-E 2 Text-to-Image model.
On this episode of Decoder Nilay Patel talks to Chris Dixon about a lot of topics related to web3. It was a very interesting hour of discussion about the future of the internet and what will happen online.
Nilay and friends were really excited this week on The Verge podcast where they discussed a couple of topics including Mark Zuckerberg’s big plans for AR glasses, Google’s apparent lack of interest in AR hardware, and Elon Musk’s Twitter drama.

Top 5 Tweets of the week:

Footnotes:

[1] https://www.wired.com/story/dangerous-ai-open-source/

[2] https://www.theverge.com/2019/11/7/20953040/openai-text-generation-ai-gpt-2-full-model-release-1-5b-parameters

[3] https://www.theverge.com/2019/2/21/18234500/ai-ethics-debate-researchers-harmful-programs-openai

[4] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=machine+learning+api+marketplace&oq=machine+learning+API+mark

What’s next for The Lindahl Letter?

Week 69: A machine learning cookbook?

Week 70: ML and Web3 (decentralized internet)

Week 71: What are the best ML newsletters?

Week 72: Open source machine learning security

Week 73: Symbolic machine learning

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Things got shuffled around a little bit and instead of hearing about, “Who still does ML tooling by hand?,” this week my attention shifted to writing down my thoughts on NFTs. This happened in part due to a lengthy conversation I had about the future of NFTs this week. Part of that conversation was distilled into this series of thoughts on the subject. This was almost an interview edition of the podcast. Given the length of my current topic backlog the topic being replaced may not surface again or will have to be incorporated into something else along the way as a sort of bonus topic. I’m on the lookout for the first mainstream use case for NFTs. To qualify that statement I’m going to describe mainstream as millions of users doing something (that something is the basis of what I would consider a use case) on a regular basis with NFTs facilitated by a marketplace.

For those of you who wanted a quick primer on NFTs, then please consider watching the 8 minute video by The Verge team called, “NFTs and the $13B marketplace, explained,” from February 2, 2022 or reading the “NFTs, explained” post they have [1].

Earlier this month Mark Zuckerberg said that NFTs are coming to Instagram [2]. We are already seeing NFTs being added to Twitter profile pictures in a special way [3]. Both of these use cases for NFTs create a use case for the technology or the foundation for where a marketplace could exist with a sizable audience. That is the key element I have been waiting to see within the NFT space. People can 100% get non fungible tokens associated with a blockchain and keep that token in their cryptocurrency wallet. This whole process requires a bunch of things to work which is why I think having some of this background stuff built into an application like Twitter or Instagram will help create a more mainstream experience.

What exactly do you need to make an NFT work?

Blockchain - You need a functioning blockchain that can be accessed to view and confirm ownership of the token. This is a really important thing as if the blockchain ceases to exist or becomes so slow from lack of operational activity your token could become worthless. It is a digital asset and is inherently ephemeral.

Marketplace - The person needs a marketplace or some method of transacting the exchange of an NFT. This is where I think the mainstream social media applications getting involved will make this easier for everybody.

Wallet - Somebody has to have a cryptocurrency wallet to be able to store the information related to the token and other elements for you. This is a collection of keys that are necessary to prove ownership of your token or cryptocurrency.

Minting - Within the process one of the parties has to be able to generate NFts and bring them to a marketplace or distribute them as gifts. I’m assuming that Instagram will give people the ability to make NFTs from pictures they upload.

Right now as it stands somebody who is reasonably tech savvy could go out and buy, own, and trade NFTs right now. I don't believe that all of the blockchains being used for these NFTs will survive. I think a bunch of people are going to be disappointed when this happens. However, a bunch of people will probably have forgotten they have had the NFTs on that blockchain to begin with so they won’t be disappointed. Sure people are working on some interoperability between blockchains so you can move NFTs if you think your current blockchain might be on the path to ruin. All of that complexity will fade away when people are just using the ecosystem related to Instagram or Twitter to make the process frictionless. A process for NFTs that includes millions of people within the marketplace and zero barrier to entry in terms of technology will kickstart an actual economy around the content. It is also a lot less likely that Instagram will fade away. People will keep sharing photos in some form now and in the metaverse.

Soon enough I’ll be focused on building out 8 sections of that machine learning syllabus. My plan for that effort is to really build out and potentially revise that syllabus into something really good. I’m talking about taking it from academic articles all the way to easy to consume lectures from other academics. My goal with that effort is to pull together the very best content from all over the place into a one stop shop for somebody to come up to speed on the subject of machine learning. I’ll put each one of them into a Jupyter notebook format and publish them on GitHub as well to allow people to do push and pull requests against the content. That seems like the most reasonable way to begin the sharing process in an open and earnest way.

Links and thoughts:

You can watch this video about “NFTs and the $13B marketplace, explained” from the team over at The Verge. I enjoyed it and thought it was a good introduction.
This episode of the Decoder podcast really digs into the possibility of what is going to happen with NFTs and crypto. Nilay Patel really gets into the future of things during the podcast, “Steve Aoki on why he’s a ‘crypto believer’” This was one of the most meta conversations I have listened to in some time.

Top 5 Tweets of the week:

Footnotes:

[1] https://www.theverge.com/22310188/nft-explainer-what-is-blockchain-crypto-art-faq

[2] https://www.engadget.com/mark-zuckerberg-confirms-nf-ts-are-coming-to-instagram-204435805.html

[3] https://www.theverge.com/2022/1/20/22893502/nft-twitter-profile-picture-crypto-wallet-opensea-coinbase-right-click

What’s next for The Lindahl Letter?

Week 68: Publishing a model or selling the API?

Week 69: A machine learning cookbook?

Week 70: ML and Web3 (decentralized internet)

Week 71: What are the best ML newsletters?

Week 72: Open source machine learning security

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Let’s start with a brief aside. You may have noticed the podcast audio from last week’s episode was a little different. I accidentally pressed the pattern button on the back of my Blue Yeti X microphone and moved from stereo mode to omni mode. The change did not take away from the listenability of the overall podcast episode, but it does change the temperature or color of the overall audio recording feel. Personally, I strongly prefer the results of the stereo mode that have a slight natural reverb and increased depth. You may recall that I tried the cardioid mode as well earlier as it is recommended by Blue microphones for podcasts, but accidental button presses aside I’m going to stick with the stereo mode for recording going forward. I’ll take a moment now to kindly remember the iconic words of science fiction writer Douglas Adams, “Buttons aren't toys!” That is good advice to remember in the future when moving the Yeti X around my desk.

We can now return to the question at hand related to the title of this essay, “Does a digital divide in machine learning exist?” Yes. A digital divide exists. A world of online content exists, but it has a certain barrier to entry or access. You need a smartphone, computer, or tablet with access to the internet to participate with and access the digital world. It is distinct and separate. To that end a divide exists. It really is a digital divide between technology usage and a normative set of functions distinctly separate. Within the machine learning landscape I would argue that a digital divide exists and we could probably categorize it in multiple ways. First, you have the layer of digital divide that exists between those who have access to technology and elect to use it. Beyond that first layer you have to consider the complexity of machine learning models and the underlying data. As a second layer, you probably have to consider that the digital divide creates an inherent bias where under representation or even complete exclusion exists inside the data being used to train and implement machine learning models. That is probably a structural data inequality that is not easily corrected to remove bias related to a lack of inclusion.

I did read an article titled, “Exploring the Intersection of the Digital Divide and Artificial Intelligence: A Hermeneutic Literature Review,” from 2020 [1]. The paper is free to download and does look at a lot of literature. You can generally jump to “Appendix A: Digital Divide Research” to see some extra content about the digital divide if that is an area you are interested in better understanding. They had a focus on visible and invisible AI that I found interesting. A focus on AI that is visible to a user and elements that would not be visible. A lot of the machine learning models at work today are not visible to the users they are impacting.

The metaverse is not visible to most people. For most of us the metaverse is an example where a very real digital divide will exist within two distinct worlds of interaction. I don’t participate in the metaverse. A more pressing example than the metaverse would be access to care between those that can utilize digital access as a vector and people who are unable to use technology for scheduling. Within that frame of reference, I read an article from Anita Ramsetty and Cristin Adams about the “Impact of the digital divide in the age of COVID-19” from the Journal of the American Medical Informatics Association [2]. It was only 2 pages long, but it was directly looking at the topic I wanted to read about. The authors very carefully argued about how underserved communities could exist from a healthcare perspective based on a digital divide.

Machine learning itself also creates problems based on the efforts required to make it work [3]. A lot of gig workers help train and work with the data necessary to make machine learning work. The article listed above from the MIT Technology Review really was focused on the content the title indicated, “AI needs to face up to its invisible-worker problem.” Within the article a NeurIPS talk was referenced by Saiph Savage [4]. That talk is over an hour long and will make you really think about how the largest datasets got labeled and who did that work. It really is something to consider and understand about how foundations are built within the largest language models.

Links and thoughts:

I listened to a New York Times podcast from Kara Swisher this week called Sway that covered how Elon Musk might shape the future of Twitter. Casey Newton showed up and they dug into the potential changes at Twitter now that Elon Musk is the largest single shareholder.
This week you are getting a second podcast link. This one was titled “Is streaming just becoming cable again? Julia Alexander thinks so” from the Decoder podcast with Nilay Patel. It was an interesting conversation with two people who have obviously spent a lot of time talking.

Top 5 Tweets of the week:

Footnotes:

[1] Carter, L., Liu, D., & Cantrell, C. (2020). Exploring the Intersection of the Digital Divide and Artificial Intelligence: A Hermeneutic Literature Review. AIS Transactions on Human-Computer Interaction, 12(4), 253-275. https://doi.org/10.17705/1thci.00138

[2] Anita Ramsetty, Cristin Adams, Impact of the digital divide in the age of COVID-19, Journal of the American Medical Informatics Association, Volume 27, Issue 7, July 2020, Pages 1147–1148, https://doi.org/10.1093/jamia/ocaa078

[3] https://www.technologyreview.com/2020/12/11/1014081/ai-machine-learning-crowd-gig-worker-problem-amazon-mechanical-turk/

[4] https://nips.cc/virtual/2020/protected/invited_16164.html

What’s next for The Lindahl Letter?

Week 67: My thoughts on NFTs

Week 68: Publishing a model or selling the API?

Week 69: A machine learning cookbook?

Week 70: ML and Web3 (decentralized internet)

Week 71: What are the best ML newsletters?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Ethics should be a part of every machine learning course. It has to be a part of every machine learning journey. Perhaps the best way to sum it up as an imperative would be to say, “Just because you can do a thing does not mean you should.” Machine learning opens the door to some incredibly advanced possibilities for drug discovery, medical image screening, or just spam detection to protect your inbox. The choices people make with machine learning use cases is where the technology and ethics have to be aligned. Full stop. That is the point I’m trying to make today and this essay could stop right here. I’m going to carry on anyway to celebrate the point as I consider it to be vitally important.

No one really solid essay or set of essays on AI/ML ethics jumped out and caught my attention this week during my search. Part of my search involved digging into results from Google Scholar that yielded a ton of different options to read about “ethics in machine learning” [1]. A lot of those articles were about how to introduce ethics to machine learning courses and about the need to consider ethics when building machine learning implementations. Given that those two calls to action are the first things that come up and they are certainly adjacent to the primary machine learning content being shared it might make you take a moment to pause and consider how much the field of machine learning should deeply consider the idea that just because it can do something does not mean you should. Some use cases are pretty basic and the ethics of what is happening is fairly settled. Other use cases walk right up to the edge of what is reasonable in terms of fairness and equity.

An open access article from Nature did catch my attention by Samuele Lo Piano called, “Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward” [2]. That 7 page article has almost 2 pages of references which was pretty intense as citation to content ratios go in published articles. Within my search I was looking for a foundational article or essay that is commonly referenced. I never really did find one. I ended up moving on to an industry driven essay from the team over at Toward Data Science about, “Ethics in machine learning” [3]. That essay did scale back to the basics of the question at hand in terms of how ethical considerations are applied to building machine learning models.

I wanted to refocus my efforts on the macro considerations related to ethics in machine learning at this point. I remembered that Rob May shared a weekend commentary as a part of the Inside AI newsletter recently about the dark side of reducing friction in taking action with advanced technology [4]. Rob even went as far as sharing an article from one of my favorite technology related sources “The Verge” about just how easy and low friction it was to use machine learning to suggest new chemical weapon builds [5]. That is a very real example of where reducing friction to doing a thing opens the door to very problematic actions that illustrate the need for a foundational set of ethics.

If my call to action and introduction of an imperative to the machine learning ethics space were not enough to compel you to ground your efforts, then please consider a hand curated selection of three videos to assist you in your journey. Maybe one of them will catch your attention and help spread the word about ethics in machine learning.

In under 2 minutes Dan Frey who is a professor of mechanical engineering at the Massachusetts Institute of Technology (MIT) introduces this video, “Exploring fairness in machine learning.” This pitch goes back to the Comprehensive Initiative on Technology Evaluation (CITE) from the MIT D-Lab which was launched in 2012 and provides a framework you can use for evaluation [6].
If you were looking for something less tactical and more discussion oriented, then please consider this much longer 90 minute video from the New York University (NYU) Stern School of Business, Fubon Center for AI, Business Speaker Series, titled, “Machine Learning, Ethics, and fairness.” The video is from back on Monday, April 15, 2019 by Dr. Solon Barocas of Cornell University and Professor Foster Provost who is director of the Fubon data analytics and AI intuitive and it really digs into the question of ethics in machine learning.
Finally the third curated selection for you is a shift to a 6 minute video from an industry leader. The IBM Technology and the IBM Cloud group shared a video with Phaedra Biondiris whose title in the video is noted as, “Trustworthy AI Leader: IBM Global Business Services.” This video is more grounded and is probably a good place to wrap up this essay.

Links and thoughts:

https://www.youtube.com/playlist?list=PLZHnYvH1qtOYXzWxVdIU1ZDpbLvxbZdyQ

Top 5 Tweets of the week:

Footnotes:

[1] https://scholar.google.com/scholar?q=ethics+in+machine+learning&hl=en&as_sdt=0&as_vis=1&oi=scholart

[2] Lo Piano, S. Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward. Humanit Soc Sci Commun 7, 9 (2020). https://doi.org/10.1057/s41599-020-0501-9

[3] https://towardsdatascience.com/ethics-in-machine-learning-9fa5b1aadc12

[4] https://inside.com/campaigns/inside-ai-31781/sections/inside-ai-commentary-by-robmay-275419

[5] https://www.theverge.com/2022/3/17/22983197/ai-new-possible-chemical-weapons-generative-models-vx

[6] http://d-lab.mit.edu/research/mit-d-lab-cite

What’s next for The Lindahl Letter?

Week 66: Does a digital divide in machine learning exist?

Week 67: My thoughts on NFTs

Week 68: Publishing a model or selling the API?

Week 69: A machine learning cookbook?

Week 70: ML and Web3 (decentralized internet)

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Maybe revisiting large language models should have been saved for a few weeks from now, but we are going to begin that journey into the foundations of machine learning anyway. My opening question within this chautauqua should be about how large language models in the machine learning space will change society. To that end it might be good to read a post from the Stanford University HAI or Human Centered Artificial Intelligence Institute, “How Large Language Models Will Transform Science, Society, and AI,” by Alex Tamkin and Deep Ganguli [1]. That institute has a mission of, “Advancing AI research, education, policy, and practice to improve the human condition.” While that sounds like an interesting mission statement to attempt to fulfill, it probably ignores the darker possibilities of what could happen. I went out and read the 8 page paper from the post Alex and Deep that they shared, “Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models” [2]. Understanding how large language models could impact the economy and potential misuses are considered in that research which made it a very solid place to start my analysis for this week.

Some really large blocks of content for machine learning exist online. The amount of written work being shared related to machine learning is exponentially growing. It is seriously out of control and beyond what anybody can really reasonably track anymore. One of those blocks of content that caught my attention this week was the ML Compendium by Dr. Ori Cohen [3]. First, this pretty deep work made me wonder about how GitBook works and what other content might be on that platform. Second, it made me wonder about how interactive delivery formats might change the future of textbooks in college settings. A quick search for language models in that collection of links and other content took me to a section on “attention” that included BERT, GPT-2, and GPT-3 [4]. It was not really what I was looking to read about this week and my attention quickly turned elsewhere.

What I was expecting to dig into was the paper on foundational models from a bunch of Stanford University related contributors noted as, “On the Opportunities and Risks of Foundation Models: A new publication from the Center for Research on Foundation Models (CRFM) at Stanford University, with contributions by Shelby Grossman and others from the Stanford Internet Observatory” [5]. You can get the full 212 paper over on ArXiv [6]. By this time in our journey together you have downloaded that paper a couple of times. Yannic theorized that the paper will end up being a key referenced work due to the number of contributors and the volume of things covered. I can see it becoming a part of curriculums for years to come as it has so much reference material in one place and it is free to download.

I’m going to backtrack for a minute here and let you know that after a bit of review it appears that GitBook was designed to provide living documentation [7]. Teams use it to maintain and share technical documentation for software and APIs. It appears that it is also used for some projects like the one shared above. I really do think that type of content curation is probably the future of academic publishing for coursework. Really large static textbooks will be replaced by interactive content that could survive in the metaverse. Students' expectations for the delivery of content to them will fundamentally change in the next 10 years and courses that demand a rigid reading of chapter by chapter in a textbook will fall out of favor.

Links and thoughts:

Top 6 Tweets of the week:

Footnotes:

[1] https://hai.stanford.edu/news/how-large-language-models-will-transform-science-society-and-ai

[2] https://arxiv.org/abs/2102.02503

[3] https://mlcompendium.gitbook.io/machine-and-deep-learning-compendium/

[4] https://mlcompendium.gitbook.io/machine-and-deep-learning-compendium/deep-learning/deep-neural-nets#attention

[5] https://fsi.stanford.edu/publication/opportunities-and-risks-foundation-models

[6] https://arxiv.org/pdf/2108.07258.pdf

[7] https://docs.gitbook.com/

What’s next for The Lindahl Letter?

Week 65: Ethics in machine learning

Week 66: Does a digital divide in machine learning exist?

Week 67: Who still does ML tooling by hand?

Week 68: Publishing a model or selling the API?

Week 69: A machine learning cookbook?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Thank you for tuning in to this audio only podcast presentation. This is week 63 of The Lindahl Letter publication. A new edition arrives every Friday. This week the machine learning or artificial intelligence related topic under consideration is, “Sentiment and consensus analysis.”

My coding experience probably most closely aligns to this topic. Crawlers, bots, and other automation use sentiment analysis. A lot of my original automated coding efforts were related to trying to understand sentiment analysis. A lot of people built web crawling software that ran on some pretty tight schedules to collect a set of target pages. That content was then scraped to find companies trading on public stock exchanges. Those company names and more importantly stock symbols would be evaluated for settiment. Initially that analysis was brute force or you could say explicitly hardcoded values to see how many words near the stock symbol or name were positive or negative. You can get lists of words like a dictionary with sentiment scoring that make that relatively easy to accomplish. You probably can imagine that information was used to try to figure out if a stock was going to go up or down. I personally only mapped out the system for paper trading, but it was a very interesting technology to build.

You can go out to Google Scholar and find a ton of articles to read with a search for, “machine learning sentiment analysis,” [1]. You will see a ton of natural language processing and machine learning topics that intersect with the phrase sentiment analysis. Understanding the sentiment of a block of prose is something that is highly desirable for a variety of reasons. The use cases in some cases are very valuable which makes this particular topic something that a lot of different researchers have worked to understand. That Google Scholar search directed me to an academic publication with 501 citations to initially take a look at for this review. It was another Springer publication where they wanted $39.95 to download that single PDF. Those prices for access to academic works is why pre-prints and other open access journals are so popular. Erik Boiy and Marie-Francine Moens produced a work called, “A machine learning approach to sentiment analysis in multilingual Web texts,” [2]. You could go out to Research Gate and get the publication without the paywall as it was shared by one of the authors [3]. The article is surprisingly easy to read and happens to be very direct in the delivery of content. Back in 2008 and 2009 they spent time talking about manually annotating and working with different techniques. Current machine learning efforts have come a long way thanks to sample training datasets where a lot of this type of mapping or annotation has been done already. To that end you can get a lot of off the shelf machine learning models for this type of sentiment analysis that outperform the 83% accuracy they found for English texts.

After digging into that foundational article in my review I’ll admit my focus shifted to looking at how people use sentiment analysis to mine data from Tweets on Twitter. One of the articles that caught my attention was, “Machine Learning-Based Sentiment Analysis for Twitter Accounts,” from 2018 [4]. That article also very quickly referred to sentiment analysis as opinion mining which seems to be a popular convention at the start of research papers. On the Twitter development webpage you can even find instructions related to, “How to analyze the sentiment of your own Tweets,” that comes complete with code snippets [5]. Other coding examples exist as well that show step by step methods of doing this type of sentiment analysis [6]. The paper referenced above and the step by step guide I shared both use the Tweepy code package to work with the Twitter API. Using that as the starting point for gathering Tweets the next step in the process involves getting ready to do some type of sentiment analysis. That happens to be the part of the equation I generally find interesting. The researchers from the above article basically made a classification model that yielded percent positive and percent negative. The actual methods for determining the positive or negative sentiment are very interesting. That type of explicit encoding hardly requires a machine learning model. You would not have to train the model on anything as every word is already encoded with a value.

You could spend an entire day reading the guides hosted on the Berkeley Library website [7]. You can go find a bunch of different sentiment analysis dictionaries with scoring and other elements to do this type of analysis. It used to be a lot harder to achieve before the scoring was just something you can call or reference. I wonder about how much the scoring shifts over time and based on the collective national mode. Languages are always shifting based on the way we use and apply meaning to words. Something with a relative low score could shift rapidly with a single meme, viral post, or major world news event. Our language and how we share things with people is constantly changing. You can go read a recent article published in March of 2022 from the Nature publication covering how, “Restoring and attributing ancient texts using deep neural networks,” [8]. That article covers the Ithaca model and how it attributes location and otherwise uses a deep neural network for restoration of ancient Greek inscriptions. You can see that our modeling of language has increased a lot between 2008 and 2022. Our ability to glean insights out of sentiment modeling and provide context is potentially increasing daily.

Links and thoughts:

“[ML News] DeepMind controls fusion | Yann LeCun's JEPA architecture | US: AI can't copyright its art”

“Never Hate On Your Community - WAN Show March 4, 2022”

Top 5 Tweets of the week:

Footnotes:

[1] https://scholar.google.com/scholar?q=machine+learning+sentiment+analysis&hl=en&as_sdt=0&as_vis=1&oi=scholart

[2] Boiy, E., Moens, MF. A machine learning approach to sentiment analysis in multilingual Web texts. Inf Retrieval 12, 526–558 (2009). https://doi.org/10.1007/s10791-008-9070-z

[3] https://www.researchgate.net/publication/220479915_A_Machine_Learning_Approach_to_Sentiment_Analysis_in_Multilingual_Web_Texts

[4] Hasan A, Moin S, Karim A, Shamshirband S. Machine Learning-Based Sentiment Analysis for Twitter Accounts. Mathematical and Computational Applications. 2018; 23(1):11. https://doi.org/10.3390/mca23010011

[5] https://developer.twitter.com/en/docs/tutorials/how-to-analyze-the-sentiment-of-your-own-tweets

[6] https://towardsdatascience.com/step-by-step-twitter-sentiment-analysis-in-python-d6f650ade58d

[7] https://guides.lib.berkeley.edu/c.php?g=491766&p=7826401

[8] Assael, Y., Sommerschield, T., Shillingford, B. et al. Restoring and attributing ancient texts using deep neural networks. Nature 603, 280–283 (2022). https://doi.org/10.1038/s41586-022-04448-z

What’s next for The Lindahl Letter?

Week 64: Language models revisited

Week 65: Ethics in machine learning

Week 66: Does a digital divide in machine learning exist?

Week 67: Who still does ML tooling by hand?

Week 68: Publishing a model or selling the API?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

For years it has been a running joke with some of my close friends that I have been perpetually writing a book about the intersection of technology and modernity. To that end my consideration of how we build and what it means for our shared social fabric in terms of civility and technology has been a major part of my creative efforts over the years. Modernity as a word is an attempt to signal that the now or more to the point something very current is being considered. Singularity is a word that has been used in a variety of ways. When you are describing a phenomenon in space the word singularity could be used to describe the formation of a black hole event. Some type of force would compel matter to keep compressing into a single spot until the density of that formation becomes a singularity. In this case the word singularity is being applied to a technological based scenario where in a similar fashion the advance of technology continues to the point where the advance itself cannot be stopped anymore. Within that metaphor technology would have advanced to a point where no matter what action was taken the advance of technology would continue in an uncontrolled way.

A lot of science fiction authors have dealt with the rise and fall of the singularity. You could easily hide under a pile of them and read for years. It is a very interesting subject to take on as it allows for a myriad of very advanced problems to deal with and typically creates all sorts of drama. My title for this particular essay deals with the possibility of touching the singularity. To that end I’m describing a point of time before the intersection of technology and modernity. You can argue with the advance of large language models or foundation models, potentially the advance of technology is starting to become a snowball that will roll down the mountain either slowly or very quickly toward the singularity. I went back to look at the paper published by a ton of researchers at Stanford University, “On the Opportunities and Risks Foundation Models,” only to find that the words “singularity” and “modernity” do not occur [1]. All four of the references to “awareness” were about public understanding and not related to the state of an AI’s understanding of the world. The idea of an artificial consciousness is typically adjacent to arguments related to the formation of a technological singularity.

This problem is a complex one to begin to dig into and it took me about 500 words split between two paragraphs to start to set up the groundwork necessary to begin to question the intersection of technology and modernity. A simpler way to get to this point probably exists, but it will take some more practice on my part to introduce in the future. At this point, I’m very curious and you may be as well about just how close we are to touching the singularity. I looked at a very well referenced paper, “Future Progress in Artificial Intelligence: A Survey of Expert Opinion,” published in 2016 by Vincent C. Müller and Nick Bostrom [2]. That analysis of expert opinions found that there is a 50/50 chance between 2040 and 2050 that a general artificial intelligence or AGI would spring into existence or be created. Arguments can be made and are being made about if the singularity is inherently good or bad for civil society and civility in general. That is not a consideration I’m working with at the moment. My consideration of this is as an event or more to the point right before the event occurs. I did go back and read an article from a 2015 issue of the New Yorker magazine online called, “The Doomsday Invention: Will artificial intelligence bring us utopia or destruction?” [3]. That article is principally about Nick Bostrom and does consider utopia and destruction if you want to go give that a read.

We probably do not have the robotic automation available for an AGI to manifest as both being capable of awareness and constructing things. We are building highly complex 3D printing and other automation methods, but the supply chain for that is pretty complex at the moment. We would probably be able to see the rise of that type of technology before a more gradual intersection of technology and modernity occurred. It would have to get pretty far along to be a true singularity in the sense that it would be something that could continue on as an unstoppable force. Right at that point of moving toward the singularity is where my thoughts refocus on the social impacts to civility, our social fabric, and the normative patterns that make up our shared perspective. As I continue to dig into that point of time and really try to think about the branching of outcomes I do think it will eventually turn into a lengthy manuscript or a full book. Over the last twenty years or so I have not been ready to complete that writing effort. I have written extensively about normative patterns, civil society, and how technology relates to those concepts. I have always just fallen short of really trying to map out the intersection of technology and modernity.

At this point in the journey week 62 is now wrapped up and I’m going to start to produce the audio version of the above text. This week’s narration will include some coverage of the links and thoughts being shared. I have to remember to create an outro style ending for these recordings. The first ones abruptly ended without any sign posting to the listener that the end had occurred. A quick look at the actual post would have been enough for somebody to realize that those podcasts just stopped at the end of the main section prose. That is probably not the best method of gracefully handling the conclusion of a podcast so I have adapted.

Links and thoughts:

This week on The Vergecast Nilay and friends talked about technology. The episode titled, “Steam Deck review / Samsung Galaxy S22 Ultra review,” was pretty decent. It included some discussion of technology embargos, mega gadgets, and the future of technical things.
On the Lex Fridman podcast episode number 267 the guest was Mark Zuckerberg. Generally speaking it is interesting to hear a long form interview with Mark Zuckerberg. I enjoyed a couple of them that were hosted by Casey Newton on Platformer.

or you can watch it here:

You can find that podcast interview with Casey Newton and Mark Zuckerberg here:

https://megaphone.link/VMP6088507415

During the course of walking around Target this week I pushed play on this podcast and had no idea the topic would pull me into totally focusing on what the writers had to say. Go check out, “The Sunday Read: 'The Battle for the World’s Most Powerful Cyberweapon,'” podcast episode of The Daily and you won’t be disappointed if you enjoy thrillers, spycraft, and cyber security.
Nilay talked to the CEO of Sonos about some recent lawsuits. This is an example of the excellent reporting that Nilay had done with the Decoder and Vergecast publications. This is an example of how long format conversation is superior to sound bytes. To that end, most reporters could learn from curiosity based questioning and build up of conversations that occur in this type of forum.
Over the last decade I have listened to more hours of Josh Topolosky, Nilay Patel, Paul Miller, and Dieter Bohn on podcasts than I care to admit. From Endgadet, This is my next, The Vergecast, and Tomorrow I have tuned in on a weekly basis to hear about technology. This week one of the best technology reviewers in the game Dieter Bohn has elected to leave journalism and take a job related to product strategy at Google. People move from politics into journalism and from journalism into politics. I know people move from technology journalism into the business community, but this is a pretty high profile move. Legitimate and independent product testing and the communication about industry trends is a segment of journalism that has both flourished online and diminished within the traditional journalism landscape. That is probably due to a number of facts we cannot blame on Dieter. This little aside is my heartfelt goodbye to an interesting voice in a landscape of technology that will no longer be a part of my weekly listening routine.

Top 5 Tweets of the week:

Footnotes:

[1] https://arxiv.org/pdf/2108.07258.pdf

[2] https://philpapers.org/rec/MLLFPI

[3] https://www.newyorker.com/magazine/2015/11/23/doomsday-invention-artificial-intelligence-nick-bostrom

What’s next for The Lindahl Letter?

Week 63: Sentiment and consensus analysis

Week 64: Language models revisited

Week 65: Ethics in machine learning

Week 66: Does a digital divide in machine learning exist?

Week 67: Who still does ML tooling by hand?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Without question a good number of you reading these words right now have interacted with at least one network platform today. A futurist who was trying to predict the rise or fall of network platforms would probably side with a foreseen expansion and increased clustering of network platforms going forward. Facebook (Meta), Twitter, Slack, and other places where a technology driven platform brings a community together by hosting and maintaining a network are going to end up being categorized as network platforms. Even this very Substack post that you are reading right happens to have been distributed by a network platform of a much smaller scale than the previously mentioned ones.

Adding an element of artificial intelligence as the base of a network platform is an interesting proposition. First you have to assume that an artificial intelligence could be created that is capable of being the foundation of a network platform. Second, artificial intelligence would have to be interesting enough to maintain or sustain the network platform without destroying the hosted community. That very well could be a case of destruction through stability or interaction. You have to remember that an artificial intelligence would be able to create an asymmetric amount of content compared to the user community. Orders of magnitude would be required as a measure to describe the potential flooding that could very well occur without some degree of regulated order and consistency. For the most part all of that could be done without the general artificial intelligence being aware. It could be an AI network platform capable of a multitude of tasking, but not directly aware of what the culmination of that tasking really involves.

Last week while we were digging into what you get with a general artificial intelligence more time should have been spent on the concept of awareness and what that means. A lot of science fiction works have referred to computers, homes, or ships that have built in general artificial intelligence (GAI). That amazingly advanced GAI will handle requests and complete tasks both requested and simply needed. All of that could end up getting transferred to the needs of future AI network platforms, but we are pretty far away from getting to that point based on what we have right now. Voice assistants on phones are just barely branching into completing more than a few tasks. They are making progress for sure and we are getting closer to being able to ask the computer to complete a task with the advent of models like GPT-3 and how it is being extended to create things.

I’m working through the process of incorporating my links and top 5 tweets of the week into the main podcast recording each week. Getting to that point will require providing coverage of the content in a written format within the newsletter that can be read which would facilitate inclusion in the podcast. Early issues of The Lindahl Letter did include more commentary with the links and Tweets, but over time I shifted over to a method of just including 7-10 pieces of content that caught my attention that week and ended up becoming curated content. Providing clear voice overs for why a YouTube video was included will require more effort than the simple act of embedding the link as a curator of content. For this penultimate episode which essentially is an homage to Marvin the android who Douglas Adams brought to life in the Hitchhiker's Guide to the Galaxy, I’m going to give integrating the content a try.

Links and thoughts:

Some of you are aware that I keep recordings of classic political speeches on my phone and listen to and break them apart for fun. That interest in politics has existed for decades. From being captain of a high school debate team to just loving deep political trivia this podcast episode from actor Rob Lowe with White House Press Secretary Jen Psaki was an excellent conversation. Rob did an amazing job of bringing trivia to life during the episode, “Jen Psaki: The Go-To Person.”
This week Linus and Luke are super excited about the launch of the Steam Deck handheld gaming experience. They shared an episode of the WAN show called, “Steam Deck Review: PC Gaming in Your Palm, at Long Last.” Linus indicated this week that drama was off the table, but that never really happens on the WAN show.
You might be wondering why this episode was included based on just seeing the thumbnail alone. AJ and the team over at DFB shared an episode titled, “The Truth About Disney's Star Wars Hotel -- Star Wars: Galactic Starcruiser.” It really is a video with 25 minutes of commentary and thoughts on a really expensive Disney experience. I found it fascinating and included it this week.

In order to facilitate the process of reading the commentary some numbering has been added above to help signpost the change in what link is being discussed. Including this content could end up pushing the recording length from the 5 minute range to somewhere near 10 minutes. Overall the recording and editing experience will be the same and should produce the same type of output each week. It will just produce more of it providing a more complete experience to people who are just listening to the content from just the regular RSS link, Apple Podcasts, Stitcher, or Pocket Casts.

Footnotes:

[1] https://scholar.google.com/scholar?q=artificial+intelligence+network+platforms&hl=en&as_sdt=0&as_vis=1&oi=scholart

What’s next for The Lindahl Letter?

Week 62: Touching the singularity

Week 63: Sentiment and consensus analysis

Week 64: Language models revisited

Week 65: Ethics in machine learning

Week 66: Does a digital divide in machine learning exist?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Thank you for tuning in to this audio only podcast presentation. This is week 60 of The Lindahl Letter publication. A new edition arrives every Friday. This week the machine learning or artificial intelligence related topic under consideration is, “General artificial intelligence.”

We made the ten hour drive from Denver, Colorado to Kansas City, Kansas this week. Last time we made the lengthy drive on I-70 we listened to the audio book recording of, “The Age of AI: And Our Human Future,” by Henry Kissinger, Eric Schmidt, and Daniel Huttenlocher that was published toward the end of last year on November 2, 2021. It was a pretty decent cut at the current state of artificial intelligence and its limitations. I had looked around for another artificial intelligence related audiobook to listen to during this trip. Instead of going with an audiobook this time around, I ended up listening to some of my favorite podcasts during the 10 hours of driving time.

According to a Pew Research Center survey of adults in the United States nearly a quarter of people get their news from podcasts [1]. The other key finding from that survey was that younger adults 18-29 were even more likely to consume news by podcast at 33%. Some of the curated news podcasts are just a higher quality of content than what you would see on cable news. Consolidated and well packaged content comes from solid editing and well thought out efforts. Getting news on demand when you want it is much easier than waiting for cable news to gather enough momentum to get to the point. Based on the survey data a shift is occurring. Podcast subscriber numbers are also indicating that based on audience size the shift may have already occurred.

For those of you who are curious what podcasts made up my 10 hours of traveling time I made a list. Included in that list were The Daily from the New York Times, Start Here from ABC News, The Vergecast, The NPR Politics Podcast, This Week in Google by Leo Laporte, Bourbon Pursuit, and Sway by Kara Fisher. The episode of Sway we listened to was about, “Tech’s Love Affair With Miami,” which was a very interesting look at how the clustering of technology focused people is changing. Kara and Keith talked about the migration from Silicon Valley to Miami. I previously shared a look at where the main AI labs are located back during my week 34 post about, “Where are the main AI Labs?” I had referenced a report that came out from The Brookings Institute by Mark Muro and Sifan Liu titled, “The geography of AI: Which cities will drive the artificial intelligence revolution?” [2]. The locations according to that report, “...include eight large tech hubs—New York; Boston; Seattle; Los Angeles; Washington, D.C.; San Diego; Austin, Texas; and Raleigh, N.C.—and five smaller metro areas that have substantial AI activities relative to their size: Boulder, Colo.; Lincoln, Neb.; Santa Cruz, Calif.; Santa Maria-Santa Barbara, Calif.; and Santa Fe, N.M.” You may have noticed that none of that seemed to include Miami, Florida.

Previously I shared a book called, “Artificial Intelligence: A Modern Approach,” by Stuart Russell and Peter Norvig [3]. This is probably my favorite topic within the artificial intelligence space. It happens to be the topic that powers a ton of science fiction movie and book plots. General artificial intelligence is much more exciting than any special or specific applications of artificial intelligence. The ability to solve a variety of problems and work beyond a single use case is exciting.

I started reading an article from Ragnar Fjelland about, “Why general artificial intelligence will not be realized” [4]. Working through the article was like a refresher on philosophy and the nature of intelligence. It certainly is one perspective on general artificial intelligence. You could go with a more positive perspective from McKinsey and Company called, “An executive primer on artificial general intelligence” [5]. That article starts out by acknowledging that technology is changing, but getting to a true general intelligence is probably pretty far off. However, unlike Ragnar they don’t argue that it is an impossible task. Next I turned to Forbes from 2021 to read a little more about, “The Future Of Artificial General Intelligence” [6]. Some of the coverage is hopeful and some of it theorizes the endeavor is hopeless.

Now at the end of this post is the time to share a short update on my audio recording methods. This week's audio was recorded using the Yeti X professional microphone’s cardioid mode instead of the stereo mode which apparently kicks on automatically. The cardioid mode is geared toward recording sounds directly in front of the microphone which should be better for a podcast. You are welcome to let me know in the comments if you prefer the reverb of the stereo mode of previous episodes or the more targeted audio of the cardioid mode that delivered this recording.

Footnotes:

[0] https://www.pewresearch.org/fact-tank/2022/02/15/nearly-a-quarter-of-americans-get-news-from-podcasts/

[1] https://www.brookings.edu/research/the-geography-of-ai/ make sure to download the report from the link on the left vs. just reading the page

[2] http://aima.cs.berkeley.edu/

[3] https://www.nature.com/articles/s41599-020-0494-4

[4] https://www.mckinsey.com/business-functions/operations/our-insights/an-executive-primer-on-artificial-general-intelligence

[5] https://www.forbes.com/sites/forbestechcouncil/2021/07/16/the-future-of-artificial-general-intelligence/?sh=1c59eaaa3ba9

What’s next for The Lindahl Letter?

Week 61: AI network platforms

Week 62: Touching the singularity

Week 63: Sentiment and consensus analysis

Week 64: Language models revisited

Week 65: Ethics in machine learning

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

You might well be aware that multimodal machine learning (MMML) is a slice of the machine learning universe. Even typing the title of this post was challenging. I really wanted to type multi model vs. modal with an “E” before the last “L” instead of an “A” like the actual wording requires in this case. The definition of multimodal is really direct and does not include a ton of mystery. The word is used to describe something with more than one mode. You might end up quickly running down the path of multimodal deep learning to help describe it. As a person is capable of taking in the world with multiple senses and converting those signal paths into one stream for analysis. A multimodal deep learning network could be built to evaluate multi types of inputs. That really becomes a lot more complicated than it sounds within the modeling space. Our current class of models does not demonstrate the practical skill a person would at differentiating senses and understanding them in real time [1].

Within the machine learning space the first generation of models were really focused on achieving very specific tasks. They received highly defined training on very specific data problems with very curated training datasets. At some point, machine learning models or builds will need to be able to take on more than one type of tasking. I think that is a really fascinating part of machine learning to study. Expanding the capabilities and ultimately what is possible changes future trajectories. I ended up reading a paper titled, “Recent Advances and Trends in Multimodal Deep Learning: A Review,” from 2021 [2]. It was pretty much the exact paper I was looking to read to really dig into the topic under consideration today. That paper really focused on video and language examples which really put things in context.

One of the things that I realized during the course of my research on this topic was that a treasure trove of recorded lectures exist on YouTube. A lot of them are related to computer science, machine learning, and artificial intelligence. That is a really good thing if you were trying to put together a syllabus geared toward providing an introduction to machine learning. The video that I spent the most time watching this time around was from Victoria Dean’s, “MIT 6.S191 Lecture 5 Multimodal Deep Learning,” lecture from 2017 [3]. Somebody who was willing to put in the work to curate the content could easily pull together all the lectures necessary from a multitude of different sources.

After reviewing the current trends in multimodal deep learning my interests shifted to one particular topic related to automated ICD coding. A quick Google Scholar search for "automated icd coding" in quotes and multimodal machine learning produced a ton of interesting results [4]. My search returned 94 results which was pretty surprising given the targeted nature of the terms used. Some of the articles were related to feature extraction and others were really keyed in on trying to get to the point of working with an ICD code or completing the action of coding content. One of the results that dug into automated ICD coding caught my attention was titled, “A Deep Learning Framework for Automated ICD-10 Coding” [5]. Ultimately the automation would help physicians be more productive and accurate in working toward a diagnosis. That seems like a noble effort to assist physicians and help patients.

Links and thoughts:

“MIT 6.S191 Lecture 5 Multimodal Deep Learning”

“Steam Deck: What I Didn't Say In My Review - WAN Show February 25, 2022”

Top 5 Tweets of the week:

Footnotes:

[1] https://towardsdatascience.com/multimodal-deep-learning-ce7d1d994f4

[2] https://arxiv.org/pdf/2105.11087.pdf

[3]

[4] https://scholar.google.com/scholar?hl=en&as_sdt=0,6&as_vis=1&qsp=6&q=%22automated+icd+coding%22+multimodal+machine+learning&qst=br

[5] https://books.google.com/books?id=81A2EAAAQBAJ&lpg=PA347&ots=2k6gK3lL_d&dq=%22automated%20icd%20coding%22%20multimodal%20machine%20learning&lr&pg=PA347#v=onepage&q=%22automated%20icd%20coding%22%20multimodal%20machine%20learning&f=false

What’s next for The Lindahl Letter?

Week 60: General artificial intelligence

Week 61: AI network platforms

Week 62: Touching the singularity

Week 63: Sentiment and consensus analysis

Week 64: Language models revisited

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Let me start with a quick update on something Substack related. Throughout the last month I attended an invitation-only weekly forum hosted by Substack called Substack Go where writers were brought together to help encourage each other. For me writing on Substack is generally a solitary activity that happens mostly on Saturday morning. That effort is followed by a Sunday morning of editing and expansion. Finally the rest of the week is spent tinkering to get to a final post that goes out on Friday. A lot of the content being presented during Substack Go is about how to craft and write a newsletter which is interesting to hear about. This particular series has reached 58 weeks and the weekly rhythm of writing and publishing seems to be working. Even my new podcast variation of this series has been working.

The two topics that I have spent the most time writing about within the machine learning space are strategy (ROI/KPIs/budgeting) and training efforts like bringing somebody or a team up to speed. Training is wholesale an element that should be a part of any organizational ML strategy. This post certainly could start by harkening back to Week 4 that appeared on February 19, 2021. During that week I tackled the topic, “Have an ML strategy… revisited.” Contained within that analysis were two questions targeting “What exactly is an ML strategy?” and “Do you even need an ML strategy?” The answer to that pivotal question is still of course an organization should have a machine learning strategy. Beginning with the end in mind and having that direction tied to budget level KPI is really a minimum standard at this point.

Within this analysis the question really is if teaching or training machine learning skills is the right path to take. Inherent within that analysis has to be a question about the reason for wanting to learn the machine learning skills or acquire that specific knowledge. Bringing team members up to speed is certainly a reason to champion the training process for introducing machine learning skills. On the other side of that consideration is a scenario where learning about the foundations and building up knowledge inspired the need to pick up machine learning skills. In that case the collecting of knowledge simply compels more collecting. Continuing this general line of thought about training; I am going to break up the next two sections into concentrated coverage of individual enrichment followed by coverage of team building.

Individual enrichment as a reason for advancing machine learning skills covers a significant, but not overwhelming number of people that comprise this use case universe. Students and lifelong learners alike see the world of machine learning and either have a fear of missing out or are wondering what the hype is all about. The hype in the marketplace is all about a new technology that can be applied to business use cases. People look at potential and are willing to make a leap to what it could possibly mean for them or what it could possibly do for the organization. Within the machine learning space going from model to production is a journey. Even the process of keeping a model well tuned and running in production is a consideration that means that the journey never really ends.

Team building or more to the point the individual practitioners within teams or solo efforts that advance machine learning within organization certainly comprise the overwhelming bulk of the people contained in this use case universe. People at all sorts of different skill levels are working and learning within the machine learning space. Very few of them have the knowledge, skills, and abilities to create a machine learning system like TensorFlow or PyTorch. The preponderance of work in the machine learning space is done on the shoulders of giants based on using frameworks and systems that were already built. However, we are starting to see a lot of the newest innovations coming out of organizations like Hugging Face and EleutherAI. The developers with the skills to produce those foundational platforms and tooling that people are going to be using in the machine learning space will continue to be a tier above the much larger pool of people using the products. That asymmetrical dynamic is unlikely to change.

To end this post, I thought I would provide you with a short update on my audio editing strategy. Within the Audacity software I’m using three different effects to clean up my podcast audio. First, a noise reduction effect which I have used for a long time in the product is kicked off and executed. The noise gate effect would probably catch it all, but I still run it first. Second, a loudness normalization effect is run which is key for any podcast as it makes sure nothing shocking in terms of a volume spike happens to the listener. Third, I have started using the built in noise gate effect function to trim out background noises and any breaths that might remain. Between those three effects in Audacity my podcast audio is being edited for your listening pleasure.

Links and thoughts:

From Machine Learning Street Talk, “#063 - Prof. YOSHUA BENGIO - GFlowNets, Consciousness & Causality”

“Valve's Making Everyone Else Look Bad - WAN Show February 18, 2022”

“AI Show Live | Ep 52 | Analyze unstructured docs and more with Azure Form Recognizer”

Top 5 Tweets of the week:

Footnotes: None.

What’s next for The Lindahl Letter?

Week 59: Multimodal machine learning revisited

Week 60: General artificial intelligence

Week 61: AI network platforms

Week 62: Touching the singularity

Week 63: Sentiment and consensus analysis

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

This is the third week of including audio integration as a part of the weekly Substack post for The Lindahl Letter. Recording a reading of the first section of the post that contains the main content is easy enough to achieve using Audacity with a basic use of both the noise reduction and loudness normalization effects. Between those two effects it will keep a noise floor in place to keep anything from being too loud during the recording. For those of you who are curious to complete the recordings I’m using a Blue Microphones Yeti X professional microphone on my desk in my office. It has proven to be a little more dynamic than the Samson microphone that normally accompanies my audio recordings.

My very high level introduction to the creation of a machine learning syllabus would center on 8 core topics:

Bayesian optimization

Deep learning

Classic ML algorithms

Classic neural networks

Neuroscience

Reinforcement learning

Graph neural networks

Ethics (fairness, bias, privacy)

I would probably break out the systems and tools part of the conversation to a separate course or keep that content included as supplemental reading. Getting to the point of being able to work toward an 8 topic introduction to machine learning involved really digging into a few other machine learning course introductions. I’m going to share links to six of them with you today, but that should not be considered an exhaustive list. It is just a list of content that caught and held my attention enough that I felt curating it in a list would be helpful.

Stanford CS229: Machine Learning [1]

MIT Opencourseware: Introduction to Machine Learning [2]

NYU Introduction to Machine learning [3]

Tufts CS COMP 134 Intro ML [4]

Berkeley CS 189/289A Introduction to Machine Learning [5]

Washington: CSE/STAT 416, Summer 2020: Introduction to Machine Learning [6]

Taking a step back and thinking about the most direct solution moving forward on this path I would probably just assign a class or at least highly recommend the students to read a book called, “Artificial Intelligence: A Modern Approach,” by Stuart Russell and Peter Norvig [7]. The current version appears to be the 4th edition [8]. My bookshelf has a copy of the 3rd edition and that is the one that I picked up and started reading a few years ago. Oddly enough during the course of reading about the 4th edition the Pearson website suggested that I read, “Quantum Computing Fundamentals,” [9]. That would take this dialogue into another direction. At some point, that pivot might happen for a few weeks. I did check eBay to see if anybody was selling a used copy of that book on quantum computing. Nobody has let go of that tome of quantum goodness just yet. At some point, I’m sure one will show up for resale or my research interests will veer that direction enough to justify the price.

Getting back to the main point of talking about that book by Russell and Norvig. Machine learning elements always feel like a foundational build up to work with more advanced topics in artificial intelligence. Machine learning models that are able to handle a multitude of problems are potentially becoming more likely. Week 79 to 86 of this newsletter will be devoted to creating outlines or a first draft of what the 8 core topics in my proposed syllabus would need to contain. I’m going to try to intermix a lot of artificial intelligence related observations into the machine learning focused content for each of those Substack posts. Bringing the two content sets together in a very intentional way should provide a solid written outcome.

Links and thoughts:

“Newegg... More Like Rotten Egg! - WAN Show February 11, 2022”

“AI Show Live - Episode 51 - NVIDIA DeepStream development with Microsoft Azure”

“HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning (w/ Author)”

“[ML News] Uber: Deep Learning for ETA | MuZero Video Compression | Block-NeRF | EfficientNet-X”

Top 5 Tweets of the week:

Footnotes:

[1] Stanford University. (n.d.). Syllabus and Course Schedule. CS229. Retrieved February 13, 2022, from https://cs229.stanford.edu/syllabus.html

[2] Massachusetts Institute of Technology. (n.d.). Introduction to Machine Learning. MIT OPENCOURSEWARE. Retrieved February 13, 2022, from https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-036-introduction-to-machine-learning-fall-2020/

[3] Mehryar Mohri. (n.d.). Introduction to Machine Learning. Retrieved February 13, 2022, from https://cs.nyu.edu/~mohri/mlu11/

[4] Hughes, M. (n.d.). Introduction to Machine Learning. Syllabus. Retrieved February 13, 2022, from https://www.cs.tufts.edu/comp/135/2020f/index.html

[5] Shewchuk, J. (n.d.). CS 189/289A Introduction to Machine Learning. Retrieved February 13, 2022, from https://people.eecs.berkeley.edu/~jrs/189/

[6] Swamy, V. (n.d.). CSE/STAT 416, Summer 2020: Introduction to Machine Learning. Retrieved February 13, 2022, from https://courses.cs.washington.edu/courses/cse416/20su/

[7] Russell, S., & Norvig, P. (n.d.). Artificial Intelligence: A Modern Approach, 4th US ed. Retrieved February 13, 2022, from http://aima.cs.berkeley.edu/

[8] Link to buy the book: https://www.pearson.com/store/p/artificial-intelligence-a-modern-approach/P100000291856/9780137505135

[9] Link to buy the book: https://www.pearson.com/store/p/quantum-computing-fundamentals/P100002994167/9780137460328#

What’s next for The Lindahl Letter?

Week 58: Teaching or training machine learning skills

Week 59: Multimodal machine learning revisited

Week 60: General artificial intelligence

Week 61: AI network platforms

Week 62: Touching the singularity

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

This topic has been added to my list of academic papers that I need to one day complete. Beyond that scholarly realization the core of this topic really has me conflicted about how to approach consideration about what a national AI strategy really entails. I’m fundamentally questioning what the trajectory of the strategic action would be for the different players involved. Within that conflict I have to wonder if the corporate players within a nation taking action would be inclusive of the strategy or only the official government plan should be included. Those sets of actors do not have to work together. It is an asymmetrical dynamic where one group needs the other a lot more to implement a strategy. To that end I set out to find government documents on the subject. Finding a national AI strategy document in English for a variety of different nations is actually harder to achieve from basic searching than I expected at the start of this journey. It might be an opportunity for an industrious or benevolent researcher to index all of the current plans and build a dataset for comparative analysis.

Getting to the National AI Strategy for the United Kingdom online is really easy [1]. The PDF of that content comes in at a slim 35 pages. It is well written and easy to read. Clearly a 10 year plan exists to make sure Britain is competitive in the AI space. The only really big problem with that is that several of the key corporations in the artificial intelligence space are located several thousand miles away. Getting to the Artificial Intelligence Strategy document from the United States Department of Health and Human Services was fairly easy to achieve. It was published in January of 2021 and is only 8 pages in PDF format [2]. Reading both of those reports was pretty easy and only took a few minutes. From that point forward I started to wonder about the possibility of reading some comparative reports that might have been recently widely published. One of those reports that caught my attention was from the Brookings Institute titled, “Winners and losers in the fulfillment of national artificial intelligence aspirations,” [3]. That document let me know that 44 countries have entered the race by creating a national AI strategic plan. I immediately wanted to get the list and hoped that they shared links to this potential treasure trove. The Brookings Institute had two previous reports one about differing views and the other focusing on the analysis of 34 plans [4][5].

Yes, my primary focus of this inquiry so far was to dig into the plans of the United States and the United Kingdom and then reading three different reports from the Brookings Institute. I have no real preference for their reporting or research, but in this instance it was good quality reading material on the exact topic I wanted to take into consideration. The researchers did take the time to prepare a figure that shows the leaders in national AI strategy implementation which included India, Germany, China, South Korea, United Kingdom, and Canada. The United States actually fell distinctly below the line and was in an entirely different quadrant than the leaders called technology prepared. You could go read the Global AI Vibrancy Tool page from Stanford University to see a different perspective on rankings [6]. After reading all that content and starting to wonder why they did not share the actual dataset with links to all these national AI strategies I may have to reach out to some of the researchers to see if they published it somewhere else that I may have just missed during my exploration.

It seemed like a good idea to go and dig into Google Scholar to see if anything stood out comparing national AI strategies [7]. It does look like some of the comparative studies schools in public policy and political science have done some groundwork. None of it really dug in and provided the higher level statistical analysis with good coding on key metrics I was looking for to better understand which strategy and trajectory would be the most successful or is being primarily implemented. It is entirely possible that the free market will end up defining where AI is going to end up flourishing. Regulation could certainly constrain the free market innovation machine enough to diminish a national AI strategy. The other consideration is that the multinational nature of modern corporations means that even the best national AI strategy could be confounded by organizational reach.

Links and thoughts:

“[ML News] DeepMind AlphaCode | OpenAI math prover | Meta battles harmful content with AI”

“Is AI just statistics? | Yann LeCun and Lex Fridman”

From SpaceX, “Starship Update”

Top 5 Tweets of the week:

Footnotes:

[1] National AI Strategy. United Kingdom. (2021, September 22). Retrieved February 12, 2022, from https://www.gov.uk/government/publications/national-ai-strategy or you can go directly to the document here: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1020402/National_AI_Strategy_-_PDF_version.pdf
[2] U.S. Department of Health and Human Services. (2021, January). Artificial Intelligence (AI) Strategy. Retrieved February 12, 2022, from https://www.hhs.gov/sites/default/files/final-hhs-ai-strategy.pdf
[3] Fatima, S., Dawson, G. S., Desouza, K. C., & Denford, J. S. (2021, October 21). Winners and losers in the fulfillment of national artificial intelligence aspirations. Brookings Institute . Retrieved February 12, 2022, from https://www.brookings.edu/blog/techtank/2021/10/21/winners-and-losers-in-the-fulfilment-of-national-artificial-intelligence-aspirations/
[4] Fatima, S., Desouza, K. C., & Dawson, G. S. (2020, June 17). How different countries view artificial intelligence. Brookings Institute . Retrieved February 12, 2022, from https://www.brookings.edu/research/how-different-countries-view-artificial-intelligence/
[5] Fatima, S., Desouza, K. C., Dawson, G. S., & Denford, J. S. (2021, May 13). Analyzing artificial intelligence plans in 34 countries. Brookings Institute. Retrieved February 12, 2022, from https://www.brookings.edu/blog/techtank/2021/05/13/analyzing-artificial-intelligence-plans-in-34-countries/
[6] Global AI Vibrancy Tool: Who’s leading the global AI race? Stanford University. (n.d.). Retrieved February 12, 2022, from https://aiindex.stanford.edu/vibrancy/
[7] national ai strategy comparison. Google Scholar. (n.d.). Retrieved February 12, 2022, from https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=national+ai+strategy+comparison&btnG=

What’s next for The Lindahl Letter?

Week 57: How would I compose an ML syllabus?

Week 58: Teaching or training machine learning skills

Week 59: Multimodal machine learning revisited

Week 60: General artificial intelligence

Week 61: AI network platforms

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

View Details

Considering this topic really involved taking things in two different directions. First, I started to really try to understand the world and landscape of machine learning patents. Second, it became abundantly clear that acquisitions have fueled a lot of change in the machine learning space. A history of machine learning acquisitions could be a topic one day. That will probably happen in week 78. Some of that acquisition has been about bringing talent to internal teams and some of it has been about patent acquisition [1]. After those two considerations bubbled up to the forefront of my thoughts I also started to wonder about patenting ML models compared to patenting the underlying technology. You can very quickly go out and find out that last year the United States Patent and Trademark Office’s (USPTO) released a new artificial intelligence patent dataset [2].

That gigabyte sized dataset download included 13.2 million patents and pre-grants that you can study on the topic of artificial intelligence [3]. The actual download speed to get that file was exceedingly slow and even with my gigabit internet connection took over 20 minutes. While I was waiting to review the actual data I tried to read the linked journal article about the dataset from Giczy, A.V., Pairolero, N.A. & Toole, A.A. Identifying artificial intelligence (AI) invention: a novel AI patent dataset. The Journal of Technology Transfer (2021) [4]. That article is behind a Springer paywall and costs $39.95 which is a little wild for access to a full PDF of an article. However, it was much easier and surprisingly informative to just watch a narrated PowerPoint presentation from Nicholas A. Pairolero from June 25, 2021 that was presented at the AI and Patents Workshop at ICAIL 2021 [5].

Let’s pivot back to understanding the landscape of machine learning patents. You will very quickly find out that the team over at Google is proudly displaying a patent from December 1, 2005 about machine learning systems and methods [6]. The patent in PDF format is 17 pages long and a pretty easy read. Pivoting from that patent I started to wonder about who is getting patents now and if that really matters anymore in the machine learning space [7]. I learned during that research that it might be fun to work with a website called Patent Guru (patentguru.com) to visualize frequency [8]. I pretty quickly hit the limits of what free access to that website using the analysis features would provide and ended up circling back to the dataset I downloaded. During the course of getting that dataset loaded up I started to consider the idea of patenting AI algorithms and even the future consideration of if an AI built for drug discovery could own the patent or resulting intellectual property from that discovery.

Links and thoughts:

“Identifying Artificial Intelligence Invention: A Novel AI Patent Dataset (AI & Patents 2021)”

Data Science Grandmaster Series from NVIDIA

https://www.youtube.com/playlist?list=PL5B692fm6--uXbxtmPJz5nu3Xmc1JUm3F

“tinyML Talks: Energy-Efficiency and Security for TinyML and EdgeAI: A Cross-Layer Approach”

“GPT-NeoX-20B - Open-Source huge language model by EleutherAI (Interview w/ co-founder Connor Leahy)”

“Getting Started with Sentiment Analysis using Python”

https://huggingface.co/blog/sentiment-analysis-python

“They Almost Got Away With It! - WAN Show February 04, 2022”

Top 5 Tweets of the week:

Footnotes:

[1] Sawers, P. (n.d.). 13 acquisitions highlight Big Tech’s AI talent grab in 2020. Venture Beat. Retrieved February 6, 2022, from https://venturebeat.com/2020/12/25/13-acquisitions-highlight-big-techs-ai-talent-grab-in-2020/

[2] USPTO releases new Artificial Intelligence Patent Dataset. USPTO. (n.d.). Retrieved February 6, 2022, from https://www.uspto.gov/about-us/news-updates/uspto-releases-new-artificial-intelligence-patent-dataset

[3] Artificial Intelligence Patent Dataset. USPTO. (n.d.). Retrieved February 6, 2022, from https://www.uspto.gov/ip-policy/economic-research/research-datasets/artificial-intelligence-patent-dataset

[4] Giczy, A.V., Pairolero, N.A. & Toole, A.A. Identifying artificial intelligence (AI) invention: a novel AI patent dataset. J Technol Transf (2021). https://doi.org/10.1007/s10961-021-09900-2

[5]

[6] https://patents.google.com/patent/US20050267850A1/en

[7] Obee, E. (2021, August 18). Artificial Intelligence Patent 101. Towards Data Science. Retrieved February 6, 2022, from https://towardsdatascience.com/artificial-intelligence-patent-101-3eebf93f5297

[8] "machine learning". PatentGuru. (n.d.). Retrieved February 6, 2022, from https://www.patentguru.com/analysis?area=US&q=%22machine+learning%22

What’s next for The Lindahl Letter?

Week 56: Comparative analysis of national AI strategies

Week 57: How would I compose an ML syllabus?

Week 58: Teaching or training machine learning skills

Week 59: Multimodal machine learning revisited

Week 60: General artificial intelligence

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit nelslindahl.substack.com

The Lindahl Letter: Recent Episodes

Dr. Nels Lindahl