<- home

GitHub
Thumbnail

Non-technical LLM resources

I get asked frequently what kind of resources I'd recommend to people who want to start reasoning about the state and future of AI. The following are what I think are some good resources to get started, from which you can derive more sources depending on your interests and tastes. They are ranked in order of importance (in my opinion) within their category. I will update this list as I find new resources.

Note that these are not technical resources. I personally don't think it is important for everyone to have a deep mechanistic understanding of how LLMs work, but I do think it is important to have a high level understanding of the theory and conversations taking place around the LLMs. This is what this list is for.

TL;DR

If I had one resource to recommend, it would be Don't Worry About the Vase. It is a weekly newsletter that covers the latest in LLM theory and practice. It is written by Zvi Mowshowitz, a rational and talented writer.

Blogs to follow

  1. Don't Worry About the Vase - The comprehensive walkthrough of the weekly events and breakthroughs, mostly LLM-related.
  2. One Useful Thing - Ethan Mollick is a professor at Wharton studying innovation. He translates AI research into actionable advice and provides commentary on recent events and breakthroughs.
  3. Gwern - Gwern is a prolific writer on a wide range of topics, but his AI writing is particularly good. He has a knack for finding the most interesting papers and writing about them in a way that is accessible to non-technical readers.

Important podcast episodes to listen to

  1. Dario Amodei (Anthropic CEO) on Dwarkesh - https://youtu.be/Nlkk3glap_U?si=rfU_ZOeWcIuZw_9a
  2. Max Tegmark on Lex Fridman - https://youtu.be/VcVfceTsD0A?si=4VbNv6o9BzUco5Sg
  3. Sam Altman on The New York Times - https://www.youtube.com/watch?v=wBX4xeefPiA
  4. Neel Nanda on Machine Learning Street Talk - https://youtu.be/_Ygf0GnlwmY?si=L7BM3rNY7b2OfqSs - Neel is a researcher at Anthropic and a leading voice in the field of Mechanistic Interpretability. This episode is a great introduction to the field of MI and the challenges it faces. It may be too niche for some.

Twitter profiles to follow to stay in the loop

  1. Max Tegmark is a professor and AI researcher at Massachusetts Institute of Technology. His insightful updates convey his accomplishments, which in turn shape the AI environment. His research, covering cosmology, physics, and AI, contribute valuable perspectives.
  2. Riley Goodside maybe the worlds first actual Prompt Engineer, he gives updates on the current state of the art in interacting with language models, e.g. new techniques as they emerge and other tidbits.
  3. Geoffrey Hinton, a pioneer in Artificial Intelligence, shares his wisdom in Deep Learning and AI.
  4. David Krueger is a Ph.D. student in deep learning. He shares deep insights into the world of AI and offers updates on current works, which often feature novel approaches to challenges in Machine Learning.
  5. Jeffrey Ladish is a cybersecurity researcher and engineer expert who voices concerns about AI, especially in the context of security.
  6. Connor Leahy contributed majorly to Eleuther, and now runs Conjecture. He is often engaged in discussion around x-risk from AI, e.g. his recent Oxford Union talk.
  7. Zvi Mowshowitz offers his perspective on forecasting, rationality and decision making in the AI domain. His blog is an exhaustive summary of major AI-happenings. By following him, you will get insights on the intersection of technology trends, societal issues, and AI.
  8. Alexander Wang is the co-founder of Scale AI. His Twitter is valuable to follow as he provides updates about his company developments and shares his thoughts on the AI field.
  9. Chris Olah is a Mechanistic Interpretability researcher working at Anthropic. A leading voice in the field of alignment and MI.

Important individual blog posts to read

  1. The Bitter Lesson - Rich Sutton - a seminal piece on the history of AI and always worth a read.
  2. GPT-3 Creative Fiction - Gwern - Although now seemingly a bit dated, the insights that Gwern drew at GPT-3 launch are still pinppoint accurate and worth reading.
  3. The Scaling Hypothesis - Gwern - the most comprehensive early analysis of the scaling hypothesis, as deeply relevant today as it was when it was written.
  4. Simulators - Janus - more LLM specific, this post gives a framework for thinking about how LLMs interface with reality. I think the Simulator ontology gives a solid foundation for further reasoning about LLMs, even if it is a bit esoteric.
  5. A Mechanistiic Interpretability Analysis of Grokking - Neel Nanda - MI is a complex field of study and very new, so it's perhaps quite opaque to a new non-technical user. But I've linked a tweet thread here which gives you a good gist of an import paper. It's a paper that tells us that we can look inside models and figure out what they're doing. We're in the very early days of this, but you'll no doubt hear more about MI in the future. You can dig into the more formal writeups if you want.
  6. Why AI Will Save The World - Marc Andreessen - a techno-optimist, Marc delivers the other perspective on AI being solely a force for good with very little concern for unintended consequences. I don't agree with all of Marc's takes, and some of the voices in the e/acc movement in general are of poor quality, but Marc is a unique orator and I think it's important to give all perspectives a fair hearing.