I like hard sci-fi and read for intriguing technical ideas, world-building, and future forecasting.
— Andrej Karpathy
This list is curated from 439 mentions and sorted by most mentioned, then by date of most recent mention. The more a book is mentioned, the more likely it's recommended and a favorite... or they just like talking about it a lot!
Last updated: .
The Vital Question
by Nick Lane
Easily one of my favorite books ever - Nick Lane is an excellent scientist author, mixing engaging presentation, highly intriguing ideas, and tons of technical details. I learned a lot about origins of life and unlocked a whole new level of appreciation of the fantastic cosmic story every one of us is a part of, with all of the happy accidents along the way. 5/5
— Andrej Karpathy
His Master's Voice
by Stanisław Lem
On the topic of sci-fi’s I really like books written by scientists turned writers because I find the world building to be much more compelling, interesting and logically consistent. Recently I enjoyed [this].
— Andrej Karpathy
Lord of the Rings (3 books)
by J. R. R. Tolkien
Nice! I really want to build a reading companion app for books. E.g. I am re-reading LoTR again, you could imagine stuffing all of it (and discussion boards related commentary and chatter) into context and making it very easy to ask questions, clarifications, discussions. There's…
— Andrej Karpathy
Contact
by Carl Sagan
Did the creator of the universe give us a message? For example in the book "Contact", Carl Sagan, there's a message for any civilization in digits in the expansion of Pi and base 11, eventually, which is kind of an interesting thought.
— Andrej Karpathy
A Fire Upon The Deep
by Vernor Vinge
Only chapter 1, describing a flowering Superintelligence, really awesome read. Later chapters went downhill.
— Andrej Karpathy
Life Ascending
by Nick Lane
I used to think that the origin of life was this magical rare event. But then you read books like, for example, Nick Lane, "The Vital Question", "Life Ascending", et cetera. And he really gets in, he really makes you believe that this is not that rare. [...] I do feel like the story in these books, like Nick Lane's books and so on, sort of makes sense and it makes sense how life arose on earth uniquely. And yeah, I don't need to reach for more exotic explanations right now.
— Andrej Karpathy
Deep Learning
by Ian Goodfellow, Yoshua Bengio, Aaron Courville
There were very few books to draw on during my PhD for DL. Now there’s Ian Goodfellow et al. Deep Learning book and other resources (e.g. many talks, CS231n, etc).
— Andrej Karpathy
Reinforcement Learning
by Richard Sutton
I also liked Sutton’s Reinforcement Learning book, which I methodologically read cover to cover over few weeks and reimplemented a lot of in ReinforceJS.
— Andrej Karpathy
Profiles of the Future
by Arthur C. Clarke
First I would highly recommend Arthur C. Clarke’s “Profiles of the Future” (see my review on Goodreads). It’s a wonderful study of the science and art of predicting the future.
— Andrej Karpathy
Fiasco
by Stanisław Lem
On the topic of sci-fi’s I really like books written by scientists turned writers because I find the world building to be much more compelling, interesting and logically consistent. Recently I enjoyed [this].
— Andrej Karpathy
Ready Player One
by Ernest Cline
Ready Player One, which paints a likely future where large portion of the population spends time in VR in a Second Life - like environment. This is another example of sci-fi that has repercussions for AI research. Suppose this were true, how amazing would that source of data be, of millions of people interacting in real time in virtual worlds, etc. What does it enable? What kinds of techniques would flourish?
— Andrej Karpathy
The Black Cloud
by Fred Hoyle
On the topic of sci-fi’s I really like books written by scientists turned writers because I find the world building to be much more compelling, interesting and logically consistent. Recently I enjoyed [this].
— Andrej Karpathy
Foundation (7 books)
by Isaac Asimov
On the topic of sci-fi’s I really like books written by scientists turned writers because I find the world building to be much more compelling, interesting and logically consistent. Recently I enjoyed [this]. Especially the concepts of psychohistory and dark ages.
— Andrej Karpathy
Rendezvous with Rama
by Arthur C. Clarke
quite enjoyed, thanks (again) for another great recommendation!
— Andrej Karpathy
The Martian
by Andy Weir
I rarely give out 5/5, but this book was SO. GOOD. My usual complaints about many sci-fi books is that they spend a lot of time frolicking around with extended descriptions of vistas or facial features, or other basic literature mambo jambo. Instead, I am drawn to technical consistency, details and intriguing ideas. If you're like me, you will LOVE this book - it gets very nerdy very fast and stays that way for the entire duration of the book. The book offers a thrilling ride filled with science, calculations, and humor mixed in. There are many references to technical details spanning chemistry, biophysics, mechanical engineering, orbital mechanics, etc. The result is a believable and consistent backdrop that envelops the story. I had a lot of fun, found plenty food for thought, and I learned a lot! What else can you ask for? 5/5. 5/5
— Andrej Karpathy
The Hobbit
by J.R.R. Tolkien
A delightful adventure story full of charming characters and unexpected twists and turns, The Hobbit is a classic in its own right. Building on the rich world of Middle-earth that Tolkien created, this book is a must-read for anyone who loves fantasy or adventure stories.
— Andrej Karpathy
Programming Massively Parallel Processors
by David B. Kirk, Wen-mei W. Hwu
I read this book and then I was surprised that I still understood so little of the kernels that started to appear as llm.c contributions, beating mine. It's a pretty good 101 intro. Learning CUDA is like that horse meme, all the learning resources you can find on the left, then… pic.twitter.com/C0k1WZqkQM
— Andrej Karpathy
The Making of the Atomic Bomb
by Richard Rhodes
> You could also read Richard Rhodes "The Making Of The Atomic Bomb" Already did, great book
— Andrej Karpathy
The Selfish Gene
by Richard Dawkins
I was very impacted by "The Selfish Gene". I thought that was a really good book, it helped me understand altruism as an example and where it comes from. And just realizing that the selection and the levels of genes was a huge insight for me at the time and it cleared up a lot of things for me.
— Andrej Karpathy
Molecular Biology of the Cell
by Bruce Alberts
I like to reach for textbooks sometimes. I feel like books are for too much of a general consumption sometimes and they're too high up in the level of abstraction and it's not good enough. So I like textbooks, I like "The Cell". I think "The Cell" was pretty cool. [...] And then I'm also suspicious of textbooks honestly because as an example in deep-learning there's no amazing textbooks and the field is changing very quickly. I imagine the same is true in say synthetic biology and so on, these books like "The Cell" are kind of outdated. They're still high-level, like what is the actual real source of truth? It's people in wet labs working with cells. Sequencing genomes and, yeah, actually working with it. And I don't have that much exposure to that or what that looks like. So I still don't fully, I'm reading through the cell and it's kind of interesting, and I'm learning but it's still not sufficient I would say in terms of understanding.
— Andrej Karpathy
We Are Legion (We Are Bob)
by Dennis E. Taylor
It's a bit like The Martian with its upbeat, lighthearted, comic-relief tone, but without any of the intriguing hard sci-fi components. Fun and interesting in the first 25% but dramatically downhill from there. Naive ideas about alien life and civilizations. Doesn't really have an ending. 2/5
— Andrej Karpathy
Solaris
by Stanisław Lem
Honestly, I love Lem's ideas and treatment of aliens and this book definitely does not disappoint on that front, but everything else outside of that was just not great, making the book very difficult and tedious to finish. The characters are drawing out every single dialog, everything is always kept so very mysterious, everyone is always confused or unwilling to communicate properly, no-one acts like a good scientist, and the technological capability of the ocean is dubious. The story and its events just did not really add up. 3/5
— Andrej Karpathy
The Andromeda Strain
by Michael Crichton
My favorite part about this book is that it is a bio-heavy hard sci-fi from an era that was otherwise decidedly all about space. An alien microscopic organism makes first contact with humans - super cool concept! I also very much appreciated the writing style, which spares the reader of the typical English major literally minutiae of the color, contours or feel of every single person, scene or thing, and spends its effort on the story, ideas and world-building. Subtract a star because I still feel like there is plenty of missed opportunity in this book around the fascinating concept, and the ending is oddly rushed. I also didn't really understand some parts that felt a bit non-sensical, e.g. around the evidence and presence of conversion between mass and energy. Ah well still worth a read! 4/5
— Andrej Karpathy
Childhood's End
by Arthur C. Clarke
I'm sorry but I am unable to accept or tolerate tales that feature biological humanoid aliens who speak English and have faces and etc. I cannot concentrate on anything else in the plot, it is drowned out by the persistent screech of rage in the brain. A whole cast of characters come and go and become relevant and then irrelevant. If you like The Independence Day but wish the aliens were friendly little best buddies forever you will like this book. Sorry I'm being too mean but it just didn't resonate. 2/5
— Andrej Karpathy
Understand
by Ted Chiang
It's beautiful and the closest I've read to what it may think like to be a superintelligence.
— Andrej Karpathy
Nexus
by Ramez Naam
Nexus is a (programmable) operating system layer over the brain that allows people to program their minds (e.g., download a "Bruce Lee" package), and communicate directly with the minds of others. The book's plot involves a protagonist scientist who wants to release the technology for good, and a government organization who wants to stop it (or at least massively slow it down) in its tracks for fear of unintended consequences. I quite enjoyed the world-building pieces of this book. The Nexus operating system is interesting and is described in quite a lot of technical detail. More generally, the world features a large number of human body/mind augmentations that can be purchased. We also get a glimpse of some post-humans and we're teased with ideas of human hive minds. Unfortunately, after the awesome world-building is over in the first ~third of the book, the plot mostly transitions into what feels like a long chase sequence / thriller, and loses some of its grandeur. 5/5
— Andrej Karpathy
Project Hail Mary
by Andy Weir
Nerdy, quirky and fun! The characteristic Andy Weir - style enthusiasm and glee over science permeates the plot. The challenges of the plot are met by one of the most unique and entertaining partner duos I've seen. The book doesn't shy away from a technically elaborate, interesting and engaging portrayal of an alien species. 4/5
— Andrej Karpathy
Surely You're Joking, Mr. Feynman!
by Richard P. Feynman
Surely You're Joking is one of my all time favorite books, for sure.
— Andrej Karpathy
Children of Time
by Adrian Tchaikovsky
Interesting premise on a very high level - follows an alien civilization "booting up" from scratch, intertwined with the shenanigans of a rebooted spacefaring human civilization. Enjoyed the idea of the "classicist". Severe lack of technical bits and pieces, making it overall a quite "soft" sci-fi. A little too long, could probably be compressed by 5X of more. The attempt at world building is valiant, but ultimately quite shallow and not very believable. 2/5
— Andrej Karpathy
The End is Always Near
by Dan Carlin
I understand that some of the book's content has appeared in Dan's Hardcore History podcasts, but since I've only listened to a sparse few a lot of the book was relatively new material to me. I thought the premise of the book was excellent: Things look quite good right now and it's hard to imagine civilization regressing substantially, but history is filled with examples of exactly that over and over again. Just how optimistic should we be today that we can avert the same fate? I expect that Dan could write an excellent book laser focused on exploring this, but while the book does do a bit of it now and then, more often than not it also distracts itself and goes off on tangents of what feels like filler / irrelevant content. For example, we're discussing the Assyrian empire, the Roman empire and their fall, but then we also randomly learn a little too much of the treatment of children in history, or the details of various bombings during the second world war. What is the point of these? I would have preferred if the book stuck more closely to its core theme, with multiple examples of powerful empires rising and falling unexpectedly, with an analysis of what made that happen, and whether that analysis applies to today. This is something we half get, which is still fun. Enjoyed overall! 4/5
— Andrej Karpathy
Old Man's War
by John Scalzi
This is a humorous medium-ambitious space opera sci-fi. As far as ideas and world building go, I enjoyed some of the concepts (e.g. "smart blood"), but found others highly naive / dubious (silly biological aliens, etc). The story is enjoyable, but doesn't try too hard to wow. Overall, a satisfying bite of a story-driven sci-fi if you can forgive the unrealistic universe. 3/5
— Andrej Karpathy
Dragon's Egg
by Robert L. Forward
This book must absolutely be commended for its inventiveness, while staying within the limits of the scientifically plausible. Overall a very enjoyable hard scifi read, but if I had to critique some things, it would be: 1) the aliens are slightly annoyingly too human-like (would have appreciated an attempt at something more perplexing / foreign), 2) some parts of the book drag on for a very long time without being interesting (e.g. the various escapades of the cheela that take up a large portion of the book), 3) the cheela civilization is not imagined in a satisfying detail, and 4) the ending is a little too abrupt and naive for my tastes. A little bit like Sagan's Contact, where I would prefer a bit more of the more likely Lem's Master's Voice. overall a recommended read for anyone who loves hard scifi! Just feel free to skim some of the boring parts until you get to the last ~20% of the book, and prepare to have your intelligence insulted just a little bit when it comes to antrophology instead of the physics. 4/5
— Andrej Karpathy
Ender’s Game
by Orson Scott Card
It's alright. It's a bit like like Harry Potter in space, but not as fun or inventive. Another example a basic scifi that many people like, and I just can't really understand. 3/5
— Andrej Karpathy
Dune (6 books)
by Frank Herbert
Dune is simply not my thing, and makes a great example of where my sci-fi tastes strongly diverge from sci-fi tastes of others (judging by the 4.19 average rating). This book has almost no elements of sci-fi except for some details that don’t really matter to the plot (ships, shields, etc.). It also starts very slowly; at some point you’re halfway into it and really nothing of substance has happened yet. The last 10% get a little better but the end is highly abrupt. The plot amounts to a chosen one taking revenge on a cartoonish bad guy. I feel robbed of my short time here on Earth as a result of reading this book. I did enjoy some of the inventiveness of the world, w.r.t. the Fremen culture, etc. “it’s okay”, or 2/5 on Goodreads scale. 2/5
— Andrej Karpathy
Seveneves
by Neal Stephenson
I thought I would really enjoy this book: a problem of epic proportions, a struggle for survival through science/technology... Unfortunately, this book is like taking The Martian, removing many of the best parts (humor, compelling characters you actually care about), and then making it (what feels like) 10 times longer. It's dry, it lacks focus, pace and clarity. For example, I was frustrated to read about Dinah's problems with her robots or other trivialities when the entire plant Earth downstairs is about to burn. We just barely get to learn something about how the social order copes with the impending doom - a copout. In the end I couldn't take it anymore so I skipped through some of the later parts and then read the synopsis on Wikipedia. More importantly, this experiment confirmed to me that I loved the The Martian not just because it was about science and had lots of nerdy details, but because it was legitimately a fun, interesting, compelling and _appropriately sized/paced!_ story. Seveneves is not. EDIT: found this gem on another review: "The moon exploded, humanity is on the brink of extinction and I just might die of boredom." +1. 2/5
— Andrej Karpathy
The Three-Body Problem
by Liu Cixin
This book was a drawn-out, tedious read that I was in danger of aborting several times until it finally pulled off a miraculous redemption in the last few chapters. I was going to give it a 2/5 when I was 90% through the book, but having read the last 10% I'm happy to upgrade that to 4/5 and looking forward to reading the 2nd book. 4/5
— Andrej Karpathy
Daemon
by Daniel Suarez
A dying man leaves behind a distributed set of daemon programs that infect the world, acting collectively as a kind of superintelligence. Unfortunately, the book lacks in logical consistency: Wait, the program can run a large chunk of the global economy, reads minds with complex invented equipment, and operates a fleet of self-driving vehicles with complex sensing and yet when it speaks to you you must answer either yes or no or otherwise it cannot parse your response? The book also lacks in pacing, with long stretches that become boring or tedious. The real objective of the program is not revealed for a long time, and even once it is, it’s not very convincing and a letdown. The ending is abrupt and the story doesn’t build up into anything. It feels as though the author became tired of the story and just wanted to finish the book already. Inconsistent, tedious, and ultimately unsatisfying. 2/5. 2/5
— Andrej Karpathy
Blindsight
by Peter Watts
I thought I was going to enjoy this book because I was told that it has nice, hard-sci-fi-like aliens. Unfortunately, I learned that that this is a necessary but not a sufficient condition. The aliens were great and fun to contemplate, but there's something about the writing, the way the story is structured and the events that unfold that was simply off. The story is difficult to parse - Peter Watts doesn't hand it to you on the silver platter, for your enjoyment. He makes you work for it, and writes the story in a way that, I thought, required a lot of inferences and reading between the lines. There were also large passages containing some back story for the main character that I didn't fully understand the point of. In summary, I don't think I fully got all the details of this book in a first reading and I emerged somewhat confused about what just happened, and I'll just blame it on the book :) 3/5 3/5
— Andrej Karpathy
Star Maker
by Olaf Stapledon
I wanted to like this book a lot, but unfortunately I struggled to finish reading and ultimately emerged disappointed. The book is ambitious in its grandeur but falls short of delivering a punch. We get confronted with many alien worlds and ideas but none of them intrigued me simultaneously with inventiveness and plausibility. The book gets more and more abstract and religious towards the end. Wait, the stars and nebulae have their own minds and consciousness? I'm not prepared to accept this proposition based on a few vague paragraphs. I reject the idea. I reject the rest. I thought I was reading a scifi book but found myself inexorably reading something much closer to the Holy Bible remixed by someone drunk on the scale of the cosmos. In addition to critiquing the inclusions and choices I could also critique plenty of glaring omissions. For instance, we didn't get to see a single synthetic species? I understand that the book was written in 1930s, but it still bugs me. 2/5: It was okay. 2/5
— Andrej Karpathy
Flowers for Algernon
by Daniel Keyes
A man with low IQ becomes a subject in an experiment that promises to increase intelligence. The book is written in a journal form and chronicles the transformation. Whether you will enjoy the book comes down to your motivations coming in. I read this book primarily because I find the the topic of increasing intelligence / superintelligence to be interesting. Hence I enjoyed the first half of the book. Unfortunately, the book later turns into something more similar to a drama, having little to do with scifi and more with human relationships. However, if you're only looking for a good story with a fun speculative added element and a hint of philosophy then you might just enjoy it! 3/5
— Andrej Karpathy
The Player of Games
by Iain M. Banks
This is another example of a popular sci-fi that I simply cannot stand. I forced myself to make it through hundreds of pages thinking that it might get better later but gave up halfway through, at a point where some alien females were described as wearing jewels. I'm sorry, but I cannot accept human-like qualities naively attributed to alien beings, it's one of my greatest pet peeves. This is another one of those sci-fi that are really a vanilla story that _happens_ to take place in the future. It is a story first and a sci-fi second, and I like my books the other way around. I debated between 2/5 and 1/5, but I hated the sheer naivety and childish "little green men on mars"-like ideas in the book so much that I'm going to go with 1/5. 1/5
— Andrej Karpathy
Why Nations Fail
by Daron Acemoglu, James A. Robinson
Finished reading "Why Nations Fail" as part of Mark Zuckerberg's book club :) Good read, interesting topic. More https://www.goodreads.com/review/show/1393566633
— Andrej Karpathy
Nineteen Eighty-Four
by George Orwell
I went into this book with great expectations since I've heard many references to it, but then found myself slightly disappointed with a thin plot and long sections that read like romance and then horror. Once I got past my initial reaction I realized that I've quite enjoyed the book in retrospect. It reads as a cautionary essay wrapped with a thin plot, attempting to sourly extrapolate societies built on communism/socialism. An interesting point was made that technology helps drive the enforceability of such regimes, which we should clearly be very mindful of. I also liked the idea that language ("Newspeak") shapes thoughts, which is of course quite convenient since language can be simply designed with specific thoughts in mind and then enforced. This makes it possible to manipulate people's thoughts, in case you thought that your own mind is the only definite place of sanctity. Lastly, I enjoyed the completeness and consistency with which the world was developed, even in the presence of on-a-surface insane elements such as "doublethink", or the insistence that 2+2=5, if the party says so. These sound insane, until by the end of the book when you're suddenly not so sure. As a closing remark I thought the book paints a picture that is slightly too much of a caricature. It's relatively easy to consider the party as clearly evil, or that everyone in its plot is basically insane. A more effective approach would present a more nuanced world that looks mostly right and believable, but is still rotten at its foundation with a more subtle form of evil - not black or white but a shade of gray. This would make it easier to draw parallels to a world we live in today, which already unfortunately contains some elements of this book, but is in many other ways of course quite significantly less bleak (phew!). 4/5 4/5
— Andrej Karpathy
Hyperion
by Dan Simmons
This book was exactly what I am usually most afraid of: a "basic" sci-fi written by a non-technical author. It is a nicely written tale with grandeur whose primary focus are people and relationships, and oh, it happens to be set in the future. If you enjoy reading flavored and elaborate descriptions of sunsets, environments, or people's facial features with a few scattered mentions of different solar systems here and there (so that it qualifies as a sci-fi), this book is for you. But if you're interested in exploring possible futures, with a specific focus on ideas, technologies and their consequences in a consistent and elaborate universe, you'll get bored very quickly. 2/5
— Andrej Karpathy
Superintelligence
by Nick Bostrom
Struggling hard to finish the superintelligence book. One 2 chapters left. Must... be... strong...
— Andrej Karpathy
A for Andromeda
by Fred Hoyle
You know how Fred Hoyle's Black Cloud is a fantastic and interesting hard scifi book? This is nothing like that. It is a sloppy, boring, linear and shallow disaster that reeks of missed opportunities and dubious ideas about artificial intelligence and alien life. I can't imagine a less exciting portrayal of receiving intelligible communication from a different galaxy. Any sciency details (which I've enjoyed the most in Black Cloud) and descriptions of the nature of the code or how it is decrypted, how the computer works, details of the communication protocol, any deeper conversations with the computer -- all missing. There are so many missed opportunities everywhere that it literally hurts in my chest to think about it. Some Spoiler Alerts below: By far the biggest disappointment was Fred's portrayal of the artificially intelligent computer which seems anything but intelligent. It appears to experience emotions such as anger (seriously, Fred?), it consistently makes clearly suboptimal decisions (such as killing people early - wouldn't it try to be very friendly to lure humans into false sense of security?), and it supposedly doesn't understand emotions. I would expect an AI as the one described to have a very good understanding of emotions and why they are there in biological bodies evolved through natural selection, as they could be perfectly logically understandable as evolutionary heuristics for successful survival and reproduction. I would have expected more deeper insights from Fred Hoyle but was consistently disappointed with how shallow and mainstream his ideas were. The super-intelligent alien thing in the end is stupefied by the power of love and rebels against its master - are you f*** kidding me? Did a 6-year-old come up with these ideas after reading Galaxy Zack: Monsters in Space? I half expected the alien body to be green, have tentacles and antennas. There are many other problems with the plot line, various logical inconsistencies and the terrible and unintelligible writing, but I'll just stop here. Just awful. 1/5
— Andrej Karpathy
JavaScript
by Douglas Crockford
Highly informative and helpful for understanding the nuances of JavaScript
— Andrej Karpathy
The One World Schoolhouse
by Salman Khan
To someone well-versed on education discussions, this book contains a lot of obvious. I feel like most reasonable people will nod along as they read this book without much disagreement, as a lot of it is not very controversial. For example, a very "insightful" chapter spends several paragraphs getting the point across that some kids learn differently from other kids, or at different pace. The parts I did find interesting were first, Salman's historical account about some of the beginnings of Khan Academy, and second his vision for the ideal classroom (even though it was only discussed in a few rather vague paragraphs). I am a fan of Khan Academy (their mission, not their current execution), but I did not get very much out of this book, so 3 stars seems appropriate. 3/5
— Andrej Karpathy
How to Create a Mind
by Ray Kurzweil
Kurzweil's book offers an overview of the biological brain and briefly overviews some attempts toward replicating its structure or function inside the computer. He also offers his own high-level ideas that are mostly a restatement of what can already be found in other books (such as Hawkins' On Intelligence) with a few modifications (he admits this himself though at one point, for which he gets bonus points). Finally, he applies his Law Of Accelerating Returns (LOAR) to field of AI and produces some predictions for the future of this field. The good: Nice thought experiments section, nice overview of the biological brain (both old brain/cortex and their function), reasonably ok philosophical mambo jambo parts about consciousness and whether it is possible for a computer to be a mind (if you're into that), some analysis of relevant computational trends. By the end, you're almost convinced we're almost there! The bad: First, his own theories are extremely vague and half-baked (though I forgive this. If he knew more he would be busier with things other than writing this book) and essentially reduce to some form of Hierarchical Hidden Markov Model. That's not especially exciting, I think most researchers in the field will agree on such high-level things. I also find it puzzling that he claims to be talking about the mind in its entirety, but then his exposition focuses almost entirely on temporal modeling/prediction aspects and mostly ignores a lot of other magical components of a mind, such as a flexible and efficient knowledge representation / inference engine, or a reinforcement learning - like actor /critic system that surely exists somewhere at the core of all of our learning and reasoning. All in all, I would recommend this book to anyone who's interested in some pointers to our efforts to replicate a brain in the computer, who wants to learn a bit about the biological brain, or who's into the philosophy of it all. 3/5
— Andrej Karpathy
In Defense of Food
by Michael Pollan
This book really changed the way I look at food and will certainly lead to changes in my eating habits. I didn't want to ruin life for my older self (as I found out, almost all most serious western diseases can be attributed to mostly nutrition) so I resolved to drill down into nutrition science over the last few weeks in an attempt to identify a healthy diet. I've skimmed several books, read a number of articles, a few papers, blogs and so on, but it was all a trip down the rabbit hole of complexity that is the human body and its interaction with digested food. I was exhausted and become more confused than certain about anything as I read about all the conflicting diets out there and all the evidence supporting or conflicting all of them. What a mess. I was about to give up in confusion when a friend recommended this book to me. I come from a scientific background, so when I am faced with a problem (such as nutrition) I have a sudden impulse to right away try to drill down into details: of all components of a human body, the nutrients in foods, studies that show how they behave and interact in the body, etc. This book champions an approach that I ordinarily look down on, but it does a great job of convincing the reader that it is the best approach we have at the moment. Mainly, it argues that we should keep it simple, look at the few uncontroversial nutrition facts we have established, consider some history, and apply some common sense. In short, this book is the most honest, balanced and frank attempt I've seen so far to exploring the problem of healthy nutrition and by the end the conclusions seem clear. Warmly recommended! 4/5
— Andrej Karpathy
The Omnivore's Dilemma
by Michael Pollan
3/5. Few thoughts in review: http://www.goodreads.com/review/show/490735236
— Andrej Karpathy
Sustainable Energy
by David J.C. MacKay
This book offers a great overview of issues surrounding Energy policy in the UK (and the world). There is a lot of analysis of the various renewable energy sources and their potential of helping us replace fossil fuels over the next few decades. Economics of every choice is only briefly touched on, however. Still, the book offers a nice and fairly exhaustive exploration of our options and lays out most of the issues surrounding the implementation. David MacKay (author) is also first and foremost a revered scientist who studies this because he cares, which in my view makes him easier to trust on facts, figures and opinions. 4/5
— Andrej Karpathy
I, Robot
by Isaac Asimov
This was certainly an interesting experience, given that the short stories in this book were written at about 1940-1950, but the events the in book take place at about 2070. (i.e. right now in 2012 we are almost exactly half way there) The book contains 9 short stories, from which the ones I would most recommend are Reason and Evidence. What strikes me as most interesting is the nature of predictions in the book. Some predictions are too pessimistic and some are too optimistic, but in funny ways. Here are examples: - The robots in 2070 are described to be heavy, metalic, and have diaphragms. More likely, we'd now think that robots at that time will be made of super light-weight carbon fibers, and they certainly won't have diaphragms when we can just use speakers? - Most interestingly, in charge of the hardcore theory of robots are ... mathematicians. In fact, the positronic brains are seen to yield behavior based on solutions of differential equations. These days, we would most likely not think of including (pure) mathematicians in robotics, and we rarely ever think of algorithms in AI/Machine Learning in terms of differential equations. (wait, should we? ) - One story mentions that the protagonists recorded a video, and that he had to to get it developed. Interesting that it was not obvious that this limitation would not be overcome by 2070, and that we wouldn't be using film. - Even though some of the above contain severely pessimistic views of the world, Isaac imagines us to have hyperatomic drives in 2070, that allow for easy interstellar travel. It is strange to think that we can conquer space, but still need to "develop" a video. Anyway, overall I liked the stories. Many of them essentially come down to an almost detective-like story, where there is something wrong with the robots, and the protagonist has to figure out how the observed behavior has come about from the 3 laws and logical inferences. In general I like the idea that sufficiently advanced robots will become so complicated that we will lose the ability to fully interpret their behavior. There will simply be too many moving parts, and what we observe in terms of the behavior will only ever be the tip of the iceberg. The underlying, perfectly deterministic and individually understandable complexity will simply collapse all together into one term, and we will call it personality. I look forward to these times, at some point around 2070 (sounds reasonable to me). 4/5
— Andrej Karpathy
A Poison Like No Other
by Matt Simon
Actually I was reading the book "A Poison Like No Other: How Microplastics Corrupted Our Planet and Our Bodies" just last week. I didn't realize the extent to which plastics have come to permeate and mess with our entire environment. It's not just about the polymer granules of… https://twitter.com/rezendi/status/1826299346239009263
— Andrej Karpathy
Countdown to Zero Day
by Kim Zetter
In the Stuxnet book it was alleged that these programs are quite large and developed across multiple teams often of different capabilities, e.g. in that case an Israel team (of relatively lower sophistication) owned the delivery mechanism and US the (more sophisticated) payload.
— Andrej Karpathy
How to Live
by Derek Sivers
I love this book and I've recommended it a number of times because it charts its way through the space of self-consistent philosophies for life, none of them strictly right or wrong. I love the idea that, as miraculously conscious entities that we are, we experience a kind of awareness and are empowered to adopt a philosophy and a system for life. This book is an enumeration of ways of being, view on life and its purpose and morality. For me the enumeration in this book is only a step 1, and has given me a lot of food for a more fundemantal theory. E.g. a slider that controls how much you care about people at a different radius away from you (you alone, family, community, all people alive, people alive in the future), how you measure the distance function (e.g. proximity/genetics), and over what time (e.g. right now or in the future and how far). Or how much you value hedonism vs. meaning. Or whether your sense of worth/meaning is more internally or externally driven. Etc. But the 27 answers are then the narrative that emerges out of a certain setting of these more fundamental variables in some interesting high-dimensional space of personal philosophy. 5/5
— Andrej Karpathy
The Accidental Superpower
by Peter Zeihan
Actual quite excellent and solid recommend to anyone interested in a framework for thinking through geopolitics, with the caveats of the same points I raised in my earlier review of Disunited Nations. 5/5
— Andrej Karpathy
How Innovation Works
by Matt Ridley
Fun little stories and pop takes and tales on innovation. 3/5
— Andrej Karpathy
The Bitter Lesson
by Rich Sutton
One of the best compact pieces of insight into the nature of progress in AI.
— Andrej Karpathy
This is Vegan Propaganda and Other Lies the Meat Industry Tells You
by Ed Winters
to-read
— Andrej Karpathy
A Guide to the Good Life
by William B. Irvine
Stoicism was a philosophical movement developed during the Hellenistic period and later practiced by a number of prominent historical figures, including Seneca and Marcus Aurelius. I was reminded again that this time period saw a flourishing of many competing schools of philosophy with their own ideas about the human condition and their own physical meeting locations to concentrate their followers; these have all but disappeared in the modern times. Human psychology has not changed much on the scale of 2,000 years, making a lot of these schools of thought quite relevant to this day. This work organizes this body of literature into a kind of classical antiquity self-help book. I realized that, without knowing it, I was already following many of the psychological tips and tricks that, according to the stoics, leads to tranquility and the absence of negative emotion. The book goes into much more detail, but among these are negative visualization to fight our psychological natural tendency of "hedonistic adaptation" (the practice of imagining a regression to your living condition along various dimensions to appreciate its presence), projective visualization, active self-denial (e.g. temporary practice of poverty to appreciate wealth), categorization of worries into those you do or do not have control over, which e.g. leads to fatalism w.r.t. the past and the present, meditation (not in the buddhist sense of clearing your mind, but quite opposite a deliberate, intense psychological practice of the above techniques), etc. A little bit too long, a little bit too bloated and repetitive, but good fun. 3/5
— Andrej Karpathy
Numbers Don't Lie
by Vaclav Smil
A quick read whirlwind tour of a number of topics, at a pace of only about 2-3 pages per topic. Some fun notes and examples: - 75% of all births between 2020 - 2070 will be in Africa - vaccinations have an extraordinarily high benefit-cost ratio, an approx. ~44X return on investment - estimated heritability of linespace is only ~15-30% (?) - humans are sweating champions in the animal kingdom, very useful for thermoregulation and endurance even in hot weather - synthesis of ammonia not only averted the Malthusian catastrophe, but also just in time allowed a blockaded Germany to continue manufacturing explosives and prolonged WWI by years. - renewables only provided ~4.5% of electricity in 2017, and electricity is only 27% of global energy consumption ;s - many large uses of fossil fuels have no clear non-carbon alternatives, including long-distance transport (air, water), production of primary iron, cement, synthesis of ammonia, plastics, space heating - no other domesticated land animal can covert feed to meat as efficiently as broilers (chickens raised for meat production), at a feed-to-meat conversion efficiency of ~15% (pork 10%, beef 4%). The lives of these chicken are straight up unethical - they live 7 weeks (normal lifespan is ~8 years), have malformed bodies, spend life in dark confinement. But cost $2.94 / pound of boneless breast, great. - in North America / Europe about 60% of total crop production is for animal feed, not human feed - four pillars of modern civilization, allegedly: ammonia, steel, cement, plastics This is a fun / quick read, though mostly a large collection of mostly disconnected cliff-notes style quick fact summaries that are dense in numbers and comparisons and can become a bit exhausting. Perhaps due to speed, some topics get a somewhat questionable, almost misleading treatment, for example I found the EV and GDP sections mildly annoying. 3/5
— Andrej Karpathy
Atlas Shrugged
by Ayn Rand
Since a few people asked - I rated 2/5 not because of the implied philosophy (which I am obviously sympathetic to), but because the actual presentation is overly cartoonish, 10X too long, and highly tedious. 2/5
— Andrej Karpathy
Advanced Nutrition and Human Metabolism
by
I liked “Advanced Nutrition and Human Metabolism”
— Andrej Karpathy
Disunited Nations
by Peter Zeihan
Peter Zeihan presents a biased, incomplete and seemingly just slightly rushed geopolitical analysis of the past present and future world (dis)order, which despite its flaws makes for an informative, interesting and entertaining (due to his writing style) read. In particular, in stark contrast to popular narratives over the last decade or so, Peter argues strongly for an imminent collapse and fracturing of China. The core thesis is that the last ~60 years of rising global prosperity and peace (relatively speaking) is a highly anomalous state of affairs in the backdrop of history. This period has primarily been enabled by the global "Order" established by the American superpower (by far the economic and military "last man standing" after the second world war) to fight the Cold War. However, after the collapse of the Soviet Union the chief strategic justification for the Order is claimed to be severely dissipated. Combined with the energy independence afforded by the shale revolution, America as a highly self-sufficient nation is turning inward, and becoming increasingly indifferent to the world's affairs. It looks unwilling to get involved in wars like those in Korea and Vietnam, to continue nation building experiments in Middle East, or to continue to protect international trade routes. In Peter's view, this has dire repercussions for the global physical/maritime security that a lot of other nations have come to rely on. He examines the world's countries through 4 lenses (viable home territories, agriculture capacity, energy access and demographic structure) to divine whether they will thrive, or fall apart. From here on the book reads like a slightly watered down version of Guns Germs and Steel but with a modern focus. Much is said about mountains, (navigable) rivers, deep water ports, climates, natural resources, arable land, global supplies of oil, coal and natural gas, geographic viability of solar/wind, and demographic distribution (male/female ratios and age histograms). In contrast and unfortunately, not very much at all is said about culture / institutions (e.g. why isn't Argentina be a superpower? And why are Singapore, Japan South Korea doing so well?). Similarly, how can an analysis with a claimed scope of multiple decades not include climate change and its likely effects? Why not attempt at least a mention of the increasing relevance of cyber warfare, when our economies and national security are increasingly based on information processing and computers? In this sense, Peter's analysis and focus on physical barriers and the number of super-cruisers each country commands comes off as a bit outdated. By the way, why is India barely discussed? Finally, why should America not have a continued vested interest in the security of the global financial system and trade? The analysis jumps a little too quickly from "America is the de facto world police" to "America checks out completely, global chaos ensues and oceans become plundered by pirates". Overall, this was an informative read that I am happy to recommend, as long as one is prepared for and willing to forgive what seems to me like a little bit of tunnel vision and a little bit of jumping to unsupported conclusions where it counts. But if you're willing to cherry pick some of the analysis and fit it into a slightly more broader picture you'll have a good time! 4/5
— Andrej Karpathy
Prisoners of Geography
by Tim Marshall
Isn't particularly thorough or convincing but gives a quick intro to many regions of the world and some of their history (e.g. Russia, China, USA, Western Europe, Japan+Korea, Middle East, Afric, Arctic ). Unlike the title's claimed focus, geography is not featured as prominently and we're mostly treated to a quick history lesson of each region in turn; i.e. I'm not sure that I learned all that much about how the geography of each region influences that history. In summary, a decent intro to the topic but doesn't present any particularly coherent, well-supported theme, and instead meanders around as a summary of the first 3 paragraphs of each region's "history" section in Wikipedia. 3/5
— Andrej Karpathy
Life at the Speed of Light
by J. Craig Venter
The book covers a brief history of the discovery of DNA as the "software of life", the work that went into the sequencing of it (read), the ability to synthesize it (write), the process of introducing it to a host cell (boot), and speculations about what all of this technology means for the future. The book does a pretty good job answering the question of what the sequencing of the human genome looked like "on the ground" for the people involved, and gives a good sense and quite a few details into what kinds of problems there were along the way and how they were being resolved over time in the process of science/engineering of the achievement. I thought that the core thesis of the book "life at the speed of light" was a bit of a stretch. At one point, I wasn't sure why, a number of pages are devoted to teleportation in Stark Trek and then quantum entanglement and quantum teleportation. This is one of my favorite pet peeves because I believe I think it really confuses the public (we're not literally teleporting matter, we are moving a quantum state from one physical ensemble to another). It's also HIGHLY unrelated to the type of "telepotation" discussed in the book, which is of the form: take life, sequence DNA, transmit the DNA (information over light), synthesize DNA, introduce into a living cell to "boot" it. I understand that we can both read and write DNA and transmit it at will (it's just information), but there are still A LOT of nuances involved (e.g. especially the need for a physical and related host cell from somewhere that can also take in that DNA). The discussion of these nuances are summed up to only a few quick paragraphs, so I think a non-expert reader may think we're actually talking about teleporting living things around Star Trek style with their memories intact and all, or "beaming down" Martian life from a drone, as discussed near the end of the book. These are very misleading and unnecessary analogies, imo. I would advise a selective reading for anyone interested in a first-hand account of what the research challenges were in a lot of the early work on reading/writing DNA of tiny organisms all the way to humans and, on a high level, how they were overcome. 3/5 (I liked it) 3/5
— Andrej Karpathy
More from Less
by Andrew McAfee
A fairly unconvincing, high level, pop-econ take on dematerialization in the economy. The first 7 chapters lay out the context: Malthusian condition, the Industrial Revolution, Earth Day, etc. Chapter 5,6,7 form the core of the book where we are treated to some pretty sketchy diagrams with everything improving up and to the right while our physical consumption of raw resources reverses. Very little is said about a number of obvious objections, eg the ongoing globalization and its effects. After Ch7 the book transitions into talking about capitalism and why it's the best thing since sliced bread. Did you know that in 1950 we got 117B pounds of milk from 22M cows, but in 2015 we got 209B pounds of milk from just 9M cows? Truly something wonderful to celebrate. For anyone interested in the topic I'd recommend wikipedia and journal articles, e.g. I found the following to be much better "bang for the buck": - https://mk0eeborgicuypctuf7e.kinstacd... - https://www.pnas.org/content/pnas/112... 2/5
— Andrej Karpathy
The Great CEO Within
by Matt Mochary
A very short, practical, to the point guide discussing topics of interest for a smallish startup CEO. I enjoyed a number of ideas I saw for the first time, e.g. "Energy Audit" (forces you to explicitly think about where you can uniquely add the most value by examining your last week's calendar), the "top goal" heuristic, or the discussion around group buy-in. It's good, but a bit more if a good checklist for "topics to discuss" with someone you respect in more depth in person, and more grounded in a concrete organization. 3/5
— Andrej Karpathy
Ketofast
by Joseph Mercola
2/5 ("it was ok"). I'm already on Keto+IF and enjoying it quite a bit so I got this book to collect more data about how/why the diet works, and some details on how to maximize its benefits. Unfortunately what I found was a seemingly hastily put together high-level hodge podge of related information that is breezily presented and then re-iterated a few times over and over, as if repeating the same high-level points would make a person understand things better. The book can't decide if it's a popular science book or not, and awkwardly ends up a bit of both. The un-cliffnoted version could earn more stars. 2/5
— Andrej Karpathy
The History of Money
by Jack Weatherford
This book is a relatively short whirlwind tour of some aspects of the history of money, presented chronologically from antiquity to approx. 1997, when this book was published. Basically, due to a number of properties of asset creation in society (e.g. role of skill/expertise, need for up-front investment in production, batch efficiencies, etc.) various assets regularly end up in surplus or scarcity for any one person or organization, leading to a need for some system of exchange. Money fills this need and becomes the scalar signal that communicates scarcity across society in an emergent, decentralized manner. The book goes over multiple forms of money that have existed over the course of history stopping by different places of spacetime in each chapter. It touches on gift economy and debt in small societies in presence of trust, barter in cases of lack of trust, early commodity money that typically has intrinsic value and is a convenient means of exchange (e.g. cacao seeds used in the Aztec empire to "top off" an uneven barter, cigarettes, manufactured items (furs), coins (gold/silver), animals (cattle), or even people (slaves)), early representative money (checks, notes, receipts, or even paper money as long as it was backed by gold), fiat money ("worthless" paper not backed by anything tangible), and electronic money where we transition the whole system from atoms to bits. Not covered are cryptocurrencies as since these were not around in 1997 (eg bitcoin, which on top of all that innovation is also not governed by any central authority and is secured cryptography due the computational hardness of some inverse operations in our Universe). I enjoyed the discussion of various societies and their relationship to money as it evolved. E.g. Sparta/Egypt/Persians and other traditional empires largely rejected money in favor of government as the organizing principle, while other empires (Greeks, Romans, etc) quickly embraced it. Plato/Aristotle both had odd views on it. The Aztecs and others regulated it and its exchange tightly. Merchants were not historically part of high stratums of society, and the pursuit of money was not pervasive or even looked up to. This is fun to contemplate against the backdrop of our capitalist way of life. I also didn't realize, until reading this book, just how common and pervasive it was for governments to tamper with the money supply and monetary policy over the ages, often to no good outcomes in the long term. Eg Nero in Ancient Rome minted coins with progressively less silver as he struggled to fund the bloating Roman government beurocracy. Various other governments over the years printed a ton of money causing massive inflation and crumbling the entire economic system. On the unintended side of things, it was interesting to read about the discovery of the New World and the California gold rush in mid 1800, which both flooded the market with gold, again causing some havoc. Anyway, I think the book can be a little too hasty in glossing over some developments and leans more heavily on description instead of explanation, but overall covers a large portion of interesting historical aspects of money and its place in a number of ancient and recent societies. I would think of it more as a series of pointers and anecdotes instead of a definitive first principles guide. 3/5
— Andrej Karpathy
The Big Picture
by Sean Carroll
I love Sean Carroll, but I can't bring myself to finish this book. This is not some kind of cool science book. It's a little too much too long too high-level philosophy, re-iterating the same things over and over again, and just overall meh. Someone else less familiar with physics might like it. EDIT: I accidentally read this book again forgetting that I already had. I have to start by saying that I actually love Sean Carroll and adore his presentation/teaching ability. That said, I had to skip over a bunch of it because the book is primarily a work of philosophy and history, and a concatenation of the first 2 paragraphs of popular science articles on topics of physics. The part I actually loved was the Appendix, where he discusses the Core Theory in quite a lot of detail. If that 2 page explanation was expanded into the length of this book it would be an easy 5/5. 3/5
— Andrej Karpathy
A Crack in Creation
by Jennifer A. Doudna
Very similar to Watson's "The Double Helix", this book is a part story of discovery and part a textbook, in this case on the topic of CRISPR, from a scientist deeply and technically involved in the technology. The first half of the book explains the basics of DNA, the central dogma, the massive, ancient (and still ongoing) virus-bacteria molecular warfare, the historical context of gene editing research, and finally how CRISPR was discovered and why it is such a big deal. CRISPR evolved as part of a bacterial immune system where the bacteria stores explicit records of viral DNA segments in their genome. Together with a number of Cas (CRISPR associated) proteins encoded by genes in the vicinity, the resulting molecular assembly is able to search the genome for stretches that "match", inducing a double stranded break that disables the gene. It's incredible that this molecular machine was found by evolution and that there are so many forms of it. It's hard to imagine the Cas9 protein whizzing about the nucleus in brownian motion (it does not hydrolize ATP!), interacting with chromatin/histones and somehow cutting up matches. The biophysics of this process elude me. Anyway, this immune system mechanism can be repurposed, improved and generalized to perform very targeted and cheap gene editing (delete, insert, substitute, invert, ...), gene expression up/down regulation, tagging, etc. This is now actively utilized in animals and plants (in both somatic and germ cells), and also on humans (in somatic cells for treatment of many diseases, or more worryingly in the germ line for making permanent targeted changes to human DNA). The book also discusses gene drives, which allow us to hack evolution itself, e.g. giving us the ability to wipe out the entire population of mosquitos, on which I have very mixed opinions. It also goes into some remaining challenges such as specificity, delivery, etc. In some aspects it is not as comprehensive as I'd like (e.g. how the adaptation part works, or what the limits are). In summary, this is really the beginning of a powerful set of technologies with broad societal implications, as we begin to reprogram both us and the nature around us in hyper-targeted ways. It's refreshing to find a book that does such a good job describing large portions of it without dumbing it down too much, and also doing a good job hinting at some of the associated ethical dilemmas ahead of us. 4/5
— Andrej Karpathy
Leonardo da Vinci
by Walter Isaacson
This is an engrossing, well-paced biography that is a pleasure to read. I'm left with a deep and motivating admiration of many facets of Leonardo's character, especially his eager mind, attention to detail, thinking from first principles, his use extensive use of physical notebooks, and, most intriguingly, his use of art as a thinking tool. It's fascinating to get a glimpse of someone so far ahead of his time, and it's fun to think about what he would make of today. 4/5
— Andrej Karpathy
The New One Minute Manager
by Kenneth H. Blanchard
A very short book that employs clever writing to turn otherwise bland material into both fun and insightful reading. 4/5
— Andrej Karpathy
The Other Side of History
by Robert Garland
A comprehensive and rare study of an ordinary human experience through the ages. It's sometimes surprisingly pretty (e.g. ancient romans/greeks in the upper echelons of society seems to have had it quite nice), but most often shockingly not so much. 4/5
— Andrej Karpathy
Faraday, Maxwell, and the Electromagnetic Field
by Nancy Forbes, Basil Mahon
to-read
— Andrej Karpathy
The Better Angels of Our Nature
by Steven Pinker
The book argues that things have actually improved. I find it hard to believe that any sane and half-well-read person would think otherwise, without entire chapters on medieval torture. Someone else might like it, but I found it too exhausting. Quitting ~1/3rd through. 3/5
— Andrej Karpathy
Introduction to High Performance Scientific Computing
by Victor Eijkhout
Good & quick to the point reading
— Andrej Karpathy
The Gene
by Siddhartha Mukherjee
This book offers a comprehensive and engaging overview of genetics. It includes the history of the field, anecdotes of its development, a well-paced technical explanation of the high level aspects, and quite a lot of discussion on the associated moral dilemmas that we are faced with as we understand how we can use this technology to change our own species. Unfortunately, the book does not delve into some of the aspects of modern genetics that I find most interesting, such as gene drive. These are discussed near the very end almost as an afterthought, and are hardly given enough focus. Similarly, epigenetics is only briefly touched on. Lastly, the book is very human-centric and does not discover genetics in a broader context of evolution in animals (e.g. selfish genes), which I find fascinating. Overall, this will likely become my default recommendation for the reference Genetics book for a general interested reader who is mostly interested in the history of genetics, who enjoys thinking about the ethics of genetics in humans, and who wants to get a good high-level overview of the technical aspects. 4/5 4/5
— Andrej Karpathy
Homo Deus
by Yuval Noah Harari
This book reads like the author read a number of popular science articles, watched some sci-fi movies, attended a transhumanist meetup, got just a bit high on weed and then started writing. 2/5
— Andrej Karpathy
The Retreat of Western Liberalism
by Edward Luce
This book offers a pessimistic view on the future of liberal democracy, and interprets its recent spread not as an unstoppable or monotonic trend of history, but as a fragile system, open to an attack from both within (e.g., election of Donald Trump), and outside (e.g., Russia, China). The West is not a natural outcome of a linear progress of history and it might not represent mankind’s enlightenment on morality and virtue. As the book puts it, “We are on a menacing trajectory brought about by ignorance of what it took to build the West, arrogance towards society’s economic losers, and complacency about our system’s durability”. Overall, this book is a great way to become more depressed about humanity’s future. The topics are interesting but I’ve come to dislike the author’s writing stye. Perhaps this is because my mathematical training begs for a very clearly stated sequences of statements that are supported by arguments at each stage. Instead, I found Luce’s writing to be sprawling, overly abstract, unfocused and even rambling. As if each section started as a 2am entry into a “random thoughts” bed-side journal, and was later put through a best-effort conversion into a book by an editor. The book is quite short, and hence worth a read if you're interested in the topics, or if you are too optimistic about the future. 4/5
— Andrej Karpathy
U.S. History For Dummies
by Steve Wiegand
A nice reference that paints the broad strokes of the U.S. history. I appreciated the author's approach to the topics - the book does not try to be overly academic and instead has a conversational tone to it. I always found the existence of the U.S. somewhat magical and the book shines some light on how that came about. A big open question that remains in my mind is why the U.S. became such a superpower relative to the other countries. Either way, good read! 3/5
— Andrej Karpathy
The Everything Store
by Brad Stone
This is the story of Amazon.com and how it became a ~$500B company. The book is fun and engaging to read. The chapters focus on painting a picture of Jeff Bezos and his philosophy, and the various adversities that the company has faced over its 20 years of existence. I am generally not a huge fan of worship-fiction (which is very common when it comes to books about "visionary founders"), but luckily this book is only about 50% that. The other 50% is a genuinely fun read about Amazon's beginnings, struggles, and its now-sprawling empire from a high-level business perspective. My favorite parts included: 1) the clear-headed analysis that went into the original spark behind Amazon, 2) the repeating pattern of the "flywheel" positive feedback loops that was the energy source of Amazon's growth, 3) the amusing inability of the incumbents to realize what was happening and how to address it, and finally 4) the anecdotes related to all of the above. good/fun read. Would recommend to anyone interested in the history of internet and the dot com bubble and general high-level business strategy grounded in the examples from Amazon's history. 4/5
— Andrej Karpathy
The Elements of Style
by William Strunk Jr.
This is a very short book that is skimmable in 1hr, readable in few, but absorbable only on the scale of months/years. It is a dense instruction manual for writing well: it starts with simple grammatical rules of form in part 1, progresses to various tips/tricks for the connective structure in part 2, and ends with a discussion of how one can infuse their writing with a soul, a kick, and rhythm. I found myself disagreeing several times and found some parts to be contradictory, but I think it's still well-worth a skim. You'll almost certainly find a few things you've been doing wrong all along, or not doing enough of. 3/5 - I liked it. 3/5
— Andrej Karpathy
Death's End
by Liu Cixin
I finally finished the Three Body series, as a result of enthusiastic recommendation from several friends, but I emerge disappointed, and even perplexed about the scale of the discrepancy in people’s reception of sci-fi. TLDR: there are several fantastic diamonds of novel ideas sprinkled around, but they are mixed in with a very large mass of goo, full of soulless characters, narrative/logical inconsistencies, poor choices of what to expand on and what to omit, and a really disappointing conclusion. Okay lets get more concrete. **Spoiler alert.** I loved the grand scope of the story - the idea of a dark forest universe (a fun semi-resolution to the Fermi paradox), new physics (although the dimensionality manipulation was stretching it), the idea of fundamental physics as a weapon or a defense (e.g. space folding, or “slow fog”/dark domain), and the idea that there is a huge technological disparity between different civilizations. I also really liked that the story spans a huge amount of time, that there are different “eras” (the post-deterrence era in Australia era was my favorite), etc. A lot of these concepts were truly a joy to contemplate. Unfortunately, for every 1 awesome nugget, you must pay with 1 hour of your life spent on something unbelievable, irrelevant, annoying, or naive. Sometimes you have to spend several hours, such as when the author decides that he’s going to insert a whole different book into this book, which comprises three fairy tales that end up having a limited impact on the story. You also have to suffer through exceedingly under-developed, non-sensical and outright unbelievable characters, who have no soul. As for the world itself, some "new physics" is intriguing (strong interaction droplets, short/medium/fast comms, photoid strikes), some are dubious (dimension folding, people traversing dimensions), and some are very hard to understand the purpose of (e.g. mini-universes at the very end). Some other concepts you would expect in a universe are also missing entirely. In particular, Artificial Intelligence plays no role in this universe, except for some AI side kicks in your spaceship. I was also reading through a few other reviews, and found a paragraph that resonated with my experience, in that it gave a lot of examples of the neurotic/rash writing style, so I’ll quote it entirely: “However, the story arc is very, very unsatisfying. Lots of cool things happen, and then the story stops or changes direction. Wade has built light speed technology? Better kill that story line so we can do more hibernation. The second Trisolaran fleet had this epic battle - but we're not going to tell you anything about it... just that it was epic... They managed to make little pocket universes, but we're not telling you how they did it, or how they brought one over to the Blue Planet. Galactic humans established colonies, but yes, you guessed, never going to see them. 4-dimensional civilization encountered? Nope, not gonna revisit it apart from some cryptic conversation with its AI. Evil 4-dimensional-converted-to-3-dimensional aliens who drive the annihilation of the universe with dimensional strikes? Yep, not gonna mention or encounter them again. Love story between the main character and her brain-rehydrated lover? Yeah, they just miss each other by a few minutes/million years and he hooks up with her best friend. Final fate of main characters? You guessed! It stays a mystery! 10 dimensional "Eden" reborn universe that the main characters were going to visit? Oh, never mind, we're going to save a few kilograms worth of mass, we don't really want to see it that bad…” Recommendation? Read if you have a lot of time on your hands, but do not be afraid to skip large sections. 3/5 - "it was okay". 3/5
— Andrej Karpathy
The Dictator's Handbook
by Bruce Bueno de Mesquita
This book examines positions of power (e.g. country leadership, mayors, CEOs, deans, etc.) by assuming entirely self-interested actors who seek to gain and retain power, and argues through examples that this relatively simple model gives the first order explanation of many world events. If you really grasp the message you'll adopt a much more cynical world view, but you'll also stop torturing yourself over stupid questions like what a country "ought" to do, what is "right", or why the people in power just can't see it. At the same time, spending some time in reality will reveal ways of remedying various suboptimal situations (e.g. the inefficacy of foreign aid) with solutions that recognize the root cause and manipulate incentive structures of those in power. The book supplements its thesis with various examples. For instance, resource-rich autocracies with small winning coalitions tend to oppress the population, which is irrelevant to the revenue needed to retain power. Conversely, there exists a curious tension in countries that cannot extract riches from the ground and instead rely on a productive population to generate wealth. This leads to the development of technologies that empower people, such as better communication networks, transportation infrastructure, education, etc., but these in turn pose a threat to those in power. There is also an element of "survival of the fittest" to systems with small winning coalitions, where even if a benevolent leader rises to power who wants to raise the standard of living for the masses, they are likely to become replaced by those who promise to redirect that wealth to the key supporters (e.g. those in charge of the police, military, treasury, etc.). A coup is significantly easier if these institutions turn a blind eye. The outlooks are somewhat better for an average person living in a democracy, because the incentives of the ruler are aligned with making the average person better off to win a re-election. In short, to understand the dynamics of a system of power the first order features to consider are 1) the nominal electorate (people who theoretically have influence), or the "interchangeables", 2) the real selectorate (the people who actually have the influence), or the "influentials" and 3) the winning coalition (the number of people required to keep power), or the "essentials". You can then solve for the dynamics. My main critique of the book is that it is simply too damn long, too repetitive, and badly in need of an experienced editor. You'll hear the same statements re-iterated ad nauseam, and in many cases you'll wish the author was more concrete instead of arguing in generalities, at a level where the abstraction washes out the complexity and makes the conclusions self-evident under the simple model. Therefore, I'd recommend that the reader selectively skips through the book, or watch CGPGrey's summary video ( https://www.youtube.com/watch?v=rStL7...), or the EconTalk podcasts featuring the author (e.g. http://www.econtalk.org/archives/_fea...). 3/5
— Andrej Karpathy
Pattern Recognition and Machine Learning
by Christopher Bishop
I didn’t rely on books too much. I liked Bishop’s book, which I’ve read through early in my PhD.
— Andrej Karpathy
The Dark Forest
by Liu Cixin
Similar to the previous book in the series (Three Body Problem), this book is a mix of diamonds of interesting ideas sprinkled around a mass of gray, featureless goo that the reader has to painfully wade through. Some parts are not too logically consistent or believable, some parts are annoyingly spurious, some parts are annoyingly fast, but similar to the first book, some parts are ambitious and interesting enough that the whole is worth a read. I liked it (3/5), but why does it have to be so painful? Apparently the third one is Even longer. I hate it, but I'll probably read it. *cries* 3/5
— Andrej Karpathy
The Lifecycle of Software Objects
by Ted Chiang
The story starts strong, interesting and pregnant with potential but ultimately fizzles, dissipates and meanders around relatively silly and implausible dilemmas. 3/5
— Andrej Karpathy
A Short History of Nearly Everything
by Bill Bryson
This is an ambitious book covering the history and development of ideas from a very wide range of topics pertaining to how we (humans) got here. Bryson starts all the way with the big bang, to the formation of the cosmos, the solar system and the planet, abiogenesis, evolution, and finally human prehistory. The book spices up a relatively high-level discussion of the relevant scientific topics with the lives and idiosyncrasies of the scientists who made the critical discoveries. I admire the book for its scope, but an unfortunate side effect is that the book doesn’t get a chance to dive in to any one topic in a satisfying manner. Instead we get an overview of many topics and ideas which is fun sometimes unless you’re already familiar with the topics (e.g. Darwin’s beagle voyages, Watson&Crick’s DNA discovery, spread of humans from Africa, etc.). I personally somehow found the book a little tedious by the end and I was happy when it was over. Marking read, phew. 3/5 - it was okay. I suspect someone less familiar with many of the topics who is content with a broad but shallow overview and willing to hear a lot about the lives of scientists who made the relevant discoveries could enjoy it much more. 3/5
— Andrej Karpathy
Narconomics
by Tom Wainwright
Once in a while you read a book that shatters your preconceptions and updates your world view. In the wonderful "Narcoeconomics: How to Run a Drug Cartel", Tom Wainwright (an editor at The Economist), explores the narcotics industry through an economic lens. You'll see how drug cartels are much more like McDonalds or Walmart than you previously thought: optimizing their supply chains, competing, forming mergers, colluding, worrying about human resources, public relations and brand building, offshoring, franchising, investing in R&D, dealing with rise of disruptive online marketplaces, diversifying (kidnapping, prostitution, human trafficking). You'll see flawed prison systems much more as recruiting grounds, jobs fairs or networking events. You'll see full-body tattoos as an employee retention strategy. By the end of it, you'll emerge with a more complete and coherent picture of the narcotics industry and its dynamics, understand why Nixon's war on drugs has been so ineffective, and maybe get a few hints of how we could do better. 5/5. 5/5
— Andrej Karpathy
The Greatest Show on Earth
by Richard Dawkins
44% of Americans believe that humans were created by god about ten thousand years ago. I admire that Richard Dawkins' stamina in fighting the good fight but he's not going about it very effectively. He sounds pretentious. He uses technical terms quite frequently, to the point that I can barely keep up (and I have reasonable background in the area already). He repeatedly insults and mocks people who do not accept what he sees to be completely self-evident. The argument is not strung together too neatly and his time/resource allocation is also slightly questionable. You can feel his pain and sympathize, but if he wanted to be more effective he would have to surrender some of his ego first, which I don't think he can do very well. I personally read this book because I was curious to see what evidence from the animal kingdom Dawkins would select as his best ammunition and how he would structure the argument. I emerged slightly disappointed: by the tone, the flow, the structure, and the selection. If there is someone out there who would like to learn more about evolution I would instead recommend looking at some very well-produced videos on YouTube. 3/5
— Andrej Karpathy
Guns, Germs, and Steel
by Jared Diamond
In the ~mid 16th century the Spanish conquistador Pizarro captured the Incan Emperor Atahualpa despite being outnumbered a few hundred to few tens of thousands. Why didn’t Atauhalpa’s warriors instead land in Spain and capture King Charles I? More generally, why did Europeans conquer the world? And what factors govern the broad trends of societies in the history of our species? Are they predominantly biological? cultural? geographical? is it all luck and chaos? Guns, Germs, and Steel seeks to answer these questions. The central thesis is that if you carefully trace the major factors they will lead you to geography. Do you play Civilization? Of course you do. The book argues that the winner of the game of Civilization has very little to do with the players and almost everything to do with the distribution of resources near the starting locations of different civs. And if there are any biological factors (e.g. civ-specific bonuses), they are negligible in the scheme of things (as it is in Civ games as well). The Europeans, for example, happened to spawn next to a lot of bonus tiles for a wide variety of nutritious plants giving +10 food, and also bonus tiles for several domesticable animals, giving +5 food and +5 production. Any civ player can tell you that if your population grows faster and the cities become bigger and denser you will get lots of nice bonuses: specialist civilians, social stratification, faster research, more culture, etc. Over several hundred years these bonuses add up and the next thing you know you're showing up with 10 +28 attack Musketeers in a territory full of gold tiles defended by a +8 attack Aztec warrior. Unmodeled by Civilization games, you also enjoy carrying more germs that you’re also resistant to, and it's cheaper to establish trade routes west-east instead of south-north due to climates (convenient for Eurasia, not so much for America). These are very important questions and it’s all great to know and contemplate, but dammit is it painful - the book feels like eating your vegetables. Be prepared to hear (in excruciating detail) about different protein:carb:fat ratios of dozens of plants species you’ve never heard of. Be prepared to learn much more than you ever wanted about all possible candidate animals for domestication. Jared in fact manages to enumerate them all. These sections are definitely interesting and you can’t help but admire the thoroughness and attention to detail here, but it’s exhausting. There are a few more things that can be critiqued: - the hypocrisy in the dubious claim that Papa New Guineans are smarter than whites is jarring, considering that the whole point of the book is to discredit biological determinism. - the book is almost certainly sometimes guilty of post-hoc reasoning, but admittedly Jared is aware of this criticism and tries hard to avoid it. - the pacing is slightly off: there are hundreds of pages devoted to plant seeds, and only dozens of pages devoted to much more interesting patterns of development (especially in the last chapter). I can’t decide if I should recommend this book. I personally liked it (i.e. 3/5 on Goodreads scale). It was a significant undertaking and I feel that I’ve gained quite a bit of insight into the development of societies in the history of our species, but… at what cost? **shudders**. 3/5
— Andrej Karpathy
Economics in One Lesson
by Henry Hazlitt
The main thesis of this book is that the economy is a complex dynamical system and government's efforts to tamper with a free market economy is a game of whac-a-mole where a variety of hard-to-see n-th order (n>1) negative consequences dominate the intended easy-to-see positive consequences, resulting in an overall net loss for everyone. This thesis is illustrated with the use of few dozen example settings per chapter that are seemingly different (e.g. tariffs, rent control, unions, minimum wages, government infrastructure projects, technological creative destruction, price fixing, savings, etc), but are in fact argued to be instances of the same general pattern and the same recurring fallacies. If you already share the philosophy (e.g. you like Austrian school of economics, libertarian philosophy, capitalism, Ron Paul, and you worship the free market as the ultimate decentralized planning and allocation system), and you already have a working knowledge of economics then this book will supply you with a lot of additional ammunition of examples and arguments for fighting your socialist or keynesian friends. You’ll love it and walk away with an even peakier posterior over your mastery of economic philosophy. Unfortunately I do have to critique the book in some respects: - It does assume quite a bit of economics knowledge that it does not bother to explain sufficiently, leaving me a little perplexed in some sections. i.e. this is not a textbook. More worryingly, - The author just can’t hold back his feelings and resorts to ad hominem attacks too frequently, multiple times citing unidentified individuals that have clearly caused him a lot of emotional pain as too stupid to understand his very basic lesson. These attacks add nothing. I was strongly reminded of Richard Dawkins’ related and unfortunate tendency to mock those who do not accept what he views as self-evident. - The book is, as is often the case, a very one-sided account of the central thesis, frustratingly lacking in any hints of counterexamples or uncertainties. For example I would have loved to see the discussion at least touch on, e.g. wealth distribution inequality and the related and tightly coupled inequality in power, externalities, social darwinism, historical precedents of government projects (e.g. atom bomb, space program, etc), etc. In summary, I enjoyed the book overall but I was hoping for less of a "I have it all figured out, look it’s so elementary, and there are no good arguments to the contrary" vibe and a more complete treatment of the topic (and preferably without ad-hominem attacks for bonus points). Still an overall recommended read. 4/5
— Andrej Karpathy
The Power Broker
by Robert A. Caro
I could not finish this book because it is a little too long and too detailed, making the "bang for buck" ratio slightly too low for my liking. It is a relatively pleasant read, but I also thought that the author assumed too much about what the reader knows regarding public policy, politics, and so on. Many times I felt myself lost in the subtleties of the politics that was unfolding, and there was no decent attempt made at explaining the situation more clearly.
— Andrej Karpathy
Humans Need Not Apply
by Jerry Kaplan
This book provides a decent exploration of the future of automation. The first part of the book talks about AI/Machine Learning. This may have been a decent intro for someone completely new to the field, but for someone very much inside the field it was a little frustrating to read because of explanations that I think confused concepts in artificial intelligence, sometimes for example using the terms "machine learning" and "neural networks" interchangeably. I was also put off by some silly examples of what the future looks like, such as "Trying on an outfit? Instead of asking a sales assistant if you look nice, why not take a snapshot of yourself and seek crowdsourced opinions?". To me, these silly and quite speculative examples of small use cases give off too much of a singularity hype hype vibe. The later part of the book is where things finally take off and the book goes into some social-economical repercussions of automation and the likely more dramatic income inequality. This is mostly why I got the book and I was looking forward to these parts, but unfortunately the book dives in quite quickly and became a bit of a stream-of-consciousness that assumed quite a lot of knowledge of economics, law, etc. I did not have enough background to appreciate entire chapters (e.g. surrounding the proposed job mortgage concept and its merits) and went from being bored in the first half to mostly confused in the second half. I think these chapters should have been expanded, introduced more slowly, put in wider context, and made more concrete with more frequent examples. 3/5
— Andrej Karpathy
Thinking, Fast and Slow
by Daniel Kahneman
I could not bring myself to finish this book. The book is filled with shady experiments on undergraduates and psychology grad students and wild extrapolations of the associated results. I find it exceedingly difficult to take many of the conclusions seriously. I can't read into them. I can't trust them. I can't base my decisions on them and I resist incorporating them into my world view with anything more than 0.01 weight. In fact, several of the experiments that this book mentions were also found to be not reproducible by a recent meta-study on reproducibility in psychology studies. Here's a characteristic example of me reading the book. The author says: "Consider the word EAT. Now fill in the blank in the following: SO_P. You were much more likely to fill in the blank with a U to make SOUP than with an A to make soap! How amazing. We call this phenomenon priming, system 1, something something". In fact, no, SOAP came to my mind immediately. All I could think about when I read this book is my own experience of participating in a friend's psychology study once. He designed an experiment and asked me to do some things and answer some questions, but at some point it became extremely clear to me what the experiment was about, or how he hoped I would behave. I went along with it, but I couldn't believe that this would eventually become part of a paper. It was a joke. I'm afraid you can't go through a similar experience and take these studies seriously from then on. All that being said I do find the broad strokes of the system1/system2 division proposed in this book to be interesting and appealing. A small few of the examples were fun to contemplate, and it was okay. 3/5, aborting reading. 3/5
— Andrej Karpathy
Dealers of Lightning
by Michael A. Hiltzik
This book details the history of Xerox PARC, which set up a research lab that invented many aspects of modern computing and then failed to capitalize on it (at least to the extent that many people thought they should have). I was happy to see the author resist the obvious and often-retold narrative of a corporation that was simply too dumb to realize what their visionary research division had. The book instead paints a more realistic picture, mentions some of the tensions present between a corporation and a research lab and dispels the overly simplistic notion that Xerox would have clearly become immensely successful if they only followed up on the research. This is mostly our benefit of hindsight and there are many other variables at play. You'll get a sense of the history, some of the drama, some of the background story behind the inventions. But unfortunately the book doesn't spend a lot of time talking about the layout of the lab, or some of the philosophy that led to its success in research. Neither does it try to generalize, observe, or contrast. Therefore, sadly the book felt mostly as an enumeration of facts rather than an attempt at their interpretation in a wider picture. This is understandable because the former is relatively easy, but the latter is not. 3/5 - I liked it. 3/5
— Andrej Karpathy
Einstein
by Walter Isaacson
This was my second read of an Einstein biography, this time by Isaacson. Coming from Isaacson, the book is well-written and seemed very thoroughly researched. Overall a great read, but if I had to complain my biggest issue is that the emphasis was not allocated very well. For instance, a huge portion of the book is devoted to Einstein’s personal life, reading through his correspondence with his love interests. It’s interesting for a while, but after some point I thought we were intruding a little too much, and that it was stretched out and uninformative. Conversely, some very interesting portions of his life are under-represented. In one chapter he publishes his streak of 1905 papers, and in what feels like a few pages later he is a scientific celebrity. This period, where the community is discovering and processing him as a person from nowhere who made sudden and large contributions is among the most interesting, and very sparsely covered. There could have also been much more space for his works’ retrospectives - how do scientists today see his theories, in what ways was he right or wrong based on our current understanding of physics? This book was written in 2007 but so few of these interesting retrospectives are present that it may have as well been published in 1955. I thought this was a huge missed opportunity. A few more fun parts of the book I enjoyed: - Einstein did not describe himself as atheist and in fact frowned on them. Instead, he subscribed to something similar to Spinoza’s god - an abstract, pantheistic, impersonal god. I think I mostly self-identified as an atheist until now but I’ve been swayed to Einstein’s view by this book, as it was nicely presented by Walter Isaacson with help of original texts by Einstein. - Einstein strongly disliked nationalism, and thought of himself as a citizen of the world. An interesting view, expanded on nicely in the book. - I liked the anecdotes surrounding Einstein’s Nobel prize. Most people felt strongly that he should get one, but the situation was more politically charged than may seem at a first glance. In the end, Einstein received the Nobel for photoelectric effect, not for his much more impactful theory of general relativity. - The book goes into quite a lot of detail on how Einstein was rejected by almost every single academic institution prior to his 1905 papers. Luckily, it turns out that a patent office is not a bad place for an academic tenure. - The book goes into quite a bit of fun details about the massive Einstein hysteria in the public. A scientific celebrity of that scale is quite singular in our history - it was relatively unprecedented back then, and we also haven’t seen quite the same phenomenon since. I wish we did. - It was also fun to think about Einstein’s stubborn refusal to accept Quantum Mechanics despite mounting evidence throughout his life (“He does not play dice”). The irony is that many established senior scientists were on the defense of the old order when Einstein first formulated GR, and now here he was much later (as an established senior scientist) stubbornly defending the old order in face of attacks from QM. This irony was not lost on Einstein at all either, but he still refused to correct for this persistently observed bias across history. As a scientist, I hereby resolve to overcompensate in accepting new paradigms once I’m older :) I developed a new appreciation for Einstein after reading the book, and there were plenty of fun parts and anecdotes that made this quite worth the read. 4/5
— Andrej Karpathy
Creativity, Inc.
by Ed Catmull
I appreciate what Ed has tried to do with this book, and I'm a fan of his and Pixar. It was an enjoyable read, but if I was to complain about something it's that I felt the book was trying to be contrarian quite often in a slightly forced way. I ended up noticing a formula repeated many times: In this situation most people would think X... (I didn't, and I wasn't fully sold that most would), but I think Y (I agree and I've heard something similar said several times before). Of course, there was a distribution of reactions to these statements - some I thought were relatively unique and interesting (e.g. negative spaces in art, a well-articulated defense of errors), and some were somewhat forceful (e.g. do not resist change - okay, thanks). I could also notice a persistent undertone of pride for Pixar employees, with statements such as "Do you see now why I love to work here?". Somehow I have slight negative reaction to it. I understand he's very excited and Pixar is his baby, but for me these parts started to feel like PR. There were a few things like this that slightly rubbed me the wrong way as I read through it, but I don't mean to sound too negative and overall enjoyed several interesting parts, Ed's quite obvious desire to understand these topics as well as he can is fun and contagious. I liked the chapter on the Notes day near the end, that sounds like a quite nice idea that I also see a lot of merit in - the idea of recognizing that your employees have brains, and distributing decision making through the ensemble of smart people throughout the company to some degree, and giving them a sense of ownership. A powerful strategy, and this was well articulated. Also, I liked the reflections on Steve Jobs, and the dispelled myths about his personality. I do notice quite consistently that people who knew him best cringe the most when they are asked about popular media coverage about his life. Somewhere between 3 (I liked it) and 4 (I really liked it) /5. I'll round up because it's Ed. 4/5
— Andrej Karpathy
Logicomix
by Apostolos Doxiadis
Soooo good. A very fun graphic novel about people and their maddening quest for foundational mathematics and objective truth. The novel does a great job at faithfully portraying and expressing the obsession for solving these fundamental problems that consumes great minds. And no wonder-- it is written by mathematicians! 5/5
— Andrej Karpathy
Where Good Ideas Come from
by Steven Johnson
There are really only two core ideas in this book: 1. That innovations are best modeled as ideas having sex, in the sense that they don't pop into existence but instead each idea is formed by the process of mixing elements from previous ideas (recombination), or slightly improving on an aspect of the idea (mutation). This view makes all of our innovations look similar to intellectual animals, with their own family trees. And 2. That these innovations don't happen in sudden eureka moments inside the mind of one person, but instead happen over time through "slow hunches" that incubate inside an obsessed mind, while that mind is engaged in liquid networks of other minds. These two core tenants are supported with several briefly-discussed fun historical examples. However, I think the book could have been slightly shorter, it rambles a bit and stretches some assertions a bit, but overall it gives some food for thought. 3/5. 3/5
— Andrej Karpathy
The Moral Animal
by Robert Wright
"The Moral Animal" explores questions of morality and ethics through the lens of evolutionary biology. Spicing up the book is a parallel thread that supports its evolutionary psychology conclusions with examples from Darwin's own personal life. If you enjoy thinking about evolution, the forces shaping our minds, feelings and instincts, and haven't read much about this topic before then you might enjoy it. However, if you've comfortable with the subtleties of the term "selfish" in "The Selfish Gene" you will find the first half repetitive and the second half overly hand-wavy, vague and unconvincing. Final recommendation: Go ahead if you'd like a quick intro to evolution and its effects on our minds. Expect to be tickled by arguments, but ultimately unconvinced of anything. 3/5
— Andrej Karpathy
The Rise and Fall of the Third Reich
by William L. Shirer
I resolved to read this book (in form of 60 hours of audibook - it took a while!) as a cautionary tale, and in an effort to understand the forces that enabled WW2, Nazi Germany and the Holocaust. I’ve been meaning to do this for a long time, since i consider this period to be an important section of general knowledge. The book provides an engaging narrative and a relatively complete account of some of the most important events. If I had to complain about something it would be that the book does not end up painting a complete picture and leaves many of my questions unanswered. For instance, it is one thing to state that this or that event happened on some date. This is an “easy” task of collecting, indexing and then regurgitating historical documents and records. It is entirely another to explain why an event happened rather than any other event, and why or how it came about in the context. This latter task is much harder - it is one of interpretation, summary, knowledge and insight. Ultimately, I felt that while the book was filled with well-researched instances of the former, insights of the latter form were unfortunately more scarce. That being said, overall, I still consider the book well worth reading. A few highlights/thoughts that caught my attention follow. The philosophical underpinnings. Several early chapters are devoted to studying Hitler’s background and ideology, which of course directly influence how future events transpire. In particular it seems he has developed most of his views early during his bohemian lifestyle in Austria and then kept them largely fixed thereafter. The chief ideas are those of racial hierarchy and Social Darwinism (the belief that the strong should see their wealth and power increase over the weak). From these two core tenets a lot of the rest follows as a corollary. When you add a pinch of delusions about destiny and embody the whole in a persuasive and eloquent rhetoric in a time of disillusionment and general social/political unrest you start to get a potent mix. The support of the people. One of my biggest questions going into the book was how a regular German person has interpreted all the events leading up to and during the war. It was clear that Hitler had in fact amassed wide support. Did no-one go “So hold on… all Jews are categorically unworthy people and responsible for everything bad in this country?”, or “Wait a second, why are we bullying the neighboring countries and invading them again?”, and then much later “Wait, why am I wielding this weapon and why do I have to shoot these other random people again, instead of hanging out at home with my wife/children?”. I don’t think these questions were adequately explored in the book. Indeed, my most favorite parts (though there were few) were those where the author provides personal comments and anecdotes to the general atmosphere, which he had himself witnessed. My best explanation is that early on while Hitler was rising to power he relied mostly on his strong rhetoric skills and the hard times that have befallen the country in that time. It also seems that things genuinely seemed to improve during that time with respect to unemployment rates, etc. And once he came to full power it became clear that the trap has been sprung. The book is filled with references to pervasive propaganda, and the author himself confesses to have been influenced by it. Then the freedom of the press was given up, which enabled the regime to twist events in their favor. Lastly, even if some eventually did suspect that things went awry, there was no recourse. People were set against each other and if anyone got accused of anything undesirable they would quickly get shipped off to a concentration camp. The image one gets in the end is that the people were simultaneously brainwashed with propaganda, lied to with press, and paralyzed with fear. There were some who did try to do something, either by political means or more often by plotting an assassination. Fun fact: did you know that apparently there were on order of 30 assassination attempts made on Hitler? None of them worked out, in large part due to Hitler’s paranoia and conscious effort of constantly changing his schedule last second and generally acting unpredictably. The politics of deceit. The next question I had was how did the other countries (in particular France/England) not see this coming from miles away and why did they not intervene somehow before Germany gained so much strength and momentum? As the author points out repeatedly, Hitler had also very clearly articulated everything he would do when he came to power in Mein Kampf (which was published by then), including taking over all other countries to provide “living space” for Germans and exterminating Jews, enslaving Slavs, etc. Judging from the book it became clear that it was a combination of two things: 1. Hitler’s constant deceit and 2. Naivety of the other powers, likely due to fear of another war. As an example of deceit, I found the invasion of Poland interesting, in which Hitler looked for an “excuse” to invade, and being unable to find one his men (dressed in Polish uniforms) staged a fake attack on a German outpost. Another example was Hitler’s constant message in his speeches of how reasonable and peace loving Germany was (while plots were made to take over the whole Europe and beyond). Last example was giving an ultimatum with a lifespan of only 24 hours, which of course could not have even been discussed in time, with the intention of later being able to claim that an ultimatum was given (and leaving out the 24hr part). Therefore it turns out to be somehow quite easy to deceive a country’s citizens, to deceive other countries’ leaders, and fabricate events to provide justification for whatever. The same strategies are without a doubt used to this day to achieve all kinds of ulterior motives. The holocaust. The “New Order” chapter of the book was by far the most damaging read. We get a glimpse into exterminations at a massive scale, where people get treated as livestock or worse, and where methods are devised to most efficiently kill large batches of people. In fact, their problem was that they could not kill people fast enough. Even more disturbingly, the people involved (when interviewed afterwards) do not seem very phased about contributing to such events in retrospect. They shrug saying they “followed orders”, and have somehow managed to become indifferent towards these “lesser” people. One former concentration camp guard interviewed in his late life said he didn’t regret anything, and eventually, frustrated, simply said “we just hated them”. Simple as that. Indeed, some have grown to enjoy the whole thing. It seems childish to shrug and simple label all of these people as “evil”. Instead, there’s a lesson here about the malleable nature of a person’s morality, the power of propaganda and the strength of wanting to belong to a group. In summary, a great book with a lot of food for thought, though ultimately it provides more description than understanding. I tried my best to do the latter for myself but I probably got 50% of it wrong anyway, and I hope no historian reads any of this. 4/5. 4/5
— Andrej Karpathy
Harry Potter and the Methods of Rationality
by Eliezer Yudkowsky
Wonderful alternate Harry Potter timeline story. In many aspects I prefer it to the "real" Harry Potter books. It is kind of like hard sci-fi, but in Harry Potter universe. It starts strong, then drags a bit for a while, but then it ends very strong. Very enjoyable read overall! 5/5
— Andrej Karpathy
A Peek into Einstein's Zurich Notebook
by John D. Norton
interesting read, glimpse into how theory of GR developed
— Andrej Karpathy
Sapiens
by Yuval Noah Harari
Sapiens offers a good overview of broad trends that have shaped the human civilization from prehistoric to modern times, but suffers from inconsistent assumptions about its target audience and, due to its scope, an abundance of high-level statements that don’t connect. My chief criticism of the book is that it does seem to address a consistent audience. One minute the book discusses and explains a broad trend (e.g. the empire, or capitalism) and then abruptly dives into very specific examples from history to support its statements (e.g. a briefly alluded to specific battle during the expansion of the British empire). The problem is that as an ordinarily-educated person, I am familiar with the basic ideas such as capitalism so these sections bore me, but I am not at all familiar with the details of specific conflicts or related circumstances. As such, I spent the majority of the book either bored, or confused and overwhelmed with information that was seemingly assumed. Overall, the book often felt as a work of a historian written for other historians, selling and supporting certain historical interpretations that the author believes to be true. Not an attempt to teach the general audience about history. I enjoyed the first part of the book dealing with prehistoric times - these chapters were fun to read, perhaps partly due to my relative lack of knowledge of this era. From there, my enjoyment of this book diminished monotonically until the end, which suddenly features several hours of philosophical musings about the point of life and happiness. If you’re looking to get a basic idea about some broad strokes of our history, you might enjoy the first parts of this book. If you’re, like me, trying to get a better and more concrete sense of what life was like in different times of our history, this book never spends enough time and depth to paint and communicate a coherent picture and you will end up disappointed. 3/5
— Andrej Karpathy
Zero to One
by Peter Thiel
There are several good parts in this book and I've felt some of my views shift as a result of reading the book (which is all you can ask for). A lot of Peter's arguments are supported with examples and anecdotes which is great because it grounds the discussion, but I also kept thinking that some seemed cherry picked. In a pool of 20 successful companies it seems it would be easy to always find the one that fits a narrative. Regardless, interesting read; These are useful ideas to be aware of. 3/5
— Andrej Karpathy
The Elements of Moral Philosophy
by James Rachels
A reasonably comprehensive introduction to Moral Philosophy. In first portion it touches on basic issues such as what is morality and how objective is it? How is it related to religion and can it exist without it? The second portion discusses a few historically important ideas and people in the field: - Ethical egoism (do what's best for you) - Utilitarianism (do what's best for an average person in expectation) - Kant's theory (follow rules you'd think would make good universal rules) - Social contract theory (follow rules that self-interested rational people can agree on for mutual benefit) - Theory of virtues (just be fair, honest, just, courageous, civil, .... 50 other things here) It's a good book and I liked it (i.e. 3 stars), but there's also several things I disliked: - The chapters and topics seem strangely chosen and there is no coherent theme throughout the book that emerged. It seems to be a set of individually chosen, arbitrarily split or merged and randomly arranged set of topics. There is no overlaying theme or flow. One of the chapters near the end just suddenly and randomly includes whole paragraphs summarizing the book so far. Why is that placed there? Sometimes this chaos can be distracting. - The author seemingly can't resist injecting his own views into all parts of the exposition, sometimes strongly agreeing or mocking various historical ideas without fully expanding on his reasons. This can be distracting: I wanted an as-objective-as-possible exploration of different ideas and a collection of arguments/counter-arguments that have been raised against them in the past. I don't need to know the author's personal opinion every single paragraph, especially when there is an entire section devoted to the author's views at the very end. It's good and useful stuff, but I wish it was all neatly at the end, or at least at the end of each chapter. - All his examples about what is morality involve some terminally ill patients or other medical dilemmas. The field is so much wider and touches on all kinds of interesting issues in politics and law! Very few of these connections were made. - The final chapter on author's own views is very poorly written. I tried twice and I don't understand what he's proposing. - Sometimes some arguments are not fully developed and seem almost trivially refutable, but this is not followed up on. But to be fair, it's only an introduction book and serves reasonably as such. All in all a recommended skim, especially if you're very new to the topics as I am! 3/5
— Andrej Karpathy
The Count of Monte Cristo
by Alexandre Dumas
I don't usually read this kind of literature, but Count of Monte Cristo is a unique and marvellous book that explores a variety of themes: life/death, revenge, justice, hope, etc through a fascinating story. Reading the book is comparable to seeing a large puzzle coming together piece by piece, as Edmond slowly but surely achieves his revenge, until it all comes together in a brilliant and emotional ending. The style of writing is amusingly unique (remember this book was written in ~1840!), especially the chapters rich in noble banter. Lastly, before you endeavour to read this book, keep in mind that it is a significant undertaking that takes many many days to get through. Sometimes the story drags on and on with all kinds of details and "gay" chitchat when all you really want to read about is Edmond finally meeting Mercedes again. So be prepared to fight your eagerness, and try to enjoy the ride! 4/5
— Andrej Karpathy
The Great Gatsby
by F. Scott Fitzgerald
The author does a good job of verbally sugar-coating an otherwise boring story about an obsessed bootlegger and his unstable acquaintances. A book perfectly suited to teach helpless K-12 children to value prose over content and that unfaithfulness and obsession lead to despair and death. 2/5
— Andrej Karpathy
Lean In
by Sheryl Sandberg
I got the impression that this book was primarily intended for women, but it was an interesting read for me nonetheless because it explores a variety of barriers, issues and concerns that women face as they progress through their careers. Barriers that are, at least to a white man, not immediately obvious. I liked that Sheryl supplements her arguments with a variety of anecdotes from her and her friends' personal lives. These make the exposition more authentic and enjoyable. There are also a number of interesting passages that address topics of leadership and communication in general. One of my favourites that comes to mind was a passage about the relativity of truth, and how one can more effectively communicate with others when this is recognized (Seek and Speak your Truth chapter, page 79). I've also developed a certain admiration for Sheryl that this book helped reinforce. As she explains in the book, you can't have it all, but she clearly has a strong drive to come as close to it as possible. That's inspiring. Enjoyable read! 3/5
— Andrej Karpathy
Journey to the Ants
by Bert Hölldobler
Journey to the Ants paints a very interesting picture of an ant colony as an intricate super-organism in which individual ants are only small, dispensable, fairly mechanical and easily replaceable walking batteries of exocrine glands that sense their world primarily through array of chemical words, touch, sound, and very poor vision in some cases. The fascinating image I take away from this book is that the colony is the individual, and every ant is like a protein flowing through the veins of the individual, doing various tasks to support the organism. Throughout the book you'll gain an understanding of an ant colony, how it functions, how it is divided into castes, how it controls its environment, migrates, cooperates with surrounding species through various symbioses, fights, and forages. There is a also a lot of discussion of various types of species in the vast and varied ant species universe, together with discussion on how they have evolved over time from wasps. Excellent! I'm leaving out a star because it could have been even better! Very often a stunningly interesting behavior is described, but there is no effort made to explain on a reductionist level how it is achieved through simple rules that the ants may follow. For example, ants can build bridges across leaves with their bodies. How does a single ant decide to become part of the bridge? How does it know where to attach, or how long to stay? I wish there was an attempt to unravel the algorithm every ant follows. Surely, these kinds of experiments can be carried out in laboratory conditions and monitored closely? More generally, I found the book to be fairly light on this process of trying to "debug" an ant. Instead, much of the focus is on simply cataloging the behaviors on a high level. Maybe it's just the grumpy Computer Scientist in me... Oh well. 4/5
— Andrej Karpathy
The China Study
by T. Colin Campbell, Thomas M. Campbell
data-driven book on nutrition and diet
— Andrej Karpathy
Vision
by David Marr
David Marr proposes a complete framework of how the brain could process visual information from 2D image all the way to 3D geometries at the very end. Even coming up with a not-obviously-wrong hypothesis of this entire pipeline is no small feat and David almost makes it sound consistent and as if it could work if it was only implemented with a few details filled in here and there. I do wonder if he was slightly ahead of his time, I'm sure he would have loved to play around with Kinect RGBD videos :) 4/5
— Andrej Karpathy
Sum
by David Eagleman
I found this to be a frustrating read because every story featured ridiculous metaphysical stuff that can't possibly be made consistent or interesting. The entire notion of afterlife is intellectually flawed from the get go, so I feel that the author has made a mistake in the very premise of the book. A more interesting book would be something like "forty tales of how we got here", where he could describe possible ways we got here to the present moment without needing any mention of afterlife. Some of them could be metaphysical mambo jambo involving god(s) of various forms, shapes, sizes and colors (and afterlife, possibly) but a larger portion could be about more plausible scientifically-consistent possibilities, more along lines of abiogenesis and similar. I was going to rate it a 1/5 but two stories I thought contained semi-interesting ideas were "Microbe" and "Reversal". 2/5
— Andrej Karpathy
Microcosm
by Carl Zimmer
This is a good exploration of E. coli bacteria / associated topics for a layman, and a pleasant read overall. The book begins with a brief description of historical context under which E. coli bacteria was discovered, but quickly transitions to describe the life of E. coli. An amazing picture of a complicated and intricate molecular machine emerges. The book goes on to describe populations of E. coli, their chemical warfare/symbiosis, different strains, genetics, and (inevitably) evolution. It finishes with synthetic biology and societal considerations regarding our present use of E. coli through genetic engineering. I knew a lot already but I imagine that for someone who does not know much about molecular biology, genetics or evolution this will be a dense but exceptionally rewarding read. 4/5
— Andrej Karpathy
Chaos
by James Gleick
I read this a while ago but I can't remember it being a very spectacular or enjoyable read. Disclaimer: I took chaos mathematics at school so I was reasonably familiar with most presented concepts, which could have made it a little more boring. 2/5
— Andrej Karpathy
The Double Helix
by James D. Watson
I read this quite a long time ago but I remember it being an interesting account of what science looks like from the trenches. The struggles, uncertainty, the thrill of the race with other labs. It also paints a realistic picture of discoveries: it's not one sudden eureka moment that changes everything but a process of gradual narrowing down of the truth with a sequence of smaller eureka moments in between. 4/5
— Andrej Karpathy
Gödel, Escher, Bach
by Douglas R. Hofstadter
This book is a must read or at least must selectively skim for anyone interested in intelligence. Some of the ideas regarding intelligence and how it should be implemented are perhaps slightly outdated (you would see much more statistical reasoning if you asked experts today), which is largely absent in "old AI" approaches to intelligence. 5/5
— Andrej Karpathy
On Intelligence
by Jeff Hawkins
I liked this book: it contains a few nice thought experiments about intelligence. Bare in mind, however, that Jeff Hawkins' implementation of these ideas has not proven to be fruitful so far in his company Numenta. 3/5
— Andrej Karpathy
Life of Pi
by Yann Martel
I guess I was never smart enough to "get" this book. I was forced to read it by my English teacher and it was ... meh. 2/5
— Andrej Karpathy
A New Kind of Science
by Stephen Wolfram
this book is a mixed bag. You really have to selectively skim chapters that look interesting because you will never make it fully through. I thought some of the chapters had some very interesting results, however, and the notion of a computational universe is very intriguing and interesting. 4/5
— Andrej Karpathy
Harry Potter series
by J.K. Rowling
I don't think any one of the books are Amazing, but the sum total of the entire Harry Potter story line and universe is. 5/5
— Andrej Karpathy
