This is my 3rd post summarizing the key takeaways I got from reading the book Modeling Life. I recommend reading my earlier posts first to get a good grounding on the foundations covered in the book. A system can exhibit three different types of behavior: equilibrium, oscillation, and chaos. The E. coli bacteria we encountered…
This is my 2nd post summarizing the key takeaways I got from reading the book Modeling Life. I recommend reading my earlier post first to get a good grounding on the foundations covered in the book. A system can exhibit three different types of behavior: equilibrium, oscillation, and chaos. The shark-tuna system we saw in…
Herman Wouk was researching for a novel he planned to write about World War II. He interviewed several physicists at Caltech who had worked on the bomb, including Richard Feynman. After the interview, Feynman asked Wouk if he knew Calculus. “No,” said Wouk. To which Feynman replied, “You’d better learn it. It’s the language God…
I spend a couple of weeks every February keeping up with my investments, timing it to coincide with the release of Buffett’s annual letter to shareholders. I found something interesting while performing this ritual this time. I came across Aswath Damodaran’s website, where he has compiled data on the S&P 500 index, including earnings, dividends,…
This is the 11th and final post in my series on building a toy GPT. Read my earlier posts first for better understanding. I concluded the previous post by explaining how Attention works. You can download the Excel sheet and Python code that explains how Attention from here and here. The Transformer architecture was originally…
This is the 10th post in my series on building a toy GPT. Read my earlier posts first for better understanding. I asked ChatGPT to complete the sentence given the phrase: “I chose that bank for”. It completed the sentences sensibly. Here are the four sentences it generated: In order to generate the right words…
This is the 9th post in my series on building a toy GPT. For better understanding, I recommend reading my earlier posts first. Word embeddings convert words into fixed-length numerical arrays. Each number in these arrays corresponds to a specific characteristic of the word, such as its association with a place, person, gender, or concept.…
This is the 8th post in my series on building a toy GPT. For better understanding, I recommend reading my earlier posts first. I love playing and watching cricket. The dominance India showed in the recently concluded World Cup is astounding. I have never seen anything like it in the four decades I’ve been following…
This is the seventh post in my series on making a toy GPT. For better understanding, I recommend reading my earlier posts first. The MNIST dataset is the “hello world” of machine learning, containing images of handwritten digits that are used to train machine learning models. It includes 60,000 training images and 10,000 test images…
This is my sixth post in a series on building a toy GPT. I recommend that you read my previous posts before reading this one. When using linear and logistic regression, which involve just a single linear equation, calculating derivatives is easy since there is only one equation to work with. Those derivatives help us…
Seeking Wisdom 