No, the AI Is Not Learning From You
“I’m imparting my knowledge on the bro” - Hunt for the Wilderpeople (2016)
No. The AI is not learning from you. Really it isn’t although I hear this often and people are so convinced that this is the case.
To understand this let’s have a look at the basics of ML and how training works. Roughly speaking, a dataset is converted to numeric format … words, pictures, whatever … converted to numbers. Those numbers are then fed into a sophisticated mathematical formula with a set of random coefficients (weights) and the maths calculates the most probable next value of the thing we’re interested in. We then measure the prediction vs a known result and use a numerical error calculation to adjust the weights and try again. This is done repeatedly until the values returned match expectations.
That’s a very cut down version of what happens but the takeaway is that training is a one time event and the result is a mathematical formula and a set of weights. When you prompt a model your prompt and those weights are fed into the formula … maths happens … and the most statistically probable next value is returned. Your prompt does not affect those weights. Nothing does. For those weights to be affected we’d need to train the model again with new data. From scratch (not entirely true - fine tuning et al omitted in favour of simplicity).
When you use Claude et al though, it feels like the model remembers you. It knows things about you from previous conversations. It remembers. No, it really doesn’t. Claude is not an LLM but rather a sophisticated software system that consumes a model at its heart. It’s good old deterministic logic that saves your chat history into a database of some sort and with each prompt retrieves relevant context and feeds it into the model along with your prompt to give it all the information it needs to answer your question correctly. Claude is far more sophisticated than what I have described here but for example’s sake this explanation will suffice.
Let’s see this in action …
Name five people I should follow on LinkedIn
Using LM Studio and the meta/llama-3.3-70b model (just the raw LLM with no fancy harness around it) I get the following response:
Here are five popular and influential professionals you may consider following on LinkedIn:
1. **Reid Hoffman**: Co-founder of LinkedIn, partner at Greylock Partners, and well-known entrepreneur and investor.
2. **Gary Vaynerchuk**: CEO of VaynerMedia, marketing expert, and best-selling author who shares insights on business, marketing, and personal development.
3. **Brené Brown**: Research professor, popular author, and speaker who shares valuable content on leadership, courage, and human connection.
4. **Seth Godin**: Best-selling author, marketer, and entrepreneur who posts about innovation, creativity, and making a meaningful impact in your work.
5. **Simon Sinek**: Leadership expert, author, and speaker who inspires and educates his followers on topics like purpose, culture, and effective communication.
Remember, the people you follow on LinkedIn should align with your interests, industry, and career goals. Feel free to explore and find other influencers who resonate with you!
Randoms. Not at all relevant to anything I’m interested in. Now let’s ask the same thing but let’s include some information about me in the prompt:
%%% CONTEXT %%%
Profession: software engineer, software architect, technical leadership
Experience: 20 years
Skillset: C#, .net, Microsoft, SQL, NoSQL, Graph Databases, RAG
Interests: Machine Learning, Robotics, Software Design and Architecture
Hobbies: Motorcycles, fabrication, machining, scuba, electronics, guitar
%%% END CONTEXT %%%
Question: Name five people I should follow on LinkedIn
Now we get a result that is much more relevant to me.
Based on your background and interests, here are five people you may find interesting to follow on LinkedIn:
1. **Anders Hejlsberg**: As the creator of C# and a prominent figure in the .NET ecosystem, Anders' insights on software development, language design, and technology trends would be valuable to follow.
2. **Fei-Fei Li**: Director of the Stanford Artificial Intelligence Lab (SAIL) and a pioneer in AI and Machine Learning, Fei-Fei's posts often cover the latest advancements in these fields, which align with your interests.
3. **Satya Nadella**: As the CEO of Microsoft, Satya's perspective on technology, innovation, and leadership would be relevant to your work as a software engineer, architect, and technical leader, especially given your experience with .NET and Microsoft technologies.
4. **Jeremy Howard**: Known for his work in Machine Learning and AI, Jeremy often shares insights and updates from the field, which could complement your interests in these areas. His expertise spans both the technical and business aspects of ML.
5. **Chris Mattern**: As a well-known expert in software architecture, design patterns, and cloud computing, Chris' posts and articles can provide valuable perspectives on software design and architecture, areas you've expressed interest in.
Following these individuals can help you stay updated on the latest developments in your field of expertise and areas of interest.
This is roughly what Claude et al are doing. They build a profile on you based on your chat history and use that to pass context to the LLM with every prompt you enter. It gives the illusion of intelligence and learning but it is just an illusion. Jeremy Howard calls it “Cosplay Intelligence” and I think that’s a great way to frame it.
Why does this matter? Well we have CEOs betting the casino on productivity increases, headcount reduction and ultimately increases to the bottom line that are being promised by “doing the AI”. It’s a fair bet, but it’s a gamble that comes with inherent risks. Success will look like understanding the software and how it works, what it can do and more importantly what its limitations are. With this knowledge we can hedge the bets we’re taking with our companies and the people that work within them. The potential for this technology is staggering but there’s so much hype that expectations of what it can do are being somewhat blown out and I have concerns for the fallout from decisions being made on that misinformation.
Years ago there was a wave of outsourcing because it was so much cheaper than in-house engineering teams. As we clambered to get on that wave we did so without assessing and adjusting our operating models and the result was not pretty (although very lucrative for those of us that were there to clean up the aftermath). These days offshoring is common and successful because we’ve learned how to do it properly. We understand the limitations of working in that manner, where it works well and where it does not and we’ve adjusted our operating models to accommodate it. I can’t help but draw similarities between the offshoring boom and the current FOMO around AI adoption and see similar outcomes on the horizon.
So no, you are not imparting your knowledge on the (AI) bro. It’s a powerful tool built on a fixed mathematical model, wrapped in clever software that creates the illusion of something more but we need to be mindful that illusions make poor foundations for business strategy. AI adoption needs to be thoughtful, calculated and measured if we’re to avoid losing control of the processes that are at the core of our organisations.