Somebody here posted about a Socratic method skill for Claude (or whatever your LLM of choice is). I’ve found this to be fantastic. Dropped in a dense PDF about Linear Algebra, Postgres (those are the two I’ve gotten into so far). It’s basically just a skill.md that reads, “Here’s a document, give me an opening statement about the material and ask me a question. As I provide answers, guide me to simpler or more complex areas of the topic. Something something Socratic method.” Works great.
Spaced Repetition flashcards or flashquestions to progress through a game might be good; configure Anki decks to surface cards from when for example the user jumps to crush a bonus block.
May I ask why? If by "not the same" you meant: AI is not at the level of a good human educator, it is a fair argument - but not many humans can afford to have a one-to-one human educator either.
Perhaps I am a bit more optimistic, but if AI could give me, let's say, 90% of an average lecturer's capability, at 10% of the price or even less, that is probably extremely valuable. (For one, I would certainly be interested in learning new things that otherwise wouldn't make sense financially)
Moreover, there are areas of expertise that are very hard to reach without special help. I have a PhD in computer science. Earning that degree was a major time commitment, and it gave me close contact with one of the few experts in a niche area. I absolutely loved the experience.
But I did not learn everything I wanted to know. And I can’t afford to take a time-out to go to grad school again for a second PhD. On the other hand, an LLM, in some sense, has access to the world’s knowledge. With some patience and persistence, you can mine almost-expert-level instruction from one.
When ChatGPT 3.5 came out, I excitedly tried this for a topic that I WAS an expert in. The results were discouraging. The model’s answers seemed to reflect the general misunderstandings that people had about the topic. More recently though, I had to review a paper in the same area, and I used a late-model ChatGPT to help check my work. It was an eye-opening experience because it was no longer confused. And it found longstanding misunderstandings I had, buttressing its answers against my skepticism with citations to original work. I came away very impressed. This kind of AI “rubber duck programming” is my preferred style of use now. I used it just today to help me learn an area of statistics I have always been fuzzy about. This approach definitely requires some careful prompting, but I am optimistic that AI tutors will one day be a real thing. My only worry is that people lose the ability to understand what makes an answer a good one and why we should care about good answers.
A lot of knowledge in such niches has never been written down. Papers don't contain it. Tacit knowledge, it's in unpublished config files and unpublished experiments that some prof or PhD student remembers, all the things that are not "publishing-worthy" (because not flashy, not novel-sounding enough, or a negative result etc) but crucial for the method to actually work. Most of expertise is like this and this is why spending years in a research group, talking to the researchers, participating in their culture, understanding their thinking over years (or even just months) can be invaluable. There are also many things that people only say after a beer, after you know each other well. Claude and ChatGPT haven't ingested any of this. They ingested the sanitized surface version, the press release / advertisement version. Because papers are basically career marketing and advertisement mostly. The real expertise is guarded and hidden.
This is of course about the PhD level and beyond. For regular university-level education, where the goal is to pass the usual exams, ChatGPT and Claude are more than enough. But for real-world "doing", not (yet).
I got to see Luis von Ahn the inventor of CAPTCHA discuss his latest crowd sourced venture, Duolingo, which I'm sure everyone has an opinion about, back in 2013 or 2014 at The LAB Miami in Wynwood.
At that point they had ~24 employees. He said his interest was crowdsourcing and thought most of the engineering effort would go toward the crowdsourced translation, but at that time, it was being handled by just two employees.
They brought on two prominent consultant researchers, leaders in the field of language learning and linguistics. The first question they asked the consultants was which part of speech should they present to the learner first. The consultants didn't have an answer. They decided to dedicate engineers to collecting data in order to answer that question.
Very cool to hear that you were at the LAB Miami in 2014, Around 2016, I was trying to organize a group of folks to build a variable spaced repitition application that tied to a off the shelf eeg, to figure out the optimal time to learn at one of the local hackathons.. We had a whole team of folks, some of the Flomio hardware people, and other local startup folks. Never was able to build the prototype at a hackathon, but always thought there is a physiological element to learning, and ed tech in general.
We know that mild stress can narrow focus and help learning. "Neural connections do form when you struggle." [0] A little stress is not only good but necessary for learning while a lot of stress causes a breakdown. The trick is being able to judge the stress level of the student and every person is different.
The idea of using some type of bio metric data from EEG or blood pressure sensors will help determine a little struggle from a lot of struggle in formative assessment.
I don’t understand what you mean by “we don’t know”? Are you saying tech entrepreneurs don’t know education? I agree with that. Or are you saying we don’t know what problem to solve?
My interpretation of “we don’t know” is that the space of possible instruction methods is huge, and we don’t know what works best. It is easy to think of it as a kind of optimization problem, but that line of thinking quickly runs into a bunch of practical obstacles: different people learn differently, studies with good experimental design take time and resources so it is slow to iterate, and good teaching seems to require deep insights into student motivations and incentives, aspects of teaching that are hard to quantify. We also don’t have great ways to measure outcomes; grades are weakly predictive of future success.
The best teachers seem to combine deep thinking about how to teach with intuition and charisma. The ingredients are hard to replicate.
I do think that this is a nut that we will eventually crack, but it is proving to be a very tough nut.
I’ll give him the benefit of the doubt and hope for the best.
My wife has been a high school math teacher for 25 years and has seen waves of tech luminaries give their visions of how tech will radically improve education and learning. She can’t point to any startup that has had a major impact in improving outcomes. I wish I could point her to something that really will have an impact in, or even out, of the classroom for her students.
That's not unique to tech startups. Almost nothing in terms of explicit methodology has any effect on education outcomes. The literature on education research is of abysmal quality too.
There is not a huge market of people who actually want to learn something on their own motivation. Get some piece of paper, yes. So the huge market will only materialize if the degree granting institutions adopt something like this.
I'm working on a much smaller idea, but so far it's just me using it for grad school, and a friend using it for his master ciccerone prep. It has been really useful in the class I used it for while building it. Looking forward to playing with more as I start my fall semester(UIUC CS 425, Distributed Systems).
I need to get some model pricing evals in place to see if I can use open weight models to get it cheap enough, since it's just me and not VC money. The real product potential seems to be getting reusable materials like cert prep, continuing education for careers that require it, and the like. But then you're fighting incumbents that fiercely control their moats.
Ive actually started trying to build games, starting with Roblox, to eventually build a catalog for this same problem. 1:1 learning in the future is probably going to be 80 - 90% gamified because people like playing games. Spent the last 6 months traveling Asia and everybody was playing games on their phones...everywhere. The first game I released is Math Cadet, basically an adventure game to build up math skills for kids elementary and middle school. I have another one in the chamber thats more science related. Any feedback would be great: https://www.roblox.com/games/129132556457079/Math-Cadet-Adve...
I do think Andrew is the right person doing this. Someone mentioned that the return of financial outcome is not good in edTech, and this is exactly why backing by Coursera is a huge advantage. Education-related product is slow, and it needs patience.
The concern, though, is that how to achieve those 3 thesis. Do they train/fine-turn their own model? according to my own experience, the current models are not well trained in personal education context, and it is either too textbook-like or too task-driven.
So the founder of Coursera is starting a new venture backed by Coursera ? That sounds like the Musk playbook.
I don't really get how this is new or even "AI". Can't you already ask Claude to teach you something ?
"Patiently stays with you until you've mastered new skills", that's so nice. Maybe in the future it will be the other way around, your computer will complain about how slow the user is :)
The AI sloppification is palpable.
Anywho, curious to see what comes out of this! EdTech & AI is a really promising space.
So people will replace AI?
These look like system and agent prompts and agent instructions: "Academic Assistant Pro.md", "Code Tutor" https://github.com/linexjlin/GPTs/blob/main/prompts/
Gemini Notebook does flashcards, study guides: https://edu.google.com/ai-gemini-notebook/
Spaced Repetition flashcards or flashquestions to progress through a game might be good; configure Anki decks to surface cards from when for example the user jumps to crush a bonus block.
"Ask HN: How can I learn to read mathematical notation?" (2018) https://news.ycombinator.com/item?id=18511718
"[Feature Request]: flashcards to advance in the game and learning mode" Issue #3054 ; SuperTux/supertux https://github.com/SuperTux/supertux/issues/3054
Yeah, prominent thing run by famous guy … but why so much funding so early?
What does $100M enable in the next 5-10 years that $25M doesn’t?
Perhaps I am a bit more optimistic, but if AI could give me, let's say, 90% of an average lecturer's capability, at 10% of the price or even less, that is probably extremely valuable. (For one, I would certainly be interested in learning new things that otherwise wouldn't make sense financially)
But I did not learn everything I wanted to know. And I can’t afford to take a time-out to go to grad school again for a second PhD. On the other hand, an LLM, in some sense, has access to the world’s knowledge. With some patience and persistence, you can mine almost-expert-level instruction from one.
When ChatGPT 3.5 came out, I excitedly tried this for a topic that I WAS an expert in. The results were discouraging. The model’s answers seemed to reflect the general misunderstandings that people had about the topic. More recently though, I had to review a paper in the same area, and I used a late-model ChatGPT to help check my work. It was an eye-opening experience because it was no longer confused. And it found longstanding misunderstandings I had, buttressing its answers against my skepticism with citations to original work. I came away very impressed. This kind of AI “rubber duck programming” is my preferred style of use now. I used it just today to help me learn an area of statistics I have always been fuzzy about. This approach definitely requires some careful prompting, but I am optimistic that AI tutors will one day be a real thing. My only worry is that people lose the ability to understand what makes an answer a good one and why we should care about good answers.
This is of course about the PhD level and beyond. For regular university-level education, where the goal is to pass the usual exams, ChatGPT and Claude are more than enough. But for real-world "doing", not (yet).
Given how much people spend on education, there’s no reason AI education software wont be a huge market.
And there’s few people better suited than Andrew Ng to execute this
At that point they had ~24 employees. He said his interest was crowdsourcing and thought most of the engineering effort would go toward the crowdsourced translation, but at that time, it was being handled by just two employees.
They brought on two prominent consultant researchers, leaders in the field of language learning and linguistics. The first question they asked the consultants was which part of speech should they present to the learner first. The consultants didn't have an answer. They decided to dedicate engineers to collecting data in order to answer that question.
A big problem with edTech is that we don't know.
We know that mild stress can narrow focus and help learning. "Neural connections do form when you struggle." [0] A little stress is not only good but necessary for learning while a lot of stress causes a breakdown. The trick is being able to judge the stress level of the student and every person is different.
The idea of using some type of bio metric data from EEG or blood pressure sensors will help determine a little struggle from a lot of struggle in formative assessment.
[0] https://www.youtube.com/watch?v=wh0OS4MrN3E
The best teachers seem to combine deep thinking about how to teach with intuition and charisma. The ingredients are hard to replicate.
I do think that this is a nut that we will eventually crack, but it is proving to be a very tough nut.
My wife has been a high school math teacher for 25 years and has seen waves of tech luminaries give their visions of how tech will radically improve education and learning. She can’t point to any startup that has had a major impact in improving outcomes. I wish I could point her to something that really will have an impact in, or even out, of the classroom for her students.
I need to get some model pricing evals in place to see if I can use open weight models to get it cheap enough, since it's just me and not VC money. The real product potential seems to be getting reusable materials like cert prep, continuing education for careers that require it, and the like. But then you're fighting incumbents that fiercely control their moats.
https://studyengine.app/
The concern, though, is that how to achieve those 3 thesis. Do they train/fine-turn their own model? according to my own experience, the current models are not well trained in personal education context, and it is either too textbook-like or too task-driven.
I don't really get how this is new or even "AI". Can't you already ask Claude to teach you something ?
"Patiently stays with you until you've mastered new skills", that's so nice. Maybe in the future it will be the other way around, your computer will complain about how slow the user is :)