I'm a CS/AI teacher in an engineering school. A few days ago, towards the end of my course on convolutional neural networks, I asked my students to explain why tha first linear layer of the example PyTorch network had a specific number of neurons. This is a non-trivial question whose answer isn't directly available online (it depends on the input dimensions and the nature of all previous layers).
They struggled for a while, and the first student who gave the right answer explained how he did it. All morning, he interacted with ChatGPT while following my course, asking questions each time my own explanations weren't sufficient for him to understand. He managed to give the LLM enough context and information for it to spit not only the right answer, but also the whole underlying process to obtain it. In French, qui plus est ;)
This was for me an eye-opening, but also a bit unsettling experience. I don't use ChatGPT & co much for now, so this might seems pretty mundane to some of you. Anyway, I realized that during any lecture or lab, teachers will soon face (or are already facing) augmented students able to check and consolidate their understanding in real time. This is great news for education as a whole, but it certainly interrogates our current teaching model.
The fun side of ChatGPT is that if you probe it for information like this, it'll also generate complete fantasy. Without an expert to consult, the generated explanation may as well conclude the earth is flat and the sky is green.
No, this is not accurate in my trials. I use Claude.ai daily. If you ask questions on niche topics or dive down too deep, it says that resources on the topic are limited and you should consult a book.
I'm curious to hear more about this. I've seen very little hallucination with mainstream LLMs where the conversation revolves around concepts that were well-represented in the training data. Most educational topics thus have been quite solid. Even asking for novel analogies between distant and unrelated topics seem to work well.
I haven't messed with it in a few months but something that used to consistently cause problems was asking specific questions about hypotheticals where there may be a non-matching real example in the dataset.
Kind of hard to explain but for example giving a number of at-bats and hits for a given year for a baseball player and asking it to calculate their batting average from that. If you used a real player's name it would pull some or all of their actual stats from that year, rather than using the hypothetical numbers you provided.
I think this specific case has been fixed, and with stats-based stuff like this it's easy to identify and check for. But I think this general type of error is still around.
Thanks, that makes sense. I avoid using LLMs for math because it is only a text token prediction system (but a magical one at that), and can't do true numeric computation. But making it write code to compute works well.
I might be misunderstanding you, but the question you posed is all over the internet. First try, first page. It does not surprise me an LLM can “help” here.
My deeper issue with this tech is not its “knowledge”, it’s the illusion of understanding that I am afraid it fosters.
Lots of people will nod and agree when a competent teacher/mentor figure shows them something and walks them through it. They even think they understand. However, when given an actual new problem that they have to solve themselves without help they completely break down.
I am all for shallow learning as a hobby. Myself I love it, but I think it is dangerous if we misunderstand the nature of the problem here. Understanding is only partly based on consumption. A significant part of any craft is in the doing.
Take something like calculus. There are mountains of beautifully crafted, extraordinary videos on just about every nuance calculus has to offer and you can watch it all. It will give you a lot of concepts and this alone might be worth something but your time is better spent watching one or two videos and then practicing problems for hours.
My personal impulse was to grab to videos or books the moment I was stuck in my younger years. I now recognize how flawed this strategy was. Sure, it was “productive”. I got stuff “done”, but my knowledge was superficial and shallow. I had to make up for it later. By doing, you guessed it, a shit ton of exercises.
One thing I do appreciate is the availability of good quality content nowadays. Something like 3blue1brown is amazing and my university actually recommends watching his videos to supplement and ground your understanding.
No matter how many videos (or LLM podcasts) you consumed though, there is no way around “doing the work”.. as some painful questioning by any professional will quickly show you.
OP here: I definitely agree that shallow learning is an issue, and that it's an intoxicating effect. I've done it a few times — spent a few minutes learning a new topic, only to realize when I put it into practice that I'd been lied to.
But that's why it's critical to engage kids in this. There's a skill in using AI. Resisting the urge to take it at it's word, yet still using it for what it's good at. You can't build a skill without practice.
"Check and consolidate their understanding" by reading generated text that is not checked and has the same confident tone whether it's completely made-up or actually correct? I don't get it.
>interrogates our current teaching model
Jesus, many many things put our current teaching model in question, chatgpt is NOT one of them. Tbh this excitement is an example of focusing on the "cool new tech" instead of the "unsexy" things that actually matter.
> by reading generated text that is not checked and has the same confident tone whether it's completely made-up or actually correct? I don't get it.
This is a valid point, but it's referring to the state of things as of ~1.5 years ago. The field has evolved a lot, and now you can readily augment LLMs answers with context in the form of validated, sourced and "approved" knowledge.
Is it possible that you are having a visceral reaction to the "cool new tech" without yourself having been exposed to the latest state of that tech? To me your answer seems like a knee-jerk reaction to the "AI hype" but if you look at how things evolved over the past year, there's a clear indication that these issues will get ironed out, and the next iterations will be better in every way. I wonder, at that point, where the goalposts will be moved...
No, ChatGPT and others still happily make stuff up and miss important details and caveats. The goalpost hasn't moved. The fact that there are specialized LLMs that can fact check (supposedly) doesn't help the most popular ones which can't.
Have you tried Claude.ai. In my experience on computer science topics, the LLMs are very good. Because they have been trained on a vast amount of information online. I just had a nice conversation about mutexes and semaphores with claude and was able to finally grasp what they were.
I do not know if this is the case for example for mathematics or sciences.
>To me your answer seems like a knee-jerk reaction to the "AI hype" but if you look at how things evolved over the past year
It's not a kneejerk traction, like you said it's been 2 years of nonstop AI hype. I have used every chatbot model from openAI (3.5, 4, 4o, even o1) and a few from other companies as well. I've used code copilot tools. I've yet never not been disappointed.
> there's a clear indication that these issues will get ironed out, and the next iterations will be better in every way
On the contrary, there's NO indication of meaningful progress since the release of GPT 3.5. There's incremental progress, sure, as models get larger and larger and things get tweaked and perfected, but NO breakthrough and NO indication of an imminent one. Everything points to the fact that the current SotA, more or less, is at good as it gets with the transformer model.
> now you can readily augment LLMs answers with context in the form of validated, sourced and "approved" knowledge.
The student isn't an idiot, they'd use what the teacher says as their ground truth and chatgpt would be used to supplement their understanding. If it's wrong, they didn't understand it anyway, and reasoning/logic would allow them to sus out any incorrect information along the way. The teaching model can account for this providing them the checks to ensure their explanation/understanding is correct. (This is what tests are for, to check your understanding).
How is someone who is learning something supposed to figure out if what chatgpt is saying is bullshit or not? I don't understand this.
It's a kind of Gell-Mann effect. When I ask it a question of which I know the answer (or at least enough to understand if the answer is wrong) it fails miserably. Then I turn to ask if something which I don't know anything about and... I'm supposed to take it at its word?
You have what the teacher has told you as your primary correct reference point (your ground truth). It should align with that, if not the LLM is wrong.
Obviously the gaps between is where the issue would be but as I say the student can think this through (most lessons are built on previous foundations so they should have an understanding of the fundamentals and won't be flying in the dark).
The fact here is that a student, using ChatGPT, managed to give the right answer. And I agree with GP that the teaching model must evolve. The cat is out of the bag now and clearly students, of (unfortunately) almost all ages, are using it. It being "cool new tech" or anything else doesn't matter and as a teacher it must not be dismissed or ignored.
Not all subjects taught have to evolve in the same way. For example, it is very different to use ChatGPT to have a technical discussion than to simply ask it to generate a text for you. Meaning this tech is not having the same impact in a literature class and here in a CS one. It can be misused in both though.
I always come back to the calculator analogy with LLMs and their current usage. Here in the context of education, before calculators were affordable simply giving the right answer could have meant that you knew how to calculate the answer (not entirely true but the signal was stronger). After calculators math teachers were clearing saying "I want to see how you came up with the answer or you won't get any points". They didn't solve the problem entirely, but they had to evolve to that "cool new tech" that was clearly not helping there students learn as it could only give them answers.
I don’t know if you have been teaching, but I have (for nearly 19 years now ) to a lot of different people of various ages. I’m also a daily user of LLMs.
I’m firmly convince that LLMs will have an impact on teaching because they are already used in addition / superimposed on current classes.
The physical class, the group, has not been dislodged even after hundred of thousands of remote classes during confinement. Students were eager to come back, for many reasons.
LLMs have the potential to enhance and augment the live physical class. With a design school I teach at, we even have proposed a test program for a grant, where my History of Tech Design will be the in-vivo test ground of new pedagogical strategies using LLMs.
Tools that graft into the current way of teaching have had more impact that tools that promise to “replace university/schools”
I'm no LLM fanboy and I do know about their issues and shortcomings.
I also think that asking the right questions to a model while following a lecture, assessing its answers and integrating them into one's own reasoning is difficult. There is certainly a minimum age/experience level under which this process will generally fail, possibly hindering the learning outcome.
Nevertheless, I saw with my own eyes a mid-level student significantly improving his understanding of a difficult topic because he had access to a LLM in real time. I believe this is a breakthrough. Time will tell.
I don't know, seeming 'conversationally smarter' when having access to a language model is much different from just looking stuff up and pattern matching answers?
I'm afraid these models are making people sound smarter and feel smarter without any actual gains in real world problem solving skills.
Any smartwatch will become unusable, polluting garbage a few years (months?) from now: a canonical example of planned obsolescence. Their self-tracking functions are a double-edged sword, a source of stress as much as relief.
Any well-built and well-maintained mechanical watch will last you decades. No dependencies on electricity and network connectivity, it's a self-contained and entirely autonomous piece of human engineering. Mine was built in 1975 and is one year older than me. In a world where everything fades away so fast, wearing it everyday feels like owning a precious relic.
>No dependencies on electricity and network connectivity, it's a self-contained and entirely autonomous piece of human engineering.
This already veers straight back into the marketing territory that everyone in this thread remarks was an eye opener when they actually got a mechanical watch.
I have a mild prepper tendency and I had to eventually kill my romantic views of mechanicals when I realized it just time drift and wouldn't last long without regular maintenance from someone with the tools and knowledge/skill, not to mention someone in this very comment section mentions a mechanical watch suffering a death from drop onto carpeted floor.
Mechanical watches are cool, but I easily spend less time without my PineTime (which I'm surprised nobody else in these comments has even mentioned) working than my friend spends manually syncing his seiko back to time/maintaining it.
I never heard about PineTime until now! Looks like a cool gadget. What has your experience been with it, apart from it being more accurate than a mechanical?
It's an interesting experience, I have github send me an email on the odd occasion the community developed "OS" gets an update. Then I download the zip file on my phone browser and upload the file on Gadgetbridge for the update.
I sometimes call it my "soviet in a good way" watch, it ended up becoming my "function over fashion" watch, which means almost all day every day wear for a few years now.
pros:
decent battery life (1-2 weeks, i turn off bluetooth and gps on my phone overnights which helps both devices)
"good enough" design (durable enough for all but swimming/showering)
easily replaced or modified (even takes standard watch bands)
flashlight, notifications and all traditional digital watch functions
multiple community "OS" options
cons:
community development can be slow, buggy
water droplets particularly from natural rain can trigger the touchscreen, not amazing if you bike in seattle or something
anemic hardware
The charger is cheap and isn't that quick but again, the pinetime kind of excels in knowing the difference between good and good enough, as I once heard an engineer say (about something else), and I rarely find myself bothered by it's lack of luxuries.
>What do you think is the PineTime's biggest strength when compared with a mainstream smartwatch?
Frankly, I think the combination of price/replaceability and privacy are the only things unique (besides niche FOSS modding) to it among smart watches; and I like the open aspect of essentially every detail.
It makes it the only smart watch I've used that feels like it respects my dignity, frankly. A minor philosophical quibble but one I take stoic pleasure in. It is a tool, and technology that serves me, not another.
> Have you found the watch to be hackable? Is there any sort of customization that you've done to it?
I actually got it hoping I'd have the inclination to tinker with it, but my only idea that wasn't already being worked on by the default "OS" is a red flashlight mode, which with the IPS screen is a moot point anyways, since the black pixels when turned on make for a low-light flashlight anyways, heh. A hardware drawback that ironically makes it a more accessible tool in my experience.
The PineTime is still an electronic gadget that you won't use for very long.
My watch takes 15 seconds to rewind each day, and 5 seconds to be time adjusted by one minute twice a week. Service is every 5 years, the last one cost me 88 €. It gets more valuable each year, and I plan to bequeath it to my son in a (hopefully) very distant future.
Yes, it's manual. I have no experience with automatic ones.
Admittedly, it is a rather pricey model (Omega Speedmaster). I bought it second hand for 1500 € a few years ago. Unfortunately, prices have since skyrocketed for many emblematic watches like this one.
I beg to differ: for all its qualities, Eloquent JavaScript is not a very beginner-friendly book. It goes quite deep into many JS intricaties and contains a lot of challenging tasks for people just discovering programming.
I have recommended to beginners before and most of them thanked me. I skimmed the page for your book but did not see the link to the free version. I am on mobile so maybe just missed it. Says read free here, but here is not a link. Clicking the book takes you to Amazon.
The modern JS tutorial is indeed a very good and comprehensive resource.
For all its qualities, Eloquent JS is not a very beginner-friendly book. For a smoother learning curve, you might consider https://github.com/thejsway/thejsway.
If you don't have much programming experience, you should start with The JavaScript Way (https://github.com/thejsway/thejsway), which was written with beginners in mind (disclaimer: i'm the main author). Then you should study the YDKJSY series like many others have said.
IMHO, Eloquent JavaScript is a great book but not really beginner-friendly. It's a solid choice if you are already proficient with another language.
Nine months later, the book earned a little more than $3,000 in Leanpub royalties. Like other said, marketing is definitely the hardest part, especially if (like me) you had no initial audience. Since the book is free to read on GitHub, I hoped word-of-mouth would be enough to make it widely known and popular. Boy was I wrong.
The experience was worth it for a number of reasons, honing one's skills being the number one. That said, would-be authors without an existing follower base should prepare themselves for a multi-year commitment in order to reap any significant rewards.
They struggled for a while, and the first student who gave the right answer explained how he did it. All morning, he interacted with ChatGPT while following my course, asking questions each time my own explanations weren't sufficient for him to understand. He managed to give the LLM enough context and information for it to spit not only the right answer, but also the whole underlying process to obtain it. In French, qui plus est ;)
This was for me an eye-opening, but also a bit unsettling experience. I don't use ChatGPT & co much for now, so this might seems pretty mundane to some of you. Anyway, I realized that during any lecture or lab, teachers will soon face (or are already facing) augmented students able to check and consolidate their understanding in real time. This is great news for education as a whole, but it certainly interrogates our current teaching model.