College professors are going back to paper exams and handwritten essays to fight students using ChatGPT

Really sucks for people with disabilities and handwriting issues.

Here's a somewhat tangential counter, which I think some of the other replies are trying to touch on ... why, exactly, continue valuing our ability to do something a computer can so easily do for us (to some extent obviously)?

In a world where something like AI can come up and change the landscape in a matter of a year or two ... how much value is left in the idea of assessing people's value through exams (and to be clear, I'm saying this as someone who's done very well in exams in the past)?

This isn't to say that knowing things is bad or making sure people meet standards is bad etc. But rather, to question whether exams are fit for purpose as means of measuring what matters in a world where what's relevant, valuable or even accurate can change pretty quickly compared to the timelines of ones life or education. Not long ago we were told that we won't have calculators with us everywhere, and now we could have calculators embedded in our ears if wanted to. Analogously, learning and examination is probably being premised on the notion that we won't be able to look things up all the time ... when, as current AI, amongst other things, suggests, that won't be true either.

An exam assessment structure naturally leans toward memorisation and being drilled in a relatively narrow band of problem solving techniques,¹ which are, IME, often crammed prior to the exam and often forgotten quite severely pretty soon afterward. So even presuming that things that students know during the exam are valuable, it is questionable whether the measurement of value provided by the exam is actually valuable. And once the value of that information is brought into question ... you have to ask ... what are we doing here?

Which isn't to say that there's no value created in doing coursework and cramming for exams. Instead, given that a computer can now so easily augment our ability to do this assessment, you have to ask what education is for and whether it can become something better than what it is given what are supposed to be the generally lofty goals of education.

In reality, I suspect (as many others do) that the core value of the assessment system is to simply provide a filter. It's not so much what you're being assessed on as much as your ability to pass the assessment that matters, in order to filter for a base level of ability for whatever professional activity the degree will lead to. Maybe there are better ways of doing this that aren't so masked by other somewhat disingenuous goals?

Beyond that there's a raft of things the education system could emphasise more than exam based assessment. Long form problem solving and learning. Understanding things or concepts as deeply as possible and creatively exploring the problem space and its applications. Actually learn the actual scientific method in practice. Core and deep concepts, both in theory and application, rather than specific facts. Breadth over depth, in general. Actual civics and knowledge required to be a functioning member of the electorate.

All of which are hard to assess, of course, which is really the main point of pushing back against your comment ... maybe we're approaching the point where the cost-benefit equation for practicable assessment is being tipped.

1. In my experience, the best means of preparing for exams, as is universally advised, is to take previous or practice exams ... which I think tells you pretty clearly what kind of task an exam actually is ... a practiced routine in something that narrowly ranges between regurgitation and pretty short-form, practiced and shallow problem solving.

Case in point, ALL students on my course with low (<60%) attendance this year scored 70s and 80s on the coursework and 10s and 20s in the OPEN BOOK exam. I doubt those 70s and 80s are real reflections of the ability of the students

I get that this is a quick post on social media and only an antidote, but that is interesting. What do you think the connection is? AI, anxiety, or something else?

Graduated a year ago, just before this AI craze was a thing.

I feel there's a social shift when it comes to education these days. It's mostly: "do 500 - 1,000 word essay to get 1.5% of your grade". The education doesn't matter anymore, the grades do; if you pick something up along the way, great! But it isn't that much of a priority.

I think it partially comes from colleges squeezing students of their funds, and indifferent professors who just assign busywork for the sake of it. There are a lot of uncaring professors that just throw tons of work at students, turning them back to the textbook whenever they ask questions.

However, I don't doubt a good chunk of students use AI on their work to just get it out of the way. That really sucks and I feel bad for the professors that actually care and put effort into their classes. But, I also feel the majority does it in response to the monotonous grind that a lot of other professors give them.

I recently finished my degree, and exam-heavy courses were the bane of my existence. I could sit down with the homework, work out every problem completely with everything documented, and then sit to an exam and suddenly it's "what's a fluid? What's energy? Is this a pencil?"

The worst example was a course with three exams worth 30% of the grade, attendance 5% and homework 5%. I had to take the course twice; 100% on HW each time, but barely scraped by with a 70.4% after exams on the second attempt. Courses like that took years off my life in stress. :(

We have no way to determine if you did the work, or if an AI did, and if called into a court to certify your expertise we could not do so beyond a reasonable doubt.

Could you ever though, when giving them work they had to do not in your physical presence? People had their friends, parents or ghostwriters do the work for them all the time. You should know that.

This is not an AI problem, AI "just" made it far more widespread and easier to access.

Student here - How does that cursive longhand thing go again?

Is AI going to go away?

In the real world, will those students be working from a textbook, or from a browser with some form of AI accessible in a few years?

What exactly is being measured and evaluated? Or has the world changed, and existing infrastructure is struggling to cling to the status quo?

Were those years of students being forced to learn cursive in the age of the computer a useful application of their time? Or math classes where a calculator wasn't allowed?

I can hardly think just how useful a programming class where you need to write it on a blank page of paper with a pen and no linters might be, then.

Maybe the focus on where and how knowledge is applied needs to be revisited in light of a changing landscape.

For example, how much more practically useful might test questions be that provide a hallucinated wrong answer from ChatGPT and then task the students to identify what was wrong? Or provide them a cross discipline question that expects ChatGPT usage yet would remain challenging because of the scope or nuance?

I get that it's difficult to adjust to something that's changed everything in the field within months.

But it's quite likely a fair bit of how education has been done for the past 20 years in the digital age (itself a gradual transition to the Internet existing) needs major reworking to adapt to changes rather than simply oppose them, putting academia in a bubble further and further detached from real world feasibility.

Sorry but it was never about OUR abilility in the firts place.

In my country exams are old, outdated and often way to hard. In my country all classes are outdated and way to hard. It often feels that we are stucked in the middle of the 20th century.

You have no change when you have a disability. When you have kids, parents to take care of. Or hell: you have to work, cause you can't effort university otherwise.

So i can totaly understand why students feel the need to use AI to survive that torture. I don't feel sorry for an outdated university system.

When it is about OUR abilility, then create a System that is for students and their needs.

Millennial here, haven't had to seriously write out anything consistently in decades at this point. There's no way their handwriting can be worse than mine and still be legible lol.

You'd be surprised. My daughter (13) has better penmanship than I do (46). Although I'm sure my left-handedness doesn't help there.

I have horrible penmanship as a millennial, I was typing from around 6 on

We get in trouble if we fail everyone because we made them do a novel synthesis, instead of just repeating what we told them.

Particularly for an intro course, remembering what you were told is good enough.

My favourite lecturer at uni actually did that really well. He also said the exam was small and could be done in about an hour or two but gave us a 3 hour timeslot because he said he wanted us to take our time and think about each problem carefully. That was a great class.

IME, a lot of professors liked to write exams that specifically didn't cover anything from the PowerPoint presentations lol

Same in Germany

Norway has been pushing digital exams for quite a few years, to the point where high school exams went to shit for lots of people this year because the system went down and they had no backup (who woulda thought?). In at least some universities most of or all exams have been digital for a couple years.

I think this is largely a bad idea, especially on engineering exams, or any exam where you need to draw/sketch or write equations. For purely textual exams, it's fine. This has also lead to much more multiple-choice or otherwise automatically corrected questions, which the universities explicitly state is a way of cutting costs. I think that's terrible, nothing at university level should be reduced to a multiple choice question. They should be forbidden.

Covid forced the transition to electronic exams in many areas.

The university I went to explicitly did in person written exams for pretty much all exams specifically for anti-cheating (even before the age of ChatGPT). Assignments would use computers and whatnot, but the only way to reliably prevent cheating is to force people to write the exams in carefully controlled settings.

Honestly, probably could have still used computers in controlled settings, but pencil and paper is just simpler and easier to secure.

One annoying thing is that this meant they also usually viewed assignments as untrusted and thus not worth much of the grade. You'd end up with assignments taking dozens of hours but only worth, say, 15% of your final grade. So practically all your grade is on a couple of big, stressful exams. A common breakdown I had was like 15% assignments, 15% midterm, and 70% final exam.

Not during class. Most likely a proctored exam. No laptops, no phones, teacher or proctor watching.

Only if you memorize it. In which case, why not just memorize the test material?

My first thought was exactly this.

Sounds like effort, I'm making a font out of my handwriting and getting a 3d printer to write it

You can always write down what gpt shows on the screen onto a paper

Hate to break it to you, but you picked probably the one law in physics that is empirically derived. There is no mathematical equation to derive newton's law of gravity.

Or you have the classic, "you can write anything down that you'd like, but it won't tell you how to answer the questions".

And, in fact, it doesn't help at all beyond a few formulas. I was ChemE, our cheat sheets never saved us.

So what you’re telling me is that written tests have, in fact, existed before?

What are you some kind of education historian?

I had some teachers ask for handwritten programming exams too (that was more like 20 years ago for me) and it was just as dumb then as it is today. What exactly are they preparing students for? No job will ever require the skill of writing code on paper.

Same. All my algorithms and data structures courses in undergrad and grad school had paper exams. I have a mixed view on these but the bottom line is that I'm not convinced they're any better.

Sure they might reflect some of the student's abilities better, but if you're an evaluator interested in assessing student's knowledge a more effective way is to make directed questions.

What ends up happening a lot of times are implementation questions that ask from the student too much at once: interpretation of the problem; knowledge of helpful data structures and algorithms; abstract reasoning; edge case analysis; syntax; time and space complexities; and a good sense of planning since you're supposed to answer it in a few minutes without the luxury and conveniences of a text editor.

This last one is my biggest problem with it. It adds a great deal of difficulty and stress without adding any value to the evaluator.

Noooo. That's a genious countermeasure without any obvious drawbacks!!1! /s

And forget about having any sort of integrity or explaining to kids why it's important for them to know how to do shit themselves instead of being wholly dependent on corporate proprietary software whose accessibility can and will be manipulated to serve the ruling class on a whim 🤦

Good luck doing one on one assessments in a uni course of 300+

It actually is artificial intelligence. What are you even arguing against man?

Machine learning is a subset of AI and neural networks are a subset of machine learning. Saying an LLM (based on neutral networks for prediction) isn't AI because you don't like it is like saying rock and roll isn't music

But then the investor wont throw wads of money at these fancy tech companies

Maybe machine learning models technically fit the definition of "algorithm" but it suits them very poorly. An algorithm is traditionally a set of instructions written by someone, with connotations of being high level, fully understood conceptually, akin to a mathematical formula.

A machine learning model is a soup of numbers that maybe does something approximately like what the people training it wanted it to do, using arbitrary logic nobody can expect to follow. "Algorithm" is not a great word to describe that.

Please let's not defame Djikstra and other Algorithms like this. Just call them "corporate crap", like what they are.

Training how to use "AI" (LLMs demonstrably possess zero actual reasoning ability) feels like it should be a seperate pursuit from (or subset of) general education to me. In order to effectively use "AI", you need to be able to evaluate its output and reason for yourself whether it makes any sense or simply bears a statitstical resemblance to human language. Doing that requires solid critical reasoning skills, which you can only develop by engaging personally with countless unique problems over the course of years and working them out for yourself. Even prior to the rise of ChatGPT and its ilk, there was emerging research showing diminishing reasoning skills in children.

Without some means of forcing students to engage cognitively, there's little point in education. Pen and paper seems like a pretty cheap way to get that done.

I'm all for tech and using the tools available, but without a solid educational foundation (formal or not), I fear we end up a society snakeoil users in search of the blinker fluid.

That's just what we tell kids so they'll learn to do basic math on their own. Otherwise you'll end up with people who can't even do 13+24 without having to use a calculator.

There are some universities looking at AI from this perspective, finding ways to teach proper usage of AI. Then building testing methods around the knowledge of students using it.

Your point on checking for accuracy is on point. AI doesn't always puke out good information, and ensuring students don't just blindly believe it NEEDS to be taught. Otherwise wise you end up being these guys... https://apnews.com/article/artificial-intelligence-chatgpt-courts-e15023d7e6fdf4f099aa122437dbb59b

It's like the calculator in the 80s and 90s. Teacher would constantly tell us "no jobs just gonna let you use a calulator,, they're paying you to work"..

I graduated, and really thought companies were gonna make me do stuff by hand, cause calulators made it easy. Lol.

That's actually something that is done (PhD viva). If I had the budget to hire another 6 assistant profs to viva my 120 students, I'd probably do it for my module too!

I love this method and would use it if it weren't so incredibly time consuming. How are you supposed to test 30 students that way? Nevermind 300.

We still have orals in smaller seminars, and for PhDs. As another poster said there's too many students in most courses to do it, but we absolutely do oral exams for smaller cohorts.

This went away?

For this case you can get an assistant to write for you while you narrate

Me too. And on top of that, I'm left-handed, so any time I write, I get ink or graphite all over my hand.

Your comment is full of errors, interestingly enough...

You became the smartest kid because everyone else had a stroke trying to read what you wrote.

Universities have accomodation systems for issues like this. People with disabilities can go to the accomodations office and get what they need to be able to do the work

Big brain

Are you going to bring your 3d printer into your university's exam hall to write your final?

The problem with those programs is that you do not always know what the tests are going to be.

Also, it'd be hard to pull out the sheet if you have to store your bags away from you and have to empty your pockets.

Stuff Made Here did a great Video about this:

https://www.youtube.com/watch?v=cQO2XTP7QDw

Here is an alternative Piped link(s): https://piped.video/cQO2XTP7QDw

Piped is a privacy-respecting open-source alternative frontend to YouTube.

I'm open-source, check me out at GitHub.

Cool video if a bit impractical. Also teachers don’t have time to play detective with handwriting comparisons xD