Texas is replacing thousands of human exam graders with AI

Texas sabotaging public education again? Color me shocked. No doubt the lower test scores will be used to justify privatizing more schools.

Also 3000 exam responses is luaghably low to train an LLM. These tests are for every 3rd-8th grader. That's less responses than you'd get from a single mid sized school - expected to train an LLM how to grade probably millions of answers across the entire state.

They claim its not an LLM because it doesn't learn as it goes. I'm fairly certain that's been the common implementation since we learned from the older generation of chatbots all turning to Nazis after being trolled by 4chan.

Ugh... I'm deep on the ai sphere, and this seems like a bad idea to me. Gpt (let's face it, they are probably using open ai) can be deeply biased and arbitrary in it's evaluations.

For example, "Two apples and four oranges," might score better than: "4 oranges and 2 apples." for inscrutable reasons. Say, if the question spelled out the numbers, and the LLM has a weighted bias to favor overall textual consistently, it might produces a reason to dock points apparently unrelated to that weight, such as: "incomplete sentence." for the second answer, but not the first.

Students may also receive lower scores due to cultural biases towards certain phrases, and factors as straightforward as their name.

Finally, AI will hallucinate errors constantly if you ask it to evaluate text without any errors. Constantly. Consistently.

This is good, actually. Teach the kids to game robots from a young age!

Hopefully itll be easier for the kids to cheat

A friend of mine used to mark undergrad papers, to be honest this would be a kindness to teachers.

I wonder to what sort of standard. I know I was shocked how poor things were when I started grading college students work as a TA. Same later in the work world reviewing nominations for an award.

🤖 I'm a bot that provides automatic summaries for articles: ::: spoiler Click here to see the summary Students in Texas taking their state-mandated exams this week are being used as guinea pigs for a new artificial intelligence-powered scoring system set to replace a majority of human graders in the region.

The STAAR exams, which test students between the third and eighth grades on their understanding of the core curriculum, were redesigned last year to include fewer multiple-choice questions.

According to a slideshow hosted on TEA’s website, the new scoring system was trained using 3,000 exam responses that had already received two rounds of human grading.

Some safety nets have also been implemented — a quarter of all the computer-graded results will be rescored by humans, for example, as will answers that confuse the AI system (including the use of slang or non-English responses).

While TEA is optimistic that AI will enable it to save buckets of cash, some educators aren’t so keen to see it implemented.

The attempt to draw a line between them isn’t surprising — there’s no shortage of teachers despairing online about how generative AI services are being used to cheat on assignments and homework.

Saved 63% of original text. :::