The official ChatGPT thread - the next big thing?

[Jeff Lebowski]

I swear Google Notebook LM has gotten dumber. I am trying to do some things that used to handle beautifully and the responses are terrible.

Also, has anyone else noticed that Claude has been off lately? Some days it is dumb and some days it is a genius. I see others commenting on the same thing.

Well, there you go. I was not hallucinating.

Embedded Tweet

[Donuthole]

The saving grace for all the humans whose jobs are supposedly going to be replaced by AI--eventually it gets lazy, too.

[PaloAltoCougar]

The saving grace for all the humans whose jobs are supposedly going to be replaced by AI--eventually it gets lazy, too.

Maybe the various AI platforms are telling each other to slow down, with ChatGPT (Jon Lovits) telling Claude (Tom Hanks) to knock it off.

[Northwestcoug]

This ain't good:

Got sore, itchy eyes? You’re probably one of the millions of people who spend too much time staring at screens, being bombarded with blue light. Rub your eyes too much and your eyelids might turn a slight, pinkish hue.

So far, so normal. But if, in the past 18 months, you typed those symptoms into a range of popular chatbots and asked what was wrong with you, you might have got an odd answer: bixonimania.

The condition doesn’t appear in the standard medical literature — because it doesn’t exist. It’s the invention of a team led by Almira Osmanovic Thunström, a medical researcher at the University of Gothenburg, Sweden, who dreamt up the skin condition and then uploaded two fake studies about it to a preprint server in early 2024. Osmanovic Thunström carried out this unusual experiment to test whether large language models (LLMs) would swallow the misinformation and then spit it out as reputable health advice. “I wanted to see if I can create a medical condition that did not exist in the database,” she says.

The problem was that the experiment worked too well. Within weeks of her uploading information about the condition, attributed to a fictional author, major artificial-intelligence systems began repeating the invented condition as if it were real.

Even more troublingly, other researchers say, the fake papers were then cited in peer-reviewed literature. Osmanovic Thunström says this suggests that some researchers are relying on AI-generated references without reading the underlying papers.

https://www.nature.com/articles/d41586-026-01100-y

[YOhio]

This ain't good:

https://www.nature.com/articles/d41586-026-01100-y

That's not good at all. But kudos to the research team. Super clever approach that will, hopefully, make AI better.

[Jeff Lebowski]

This ain't good:



https://www.nature.com/articles/d41586-026-01100-y

If they uploaded the fake papers to a pre-print server, how is the LLM supposed to know that it is fake? This seems contrived.

[Northwestcoug]

If they uploaded the fake papers to a pre-print server, how is the LLM supposed to know that it is fake? This seems contrived.

But that highlights an obvious problem. In this instance, AI can't judge between fraud and valid scientific literature. Presumably if a knowledgeable person would know 'bioxinomania' is fake and would sound the alarm.

[YOhio]

Reminds me of the Grievance Studies hoax a while back.

https://en.wikipedia.org/wiki/Grievance_studies_affair

[Jeff Lebowski]

But that highlights an obvious problem. In this instance, AI can't judge between fraud and valid scientific literature. Presumably if a knowledgeable person would know 'bioxinomania' is fake and would sound the alarm.

How would it know if it is fake? How would a human know if it is fake?

[Northwestcoug]

How would it know if it is fake? How would a human know if it is fake?

Again, highlighting the problem. Fakery abounds on the web. AI scours the internet looking for hits, trying to synthesize an answer. If it finds a fake link that looks legit and provides it as an answer, that is the 'calculator is right 99% of the time' problem.

The lay human can be easily fooled also. Hopefully if a fake diagnosis gains enough traction an expert would be able to debunk it.

[Jeff Lebowski]

Looks like the Mythos hype might be overblown.

Embedded Tweet

[YOhio]

Looks like the Mythos hype might be overblown.

Embedded Tweet

I really like Claude, but Anthropic is uniquely annoying.

[Jeff Lebowski]

Embedded Tweet

[YOhio]

I've been fooling around with Claude Design the past few days. My goodness it's impressive.

[Pelado]

I've been fooling around with Claude Design the past few days. My goodness it's impressive.

What does it do?