GPT and other large language models are aesthetic instruments rather than epistemological ones. Imagine a weird, unholy synthesizer whose buttons sample textual information, style, and semantics. Such a thing is compelling not because it offers answers in the form of text, but because it makes it possible to play text—all the text, almost—like an instrument.
This is pretty much where my head is on this stuff right now.
The hype and the anti-hype noise is really unfortunate, because the emergent capabilities are fascinating.
But, you know, sometimes a human on LSD also does interesting things. I just wouldn't trust them with heavy machinery or accept medical advice.