there's still goofy magic sci-fi designs now, and there have always been more 'realistic' designs based on (to varying degrees) real scientific and engineering knowhow. the only way one could come to this conclusion would be by cherry-picking your examples. if you compare 'the jetsons' to 'mass effect', sure, it supports your conclusion, but on the other hand, contrasting 'rick and morty' against '2001: a space odyssey' would give a rather different conclusion.
I finished reading Dragon's Egg (1980) recently and at the back of the book was schematics, drawings, and layouts describing the various things in the novel.
There, indeed, have always been varying levels of "hardness" in sci-fi.
Yup. And in the past they didn't have as much scientific knowledge as we do now. So past sci-fi has a lot of concepts that seemed legit back then, but later we found out things simply don't work that way. And there's also ideas of things that would work, but we found better solutions that are more economically feasible later on.
A shuttlecraft that's basically a box with some cylinders strapped onto it made sense in the 1960s. Cylinders are like rockets, what we used for propulsion to get to space. Aerodynamics? Who cares?
But knowing what an actual space shuttle looks like makes people more likely to design something closer to that. And there's a greater understanding of the importance of aerodynamics when entering an atmosphere by everyone so people are less likely to buy into a new design that doesn't make these considerations unless there's some nostalgia involved.
I just want to say, i loved Dragon's Egg for this level of detail to the physics. I even did some quick calculations why you want 6 compensator masses not less to reduce the effect of tidal forces. Or the black holes inside the sun, at first i thougt, this is impossible. Then i read some more on it an noticed its well researched.
I usually love and prefer the goofy look of old SciFi, but seeing the brutalist, utilitarian aesthetics of the tech in the two newest Dune movies convinced me of that the new designs can be awesome when done right.