5mo ago

xkcd 1683

[xkcd 1683] Digital Data

cross-posted from: https://lemmy.dbzer0.com/post/36672698

Source: https://xkcd.com/1683/

196 @lemmy.blahaj.zone

xkcd 1683

Digital Data Rules

110 3

It's Me @lemm.ee

xkcd 1683

Digital Data

44 3

56 comments

- I'm doing my part
  
  Am I doing my part?

I'm doing my part
- I love how unoriginal the human brain is sometimes. I had the same exact thing I was about to comment

I've always thought that argument only works as long as data is free or close to free. Once it incurs a cost, I think copies end up getting removed. I think it's fundamentally flawed to say the internet will never forget.
- The media on the internet will all eventually be behind a paywall. It seems like we're heading in that direction.
  
  That seems to imply everything you're willing to pay for would still be accessible. That's just not the case I think. Things dissappear full stop, also if you do want to pay for it.
  A lot of non super popular, not very internationally known media eventually disappears into non-accessible copies in private collections: hard drives, non public accessible computers etc and at the same time becomes nearly impossible to purchase or otherwise retrieve online. For example public broadcasters in Europe: they don't want to put in the money and effort to preserve their entire archives, they don't make everything from the past accessible, things do get lost in their archives (sometimes as a conscious choice) and at the same time it is illegal for private people to archive it... until it is too late. For example lots and lots of radio plays are probably already lost forever.

Ironically this is original data we are viewing now.
- Well it wasn't even posted on your instance, so you're already just viewing a thirdhand copy of it
  
  Despite that its still the same actual bits of data

Well, for thot pics, there’s always more jpeg. For everything else, there’s lossless data formats.
- Even with jpeg, you only lose data each time it's encoded. If you save the file instead of taking a screenshot, the quality remains the same.
  That said, I don't know if there's a digital storage method widely used that will last longer than a book without some sort of active aspect to the storage (like copying the files to a new medium every now and then).
  I think punch cards are one that can, but they aren't used much anymore due to poor density and speed, plus being susceptible to literal bugs. It's possible to encode digital information into carved rock, but that would also have density issues (higher density means less reliability because the amount of damage required to make it unreadable is lower).
  I think there's a good chance that a lot of the knowledge we have today could be lost entirely if civilization collapses to a certain degree just due to how we store it.
  
  I think there’s a good chance that a lot of the knowledge we have today could be lost entirely if civilization collapses to a certain degree just due to how we store it.
  We do have some backups.
  https://archiveprogram.github.com/arctic-vault/
  
  You’re right, the format’s integrity is only as good as the medium upon which it’s stored. Hard disks are really only good for a few decades if left untouched. Punchcards maybe a few thousand years if sealed up well.
  The “active aspect” you mentioned is the key. There are file storage systems which employ regenerative error correction to achieve exactly this sort of desired outcome. I use one on my home server called ZFS. It was originally developed by Sun Microsystems and works great. The only catch is that there is a limit to the number of drives in your storage array which can fail before data becomes unrecoverable. So, you have to be constantly vigilant, and if a drive is starting to go, replace it before a potential worst case scenario of cascade failure.
  Unfortunately, I don’t know of a way we could store something indefinitely without this kind of active monitoring and occasional TLC. If a sort of caretaker is required, this might be a good job for AI with real world robotic hooks - have it monitor the array and fabricate replacement drives for installation as needed.
  
  We have stone tablets from back when humans invebted written language. I vote we back up critical data using this method.
  
  There's M-discs which are supposed to last for 1000 years, but obviously nobody has fully tested that yet. They're readable by normal DVD and Blu-ray players and go up to 100GB capacity.
  There's LTO tapes as well of course, but they're not rated to last for as long (only 30ish years).

Good luck finding the raw original video of anything these days. The amount of 3gp an rm files that used to float around compared to the reactionary emoji text bs you see today. Get off my lawn.

Trying not to spoil it but this is a plot point in a relatively famous, relatively recent sci-fi book, where the characters need to record a warning that lasts for millennia. They end up carving it into the rock of Pluto since all other data mediums would fail over that timespan.
- That sounds like a great book, if only I new what it was called.
  
  Three Body Problem
- I do not know of this movie, but your description reminded me of the most excellent read Deep Time, by Gregory Benford. Didn't consider other planets though, as this is actually a non-fiction work.

Only downside is that only data that people care about right now is being saved. But what seems useless now might become valuable in the future. It's hard to grasp how much data has been forgotten on some old computers, or some CDs, or websites that have gone dark.
- It's hard to imagine how much data is lost on old notepads, journals, even personal voice recordings.
  
  I want my journal to die with me. It's got a lot of painfully honest stuff that could hurt the people I love.
- I really hope the Internet Archive survive their lawsuits. They're important not just for archival of websites (the Wayback Machine) but also of books, audio recordings, etc. There's a large amount of old content that's only archived at the Internet Archive.

About that... we could record someone's every word and different people would read entirely different things into it. Consider how strangers have reacted to your own internet comments.
- No vehicles in the park

Needs more jpeg.
- Needs more 9gag logos

Storing data for decades or even centuries is a difficult thing. But the problem isn't the storage it's the data format!
Who knows if a person 300 years from now has a program that can open .png or .jpg? Or the dreaded .doc and .xls that even Microsoft has problems with today. This poor future fellow probably won't have the capatibilities and might need a few years or decades to develop a reader app.
- .txt ftw
- You could archive a description of the file format alongside the files. Maybe a pseudocode implementation too, or actual code (although who knows which programming languages will exist 300 years from now).
  Or the dreaded .doc and .xls that even Microsoft has problems with today.
  The US Library of Congress recommends archiving data in SQLite databases, since it's a simple, well-documented format, SQLite is public domain, and SQLite devs have promised to support it for a long time and retain backwards compatibility indefinitely.
  CSV and TSV are okay too of course, but it's often much easier to deal with large datasets if they're in an actual database format.

See, that's why I started using JPEG-XL for long-term storage. Apart from being better in every aspect for lossless and near-lossless still images than any competitor, the generation loss even over 1000 lossy save and load cycles is negligible.
- But converting from a format to another is a lossy process. It's best to just keep whatever original format you have, unless you are creating the images yourself.
  
  I believe jpeg xl allows lossless conversion from common jpeg
- That really doesn't matter when someone screenshots your JPEG-XL and posts it in a website that transcodes it to WEBP and adds a water mark.

I like how /u/gofsckyourself didn’t show up with a higher quality version.

9gag
That's a name I haven't heard in a while.

56 comments