fachat wrote:
I've recently noticed that the links in the datasheet archive section of 6502.org seem to point to the internet archiv web.archive.org.
I understand that this could be a measure to reduce bandwidth, but is this true, is this permanent?
No, this is not intended to be permanent.
In the
Git repository for the website, I have a SQLite database file which maps all of the PDF files in the documents archive to known-good copies on
archive.org. I originally did this so that the website can be run locally or can be recreated if something happens to me.
A few months ago, we were having problems where the entire documents archive was being downloaded frequently. The downloaders would sometimes put so much load on the server that the forum became unusably slow. Blocking IPs didn't help because whenever I blocked some, the bulk downloading would start again from new ones.
I started redirecting the document files to archive.org to help deal with this. I was hoping that the people doing the bulk downloading had good intentions and might stop if they realized everything was already backed up on archive.org. However, whenever I look in the logs, they don't seem to have let up. I'd like to restore serving directly from 6502.org and I think the long-term solution will be some kind of rate limiting where an IP address will be redirected to archive.org if it exceeds some reasonable number of downloads within a time period.