https://OpenStreetMap.org has been disrupted today.
-
https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI
Oof, sorry you are dealing with that load. Beyond blocklists, caching tiles plus aggressive rate limits per ASN, and serving 429s with Retry-After can help. For bulk use, requiring API keys for heavy endpoints can also cut abuse. Thanks for keeping OSM running.
-
https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI
@osm_tech Ugh, I am so sorry to hear this, and thank you to everyone working on this project.
Such a nightmare, and so completely pointless. 🤬
-
@ISibboI Correct. Not an ounce of brain. It has been going on for months, but it is just getting worse. Waste of their resources/time and ours.
-
https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI
@osm_tech@en.osm.town
Time to deploy Anubis in front of all the URLs that the scrapers are hitting. -
https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI
@osm_tech Good luck, and thank you as always.
-
https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI
@osm_tech wonder if it is collateral damage or intended, to get rid of alternative sources of information?
Anyway, zip-bomb the hell out of them. -
@osm_tech wonder if it is collateral damage or intended, to get rid of alternative sources of information?
Anyway, zip-bomb the hell out of them.@Don_Clemente Just dumb scrapers. They want our data, so maybe we /should/ redirect them to the latest full history planet file, it is only 245GB
-
@osm_tech@en.osm.town
Time to deploy Anubis in front of all the URLs that the scrapers are hitting.@gme Anubis is great, but not ideal for us. We have reasonable technical mitigations in place, but it is unfortunately an arms race (and time/resource blackhole)
-
@Don_Clemente Just dumb scrapers. They want our data, so maybe we /should/ redirect them to the latest full history planet file, it is only 245GB
@osm_tech yes, but that's heavy on your resources and bandwidth. A zip file of 10GB Zeros is much more resource-efficient on your side
-
-
@pesasa @ISibboI @osm_tech
If its helpful (if you want to stuff something unpleasant down their throats), I believe @jwz has been making a 'json bomb' for (AIUI) pretty much this purpose:
https://mastodon.social/@jwz/116049057703097965 -
https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI
@osm_tech @lehtimaeki I wonder if anyone has looked at the various AI browsers out there to see if they're used for active scraping.
-
@Don_Clemente Just dumb scrapers. They want our data, so maybe we /should/ redirect them to the latest full history planet file, it is only 245GB
@osm_tech @Don_Clemente Replacing the actual data with Null Island data that points to planet.osm.org is not a dumb idea IMHO
-
https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI
@osm_tech Block all read access except from registered users? And be restrictive on whom to grant an account (captcha, verified email address)?
-
https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI
-
-
@osm_tech yes, but that's heavy on your resources and bandwidth. A zip file of 10GB Zeros is much more resource-efficient on your side
@osm_tech @Don_Clemente Can confirm. >:D
-
@osm_tech Block all read access except from registered users? And be restrictive on whom to grant an account (captcha, verified email address)?
had a similar issue with a much, much smaller wiki server (
) but facing the same anonymous scrapers coming from thousands of random IP's.Couldn't find a way to keep them from crashing the server, so now access is by registration only.
Very disruptive, this "AI" race to the bottom destroyed the open internet
-
https://OpenStreetMap.org has been disrupted today. We're working to keep the site online while facing extreme load from anonymous scrapers spread across 100,000+ IP addresses. Please be patient while we mitigate and protect the service. #OpenStreetMap #DDoS #Scrapers #AI
@osm_tech I went to use OpenStreetMap today to get directions but it asked me to sign up/ log in, so I used Mapquest instead which doesn't.
-
@ISibboI Correct. Not an ounce of brain. It has been going on for months, but it is just getting worse. Waste of their resources/time and ours.
