Web Applications Stack Exchange is a question and answer site for power users of web applications. It only takes a minute to sign up.
Sign up to join this community
When I try to access http://archive.is/ with DNS through 1.1.1.1, it doesn't work. Why?
% dig +nocmd +nocomments @one.one.one.one archive.is
;archive.is. IN A
archive.is. 49050 IN A 127.0.0.3
;; Query time: 139 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Thu Oct 3 17:12:50 2019
;; MSG SIZE rcvd: 44
%
archive.today had this to say about the issue:
https://twitter.com/archiveis/status/1017902875949793285
2018-07-13T1545: yes, unlike other public DNS services, 1.1.1.1 does not support EDNS Client Subnet
https://twitter.com/archiveis/status/1018691421182791680
2018-07-15T1958: "Having to do" is not so direct here. Absence of EDNS and massive mismatch (not only on AS/Country, but even on the continent level) of where DNS and related HTTP requests come from causes so many troubles so I consider EDNS-less requests from Cloudflare as invalid.
From a technical perspective, the claim could easily be verified by running the following commands; one can notice that the vast majority of public resolvers, other than 1.1.1.1, indeed do provide an "edns0-client-subnet XX.XX.XX.0/24" answer, which is necessary in order for the various CDN functionalities to work their best.
% dig +nocmd @dns.google. -t txt o-o.myaddr.l.google.com +nocomments +noall +answer +stats
o-o.myaddr.l.google.com. 59 IN TXT "172.217.34.2"
o-o.myaddr.l.google.com. 59 IN TXT "edns0-client-subnet XX.XX.XX.0/24"
;; Query time: 28 msec
;; SERVER: 8.8.4.4#53(8.8.4.4)
;; WHEN: Thu Oct 3 17:41:29 2019
;; MSG SIZE rcvd: 113
% dig +nocmd @resolver1.opendns.com. -t txt o-o.myaddr.l.google.com +nocomments +noall +answer +stats
o-o.myaddr.l.google.com. 60 IN TXT "2620:0:cc7::68"
o-o.myaddr.l.google.com. 60 IN TXT "edns0-client-subnet XX.XX.XX.0/24"
;; Query time: 20 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Thu Oct 3 17:41:32 2019
;; MSG SIZE rcvd: 115
% dig +nocmd @one.one.one.one. -t txt o-o.myaddr.l.google.com +nocomments +noall +answer +stats
o-o.myaddr.l.google.com. 60 IN TXT "162.158.82.18"
;; Query time: 23 msec
;; SERVER: 1.0.0.1#53(1.0.0.1)
;; WHEN: Thu Oct 3 17:41:42 2019
;; MSG SIZE rcvd: 67
% host 162.158.82.18
Host 18.82.158.162.in-addr.arpa not found: 2(SERVFAIL)
%
Even the less popular public resolvers that don't provide ECS, still qualify under archive.today's exception where the geolocation of the resolver is trivial to determine in a programmatic way:
% dig +nocmd @a.resolvers.level3.net -t txt o-o.myaddr.l.google.com +nocomments +noall +answer +stats
o-o.myaddr.l.google.com. 60 IN TXT "8.0.18.0"
;; Query time: 14 msec
;; SERVER: 4.2.2.1#53(4.2.2.1)
;; WHEN: Thu Oct 3 19:24:44 2019
;; MSG SIZE rcvd: 62
% host 8.0.18.0
0.18.0.8.in-addr.arpa domain name pointer cns1.Frankfurt1.Level3.net.
%
% dig +nocmd @ordns.he.net -t txt o-o.myaddr.l.google.com +nocomments +noall +answer +stats
o-o.myaddr.l.google.com. 60 IN TXT "216.66.80.30"
;; Query time: 16 msec
;; SERVER: 74.82.42.42#53(74.82.42.42)
;; WHEN: Thu Oct 3 19:26:56 2019
;; MSG SIZE rcvd: 66
% host 216.66.80.30
30.80.66.216.in-addr.arpa domain name pointer tserv1.fra1.he.net.
%
If you're a savvy internet operator, it doesn't take long to see a conflict of interest at play as well.
Cloudflare's main line of business is as a Content Delivery Network, as well as associated services like DDoS-protection and bot hinderance.
To be most effective, they require their customers (website owners) to completely give up control over the technical setup of their website. For example, this includes a mandatory requirement of delegating your domain name, example.org, to a unique set of cloudflare.com. nameservers — Cloudflare does not allow their customers to make any assumptions about any IP addresses of any services at all — no IP address hardcoding. This applies not just to HTTP/HTTPS servers, but also to the authoritative DNS as well.
Basically, unlike Linode and HE.net, they at Cloudflare don't even let you whitelabel their NS servers for free (i.e., use Cloudflare's IP addresses with your own domain name, like ns1.example.org. if you're the owner of example.org); this is done in order for Cloudflare to have the maximum and complete control over all available DoS remediation techniques, to be able to change any IP address of any service as seen by any client at any given time, as well as to facilitate request tracking for data collection, machine learning and traffic anomaly analysis.
As such, with their new 1.1.1.1 service, free to end users, and subsidised out of their massive CDN business, Cloudflare's decision to deny their competitors and non-customers from having access to the very same level of information for their decision making that Cloudflare itself always has had access to — archive.today runs their own CDN network here — doesn't seem exactly like a level-playing field.
This effectively forces operators like archive.today to either succumb to DoS attacks by not having all the tools available at their disposal to protect themselves against such attack (by giving misbehaving clients or subnets a distinct name resolution, as well as doing anomaly detection), or to become a Cloudflare CDN customer — how convenient for Cloudflare!
Cloudflare touts their decision to omit EDNS Client Subnet as a privacy initiative (which is a rather disingenuous claim, as ECS is only specific to a /24 (/56 with IPv6), and after the DNS resolution is complete, you'd still have to issue your HTTP/HTTPS request from your own IP address anyways), but I think it's easy to read between the lines that the only known monetisation from 1.1.1.1 would be tracking, machine-learning and upselling of Cloudflare's CDN offering.
Failing to provide EDNS Client Subnet makes it significantly more difficult for someone like archive.today to do the exact same things in their own CDN that Cloudflare itself enjoys on doing in their acclaimed commercial offering.
Does archive.is have a CoI as well? Perhaps. Bot-hindering CDNs like Cloudflare has had a net-positive effect for website owners at the price of a significant net-negative effect on certain less common internet users, where some unlucky ones may now be required to solve captchas on a daily basis all day long. Obviously, it's easy to see how Cloudflare's oblivious captchas may have a significant negative effects on website archiving, too.
Nothing is exactly black and white, so, I'll leave you the reader to form your own conclusion.
This is the statement from the CEO & co-founder of CloudFlare on Hacker News in May 2019 about this issue:
We don’t block archive.is or any other domain via 1.1.1.1. Doing so, we believe, would violate the integrity of DNS and the privacy and security promises we made to our users when we launched the service. Archive.is’s authoritative DNS servers return bad results to 1.1.1.1 when we query them. I’ve proposed we just fix it on our end but our team, quite rightly, said that too would violate the integrity of DNS and the privacy and security promises we made to our users when we launched the service.
The archive.is owner has explained that he returns bad results to us because we don’t pass along the EDNS subnet information. This information leaks information about a requester’s IP and, in turn, sacrifices the privacy of users. This is especially problematic as we work to encrypt more DNS traffic since the request from Resolver to Authoritative DNS is typically unencrypted. We’re aware of real world examples where nationstate actors have monitored EDNS subnet information to track individuals, which was part of the motivation for the privacy and security policies of 1.1.1.1.
EDNS IP subsets can be used to better geolocate responses for services that use DNS-based load balancing. However, 1.1.1.1 is delivered across Cloudflare’s entire network that today spans 180 cities. We publish the geolocation information of the IPs that we query from. That allows any network with less density than we have to properly return DNS-targeted results. For a relatively small operator like archive.is, there would be no loss in geo load balancing fidelity relying on the location of the Cloudflare PoP in lieu of EDNS IP subnets.
We are working with the small number of networks with a higher network/ISP density than Cloudflare (e.g., Netflix, Facebook, Google/YouTube) to come up with an EDNS IP Subnet alternative that gets them the information they need for geolocation targeting without risking user privacy and security. Those conversations have been productive and are ongoing. If archive.is has suggestions along these lines, we’d be happy to consider them.
This is a very interesting question, and I've given it a lot of thought.
The one sentence answer is "because archive.is blocks DNS requests from Cloudflare's datacenters". That of course leads to the core question: "why?".
This behavior of archive.is is especially intriguing to me because archive.is seems to have an emphasis on being anti-censorship and being resilient, but blocking 1.1.1.1 users appears antithetical to that. cnst's answer is definitely interesting and plausible, but I've come up with my own pet hypothesis that's somewhat similar.
Let's say you operate a website, and must censor illegal content. You might come up with 3 rules:
If content is illegal in country X, that content must not be served to users in country X.
If content is illegal in country X, that content must never touch servers in country X. So it must not be stored on those servers, nor be proxied through those servers.
If content is legal in country X, we will attempt (within reason) to make that content available to users in country X.
Let's have a simple example: you have a piece of content A that is illegal in every country in the world except country X. How can we operate our website within the stated rules? This is fairly simple, put all our servers in X, and if a request for A comes from country X, serve it. If a request for A comes from country Y, give a 404.
Now let's have a more complicated example: you have a piece of content A that is illegal in every country in the world except country X. You have a piece of content B that is illegal in every country in the world except country Y. Now there's no longer a simple solution. But here's a complicated solution:
Operate servers in X that have A but not B. If servers in X receive a request for A from X, serve it. If servers in X receive a request for A from outside X, give a 404. And similarly for Y and B. If any servers receive a request for content they don't have they give a 404.
Operate a custom authoritative DNS server for your site. If it receives a DNS request with the EDNS as an IP in X, respond with the IP address of a server in X. If it receives a DNS request with the EDNS as an IP in Y, respond with the IP address of a server in Y. If it receives a DNS request with the EDNS as an IP in some non-X and non-Y country, arbitrarily choose a server IP address to respond with.
If you try to actually implement that solution, you will have a problem: some DNS resolvers (such as 1.1.1.1) don't give you the EDNS, so it's not possible. So archive.is might have done this, then realized that it fails for some DNS resolvers, and so rather than have a semi-broken site, they decided to block users who use those resolvers. There is still the open question though about why block only some EDNS-less resolvers (such as 1.1.1.1) and not other EDNS-less resovers.
I've found some evidence in favor of this explanation of archive.is's behavior.
Archive.is runs some special DNS servers on Linode and DigitalOcean. When Linode complained that these DNS servers were used in some way to help distribute controversial content, archive.is defended themselves by saying those servers are just DNS servers, content never touches those servers. This defense uses the same reasoning as rule 2 from above, namely that what matters is whether the content touches the specific servers.
Additionally, in that same conversation, Linode complained that archive.is was doing blocking based on user IP. Archive.is responded that yes, they do censorship based on the country the user is in. This is very similar behavior to what I proposed in the complex solution. As an interesting side note, archive.is viewed this type of censorship as improving legality by ensuring users didn't see banned content, whereas Linode viewed this exact same behavior as hurting legality, by obfuscating behavior, hiding evidence, and making it hard for Linode to investigate.
Archive.is says they are using modern deployment tools and the highly competitive cloud market to prevent wrongful takedowns, which could indicate a complicated setup with servers across many countries and some sort of orchestration system to manage them.
We don't know for certain archive.is's reasoning for their behavior because they haven't fully explained themselves. If their reasoning is legal-based, maybe they fear that explaining their legal setup will expose legal holes in their setup.
The gaps in explanation that archive.is has left provide an opportunity for some interesting technical and philosophical speculation.
