Mirroring a particular web page?

Question

I have a particular page that I want to refer users who have DNS problems:

How do I diagnose not being able to reach a specific website as an end user?

However, there is a catch-22 -- users who are having DNS problems may not be able to reach our sites and read that page!

Thus, I need a reliable long-term mirror of this web page on another domain, either for free or as a paid service. Ideally, one that would periodically ping the source and keep it up to date with any changes as well.

I found some community built mirroring services to make sites "Digg-proof" or "Reddit-proof" but these are ad-hoc and not guaranteed to work for the long term.

Are there any webapps that offer reliable long term mirroring of individual web pages? Or any other webapp I can use to achieve this functionality?

Answer 1

The only semi-reliable thing I could think of was linking to the page in the google cache:

http://webcache.googleusercontent.com/search?q=cache%3Ahttp%3A%2F%2Fsuperuser.com%2Fquestions%2F231977%2Fhow-do-i-diagnose-not-being-able-to-reach-a-specific-website-as-an-end-user

But you can't control how frequently it's updated, etc. Also, not the prettiest of URIs (although there's always the shortened link). And the Google cached URI scheme has changed over the years. Maybe using the stock google.com query with the cache operator would be more reliable.

Answer 2

If your users are having DNS problems, they are unlikely to be able to resolve any host names at all. In that cause, no mirror will help, as any mirror still requires a DNS lookup to resolve (http://something/some-page) into an IP address.

So the fail safe way is to give the address raw (eg 164.34.119.12/page). However, this doesn't work with virtual hosts. In this case, the IP address that superuser.com resolves to is also used for stackoverflow, and the web server doesn't know which website the request is for.

So, starting with

superuser.com/questions/231977/how-do-i-diagnose-not-being-able-to-reach-a-specific-website-as-an-end-user

We can bypass a DNS lookup by using 64.34.119.12/questions/231977/how-do-i-diagnose-not-being-able-to-reach-a-specific-website-as-an-end-user ... but on a webserver with multiple sites, it will resolve to the default website (and in this case, will result in a page on stackoverflow).

Last step then: On the webserver, you would then have to set up proxy rules to redirect the request. By changing the url to something like 64.34.119.12/superuser/question, you can setup a rule on the webserver to foward any requests to the correct website.

Your resulting url then looks like http://64.34.119.12/superuser/questions/231977/how-do-i-diagnose-not-being-able-to-reach-a-specific-website-as-an-end-user

This will require a change in the setup of the server to redirect the url to the correct virtual host. It will also completly bypass any DNS problems.

Answer 3

Continuing on from the chat...

I think the best method would be some sort of status checker where if it detects the site is up, it can display helpful hints / a copy of the answer.

I have made a sample site that demonstrates what I mean - without any sort of style/Ajaxy effects. It can be viewed at http://ezpcinternal.com/v2.aspx

Answer 4

I usually use The Coral Cache, but I'm not sure what's its latency in the States.

Answer 5

Why not create a simple shell script that runs on a CRON job and creates a local mirror of the page automatically using wget's built-in mirroring feature?

Maybe something like:

#!/bin/bash
rm -rf ./mirror
wget -E -H -k -K -p -P ./mirror "http://superuser.com/questions/231977"

The "mirror" folder could be symlinked to a folder served by the web server that is directly accessible by IP address, to eliminate any DNS-related doubt.

Answer 6

I was looking at the using the Google App Engine as my own CDN and found a nice article on how to do it here

http://www.digitalistic.com/2008/06/09/10-easy-steps-to-use-google-app-engine-as-your-own-cdn/

I am sure it is fairly easy to also host this page or pages there, could probably even use web services to connect back you your site to do any updating or real time integration. This is free and I am pretty sure will be around for awhile.

Answer 7

You could try mirroring a particular page with AWS Cloudfront. Using the ability to have Cloudfront pull from an origin server your page should always be up to date. A user who requested the page from the distribution would in turn cause Cloudfront to request the content from the origin.