-2

I am able to access the directory listing of a web-site. I need to download all the files from a specific folder. Is there some application to help me do this? This is the site.

Edit: The application whose content I need is a web application like :

www.example.com/ab/cd

The application which gives me the list of files from the server can be anything - either a webapp or a desktop application.

|improve this question
  • 1
    This question would fit superuser.com better in my opinion – Joe Phillips Jul 1 '10 at 3:52
  • Depends on whether the application is web or desktop. For example a we b app that sends me a zip file with all the files from url.com/directory/ then I am not sure, if it is directory listing in terms of server usr/blah/blah then yes @bdhar please clarify – phwd♦ Jul 1 '10 at 3:55
  • @phwd - edited the question for clarification – bdhar Jul 1 '10 at 4:20
  • Ok then I agree with @Joe Philllips . This question is about downloading from a server which falls in superuser.com scope not necessarily a non-hosting directory browser checking as in Plesk. In terms of desktop try HTTrack the guys at superuser.com would be better for this. – phwd♦ Jul 1 '10 at 4:41
  • should be migrated to superuser.com – akira Jul 1 '10 at 4:53
1

Ref: http://blog.incognitech.in/download-files-from-apache-server-listing-directory/

You can use following command:

wget --execute="robots = off" --mirror --convert-links --no-parent --wait=5 <website-url>

Explanation with each options

  • wget: Simple Command to make CURL request and download remote files to our local machine.
  • --execute="robots = off": This will ignore robots.txt file while crawling through pages. It is helpful if you're not getting all of the files.
  • --mirror: This option will basically mirror the directory structure for the given URL. It's a shortcut for -N -r -l inf --no-remove-listing which means:
    • -N: don't re-retrieve files unless newer than local
    • -r: specify recursive download
    • -l inf: maximum recursion depth (inf or 0 for infinite)
    • --no-remove-listing: don't remove '.listing' files
  • --convert-links: make links in downloaded HTML or CSS point to local files
  • --no-parent: don't ascend to the parent directory
  • --wait=5: wait 5 seconds between retrievals. So that we don't thrash the server.
  • <website-url>: This is the website url from where to download the files.

Happy Downloading :smiley:

|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.