Web Applications Stack Exchange is a question and answer site for power users of web applications. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am able to access the directory listing of a web-site. I need to download all the files from a specific folder. Is there some application to help me do this? This is the site.

Edit: The application whose content I need is a web application like :

www.example.com/ab/cd

The application which gives me the list of files from the server can be anything - either a webapp or a desktop application.

share|improve this question

closed as off-topic by Al E., serenesat, Bookend, jonsca Jun 19 at 21:11

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions asking for us to recommend or find a Web Application are off-topic and out of scope, as they tend to attract opinionated answers and spam. However, your question may be on topic at Software Recommendations if you can rewrite it so that it meets their quality guidelines" – Al E., serenesat, Bookend, jonsca
If this question can be reworded to fit the rules in the help center, please edit the question.

1  
This question would fit superuser.com better in my opinion – Joe Philllips Jul 1 '10 at 3:52
    
Depends on whether the application is web or desktop. For example a we b app that sends me a zip file with all the files from url.com/directory/ then I am not sure, if it is directory listing in terms of server usr/blah/blah then yes @bdhar please clarify – phwd Jul 1 '10 at 3:55
    
@phwd - edited the question for clarification – bdhar Jul 1 '10 at 4:20
    
Ok then I agree with @Joe Philllips . This question is about downloading from a server which falls in superuser.com scope not necessarily a non-hosting directory browser checking as in Plesk. In terms of desktop try HTTrack the guys at superuser.com would be better for this. – phwd Jul 1 '10 at 4:41
    
should be migrated to superuser.com – akira Jul 1 '10 at 4:53
up vote 1 down vote accepted

See this post: http://webapps.stackexchange.com/questions/570/downloading-code-from-google-code-without-using-a-version-control-system/577#577

It applies to this problem too.

share|improve this answer
    
Thanks Feniix, that's exactly what I was looking for.. – bdhar Jul 1 '10 at 6:25

Ref: http://blog.incognitech.in/download-files-from-apache-server-listing-directory/

You can use following command:

wget --execute="robots = off" --mirror --convert-links --no-parent --wait=5 <website-url>

Explanation with each options

  • wget: Simple Command to make CURL request and download remote files to our local machine.
  • --execute="robots = off": This will ignore robots.txt file while crawling through pages. It is helpful if you're not getting all of the files.
  • --mirror: This option will basically mirror the directory structure for the given URL. It's a shortcut for -N -r -l inf --no-remove-listing which means:
    • -N: don't re-retrieve files unless newer than local
    • -r: specify recursive download
    • -l inf: maximum recursion depth (inf or 0 for infinite)
    • --no-remove-listing: don't remove '.listing' files
  • --convert-links: make links in downloaded HTML or CSS point to local files
  • --no-parent: don't ascend to the parent directory
  • --wait=5: wait 5 seconds between retrievals. So that we don't thrash the server.
  • <website-url>: This is the website url from where to download the files.

Happy Downloading :smiley:

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.