Fetch files over web server

Discussion in 'Java' started by Juuno, Feb 3, 2009.

  1. Juuno

    Juuno New Member

    Joined:
    Feb 3, 2009
    Messages:
    10
    Likes Received:
    0
    Trophy Points:
    0
    Hi, I am just a newbie and I need to write some programs.

    I have one directory and under that directory, I have 35 xml files in the Apache Web Server. The path for the directory is like //localhost:8081/dirxml/

    And path for the xml files are like..
    //localhost:8081/dirxml/mycontacts.xml
    //localhost:8081/dirxml/favourtieactor.xml
    //localhost:8081/dirxml/favouritesinger.xml
    ....
    ....
    .... and so on....


    I would like to fetch them from my java program and then produce an xml file which is a combination of those xml files.

    So, how can I fetch those xml files from web server simultaneously? Is it ok if I use java.net.URL class. But what I found so far is it needs to give the exact URL path like //localhost:8081/dirxml/mycontacts.xml?? But I can't do it coz I don't want to give those xml file names and fetches. What I want to do is to fetch the files under that directory without knowing the file name. It is like to fetch all the files under that directory which ends with .xml

    How can I do this? And is there any reference for it.

    Thanks in advance.
     
  2. devunion

    devunion New Member

    Joined:
    Sep 26, 2008
    Messages:
    19
    Likes Received:
    2
    Trophy Points:
    0
    You can use next code to retrieve list of files:

    Code:
            try {
                URL url = new URL(YOUR_URL_HERE);
                InputStream inputStream = url.openStream();
                System.out.println("inputStream.available() = " + inputStream.available());
    
                BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
    
                String line = null;
    
                while ((line = br.readLine()) != null) {
                    System.out.println("line = " + line);
                }
            } catch (MalformedURLException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            }
    
     
    shabbir likes this.
  3. graveyard220

    graveyard220 New Member

    Joined:
    Feb 19, 2009
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    0
    there is no simple way to do that because this is not a real path for a file system so this will be some huge code because you want to parse an html resulted from specified url and get the resulted inline files,
    but fortunately apache did that effort and make a package called ivr this have more utilities to do such that things,
    so,
    you can download the binary from this location:
     
  4. graveyard220

    graveyard220 New Member

    Joined:
    Feb 19, 2009
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    0
    ht tp:/ /a nt. apa che. org /i v y/d o w n lo ad. cgi

    and then use it as simple as:
     
  5. graveyard220

    graveyard220 New Member

    Joined:
    Feb 19, 2009
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    0
    sorry for that cutting
    Code:
    package networkanddatabase;
    
    import java.io.BufferedInputStream;
    import java.io.BufferedOutputStream;
    import java.io.File;
    import java.io.FileOutputStream;
    import java.io.IOException;
    import java.net.URL;
    import java.net.URLConnection;
    import java.util.Iterator;
    import java.util.List;
    
    import org.apache.ivy.util.url.ApacheURLLister;
    
    public class FetchFilesFromHttpURL {
    	public static void main(String[] args) {
    		URL url;
    		try {
    			url = new URL(youurl);
    			File destFolder = new File("c:\\test");
    			ApacheURLLister lister = new ApacheURLLister();
    			// this list of URLs objects
    			List files = lister.listAll(url);
    			System.out.println("list file is complete.."+files);
    			for (Iterator iter = files.iterator(); iter.hasNext();) {
    				URL fileUrl = (URL) iter.next();
    				httpFileDownload(fileUrl, destFolder);
    			}
    			System.out.println("download is complete..");
    		} catch (Exception e) {
    			e.printStackTrace();
    		}
    
    	}
    
    	public static void httpFileDownload(URL url, File destFolder) throws Exception {
    		File destination = new File(destFolder, url.getFile());
    		destination.getParentFile().mkdirs();
    		BufferedInputStream bis = null;
    		BufferedOutputStream bos = null;
    		try {
    			URLConnection urlc = url.openConnection();
    
    			bis = new BufferedInputStream(urlc.getInputStream());
    			bos = new BufferedOutputStream(new FileOutputStream(destination.getPath()));
    
    			int i;
    			while ((i = bis.read()) != -1) {
    				bos.write(i);
    			}
    		} finally {
    			if (bis != null)
    				try {
    					bis.close();
    				} catch (IOException ioe) {
    					ioe.printStackTrace();
    				}
    			if (bos != null)
    				try {
    					bos.close();
    				} catch (IOException ioe) {
    					ioe.printStackTrace();
    				}
    		}
    	}
    }
    
    
    
     
  6. devunion

    devunion New Member

    Joined:
    Sep 26, 2008
    Messages:
    19
    Likes Received:
    2
    Trophy Points:
    0
    Good solution, thanks!
     

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice