How do you request a web page from within Java?
Regards,
Jacky
It seemed a bit complicated. Is there a source code sample, that
requests "http://www.google.com" for example?
Thanks,
Jacky
Roland de Ruiter
___ ___
/__/ w_/ /__/
/ \ /_/ / \
There's also Jakarta Commons HTTPClient, which hides some of the complexity.
http://jakarta.apache.org/commons/httpclient/
--
Wendy
try {
URL url = new URL(http://www.google.com);
BufferedReader in = new BufferedReader(new
InputStreamReader(url.openStream()));
String str;
StringBuffer page = new StringBuffer();
while ((str = in.readLine()) != null) {
page.append(str + "\n");
}
in.close();
}
catch (MalformedURLException e) {
//...
}
catch (IOException e) {
//...
}
// And now 'page' contains the webpage...
--
Dag.
If you know you're going to get textual data back, something like the
following might work:
StringBuffer bigBuf = new StringBuffer();
try {
InputStream strm = new URL("http://www.google.com/").openStream();
BufferedReader rdr = new BufferedReader(new InputStreamReader(strm));
String thisLine = rdr.readLine();
while (thisLine != null) {
bigBuf = bigBuf.append(thisLine);
thisLine = rdr.readLine();
}
rdr.close();
} catch (Exception e) {
/* You might catch URLException, network exception, or IOException
}
String wholeThing = bigBuf.toString();
After which the 'wholeThing' string would contain all the data, unless
an exception occured.
You could easily modify this to check the URL's Content type - Read up
on the classes others have suggested. This is just a starter :)
Hope this helps,
Ross
--
[Ross A. Bamford] [ro...@synergix-systems.co.uk]
Synergix Systems +++ Linux Admin + Support + Consultancy
www.synergix-systems.co.uk + in...@synergix-systems.co.uk
I have the following code.
It displays alright when URL = http://google.com
It gives me an error when URL =
http://www.google.com/search?hl=en&safe=off&q=Media+studies+and+gender+studies+with+content+on+queer+theory%2C+gender&meta=
===============================================================
Error Message:
===============================================================
Exception in thread "main" java.io.IOException: Server returned HTTP
response code: 403 for URL: htt
p://www.google.com/search?hl=en&safe=off&q=Media+studies+and+gender+studies+with+content+on+queer+th
eory%2C+gender&meta=
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:789)
at java.net.URL.openStream(URL.java:913)
at Project.main(Project.java:19)
Press any key to continue...
===============================================================
Code:
===============================================================
import java.io.*;
import java.net.*;
import java.lang.*;
public class Project {
public static void main(String[] args) throws Exception {
URL search_term = new URL("http://www.yahoo.com/");
BufferedReader in = new BufferedReader(
new InputStreamReader(
search_term.openStream()));
String inputLine;
StringBuffer bigBuf = new StringBuffer();
search_term = new URL("http://www.google.com/search?hl=en&safe=off&q=Media+studies+and+gender+studies+with+content+on+queer+theory%2C+gender&meta=");
in = new BufferedReader(new
InputStreamReader(search_term.openStream()));
while ((inputLine = in.readLine()) != null) {
bigBuf = bigBuf.append(inputLine);
bigBuf = bigBuf.append("\n\n\n");
}
String wholeThing = bigBuf.toString();
System.out.println(wholeThing);
in.close();
}
}
Any ways to get around it?
You need to pretend to be a "real" browser by setting a request property.
See http://www.rgagnon.com/javadetails/java-0399.html
for an example.
bye.
--
Real Gagnon from Quebec, Canada
* Looking for Java or PB code examples ? Visit Real's How-to
* http://www.rgagnon.com/howto.html
btw, the code at the page has some "bugs":
webData = new BufferedReader(new
InputStreamReader(connection.getInputStream()));
connection should be replaced with con, because:
URLConnection con = urlObject.openConnection();
was declared eariler...
Here's the code:
=====================================================
import java.io.*;
import java.net.*;
import java.lang.*;
public class Project {
public static void main(String[] args) throws Exception {
String search= "dogs";
String google="http://www.google.com/search?q=" + search +
"&hl=en&ie=UTF-8&oe=UTF8";
URL urlObject = new URL(google);
URLConnection con = urlObject.openConnection();
con.setRequestProperty
( "User-Agent", "Mozilla/4.0 (compatible;MSIE 5.5; Windows NT 5.0;
H010818)" );
BufferedReader in = new BufferedReader(new
InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer bigBuf = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
bigBuf = bigBuf.append(inputLine);
bigBuf = bigBuf.append("\n\n\n");
}
String wholeThing = bigBuf.toString();
System.out.println(wholeThing);
in.close();
}
}
=====================================================