Request de website

[carvalho2707]

boas, precisava de aceder ao código de um site, mas estou com alguns erros.. acham que me podem ajudar?

usei dois metodos diferentes e ambos falham:

URL url = new URL("https://www.google.com/search?q=ice+age&ie=utf-8&oe=utf-8&aq=t");
URLConnection yc = url.openConnection();
BufferedReader br = new BufferedReader(new InputStreamReader(yc.getInputStream()));

e da um erro:

java.io.IOException: Server returned HTTP response code: 403 for URL: https://www.google.com/search?q=ice+age&ie=utf-8&oe=utf-8&aq=t

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)

at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)

at bd.Programa.descobreLinkImdb(Programa.java:304)

at bd.Main.main(Main.java:22)

por outrlo lado:

URL url;
HttpURLConnection connection = null;
url = new URL("https://www.google.com/search?q=ice+age&ie=utf-8&oe=utf-8&aq=t");
connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
connection.setRequestProperty("Content-Language", "en-US");
connection.setRequestProperty("User-Agent","Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11");
connection.setUseCaches(false);
connection.setDoInput(true);
connection.setDoOutput(true);
java.io.InputStream is = connection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));

java.io.IOException: Server returned HTTP response code: 411 for URL: http://www.google.com/search?q=ice+age&ie=utf-8&oe=utf-8&aq=t

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)

at bd.Programa.descobreLinkImdb(Programa.java:312)

at bd.Main.main(Main.java:22)

[HappyHippyHippo]

http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

experimenta tirar o "s" de "https"

[KTachyon]

Pah, o Google filtra as tentativas de automatizar, pelo que tens que acertar nos pormenores certos. A segunda tentativa seria melhor se acertasses no método correcto (que é GET, não POST).

De qualquer forma, não sei se não existirá uma clausula nos termos deles que te deveria impedir de fazer isso.

Podes sempre utilizar um dos APIs deles em vez de fazeres pedidos directos.