Cannot access google trends using HttpClient - httpclient

Question

I'm kind of newbie to this...Basicly I need to run a script to download .csv files from google trends. I wrote the following code according to this reference , the code is like:
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("https://www.google.com/accounts/ClientLogin");
try {
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>;
nameValuePairs.add(new BasicNameValuePair("Email", "myEmail"));
nameValuePairs
.add(new BasicNameValuePair("Passwd", "myPasswd"));
nameValuePairs.add(new BasicNameValuePair("accountType", "GOOGLE"));
nameValuePairs.add(new BasicNameValuePair("source",
"Google-cURL-Example"));
nameValuePairs.add(new BasicNameValuePair("service", "xapi"));
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = client.execute(post);
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line = "";
while ((line = rd.readLine()) != null) {
System.out.println(line);
if (line.startsWith("SID=")) {
String key = line.substring(4);
// Do something with the key
} catch (Exception e) {
}
I got the information about SID, LSID, Auth, but don't know how to use these information. I guess I should add these cookies in my following request, but don't know exactly how. I wrote another piece of code to connect to the certain URL, but I keep getting this message "You must be signed in to export data from Google Trends." The code is here if it helps:
URL url = new URL(myUrl);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setInstanceFollowRedirects(true);
conn.addRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.addRequestProperty("Authorization", "SID"+key);
conn.addRequestProperty("Email", "myEmail");
conn.addRequestProperty("Passwd", "myPasswd");
conn.setReadTimeout(5000);
conn.connect();
I searched around and found few useful information, anyone could help?

Answer 1

Does it have to be in Java? In python, it's as simple as this:
from pyGTrends import pyGTrends
connector = pyGTrends('google username','google password')
connector.download_report(('keyword1', 'keyword2'))
print connector.csv()
You'll need the google trends api library.
If it has to be Java, you may want to look at the HttpClient examples from Apache. "Form based logon" and "client authentication" may both be relevant.

Answer 2

I have just coded this:
https://github.com/elibus/j-google-trends-api
It is an unofficial Java implementation of Google Trends API. You could use it to easily access Google Trends or you might want to have a look at the code to see it works.
Anyway the authentication flow works as follows (all the steps are required):
Fetch https://accounts.google.com/ServiceLoginAuth and parse the GALX id
Post username/password + GALX
Get http://www.google.com
Then you can access Google Trend with relaxed QoS policies for authenticated users.