Why multi node config server fetch the configuration from the same node each time - spring-cloud

There are three nodes running in the configuration center, when the server starts, i found that
the server fetch configuration from the same config-server each time, and only the config-server
processes the request.
My server start log:
Multiple Config Server Urls found listed.
Fetching config from server at: http://***:20001/
Located environment: name=onip-boot, profiles=[prod], label=null, version=..., state=null
I read the source code of ConfigServicePropertySourceLocator.java, at line 205, I found it always seems to ask the first to get the configuration and then return, unless it fails and loops the next one. So, Spring Cloud Config does not have the ability of load balancing?
for (int i = 0; i < noOfUrls; i++) {
Credentials credentials = properties.getCredentials(i);
String uri = credentials.getUri();
String username = credentials.getUsername();
String password = credentials.getPassword();
logger.info("Fetching config from server at : " + uri);
try {
HttpHeaders headers = new HttpHeaders();
addAuthorizationToken(properties, headers, username, password);
if (StringUtils.hasText(token)) {
headers.add(TOKEN_HEADER, token);
}
if (StringUtils.hasText(state) && properties.isSendState()) {
headers.add(STATE_HEADER, state);
}
final HttpEntity<Void> entity = new HttpEntity<>((Void) null, headers);
response = restTemplate.exchange(uri + path, HttpMethod.GET, entity,
Environment.class, args);
}
catch (HttpClientErrorException e) {
if (e.getStatusCode() != HttpStatus.NOT_FOUND) {
throw e;
}
}
catch (ResourceAccessException e) {
logger.info("Connect Timeout Exception on Url - " + uri
+ ". Will be trying the next url if available");
if (i == noOfUrls - 1)
throw e;
else
continue;
}
if (response == null || response.getStatusCode() != HttpStatus.OK) {
return null;
}
Environment result = response.getBody();
return result;
}

Related

Server returned HTTP response code: 401 for URL: https://accounts.google.com/o/oauth2/token during generating access token

I am using the admin sdk API to retrieve all G Suite users. We require an access token for this. AWS is used to host our website. I've tried a few different codes to generate access token, but they always return error
"Server returned HTTP response code: 401 for URL: https://accounts.google.com/o/oauth2/token."
I have no idea why this error is occurring. My code is running smoothly, generating access token and retrieving every user domain wise in a local environment. Any help in why actually I am getting this error. have i missed something? any help in it.
This is my code.
private String getAccessToken()
{
String accessToken="";
try
{
Map<String,Object> params = new LinkedHashMap<>();
params.put("grant_type","refresh_token");
params.put("client_id",client_id);
params.put("client_secret",client_secret);
params.put("refresh_token",refresh_token);
StringBuilder postData = new StringBuilder();
for(Map.Entry<String,Object> param : params.entrySet())
{
if(postData.length() != 0)
{
postData.append('&');
}
postData.append(URLEncoder.encode(param.getKey(),"UTF-8"));
postData.append('=');
postData.append(URLEncoder.encode(String.valueOf(param.getValue()),"UTF-8"));
}
byte[] postDataBytes = postData.toString().getBytes("UTF-8");
URL url = new URL("https://accounts.google.com/o/oauth2/token");
HttpURLConnection con = (HttpURLConnection)url.openConnection();
con.setDoOutput(true);
con.setUseCaches(false);
con.setRequestMethod("POST");
con.getOutputStream().write(postDataBytes);
BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));
StringBuffer buffer = new StringBuffer();
for (String line = reader.readLine(); line != null; line = reader.readLine())
{
buffer.append(line);
}
JSONObject json = new JSONObject(buffer.toString());
accessToken = json.getString("access_token");
return accessToken;
}
catch (Exception ex)
{
ex.printStackTrace();
}
return accessToken;
}

Http Post Flutter to SAP

Im trying to use http post to transfer data from flutter to SAP. I can get data without any problem, but post attempt is failing with code 403 (x-csrf-token invalid)
I had the same problem while working in C# but that was resolved using event handler, that triggers just before save (please see below extract of C# code) but i'm unable to find option in flutter. Please guide..
zZSSALE1SRVEntity.SendingRequest2 += new EventHandler<SendingRequest2EventArgs>(_container_SendingRequest_Enhance);
zZSSALE1SRVEntity.SaveChanges();
private void _container_SendingRequest_Enhance(object sender, SendingRequest2EventArgs e)
{
HttpWebResponse response;
string empty = string.Empty;
string str = string.Empty;
CookieContainer cookieContainer = new CookieContainer();
OdataSsaleDEV.ZZSSALE1_SRV_Entities zZSSALE1SRVEntity = new OdataSsaleDEV.ZZSSALE1_SRV_Entities(app_uri)
{
Credentials = credentials
};
string str1 ;
if (empty == string.Empty)
{
HttpWebRequest credentials = (HttpWebRequest)WebRequest.Create(zZSSALE1SRVEntity.BaseUri);
credentials.Method = "GET";
credentials.Headers.Add("X-CSRF-Token", "Fetch");
credentials.Credentials = zZSSALE1SRVEntity.Credentials;
cookieContainer = new CookieContainer();
credentials.CookieContainer = cookieContainer;
try
{
response = (HttpWebResponse)credentials.GetResponse();
}
catch (WebException webException)
{
MessageBox.Show(webException.Message);
return;
}
catch (Exception exception)
{
MessageBox.Show(exception.Message);
return;
}
empty = response.Headers.Get("X-CSRF-Token");
str = response.Headers.Get("Set-Cookie");
credentials.Abort();
}
if (empty != string.Empty)
{
e.RequestMessage.SetHeader("x-csrf-token", empty);
foreach (Cookie cooky in cookieContainer.GetCookies(zZSSALE1SRVEntity.BaseUri))
{
str1 = string.Concat(str1, ";", cooky.ToString());
}
e.RequestMessage.SetHeader("Cookie", str1.Substring(1));
}
Issue resolved.
Actually server requires session cookies (MYSAPSSO and SAP_SESSIONID) along with x-csrf-token.

Websphere App Server 7.0.0.43 - IBM Java 1.6, Apache Wink, Rest Client TLSv1.2

Recently upgraded websphere app server to TLSv1.2. Prior, with TLSv1, was able to interface with REST client using combination of javax.ws.rs, org.glassfish.jersey and javax.net.ssl. Wasn't able to get working once app server upgraded TLS. IBM instructed to apply Feature Pack Web 2.0 and interface with REST client using Apache Wink. Haven't been able to get this solution to work. Previously, Keystore and Truststore were set on SSLContext.
private static String getClient(String servicePath){
try {
javax.ws.rs.client.Client client = null;
org.glassfish.jersey.client.ClientConfig clientConfig = new org.glassfish.jersey.client.ClientConfig();
int readTimeOut = Integer.parseInt(commonProp.getProperty(READ_TIMEOUT));
int connTimeout = Integer.parseInt(commonProp.getProperty(CONNECTION_TIMEOUT));
clientConfig.property(ClientProperties.CONNECT_TIMEOUT, connTimeout);
clientConfig.property(ClientProperties.READ_TIMEOUT, readTimeOut);
org.glassfish.jersey.SslConfigurator sslConfig;
String trustStoreName = commonProp.getProperty(TRUSTSTORE_NAME);
File file1 = new File("\\"+PROJECT_FILE_DIR+ "/ISB"+trustStoreName.trim());
String keyStoreName = commonProp.getProperty(KEYSTORE_NAME);
File file2 = new File("\\"+PROJECT_FILE_DIR+ "/ISB"+keyStoreName.trim());
FileInputStream fis1 = new FileInputStream(file1);
FileInputStream fis2 = new FileInputStream(file2);
String trustStorePassword = commonProp.getProperty(TRUSTSTORE_PASSWORD);
String keyStorePassword = commonProp.getProperty(KETSTORE_PASSWORD);
sslConfig = org.glassfish.jersey.SslConfigurator.newInstance().trustStoreBytes(ByteStreams.toByteArray(fis1))
.trustStorePassword(trustStorePassword).keyStoreBytes(ByteStreams.toByteArray(fis2))
.keyPassword(keyStorePassword);
javax.net.ssl.SSLContext sslContext = sslConfig.createSSLContext();
client = javax.ws.rs.client.ClientBuilder.newBuilder().sslContext(sslContext).withConfig(clientConfig).build();
String url = commonProp.getProperty(ENDPOINT_URL)+servicePath;
Response response =
client.target(url)
.request(MediaType.APPLICATION_JSON)
.get();
String responseAsString = "";
if(response != null){
responseAsString = response.readEntity(String.class);
}
return responseAsString;
} catch (Throwable e) {
logger.severe(e.getMessage() + e.getLocalizedMessage());
e.printStackTrace();
throw new RuntimeException(e);
}
}
With Installed Feature Pack and use of JSSEHELPER, retrieved SSL alias
information from WAS. Seem to make client connection now, but still have
authentication issue
Blockquote
response statusCode: 200
called close()
called closeInternal(true)
WebContainer : 0, SEND TLSv1.2 ALERT: warning, description = close_notify
An attempt to authenticate with a client certificate failed. A valid client
certificate is required to make this connection.
Blockquote
Believe issue may be that previously the KEYSTORE and TRUSTSTORE information
was read from file. Whereas the JSSEHELPER is simply putting file name.
com.ibm.ssl.clientAuthenticationSupported = false
com.ibm.ssl.keyStoreClientAlias = isbgatewaytst
com.ibm.ssl.contextProvider = IBMJSSE2
com.ibm.ssl.trustStoreProvider = IBMJCE
com.ibm.ssl.protocol = TLSv1.2
com.ibm.ssl.keyStoreReadOnly = false
com.ibm.ssl.alias = ISBGatewaySSL
com.ibm.ssl.keyStoreCreateCMSStash = false
com.ibm.ssl.securityLevel = CUSTOM
com.ibm.ssl.trustStoreName = ISBGatewayTrust
com.ibm.ssl.configURLLoadedFrom = security.xml
com.ibm.ssl.trustStorePassword = ********
com.ibm.ssl.keyStoreUseForAcceleration = false
com.ibm.ssl.trustManager = PKIX
com.ibm.ssl.validationEnabled = false
com.ibm.ssl.trustStoreInitializeAtStartup = false
com.ibm.ssl.keyManager = IbmX509
com.ibm.ssl.keyStoreFileBased = true
com.ibm.ssl.keyStoreType = JKS
com.ibm.ssl.trustStoreFileBased = true
com.ibm.ssl.trustStoreCreateCMSStash = false
com.ibm.ssl.trustStoreScope = (cell):ESB_DEV
com.ibm.ssl.trustStore = E:/IBM/content/resources/dev_projectfiles_dir/ISB/isb-truststore.jks
com.ibm.ssl.keyStoreProvider = IBMJCE
com.ibm.ssl.enabledCipherSuites = SSL_ECDHE_RSA_WITH_AES_128_GCM_SHA256 SSL_ECDHE_RSA_WITH_AES_128_CBC_SHA256 SSL_ECDHE_RSA_WITH_AES_128_CBC_SHA
com.ibm.ssl.daysBeforeExpireWarning = 60
com.ibm.ssl.keyStoreServerAlias = isbgatewaytst
com.ibm.ssl.clientAuthentication = false
com.ibm.ssl.keyStore = E:/IBM/content/resources/dev_projectfiles_dir/ISB/isb-keystore-tst.jks
com.ibm.ssl.trustStoreUseForAcceleration = false
com.ibm.ssl.trustStoreReadOnly = false
com.ibm.ssl.keyStoreScope = (cell):ESB_DEV
com.ibm.ssl.tokenEnabled = false
com.ibm.ssl.keyStoreName = ISBGatewayPrivatekey
com.ibm.ssl.keyStorePassword = ********
com.ibm.ssl.keyStoreInitializeAtStartup = false
com.ibm.ssl.trustStoreType = JKS
My issue is how to put the KEYSTORE and TRUSTSTORE information on the org.apache.wink.client.ClientConfig or else where, similar to previously setting the javax.net.ssl.SSLContext? Did set the WAS SSL alias information on the ClientConfig properties, but believe that is just adding the Store's file location which isn't of use for the REST Client.
public static String getPPLUResponseString(String servicePath) {
String responseAsString = "";
com.ibm.websphere.ssl.JSSEHelper jsseHelper = com.ibm.websphere.ssl.JSSEHelper.getInstance();
try {
Properties sslProps = null;
String alias = "ISBGatewaySSL";
sslProps = jsseHelper.getProperties(alias, getConnectionInfo(), null);
org.apache.wink.client.ClientConfig clientConfig = new org.apache.wink.client.ClientConfig();
clientConfig.readTimeout(Integer.parseInt(commonProp.getProperty(READ_TIMEOUT)));
clientConfig.connectTimeout(Integer.parseInt(commonProp.getProperty(CONNECTION_TIMEOUT)));
clientConfig.setProperties(sslProps);
Enumeration keys = clientConfig.getProperties().keys();
while (keys.hasMoreElements()) {
String key = (String) keys.nextElement();
String value = (String) clientConfig.getProperties().get(key);
System.out.println(" clientConfig.getProperties(): " + key + ": " + value);
}
org.apache.wink.client.RestClient restClient = new org.apache.wink.client.RestClient(clientConfig);
String url = commonProp.getProperty(ENDPOINT_URL) + servicePath;
System.out.println(" url: " + url);
org.apache.wink.client.Resource restResource = restClient.resource(url);
System.out.println(" before client response");
org.apache.wink.client.ClientResponse clientResponse = restResource.accept(MediaType.APPLICATION_JSON_TYPE).get();
int statusCode = clientResponse.getStatusCode();
System.out.println(" response statusCode: " + statusCode);
String responseEntity = clientResponse.getEntity(String.class);
System.out.println(" responseEntity start: " + responseEntity);
System.out.println(" responseEntity end: ");
if (responseEntity != null) {
responseAsString = responseEntity;
}
} catch (com.ibm.websphere.ssl.SSLException e) {
System.out.println(" com.ibm.websphere.ssl.SSLException");
e.printStackTrace();
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return responseAsString;
}

HttpWebRequest maintenance and http web errors causing it to return "HRESULT E_FAIL" and "server not found"

I am iterating through a large list of objects (1503) and calling a save method on a ServiceProxy I have written. The service proxy uses the new networking stack in Silverlight 4 to call BeginGetRequestStream to start the process of asynchronously sending my objects to an azure REST service I have written for saving off the objects. The Http method I am using is POST. I know HttpWebClient is smart enough to reuse the Http connection so I am not concurrently opening 1503 connections to the server. Saving works fine and all 1503 objects are saved very quickly. However, when I try to save the same objects again, I expect to recieve an HttpStatus code of forbidden because the objects already exist and that is the code I set my azure web service to return. On small groups of objects, it works as expected. However, when I try saving the entire list of 1503 objects, I receive only 455 correct responses and 1048 errors such as "server not found" and
System.Exception ---> System.Exception:Error HRESULT E_FAIL has been returned from a call to a COM component.
at
System.Net.Browser.ClientHttpWebRequest.InternalEndGetResponse(IAsyncResult asyncResult)...
I wonder if there is some sort of book keeping or maintenance I am supposed to be performing on my HttpWebClient instances that I am neglecting and that is what is causing the http errors to throw exceptions but the new saves to work perfectly. Here is my code for handling the error cases:
private static void SendAncestorResponseCallback(IAsyncResult result)
{
var info = (SendAncestorInfo)result.AsyncState;
try
{
var response = info.Request.EndGetResponse(result);
info.Response = response;
}
catch ( Exception ex)
{
info.Error = ex;
}
info.MainThreadContext.Post(SendAncestorMainThreadCallback, info);
}
private static void SendAncestorMainThreadCallback(object state)
{
var info = (SendAncestorInfo)state;
IAncestor origAncestor = info.Content;
HttpWebResponse response = null;
if (info.Error != null)
{
if ((info.Error as WebException) == null)
{
info.Callback(false, origAncestor, null, info.Error);
return;
}
else //get response from WebException
{
response = (HttpWebResponse)(info.Error as WebException).Response;
}
}
else //get response from info.Response
{
response = info.Response as HttpWebResponse;
}
if (response.StatusCode == HttpStatusCode.Created || response.StatusCode == HttpStatusCode.Forbidden)
{
var stream = response.GetResponseStream();
using (var reader = new StreamReader(stream))
{
IAncestor retAncestor = XMLSerializerHelper.DeserializeObject<Ancestor>(reader.ReadToEnd());
info.Callback(response.StatusCode == HttpStatusCode.Created, origAncestor, retAncestor, null);
}
}
else info.Callback(false, origAncestor, null, info.Error);
}
considering how the web service is written I should only expect http status codes of created or forbidden and like I said with small groups this is the case. The fact that I only start getting the errors mentioned earlier makes me feel like I am doing something wrong with the HttpWebRequest objects etc. Any assistance would be greatly appreciated. Thanks.
--update here is the code that generates the HttpWebRequest:
foreach (IAncestor ancestor in ancestors)
{
AncestorViewModel ancestorVM = new AncestorViewModel(ancestor);
ancestorVM.Status = SaveStatus.Undefined;
ParsedAncestors.Add(ancestorVM);
_service.CreateAncestor(UserSrc, ancestor, (success, origAncestor, retAncestor, exception) =>
{
AncestorViewModel result = ParsedAncestors.First(a => a.Model.IdNo == origAncestor.IdNo);
if (exception == null)//web response was either Created or Forbidden
{
if (success)//Ancestor successfully created
{
savedAncestors++;
SuccessMessage = string.Format("{0} Saved\n", savedAncestors);
result.Status = SaveStatus.Saved;
}
else //Ancestor already existed
{
conflictAncestors.Add(origAncestor, retAncestor);
ConflictMessage = string.Format("{0} Conflicts\n", conflictAncestors.Count);
result.Status = SaveStatus.Conflicted;
}
}
else //Show exception recieved from remote web service
{
//if (exception as WebException != null)
//{
// //if exception is WebException get status code and description
// HttpWebResponse rs = (HttpWebResponse)(exception as WebException).Response;
// Message += string.Format("WebServer returned status code {0}: '{1}'\n", (int)rs.StatusCode, rs.StatusDescription);
//}
errors.Add(origAncestor, exception);
ErrorMessage = string.Format("{0} Errors\n", errors.Count);
result.Status = SaveStatus.Error;
}
});
}
public void CreateAncestor(string userSrc, IAncestor ancestor, Action<bool, IAncestor, IAncestor, Exception> callback)
{
WebRequest.RegisterPrefix("http://", WebRequestCreator.ClientHttp);
var request = (HttpWebRequest)WebRequest.Create(
new Uri(string.Format("{0}/{1}/{2}", rootUri, AncestorsRestPoint, userSrc)));
request.Method = "POST";
request.ContentType = "application/xml";
var info = new SendAncestorInfo
{
Request = request,
Callback = callback,
Content = ancestor,
MainThreadContext = SynchronizationContext.Current
};
request.BeginGetRequestStream(SendAncestorRequestCallback, info);
}

apache HttpClient to access facebook

Any examples, tips, guidance for the following scenario?
I have used Apache HttpClient to simulate the functionality of browser to access facebook through java application. to do that first i have to provide user credentials. i have used examples provided in the following web site.
http://svn.apache.org/viewvc/httpcomponents/oac.hc3x/trunk/src/examples/
But non of these methods works for facebook, following is the test code i have written for this purpose. i have not provided the all the methods written, only the method used to login to the facebook account is given here. relay appreciate any help
private static int connectAndLogin(String email, String pass){
logger.trace("Facebook: =========connectAndLogin begin===========");
String httpResponseBody = getMethod("http://www.facebook.com/login.php");
if(httpResponseBody == null){
//Why don't we try again?
try
{
Thread.sleep(1000);
}
catch (InterruptedException e)
{
logger.trace(e.getMessage());
}
httpResponseBody = getMethod("http://www.facebook.com/login.php");
}
logger.trace("Facebook: ========= get login page ResponseBody begin===========");
logger.trace(httpResponseBody);
logger.trace("Facebook: +++++++++ get login page ResponseBody end+++++++++");
logger.trace("Facebook: Initial cookies: ");
List<Cookie> cookies = getCookies();
if (cookies.isEmpty()) {
logger.trace("Facebook: None");
} else {
for (int i = 0; i < cookies.size(); i++) {
logger.trace("Facebook: - " + cookies.get(i).toString());
}
}
if(httpResponseBody == null){
logger.warn("Facebook: Warning: Failed to get facebook login page.");
}
try
{
HttpPost httpost = new HttpPost("http://www.facebook.com/login.php");
List <NameValuePair> nvps = new ArrayList <NameValuePair>();
nvps.add(new BasicNameValuePair("email", email));
nvps.add(new BasicNameValuePair("pass", pass));
//don't know if is this necessary
nvps.add(new BasicNameValuePair("login", ""));
httpost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));
logger.info("Facebook: #executing post method to:" + "http://www.facebook.com/login.php");
HttpResponse loginPostResponse = getHttpClient().execute(httpost);
HttpEntity entity = loginPostResponse.getEntity();
logger.trace("Facebook: Login form post: " + loginPostResponse.getStatusLine());
if (entity != null) {
logger.trace("Facebook: "+EntityUtils.toString(entity));
entity.consumeContent();
} else {
logger.error("Facebook: Error: login post's response entity is null");
return FacebookErrorCode.kError_Login_GenericError;
}
logger.trace("Facebook: Post logon cookies:");
cookies = getCookies();
if (cookies.isEmpty()) {
logger.trace("Facebook: None");
} else {
for (int i = 0; i < cookies.size(); i++) {
logger.trace("Facebook: - " + cookies.get(i).toString());
}
}
int statusCode = loginPostResponse.getStatusLine().getStatusCode();
logger.info("Facebook: Post Method done(" + statusCode + ")");
switch(statusCode){
case 100: break;//we should try again;
case 301:
case 302:
case 303:
case 307:
{
//redirect
// Header[] headers = loginPostResponse.getAllHeaders();
// for (int i=0; i<headers.length; i++) {
// logger.trace("Facebook: "+headers[i]);
// }
// Header locationHeader = loginPostResponse.getFirstHeader("location");
// if(locationHeader != null){
// homePageUrl = locationHeader.getValue();
// logger.info("Facebook: Redirect Location: " + homePageUrl);
// if(homePageUrl == null
// || !homePageUrl.contains("facebook.com/home.php")){
// logger.error("Facebook: Login error! Redirect Location Url not contains \"facebook.com/home.php\"");
// return FacebookErrorCode.kError_Login_GenericError;
// }
// } else {
// logger.warn("Facebook: Warning: Got no redirect location.");
// }
}
break;
default:;
}
}
catch (IOException ioe)
{
logger.error("Facebook: IOException\n" + ioe.getMessage());
return FacebookErrorCode.kError_Global_ValidationError;
}
logger.trace("Facebook: =========connectAndLogin end==========");
return FacebookErrorCode.Error_Global_NoError;
}
The following code, based on that sample, worked for me:
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet("http://www.facebook.com/login.php");
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
System.out.println("Login form get: " + response.getStatusLine());
if (entity != null) {
entity.consumeContent();
}
System.out.println("Initial set of cookies:");
List<Cookie> cookies = httpclient.getCookieStore().getCookies();
if (cookies.isEmpty()) {
System.out.println("None");
} else {
for (int i = 0; i < cookies.size(); i++) {
System.out.println("- " + cookies.get(i).toString());
}
}
HttpPost httpost = new HttpPost("http://www.facebook.com/login.php");
List <NameValuePair> nvps = new ArrayList <NameValuePair>();
nvps.add(new BasicNameValuePair("email", "******"));
nvps.add(new BasicNameValuePair("pass", "*******"));
httpost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));
response = httpclient.execute(httpost);
entity = response.getEntity();
System.out.println("Double check we've got right page " + EntityUtils.toString(entity));
System.out.println("Login form get: " + response.getStatusLine());
if (entity != null) {
entity.consumeContent();
}
System.out.println("Post logon cookies:");
cookies = httpclient.getCookieStore().getCookies();
if (cookies.isEmpty()) {
System.out.println("None");
} else {
for (int i = 0; i < cookies.size(); i++) {
System.out.println("- " + cookies.get(i).toString());
}
}
httpclient.getConnectionManager().shutdown();
I am not sure if your code was managing properly cookies (and session id kept within one of them), maybe that was the problem. Hope this will help you.
Just to make clear version issue: I was using HttpClient version 4.X, not the old one (3.X). They differ significantly.
Perhaps you should use a tool, such as Selenium
Have you taken a look at HtmlUnit. It wraps the HttpClient to create a headless Java browser, with javaScript execution. This way you are not trying to hack the individual forms all the time.