Creating a Signed URL for Google Cloud Storage - google-cloud-storage

We have an ERP application running on GCP .
For downloading data spanning more than three months or so ,we're uploading a file on GCS. Now i want to create a signed url so that to give limited access to the end users .
I have been trying this. But i get this error :
Signature does not match. Please check your Google secret key.
Can anyone tell how to go about this?
private static final int EXPIRATION_TIME = 5;
private static final String BASE_URL = "https://storage.googleapis.com";
private static final String httpVerb = "GET";
/*
* private static final String BUCKET = "my_bucket"; private static final String
* FOLDER = "folder";
*/
private final AppIdentityService identityService = AppIdentityServiceFactory.getAppIdentityService();
public String getSignedUrl(String bucket, final String fileName, String contentTpe) throws Exception {
final long expiration = expiration();
final String unsigned = stringToSign(bucket, expiration, fileName, contentTpe);
final String signature = sign(unsigned);
return new StringBuilder(BASE_URL).append("/").append(bucket).append("/").append(fileName)
.append("?GoogleAccessId=").append(clientId()).append("&Expires=").append(expiration)
.append("&Signature=").append(URLEncoder.encode(signature, "UTF-8")).toString();
}
private static long expiration() {
final long unitMil = 1000l;
final Calendar calendar = Calendar.getInstance();
calendar.add(Calendar.MINUTE, EXPIRATION_TIME);
final long expiration = calendar.getTimeInMillis() / unitMil;
return expiration;
}
private String stringToSign(String bucket, final long expiration, String filename, String contentType) {
final String contentMD5 = "";
final String canonicalizedExtensionHeaders = "";
final String canonicalizedResource = "/" + bucket + "/" + filename;
final String stringToSign = httpVerb + "\n"+ contentMD5 + "\n" + contentType + "\n" + expiration + "\n"
+ canonicalizedExtensionHeaders + canonicalizedResource;
return stringToSign;
}
protected String sign(final String stringToSign) throws UnsupportedEncodingException {
final SigningResult signingResult = identityService.signForApp(stringToSign.getBytes());
final String encodedSignature = new String(Base64.encodeBase64(signingResult.getSignature()), "UTF-8");
return encodedSignature;
}
protected String clientId() {
return identityService.getServiceAccountName();
}

URL signing code is a bit tricky because by its nature it can be difficult to know what you've gotten wrong, other than just seeing that it's wrong. There are a few general tips that make it easier:
First, if possible, consider using URL signing functions in the google-cloud libraries. For example, the Java google-cloud library provides a Storage.signURL method, and you can use it like this:
URL signedUrl = storage.signUrl(
BlobInfo.newBuilder(bucketName, blobName).build(),
2, TimeUnit.DAYS);
Second, if you look at the error message, you'll notice that there's a <StringToSign> section. This section contains the exact string that GCS would calculate a signature for. Make sure that the string you're signing matches this string exactly. If it doesn't, that's your problem.
In your code's particular case, I didn't find the problem, but it might be that you're including a content-type line when signing the string, but GET requests don't provide a Content-Type header. It's just an idea, though, since I don't see your invocation of getSignedUrl.

Related

Specifying multiple "implementation=" tags in #requestbody openapi tag in quarkus REST api

I have a quarkus REST api that accepts JSON strings (various kinds) as events and then propagates them to a KAFKA topic.
I want to now put the structure of the possible JSON strings in the openAPI contract (without having to type it out myself manually.
I have managed to find the #requestbody tag that allows me to specify the one structure but I have multiple types of events that all have to go to the same Kafka topic.
example:
#POST
#Path("/student")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
#Tag(name = "Student Events Service")
#Operation(summary = "Accepts student type events", description = "Accepts student type events")
#APIResponse(responseCode = "200", description = "Event sent successfully")
#APIResponse(responseCode = "400", description = "The JSON packet sent did not conform to any known message or the primary key or shared data in the packet is invalid. \n "
+ "See Response Body for detail. - "
+ "Error code 400-1 = Bad message structure or unknown message. Log error and continue to next message - "
+ "Error code 400-2 = Bad primary key. Fix data (client or server side) and retry later - "
+ "Error code 400-3 = Invalid shared data. Fix shared data and retry later",
content = #Content(mediaType = "application/json",
schema = #Schema(implementation = ErrorResponse.class)))
#APIResponse(responseCode = "500", description = "Unexpected server error. See Response Body for detail. - "
+ "Error code 500-1 = Unable to send to KAFKA, please wait and retry later - "
+ "Error code 500-2 = Unable to connect to verification database, please wait and retry later - "
+ "Error code 500-3 = General server error, Please wait and retry later",
content = #Content(mediaType = "application/json",
schema = #Schema(implementation = ErrorResponse.class)))
public Response postStudentEvent(String eventJSON) {
Now I can put the following tag in to generate the openapi contract for 1 class of eventJSON string
#RequestBody(
required = true,
content = #Content(
schema = #Schema(implementation = Application.class)
)
)
But lets say I also want to be able to accept JSON string with the structure of CourseEnrollment.class (These classes are all part of the project btw).
public class CourseEnrollment {
private String eventID;
private String eventTime;
private String eventType;
private String eventUserSource;
private String externalReference1;
private String firstName;
private String lastName;
private String emailAddressPrimary;
private String emailAddressAlternate;
private String periodStartDate;
private String periodEndDate;
private String intakeIntakeYearCode;
private String programmeCode;
public class Application {
private String eventID;
private String eventTime;
private String eventType;
private String eventUserSource;
private String eventTrigger;
private ApplicationInfo applicationDetails = null;
private Person personDetails = null;
private School schoolDetails = null;
Its a wierd scenario I know, but thats what I have to deal with as we have an external system generating the events and sending the JSON to my REST service and I have to propagate the JSON to the KAFKA topics locally.
Thank you for your inputs
You can use oneOf:
#RequestBody(
required = true,
content = #Content(
schema = #Schema(oneOf = {Application.class, CourseEnrollment.class})
)
)

JWT signature different from expected [duplicate]

I wrote a method that takes a JWT as a request and checks if the signature is valid.
This is the unit test:
#Test
public void isValid() {
final JwtValidator jwtValidator = JwtValidator.getInstance();
final boolean valid = jwtValidator.isValid("eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c");
Assert.isTrue(valid);
}
and this is the code:
#SneakyThrows
public boolean isValid(String extractedToken) {
final String[] tokenParts = extractedToken.split(Pattern.quote("."));
String header = tokenParts[0];
String payload = tokenParts[1];
String signature = tokenParts[2];
final byte[] calcHmacSha256 = HMAC.calcHmacSha256("your-256-bit-secret".getBytes(), (header+"."+payload).getBytes());
final String s = Base64.getEncoder().encodeToString(calcHmacSha256);
System.out.println("'" + signature + "'.equals('"+s+"')");
return signature.equals(s);
}
The log prints two strings that differ only for 2 chars, so I feel like I'm close "but not quite" to make it work:
'SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c'.equals('SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV/adQssw5c=')
There are of course hard coded values because the implementation isn't complete, but I'm using the example values in https://jwt.io/ for ease of use right now.
Thanks!
EDIT 1:
public class JwtValidatorTest {
#Test
public void isValid() {
byte[] header64 = Base64.getEncoder().encode("{\"alg\":\"HS256\",\"typ\":\"JWT\"}".getBytes());
byte[] payload64 = Base64.getEncoder().encode("{\"sub\":\"1234567890\",\"name\":\"John Doe\",\"iat\":1516239022}".getBytes());
final byte[] calcHmacSha256 = HMAC.calcHmacSha256("your-256-bit-secret".getBytes(), (header64+"."+payload64).getBytes());
final String signature64 = Base64.getEncoder().encodeToString(calcHmacSha256);
final String input = header64 + "." + payload64 + "." + signature64;
final JwtValidator jwtValidator = JwtValidator.getInstance();
final boolean valid = jwtValidator.isValid(input);
Assert.isTrue(valid);
}
}
The difference is just caused by the different encoding used here. You used Base64 encoding, but the original signature is Base64Url encoded. Base64Url encoding is, according to RFC7519, the standard encoding for JWT:
Each part contains a base64url-encoded value
Base64Url encoding has no padding (=) on the end and the characters + and / are replaced with -and _.
This code should solve the problem:
final String s = Base64.getUrlEncoder().withoutPadding().encodeToString(calcHmacSha256);

Get Exception in thread "main" org.zaproxy.clientapi.core.ClientApiException: Does Not Exist on running form ans script authentication using zap api

public class FormAuth {
private static final String ZAP_ADDRESS = "localhost";
private static final int ZAP_PORT = 8080;
private static final String ZAP_API_KEY = null;
private static final String contextId = "1";
private static final String contextName = "Default Context";
private static final String target = "http://localhost:8090/bodgeit";
private static void setIncludeAndExcludeInContext(ClientApi clientApi) throws UnsupportedEncodingException, ClientApiException {
String includeInContext = "http://localhost:8090/bodgeit.*";
String excludeInContext = "http://localhost:8090/bodgeit/logout.jsp";
clientApi.context.includeInContext(contextName, includeInContext);
clientApi.context.excludeFromContext(contextName, excludeInContext);
}
private static void setLoggedInIndicator(ClientApi clientApi) throws UnsupportedEncodingException, ClientApiException {
// Prepare values to set, with the logged in indicator as a regex matching the logout link
String loggedInIndicator = "Logout";
// Actually set the logged in indicator
clientApi.authentication.setLoggedInIndicator(contextId, java.util.regex.Pattern.quote(loggedInIndicator));
// Check out the logged in indicator that is set
System.out.println("Configured logged in indicator regex: "
+ ((ApiResponseElement) clientApi.authentication.getLoggedInIndicator(contextId)).getValue());
}
private static void setFormBasedAuthenticationForBodgeit(ClientApi clientApi) throws ClientApiException,
UnsupportedEncodingException {
// Setup the authentication method
String loginUrl = "http://localhost:8090/bodgeit/login.jsp";
String loginRequestData = "username={%username%}&password={%password%}";
// Prepare the configuration in a format similar to how URL parameters are formed. This
// means that any value we add for the configuration values has to be URL encoded.
StringBuilder formBasedConfig = new StringBuilder();
formBasedConfig.append("loginUrl=").append(URLEncoder.encode(loginUrl, "UTF-8"));
formBasedConfig.append("&loginRequestData=").append(URLEncoder.encode(loginRequestData, "UTF-8"));
System.out.println("Setting form based authentication configuration as: "
+ formBasedConfig.toString());
clientApi.authentication.setAuthenticationMethod(contextId, "formBasedAuthentication",
formBasedConfig.toString());
// Check if everything is set up ok
System.out
.println("Authentication config: " + clientApi.authentication.getAuthenticationMethod(contextId).toString(0));
}
private static String setUserAuthConfigForBodgeit(ClientApi clientApi) throws ClientApiException, UnsupportedEncodingException {
// Prepare info
String user = "Test User";
String username = "test#example.com";
String password = "weakPassword";
// Make sure we have at least one user
String userId = extractUserId(clientApi.users.newUser(contextId, user));
// Prepare the configuration in a format similar to how URL parameters are formed. This
// means that any value we add for the configuration values has to be URL encoded.
StringBuilder userAuthConfig = new StringBuilder();
userAuthConfig.append("username=").append(URLEncoder.encode(username, "UTF-8"));
userAuthConfig.append("&password=").append(URLEncoder.encode(password, "UTF-8"));
System.out.println("Setting user authentication configuration as: " + userAuthConfig.toString());
clientApi.users.setAuthenticationCredentials(contextId, userId, userAuthConfig.toString());
clientApi.users.setUserEnabled(contextId, userId, "true");
clientApi.forcedUser.setForcedUser(contextId, userId);
clientApi.forcedUser.setForcedUserModeEnabled(true);
// Check if everything is set up ok
System.out.println("Authentication config: " + clientApi.users.getUserById(contextId, userId).toString(0));
return userId;
}
private static String extractUserId(ApiResponse response) {
return ((ApiResponseElement) response).getValue();
}
private static void scanAsUser(ClientApi clientApi, String userId) throws ClientApiException {
clientApi.spider.scanAsUser(contextId, userId, target, null, "true", null);
}
/**
* The main method.
*
* #param args the arguments
* #throws ClientApiException
* #throws UnsupportedEncodingException
*/
public static void main(String[] args) throws ClientApiException, UnsupportedEncodingException {
ClientApi clientApi = new ClientApi(ZAP_ADDRESS, ZAP_PORT, ZAP_API_KEY);
setIncludeAndExcludeInContext(clientApi);
setFormBasedAuthenticationForBodgeit(clientApi);
setLoggedInIndicator(clientApi);
String userId = setUserAuthConfigForBodgeit(clientApi);
scanAsUser(clientApi, userId);
}
}
=========================================================================================
/usr/lib/jvm/java-1.11.0-openjdk-amd64/bin/java -javaagent:/snap/intellij-idea-ultimate/319/lib/idea_rt.jar=43425:/snap/intellij-idea-ultimate/319/bin -Dfile.encoding=UTF-8 -classpath /home/arpit/IdeaProjects/maven-zap-demo/target/classes:/home/arpit/Downloads/zap-clientapi-1.9.0.jar ScriptAuth
Exception in thread "main" org.zaproxy.clientapi.core.ClientApiException: Does Not Exist
at org.zaproxy.clientapi.core.ApiResponseFactory.getResponse(ApiResponseFactory.java:50)
at org.zaproxy.clientapi.core.ClientApi.callApi(ClientApi.java:351)
at org.zaproxy.clientapi.gen.deprecated.ScriptDeprecated.load(ScriptDeprecated.java:146)
at ScriptAuth.uploadScript(ScriptAuth.java:76)
at ScriptAuth.main(ScriptAuth.java:93)
The recommended way to automate authentiation in ZAP is to configure and test it in the desktop, then export the context and import that via the API. If the authentication uses scripts then these will need to be registered with ZAP first.

How to retrieve all the Groups/Roles a user is member of using SOAP services?

I am trying to collect some user informations using SOAP services.
I was able to get the Job Title for a given user, but I don't understand how to retrieve the list of groups and roles that a user has.
Can I simply use the GroupServiceSoap.getUserPlaces(long userId, String[] classNames, int max) method? Or is there another way I can get these fields?
Currently my code:
private static URL _getURL(String remoteUser, String password, String serviceName) {
final String LIFERAY_PROTOCOL = "http://";
final String LIFERAY_TCP_PORT = "8080";
final String LIFERAY_FQDN = "localhost";
final String LIFERAY_AXIS_PATH = "/api/secure/axis/";
try {
return new URL(LIFERAY_PROTOCOL + URLEncoder.encode(remoteUser, "UTF-8") + ":"
+ URLEncoder.encode(password, "UTF-8") + "#" + LIFERAY_FQDN
+ ":" + LIFERAY_TCP_PORT + LIFERAY_AXIS_PATH + serviceName);
} catch (MalformedURLException e) {
return null;
} catch (UnsupportedEncodingException e) {
return null;
}
}
[...]
public static void main(String[] argv){
public final String LIFERAY_USER_SERVICE="Portal_UserService";
public final String LIFERAY_COMPANY_SERVICE="Portal_CompanyService";
public final String LIFERAY_GROUP_SERVICE = "Portal_GroupService";
//company.default.web.id property
public final String LIFERAY_DEFAULT_COMPANY_ID = "liferay.com";
UserServiceSoap userService = new UserServiceSoapServiceLocator().getPortal_UserService(_getURL(USER_IDENTIFIER,USER_PASSWORD, LIFERAY_USER_SERVICE));
//This code is usefull if you want to use SOAP setter.
//((Portal_UserServiceSoapBindingStub) userService).setUsername(USER_IDENTIFIER);
//((Portal_UserServiceSoapBindingStub) userService).setPassword(USER_PASSWORD);
CompanyServiceSoap companyService = new CompanyServiceSoapServiceLocator().getPortal_CompanyService(_getURL(USER_IDENTIFIER, USER_PASSWORD, LIFERAY_COMPANY_SERVICE));
long companyId = companyService.getCompanyByMx(LIFERAY_DEFAULT_COMPANY_ID).getCompanyId();
// Here I retrieve my user, and can access some properties, but not them all !
UserSoap user = userService.getUserByEmailAddress(companyId, target_user_mail);
//TODO : I got hte JobTittle that I want, later I will do something more util thant just print it, I swear it my precious !
System.out.println(user.getJobTitle());
GroupServiceSoap groupService = new GroupServiceSoapServiceLocator().getPortal_GroupService(_getURL(USER_IDENTIFIER, USER_PASSWORD, LIFERAY_GROUP_SERVICE));
//this one return an empty array
GroupSoap[] userPlaces = groupService.getUserPlaces(new String[]{"Group", "Role"}, 150);
//this return an array of size 1, but the only GroupSoap seems to be a structural groups without any usefull properties to me.
GroupSoap[] userPlaces = groupService.getUserPlaces(null, 150);
}
Use this method to get user role and group user id
UserServiceSoap.getRoleUserIds
UserServiceSoap.getGroupUserIds
HTH
It is only a partial answer.
In order to get all the User Roles one can do this :
RoleServiceSoap roleService = new RoleServiceSoapServiceLocator().getPortal_RoleService(_getURL(USER_IDENTIFIER, USER_PASSWORD, LIFERAY_ROLE_SERVICE));
RoleSoap[] userRoles = roleService.getUserRoles(user.getUserId());
with user variable an instance of UserSoap.
The SOAP access must be done by an Admin user in order to get access to the Role List. The user can't access this himself.

Google Cloud Storage - JAVA REST API - Getting SignatureDoesNotMatch

I am using jersey-client to make REST Call . I am getting SignatureDoesNotMatch error in response.
I was trying to List down Bucket names using GET Service , also tried to list Bucket object using GET Bucket method.
here is my sample code.
Any hint or solution ?
public class restSample {
private static final String HMAC_SHA1_ALGORITHM = "HmacSHA1";
private static final String PROJECT_ID = "10XXXXXXXX478";
public static String Base64Encoding()
throws java.security.SignatureException, UnsupportedEncodingException {
String access_id = "GOOGBAXXXXXXXXXXBI";
String secret_key = URLEncoder.encode("pWTXXXXXXXXXXXXXXXRo85T+XXXXXXXXX3O","UTF-8");
String bucket = "bucket_name";
String version_header = "x-goog-api-version:1";
String project_header = "x-goog-project-id:"+PROJECT_ID;
String canonicalizedResources = "/"+bucket+"/";
Calendar calendar = Calendar.getInstance();
calendar.add(Calendar.MINUTE, 30);
long expiration = calendar.getTimeInMillis();
String stringToSign = URLEncoder.encode("GET\n\n\n"+expiration+"\n"+version_header+"\n"+project_header+"\n"+canonicalizedResources,"UTF-8");
//String stringToSign = URLEncoder.encode("GET\n\n\n"+getdate()+"\n"+version_header+"\n"+project_header+"\n"+canonicalizedResources,"UTF-8");
String authSignature="";
try {
SecretKeySpec signingKey = new SecretKeySpec(secret_key.getBytes(),HMAC_SHA1_ALGORITHM);
Mac mac = Mac.getInstance(HMAC_SHA1_ALGORITHM);
mac.init(signingKey);
// compute the hmac on input data bytes
byte[] rawHmac = mac.doFinal(stringToSign.getBytes("UTF-8"));
// base64-encode the hmac
authSignature = new String(Base64.encode(rawHmac));
} catch (Exception e) {
throw new SignatureException("Failed to generate HMAC : " + e.getMessage());
}
authSignature = (access_id +":"+ authSignature);
return authSignature;
}
public static void main(String[] args) {
ClientConfig config = new DefaultClientConfig();
Client client = Client.create(config);
String authSignature = null;
try {
authSignature = "GOOG1 "+ Base64Encoding();
} catch (SignatureException e) {
e.printStackTrace();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
WebResource service = client.resource(getBaseURI());
ClientResponse response = service.accept(MediaType.APPLICATION_XML)
.header("Authorization",authSignature)
.header("Date", getdate())
.header("Content-Length", "0")
.header("x-goog-api-version", "1")
.header("x-goog-project-id", PROJECT_ID)
.get(ClientResponse.class);
System.out.println(response.getClientResponseStatus().getFamily());
System.out.println("response1 :: " + response.getEntity(String.class));
}
private static URI getBaseURI() {
String url = "https://bucket_name.storage.googleapis.com";
return UriBuilder.fromUri(url).build();
}
private static String getdate(){
SimpleDateFormat format = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss z ", new Locale("US"));
Calendar cal = Calendar.getInstance(new SimpleTimeZone(0, "GMT"));
format.setCalendar(cal);
return format.format(new Date());
}
}
Thanks!
Make sure the string you are signing matches the expected string to sign. Google cloud storage returns the expected string to sign in the HTTP response if authentication fails.
In your particular example it looks like you are adding both the version_header and project_header into the string to sign. These are not in the list of CanonicalHeaders nor CanonicalExtensionHeaders, so you are signing a different string than the server.
You can review the list here: https://developers.google.com/storage/docs/reference/v1/developer-guidev1#authentication