Google Storage JSON API NullPointException While Doing Object Downloading - google-cloud-storage

When I try to download a Storage object by using following codes I encountered a NullPointException:
Storage.Objects.Get getObject = storage.objects().get(BUCKET_LOG, entity);
getObject.getMediaHttpDownloader().setDirectDownloadEnabled(true);//directly download, which is recommended for non AppEngine Apps.
The stack trace is:
at com.google.api.client.googleapis.media.MediaHttpDownloader.download(MediaHttpDownloader.java:186)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeMediaAndDownloadTo(AbstractGoogleClientRequest.java:553)
at com.google.api.services.storage.Storage$Objects$Get.executeMediaAndDownloadTo(Storage.java:4606)
at com.alibaba.intl.google.adwords.util.GoogleStorageUtil.downloadStorageObject(GoogleStorageUtil.java:149)
So I took a look at line 186 of MediaHttpDownloader and found this:
mediaContentLength = response.getHeaders().getContentLength();
Well, here is the problem, method getContentLength might return null value, while the type of variable mediaContentLength is long, but not Long.
A piece of the snapshot for class HttpHeaders:
public final Long getContentLength() {
return getFirstHeaderValue(contentLength);
}
/**Returns the first header value based on the given internal list value.*/
private <T> T getFirstHeaderValue(List<T> internalValue) {
return internalValue == null ? null : internalValue.get(0);
}
I am using Java Library for AdWords API V201402.

Related

Using AcceptedAtActionResult with Azure Function to direct to specific method

I have a API that accepts a submission request (that when accepted to return http code 202 Accepted), for some workflow to approve, and then the caller to retrieve the result later.
The URL should be returned on successful submission and I am trying to use AcceptedAtActionResult with Azure Functions, however I do not understand how to use it as they don't have Controllers per my understanding, but this is one of the parameters.
Regardless and trying to use string.Empty, or the class name EnrollmentFunctions, or path enroll, the library is generating strange paths (in this case pointing to a different function in a different file i.e. location: http://localhost:7071/api/attest/c588484e-8e57-47f6-bf6c-973cfa5b9214?action=GetStatus&controller=enroll).
I am looking to return of the form location: http://localhost:7071/api/enroll/c588484e-8e57-47f6-bf6c-973cfa5b9214 (but avoid hardcoding URLs, both for maintenance and also running in dev mode on local machine).
I believe this would get more complicated if we used a cloud WAF (like cloudflare) on the front-end, in that case I presume would create my own class to build the URL (i.e. https://api-protected-by-waf.mydomain.com/enroll/bf920104-a630-4b58-97cb-cc3ae45c31d3)? I can see this then becoming a configuration issue was looking to avoid on the previous para.
Research points to creating my own IUrlHelper and possibly specifying key:values for retrival via Environment.GetEnvironmentVariable("your_key_here") (for use by my IUrlHelper implementation).
Thanks in advance on how to solve this issue or if i am using the libraries incorrectly and alternate best practice. Code of the Azure functions are below:
public class EnrollmentFunctions
{
[FunctionName("Enroll")]
public async Task<IActionResult> Enroll(
[HttpTrigger(AuthorizationLevel.Function, "post", Route = "enroll")] HttpRequest req)
{
var id = Guid.NewGuid();
return new AcceptedAtActionResult("GetStatus", "enroll", new { id = id }, id);
// return new AcceptedAtRouteResult("GetStatus", (new { id }, new { Result = id.ToString() }));
}
[FunctionName("GetStatus")]
public async Task<IActionResult> GetStatus(
[HttpTrigger(AuthorizationLevel.Function, "get", Route = "enroll/{id}")] HttpRequest req)
{
return new OkObjectResult("OK");
}
}

basic auth respond 500 in postman when it should be OK(Authorized), but 401 Unathorized works fine. using ktor intellij mongodb

java.lang.NoSuchMethodError: kotlinx.coroutines.channels.LinkedListChannel: method ()V not found
this is my database.kt
private val client = KMongo.createClient().coroutine
private val database = client.getDatabase("PestaDanDagangDatabase")
private val users = database.getCollection()
private val posts = database.getCollection()
suspend fun getPostsForUser(email: String) : List<> {
return posts.find(Post::members contains email).toList()
}
i'm following some tutorial... getPostsForUser return List of Post data class(it is disappear.. still need to learn to use stackoverflow), the tutorial project is running even i followed the same. i changed the return to be return posts.find(Post::members contains email).toString and the function return String both my code and my tutorial result ok in postman with same respond org.litote.kmongo.coroutine.CoroutineFindPublisher#44fc57b6(except 44fc57b6).
i tried to change the return to be return posts.find(Post::members contains email) and function return type CoroutineFindPublisher of type Post, it is OK in postman but the event time is keep ticking or running.
I've encountered kinda the same problem, but for me the implementation were updated to latest version (as today kmongo 4.4.0 & coroutines 1.6.0), the error was
kotlinx.coroutines.channels.LinkedListChannel: method 'void <init>() not found
I realized that there was a import conflict with the .toList() method
To solve it I simply, instead of
return posts.find(Post::members contains email).toList()
used
return posts.find(Post::members contains email).publisher.KMongoToList()
with as import
import org.litote.kmongo.coroutine.toList as KMongoToList

Customising Spring Boot Exception Handling to Prevent Stacktraces Being Returned in Rest Response

How do I configure my spring boot service so that errors such as 500 don't potentially leak implementation details such as stacktraces.
{
"timestamp": "2019/05/01 15:06:17",
"status": 500,
"error": "Internal Server Error",
"message": "Type definition error: [simple type, class net.i2p.crypto.eddsa.math.ed25519.Ed25519LittleEndianEncoding]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class net.i2p.crypto.eddsa.math.ed25519.Ed25519LittleEndianEncoding and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.Collections$UnmodifiableRandomAccessList[0]->........)",
"path": "/api/test"
}
Note: here the stacktrace is in the message and not the exception part of the json.
As you can see I am already formatting the timestamp with:
#Component
public class CustomErrorAttributes extends DefaultErrorAttributes {
private static final DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
private static final String TIMESTAMP = "timestamp";
#Override
public Map<String, Object> getErrorAttributes(WebRequest webRequest, boolean includeStackTrace) {
//Let Spring handle the error first
Map<String, Object> errorAttributes = super.getErrorAttributes(webRequest, includeStackTrace);
//Format & update timestamp
Object timestamp = errorAttributes.get(TIMESTAMP);
if(timestamp == null) {
errorAttributes.put(TIMESTAMP, dateFormat.format(new Date()));
} else {
errorAttributes.put(TIMESTAMP, dateFormat.format((Date)timestamp));
}
return errorAttributes;
}
}
But I need to handle the message too.
If this 500 was the only error I could just do:
errorAttributes.put("message", "Server error. Contact support.");
However, all the errors go through here and that would override all the messages.
I could check if the status is 500 and only modify it then. However, there are other errors that can be generated that also might leak stacktraces.
Using #RestControllerAdvice seems to require knowing every exception that is generated and having an #ExceptionHandler for each and knowing which status code to respond with.
Is there a cleaner way to handle this?
It may not be the "cleanest" approach, but with projects I've been on we had a "standard format" for our Error Responses across projects, so we had a custom object with the fields that matched our orgs standard (HttpStatus, Reason, ect.) that extended RuntimeException. Then in our controllers, services, repos, ect we would catch exceptions and create this object accordingly and throw the custom one up instead. Based upon where it happened in the app (repo, service, controller, ect.) we could give our own custom verbage to it, but still log out the full exception in our server logs so we could investigate later
For example if we caught an error in our repository we would create our custom error object, set the Reason to DB unavailable (really all the consumer needs to know), set the status to HttpStatus.SERVICE_UNAVAILABLE (we tracked these with reasons and httpstatus with enums to keep status the same across modules), and throw the custom object up to the controller to be returned.
Sorry if this was a longwinded answer that may not give you what you want, I'm not too familiar with how you're trying to do it so figured I'd just give an example of other methods. I'll put some sample code as well
Custom Exception:
data class MyException(
val reason: String,
val httpStatus: HttpStatus? = null
) : RuntimeException(reason)
Method for creation:
fun createApiException(errorCode: ErrorCodeEnum) = MyException(
reason = errorCode.reason,
httpStatus = errorCode.httpStatus,
)
Spring-boot provides us with a standard method to handle exceptions using spring aop concept. You can use the #ControllerAdvice and #Exceptionhandled annotations to handle exceptions from a spring-boot rest endpoint so that a custom exception is always thrown from a rest endpoint with proper error code and error response.
The #ResponseStatus() annotation can be used to customize the response code being thrown.
For example consider the custom exception :
#ResponseStatus(HttpStatus.NOT_FOUND)
public class DataNotFoundException extends RuntimeException {
public DataNotFoundException(String exception) {
super(exception);
}
}
We can throw this error from a rest GET mapping when a data is not found like :
#GetMapping("/trains/{id}")
public Resource<Student> retrieveTrains(#PathVariable long id) {
Optional<Trains> trains = trainRepository.findById(id);
if (!train.isPresent())
throw new DataNotFoundException("id-" + id);
Resource<Trains> resource = new Resource<Trains>(train.get());
ControllerLinkBuilder linkTo = linkTo(methodOn(this.getClass()).retrieveAllTrains());
resource.add(linkTo.withRel("all-trains"));
return resource;
}
Default error response provided by Spring Boot contains all the details that are typically needed.
However, you might want to create a framework independent response structure for your organization. In that case, you can define a specific error response structure.
For example :
public class ErrorDetails {
private Date timestamp;
private String message;
private String details;
public ErrorDetails(Date timestamp, String message, String details) {
super();
this.timestamp = timestamp;
this.message = message;
this.details = details;
}
To use this error node we use :
#ControllerAdvice
public class CustomizedResponseEntityExceptionHandler extends ResponseEntityExceptionHandler {
#ExceptionHandler(DataNotFoundException.class)
public final ResponseEntity<ErrorDetails> handleUserNotFoundException(DataNotFoundException ex, WebRequest request) {
ErrorDetails errorDetails = new ErrorDetails(new Date(), ex.getMessage(),
request.getDescription(false));
return new ResponseEntity<>(errorDetails, HttpStatus.NOT_FOUND);
}
#ExceptionHandler(DataNotFoundException.class) indicates that this
method would handle exceptions of the specific type.
new ResponseEntity<>(errorDetails, HttpStatus.NOT_FOUND) - Create an
error response object and return it with a specific Http Status.
For a more generalized exception handler you can define a method that handles exception of the type Exception.class, that way you don't have to know every exception.
Like :
#ExceptionHandler(Exception.class)
public final ResponseEntity<ErrorDetails> handleAllExceptions(Exception ex, WebRequest request) {
ErrorDetails errorDetails = new ErrorDetails(new Date(), ex.getMessage(),
request.getDescription(false));
return new ResponseEntity<>(errorDetails, HttpStatus.INTERNAL_SERVER_ERROR);
}
Reference from : https://www.javaguides.net/2019/02/spring-boot-2-angular-7-crud-example-tutorial.html

Spring MVC REST using #RequestBody List<?> returns HTTP 400 syntactically incorrect

I am using Spring 4 + Jackson 2 and have written a fully functional POST method using #RequestBody on a custom class. This method has no trouble unmarshalling the object.
#ResponseBody
#RequestMapping(value="store", method = RequestMethod.POST)
public ServiceResponse store(#RequestBody CustomClass list) {
...
}
// Request: { code: "A", amount: 200 }
When I attempted to add another method to handle a collection of the same class instead, my POST requests were returning with the following error.
HTTP Status 400: The request sent by the client was syntactically incorrect.
I note that this error typically occurs when the JSON submitted does not match the entity class. However, all I am doing is submitting an array of the same object instead of the object itself, which has already proven to work.
#ResponseBody
#RequestMapping(value="store-bulk", method = RequestMethod.POST)
public ServiceResponse storeBulk(#RequestBody List<CustomClass> list) {
...
}
// Request: [{ code: "A", amount: 200 }, { code: "B", amount: 400 }]
Am I missing something here?
In Java, type information for generics is erased at runtime, so Spring sees your List<CustomClass> object as List<Object> object, thus it cannot understand how to parse it.
One of ways to solve it, you could capture the type information by creating a wrapper class for your list, like this:
public class CustomClassList extends ArrayList<CustomClass> {
}
Sergey is right that the issue is due to type erasure. Your easiest way out is to bind to an array, so
#ResponseBody
#RequestMapping(value="store-bulk", method = RequestMethod.POST)
public ServiceResponse storeBulk(#RequestBody CustomClass[] object) {
...
}
The answer is that Spring 4 doesn't actually get rid of type erasure, contrary to what some other solutions suggest. While experimenting on debugging via manual unmarshalling, I decided to just handle that step myself instead of an implicit cast that I have no control over. I do hope someone comes along and proves me wrong, demonstrating a more intuitive solution though.
#ResponseBody
#RequestMapping(value="store-bulk", method = RequestMethod.POST)
public ServiceResponse storeBulk(#RequestBody String json) {
try {
List<CustomClass> list = new ObjectMapper().readValue(json, new TypeReference<List<CustomClass>>() { });
...
} catch (Exception e) {
...
}
}
Bonus: Right after I got this working, I bumped into this exception:
IllegalStateException: Already had POJO for id
If anyone gets this, it's because the objects in the list happen to reference some object that another item in the list already references. I could work around this since that object was identical for my entire collection, so I just removed the reference from the JSON side from all but the first object. I then added the missing references back after the JSON was unmarshalled into the List object.
Two-liner for the Java 8 users (the User object reference was the issue in my case):
User user = list.get(0).getUser();
list.stream().filter(c -> c.getUser() == null).forEach(t -> t.setUser(user));

Clientside error (Cannot call method 'nullMethod' of null arguments: nullMethod)

My App engine/ GWT project is spitting out a nasty little pile of stack trace whenever it attempts to return from my login method. I am using GAE version 1.5.0 and GWT version 2.3.0 .
It's a facebook app, so what I've got is this:
The player navigates to the app page.
They click a button, and are redirected to the OAuth authentication page
They are then redirected back to the app, with the authentication token in the query string
I break the query string apart to get the UID, and then use that as the primary key for my Player entity (RPC to app engine backend)
I retrieve the Player entity instance from the datastore, and turn it into a serializable type to return to the client
Epic fail.
When I spit out the exception in a JSAlert, I get a big nasty pile of stack trace (I already was thoughtful enough to compile using "pretty" instead of "obfuscated").
My login function looks like this:
#Override
public ClientPlayer login(String uid) {
PersistenceManager pm=PMF.get().getPersistenceManager();
log.warning(Player.class.getName());
log.warning(uid);
Key k=KeyFactory.createKey(Player.class.getSimpleName(), uid);
Player p;
List<List<Integer>> stats;
try{
p=pm.getObjectById(Player.class, k);
} catch (JDOObjectNotFoundException e){
p=new Player(uid);
p.setKey(k);
pm.makePersistent(p);
} finally {
pm.close();
}
stats=p.getStats();
return new ClientPlayer(p.getUID(),p.getPerm(), p.getDecks(),stats.get(0), stats.get(1), stats.get(2));
}
Unfortunately, due to NDA, I can't link to the app, but here's the output:
Failure to log in because of:
com.google.gwt.core.client.JavaScriptException: (TypeError): Cannot call method 'nullMethod' of null
arguments: nullMethod,
type: non_object_property_call
stack: TypeError: Cannot call method 'nullMethod' of null
at Object.ClientPlayer_1 (http://*.com/com.MES.Tap2/A37A2E2E9A65DB1BAAE2BFA42572F7F8.cache.html:993:89)
at Object.ClientPlayer_0 (http://*com/com.MES.Tap2/A37A2E2E9A65DB1BAAE2BFA42572F7F8.cache.html:984:18)
at Array.instantiate_1 [as 0] (http://*.com/com.MES.Tap2/A37A2E2E9A65DB1BAAE2BFA42572F7F8.cache.html:1031:10)
at $instantiate_0 (http://*.com/com.MES.Tap2/A37A2E2E9A65DB1BAAE2BFA42572F7F8.cache.html:10660:34)
at $instantiate (http://*.com/com.MES.Tap2/A37A2E2E9A65DB1BAAE2BFA42572F7F8.cache.html:1948:10)
at $readObject (http://*.com/com.MES.Tap2/A37A2E2E9A65DB1BAAE2BFA42572F7F8.cache.html:10148:95)
at Object.read_8 [as read] (http://*.com/com.MES.Tap2/A37A2E2E9A65DB1BAAE2BFA42572F7F8.cache.html:10608:10)
at $onResponseReceived (http://*.com/com.MES.Tap2/A37A2E2E9A65DB1BAAE2BFA42572F7F8.cache.html:10352:247)
at $fireOnResponseReceived (http://*.com/com.MES.Tap2/A37A2E2E9A65DB1BAAE2BFA42572F7F8.cache.html:5002:5)
at Object.onReadyStateChange (http:/*.com/com.MES.Tap2/A37A2E2E9A65DB1BAAE2BFA42572F7F8.cache.html:5222:5)
The issue was in the use of the IsSerializable interface, or rather my poor understanding of it.
When you create an IsSerialiazable object, it requires a no-argument constructor. I was passing null values from that constructor to the main constructor, so when methods were called on them, null pointer exceptions occurred. This was dumb of me, but hey, it was a learning experience.
In my particular case, it went a little like this...
public class ClientObject implements IsSerializable {
private Object field1;
private Object field2;
private String field3;
public ClientObject(){
this(null, null);
}
public ClientObject(Object arg1, Object arg2){
field1=arg1;
field2=arg2;
field3=arg1.toString()+arg2.toString();
//Error on above line, though not obviously mentioned in the message
}
}
What should have been done was...
public ClientObject(){
this(new Object(), new Object());
}
Hope this helps someone.