Is there a way to log correlation id (sent from microservice) in Postgres - postgresql

I am setting up the logging for a project and was wondering, whether it is possible to send id to postges database. Later I would collect all logs with fluentd and use the efk(elastic search, fluentd, kibana) stack to look through the logs. That is why it would be very helpful if can set the id in the database logs.

You can have your connection set the application_name to something that includes the id you want, and then configure your logging to include the application_name field. Or you could include the id in an SQL comment, something like select /* I am number 6 */ whatever from whereever. But then the id will only be visible for log messages that include the sql. (The reason for putting the comment after the select keyword is that some clients will strip out leading comments so the serve never sees them)

Thank you #jjane,
for giving me such a great idea. After some googling I found a page describing how to intercept hibernate logs
then I made some more research on how to add the inspector to spring-boot and this is the solution I came up with:
#Component
public class SqlCommentStatementInspector
implements StatementInspector {
private static final Logger LOGGER = LoggerFactory
.getLogger(
SqlCommentStatementInspector.class
);
private static final Pattern SQL_COMMENT_PATTERN = Pattern
.compile(
"\\/\\*.*?\\*\\/\\s*"
);
#Override
public String inspect(String sql) {
LOGGER.info(
"Repo log, Correlation_id:"+ MDC.get(Slf4jMDCFilter.MDC_UUID_TOKEN_KEY),
sql
);
return String.format("/*Correlation_id: %s*/", MDC.get(Slf4jMDCFilter.MDC_UUID_TOKEN_KEY)) + SQL_COMMENT_PATTERN
.matcher(sql)
.replaceAll("");
}
}
and its configuration:
#Configuration
public class HibernateConfiguration implements HibernatePropertiesCustomizer {
#Autowired
private SqlCommentStatementInspector myInspector;
#Override
public void customize(Map<String, Object> hibernateProperties) {
hibernateProperties.put("hibernate.session_factory.statement_inspector", myInspector);
}
}

Related

Why is my data not persisted/accessible in an Spring-Boot integration-test with HTTPGraphQLTester and TestEntityManager

I have a bare-bones Spring-Boot app with some GraphQL endpoints and a Postgres database and want to run an integration test against an endpoint. It should find an entity by its ID and does so without a problem when I send a request manually via Postman. However when I write an integration test for the controller it doesn't. The data seems to be saved after using
TestEntityManager (or the JpaRepository directly) an I get the entity back with its ID. I then stick that ID into a query with HttpGraphQlTester which fails with an empty result/null. I traced it with the debugger and discovered that when the endpoint calls the repository to retrieve the entity with the given ID it gets null or when I look at all the repo-contents it's just an empty list. So my data seems to be accessible in my test but not in my repo/service. Any pointers would be very much appreciated.
Test
#SpringBootTest
#AutoConfigureHttpGraphQlTester
#AutoConfigureTestEntityManager
#Transactional
public class BackboneTreeControllerTest {
#Autowired
HttpGraphQlTester tester;
#Autowired
private TestEntityManager testEntityManager;
#Test
void findTaxon() {
Taxon taxon = Taxon.builder()
.path(Arrays.asList("path", "to", "taxon"))
.nameCanonical("Cocos nucifera")
.authorship("Me")
.extinct(false)
.numDescendants(1l)
.numOccurrences(1l)
.build();
Taxon savedTaxon = testEntityManager.persistFlushFind(taxon); // (1)
this.tester.documentName("queries")
.operationName("FindTaxon")
.variable("taxonId", savedTaxon.getId())
.execute()
.path("findTaxon.authorship")
.entity(String.class)
.isEqualTo("Me");
the testEntityManager returns successfully with an ID.
Query
query FindTaxon($taxonId: ID!) {
findTaxon(id: $taxonId) {
authorship
}
}
Controller
#Controller
#AllArgsConstructor
public class BackboneTreeController {
private final TaxonService taxonService;
#QueryMapping
public Taxon findTaxon(#Argument Integer id) {
Optional<Taxon> taxon = taxonService.findTaxon(id);
return taxon.orElse(null);
}
}
Service
#Service
#AllArgsConstructor
public class TaxonService {
private final TaxonRepository taxonRepository;
public Optional<Taxon> findTaxon(Integer id) {
return taxonRepository.findById(id); // (2)
}
}
This is where I would expect the repo to return the entity but it does not. Also using .findAll here returns an empty list.
Repository
#Repository
public interface TaxonRepository extends JpaRepository<Taxon, Integer> {
}
Note that everything works fine when I just run the app and send the exact same query manually!
I don't know HttpGraphQlTester but I'd assume that it generates requests which then get processed in a separate thread.
That thread won't see the changes made in the test because they aren't committed yet.
If this is the reason resolve it by putting the setup in it's own transaction, for example by using TransactionTemplate.

How Can I have multiples instances of a Spring boot Repository(Interface), to have a complete test-state-isolation?

1) Contextualization:
In order, to have a complete test-isolation-state in all test of my Test-Class;
I would like to have a new-instance-repository(DAO) for each individual test;
My Repository is a Interface, thats the why I can not simply instantiate that.
My Goal is:
Run all tests 'Parallelly', meaning 'at the same time';
That's the why, I need individual/multiple instances of Repository(DAO) in each test;
Those multiple instances will make sure that the tests' conclusion would not interfere on those that still is running.
Below is the code for the above situation:
1.1) Code:
Current working status: working, BUT with ths SAME-REPOSITORY-INSTANCE;
Current behaviour:
The tests are not stable;
SOMETIMES, they interfere in each other;
meaning, the test that finalize early, destroy the Repository Bean that still is being used, for the test that is still running.
public class ServiceTests2 extends ConfigTests {
private List<Customer> customerList;
private Flux<Customer> customerFlux;
#Lazy
#Autowired
private ICustomerRepo repo;
private ICustomerService service;
#BeforeEach
public void setUp() {
service = new CustomerService(repo);
Customer customer1 = customerWithName().create();
Customer customer2 = customerWithName().create();
customerList = Arrays.asList(customer1,customer2);
customerFlux = service.saveAll(customerList);
}
#Test
#DisplayName("Save")
public void save() {
StepVerifier.create(customerFlux)
.expectNextSequence(customerList)
.verifyComplete();
}
#Test
#DisplayName("Find: Objects")
public void find_object() {
StepVerifier
.create(customerFlux)
.expectNext(customerList.get(0))
.expectNext(customerList.get(1))
.verifyComplete();
}
}
2) The ERROR happening:
This ERROR happens in the failed-Tests:
3) Question:
How Can I create multiple instances of Repository
Even if, it being a Interface(does not allow instantation)?
In order, to have a COMPLETE TEST-ISOLATION
Meaning: ONE different instance of Repository in each test?
Thanks a lot for any help or idea
You can use the #DirtiesContext annotation on the test class that modifies the application context.
Java Doc
Spring documentation
By default, this will mark the application context as dirty after the entire test class is run. If you would like to mark the context as dirty after a single test method, then you can either annotate the test method instead or set the classMode property to AFTER_EACH_TEST_METHOD at your class level annotation.
#DirtiesContext(classMode = ClassMode.AFTER_EACH_TEST_METHOD)
When an application context is marked dirty, it is removed from the
testing framework's cache and closed; thus the underlying Spring
container is rebuilt for any subsequent test that requires a context
with the same set of resource locations.

Detecting when the database session gets refreshed on a Spring Boot 2 application

I'm trying to execute the following SQL statement every time the Database Session gets refreshed. I have a Spring Boot 2.0.1.RELEASE with JPA application and a PostgreSQL Database.
select set_config('SOME KEY', 'SOME VALUE', false);
As the PostgreSQL documentation states the is_local parameter is used to indicate that this configuration value will apply just for the current transaction -if true- or will be attached to the session (as I require) -if false-
The problem is that I'm not aware when Hibernate/Hikari are refreshing the db session, so, in practice, the application start failing when it has a couple of minutes running, as you can imagine...
My approach -that is not working yet- is to implement a EmptyInterceptor, for that I have added a DatabaseCustomizer class to inject my hibernate.session_factory.interceptor properly in a way that Spring can fill out all my #Autowires
DatabaseInterceptor.class
#Component
public class DatabaseInterceptor extends EmptyInterceptor {
#Autowired
private ApplicationContext context;
#Override
public void afterTransactionBegin(Transaction tx) {
PersistenceService pc = context.getBean(PersistenceService.class);
try {
pc.addPostgresConfig("SOME KEY", "SOME VALUE");
System.out.println("Config added...");
} catch (Exception e) {
e.printStackTrace();
}
}
}
DatabaseCustomizer.class
#Component
public class DatabaseCustomizer implements HibernatePropertiesCustomizer {
#Autowired
private DatabaseInterceptor databaseInterceptor;
#Override
public void customize(Map<String, Object> hibernateProperties) {
hibernateProperties.put("hibernate.session_factory.interceptor", databaseInterceptor);
}
}
Obviously, there is a problem with this approach because when I #Override the afterTransactionBegin method to start another transaction I get an Infinite loop.
I tried to look something inside that Transaction tx that could help to be sure that this transaction is not being generated by my own addPostgresConfig but there is not much on it.
Is there something else I could try to achieve this?
Thanks in advance,

Problems while connecting to two MongoDBs via Spring

I'm trying to achieve to connect to two different MongoDBs with Spring (1.5.2. --> we included Spring in an internal Framework therefore it is not the latest version yet) and this already works partially but not fully. More precisely I found a strange behavior which I will describe below after showing my setup.
So this is what I done so far:
Project structure
backend
config
domain
customer
internal
repository
customer
internal
service
In configI have my Mongoconfigurations.
I created one base class which extends AbstractMongoConfiguration. This class holds fields for database, host etc. which are filled with the properties from a application.yml. It also holds a couple of methods for creating MongoClient and SimpleMongoDbFactory.
Furthermore there are two custom configuration classes. For each MongoDB one config. Both extend the base class.
Here is how they are coded:
Primary Connection
#Primary
#EntityScan(basePackages = "backend.domain.customer")
#Configuration
#EnableMongoRepositories(
basePackages = {"backend.repository.customer"},
mongoTemplateRef = "customerDataMongoTemplate")
#ConfigurationProperties(prefix = "customer.mongodb")
public class CustomerDataMongoConnection extends BaseMongoConfig{
public static final String TEMPLATE_NAME = "customerDataMongoTemplate";
#Override
#Bean(name = CustomerDataMongoConnection.TEMPLATE_NAME)
public MongoTemplate mongoTemplate() {
MongoClient client = getMongoClient(getAddress(),
getCredentials());
SimpleMongoDbFactory factory = getSimpleMongoDbFactory(client,
getDatabaseName());
return new MongoTemplate(factory);
}
}
The second configuration class looks pretty similar. Here it is:
#EntityScan(basePackages = "backend.domain.internal")
#Configuration
#EnableMongoRepositories(
basePackages = {"backend.repository.internal"}
mongoTemplateRef = InternalDataMongoConnection.TEMPLATE_NAME
)
#ConfigurationProperties(prefix = "internal.mongodb")
public class InternalDataMongoConnection extends BaseMongoConfig{
public static final String TEMPLATE_NAME = "internalDataMongoTemplate";
#Override
#Bean(name = InternalDataMongoConnection.TEMPLATE_NAME)
public MongoTemplate mongoTemplate() {
MongoClient client = getMongoClient(getAddress(), getCredentials());
SimpleMongoDbFactory factory = getSimpleMongoDbFactory(client,
getDatabaseName());
return new MongoTemplate(factory);
}
}
As you can see, I use EnableMongoRepositoriesto define which repository should use which connection.
My repositories are defined just like it is described in the Spring documentation.
However, here is one example which is located in package backend.repository.customer:
public interface ContactHistoryRepository extends MongoRepository<ContactHistoryEntity, String> {
public ContactHistoryEntity findById(String id);
}
The problem is that my backend always only uses the primary connection with this setup. Interestingly, when I remove the beanname for the MongoTemplate (just #Bean) the backend then uses the secondary connection (InternalMongoDataConnection). This is true for all defined repositories.
My question is, how can I achieve that my backend really take care of both connections? Probably I missed to set another parameter/configuration?
Since this is a pretty extensive post I apologise if I forgot something to mention. Please ask for missing information in the comments.
I found the answer.
In my package structure there was a empty configuration class (of my colleague) with the annotation #Configurationand #EnableMongoRepositories. This triggered the automatic wiring process of Stpring Data and therefore led to the problems I reported above.
I simply deleted the class and now it works as it should!

How to properly validate an EntityProxy from a ListEditor in GWT RequestFactory?

I'm havin a problem editing a proxy from a ListEditor, and flushing the modifications back to the ListEditor, if the edited proxy has Constraint Validations.
I have the following case:
A user has one or many Addresses. For a simpler example, the Address object has an ID, and a String property(that represents the proper address.
The AddressProxy it's defined in the following lines:
#ProxyFor(value = Address.class, locator = AddressLocator.class)
public interface AddressProxy extends EntityProxy {
public Long getId();
public void setId(Long id);
#NotNull(message = "The address must not be NULL.")
public String getAddress();
public void setAddress(String address);
}
So I have an UserEditor (Editor<"UserProxy>), that has a ListEditor<"AddressProxy, LeafValueEditor<"AddressProxy>>
The ListEditor is mapped to a CellTable, and I have a button after the table, to edit a certain Address from the Addresses List.
If I edit an Address from the list, and it has Constrain Validation errors(i delete the address String), the address from the ListEditor it's updated with the error values.
The address Editor looks like this:
public class AddressEditor extends DialogBox implements LeafValueEditor<AddressProxy> {
...
public interface AddressEditorDriver extends RequestFactoryEditorDriver<AddressProxy, AddressEditor> {}
AddressEditorDriver driver = GWT.create(AddressEditorDriver .class);
Validator validator = Validation.buildDefaultValidatorFactory().getValidator();
private AddressProxy proxy;
...
public void edit(UserContext context, AddresProxy proxy) {
this.proxy = proxy;
driver.edit(proxy, context);
}
...
public void onSaveButtonClick() {
// this is where the original proxy it's updated
driver.flush();
Set<ConstraintViolation<AddressProxy>> constraints = validator.validate(proxy, Default.class);
if(constraints.size() != 0) {
Window.alert("Error encountered");
}
}
}
So what am I doing wrong? How can i Validate the proxy from the editor, without flushing it to the original proxy ? How can I flush the proxy from driver, only after the modified proxy it's valid ?
You can't.
Validation needs to operate on the object, so you need to flush the AddressEditorDriver; and because proxies cannot be edited by two request contexts at once, you're forced to use the request context from the other driver, which will give you the same editable proxy as in the other driver, so when you flush the AddressEditorDriver you're modifying the original proxy.
There are workarounds but they all involve a non-negligible amount of work and code:
You could clone your proxy (for which there's no easy way to do) before editing it in the AddressEditorDriver and applying the changes back to the original proxy after flush.
Maybe you could use an EditorVisitor on your AddressEditorDriver and validate each field as you visit it, so you don't need to flush the driver (which would update the original proxy)
Similarly, you could use only sub-editors in the AddressEditor that would validate the field and report the error instead of updating the proxy (i.e. getValue returns the original value from the proxy –the one passed to setValue– rather than the invalid value from the editor)