How to set the offset.commit.policy to AlwaysCommitOffsetPolicy in debezium? - debezium

I created a Debezium Embedded engine to capture MySQL change data. I want to commit the offsets as soon as I can. In the code, the config is created including follows.
.with("offset.commit.policy",OffsetCommitPolicy.AlwaysCommitOffsetPolicy.class.getName())
Running this returns, java.lang.NoSuchMethodException: io.debezium.embedded.spi.OffsetCommitPolicy$AlwaysCommitOffsetPolicy.<init>(io.debezium.config.Configuration)
However, When I start the embedded engine with,
.with("offset.commit.policy",OffsetCommitPolicy.PeriodicCommitOffsetPolicy.class.getName()), the embedded engine works fine.
Note that the class OffsetCommitPolicy.PeriodicCommitOffsetPolicy constructor includes the config parameter while OffsetCommitPolicy.AlwaysCommitOffsetPolicy doesn't.
public PeriodicCommitOffsetPolicy(Configuration config) {
...
}
How to get the debezium embedded engine to use its AlwaysCommitOffsetPolicy?

Thanks for the report. This is partly bug (which we would appreciate if you could log into our Jira). You can solve this issue by calling a dedicated method embedded engine builder like `io.debezium.embedded.EmbeddedEngine.create().with(OffsetCommitPolicy.always())'

Tested with version 1.4.0Final:
new EmbeddedEngine.BuilderImpl() // create builder
.using(config) // regular config
.using(OffsetCommitPolicy.always()) // explicit commit policy
.notifying(this::handleEvent) // even procesor
.build(); // and finally build!

Related

neo4j using transactionListener causes read/write error

I'm trying to use TransactionEventListener in neo4j. There don't seem to be any lifecycle hooks for plugins, so I figure the only way to do it is to have a plugin procedure do it. However, trying to do that gives me this error:
Neo4jError: Writing in read access mode not allowed. Attempted write to internal graph 1 (system)
The plugin uses write mode, even if I'm not actually writing anything to the database; I'm just registering that TransactionEventListener, although that could indeed lead to writes later on. Still, I've got Write mode.
Here's my procedure:
#Procedure(name = "setTransactionListener", mode = Mode.WRITE)
public Stream<BuiltInProcedures.NodeResult> setTaxonomy(
#Name("taxonomy") Map<String, Map<String, Object>[]> taxonomy
) {
var managementService = new DatabaseManagementServiceBuilder(Path.of(".")).build();
var listener = new ValidationTransactionListener(taxonomy);
managementService.registerTransactionEventListener(db.databaseName(), listener);
return null;
}
Best guess is that I'm not supposed to register a transaction listener this way. But if not this way, then how? There don't seem to be any lifecycle hooks that get called when the database starts, so how can I possibly register an transactionEventListener?
Or is there a way I can give myself permission to do this?
What do you actually want to do?
It doesn't work like that, you need to register the listener in the database lifecycle within a KernelExtensionFactory?
See here for an example:
https://github.com/neo4j/apoc/blob/dev/common/src/main/java/apoc/ApocExtensionFactory.java#L53

Spring cloud config server - how to add custom PropertySource visible in findOne() method of EnvironmentEncryptorEnvironmentRepository

My goal is to add custom PropertySource to spring-cloud-server. What I want to achieve is to get some custom properties from that custom source in spring-cloud-config-client application.
Basing on suggestions from Adding environment repository in spring-config-server I've created spring-cloud-config-server application and separate project spring-cloud-config-custom. Second one is based on spring-cloud-consul-config code. So, I've created all necessary classes like CustomPropertySource, CustomPropertySourceLocator, CustomConfigBootstrapConfiguration and so on and configured them in spring.factories.
At the end, I've added maven dependency to spring-cloud-config-custom inside my spring-cloud-config-server.
So far so good. Everything works well. When I start server I can see that my CustomPropertySource is on the list of propertySources inside EnviromentRepository bean injected to EnvironmentController.
Problem: When I send GET request to #RequestMapping("/{name}/{profiles}/{label:.*}") (in EnvironmentController), injected EnviromentRepository bean is being used to find requested property source (repository.findOne(name, profiles, label) method).
Unfortunately my property source could not be found here. Why?
I've spent a lot of time on debugging this. I've found that repository delegates findOne() method call to other repositories: MultipleJGitEnvironmentRepository which delegates it to NativeEnvironmentRepository. Inside this delegates, findOne() method doesn't use propertySources from EnviromentRepository primary injected to controller. It creates new environment repository with new list of PropertySources and new separate SpringApplication. At the end, this list does not contain my CustomPropertySource and that is why findOne() returns empty propertySources in resulting Environment object.
Am I doing something wrong?
Is CustomPropertySourceLocator (and/or ConsulPropertySourceLocator) supposed to be used (autowired/bootstrapped) in spring-cloud-config-server or spring-cloud-config-client
Can spring-cloud-config-server deliver many different kind of PropertySources at the same time, via REST interface (saying "different" I mean all Git, Consul and Zookeeper)?
What you are doing is adding a property source to the config server itself, not the configuration it serves. Adding spring-boot-starter-actuator to your config server and viewing /env reveals:
{
"profiles": [
],
"server.ports": {
"local.server.port": 8888
},
"bootstrapProperties:custom": {
"test.prop3": "CUSTOM-VALUE-3",
"test.prop2": "CUSTOM-VALUE-2",
"test.prop1": "CUSTOM-VALUE-1"
},
}
To add something that will be served by config server, you have to implement an EnvironmentRepository.
Support for a composite EnvironmentRepository was recently added.

Eloquent error: A facade root has not been set

I have been using Eloquent as a standalone package in Slim Framework 2 successfully.
But now that I want to make use of Illuminate\Support\Facades\DB since I need to show some statistics by getting the info from 2 tables and using a Left Join and a Counter from the database like this:
use Illuminate\Support\Facades\DB;
$projectsbyarea = DB::table('projects AS p')
->select(DB::raw('DISTINCT a.area, COUNT(a.area) AS Quantity'))
->leftJoin('areas AS a','p.area_id','=','a.id')
->where('p.status','in_process')
->where('a.area','<>','NULL')
->orderBy('p.area_id');
I get the following error:
Type: RuntimeException
Message: A facade root has not been set.
File: ...\vendor\illuminate\support\Facades\Facade.php
Line: 206
How can I solve it?
So far I have found out, in this link that I need to create a new app container and then bind it to the Facade. But I haven't found out how to make it work.
This is how I started the rest of my Eloquent and working fine:
use Illuminate\Database\Capsule\Manager as Capsule;
$capsule = new Capsule();
$capsule->addConnection([
'my' => $app->config->get('settings'),
/* more settings ...*/
]);
/*booting Eloquent*/
$capsule->bootEloquent();
How do I fix this?
Fixed
As #user5972059 said, I had to add $capsule->setAsGlobal();//This is important to make work the DB (Capsule) just above $capsule->bootEloquent();
Then, the query is executed like this:
use Illuminate\Database\Capsule\Manager as Capsule;
$projectsbyarea = Capsule::table('projects AS p')
->select(DB::raw('DISTINCT a.area, COUNT(a.area) AS Quantity'))
->leftJoin('areas AS a','p.area_id','=','a.id')
->where('p.status','in_process')
->where('a.area','<>','NULL')
->orderBy('p.area_id')
->get();
You have to change your code to:
$Capsule = new Capsule;
$Capsule->addConnection(config::get('database'));
$Capsule->setAsGlobal(); //this is important
$Capsule->bootEloquent();
And at the beginning of your class file you have to import:
use Illuminate\Database\Capsule\Manager as DB;
I have just solved this problem by uncommenting $app->withFacades(); in bootstrap/app.php
Had the same issue with laravel 8. I replaced
use PHPUnit\Framework\TestCase;
with:
use Tests\TestCase;
Try uncommenting in app.php $app->withFacades();
Do not forget to call parent::setUp(); before.
fails
public function setUp(): void {
Config::set('something', true);
}
works
public function setUp(): void {
parent::setUp();
Config::set('something', true);
}
One random problem using phpUnit tests for laravel is that the laravel facades have not been initialized when testing.
Instead of using the standard PHPUnit TestCase class
class MyTestClass extends PHPUnit\Framework\TestCase
one can use
class UserTest extends Illuminate\Foundation\Testing\TestCase
and this problem is solved.
I got this error after running:
$ php artisan config:cache
The solution for me was to delete the /bootstrap/cache/config.php file. I'm running Laravel 5.5.
The seems to arise in multiple situation, and not just about facades.
I received the following message while running tests using PHPUnit v.9.5.4, PHP v.8.0.3 and Lumen v. 8.2.2:
PHP Fatal error: Uncaught RuntimeException: A facade root has not
been set. in path_to_project/vendor/illuminate/support/Facades/Facade.php:258
And that happened although I had apparently already configured my app.php to enable facades ($app->withFacades();), still I received this error message whenever I tried to run tests using Illuminate\Support\Facades\DB. Unfortunately, none of the other answers helped me.
This error was actually been thrown due to my configs in phpunit.xml, which didn't point to my app.php file, where I actually enabled facades.
I just had to change
<phpunit (...OTHER_PARAMS_HERE) bootstrap="vendor/autoload.php">
to
<phpunit (...OTHER_PARAMS_HERE) bootstrap="bootstrap/app.php">
Hope it helps.
wrong way
public function register()
{
$this->app->bind('Activity', function($app)
{
new Activity;
});
}
right way 👍
public function register()
{
$this->app->bind('Activity', function($app)
{
return new Activity;
});
}
---------------------------------- don't forget return
Upgrade version for php, I encountered this error while calling the interface.
$ php artisan config:cache
Deleting the /bootstrap/cache/config.php file is a very effective way.
In my project, I managed to fix this issue by using Laravel Dependency Injection when instantiating the object. Previously I had it like this:
$class = new MyClass(
new Client(),
env('client_id', 'test'),
Config::get('myapp.client_secret')
);
The same error message happened when I used Laravel env() and Config().
I introduced the Client and env in the AppServiceProvider like this:
$this->app->bind(
MyClass::class,
function () {
return new MyClass(
new Client(),
env('client_id', 'test')),
Config::get('myapp.client_secret')
);
}
and then instantiated the class like this:
$class = app(MyClass::class);
See more from https://laravel.com/docs/5.8/container .
In my case, for a while a ran a PHP project in PHP version 8, and that time I used some PHP 8 features like param definition and method's multiple return type declarations supported by only PHP 8 and above. When I downgraded from PHP 8 to PHP 7.4 I faced this issue. After removing the return types and param hinting the problems are gone.
Tested on Laravel 8.78
tests/bootstrap.php
<?php
use Illuminate\Foundation\Bootstrap\RegisterFacades;
use Illuminate\Foundation\Bootstrap\LoadConfiguration;
require_once __DIR__ . '/../vendor/autoload.php';
$app = require_once __DIR__ . '/../bootstrap/app.php';
(new LoadConfiguration())->bootstrap($app);// <------- Required for next line
(new RegisterFacades())->bootstrap($app);// <------- Add this line
Here is yet another instance of this error, happened to me after upgrading Laravel 8 to 9.
I had feature tests with a #dataProvider to supply data to those tests. Some of the data supplied by the data provider methods came from an application service. It was being initialised like this:
/**
* #dataProvider myDataProvider
*/
public function testSomeStuff(...)
{
...
}
public function myDataProvider()
{
$myService = app(Service::class); // This is trouble
return [
['test1_data' => $myService::SOME_CONSTANT],
[...],
...
];
}
This worked under Laravel 8, but not in Laravel 9. All other solutions listed in this SO thread were checked and were correctly set up.
The problem is that the application is not being inititialised until after the data provider method is run. It was presumably initialised before this stage in the Laravel 8 install. So app(Service::class) was failing due to it using facades internally.
One workaround could be to force the application to initialise earlier, in the data provider function: $this->createApplication(). I would not recommend this due to potential side effects of the test parts running in the wrong order, though it does appear to work when I tried it.
Best solution is to avoid accessing any part of the application functionality in the data provider methods. In my case it was easy to replace $myService::SOME_CONSTANT with MyService::SOME_CONSTANT after making sure those constants were public.
Hopefully this will help somebody suddenly hitting this problem running feature tests after a Laravel 9 upgrade.
If you recently upgrade Laravel on Homestead & VirtualBox environment or do not find any reason that causing please be sure your Vagrant is up to date.
Referance
I had Taylor lock this thread. The past several replies have restated the solution, which is to Upgrade to Virtualbox 6.x, the thread is locked to prevent other issues that are not related from being dogpiled on here.
#melvin's answer above works correctly.
In case someone is wondering about it, the mistake people do is to choose Yes when VSCode asks them if they are making a Unit Test. Remember, Unit Tests should really be unit tests, independent of other application features (models, factories, routes; basically anything that would require the Laravel app to be fired up). In most scenarios, people really actually want to make Feature Tests and therefore should answer No to the above question. A feature test inherits from Tests\TestCase class (which takes care of firing up Laravel app before running the test) unlike unit tests that inherit from the class PHPUnit\Framework\TestCase which use just PHPUnit and are therefore much faster.
credit with thanks to #Aken Roberts's answer here.
From Laravel Documentation: Generally, most of your tests should be feature tests. These types of tests provide the most confidence that your system as a whole is functioning as intended.

Grails mail plugin runtime configuration

Using grails mail plugin 1.0.7.
https://jira.grails.org/browse/GPMAIL-36 states that it's possible to change plguin configuration since 1.0.1 at runtime. Sadly it does not explains how to achieve it.
I want to be able to change the username at runtime to be able to use different mail accounts.
Thanks.
Based on this code, you should be able to change the configuration at runtime and the mail plugin will automagically re-deploy and update mail sender based on your changes.
Example:
Holders.config.grails.mail.username = 'foo'
Holders.config.grails.mail.password = 'bar'
sendMail {
to "foo#bar.com"
from "bar#foo.com"
subject "Hi"
body "This is an email"
}
Update:
It would appear that changing the configuration in this manner does not, in fact, fire the onConfigChange event. Per this, you can fire the event manually. Something like this:
Holders.pluginManager.getGrailsPlugin('mail').notifyOfEvent(GrailsPlugin.EVENT_ON_CONFIG_CHANGE, Holders.config)
I've realized this can be done accessing the mailSender bean from the context and updating it like is explained here
Changing mail configuration in runtime
However if #rmlan solution finally works it may be a much cleaner solution.
Actually thr rmlan solution works with the following fix. Since the onConfigChange compares hashCode of existing config map and new one, so if you set new configs in original configuration (Holders.config.grails.mail), then both configs are same and it never pass the condition to apply new changes, so a new structure should be created and pass it to notifyOfEvent method to mark the change as different hashCodes.
def mailConfig = [ grails: [ mail: [:] ] ]
mailConfig.grails.mail.host = newHost
mailConfig.grails.mail.port = newPort
Holders.pluginManager.getGrailsPlugin('mail').
notifyOfEvent(GrailsPlugin.EVENT_ON_CONFIG_CHANGE, mailConfig)
Still using async-mail and this one make the below exception
No qualifying bean of type [grails.plugin.mail.MailService] is defined: expected single matching bean but found 2: nonAsynchronousMailService,mailService
that is thrown because of the following part of onConfigChange
event.ctx.getBean(MailService.class).setPoolSize(mailConfig.poolSize?:null)
Commenting it let it works as a workaround, but making sendMail of mail plugin is called, not async-mail, so exception may raise if async-mail features is used on constructing mail. Hence to use async-mail in this workaround should use sendAsynchronousMail method.

How to make EF log sql queries globally?

How do I "tell" EF to log queries globally? I was reading this blog post: EF logging which tells in general how to log sql queries. But I still have a few questions regarding this logger.
Where would I need to place this line context.Database.Log = s =>
logger.Log("EFApp", s);?
Can it be globally set? Or do I have to place it everywhere I do DB
operations?
In the "Failed execution" section, the blogger wrote that, and I
quote:
For commands that fail by throwing an exception, the output contains the message from the exception.
Will this be logged too if I don't use the context.Database.Log?
Whenever you want the context to start logging.
It appears to be done on the context object so it should be done every time you create a new context. You could add this line of code in your constructor though to ensure that it is always enabled.
It will not log if you do not enable the logging.
I don't recommend to use that's functionality, because, it hasn't reason to exists in the real case.
Thats it use a lot of to debug code only. But, wether you wanna know more than details ... access link... https://cmatskas.com/logging-and-tracing-with-entity-framework-6/
In this case you can put code like this
public void Mylog()
{
//Thats a delegate where you can set this property to log using
//delegate type Action, see the code below
context.Database.Log = k=>Console.Write("Any query SQL")
//Or
context.Database.Log = k=>Test("Any query SQL")
}
public void Test(string x){
Console.Write(x)
}
I hope thats useufull