I am trying to use the Keycloak docker image with a non-standard database (HANA-DB).
Modifying the configuration as described in the documentation worked fine and keycloak connects to the database. The remaining problem seems to be that the Liquibase migrations can not run because Liquibase does not know how to handle HANA-DB out of the box.
Of course, there is an extension library (https://mvnrepository.com/artifact/org.liquibase.ext/liquibase-hanadb/4.0.0) that adds this capability to Liquibase. Now my question: How do I get Keycloak's Liquibase to use this library?
I already tried:
Packaging the library into its own module and adding a dependency to it in Keycloak's Liquibase module
Adding the library as a second resource-root to Keycloak's Liquibase module
Both did not work, i.e. Liquibase is still not recognizing the "HDB" database.
What would be the correct way to do this?
I actually made it!
The way i achieved it is not perfekt, but keycloak 11.0.2 Docker Image is running.
Keycloak with Hana and Liquibase Migration to Hana DB is working and by now all tests for us have passed.
I'll just give you a short description what I did. And maybe write a little Blog Entry for that.
Basically you answer helped me at some point and you were on the right track.
What I did:
Creating another DB Vendor for JBoss CLI Bootstrap Scripting (https://medium.com/#victor.boaventura/keycloak-using-alternative-databases-e2b13576c457).
Hana DB Driver also needed a Global Module /subsystem=ee:write-attribute(name="global-modules",value=[{"name" => "jdk.net","slot" => "main"}, {"name" => "org.liquibase","slot" => "main"}]) (JBOSS CLI)
Keycloak did not autodiscover the correct Hibernate Dialect, so had to add that as well: /subsystem=keycloak-server/spi=connectionsJpa/provider=default/:write-attribute(name=properties,value={"dataSource" => "java:jboss/datasources/KeycloakDS","initializeEmpty" => "true","driverDialect" => "org.hibernate.dialect.HANAColumnStoreDialect","migrationStrategy" => "update","migrationExport" => expression "${jboss.home.dir}/keycloak-database-update.sql"})
Updated the Liquibase Module to Version 3.6.3 and added Liquibase Hana DB Adapter 3.9.0 (https://github.com/liquibase/liquibase-hanadb) as second resource root in same module. (Was kinda tricky to get the versions working) Also the Module needed another dependency on 'org.slf4j' to get the Logging to work.
At this Point I struggled with Liquibase Package Scanning and Keycloaks exclusion on 'liquibase.ext'. Thats why I had to add the SystemProp 'liquibase.scan.packages' (Thanks to you here :))
liquibase.scan.packages=org.keycloak.connections.jpa.updater.liquibase.lock,liquibase.change,liquibase.changelog,liquibase.database,liquibase.parser.core.xml,liquibase.precondition,liquibase.datatype,liquibase.serializer.core.xml,liquibase.sqlgenerator,liquibase.executor,liquibase.snapshot,liquibase.logging,liquibase.diff,liquibase.structure,liquibase.structurecompare,liquibase.lockservice,liquibase.sdk.database,liquibase.ext
After that Keycloak recognized Hana DB for Liquibase and the Migration started. I was super happy, but one Problem solved the next comes up...
2 of the liquibase changelogs were not working with Hana DB due to some SQL specifics. Thats why I created two patches and applied them to the keycloak-model-jpa module. Basically they already handled special cases (that failed) for oracle, so I just added 'hana' to every preCondition, where 'oracle' was mentioned and it worked!
This is basically my solution. If you have any questions please feel free to ask them here or contact me.
It turns out, that adding the library as a second resource-root in Keycloaks's liquibase module is actually working. It does not get loaded because Keycloak removes liquibase.ext from the service loader used to find liquibase extensions.
This can be worked around by using the liquibase.scan.packages system property, though.
For all those trying to make Keycloak working with HANA:
Overriding the liquibase.scan.packages system property causes the library to be loaded
Now, a class (LoggingService) from liquibase that is referenced in the liquibase-hanadb library, can not be found because it was introduced in liquibase 3.6 and Keycloak comes with 3.5.5.
When hacking a more recent version of liquibase (3.6.0) into Keycloak's modules, for some reason a changeset only intended for DB2 is being run, causing errors. I suspect an incompatibility between Keycloak's changelogs and the updated liquibase version.
I gave up at this point.
Related
I am going to make a db migration file and migrate the file to database by using the a command.
But I can't find the command.
In Laravel we approach the goal by using this command.
php artisan make:migrate migration_client_table
Is there any similiar command that acts like the php artisan make:migrate command?
Coming from Laravel/Symfony background as well, I was asking this myself. Here are my findings:
Laminas uses laminad-db module for interactions with database. Documentation for this module however does not mention migrations at all.
Database and Models introduction to Laminas uses sql file to create a table and fill it with some original seed and does not mention altering tables at all.
Therefore I made assumption Laminas does not handle migrations at all.
Edit: According to this answer, we can use sqitch, doctrine/migrations or liquibase
I'm currently in the middle of upgrading our API from v0.12 of Sails to v1. Not the easiest task, but will be worth it.
The current problem I'm having, is converting our old "ModelName.query" calls to the new style, which is supposedly "sails.getDatastore". Great, fine.
Except, that when trying to do this in config/bootstrap.js, I constantly get the error "sails.getDatastore is not a function".
Yes, I am using the default sails-hook-orm, the .sailsrc has it turned on explicitly; and yes, I have globals turned on.
Is the problem that the function isn't registered until after bootstrap? Because that is not an option for us; bootstrap is validating our database schema before lift (custom code, using native queries), so our production servers fail to deploy if we missed a database update. It eliminates a ton of human error.
Thanks for taking the 1.0 plunge!
I'm not sure what you mean by the "default" sails-hook-orm -- that hook is installed directly as a dependency on each Sails 1.0 project -- but I can almost guarantee that the version you're using is not correct. I would do:
npm cache clean
npm install sails-hook-orm#beta
in your project to make sure you get the latest (currently v2.0.0-21). It adds getDatastore to the app object when it initializes.
I have a PostgreSQL(v9.6) instance running on my machine. The database is called 'postgres'. I've managed to open the application in pgAdmin 3 using localhost:5432. In my Play! application(v2.6.2) I have added the driver and the url to the application.conf file following the tutorial here and I have added the javaJdbc dependency to my build.sbt file. So I have the following:
db.default.driver=org.postgresql.Driver
db.default.url="jdbc:postgresql://localhost:5432/postgres"
db.default.username = "user"
db.default.password = "pass"
When I run the application though I get this error in the console:
Cannot connect to database [default]
Could somebody explain to me why this is the case? I can provide more information if I need to.
So after quite a bit of digging, it turns out the problem was actually me missing the postgres dependency. If anyone encounters this problem I would suggest checking your build.sbt file and adding the org.postgres dependency if you are missing it.
This documentation on the play website doesn't explain this step.
I'm using liquibase since several years and it's extremly helpful for me as application developer to bring source code and database in sync, so thank you to all contributors for this tool.
During my daily work, I usually start liquibase from the command line in order to test the changesets and database operations. If everything is wired right, I start my application (Spring Boot) and the liquibase setup within the application performs all those sync steps. These setup works perfect unless my changelog file contains changesets with loaddata in order to populate data from CSV files into the database. Every application start fails with liquibase.exception.ValidationFailedException: Validation Failed:
change sets check sum
The reason seems to be the different file locations for the CVS files mentioned in loaddata which are part of the checksum computation. If startet from the application, the changesets looks like this:
classpath:liquibase/changelog.xml: classpath:liquibase/changelog.xml::loadDefaultRolePermissions::dominik
But if started from commandline, there is no way to use classpath resources, the changeset infos looks like that
liquibase: src/main/resources/liquibase/changelog.xml: src/main/resources/liquibase/changelog.xml::loadDefaultRolePermissions::dominik
Both values differs and leads to different checksums.
If you look into liquibase.integration.commandline.Main.java, there is no classpath resource accessor used:
FileSystemResourceAccessor fsOpener = new FileSystemResourceAccessor();
CommandLineResourceAccessor clOpener = new CommandLineResourceAccessor(classLoader);
CompositeResourceAccessor fileOpener = new CompositeResourceAccessor(fsOpener, clOpener);
from liquibase.integration.commandline.Main.java
Is there any way to let liquibase be interoperable between command line AND application startup run ?
Thanks in advance
Dominik
RESOLVED by updating from liquibase 3.3.1 to 3.5.3
Can someone help me with this problem??. I want use "Reverse Engineering Code First" with entity framework used MySQL. I was installed odbc, connector .net etc but still I cant see this... I did everything in this topic (and I have this same problem): Can't use a MySQL connection for entity framework 6
but still doesnt work :(. I was record all steps what I do it.. (Sry for english): https://youtu.be/xqEgCsu7_eU
I've gotten it to work with MySql Connector 6.9.11. Its so darned difficult and fragile I'm not sure if it was ultimately a good idea. There are at least 10 things that can go wrong and suck up hours of time. Once you get it working be very careful to not change anything. Anyhow. Here's some of my notes:
WARNING use with MySQL server 5.6.39 or 5.7.19. Do NOT use MySQL server version 5.7.21 (it fails with reverse eng) Aurora seems ok.
Install "mysql-connector-net-6.9.11.msi". There seems to be a problem with newer MySQL Connector 6.10 ?? use older 6.9.11. MUST INSTALL VIA msi file. Having the local file is not enough.
Install "mysql-for-visualstudio-1.2.8.msi" or "mysql-for-visualstudio-1.2.7.msi" ? NOTE: 1.2.7 will never uninstall correctly. VS 2015 seems to require "MySQL for Visual Studio" for EF Poco to work, (Pro,Enterprise are different)
Must have "Entity Framework Power tools Beta 4" installed as a plugin to Visual Studio 2013. Its takes some effort to get it to install for 2015, 17 but it can be done.
https://visualstudiogallery.msdn.microsoft.com/72a60b14-1581-4b9b-89f2-846072eff19d
Things that can go wrong:
ISSUE 1:
you get error "One or more errors occurred while processing template 'Entity.tt'."
This ALWAYS happens the first time i use reverse engineer after restarting VS.
Simply changing the .NET assemmbly version seems to do the trick. Target .NET 4.5 (not 4.5.1) for reverse engineer.
use .net4.5 for EF. If you see exception "System.ArgumentException: Empty path name is not legal."
remember to change back to 4.5.2 or greater.
VS2015 Edit and Continue is supported for 64-bit projects that target the .NET Framework 4.5.1. (or 4.5.2)
ISSUE 2:
System.ArgumentException: The specified store provider 'MySql.Data.MySqlClient' cannot be found in the configuration, or 'MySql.Data.MySqlClient' is not valid.
This appears to be an app/web.config issue.
??? MySql.Data.MySqlClient has more than 1 entry in in app.config ?? or bindingRedirect is wrong ?? might be wrong version consistently 6.9.11 ? MySql.Data.MySqlClient
?? Install correct version 6.9.11 in GAC not just nuget ? mysql-connector-net-6.9.11.msi
ISSUE 3:
MySQL issue - "error 6003: The value for column 'IsPrimaryKey' in table 'TableDetails' is DBNull." Specified cast is not valid.
[1 Jan 2016 15:27] "Noman Khan. Run the following command on the MySQL DB and then try if this works. set global optimizer_switch='derived_merge=off'" (on the user you will use. i.e root)
ISSUE 4:
Out of memory execption. Restart VisualStudio.
ISSUE 5:
If "MySQL" option doesnt display in the "Change Data source" listbox. Install the Visual Studio MySQL Plugin
Can't use a MySQL connection for entity framework 6
NOTE:
EntityFrameworkPowerToolsBeta4 is not built for VS2015 or 17. The installer version must be modified for it to install
How to use Entity Framework Power Tools in Visual Studio 2015?
http://thedatafarm.com/data-access/installing-ef-power-tools-into-vs2015/
NOTE:
You can view the Generated SQL query by debugging/break on the object prior to deferred execution of the query.
You can Use Aliases to map to other db's
Watch out for poor pluralization rules like "Statu" to "Status"
Check out https://us.visualstudiogallery.msdn.microsoft.com/ee4fcff9-0c4c-4179-afd9-7a2fb90f5838/view/Discussions
This seems to be more configurable and better supported. But only works for MSSQL?