I have been trying to load the SQL script schema into MySQL DB using Vertx.
Though, i am able to load or update any single DB command but unable to load complete schema in one go.
The second challenge faced is that, this might be blocking code for Vertx application. If that is the case, how can it be avoided?
Here is the code snippet i have been trying to execute:
jdbcClient.getConnection(resConn -> {
if(resConn.succeeded()) {
SQLConnection connection = resConn.result();
connection.execute("<Trying to load the SQL Script schema>", resSchema -> {
connection.close();
if(resSchema.succeeded()) {
async.complete();
} else {
testContext.fail("Failed to load bootstrap schema: " + resSchema.cause().getMessage());
}
});
} else {
testContext.fail("Failed to obtain DB connection for schema write");
}
});
The MySQL JDBC driver does not allow to execute a file as a SQL script. You must parse the script and execute individual commands one by one.
Related
I have a Glue/Connection for an RDS/PostgreSQL DB pre-built via CloudFormation, which works fine in a Glue/Scala/Sparkshell via getJDBCSink API to write down a DataFrame to that DB.
But also I need to write down to the same db, plain sql like create index ... or create table ... etc.
How can I forward that sort of statements in the same Glue/Spark shell?
In python, you can provide pg8000 dependency to the spark glue jobs and then run the sql commands by establishing the connection to the RDS using pg8000.
In scala you can directly establish a JDBC connection without the need of any external library as far as driver is concerned, postgres driver is available in aws glue.
You can create connection as
import java.sql.{Connection, DriverManager, ResultSet}
object pgconn extends App {
println("Postgres connector")
classOf[org.postgresql.Driver]
val con_st = "jdbc:postgresql://localhost:5432/DB_NAME?user=DB_USER"
val conn = DriverManager.getConnection(con_str)
try {
val stm = conn.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY)
val rs = stm.executeQuery("SELECT * from Users")
while(rs.next) {
println(rs.getString("quote"))
}
} finally {
conn.close()
}
}
or follow this blog
I'm trying to use Liquibase version 3.6 to maintain database version changes. I'm able to execute database version changes when i need to execute single file changes.
I have using below code to execute version changes successfully, but my intention is to dynamically execute the change logs from a folder and not single file. I'm using only Java for all configuration of Liquibase
#Startup
#Singleton
#TransactionManagement(TransactionManagementType.BEAN)
public class InitializerBean {
#Resource(mappedName = "java:/M2M_RESOURCES")
private DataSource ds;
#PostConstruct
protected void bootstrap() {
ResourceAccessor resourceAccessor = new ClassLoaderResourceAccessor(getClass().getClassLoader());
try (Connection connection = ds.getConnection()) {
JdbcConnection jdbcConnection = new JdbcConnection(connection);
Database db = DatabaseFactory.getInstance().findCorrectDatabaseImplementation(jdbcConnection);
Liquibase liquiBase = new Liquibase("data/liquibase/", resourceAccessor, db);
liquiBase.update("Development");
} catch (SQLException | LiquibaseException e) {
}
}
}
When the first parameter of Liquibase class is single file , liquibase is able to execute changes but when I intent to execute all file of single folder is not able to track and execute changes.
I'm using JDK 1.8 and file in data/liquibase/ is dbChangelog.sql and dbChangelog_2.sq. This all code is deployed on Wildfly 10 as part of ear archive
I have successfully configured spring boot with a new project to work
with flyway
Migrated with the Postgres database from the version 0001.0 to 0008.0
I have made manually alter the script in local but
flyway migration getting failed.
Sample Error message:
org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'flywayInitializer' defined in class path
resource
[org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]:
Invocation of init method failed; nested exception is
org.flywaydb.core.api.FlywayException: Validate failed: Migration
checksum mismatch for migration version 0006.0
How to alter the database tables without affecting flyway script from the flyway_schema_history?
For example, I need to change the table name using alter command but executing the flyway migration script without failed.
Any suggestions, Kindly appreciated.
Note:- I don't want to remove the script entries from the table flyway_schema_history.
There are a few ways to do this:-
1) Create a new script file with incremented version. Put your DDL commands to alter the table in this file. Then run migration.
2) If you don't want to delete the entry from the schema_version table, you can change the checksum value in that table. To calculate checksum, use the following method copied from org.flywaydb.core.internal.resolver.sql.SqlMigrationResolver. You can pass null for resource parameter:-
/**
* Calculates the checksum of this string.
*
* #param str The string to calculate the checksum for.
* #return The crc-32 checksum of the bytes.
*/
/* private -> for testing */
static int calculateChecksum(Resource resource, String str) {
final CRC32 crc32 = new CRC32();
BufferedReader bufferedReader = new BufferedReader(new StringReader(str));
try {
String line;
while ((line = bufferedReader.readLine()) != null) {
crc32.update(line.getBytes("UTF-8"));
}
} catch (IOException e) {
String message = "Unable to calculate checksum";
if (resource != null) {
message += " for " + resource.getLocation() + " (" + resource.getLocationOnDisk() + ")";
}
throw new FlywayException(message, e);
}
return (int) crc32.getValue();
}
3) If you are using Flyway Pro version 5+, you can rollback the migration https://flywaydb.org/getstarted/undo.
The answers here are outdated but can still help you.
It sounds like you might be in one of two situations:
You want to re-run a versioned migration. This isn't really how flyway works, as Kartik has suggested, create a new versioned migration to alter the table.
A migration file has been modified and you want to leave it that way and run new ones (eg 0009.0). In this situation you can try:
Run repair. Which will recalculate the checksums (among other things).
Turn off the validateOnMigrate option which will not fail a migration if there are modified migration files.
To solve this error locally without dropping your whole db:
Fix the migration error which caused the root problem
Disconnect your db server
Open the table "flyway_schema_history" which would be created automatically
Delete the rows with the versions that are causing the mismatch problem
Open the tables that have columns depending on the conflict migrations and drop those columns (if needed)
Run again your db server with the new migrations
I am new to mongodb and mongoose. I used the below code from internet. I never created the database MEANSTACK which i am referring below. When saving the record it never complained that DB or collection does not exist like it used to happen in MYSQL. How do i make sure that DB or collection exists before doing any operation and its not created automatically.
var mongoose = require( 'mongoose' );
var dbURI = 'mongodb://localhost/MEANSTACK';
mongoose.connect(dbURI);
mongoose.connection.on('connected', function () {
console.log(chalk.green('Mongoose connected to ' + dbURI));
});
mongoose.connection.on('error',function (err) {
console.log(chalk.red('Mongoose connection error: ' + err));
});
mongoose.connection.on('disconnected', function () {
console.log(chalk.red('Mongoose disconnected'));
});
Sry for writing my note as an answer. I can't write comments because my repo is below 50. When you run your server and it runs with no mistakes, that means your database var dbURI = 'mongodb://localhost/MEANSTACK' exists or at least your mongodb server is accessible. Otherwise you would get an error:
failed to connect to server [localhost:27017] on first connect
And additionally mongoose creates a db if it doesn't exist on server run, so it do the check you need on server run like "out of the box"
I am trying to connect to DB2 in my local LAN using worklight 6.1.0 and firing a Select Query for lookup of data if exist. But i am getting below error:
{
"errors": [
"Runtime: DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=DATABASE_NAME.REGISTRATION, DRIVER=3.58.82.\nPerformed query:\nSELECT * FROM DATABASE_NAME.registration where DATABASE_NAME.registration.Mob_No = ?"
],
"info": [
],
"isSuccessful": false,
"warnings": [
]
}
My SQL adapter configuration looks like below:
<connectionPolicy xsi:type="sql:SQLConnectionPolicy">
<!-- Example for using a JNDI data source, replace with actual data source name -->
<!-- <dataSourceJNDIName>java:/data-source-jndi-name</dataSourceJNDIName> -->
<!-- Example for using MySQL connector, do not forget to put the MySQL connector library in the project's lib folder -->
<dataSourceDefinition>
<driverClass>com.ibm.db2.jcc.DB2Driver</driverClass>
<url>jdbc:db2://172.21.11.129:50000/MOBILEDB</url>
<user>db2admin</user>
<password>Newuser123</password>
</dataSourceDefinition>
</connectionPolicy>
And js file which has procedure looks like:
var selectStatement1 = "SELECT * FROM DATABASE_NAME.registration where DATABASE_NAME.registration.Mob_No = ?";
var procStmt1 = WL.Server.createSQLStatement(selectStatement1);
function registrationLookup(mobile){
WL.Logger.debug("Inside registrationLookup");
return WL.Server.invokeSQLStatement(
{
preparedStatement : procStmt1,
parameters : [mobile]
}
);
}
I did some Research about connecting DB2 with Worklight and came to know that i need to put below data in worklight.properties file.
wl.db.username=db2admin
wl.db.type=DB2
wl.db.password=Newuser123
wl.db.driver=com.ibm.db2.jcc.DB2Driver
But after adding it, i am not able to deploy Adapter and error says 'db2admin' does not exist. So i have skipped this step in the context of current question. But after going through error which i am getting without adding this worklight.properties data it seems to me that 'Object doesn't exist' as per http://www-01.ibm.com/support/docview.wss?uid=swg21613531 or user table does not exist. Any suggestion would be helpful.
NOTE:
My IP address is 172.21.11.125 from where i am invoking Adapter for DB2.
DB2 instance is running on 172.21.11.129 # 50000.
Already Added db2jcc_license_cu_9.5.jar & db2jcc_9.5.jar in server/lib. It had name appended with '_9.5' which i have removed from both jar and kept only db2jcc_license_cu.jar and db2jcc.jar.
The error message is saying that your SQL statement is invalid, and I therefore infer that your connection to the DB is fine.
To diagnose this first run the SQL using DB2 command line or other tools. My guess is you mean
LARSEN.registration
whereas you are saying DATABASE_NAME.registration
Got Answer to my own Question and its really interesting one. Thanks to https://stackoverflow.com/users/2260967/glen-misquith [Glen Misquith]
Problem was with SQL Query framing which is Different what i did with MYSQL.
Adapter's Java Script file:
var selectStatement4 = "UPDATE \"LARSEN\".\"registration\" SET \"LARSEN\".\"registration\".\"Pass\"=?, \"LARSEN\".\"registration\".\"Re_Pass\"=? WHERE \"User_Name\" = ?";
var procStmt5 = WL.Server.createSQLStatement(selectStatement4);
function updatePassword(username,pass,repass){
WL.Logger.debug("Inside updatePassword "+username+" "+pass+" "+repass);
return WL.Server.invokeSQLStatement(
{
preparedStatement : procStmt5,
parameters : [pass,repass,username]
}
);
}
This is quite a Strange thing Slashes need to be used in SQL Statement while Preparing it.
I would really like to understand this behavior of DB2. And also i cant directly write 'Select * from schema.table_name' and precisely write column name from which data needs to be fetched.