I am trying to generate DDL scripts for my JPA entities and it works just fine.
I am adding a unit test for the same which is providing me with this error
Caused by: liquibase.exception.ServiceNotFoundException: Could not find unique implementation of liquibase.executor.Executor. Found 0 implementations
at liquibase.servicelocator.ServiceLocator.findClass(ServiceLocator.java:185)
at liquibase.servicelocator.ServiceLocator.newInstance(ServiceLocator.java:211)
... 21 more
My testng test looks like this
#DataProvider(parallel = true)
public Object[][] listToArrays() {
File f=new File(PropertyReader.getInstance().getProperty("path");
File[] files=f.listFiles();
List<MetaData> list =prepare(files);
Object[][] array = new Object[list.size()][1];
for (int i = 0; i < list.size(); i++) {
array[i][0] = list.get(i);
}
return array;
}
#Test(dataProvider = "listToArrays")
public void test(MetaData s) throws ScriptGenerationException,
UnsupportedEncodingException, LiquibaseException, IOException {
String[] rdbmsTypes=new String[]
{"MSSQL","POSTGRES","ORACLE","MYSQL"};
for (String rdbmsType : rdbmsTypes) {
FileSystemResourceAccessor fsOpener = new FileSystemResourceAccessor();
CommandLineResourceAccessor clOpener = new CommandLineResourceAccessor(this.getClass().getClassLoader());
CompositeResourceAccessor fileOpener = new CompositeResourceAccessor(new ResourceAccessor[] { fsOpener, clOpener });
Database database = CommandLineUtils.createDatabaseObject(fileOpener, this.url, this.username, this.password, this.driver,
this.defaultCatalogName, this.defaultSchemaName, Boolean.parseBoolean(this.outputDefaultCatalog),
Boolean.parseBoolean(this.outputDefaultSchema), this.databaseClass,
this.driverPropertiesFile, this.propertyProviderClass, this.liquibaseCatalogName,
this.liquibaseSchemaName, this.databaseChangeLogTableName, this.databaseChangeLogLockTableName);
Liquibase liquibase=new Liquibase(d, null, database);
Writer w=getOutputWriter();
liquibase.update(new Contexts(this.contexts), new LabelExpression(this.labels), w);
w.close();
}
}
What could be the cause of the problem ?
Note : I am only using Liquibase in offline mode.
Related
I need to implement an update of the database lying in the assets. User data, namely, in the "favorite" record or not, should be saved.
I already asked a question and they helped me -https://stackoverflow.com/a/53827525/10261947
Everything worked in a test application. But when I transferred the code (exactly the same) to the real application, an error occurs - E/AndroidRuntime: FATAL EXCEPTION: ModernAsyncTask #1
Process: rodionova.lyubov.brodsky, PID: 4196
java.lang.RuntimeException: An error occurred while executing doInBackground()
at android.support.v4.content.ModernAsyncTask$3.done(ModernAsyncTask.java:161)
at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:383)
at java.util.concurrent.FutureTask.setException(FutureTask.java:252)
at java.util.concurrent.FutureTask.run(FutureTask.java:271)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
at java.lang.Thread.run(Thread.java:784)
Caused by: java.lang.IllegalArgumentException: the bind value at index 4 is null
at android.database.sqlite.SQLiteProgram.bindString(SQLiteProgram.java:169)
at android.database.sqlite.SQLiteProgram.bindAllArgsAsStrings(SQLiteProgram.java:205)
at android.database.sqlite.SQLiteDirectCursorDriver.query(SQLiteDirectCursorDriver.java:47)
at android.database.sqlite.SQLiteDatabase.rawQueryWithFactory(SQLiteDatabase.java:1397)
at android.database.sqlite.SQLiteDatabase.queryWithFactory(SQLiteDatabase.java:1239)
at android.database.sqlite.SQLiteDatabase.query(SQLiteDatabase.java:1110)
at android.database.sqlite.SQLiteDatabase.query(SQLiteDatabase.java:1278)
at rodionova.lyubov.brodsky.db.PoemsDbHelper.insertCorePoem(PoemsDbHelper.java:121)
at rodionova.lyubov.brodsky.db.PoemsDbHelper.getNewPoems(PoemsDbHelper.java:90)
at rodionova.lyubov.brodsky.db.PoemsDbHelper.onUpgrade(PoemsDbHelper.java:41)
at com.readystatesoftware.sqliteasset.SQLiteAssetHelper.getWritableDatabase(SQLiteAssetHelper.java:197)
at com.readystatesoftware.sqliteasset.SQLiteAssetHelper.getReadableDatabase(SQLiteAssetHelper.java:254)
at rodionova.lyubov.brodsky.db.PoemsProvider.query(PoemsProvider.java:45)
at android.content.ContentProvider.query(ContentProvider.java:1057)
If you do not perform the update, the application is working properly, so I will post only the code DbHelper
public class PoemsDbHelper extends SQLiteAssetHelper {
public static final String DB_NAME = "brodsky.db";
public static final int DBVERSION = 3;
public static final String TBLNAME = "poems_table";
public static final String COL_ID = "id";
public static final String COL_TITLE = "title";
public static final String COl_POEM = "poem";
public static final String COL_SUBJECT = "subject";
public static final String COL_YEARS = "years";
public static final String COL_FAVOURITE = "favorite";
Context mContext;
public PoemsDbHelper(Context context) {
super(context, DB_NAME, null, DBVERSION);
mContext = context;
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
if(newVersion > oldVersion)
getNewPoems(mContext, db);
}
private void getNewPoems(Context context, SQLiteDatabase db) {
InputStream is;
OutputStream os;
final String tempNewDbName = "temp_brodsky.db";
int buffersize = 4096;
byte[] buffer = new byte[buffersize];
String newDBPath = mContext.getDatabasePath(tempNewDbName).getPath();
File newDBFile = new File(newDBPath);
if (newDBFile.exists()) {
newDBFile.delete();
}
File newDBFileDirectory = newDBFile.getParentFile();
if (!newDBFileDirectory.exists()) {
newDBFileDirectory.mkdirs();
}
try {
is = context.getAssets().open("databases/" + DB_NAME);
os = new FileOutputStream(newDBFile);
int bytes_read;
while ((bytes_read = is.read(buffer,0,buffersize)) > 0) {
os.write(buffer);
}
os.flush();
os.close();
is.close();
}catch (IOException e) {
e.printStackTrace();
throw new RuntimeException("Ouch updated database not copied - processing stopped - see stack-trace above.");
}
long id = maxid(db) + 1;
SQLiteDatabase newdb = SQLiteDatabase.openDatabase(newDBFile.getPath(),null,SQLiteDatabase.OPEN_READONLY);
Cursor csr = newdb.query(TBLNAME,null,null,null,null,null,null);
long insert_result;
db.beginTransaction();
while (csr.moveToNext()) {
insert_result = insertCorePoem(
db,
id,
csr.getString(csr.getColumnIndex(COL_TITLE)),
csr.getString(csr.getColumnIndex(COl_POEM)),
csr.getString(csr.getColumnIndex(COL_SUBJECT)),
csr.getString(csr.getColumnIndex(COL_YEARS)),
csr.getString(csr.getColumnIndex(COL_FAVOURITE))
);
if (insert_result > 0) {
id++;
}
}
db.setTransactionSuccessful();
db.endTransaction();
csr.close();
newDBFile.delete();
}
public long insertCorePoem(SQLiteDatabase db, long id, String title, String poem, String subject, String years, String favourite) {
String whereclause = COL_TITLE + "=? AND " + COl_POEM + "=? AND " + COL_SUBJECT + "=? AND " + COL_YEARS + "=?";
String[] whereargs = new String[]{
title,
poem,
subject,
years
};
Cursor csr = db.query(TBLNAME,null,whereclause,whereargs,null,null,null);
boolean rowexists = (csr.getCount() > 0);
csr.close();
if (rowexists) {
Log.d("INSERTCOREPOEM","Skipping insert of row");
return -2;
}
ContentValues cv = new ContentValues();
cv.put(COL_ID,id);
cv.put(COL_TITLE,title);
cv.put(COl_POEM,poem);
cv.put(COL_SUBJECT,subject);
cv.put(COL_YEARS,years);
cv.put(COL_FAVOURITE,favourite);
Log.d("INSERTCOREPOEM","Inserting new column with id " + String.valueOf(id));
return db.insert(TBLNAME, null, cv);
}
private long maxid(SQLiteDatabase db) {
long rv = 0;
String extractcolumn = "maxid";
String[] col = new String[]{"max(" + COL_ID + ") AS " + extractcolumn};
Cursor csr = db.query(TBLNAME,col,null,null,null,null,null);
if (csr.moveToFirst()) {
rv = csr.getLong(csr.getColumnIndex(extractcolumn));
}
csr.close();
return rv;
}
}
I do not understand what is wrong. Identical code works great friend application. I would be grateful for the help.
Your issue is that you likely have a value of null in the years column of a row or rows in the updated database that data is being copied from.
Although you could change the code to handle (skip insertion or use provide a year value) the end result may not be desired. So the most likely fix would be to amend the database to have valid/useful year values.
I am learning zookeeper and trying out the Curator framework for service discoveries. However, I am facing a weird issue that I have difficulties to figure out. The problem is when I tried to register an instance via serviceDiscovery, the cacheChanged event of the serviceCache gets triggered three times. When I removed an instance, it is only triggered once, which is the expected behavior. Please see the code below:
public class DiscoveryExample {
private static String PATH = "/base";
static ServiceDiscovery<InstanceDetails> serviceDiscovery = null;
public static void main(String[] args) throws Exception {
CuratorFramework client = null;
try {
// this is the ip address of my VM
client = CuratorFrameworkFactory.newClient("192.168.149.129:2181", new ExponentialBackoffRetry(1000, 3));
client.start();
JsonInstanceSerializer<InstanceDetails> serializer = new JsonInstanceSerializer<InstanceDetails>(
InstanceDetails.class);
serviceDiscovery = ServiceDiscoveryBuilder.builder(InstanceDetails.class)
.client(client)
.basePath(PATH)
.serializer(serializer)
.build();
serviceDiscovery.start();
ServiceCache<InstanceDetails> serviceCache = serviceDiscovery.serviceCacheBuilder()
.name("product")
.build();
serviceCache.addListener(new ServiceCacheListener() {
#Override
public void stateChanged(CuratorFramework curator, ConnectionState state) {
// TODO Auto-generated method stub
System.out.println("State Changed to " + state.name());
}
// THIS IS THE PART GETS TRIGGERED MULTIPLE TIMES
#Override
public void cacheChanged() {
System.out.println("Cached Changed ");
List<ServiceInstance<InstanceDetails>> list = serviceCache.getInstances();
Iterator<ServiceInstance<InstanceDetails>> it = list.iterator();
while(it.hasNext()) {
System.out.println(it.next().getAddress());
}
}
});
serviceCache.start();
BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
System.out.print("> ");
String line = in.readLine();
} finally {
CloseableUtils.closeQuietly(serviceDiscovery);
CloseableUtils.closeQuietly(client);
}
}
}
AND
public class RegisterApplicationServer {
final static String PATH = "/base";
static ServiceDiscovery<InstanceDetails> serviceDiscovery = null;
public static void main(String[] args) throws Exception {
CuratorFramework client = null;
try {
client = CuratorFrameworkFactory.newClient("192.168.149.129:2181", new ExponentialBackoffRetry(1000, 3));
client.start();
JsonInstanceSerializer<InstanceDetails> serializer = new JsonInstanceSerializer<InstanceDetails>(
InstanceDetails.class);
serviceDiscovery = ServiceDiscoveryBuilder.builder(InstanceDetails.class).client(client).basePath(PATH)
.serializer(serializer).build();
serviceDiscovery.start();
// SOME OTHER CODE THAT TAKES CARES OF USER INPUT...
} finally {
CloseableUtils.closeQuietly(serviceDiscovery);
CloseableUtils.closeQuietly(client);
}
}
private static void addInstance(String[] args, CuratorFramework client, String command,
ServiceDiscovery<InstanceDetails> serviceDiscovery) throws Exception {
// simulate a new instance coming up
// in a real application, this would be a separate process
if (args.length < 2) {
System.err.println("syntax error (expected add <name> <description>): " + command);
return;
}
StringBuilder description = new StringBuilder();
for (int i = 1; i < args.length; ++i) {
if (i > 1) {
description.append(' ');
}
description.append(args[i]);
}
String serviceName = args[0];
ApplicationServer server = new ApplicationServer(client, PATH, serviceName, description.toString());
server.start();
serviceDiscovery.registerService(server.getThisInstance());
System.out.println(serviceName + " added");
}
private static void deleteInstance(String[] args, String command, ServiceDiscovery<InstanceDetails> serviceDiscovery) throws Exception {
// in a real application, this would occur due to normal operation, a
// crash, maintenance, etc.
if (args.length != 2) {
System.err.println("syntax error (expected delete <name>): " + command);
return;
}
final String serviceName = args[0];
Collection<ServiceInstance<InstanceDetails>> set = serviceDiscovery.queryForInstances(serviceName);
Iterator<ServiceInstance<InstanceDetails>> it = set.iterator();
while (it.hasNext()) {
ServiceInstance<InstanceDetails> si = it.next();
if (si.getPayload().getDescription().indexOf(args[1]) != -1) {
serviceDiscovery.unregisterService(si);
}
}
System.out.println("Removed an instance of: " + serviceName);
}
}
I appriciate if anyone can please point out where I am doing wrong and maybe can share some good materials/examples so I can refer to. The official website and the examples on github does not help a lot.
I'm looking for a way to get all the column meta data for the given table name using camel-sql component.
Though it uses spring-jdbc behind the scenes i do not see a way to get the ResultSetMetaData.
I couldn't find a direct way to get the column details from camel-sql component, For now managed to get the information using spring jdbc template and data source.
public List<String> getColumnNamesFromTable(final TableData tableData) throws MetaDataAccessException {
final List<String> columnNames = new ArrayList<String>();
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
StringBuilder query = new StringBuilder();
query.append("SELECT * FROM ").append(SINGLE_BLANK_SPACE);
query.append(tableData.getSchemaName());
query.append(tableData.getTableName()).append(SINGLE_BLANK_SPACE);
query.append("WHERE rownum < 0");
jdbcTemplate.query(query.toString(), new ResultSetExtractor<Integer>() {
#Override
public Integer extractData(ResultSet rs) throws SQLException, DataAccessException {
ResultSetMetaData rsmd = rs.getMetaData();
int columnCount = rsmd.getColumnCount();
for (int i = 1; i <= columnCount; i++) {
columnNames.add(rsmd.getColumnName(i).toUpperCase());
}
return columnCount;
}
});
return columnNames;
}
Unlike default examples where we have firehose(s) for importing rows from csv, tsv etc, do we have one so we can import records from database and insert into druid? Any thoughts?
Here is what I was thinking -
"firehose": {
"type" : "database",
"datasource" : {
"connectURI" : "jdbc:mysql://localhost:3306/test",
"user" : "druid",
"password" : "xyz123"
},
"query" : "select * from table"
"frequency" : "P1M"
}
We can possibly extend it to get connection via jndi datasource and few others. Does this sort of implementation have any issues?
How about this idea? It's custom firehose for jdbc ingestion.
In this case, only supports one time query ingestion.
https://github.com/sirpkt/druid/tree/jdbc_firehose/extensions-contrib/jdbc-firehose
This is code snippet. Using DBI library try to get result set from existing database server.
public Firehose connect(final MapInputRowParser parser) throws IOException, ParseException, IllegalArgumentException
{
if (columns != null) {
verifyParserSpec(parser.getParseSpec(), columns);
}
final Handle handle = new DBI(
connectorConfig.getConnectURI(),
connectorConfig.getUser(),
connectorConfig.getPassword()
).open();
final String query = makeQuery(columns);
final ResultIterator<InputRow> rowIterator = handle
.createQuery(query)
.map(
new ResultSetMapper<InputRow>()
{
List<String> queryColumns = (columns == null) ? Lists.<String>newArrayList(): columns;
#Override
public InputRow map(
final int index,
final ResultSet r,
final StatementContext ctx
) throws SQLException
{
try {
if (queryColumns.size() == 0)
{
ResultSetMetaData metadata = r.getMetaData();
for (int idx = 1; idx <= metadata.getColumnCount(); idx++)
{
queryColumns.add(metadata.getColumnName(idx));
}
Preconditions.checkArgument(queryColumns.size() > 0,
String.format("No column in table [%s]", table));
verifyParserSpec(parser.getParseSpec(), queryColumns);
}
ImmutableMap.Builder<String, Object> builder = new ImmutableMap.Builder();
for (String column: queryColumns) {
builder.put(column, r.getObject(column));
}
return parser.parse(builder.build());
} catch(IllegalArgumentException e) {
throw new SQLException(e);
}
}
}
).iterator();
I've read a dozen different blogs, as well as reading through the msdn examples and they just aren't working for me.
Ultimately what I'm trying to do is automate moving a DB from our production instance to our dev instance, or the other direction.
The approach I've taken is thus:
backup/restore to a temp DB
detach temp DB
copy mdf and ldf files to the other instance
reattach.
I'm stuck on 1 and I cannot understand why. Everything I've read claims this should be working.
NOTE: I've set dbName to the db I want to restore to. I have also set restore.Database = dbName, where restore is an instance of the Restore class in the smo namespace.
mdf.LogicalFileName = dbName;
mdf.PhysicalFileName = String.Format(#"{0}\{1}.mdf", server.Information.MasterDBPath, dbName);
ldf.LogicalFileName = dbName + "_log";
ldf.PhysicalFileName = String.Format(#"{0}\{1}.ldf", server.Information.MasterDBPath, dbName);
restore.RelocateFiles.Add(mdf);
restore.RelocateFiles.Add(ldf);
restore.SqlRestore(server);
This is the exception I'm getting:
The file 'D:\MSSQL.MIQ_Dev\MSSQL.2\MSSQL\Data\MIQDesign2Detach.mdf' cannot be overwritten. It is being used by database 'MIQDesignTest2'.
File 'MIQDesign' cannot be restored to 'D:\MSSQL.MIQ_Dev\MSSQL.2\MSSQL\Data\MIQDesign2Detach.mdf'. Use WITH MOVE to identify a valid location for the file.
The file 'D:\MSSQL.MIQ_Dev\MSSQL.2\MSSQL\Data\MIQDesign2Detach.ldf' cannot be overwritten. It is being used by database 'MIQDesignTest2'.
File 'MIQDesign_log' cannot be restored to 'D:\MSSQL.MIQ_Dev\MSSQL.2\MSSQL\Data\MIQDesign2Detach.ldf'. Use WITH MOVE to identify a valid location for the file.
Problems were identified while planning for the RESTORE statement. Previous messages provide details.
RESTORE DATABASE is terminating abnormally.
Why is this trying to overwrite the original mdf? Isn't the RelocateFiles stuff supposed to specify that you want it being saved to a different physical filename?
It is works.
public class DatabaseManager
{
public Action<int, string> OnSqlBackupPercentComplete;
public Action<int, string> OnSqlRestorePercentComplete;
public Action<SqlError> OnSqlBackupComplete;
public Action<SqlError> OnSqlRestoreComplete;
public bool IsConnected { get; private set; }
private ServerConnection _connection;
public void Connect(string userName, string password, string serverName, bool useInteratedLogin)
{
if (useInteratedLogin)
{
var sqlCon = new SqlConnection(string.Format("Data Source={0}; Integrated Security=True; Connection Timeout=5", serverName));
_connection = new ServerConnection(sqlCon);
_connection.Connect();
IsConnected = true;
}
else
{
_connection = new ServerConnection(serverName, userName, password);
_connection.ConnectTimeout = 5000;
_connection.Connect();
IsConnected = true;
}
}
public void BackupDatabase(string databaseName, string destinationPath)
{
var sqlServer = new Server(_connection);
databaseName = databaseName.Replace("[", "").Replace("]", "");
var sqlBackup = new Backup
{
Action = BackupActionType.Database,
BackupSetDescription = "ArchiveDataBase:" + DateTime.Now.ToShortDateString(),
BackupSetName = "Archive",
Database = databaseName
};
var deviceItem = new BackupDeviceItem(destinationPath, DeviceType.File);
sqlBackup.Initialize = true;
sqlBackup.Checksum = true;
sqlBackup.ContinueAfterError = true;
sqlBackup.Devices.Add(deviceItem);
sqlBackup.Incremental = false;
sqlBackup.ExpirationDate = DateTime.Now.AddDays(3);
sqlBackup.LogTruncation = BackupTruncateLogType.Truncate;
sqlBackup.PercentCompleteNotification = 10;
sqlBackup.PercentComplete += (sender, e) => OnSqlBackupPercentComplete(e.Percent, e.Message);
sqlBackup.Complete += (sender, e) => OnSqlBackupComplete(e.Error);
sqlBackup.FormatMedia = false;
sqlBackup.SqlBackup(sqlServer);
}
public DatabaseCollection GetDatabasesList()
{
if (IsConnected)
{
var sqlServer = new Server(_connection);
return sqlServer.Databases;
}
return null;
}
public void RestoreDatabase(string databaseName, string filePath)
{
var sqlServer = new Server(_connection);
databaseName = databaseName.Replace("[", "").Replace("]", "");
var sqlRestore = new Restore();
sqlRestore.PercentCompleteNotification = 10;
sqlRestore.PercentComplete += (sender, e) => OnSqlRestorePercentComplete(e.Percent, e.Message);
sqlRestore.Complete += (sender, e) => OnSqlRestoreComplete(e.Error);
var deviceItem = new BackupDeviceItem(filePath, DeviceType.File);
sqlRestore.Devices.Add(deviceItem);
sqlRestore.Database = databaseName;
DataTable dtFileList = sqlRestore.ReadFileList(sqlServer);
int lastIndexOf = dtFileList.Rows[1][1].ToString().LastIndexOf(#"\");
string physicalName = dtFileList.Rows[1][1].ToString().Substring(0, lastIndexOf + 1);
string dbLogicalName = dtFileList.Rows[0][0].ToString();
string dbPhysicalName = physicalName + databaseName + ".mdf";
string logLogicalName = dtFileList.Rows[1][0].ToString();
string logPhysicalName = physicalName + databaseName + "_log.ldf";
sqlRestore.RelocateFiles.Add(new RelocateFile(dbLogicalName, dbPhysicalName));
sqlRestore.RelocateFiles.Add(new RelocateFile(logLogicalName, logPhysicalName));
sqlServer.KillAllProcesses(sqlRestore.Database);
Database db = sqlServer.Databases[databaseName];
if (db != null)
{
db.DatabaseOptions.UserAccess = DatabaseUserAccess.Single;
db.Alter(TerminationClause.RollbackTransactionsImmediately);
sqlServer.DetachDatabase(sqlRestore.Database, false);
}
sqlRestore.Action = RestoreActionType.Database;
sqlRestore.ReplaceDatabase = true;
sqlRestore.SqlRestore(sqlServer);
db = sqlServer.Databases[databaseName];
db.SetOnline();
sqlServer.Refresh();
db.DatabaseOptions.UserAccess = DatabaseUserAccess.Multiple;
}
public void Disconnect()
{
if (IsConnected)
_connection.Disconnect();
IsConnected = false;
}
}
I ran into a similar problem and I found this solution to be quite helpful.
Take a look - http://www.eggheadcafe.com/software/aspnet/32188436/smorestore-database-name-change.aspx