I am trying to connect to druid database using avatica jar
Following is the code.
String url = "jdbc:avatica:remote:url=http://localhost:8082/druid/v2/sql/avatica";
Properties connectionProperties = new Properties();
try (Connection connection = DriverManager.getConnection(url, connectionProperties))
{
try (
final Statement statement = connection.createStatement();
final ResultSet resultSet = statement.executeQuery("SELECT COUNT(*) as rowcount FROM wikiticker"))
{
while (resultSet.next())
{
int count = resultSet.getInt("rowcount");
System.out.println("Total records:" + count);
}
resultSet.close();
}
}
catch (SQLException e)
{
e.printStackTrace();
}
I get following exception , Can someone please let me know whats going wrong ? I have set the runtime property to enable sql.
Exception in thread "main" java.lang.RuntimeException: Failed to execute HTTP Request, got HTTP/404
at org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:138)
at org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:34)
at org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:172)
at org.apache.calcite.avatica.remote.Driver.connect(Driver.java:175)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at com.test.druid.sql.Main.main(Main.java:17)
Looks like your broker instance is missing -Ddruid.sql.enabled=true flag on startup. You can refer to http://druid.io/docs/latest/querying/sql.html for further details.
Related
The following ItemReader gets a list of thousands accounts (acc).
The database that the ItemReader will connected to in order to retrieve the data is HIVE. I don’t have permission to create any table, only read option.
#Bean
#StepScope
public ItemReader<OmsDto> omsItemReader(#Value("#{stepExecutionContext[acc]}") List<String> accountList) {
String inParams = String.join(",", accountList.stream().map(id ->
"'"+id+"'").collect(Collectors.toList()));
String query = String.format("SELECT ..... account IN (%s)", inParams);
BeanPropertyRowMapper<OmsDto> rowMapper = new BeanPropertyRowMapper<>(OmsDto.class);
rowMapper.setPrimitivesDefaultedForNullValue(true);
JdbcCursorItemReader<OmsDto> reader = new JdbcCursorItemReader<OmsDto>();
reader.setVerifyCursorPosition(false);
reader.setDataSource(hiveDataSource());
reader.setRowMapper(rowMapper);
reader.setSql(query);
reader.open(new ExecutionContext());
return reader;
}
This is the error message that I get when using ItemReader:
Caused by: org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:153) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
Caused by: java.sql.SQLException: Error executing query
at com.facebook.presto.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:279) ~[presto-jdbc-0.243.2.jar:0.243.2-128118e]
at com.facebook.presto.jdbc.PrestoStatement.execute(PrestoStatement.java:228) ~[presto-jdbc-0.243.2.jar:0.243.2-128118e]
at com.facebook.presto.jdbc.PrestoPreparedStatement.<init>(PrestoPreparedStatement.java:84) ~[presto-jdbc-0.243.2.jar:0.243.2-128118e]
at com.facebook.presto.jdbc.PrestoConnection.prepareStatement(PrestoConnection.java:130) ~[presto-jdbc-0.243.2.jar:0.243.2-128118e]
at com.facebook.presto.jdbc.PrestoConnection.prepareStatement(PrestoConnection.java:300) ~[presto-jdbc-0.243.2.jar:0.243.2-128118e]
at org.springframework.batch.item.database.JdbcCursorItemReader.openCursor(JdbcCursorItemReader.java:121) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
... 63 common frames omitted
Caused by: java.lang.RuntimeException: Error fetching next at https://prestoanalytics-ch2-p.sys.comcast.net:6443/v1/statement/executing/20201118_131314_11079_v3w47/yf55745951e0beccc234c98f36005723457073854/0 returned an invalid response: JsonResponse{statusCode=502, statusMessage=Bad Gateway, headers={cache-control=[no-cache], content-length=[107], content-type=[text/html]}, hasValue=false} [Error: <html><body><h1>502 Bad Gateway</h1>
The server returned an invalid or incomplete response.
</body></html>
]
I was sure that the root cause is because of the driver but I have tested the driver with the same SQL this time using DriverManager and its run perfectly.
#Component
public class OmsItemReader implements ItemReader<OmsDto>, StepExecutionListener {
private ItemReader<OmsDto> delegate;
public SikOmsItemReader() {
Properties properties = new Properties();
properties.setProperty("user", "....");
properties.setProperty("password", "...");
properties.setProperty("SSL", "true");
Connection connection = null;
try {
connection = DriverManager.getConnection("jdbc:presto://.....", properties);
Statement statement = connection.createStatement();
ResultSet resultSet = statement.executeQuery(
I am not sure what is the different ? Is it the driver or sparing batch ?
I am looking for a workaround. How can I retrieve thousands of accounts via IN clauses with spring batch ?
Thank you
As soon as my code gets to my while(rs.next()) loop it produces the ResultSet is closed exception. What causes this exception and how can I correct for it?
EDIT: I notice in my code that I am nesting while(rs.next()) loop with another (rs2.next()), both result sets coming from the same DB, is this an issue?
Sounds like you executed another statement in the same connection before traversing the result set from the first statement. If you're nesting the processing of two result sets from the same database, you're doing something wrong. The combination of those sets should be done on the database side.
This could be caused by a number of reasons, including the driver you are using.
a) Some drivers do not allow nested statements. Depending if your driver supports JDBC 3.0 you should check the third parameter when creating the Statement object. For instance, I had the same problem with the JayBird driver to Firebird, but the code worked fine with the postgres driver. Then I added the third parameter to the createStatement method call and set it to ResultSet.HOLD_CURSORS_OVER_COMMIT, and the code started working fine for Firebird too.
static void testNestedRS() throws SQLException {
Connection con =null;
try {
// GET A CONNECTION
con = ConexionDesdeArchivo.obtenerConexion("examen-dest");
String sql1 = "select * from reportes_clasificacion";
Statement st1 = con.createStatement(
ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_READ_ONLY,
ResultSet.HOLD_CURSORS_OVER_COMMIT);
ResultSet rs1 = null;
try {
// EXECUTE THE FIRST QRY
rs1 = st1.executeQuery(sql1);
while (rs1.next()) {
// THIS LINE WILL BE PRINTED JUST ONCE ON
// SOME DRIVERS UNLESS YOU CREATE THE STATEMENT
// WITH 3 PARAMETERS USING
// ResultSet.HOLD_CURSORS_OVER_COMMIT
System.out.println("ST1 Row #: " + rs1.getRow());
String sql2 = "select * from reportes";
Statement st2 = con.createStatement(
ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_READ_ONLY);
// EXECUTE THE SECOND QRY. THIS CLOSES THE FIRST
// ResultSet ON SOME DRIVERS WITHOUT USING
// ResultSet.HOLD_CURSORS_OVER_COMMIT
st2.executeQuery(sql2);
st2.close();
}
} catch (SQLException e) {
e.printStackTrace();
} finally {
rs1.close();
st1.close();
}
} catch (SQLException e) {
} finally {
con.close();
}
}
b) There could be a bug in your code. Remember that you cannot reuse the Statement object, once you re-execute a query on the same statement object, all the opened resultsets associated with the statement are closed. Make sure you are not closing the statement.
Also, you can only have one result set open from each statement. So if you are iterating through two result sets at the same time, make sure they are executed on different statements. Opening a second result set on one statement will implicitly close the first.
http://java.sun.com/javase/6/docs/api/java/sql/Statement.html
The exception states that your result is closed. You should examine your code and look for all location where you issue a ResultSet.close() call. Also look for Statement.close() and Connection.close(). For sure, one of them gets called before rs.next() is called.
You may have closed either the Connection or Statement that made the ResultSet, which would lead to the ResultSet being closed as well.
Proper jdbc call should look something like:
try {
Connection conn;
Statement stmt;
ResultSet rs;
try {
conn = DriverManager.getConnection(myUrl,"","");
stmt = conn.createStatement();
rs = stmt.executeQuery(myQuery);
while ( rs.next() ) {
// process results
}
} catch (SqlException e) {
System.err.println("Got an exception! ");
System.err.println(e.getMessage());
} finally {
// you should release your resources here
if (rs != null) {
rs.close();
}
if (stmt != null) {
stmt.close();
}
if (conn != null) {
conn.close();
}
}
} catch (SqlException e) {
System.err.println("Got an exception! ");
System.err.println(e.getMessage());
}
you can close connection (or statement) only after you get result from result set. Safest way is to do it in finally block. However close() could also throe SqlException, hence the other try-catch block.
I got same error everything was correct only i was using same statement interface object to execute and update the database.
After separating i.e. using different objects of statement interface for updating and executing query i resolved this error. i.e. do get rid from this do not use same statement object for both updating and executing the query.
Check whether you have declared the method where this code is executing as static. If it is static there may be some other thread resetting the ResultSet.
make sure you have closed all your statments and resultsets before running rs.next. Finaly guarantees this
public boolean flowExists( Integer idStatusPrevious, Integer idStatus, Connection connection ) {
LogUtil.logRequestMethod();
PreparedStatement ps = null;
ResultSet rs = null;
try {
ps = connection.prepareStatement( Constants.SCRIPT_SELECT_FIND_FLOW_STATUS_BY_STATUS );
ps.setInt( 1, idStatusPrevious );
ps.setInt( 2, idStatus );
rs = ps.executeQuery();
Long count = 0L;
if ( rs != null ) {
while ( rs.next() ) {
count = rs.getLong( 1 );
break;
}
}
LogUtil.logSuccessMethod();
return count > 0L;
} catch ( Exception e ) {
String errorMsg = String
.format( Constants.ERROR_FINALIZED_METHOD, ( e.getMessage() != null ? e.getMessage() : "" ) );
LogUtil.logError( errorMsg, e );
throw new FatalException( errorMsg );
} finally {
rs.close();
ps.close();
}
A ResultSetClosedException could be thrown for two reasons.
1.) You have opened another connection to the database without closing all other connections.
2.) Your ResultSet may be returning no values. So when you try to access data from the ResultSet java will throw a ResultSetClosedException.
It happens also when using a ResultSet without being in a #Transactional method.
ScrollableResults results = getScrollableResults("select e from MyEntity e");
while (results.next()) {
...
}
results.close();
if MyEntity has eager relationships with other entities. the second time results.next() is invoked the ResultSet is closed exception is raised.
so if you use ScrollableResults on entities with eager relationships make sure your method is run transactionally.
"result set is closed" happened to me when using tag <collection> in MyBatis nested (one-to-many) xml <select> statement
A Spring solution could be to have a (Java) Spring #Service layer, where class/methods calling MyBatis select-collection statements are annotated with
#Transactional(propagation = Propagation.REQUIRED)
annotations being:
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
this solution does not require to set the following datasource properties (i.e., in JBoss EAP standalone*.xml):
<xa-datasource-property name="downgradeHoldCursorsUnderXa">**true**\</xa-datasource-property>
<xa-datasource-property name="resultSetHoldability">**1**</xa-datasource-property>
Hi I have the following method which calls a stored function in postgresql. The call works when I use a standard executequery() method but does not work when I start using batchs. Any help will be appreciated.
public void addstuff3() throws Exception {
Statement statement = null;
ResultSet resultSet = null;
Connection conn = null;
try {
// this will load the MySQL driver, each DB has its own driver
Class.forName("org.postgresql.Driver");
// setup the connection with the DB.
conn = DriverManager
.getConnection("jdbc:postgresql://localhost/newmydb?"
+ "user=new_user&password=password");
// statements allow to issue SQL queries to the database
statement = conn.createStatement();
conn.setAutoCommit(false);
statement.addBatch("SELECT ADDSTUFF('comp1', 'mdel1','power','PROPERTY','STRING','ON', '1396983600000', 'testing');");
statement.addBatch("SELECT ADDSTUFF('comp2', 'mdel2','power','PROPERTY','STRING','ON', '1396983600000', 'testing');");
conn.commit();
statement.executeBatch();
} catch (ClassNotFoundException | SQLException e) {
throw e;
} finally {
conn.close();
// resultSet.close();
statement.close();
}
This is the Error I get:
Batch entry 0 SELECT ADDSTUFF('comp1', 'mdel1','power','PROPERTY','STRING','ON', '1396983600000', 'testing') was aborted. Call getNextException to see the cause.
at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Sta tement.java:2743)
at org.postgresql.core.v3.QueryExecutorImpl$1.handleError(QueryExecutorImpl.java:461)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1928)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:405)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2892)
at jdbc.testing.MySQLAccess.addIndicators3(MySQLAccess.java:125)
at jdbc.testing.JDBCTesting.main(JDBCTesting.java:21)
Any help? I am using jdbc and postgresql.
ok thanks to #Dave I found that
e.getNextException()
Prints:
A result was returned when none was expected
I should not return a value
Works!
I want to perform the following select using JPA:
select * from permissions_table where permissions.role in ("Role1", "Role2")
What I have so far looks like this:
protected Set<String> getPermissions(Connection conn, String username, Collection<String> roleNames) throws SQLException {
PreparedStatement ps = null;
Set<String> permissions = new LinkedHashSet<String>();
try {
EntityManager em = entityManagerFactory.createEntityManager();
CriteriaBuilder builder = em.getCriteriaBuilder();
CriteriaQuery<HierarchicalPermission> criteria = builder.createQuery( HierarchicalPermission.class );
Root<HierarchicalPermission> permission = criteria.from(HierarchicalPermission.class);
criteria.select(permission).where(permission.get("Role").in(roleNames));
List<HierarchicalPermission> hPermissions = em.createQuery(criteria).getResultList();
for ( HierarchicalPermission p : hPermissions ) {
System.out.println( "Permission (" + p.getRole() +")");
}
}
catch(Exception ex)
{
System.out.println( ex.getMessage());
}
finally {
JdbcUtils.closeStatement(ps);
}
return permissions;
}
When I step over this line:
List<HierarchicalPermission> hPermissions = em.createQuery(criteria).getResultList();
I see the following in my Eclipse output window:
Hibernate: select hierarchic0_.iIdentity as iIdentity0_, hierarchic0_.timestamp as timestamp0_, hierarchic0_.szRole as szRole0_, hierarchic0_.szDescription as szDescri4_0_, hierarchic0_.iResource as iResource0_ from occ.ROLE_PERMISSIONS hierarchic0_ where hierarchic0_.szRole in (?)
and Eclipse debugger appears to stall. At this point, I can only pause or stop execution as shown in this screen shot.
What is this supposed to mean? Is this not a valid representation of the above query?
Database was locked by Sybase Interactive SQL on another machine so Hibernate was stalling while attempting to execute query. One would think that Hibernate would throw some sort of exception instead of simply stalling when it encounters resource contention but I guess this is not the case.
I'm using the IBM implentation of Open JPA on WebSphere 7 and I'm having an issue when I'm trying to reference an object that is #ManyToOne and keep getting the following error from DB2:
com.ibm.db2.jcc.b.SqlException: [jcc][t4][10120][10898][3.50.152] Invalid operation: result set is closed. ERRORCODE=-4470, SQLSTATE=null
I'm pulling my hair out as to why this doesn't work and hope that somebody can help.
Here is a simplified view the database schemas:
Table Report
record_id - integer - (primary key - generated by DB2)
agency - integer not null (foreign key to Dropdown table)
Table Dropdown
record_id - integer - (primary key - generated by DB2)
Here is the JPA entity for the Report which references the agency
#ManyToOne(fetch=FetchType.EAGER)
#JoinColumn(name="AGENCY")
private Dropdown agency;
Here is the code where I'm running a named query to get the data and then just iterating over the result set to print out the report id and the agnency. Whenever report.getAgency() is called, I get the "result set is closed" error from DB2:
#SuppressWarnings("unchecked")
public List<Report> getOpenIncidentsForUser(String aceId) throws Exception
{
List<Report> results = null;
EntityManager em = getEntityManager();
try
{
Query query = em.createNamedQuery("getOpenIncidentsForUser");
query.setParameter(1, aceId);
results = (List<Report>) query.getResultList();
Iterator<Report> it = results.iterator();
while(it.hasNext())
{
Report report = it.next();
System.out.println("Report [" + report.getRecordId() + "] Agency: [" + report.getAgency() + "]");
}
}
catch (Exception e)
{
log.fatal("Fatal error getting incidents for user", e);
throw e;
}
finally
{
em.close();
}
return (List<Report>) results;
}
if I don't ever refer to the getAgency method, I can print out anything else about the report with no problems. It only seems to be with the reference to the 2nd table. Any ideas?
I had answered this in responses to my original comment, but realized that I never marked the question as answered, so I wanted to do that officially.
The fix is documented here: https://www.ibm.com/support/knowledgecenter/SSEQTP_8.5.5/com.ibm.websphere.base.doc/ae/tejb_jpatroubleshoot.html
The fix ended up being the resulSetHoldability setting needed to be 1 instead of 2
For XA data sources you have to set downgradeHoldCursorsUnderXa to true, otherwise you could get a persistence exception with this message:
An SQL OPEN for a held cursor was issued on a XA connection
Setting DB2 resultSetHoldability=1 will only work if you are using a non-XA datasource. If you need to keep 2PC, then this is not a solution.
I had this exact problem and finally solved it by hard-coding a transaction around the offending code. This is what I have:
public class RequeueRuleList_back {
/*
* Injected resources ...
*/
#Resource UserTransaction txn;
#PersistenceUnit EntityManagerFactory emf;
:
public List<RequeueRuleBean> getRequeueRules() {
/*
* We need a hard transaction around this code even though it is just a query
* otherwise we cannot use a DB2 XA datasource to do this:
*
* com.ibm.db2.jcc.am.SqlException: [jcc][t4][10120][10898][3.63.75] Invalid operation: result set is closed. ERRORCODE=-4470, SQLSTATE=null
*/
try {
txn.begin();
} catch (Exception e) {
FacesContext.getCurrentInstance().addMessage(null,
new FacesMessage("Error starting transaction: " + e.getMessage()));
return null;
}
EntityManager em = emf.createEntityManager();
:
Query q = em.createQuery("SELECT rr FROM RequeueRule rr");
// Do useful things ...
em.close();
try {
txn.commit();
} catch (Exception e) {
FacesContext.getCurrentInstance().addMessage(null,
new FacesMessage("Error committing transaction: " + e.getMessage()));
}
:
}
}
If you just use jdbc to connect DB2 and didn't use Hibernate etc, you also have got this error. Because in the new JDBC version with DB2 9.7, many functions you shouldn't support in new vesion,althought there are no error running on old version jdbc.
These function include.
1: PreparedStatement
old version
pt.executeUpdate(sql);
new version
pt.executeUpdate();
2: Connection Iteration
old version：
try{
conn = ConnectionFactory.getConnection(ApplicationConstants.LOCAL_DATASOURCE_JNDI_NAME);
sql="select role_id,role_sname,role_sdesc from db2admin.mng_roles "+sql_condition+" order by role_id asc";
pt = conn.prepareStatement(sql.toString());
System.out.println("sql ="+sql);
rs = pt.executeQuery();
while(rs.next()){
i++;
role_id=rs.getInt(1);
role_sname=PubFunction.DoNull(rs.getString(2)).trim();
role_sdesc=PubFunction.DoNull(rs.getString(3)).trim();
role_right=PubFunction.DoNull(newright.getRightsbyRole(conn,role_id)).trim();}
new version
try{
conn = ConnectionFactory.getConnection(ApplicationConstants.LOCAL_DATASOURCE_JNDI_NAME);
sql="select role_id,role_sname,role_sdesc from db2admin.mng_roles "+sql_condition+" order by role_id asc";
pt = conn.prepareStatement(sql.toString());
System.out.println("sql ="+sql);
rs = pt.executeQuery();
while(rs.next()){
i++;
role_id=rs.getInt(1);
role_sname=PubFunction.DoNull(rs.getString(2)).trim();
role_sdesc=PubFunction.DoNull(rs.getString(3)).trim();
role_right=PubFunction.DoNull(newright.getRightsbyRole(null,role_id)).trim();}