I'm trying to add a database-enabled JSP to an existing Tomcat 5.5 application (GeoServer 2.0.0, if that helps).
The app itself talks to Postgres just fine, so I know that the database is up, user can access it, all that good stuff. What I'm trying to do is a database query in a JSP that I've added. I've used the config example in the Tomcat datasource example pretty much out of the box. The requisite taglibs are in the right place -- no errors occur if I just have the taglib refs, so it's finding those JARs. The postgres jdbc driver, postgresql-8.4.701.jdbc3.jar is in $CATALINA_HOME/common/lib.
Here's the top of the JSP:
<%# taglib uri="http://java.sun.com/jsp/jstl/sql" prefix="sql" %>
<%# taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<sql:query var="rs" dataSource="jdbc/mmas">
select current_validstart as ValidTime from runoff_forecast_valid_time
</sql:query>
The relevant section from $CATALINA_HOME/conf/server.xml, inside the <Host> which is in turn within <Engine>:
<Context path="/gs2" allowLinking="true">
<Resource name="jdbc/mmas" type="javax.sql.Datasource"
auth="Container" driverClassName="org.postgresql.Driver"
maxActive="100" maxIdle="30" maxWait="10000"
username="mmas" password="very_secure_yess_precious!"
url="jdbc:postgresql//localhost:5432/mmas" />
</Context>
These lines are the last in the tag in webapps/gs2/WEB-INF/web.xml:
<resource-ref>
<description>
The database resource for the MMAS PostGIS database
</description>
<res-ref-name>
jdbc/mmas
</res-ref-name>
<res-type>
javax.sql.DataSource
</res-type>
<res-auth>
Container
</res-auth>
</resource-ref>
Finally, the exception:
exception
org.apache.jasper.JasperException: Unable to get connection, DataSource invalid: "java.sql.SQLException: No suitable driver"
[...wads of ensuing goo elided]
The infamous java.sql.SQLException: No suitable driver found
This exception can have basically two causes:
1. JDBC driver is not loaded
In case of Tomcat, you need to ensure that the JDBC driver is placed in server's own /lib folder.
Or, when you're actually not using a server-managed connection pool data source, but are manually fiddling around with DriverManager#getConnection() in WAR, then you need to place the JDBC driver in WAR's /WEB-INF/lib and perform ..
Class.forName("com.example.jdbc.Driver");
.. in your code before the first DriverManager#getConnection() call whereby you make sure that you do not swallow/ignore any ClassNotFoundException which can be thrown by it and continue the code flow as if nothing exceptional happened. See also Where do I have to place the JDBC driver for Tomcat's connection pool?
Other servers have a similar way of placing the JAR file:
GlassFish: put the JAR file in /glassfish/lib
WildFly: put the JAR file in /standalone/deployments
2. Or, JDBC URL is in wrong syntax
You need to ensure that the JDBC URL is conform the JDBC driver documentation and keep in mind that it's usually case sensitive. When the JDBC URL does not return true for Driver#acceptsURL() for any of the loaded drivers, then you will also get exactly this exception.
In case of PostgreSQL it is documented here.
With JDBC, a database is represented by a URL (Uniform Resource Locator). With PostgreSQL™, this takes one of the following forms:
jdbc:postgresql:database
jdbc:postgresql://host/database
jdbc:postgresql://host:port/database
In case of MySQL it is documented here.
The general format for a JDBC URL for connecting to a MySQL server is as follows, with items in square brackets ([ ]) being optional:
jdbc:mysql://[host1][:port1][,[host2][:port2]]...[/[database]] » [?propertyName1=propertyValue1[&propertyName2=propertyValue2]...]
In case of Oracle it is documented here.
There are 2 URL syntax, old syntax which will only work with SID and the new one with Oracle service name.
Old syntax jdbc:oracle:thin:#[HOST][:PORT]:SID
New syntax jdbc:oracle:thin:#//[HOST][:PORT]/SERVICE
See also:
Where do I have to place the JDBC driver for Tomcat's connection pool?
How to install JDBC driver in Eclipse web project without facing java.lang.ClassNotFoundexception
How should I connect to JDBC database / datasource in a servlet based application?
What is the difference between "Class.forName()" and "Class.forName().newInstance()"?
Connect Java to a MySQL database
I've forgot to add the PostgreSQL JDBC Driver into my project (Mvnrepository).
Gradle:
// http://mvnrepository.com/artifact/postgresql/postgresql
compile group: 'postgresql', name: 'postgresql', version: '9.0-801.jdbc4'
Maven:
<dependency>
<groupId>postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.0-801.jdbc4</version>
</dependency>
You can also download the JAR and import to your project manually.
url="jdbc:postgresql//localhost:5432/mmas"
That URL looks wrong, do you need the following?
url="jdbc:postgresql://localhost:5432/mmas"
I faced the similar issue.
My Project in context is Dynamic Web Project(Java 8 + Tomcat 8) and error is for PostgreSQL Driver exception: No suitable driver found
It got resolved by adding Class.forName("org.postgresql.Driver") before calling getConnection() method
Here is my Sample Code:
try {
Connection conn = null;
Class.forName("org.postgresql.Driver");
conn = DriverManager.getConnection("jdbc:postgresql://" + host + ":" + port + "/?preferQueryMode="
+ sql_auth,sql_user , sql_password);
} catch (Exception e) {
System.out.println("Failed to create JDBC db connection " + e.toString() + e.getMessage());
}
I found the followig tip helpful, to eliminate this issue in Tomcat -
be sure to load the driver first doing a Class.forName("
org.postgresql.Driver"); in your code.
This is from the post - https://www.postgresql.org/message-id/e13c14ec050510103846db6b0e#mail.gmail.com
The jdbc code worked fine as a standalone program but, in TOMCAT it gave the error -'No suitable driver found'
No matter how old this thread becomes, people would continue to face this issue.
My Case: I have the latest (at the time of posting) OpenJDK and maven setup. I had tried all methods given above, with/out maven and even solutions on sister posts on StackOverflow. I am not using any IDE or anything else, running from bare CLI to demonstrate only the core logic.
Here's what finally worked.
Download the driver from the official site. (for me it was MySQL https://www.mysql.com/products/connector/). Use your flavour here.
Unzip the given jar file in the same directory as your java project. You would get a directory structure like this. If you look carefully, this exactly relates to what we try to do using Class.forName(....). The file that we want is the com/mysql/jdbc/Driver.class
Compile the java program containing the code.
javac App.java
Now load the director as a module by running
java --module-path com/mysql/jdbc -cp ./ App
This would load the (extracted) package manually, and your java program would find the required Driver class.
Note that this was done for the mysql driver, other drivers might require minor changes.
If your vendor provides a .deb image, you can get the jar from /usr/share/java/your-vendor-file-here.jar
Summary:
Soln2 (recommend)::
1 . put mysql-connector-java-8.0.28.jar file in the <where you install your Tomcat>/lib.
Soln1::
1 . put mysql-connector-java-8.0.28.jar file in the WEB-INF/lib.
2 . use Class.forName("com.mysql.cj.jdbc.Driver"); in your Servlet Java code.
Soln1 (Ori Ans) //-20220304
In short:
make sure you have the mysql-connector-java-8.0.28.jar file in the WEB-INF/lib
make sure you use the Class.forName("com.mysql.cj.jdbc.Driver");
additional notes (not important), base on my trying (could be wrong)::
1.1 putting the jar directly inside the Java build path doesnt work
1.2. putting the jar in Data management > Driver Def > MySQL JDBC Driver > then add it as library to Java Build path doesnt work.
1.3 => it has to be inside the WEB-INF/lib (I dont know why)
1.4 using version mysql-connector-java-8.0.28.jar works, only version 5.1 available in Eclipse MySQL JDBC Driver setting doesnt matter, ignore it.
<see How to connect to MySql 8.0 database using Eclipse Database Management Perspective >
Class.forName("com.mysql.cj.jdbc.Driver");
Class.forName("com.mysql.jdbc.Driver");
both works,
but the Class.forName("com.mysql.jdbc.Driver"); is deprecated.
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
<see https://www.yawintutor.com/no-suitable-driver-found-for-jdbcmysql-localhost3306-testdb/ >
If you want to connect to a MySQL database, you can use the type-4 driver named Connector/} that's available for free from the MySQL website. However, this driver is typically included in Tomcat's lib directory. As a result, you don't usually need to download this driver from the MySQL site.
-- Murach’s Java Servlets and JSP
I cant find the driver in Tomcat that the author is talking about, I need to use the mysql-connector-java-8.0.28.jar.
<(striked-out) see updated answer soln2 below>
If you're working with an older version of Java, though, you need to use the forName method of the Class class to explicitly load the driver before you call the getConnection method
Even with JDBC 4.0, you sometimes get a message that says, "No suitable driver found." In that case, you can use the forName method of the Class class to explicitly load the driver. However, if automatic driver loading works, it usually makes sense to remove this method call from your code.
How to load a MySQL database driver prior to JDBC 4.0
Class.forName{"com.mysql.jdbc.Driver");
-- Murach’s Java Servlets and JSP
I have to use Class.forName("com.mysql.cj.jdbc.Driver"); in my system, no automatic class loading. Not sure why.
<(striked-out) see updated answer soln2 below>
When I am using a normal Java Project instead of a Dynamic Web Project in Eclipse,
I only need to add the mysql-connector-java-8.0.28.jar to Java Build Path directly,
then I can connect to the JDBC with no problem.
However, if I am using Dynamic Web Project (which is in this case), those 2 strict rules applies (jar position & class loading).
<see TOMCAT ON ECLIPSE java.sql.SQLException: No suitable driver found for jdbc:mysql >
Soln2 (Updated Ans) //-20220305_12
In short:
1 . put mysql-connector-java-8.0.28.jar file in the <where you install your Tomcat>/lib.
eg: G:\pla\Java\apache-tomcat-10.0.16\lib\mysql-connector-java-8.0.28.jar
(and for an Eclipse Dynamic Web Project, the jar will then be automatically put inside in your project's Java build path > Server Runtime [Apache Tomcat v10.0].)
Additional notes::
for soln1::
put mysql-connector-java-8.0.28.jar file in the WEB-INF/lib.
use Class.forName("com.mysql.cj.jdbc.Driver"); in your Servlet Java code.
this will create an WARNING:
WARNING: The web application [LearnJDBC] appears to have started a thread named [mysql-cj-abandoned-connection-cleanup] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
<see The web application [] appears to have started a thread named [Abandoned connection cleanup thread] com.mysql.jdbc.AbandonedConnectionCleanupThread >
and that answer led me to soln2.
for soln2::
put mysql-connector-java-8.0.28.jar file in the <where you install your Tomcat>/lib.
this will create an INFO:
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
you can just ignore it.
<see How to fix "JARs that were scanned but no TLDs were found in them " in Tomcat 9.0.0M10 >
(you should now understand what Murach’s Java Servlets and JSP was talking about: the jar in Tomcat/lib & the no need for Class.forName("com.mysql.cj.jdbc.Driver");)
to kinda fix it //-20220307_23
Tomcat 8.5. Inside catalina.properties, located in the /conf directory set:
tomcat.util.scan.StandardJarScanFilter.jarsToSkip=\*.jar
How to fix JSP compiler warning: one JAR was scanned for TLDs yet contained no TLDs?
It might be worth noting that this can also occur when Windows blocks downloads that it considers to be unsafe. This can be addressed by right-clicking the jar file (such as ojdbc7.jar), and checking the 'Unblock' box at the bottom.
Windows JAR File Properties Dialog:
As well as adding the MySQL JDBC connector ensure the context.xml (if not unpacked in the Tomcat webapps folder) with your DB connection definitions are included within Tomcats conf directory.
A very silly mistake which could be possible resulting is adding of space at the start of the JDBC URL connection.
What I mean is:-
suppose u have bymistake given the jdbc url like
String jdbcUrl=" jdbc:mysql://localhost:3306/web_customer_tracker?useSSL=false&serverTimeZone=UTC";
(Notice there is a space in the staring of the url, this will make the error)
the correct way should be:
String jdbcUrl="jdbc:mysql://localhost:3306/web_customer_tracker?useSSL=false&serverTimeZone=UTC";
(Notice no space in the staring, you may give space at the end of the url but it is safe not to)
Run java with CLASSPATH environmental variable pointing to driver's JAR file, e.g.
CLASSPATH='.:drivers/mssql-jdbc-6.2.1.jre8.jar' java ConnectURL
Where drivers/mssql-jdbc-6.2.1.jre8.jar is the path to driver file (e.g. JDBC for for SQL Server).
The ConnectURL is the sample app from that driver (samples/connections/ConnectURL.java), compiled via javac ConnectURL.java.
I was using jruby, in my case I created under config/initializers
postgres_driver.rb
$CLASSPATH << '~/.rbenv/versions/jruby-1.7.17/lib/ruby/gems/shared/gems/jdbc-postgres-9.4.1200/lib/postgresql-9.4-1200.jdbc4.jar'
or wherever your driver is, and that's it !
I had this exact issue when developing a Spring Boot application in STS, but ultimately deploying the packaged war to WebSphere(v.9). Based on previous answers my situation was unique. ojdbc8.jar was in my WEB-INF/lib folder with Parent Last class loading set, but always it says it failed to find the suitable driver.
My ultimate issue was that I was using the incorrect DataSource class because I was just following along with online tutorials/examples. Found the hint thanks to David Dai comment on his own question here: Spring JDBC Could not load JDBC driver class [oracle.jdbc.driver.OracleDriver]
Also later found spring guru example with Oracle specific driver: https://springframework.guru/configuring-spring-boot-for-oracle/
Example that throws error using org.springframework.jdbc.datasource.DriverManagerDataSource based on generic examples.
#Config
#EnableTransactionManagement
public class appDataConfig {
\* Other Bean Defs *\
#Bean
public DataSource dataSource() {
// configure and return the necessary JDBC DataSource
DriverManagerDataSource dataSource = new DriverManagerDataSource("jdbc:oracle:thin:#//HOST:PORT/SID", "user", "password");
dataSource.setSchema("MY_SCHEMA");
return dataSource;
}
}
And the corrected exapmle using a oracle.jdbc.pool.OracleDataSource:
#Config
#EnableTransactionManagement
public class appDataConfig {
/* Other Bean Defs */
#Bean
public DataSource dataSource() {
// configure and return the necessary JDBC DataSource
OracleDataSource datasource = null;
try {
datasource = new OracleDataSource();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
datasource.setURL("jdbc:oracle:thin:#//HOST:PORT/SID");
datasource.setUser("user");
datasource.setPassword("password");
return datasource;
}
}
I was having the same issue with mysql datasource using spring data that would work outside but gave me this error when deployed on tomcat.
The error went away when I added the driver jar mysql-connector-java-8.0.16.jar to the jres lib/ext folder
However I did not want to do this in production for fear of interfering with other applications. Explicity defining the driver class solved this issue for me
spring.datasource.driver-class-name: com.mysql.cj.jdbc.Driver
You will get this same error if there is not a Resource definition provided somewhere for your app -- most likely either in the central context.xml, or individual context file in conf/Catalina/localhost. And if using individual context files, beware that Tomcat freely deletes them anytime you remove/undeploy the corresponding .war file.
For me the same error occurred while connecting to postgres while creating a dataframe from table .It was caused due to,the missing dependency. jdbc dependency was not set .I was using maven for the build ,so added the required dependency to the pom file from maven dependency
jdbc dependency
For me adding below dependency to pom.xml file just solved like magic! I had no mysql connector dependency and even adding mssql jdbc jar file to build path did not work either.
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>9.4.0.jre11</version>
</dependency>
In my case I was working on a Java project with Maven and encountered this error.
In your pom.xml file make sure you have this dependencies
<dependencies>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.11</version>
</dependency>
</dependencies>
and where you create connection have something like this
public Connection createConnection() {
try {
String url = "jdbc:mysql://localhost:3306/yourDatabaseName";
String username = "root"; //your my sql username here
String password = "1234"; //your mysql password here
Class.forName("com.mysql.cj.jdbc.Driver");
return DriverManager.getConnection(url, username, password);
} catch (SQLException | ClassNotFoundException e) {
e.printStackTrace();
}
return null;
}
faced same issue. in my case ':' colon before '//' (jdbc:mysql://localhost:3306/dbname) was missing, and it just fixed the problem.
make sure : and // are placed properly.
I ran into the same error. In my case, the JDBC URL was correct, but the issue was with classpath. However, adding MySQL connector's JAR file to the -classpath or -cp (or, in the case of an IDE, as a library) doesn't resolve the issue. So I will have to move the JAR file to the location of Java bytecode and run java -cp :mysql_connector.jar to make this work. If someone runs into the same issue as mine, I'm leaving this here.
I encountered this issue by putting a XML file into the src/main/resources wrongly, I deleted it and then all back to normal.
Problem
I am trying to run a remote Spark Job through IntelliJ with a Spark HDInsight cluster (HDI 4.0). In my Spark application I am trying to read an input stream from a folder of parquet files from Azure blob storage using Spark's Structured Streaming built in readStream function.
The code works as expected when I run it on a Zeppelin notebook attached to the HDInsight cluster. However, when I deploy my Spark application to the cluster, I encounter the following error:
java.lang.IllegalAccessError: class org.apache.hadoop.hdfs.web.HftpFileSystem cannot access its superinterface org.apache.hadoop.hdfs.web.TokenAspect$TokenManagementDelegator
Subsequently, I am unable to read any data from blob storage.
The little information I found online suggested that this is caused by a version conflict between Spark and Hadoop. The application is run with Spark 2.4 prebuilt for Hadoop 2.7.
Fix
To fix this, I ssh into each head and worker node of the cluster and manually downgrade the Hadoop dependencies to 2.7.3 from 3.1.x to match the version in my local spark/jars folder. After doing this , I am then able to deploy my application successfully. Downgrading the cluster from HDI 4.0 is not an option as it is the only cluster that can support Spark 2.4.
Summary
To summarize, could the issue be that I am using a Spark download prebuilt for Hadoop 2.7? Is there a better way to fix this conflict instead of manually downgrading the Hadoop versions on the cluster's nodes or changing the Spark version I am using?
After troubleshooting some previous methods I had attempted before, I've come across the following fix:
In my pom.xml I excluded the hadoop-client dependency automatically imported by the spark-core jar. This dependency was version 2.6.5 which conflicted with the cluster's version of Hadoop. Instead, I import the version I require.
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.version.major}</artifactId>
<version>${spark.version}</version>
<exclusions>
<exclusion>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>${hadoop.version}</version>
</dependency>
</dependency>
After making this change, I encountered the error java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0. Further research revealed this was due to a problem with the Hadoop configuration on my local machine. Per this article's advice, I modified the winutils.exe version I had under C://winutils/bin to be the version I required and also added the corresponding hadoop.dll. After making these changes, I was able to successfully read data from blob storage as expected.
TLDR
Issue was the auto imported hadoop-client dependency which was fixed by excluding it & adding the new winutils.exe and hadoop.dll under C://winutils/bin.
This no longer required downgrading the Hadoop versions within the HDInsight cluster or changing my downloaded Spark version.
Problem:
I was facing same issue while running fat jar with hadoop 2.7 and spark 2.4 on cluster with hadoop 3.x ,
I was using maven shade plugin.
Observation:
While building fat jar it was including jar org.apache.hadoop:hadoop-hdfs:jar:2.6.5 which has class class org.apache.hadoop.hdfs.web.HftpFileSystem.
Which was causing problem in hadoop 3
Solution:
I have excluded this jar while building fat jar as below.Issue got resolved.
I am trying to read data from redshift to spark 1.5 using scala 2.10
I have built the spark-redshift package and added the amazon JDBC connector to the project, but I keep getting this error:
Exception in thread "main" java.lang.NoClassDefFoundError: com/amazonaws/auth/AWSCredentials
I have authenticated in the following way:
val hadoopConf = sc.hadoopConfiguration
hadoopConf.set("fs.s3n.impl","org.apache.hadoop.fs.s3native.NativeS3FileSystem")
hadoopConf.set("fs.s3n.awsAccessKeyId", "ACCESSKEY")
hadoopConf.set("fs.s3n.awsSecretAccessKey","SECRETACCESSKEY")
val df: DataFrame = sqlContext.read.format("com.databricks.spark.redshift")
.option("url","jdbc:redshift://AWS_SERVER:5439/warehouseuser=USER&password=PWD")
.option("dbtable", "fact_time")
.option("tempdir", "s3n://bucket/path")
.load()
df.show()
Concerning your first error java.lang.NoClassDefFoundError: com/amazonaws/auth/AWSCredentials I repeat what I said in the comment :
You have forgot to ship your AWS dependency jar in your spark app jar
And about the second error, I'm not sure of the package but it's more likely to be the org.apache.httpcomponents library you need. (I don't know for what you are using it thought!)
You can add the following to your maven dependencies :
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.4.3</version>
</dependency>
and you'll need to assembly the whole.
PS: You'll always need to provide the libraries when they are not installed. You must also becareful with the size of the jar you are submitting because it can harm performance.
I'm new to Scala, Spark and Maven and would like to build spark application described here. It uses the Mahout library.
I have Scala IDE install and would like to use Maven to build the dependencies (which are the Mahout library as well as Spark lib). I couldn't find a good tutorial to start. Could someone help me figure it out?
First try compiling simple application with Maven in Scala IDE. The key of Maven project is directory structure and pom.xml. Although I don't use Scala IDE, this document seems helpful.
http://scala-ide.org/docs/tutorials/m2eclipse/
Next step is to add dependency on Spark in pom.xml you can follow this document.
http://blog.cloudera.com/blog/2014/04/how-to-run-a-simple-apache-spark-app-in-cdh-5/
For latest version of Spark and Mahout artifacts you can check them here:
http://mvnrepository.com/artifact/org.apache.spark
http://mvnrepository.com/artifact/org.apache.mahout
Hope this helps.
You need following tools to get started ( based on recent availability) -
Scala IDE for Eclipse – Download latest version of Scala IDE from
here.
Scala Version – 2.11 ( make sure scala compiler is set to
this version as well)
Spark Version 2.2 ( provided in maven
dependency)
winutils.exe
For running in Windows environment , you need hadoop binaries in
windows format. winutils provides that and we need to set
hadoop.home.dir system property to bin path inside which winutils.exe
is present. You can download winutils.exe here and place at path
like this – c:/hadoop/bin/winutils.exe
And, you can define Spark Core Dependency in your Maven POM.XML for your project, to get started with.
<dependency> <!-- Spark dependency -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.2.0</version>
<scope>provided</scope>
</dependency>
And in your Java/Scala class define this property, to run on your local environmet on Windows -
System.setProperty("hadoop.home.dir", "c://hadoop//");
More details and full setup details can be found here.