Create a mongo connection and make it alive for execution of an Entire Test Suite in Ready!API - mongodb

If you want to make an gmongo connection alive for an entire test suite and then close it in a tear down operation after the entire test suite is executed then, How could we do that?
Currently what am I doing is, I am creating an connection for an particular test step and then after the test step is executed, I close the connection by using the code mongoClient.close()
But now there is a requirement where I need to create the connection before the test suite starts executing, use the same connection throughout the test suite inside the test cases/test steps and then close the connection the connection after the entire test suite gets executed.
Could anyone please tell me how could I do this using Ready!API?
I may sound retard cause I am new to Ready API so please bear with me
This is the code that I use to create an Connection to mongo
def dbUser = context.expand( '${#Project#MongoUser}' )
def dbPassword = context.expand( '${#Project#MongoPassword}' )
def dbServer = context.expand( '${#Project#MongoServer}' )
def dbDatabase = context.expand( '${#Project#MongoDatabase}' )
def credentials = MongoCredential.createCredential(dbUser,dbDatabase,dbPassword as char[])
def mongoClient = new MongoClient( new ServerAddress(dbServer),Arrays.asList(credentials) )
context.gmongo = new GMongo( mongoClient )
context.mongoDB = context.gmongo.getDB(dbDatabase)
So i have been using the current code in order to create the connection. Actually I want this as three test suites. The First Test Suite would contain the groovy script to create the connection, The Second Test Suite would contain all of my Test Cases and the Third test suite would contain the mongo close connection script.
We use the Environment values from the properties file. Here the MongoServer has the values of the environment in which the connection is laid
I could not understand #Rao, how did you call the conn variable inside the test cases. Especially the context.testCase.testSuite.db?.connection part. What does the "?" denote and could you please tell me in the above context, how could carry out the process

Below script address how you achieve what you are looking for in ReadyAPI / SoapUI. Note that you already know how to connect to gmongo in Groovy which you need to add that logic in the place holder by following the comment inline.
Below is the test suite level Setup Script to create the db connection.
class DatabaseDetails {
def server
def user
def password
def log
def getConnection() {
log.info 'connection created'
//Write logic to create connection
}
def closeConnection() {
log.info 'Closing connection'
//Write logic to close connection
}
}
//Change server, user, password values according to your environment
def db = [ server:'localhost', user:'dbuser', password: 'dbuserpasswd', log: log] as DatabaseDetails
if (!db.connection) {
db.connection
testSuite.metaClass.db = db
}
Below is the test suite level TearDown Script to close the db connection. Since this is in tear down script, connection gets closed automatically as soon the test suite execution is completed.
testSuite.db?.closeConnection()
Now, there is no need to have step to create the db connection again and again.
You just need to use below script in Groovy Script test step to get the existing db connection.
def conn = context.testCase.testSuite.db?.connection
Using conn variable, you should be able to execute the queries.
Note : Since the db connection is done in Setup Script of test suite, if you just run the test case(i.e., test suite is not invoked or executed), you may not able to get the connection. In such cases, manually execute the Setup Script of the test suite.
EDIT: Based on OP's edit to the question and his code snippet, here is the updated test suite's Setup Script. This takes care of implementation of getConnection() and closeConnection() based on OP's edit. Please add / edit import statements for Mongo classes that are used as I am not really aware of those.
Updated Test Suite's Setup Script
import com.gmongo.*
import com.mongodb.*
class DatabaseDetails {
def context
def log
def mongoClient
def mongoDB
def getConnection() {
log.info 'Creating connection.'
//Write logic to create connection
if (!mongoDB){
def credentials = MongoCredential.createCredential(
context.expand('${#Project#MongoUser}'),
context.expand('${#Project#MongoDatabase}'),
context.expand('${#Project#MongoPassword}') as char[])
mongoClient = new MongoClient( new ServerAddress(context.expand('${#Project#MongoServer}')),Arrays.asList(credentials) )
mongoDB = new GMongo( mongoClient ).getDB(context.expand('${#Project#MongoDatabase}'))
}
mongoDB
}
def closeConnection() {
log.info 'Closing connection'
//Write logic to close connection
mongoClient.close()
}
}
def db = [ context: context, log: log] as DatabaseDetails
if (!db.connection) {
db.connection
testSuite.metaClass.db = db
}
As mentioned earlier, to get the connection, use below code and explaining it down.
context.testCase.testSuite.db?.connection
Groovy has great feature called ExpandoMetaclass. db is injected to testSuite class and db is object of DatabaseDetails class that we created and instantiated in Setup Script of test suite.
And db contains getConnection() i.e., db.getConnection() which can also same as db.connection. That is how connection is available in the above statement.

Related

Is it possible writing down to RDS raw sql (PostgreSQL) using AWS/Glue/Spark shell?

I have a Glue/Connection for an RDS/PostgreSQL DB pre-built via CloudFormation, which works fine in a Glue/Scala/Sparkshell via getJDBCSink API to write down a DataFrame to that DB.
But also I need to write down to the same db, plain sql like create index ... or create table ... etc.
How can I forward that sort of statements in the same Glue/Spark shell?
In python, you can provide pg8000 dependency to the spark glue jobs and then run the sql commands by establishing the connection to the RDS using pg8000.
In scala you can directly establish a JDBC connection without the need of any external library as far as driver is concerned, postgres driver is available in aws glue.
You can create connection as
import java.sql.{Connection, DriverManager, ResultSet}
object pgconn extends App {
println("Postgres connector")
classOf[org.postgresql.Driver]
val con_st = "jdbc:postgresql://localhost:5432/DB_NAME?user=DB_USER"
val conn = DriverManager.getConnection(con_str)
try {
val stm = conn.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY)
val rs = stm.executeQuery("SELECT * from Users")
while(rs.next) {
println(rs.getString("quote"))
}
} finally {
conn.close()
}
}
or follow this blog

Cannot run tests on h2 in-memory database, rather it runs on PostgreSQL

(I have multiple related questions, so I highlight them as bold)
I have a play app.
play: 2.6.19
scala: 2.12.6
h2: 1.4.197
postgresql: 42.2.5
play-slick/play-slick-evolutions: 3.0.1
slick-pg: 0.16.3
I am adding a test for DAO, and I believe it should run on an h2 in-memory database that is created when tests start, cleared when tests end.
However, my test always runs on PostgreSQL database I configure and use.
# application.conf
slick.dbs.default.profile="slick.jdbc.PostgresProfile$"
slick.dbs.default.db.driver="org.postgresql.Driver"
slick.dbs.default.db.url="jdbc:postgresql://localhost:5432/postgres"
Here is my test test/dao/TodoDAOImplSpec.scala.
package dao
import play.api.inject.guice.GuiceApplicationBuilder
import play.api.test.{Injecting, PlaySpecification, WithApplication}
class TodoDAOImplSpec extends PlaySpecification {
val conf = Map(
"slick.dbs.test.profile" -> "slick.jdbc.H2Profile$",
"slick.dbs.test.db.driver" -> "org.h2.Driver",
"slick.dbs.test.db.url" -> "jdbc:h2:mem:test;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=FALSE"
)
val fakeApp = new GuiceApplicationBuilder().configure(conf).build()
//val fakeApp = new GuiceApplicationBuilder().configure(inMemoryDatabase()).build()
//val fakeApp = new GuiceApplicationBuilder().configure(inMemoryDatabase("test")).build()
"TodoDAO" should {
"returns current state in local pgsql table" in new WithApplication(fakeApp) with Injecting {
val todoDao = inject[TodoDAOImpl]
val result = await(todoDao.index())
result.size should_== 0
}
}
}
For fakeApp, I try all three, but none of them work as expected - my test still runs on my local PostgreSQL table (in which there are 3 todo items), so the test fails.
What I have tried/found:
First, inMemoryDatabase() simply returns a Map("db.<name>.driver"->"org.h2.Driver", "db.<name>.url"->""jdbc:h2:mem:play-test-xxx"), which looks very similar to my own conf map. However, there are 2 main differeneces:
inMemoryDatabase uses db.<name>.xxx while my conf map uses slick.dbs.<name>.db.xxx. Which one should be correct?
Second, rename conf map's keys to "slick.dbs.default.profile", "slick.dbs.default.db.driver" and "slick.dbs.default.db.url" will throw error.
[error] p.a.d.e.DefaultEvolutionsApi - Unknown data type: "status_enum"; SQL statement:
ALTER TABLE todo ADD COLUMN status status_enum NOT NULL [50004-197] [ERROR:50004, SQLSTATE:HY004]
cannot create an instance for class dao.TodoDAOImplSpec
caused by #79bg46315: Database 'default' is in an inconsistent state!
The finding is interesting - is it related to my use of PostgreSQL ENUM type and slick-pg? (See slick-pg issue with h2). Does it mean this is the right configuration for running h2 in-memory tests? If so, the question becomes How to fake PostgreSQL ENUM in h2.
Third, I follow this thread, run sbt '; set javaOptions += "-Dconfig.file=conf/application-test.conf"; test' with a test configuration file conf/application-test.conf:
include "application.conf"
slick.dbs.default.profile="slick.jdbc.H2Profile$"
slick.dbs.default.db.driver="org.h2.Driver"
slick.dbs.default.db.url="jdbc:h2:mem:test;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=FALSE"
Not surprisingly, I get the same error as the 2nd trial.
It seems to me that the 2nd and 3rd trials point to the right direction (Will work on this). But why must we set name to default? Any other better approach?
In play the default database is default. You could however change that to any other database name to want, but then you need to add the database name as well. For example, I want to have a comment database that has the user table:
CREATE TABLE comment.User(
id int(250) NOT NULL AUTO_INCREMENT,
username varchar(255),
comment varchar(255),
PRIMARY KEY (id));
Then I need to have the configuration of it to connect to it (add it to the application.conf file):
db.comment.url="jdbc:mysql://localhost/comment"
db.comment.username=admin-username
db.comment.password="admin-password"
You could have the test database for your testing as mentioned above and use it within your test.
Database Tests Locally: Why not have the database, in local, as you have in production? The data is not there and running the test on local does not touch the production database; why you need an extra database?
Inconsistent State: This is when the MYSQL you wrote, changes the state of the current database within the database, that could be based on creation of a new table or when you want to delete it.
Also status_enum is not recognizable as a MySQL command obviously. Try the commands you want to use in MySQL console if you are not sure about it.

Why do I get 'Database is already closed' when invoking StaticQuery updateNA "shutdown;"

import scala.slick.driver.H2Driver
import scala.slick.jdbc.StaticQuery
object Main extends App {
val db = H2Driver.simple.Database forURL (url = s"jdbc:h2:mem:test", user = "sa", driver = "org.h2.Driver")
StaticQuery updateNA "shutdown;" execute db.createSession()
}
Executing this with scala 2.11.5, h2 1.4.186 and slick 2.1.0 yields a "org.h2.jdbc.JdbcSQLException: Database is already closed". What is happening here?
After executing the "shutdown" prepared statement, the slick StatementInvoker asks the database for the updateCount of the statement.
The H2 database doesn't like being asked this because it's already shut down.
I don't know which of the two is not behaving correctly. However, if you happen to have the same problem, to close the database just use
db.createSession().createStatement() execute "shutdown;"

Soap UI, REST API, update database

I am going to use soapUI to test the REST API framework.
Is there a way through which i can insert/update records inside the MongoDB with data in a file type(csv, txt etc) using soapUI tool?
What i am trying to do is validate the API calls and update the database from a data file.
If you are willing to use Groovy script, then you can do this pretty easily.
Put your jdbc driver in SoapUI's bin\ext directory.
https://github.com/mongodb/mongo-java-driver/downloads
(probably where you can it for mongodb)
Then you need roughly these things in your script:
import groovy.sql.Sql
def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
groovyUtils.registerJdbcDriver("org.postgresql.Driver") // NOT SURE WHAT STRING FOR MONGODB
def connectString = "....."
sql = Sql.newInstance(connectString) // TEST YOUR CONNECT STRING IN A SQL BROWSER
def misc = sql.firstRow("SELECT * from table")
groovy.sql.Sql is very nice!
http://groovy.codehaus.org/api/groovy/sql/Sql.html
You can easily use below groovy code in "Groovy test step" of your test case and connect to mongodb. Before that please ensure that mongodb java client jar file and gmongo are in {Installation Directory}\bin\ext folder of your soapUI installation
Gmongo: http://mvnrepository.com/artifact/com.gmongo/gmongo/1.5
Mongodb Java Client : http://mvnrepository.com/artifact/org.mongodb/mongo-java-driver/3.2.2
import com.gmongo.GMongoClient
import com.gmongo.GMongo
import com.mongodb.MongoCredential
import com.mongodb.ServerAddress
//def credentials = MongoCredential.createMongoCRCredential('admin', 'students', 'admin' as char[])
//def client = new GMongoClient(new ServerAddress("127.0.0.1:27017"))
context.gmongo=new GMongo()
def db=context.gmongo.getDB("test")
log.info db.fruit.find().count()
db.fruit.find().each{
doc->log.info doc
}

Connecting to a MongoDb in Node.js Error

Quite simply, I'm trying to connect to a MongoDB via Node.js:
Db = require('../v2/node_modules/mongodb').Db
Connection = require('../v2/node_modules/mongodb').Connection
Server = require('../v2/node_modules/mongodb').Server
console.log "before"
DbServer = new Server("localhost", 27017, {})
db = new Db("twitter", DbServer, {native_parser:true})
console.log "after"
return
That's my code and it's as simple as it gets. My output, however, seems to stop at the db = new Db... line.
It never gets to the after. It doesn't give an error either. I know I have a DB running and when I fire up MongoHub, it's there along with the twitter database
Just remove the native_parser=true would be ok
Native bson parser not compiled, please compile or avoid using native_parser=true