How to externalize mongo db queries - mongodb

I want to externalize complete mongo db query.
I know that we can externalize by splitting query into $project, $filter ,$sort and so on...
But I want the whole query to be externalized.
For example,
db.employee.find({dept:'FINANCE'},{empId:1,empName:1,empDesn:1}).sort({empSal:1}),
Here I want to externalize this mongo query, please share your thought how to achieve it using java.

I connect to mongodb using java for read and right. We use mongo-java-driver-2-XX.jar to connect it
sample code is :
import com.mongodb.MongoClient;
import com.mongodb.MongoException;
import com.mongodb.WriteConcern;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.BasicDBObject;
import com.mongodb.DBObject;
import com.mongodb.DBCursor;
import com.mongodb.ServerAddress;
import java.util.Arrays;
public class MongoDBJDBC{
public static void main( String args[] ){
try{
// To connect to mongodb server
MongoClient mongoClient = new MongoClient( "localhost" , 27017 );
// Now connect to your databases
DB db = mongoClient.getDB( "test" );
System.out.println("Connect to database successfully");
boolean auth = db.authenticate(myUserName, myPassword);
System.out.println("Authentication: "+auth);
}catch(Exception e){
System.err.println( e.getClass().getName() + ": " + e.getMessage() );
}
}
}
For more details check this link

Related

Micronaut + Vertx + testcontainers

How do I configure Micronaut app using Vert.x and testcontainers? I'm trying:
application-test.yml
datasources:
default:
url: jdbc:tc:mysql:8:///db
driverClassName: org.testcontainers.jdbc.ContainerDatabaseDriver
vertx:
mysql:
client:
uri: jdbc:tc:mysql:8:///db
Tests with micronaut-data-jdbc work, but with micronaut-vertx-mysql-client not work:
Error:
Message: Cannot parse invalid connection URI: jdbc:tc:mysql:8:///db
I'm not very familiar with testecontainers, but it seems like it doesn't come up with a fixed port, so I don't know how to configure the connection URI.
Thanks!
It might be a problem that micronaut-vertx-mysql-client does not support the Testcontainers JDBC URL scheme (hard to say without further logs).
In this case, I would suggest to use Testcontainers with database container objects instead of the special JDBC URL.
I got a solution to the problem:
Micronaut + jdbc hikari + vertx mysql client + flyway mysql
package br.com.app;
import io.micronaut.context.ApplicationContext;
import io.micronaut.context.env.PropertySource;
import io.micronaut.core.util.CollectionUtils;
import io.micronaut.http.HttpRequest;
import io.micronaut.http.HttpStatus;
import io.micronaut.http.client.HttpClient;
import io.micronaut.runtime.EmbeddedApplication;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testcontainers.containers.MySQLContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
#Testcontainers
class AuthTest {
static Logger log = LoggerFactory.getLogger(AuthTest.class.getName());
#Container
static MySQLContainer mysql = new MySQLContainer("mysql:8");
private HttpClient client;
private static EmbeddedApplication application;
private static ApplicationContext context;
#BeforeAll
public static void initTests(){
log.info("Mysql is running {}, port {}", mysql.isRunning(), mysql.getFirstMappedPort());
var port = mysql.getFirstMappedPort();
var host = mysql.getHost();
var database = mysql.getDatabaseName();
var user = mysql.getUsername();
var password = mysql.getPassword();
var url = String.format("jdbc:mysql://%s:%s/%s", host, port, database);
application = ApplicationContext.run(EmbeddedApplication.class,
PropertySource.of(
"test",
CollectionUtils.mapOf(
"vertx.mysql.client.port", port,
"vertx.mysql.client.host", host,
"vertx.mysql.client.database", database,
"vertx.mysql.client.user", user,
"vertx.mysql.client.password", password,
"datasources.default.url", url,
"datasources.default.username", user,
"datasources.default.password", password,
"flyway.datasources.default.enabled", true
)
));
context = application.getApplicationContext();
}
#BeforeEach
void beforeEach(){
this.authService = context.getBean(AuthService.class);
this.client = context.getBean(HttpClient.class);
}
#Test
void testItWorks() {
Assertions.assertTrue(application.isRunning());
}
// api tests
}
Help links:
https://dev.to/major13ua/micronaut-integration-testing-using-testcontainers-2e30
https://github.com/major13ua/micronaut-tc/blob/main/src/test/java/com/example/testcontainer/controller/DemoControllerTest.java

Parameterize the find method in python using mongo

Files to upload will be like WFSIV0101202001.318.tar.gz,WFSIV0101202001.2624.tar.gz etc.
INPUT_FILE_PATH = 'C:\Files to upload'
try:
import os
from google.cloud import storage
import sys
import pymongo
import pymongo.errors
from pymongo import MongoClient
from pymongo.errors import ConnectionFailure
except:
print("missing modules")
try:
mongo_client = MongoClient(host="xyz.com", port=27017)
Db = mongo_client['abcd']
coll = Db['shopper_journey_sitedata']
except ConnectionFailure:
print("Connection failed")
date=[]
# Thirdpartyid=[]
input_files = os.listdir(INPUT_FILE_PATH)
# looping through input files
for input_file in input_files:
x = input_file.split(".")
date.append(x[0][5:13])
tp_site_id = x[1]
# print(tp_site_id)
cur = coll.find({"third_party_site_id":tp_site_id})
for doc in cur:
print(doc)
Now i want to parameterize the find() method for every id, so that on each iteration i should get st_site_id ?
above code i tried but ist giving error as "Datas:name error"
You can do one thing
coll.find({"third_party_site_id": { $in :
[318,2624,2621,2622,102,078]}})
If Tid is an array, then you could replace 318 in your query to Tid[I]

Delete operation is failing to delete a document in mongo using groovy in Jmeter

IAm trying to delete a document i created through my script using deleteone as well as deletemany using following code:
import com.mongodb.client.MongoCollection;
import org.bson.Document;
import static com.mongodb.client.model.Filters.*;
import org.bson.Document;
import org.bson.types.ObjectId;
import java.util.Arrays;
import com.mongodb.client.MongoClients;
import com.mongodb.client.MongoClient;
import com.mongodb.MongoClientSettings;
import com.mongodb.MongoCredential;
import com.mongodb.ServerAddress;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
import org.bson.Document;
try {
String mongoUser = vars.get("mongouser");
String userDB = vars.get("userdb");
char[] password = vars.get("password").toCharArray();
MongoCredential credential = MongoCredential.createCredential(mongoUser, userDB, password);
MongoClientSettings settings = MongoClientSettings.builder()
.applyToClusterSettings {builder ->
builder.hosts(Collections.singletonList(new ServerAddress(vars.get("mongohost"), vars.get("mongoPort") as int)))}
.credential(credential)
.build();
MongoClient mongoClient = MongoClients.create(settings);
MongoDatabase database = mongoClient.getDatabase(vars.get("databaseName"));
MongoCollection<Document> collection = database.getCollection(vars.get("collectionName1"));
vars.putObject("collection1", collection);
collection.deleteone(eq("EmployeeName", "Test Automation through Jmeter"));
return "Document deleted";
}
catch (Exception e) {
SampleResult.setSuccessful(false);
SampleResult.setResponseCode("500");
SampleResult.setResponseMessage("Exception: " + e);}
getting this error:
Response message: Exception: groovy.lang.MissingMethodException: No signature of method: com.mongodb.client.internal.MongoCollectionImpl.deleteone() is applicable for argument types: (com.mongodb.client.model.Filters$
SimpleEncodingFilter) values: [Filter{fieldName='EmployeeName', value=Test Automation through Jmeter}]
Possible solutions: deleteOne(org.bson.conversions.Bson), deleteOne(com.mongodb.client.ClientSession, org.bson.conversions.Bson), deleteOne(org.bson.conversions.Bson, com.mongodb.client.model.DeleteOptions), deleteOne(com.mongodb.client.ClientSession, org.bson.conversions.Bson, com.mongodb.client.model.DeleteOptions), deleteMany(org.bson.conversions.Bson), deleteMany(com.mongodb.client.ClientSession, org.bson.conversions.Bson)
What am I doing wrong?
I believe the method is called deleteOne, not deleteone, so the correct way to call it should be:
collection.deleteOne(eq("EmployeeName", "Test Automation through Jmeter"));
See also the javadocs for the MongoCollection: https://mongodb.github.io/mongo-java-driver/3.6/javadoc/?com/mongodb/client/MongoCollection.html

How to create a mongoengine connection with ssh?

I'm trying to create a connection and add a document with mongoengine through an SSH tunnel.
A successful attempt with pymongo can be seen below, I simply want something similar with mongoengine. :-)
from auth import *
import pymongo
from sshtunnel import SSHTunnelForwarder
server = SSHTunnelForwarder(
(HOST_IP, HOST_PORT),
ssh_username = SSH_USER,
ssh_password = SSH_PASS,
remote_bind_address = ('localhost', 27017)
)
server.start()
client = pymongo.MongoClient('127.0.0.1', server.local_bind_port)
db = client[MONGO_DB]
db.authenticate(MONGO_USER, MONGO_PASS)
coll = db.queue_db
coll.insert({"testFile42":43})
server.stop()
mongoengine.connect(
db=DB_NAME,
host="127.0.0.1",
port=server.local_bind_port
)

How to provide pemfile password in pymongo mongoclient in the connection string

Question: How to provide pemfile password in pymongo mongoclient in the connection string?
import pymongo
from pymongo import MongoClient
sslCAFile = data['COMMON_SETTINGS']['sslCAFile'] //reading cafile path from configurationfile
sslpemkeyfile = data['COMMON_SETTINGS']['sslpemkeyfile'] //reading pemfile path from configurationfile(which is encrypted with password)
// now i need to connect by giving the password . but i dont see any parameter for that in pymongo documentation and in authentication examples
connection =
MongoClient(mongos_ip,int(mongos_port),ssl=True,ssl_certfile=sslpemkeyfile,ssl_ca_certs=sslCAFile)
//Help me on this!!!
Unfortunately the current version of pymongo doesn't support this feature
ref: https://jira.mongodb.org/browse/PYTHON-640
What about this:
import ssl
connection = MongoClient(mongos_ip, int(mongos_port),
ssl=True,
ssl_certfile=sslpemkeyfile,
ssl_cert_reqs=ssl.CERT_REQUIRED,
ssl_ca_certs=sslCAFile)
It is from here: http://api.mongodb.org/python/current/examples/authentication.html