I have a project with 16 micro-services using Springboot 2.0.5. One of the service that calculates OHLCV of a very large trade data stored in Mongo collection.
MongoDB 6.0 has introduced timeseries functionalities that can make our life easier. But going through Mongo and Spring docs, I realized that I'll need to upgrade my Springboot version to 2.7.x, because I am currently using spring-boot-starter-data-mongodb, which doesn't have TimeSeries support.
I have also tried to migrate entire project to 2.7.4, removing Netflix Zuul, which is literally a backbone of entire architecture, but it was way too much of change. So I have decided to roll back to what was and is working fine. And it doesn't feel logical to migrate such large code base for just one feature support.
Is there any other way we can use timeseries functionality in Mongo (or other DB) which supports Springboot 2.0.5 ? The data we are talking is millions of document....
For anyone coming to this hoping for an easy answer, there isn't (at least I couldn't find it)
The way I solved is to NOT use any dependecy hack to support latest MongoDB drivers in an old Spring boot version.
Rather, I used PostGres with TimeScaleDB extension which supports OHLCV out of the box with minimum code. Also, is way too faster, and way more easy then to work around the dependency version issues.
Related
My company runs a huge Lumen 5.1 project on MySQL. They want to add to it analytics, and they that part to use MongoDB
Is it possible to use MongoDB without any third party libraries? I one going to use
https://github.com/jenssegers/laravel-mongodb
But the tech lead thinks Laravel support MongoDB by default, I'm just asking this question to check whether that's true or not.
Edit:
If MongoDB isn't an option, does Laravel support any other NoSQL by default?
Laravel does not support MongoDB by default.
You would need to use one of several available third-party packages. I like moloquent because it maps mongo db collections to laravel models just like eloquent.
You can use Redis if you need a natively supported NoSQL db or could consider ElasticSearch (not supported natively) if you are going to store a lot of meta data and then analyse it. Tools like kibana and logstash might get very helpful.
I'm using Spring Data for Neo4j and MongoDB, I find it awesome, but now I just found out about the embedded and not embedded DB stuff.
Here's my situation :
Using Spring Data with the annotations, repositories, templates and thinking that I just need to change the DB address to make it work elsewhere.
My questions :
1) I don't even understand what they mean by embedded vs non embedded (on the same machine vs on a distant machine ?)
2) Do I have to change all the work I've done to make it work with a 'non embedded' DB ?
What I wan't to do is to deploy my Spring Boot app that is using Neo4j to Heroku or CloudFoundry and use Graphen (Neo4j paas) for the DB. But when I saw all this story about Spring Data working only for embedded, I just lost all the hope and happiness I had when building my app.
3) If 2) is Yes, is it an easy transition ? is there a lot of things to change ?
EDIT :
Here's what I'm talking about :
http://inserpio.wordpress.com/2014/04/30/extending-the-neo4j-server-with-spring-data-neo4j/
He's adding some custom boilerplate code to make it work with a non embeded DB, is it ok ? Why it doesn't work as any other DB (like with JPA, where you just specify the address of the DB).
inserpio here. Don't lose your happiness, please: Spring Data Neo4j team is working hard to implement a new release that improves remote performances.
When Spring Data Neo4j started neither Cypher nor Neo4j-Server existed, while only the embedded version was available. As the server version was delivered SDN team provided a quick solution that works well if you only use repositories, but becomes a little bit too chatty is you want to use #Entity too. The problem is matching those #Entity with the returned nodes.
Since the new version is still not completed, for the moment, you could move you persistence-logic more tight to the database as a server extension. I explained it on the link you mentioned. It's a really fast refactoring: just move your entities and repositories to a new simple java project, install the resulting jar in the 'plugins' folder, one line configuration in the neo4j-server.properties and expose your queries as simple REST services.
Hope this could help.
Do not hesitate to contact me for any further question.
Cheers,
Lorenzo
We have an EZ Publish 5 already operational with an Oracle 11g database as a persistence backend.
We planned to get rid of Oracle and we know that we could use Postgresql for sure as a new RDBMS, but we also planned to use MongoDB whenever it's possible.
So the question is: Can we pair EZ Publish 5 to a MongoDB NoSQL database ?
Quick answer: You cannot now as there is no MongoDB driver implemented.
There was a NoSQL Driver planned in the 5.x series, but it is not yet implemented in the latest eZ Publish 2013.06.
With the official roadmap not being updated since the 4.6 release, it hard to guess when the NoSQL driver will be implemented, and if MongoDB will be supported.
Theorically, as the new persistence API make it "easier" to add new storage drivers, you could write a custom MongoDB storage driver.
But in practice implementing a storage driver is quite complex and resource consuming.
Contacting the eZ Systems office in your region and directly ask them when a MongoDB driver will be available might be fastest way to get a clear answer.
I hope it helps.
It is not currently possible as there is no storage engine implemented for MongoDB.
It is indeed planned to implement a NoSQL engine, but there is no version tag on this for now.
However, the new persistence API makes it possible to "easily" implement such a storage engine, but you will be limited by the fact that eZ Publish 5 still uses the legacy back office which runs in the legacy stack (so with 4.x infrastructure which is not compatible with NoSQL).
In short: Implementing a new storage engine will make it only accessible from Public API and REST API.
Since ez5 is a symfony application you can achieve this the symfony way
http://symfony.com/doc/current/bundles/DoctrineMongoDBBundle/index.html
just remember they have renamed the app folder and the appkernel.php.
Haven't tried it yet though
I've recently came across a need to store a higher amount of files in my application and because PaaS platform used to host the application provides mongo, I've would like to use it.
However because I'm quite inexperienced with mongo I have almost no idea what is the current state of mongo related plugins and tools for grails. What should I use? As I want to keep domain classes in SQL database and use mongo only to store related files (in this case it will be mostly a bunch of PDFs and text documents related to domain instance) the mongoDB ORM [1] plugin seems too "heavy". Unfortunately mongoDB ORM is probably the only mongo plugin for grails in active development at the moment.
In short, what would be the best plugin / library tool-set for this purpose? The closest thing that matches my need I've found is grails-mongo-files plugin [2], which is probably a little bit outdated with no further development.So far it seems that I will have to use mongo's java driver (or the gmongo wrapper) and write some storage service and taglib by myself (what is not necessary a bad thing).
[1] http://grails.org/plugin/mongodb
[2] https://github.com/quirklabs/grails-mongo-file
There is also the mongodb gridfs plugin. http://grails.org/plugin/mongodb-gridfs
One thing to consider is that gridfs effectively does two calls to mongo, one to retrieve file information and one to retrieve the file. So it might not be a good fit if your files are under 16 megabytes.
Here is a post on how to do this manually if you want to bypass plugins - http://jameswilliams.be/blog/entry/171
I am using Spring Data in my Java application to connect to MongoDb and have a requirement around versioning the documents (basically storing the history).
It seems that its pretty straightforward in Ruby, if one uses Mongoid
I was wondering if spring data has something similar for Java. Or are you better of trying to implement your own.
Yes there is a very good feature in Spring data which is auditing you can refer to the following link
http://www.javacodegeeks.com/2013/11/auditing-entities-in-spring-data-mongodb-2.html
After lot of research I found that https://javers.org/documentation/spring-boot-integration/. This works like rock solid and very easy to implement.
This library helps to store all the history of the changed fields and easy to query over it and it has great support of it. The sample POC shared here: https://nullbeans.com/auditing-using-spring-boot-mongodb-and-javers/