We are planning to migrate from Prevayler (http://prevayler.org/) to db4o (http://www.db4o.com/), so we wanted to know experiences, pros and cons, and best practices to move forward. What do you think about it? Is it a good solution? Or, maybe moving forward with a NoSQL standard solution would be better? (Such as MongoDB or CouchDB). Thanks!
we use db4o as main db in our production environment (both embedded and client/server), so i am going to share some of my experiences.
Pro:
- very easy for development (you just implememt data classes)
- support both embedded/client server under the same interface, which makes it easy to unittest
- decent performance for small projects
Cons:
- db4o is no longer developed so it's quite dead project, and you wont get much of support for it
- [client/server] everytime you change model you need to redeploy server (not talking about the fact that you need to host server app yourself)
- [client/server] performance degrade with more clients connected - not possible to scale
Summary: db4o is very good as embedded db (mobile app, desktop local db), but if it comes to server application you get into troubles
Given that I did not receive so much feedback, we gave it a try. So far, it seemed to be a good option for a embedded database, that makes much easier the deployment. So, we wrote again the whole persistence layer, with their unit tests and seemed to work fine.
Then, we tried with real data, and we start to have some weird Null Pointers, and we did not know why. Then, we started to read and we found this issue: http://www.gamlor.info/wordpress/2009/09/db4o-activation-update-depth/.
We've been trying to solve for a few hours, but then we decided no to spend more time on it, and found another way. CouchDB, OrientDB or MongoDB are still on our list.
Related
I am about to start a project that I hope will some day require the capacity for scaling. The key question for me is: should I invest the effort to design for this eventuality now, or should I cross that bridge when I get there?
I know how these things go: everyone thinks their project might scale, but most projects never do.
Is there a set of best practices that will allow you to scale more easily at a later stage, while not investing too much in something that may never be needed?
Obviously I have no experience with scalable web sites. Currently I am leaning towards Scala / Play! / Akka. From what I can glean from the Akka website, it is very suitable for this kind of project (in that it provides a toolset that allows development on a single machine and scaling out to an arbitrary number of machines).
The project is a consumer-facing web site that involves some user interaction (comments, messages, voting etc.). The main focus is editorial, though. It's no Facebook :)
Not being experienced in any of these technologies (my experience mainly coming from PHP, AS3, Objective-C), I probably have a little too much on my plate right now. But then I am not going to go at it right away. I am aware that I need to get some experience using Scala and Play! first.
Any advice is greatly appreciated.
Start out by simply designing you model, build unit tests for that and then set your presentation layer on top of it. As long as your model is sensible, it will be easy to scale Play out to any number of machines. If you go for the build in JPA support, you can always deal with the question of which DB to go for.
You have larger things on your plate for the time being. So just make sure your design is consistent and sensible, then scaling will not be a problem.
You've got a head start by using a stateless web framework like Play! that won't get in the way when you need to scale. As ExxKA says, keep your model clean and sensible. This will help you keep the advantage of complexity.
Don't consider it a failure when you need to refactor your code - or even rewrite important parts of it. This is a natural part of a growing project, like a snake shedding its skin.
There are inevitably things that you'll learn in the process of writing the project, so don't try to anticipate them all right now.
There are must be a lot of apps that are designed to communicate with server. My question is only about App installed on iOS device + Server side service interaction. Web app is not what I am talking about, and there should be no webpage involved in this discussion at all. Typical examples are Apps like Instagram and Twitter, in which most of the information exchanged between the App and the server is just data like String, Image and Integers(wrapped in JSON or XML), no webpage presentation needed.
My question will be: if you are an independent app developer, and you are designing such an app from scratch without any existing website API, database structure or application(so you are not limited by any existing API or database structure or application protocol), What will be the most efficient approach?
What the sever side need to do are:
receive data send by the App;
process the data with designed logic;
interact with database(like MySQL);
do necessary data mining and analysis---this could be a constantly running service or one time task requested by the App client;
send the data back to the App upon request or spontaneously;
exchange or broadcast the data between/among different App clients (i.e.: group chatroom and peer to peer message);
As far I as know there are 3 obvious options to implement the server side:
PHP
Python
Ruby on Rails
(please feel free to add more options)
My questions are:
which one is the most appropriate choice to implement the server side?
If the App is focusing intensively on natural human language/text searching, analyzing and data mining, which one is the best choice? I heard Python is doing pretty good in this area.
Any advice on the database choices? I am using MySQL for now, and I found it's quite powerful for my purposes, I heard Twitter is switching to Cassandra. Will that be too difficult to start with?
For the server end, if you need to build a Server management interface, for you as an admin to manage and monitor the community, membership, data and such, is there any existing solution, or framework or tool for that? what will be the most efficient approach?
If a new programmer has no experience in non of them, which one you suggest he/she to start with?
Is there any good reference material or sample code on the server side in such context we can learn from?
I know there are a lot of very experienced experts on these areas on Stackoverflow, but I saw more newbies who just entered the iOS developing area without much knowledge in server/database programming experience. And I hope this thread can help these who are thinking to design an App with server/client structure but have no idea where to start with.
ps: I will keep updating this question thread and adding my findings on this topic, to help all other users at stackoverflow. :-) Please try to make your answer informative, easy to understand, and constructive. I guess most of readers for this thread will be new members of this great community.
Are you sure you want to spend time & money to develop your own Server & develop your own API?
There are lots of mBaaS (mobile Backend as a Service) providers today such QuickBlox, Parse,StackMob, which are ready to use and they have great Custom Objects API and some of predefined modules. They have great free plans with big quota. Some of them such QuickBlox has Enterprise plan - so you can buy license and they server team update server for you purpose.
So, i recommend not develop your server and think about mBaaS market.
Just about your issue - I can recommend look at QuickBlox Custom Objects code sample and also Custom Objects API. Custom Objects module provides flexibility to define any data structure(schema) you need. Schema is defined in Administration Panel. The schema is called Class and contains field names and their type. I think it's what you need.
which one is the most appropriate choice to implement the server
side?
Well that depends on what you know, there is reason to choice one of the other
If the App is focusing intensively on natural human language/text
searching, analyzing and data mining, which one is the best choice? I
heard Python is doing pretty good in this area.
This would reflect on your first question, you pick the language on you needs. Thus if python makes it easier then pick that one.
Any advice on the database choices? I am using MySQL for now, and I
found it's quite powerful for my purposes, I heard Twitter is
switching to Cassandra. Will that be too difficult to start with?
Again not one that is easy to answer, since it all has to do with requirements. But any SQL server will do. Cassandra is meant for "scalability and high availability without compromising performance" accourding to there website. Do you think you webservice will get many request then it might be a choice to consider.
For the server end, if you need to build a Server management
interface, for you as an admin to manage and monitor the community,
membership, data and such, is there any existing solution, or
framework or tool for that? what will be the most efficient approach?
This again is only going to be answered when you pick the a SQL server and server language.
If a new programmer has no experience in non of them, which one you
suggest he/she to start with?
Start with something simpler, you are really going out on a limb here.
Is there any good reference material or sample code on the server
side in such context we can learn from?
Propably there is some, but you should really start small and work from there.
Twitter started out as a Rub on Rails app and is working on scalability and availability which ruby is not really good ar (that is my person opinion). or Look at facebook they have written a php to c compiler to make php run faster.
The only thing I can say to start code, when you app does take off then tackle the some of the performance issues.
And since you state that you are new to programming do not bite of more then you can chew.
This is a huge question and I don't think there is a best answer. It most depends on what you care about, such as how quickly the development process, how easily the implementation, etc.
And which one is popular, which one is cool, I don't think it make really sense.
In my personal opinion, I'm good at ASP.NET and I can get Windows server easily, so I'll start with an ASP.NET service to provides data.
And, to be continued.
Good day,
Our school, a small high school in semi-rural New Zealand, is currently looking into online homework solutions. Being one of the IT guys, I have been asked to look into some of the options. We have checked around and there are no robust solutions that cover what we are looking for. So, we are considering development of our own system, either on our own or in collaboration with some other schools.
Before I put significant time into any one option, I would thought I should ask for some expert advice.
Please keep in mind that one of our major obstacles is that around 20% of our students are on dial-up because broadband is not available in their area.
We are also not limited to the technologies listed, they just are the ones that we have been looking into up to this point.
With that in mind, here goes.
1. Is there a way to pre-determine the bandwidth needed for these technologies?
2. If bandwidth continued to be too limiting, could the final solution stand alone so we could distribute it to students on CD or USB stick?
3. What are some pros/cons of each for use with databases, specifically mysql or postgresql? (After all we do need to keep track of lots of data)
4. What are some pros/cons of each for of these RIA development?
I appreciate everyone for sharing their time and expertise on the matter.
Cheers,
Ben
1) If you write full-AJAX application, such as in GWT, the bandwitch will be:
a) the size of application java script, images, etc., you may consider that everything is loaded when user logs in (cache for images may seems to be big, but it's easily overloaded)
b) the size of communication - in GWT it depends only from you! no magic full-frame reloading, sending is only what YOU are wanting to send
2) I do not catch your point, stand alone applications can be distributed such way, applications that use databases generally can't
3) postgresql has high compatibility with Oracle - same transaction+select for update behaviour, pgPLSQL is highly inspired by PL/SQL (easy to rewrite stored procedures).
I personally suggest MySQL for a school project for its simplicity. PostgreSQL is powerful but a bit complicate to configure and the visual tool for optimizing queries not good.
Without considering the bandwidth, I definitely suggest ZK since, again, it is much easier to learn, to develop and to maintain (also much more powerful). The bandwidth consumption and latency of GWT really depends how much effort you want to invest, and how skillful your people are familiar with distributed computing, while the network bandwidth is basically the states of UI (not data), which is reasonably small. In short, you could have the best network bandwidth and latency if you optimize it at the best with GWT, while ZK is less to worry but, if you want to improve, you have to use jQuery (i.e, in JavaScript).
Thanks lechlukasz, I appreciate your comments and insight.
I will clarify my point about stand alone applications. We have a number of students, as high as 20%, who do not have access to broadband due to their geographic location. We are considering, as part of the design, how we may be able to distribute a stand alone version.
For instance, if we were to abstract all the database calls using a separate class in GWT, we could recompile a stand alone version that didn't make the database calls. The database would likely only be for tracking results and reporting.
In reality, we would likely implement the front end product first with references to empty methods for storing the results in a database and implement those methods at a later time.
For the record, we have started to code up some test cases using GWT/SmartGWT and are pleased with the results. Although we cannot comment on the other technologies considered because we didn't try them to the same extent, we are pleased with the results to this point of the project.
Cheers,
Ben
We are doing research right now on whether to switch our postgresql db to an embedded Derby db. Both would be using glassfish 3 for our data layer. Anybody have any opinions or knowledge that could help us decide?
Thanks!
edit: we are writing some performance tests ourselves right now. Looking for answers more based on experience / first hand knowledge
I know I'm late to post an answer here, but I want to make sure nobody makes the mistake of using Derby over any production-quality database in the future. I apologize in advance for how negative this answer is - I'm trying to capture an entire engineering team's ill feelings in a brief Q&A answer.
Our experience using Derby in many small-ish customer deployments has led us to seriously doubt how useful it is for anything but test environments. Some problems we've had:
Deadlocks caused by lock escalations - this is the biggest one and happens to one customer about once every week or two
Interrupted I/Os cause Derby to fail outright on Solaris (may not be an issue on other platforms) - we had to build a shim to protect it from these failures
Can't handle complicated queries which MySQL/PostgreSQL would handle with ease
Buggy transaction log implementation caused a table corruption which required us to export the database and then re-import it (couldn't just drop the corrupted table), and we still lost the table in the process - thank goodness we had a backup
No LIMIT syntax
Low performance for complicated queries
Low performance for large datasets
Due to the fact that it's embedded, Derby is more of a competitor to SQLite than it is to PostgreSQL, which is an extremely mature production-quality database which is used to store multi-petabyte datasets by some of the largest websites in the world. If you want to be ready for growth and don't want to get caught debugging someone else's database code, I would recommend not using Derby. I don't have any experience with SQLite, but I can't imagine it being much less reliable than Derby has been for us and still being as popular as it is.
In fact, we're in the process of porting to PostgreSQL now.
Derby still is relatively slow in performance, but ... where ever your Java application goes your database server goes, completely platform independent. You don't even need to think about installing a DB server where your Java app is being copied to.
I was using MySQL with Java, but having an embedded implementation of your Database server sitting right within my Java App is just stunning and unprecedented productivity, freedom and flexibility.
Always having a DB server included whenever and wherever on any platform for me is just heaven !!!
Have not compared Postgresql to Derby directly. However, having used both in different circumstances, I have found Derby to be highly reliable. However you will need to pay attention to Derby configuration to ensure it suits your application needs.
When looking at the H2 databases stats site, it's worth reading follow up discussion which comes out in favour of Derby compared to the H2 conclusions. http://groups.google.com/group/h2-database/browse_thread/thread/55a7558563248148?pli=1
Some stats from the H2 database site here:
http://www.h2database.com/html/performance.html
There are a number of performance test suites that are included as part of the Derby source code distribution itself; they are used by Derby developers to conduct their own performance testing of Derby. So if you need examples of performance tests, or want additional ones, you could consider using those. Look in the subdirectory named java/testing/org/apache/derbyTesting/perf in the Derby source distribution.
I'm not sure what you mean that Derby is embedded. Certainly that is an option, but you can also install it as a separate server.
A job came in to me that's built with CodeCharge - had a look at it and seems to be a pretty basic point-and-click site builder tool. Has anyone got any in-depth experience with it? My first reaction is one of horror and to just rebuild the code in Rails or PHP but I thought I'd ask the question first, maybe i'm missing something...
I'm currently evaluating it for use in quickly producing a back-office environment, and it seems a very comfortable way of getting an interface up and running quickly.
Once the code is generated it is simple, but easily modified for any special needs you have.
In short it is a very good tool (though it would seem that Iron Speed Designer is much better though much more expensive) for what it does - fast prototyping and almost no coding approach to developing a web application. In my opinion, not much different than a Ruby On Rails application in terms of functionality, and, I can generate the code in any language I want.
You have to realize, it is all about speed - some quality is thrown in, but it is a very generic of quality - this is NOT a custom application, mind you, the resulting code you get here might not be pretty but it is a few level higher than your average script kiddie code.
I'm seriously considering this tool for creating back-office applications for sites I develop - a fast and easy solution instead of mucking around in tables of data and useless and repetitive SQL code.
Codecharge is a powerful tool that I have used for over 10 years to build very large content management systems, CRMs and many other management type tools.
Its far from simple once you get into it, and honestly when you use a tool like Codecharge to cleanly generate your user interfaces, you end up with a healthier application that can last many many years.
For instance, I have three clients that have been running Codecharge created portals for over 10 years and they always comment how bug free they have been.
There is a learning curve to learning CodeCharge but it will also teach you what entire applications should have in place and it will please the executives every time because they can get functionality within hours or days rather than weeks or never.
Development teams will often not like it though, because they would rather hand code everything or use the latest and greatest approach to development.