I am working on a RSS reader application. And I need to find a backend database. I want the database be embedded because I don't want the users to install a database server.
I know SQLite is a good choice, but I am wondering if there are any other nosql choices?
(I don't yet have 50 rep points to comment on, and build upon, the accepted answer; otherwise I would, sorry!)
You can embed MongoDB in your OEM solution but there are two things to consider:
It is written in C++, so if you are coding in a different language you might need to write a wrapper that launchers the database process separately.
MongoDB is licensed under Gnu AGPL-3.0 which is a copy left server license. The accepted answer, and the Google group quote, both correctly state that this would normally force you to also be AGPL licensed. However, they MongoDb states that the intention of the license is to allow refinements to their code to be submitted back, and that your product will remain separate. This makes me think that the normal copy left rules don't apply.
The goal of the server license is to require that enhancements to MongoDB be released to the community. Traditional GPL often does not achieve this anymore as a huge amount of software runs in the cloud. For example, Google has no obligation to release their improvements to the MySQL kernel – if they do they are being nice.
To make the above practical, we promise that your client application which uses the database is a separate work. To facilitate this, the mongodb.org supported drivers (the part you link with your application) are released under Apache license, which is copyleft free. Note: if you would like a signed letter asserting the above promise please request via email.
Source: http://www.mongodb.org/display/DOCS/Licensing
According to the Google Group, yes it can, but it doesn't cover how exactly.
Yes, but it isn't pretty and will
force your app to be AGPL licensed. If
you are interested take a look at how
the tools handle the --dbpath option.
Source: http://groups.google.com/group/mongodb-user/browse_thread/thread/463956a93d3fb734?pli=1
If you're using .NET, one option might be RavenDB, which is a document database, and can be embedded.
Please checkout https://github.com/Softmotions/ejdb
This project being developed to resolve this issue.
How about Couchbase Lite? It's an open source, embeddable document database. While it can function as a standalone document database, its real value is in its ability to synchronize with remote document databases. It may be aimed at iOS / Android, but it can run on anything with a JVM.
https://github.com/couchbase/couchbase-lite-java
There is no straight forwarding way to use MongoDB as an embedded library in terms of a well-reusable library. Eliot - head of 10gen - spoke of "it would be nice to have one" - but there is nothing available that could be reused in a sane way.
Looks like a lot of OEMs are trying to get Mongo on to their hardware and devices for real-time processing. A link from MongoDBs website
I usually use Buildroot to create a cross-compiled Embedded Linux root file-system along with all the user space packages.
I noticed that MongoDB is one of the packages that's already integrated as one of the Buildroot builtin packages.
You may check out MongoDB make file for some hints regarding how to built it for Embedded Linux.
Related
We are using a Variscite VAR-SOM-AM33 platform for our project, and software platform is based on OpenEmbedded/Yocto.
To ensure the hardware is running with the current software, the devices are connected to the internet. So far, we have been following the OE recipes and generating ipk and applying software updates via opkg.
However, the process is not satisfactory as some of the recipes are poorly written (fails to uninstall/install during the upgrade process). What robust technique/solution are available for OE/Yocto based systems?
Thanks in advance.
I'd like to add SWUpdate to the list of packages that you should consider. It was recommended in a 2016 paper by the Konsulko Group for Automotive-Grade Linux. That paper mentions a few other options, and provides an analysis of the various tools, so it's probably worth a read. Quoting from the paper:
It is our recommendation that the reference AGL software update strategy make use of SWUpdate in a dual copy configuration and integrate OSTree support. This allows recovery from a corrupt partition for the exception case, but also optimizes the common case where small, incremental updates can be quickly applied or rolled back as needed to [meet] OEM policy.
I don't completely agree with the paper. For example, they wrote off Mender.io because it lacks community support, but IMO the Automotive-Grade Linux group is influential enough to create popularity from scratch. Still, it's a good paper, and the fact that they settled on SWUpdate was interesting to me. I was already leaning toward it because the author, sbabic, is involved in U-Boot software development, and we use U-Boot to burn new images into our device.
At the moment I'm unsatisfied with all of the current options, but mostly because I want extra functionality. I'll probably settle on a custom system which incorporates one or more of the aforementioned packages. Unfortunately that's not the kind of definitive answer that SO prefers, but I hope that it was helpful.
I'm working on a metadata layer to integrate the Software Updater (swupd) from Clear Linux with the Yocto Project / OpenEmbedded Core.
swupd performs whole of OS updates, rather than package-based updates, using binary deltas to only update the files which change and to do so in an efficient manner.
I recently wrote some documentation (within the docs/Guide.md file in the meta-swupd repo) about adopting the "Clear Linux Way" to utilise meta-swupd from an OE/YP based distro. A wikified version of that guide, including a link to the layer git repository, are available on the Yocto Project wiki:
https://wiki.yoctoproject.org/wiki/Meta-swupd
I also have a sample layer on Github which demonstrates use of the layer (this is also the distro layer I test much of meta-swupd with):
https://github.com/incandescant/meta-myhouse
About mender.io, I have recently talked to them regarding their open-source update.
Currently, they already have their client side developed, and is working on the server side. They use HTTP and JSON. This is their git, it is only supporting Beaglebone and QEMU at the moment.
The way mender.io works is: they will have one persistence data and uboot. and 2 rootfs (active and backup) to update. So, when there is an update on the server, the users will be notified to pull it down. Give a mender -rootfs image update command. And if the upgrade is success, the user gives another mender -commit command. If there is no mender -commit, the rootfs will be rolled back to the previous rootfs in the next reboot. Mender currently only support update of kernel and rootfs.
The main role of mender.io is to ensure that the mass distributed image upgrade process is recoverable from errors. In the Server side, mender.io developed a management server to the mass distributed devices using UUID.
Not to advertise but please try out mender.io and give feedback so that the software could be more mature.
Mender Introduction pdf
Well, you can either use package based upgrades, like you do. In that case, you'll need to test and verify everything locally before you push any updates to the field. Obviously, you'll likely need to improve a number of recipes. (And I assume that you upstream those immprovements, right?)
The alternative is to use image-based upgrades. Either with full images, see for instance the discussion at Stackoverflow: Embedded Linux mechanism for deloying firmware updates or swupd
Note: I got distracted while writing this answer, so look at the answer from joshuagi; he explains a lot more of swupd.
I think they are two problems here. We (OpenEmbedded) do need to be careful that we do not break package based updates.
Also, there are image updates like swupd (mentioned above and swupdate, described at: https://sbabic.github.io/swupdate/swupdate.html
meta-updater provides support for OSTree-based updates to OE systems.
OSTree is interesting because it provides a half-way house between full image updates (which are large and tricky to handle correctly) and package based updates (which are tricky to make robust). It has a 'git-like' object representation of a root filesystem, and uses chroot and hard links to atomically switch between file system images.
(Disclosure: I'm a contributor to meta-updater)
Posts here were done years ago. Now also RAUC seems to be promising alternative to mender.io
I check iOS document and also google it for a while and get the impression that iOS does not support samba (although there is a samba app for jailbreak iPhone).
But then how do the app FileBrowser achieves that? Does that mean they implemented samba support by their own ?
I also find there is an open source library called tango that provides limited support for samba. So my question is that the best samba support I can get ?
I spent quite some time in implementing my own SMB client so I would like to share some experience here.
First do not use tango in your production code because once you become familiar with SMB you will realize that its implementation is problematic, e.g. it does not support unicode and in some several cases it is not correctly padding so you can't access the folder. And I also heard people said they can't connect window 7 with it.
Second, to summarize my experience I find jcifs guys had said the best: "anyone who wants to implement the CIFS needs to know one very important thing - the "official" CIFS documentation is not accurate
and does not reflect reality. There is NO specification. Do not believe anything you read in the IETF draft or the SNIA document (same document different formatting). Use it only as a hint. The definitive reference is whatever you see on the wire.
WireShark Rules!
... look at JCIFS for design inspiration such as how it puts the request and response into a map by MID and encodes and decodes frames.
Then implement the following commands:
SMB_COM_NEGOTIATE
SMB_COM_SESSION_SETUP_ANDX
SMB_COM_TREE_CONNECT_ANDX
SMB_COM_NT_CREATE_ANDX
SMB_COM_READ_ANDX
SMB_COM_WRITE_ANDX
SMB_COM_CLOSE
all responses for above
"
The only thing I can add is that , you also need to implement TRANS2_FIND_FIRST2 request/response to query the files inside a folder and if you want to find out how many shared folders the server exposes you need to implement NetShareEnum Request/Response.
I used libsmbclient from samba package (http://www.samba.org) for SMB operations on iOS.
You can look on my project https://github.com/kolyvan/kxsmb (objective-c wrapper on libsmbclient). For now it supports a limited set of SMB operations. It mostly was designed for browsing local net and retrieving files from SMB shares.
iOS doesn't give you access to a filesystem that you may be used to. You can read and write files inside your own App's private area, but that's all. You could potentially implement another file system in your application, but you won't be able to use normal file operations.
I'd bet FileBrowser implements the protocol inside their app and implements a file system like layer on top of that for access. I'd guess you could either try to port an existing samba library or roll your own.
iOS does not have any APIs to work with SMB. However, SMB is currently documented by Microsoft and implementing it is not impossible (although not easy too). I've created a freeware project which contains most of what you need to handle SMB. See
https://sourceforge.net/projects/smb4ios/
I'm trying to version control my system deployment bits: OS images, drivers, 3rd party installers, 1st party installers (our applications). I need to have everything in a coherent bundle (tags). And be able to get the entire bundle for any of our active releases.
I'm finding that no common version control system is going to fit the bill. And even the ones that have some capabilities (git media, annex, bup) will not run well on Windows. Digital Asset Management systems seem not entirely suited for this purpose and are very foreign to me.
I heard that people are building custom systems on document databases like MongoDB. I'm asking you for advice on how to build, structure, and maintain a system like this on RavenDB and the Windows server/client platform.
RavenDB has specific support for binary attachments, which is handled in an efficient manner.
You can handle things that way.
I'm not prepared to build a custom RavenDB app/instance for this purpose. After a lot of experimentation, I've settled on NuGet and Chocolatey, along with the original large binary data store, a network drive!
Basically, I want to get an update of the standard command-line subversion client for Windows. I used to be able to get downloads of this quite easily, but it seems like registration is required these days.
I object to registration, but equally, I prefer not to use workarounds that e.g. involve registering with details that won't stay valid.
I already have TortoiseSVN - this isn't about clients in general, but specifically about the standard command-line client. I also don't need the server stuff - just the client.
It looks like I can download the source, but building from that probably involves the usual dependency-finding issues and so on. As this is likely to be a recurring issue, I'd prefer to avoid it if possible.
I'll be more than happy with a torrent link. Googling for that specifically, though, just leads to a lot of what look like pirate versions of commercial clients.
Any ideas? Or is there some good reason for collecting these registration details that might override my objections?
EDIT
Applogies to everyone I "sigh"ed at or whatever over SlikSVN suggestions. Clearly, I need to do better at avoiding making bad assumptions.
Which site are you downloading from? Just taking a look at the download links from the SVN project home (http://subversion.apache.org/packages.html), I see four options for Windows, several of which do not have registration, at least. The SlikSVN link is free and has command line utilities.
You could also install Cygwin, which is definitely free and comes with other useful utilities.
You can get a free one here:
http://www.sliksvn.com/en/download
As you said, you can certainly download and build the source code; it's free software under the Apache License 2.0. But if you want to download an executable without Collab's registration, try SlikSvn.
Binary distributions of the subversion client are available from the official subversion site.
http://subversion.apache.org/packages.html
I'm currently working at a small web development company, we mostly do campaign sites and other promotional stuff. For our first year we've been using a "server" for sharing project files, a plain windows machine with a network share. But this isn't exactly future proof.
SVN is great for code (it's what we use now), but I want to have the comfort of versioning (or atleast some form of syncing) for all or most of our files.
What I essentially want is something that does what subversion does for code, but for our documents/psd/pdf files.
I realize subversion handles binary files too, but I feel it might be a bit overkill for our purposes.
It doesn't necessarily need all the bells and whistles of a full version control system, but something that that removes the need for incremental naming (Notes_1.23.doc) and lessens the chance of overwriting something by mistake.
It also needs to be multiplatform, handle large files (100 mb+) and be usable by somewhat non technical people.
SVN is great for binaries, too. If you're afraid you can't compare revisions, I can tell you that it is possible for Word docs, using Tortoise.
But I do not know, what you mean with "expanding the versioning". SVN is no document management system.
Edit:
but I feel it might be a bit overkill for our purposes
If you are already using SVN and it fulfils your purposes, why bother with a second system?
If you have a windows 2003 server, you can have a look at Sharepoint Services 3.0 (http://technet.microsoft.com/en-us/windowsserver/sharepoint/bb684453.aspx).
It can do version control for documents, and has a nice integration with Office, starting with Office xp, but office 2003 and 2007 are better. Office and PDF files can be indexed (via Adobe IFilter), and searched. You can also add IFilters to search metadata in your documents.
Regarding large files, by default the max filesize is 50MB, but it can be configured.
We've just moved over to Perforce and have been really happy with it. It's a commercial product, but it's so powerful and easy to use that it's worth the price per seat IMHO.
A decent folder structure and naming scheme?
VCS don't really handle images and such very well - would it be possible to have the code in a VCS (SVN/Git/Mercurial etc), along-side a sensible folder structure for the binary-assets (source photos, Photoshop PSD files, Illustrator files and so on)?
It wouldn't handle syncing, but a central file-server would achieve the same thing.
It would require some enforcing and kitten-herding to get people to name things properly, but I think having a version folder for each asset (like someproject/asset/header_logo/v01/header_logo_v01.psd) will basically be like a VCS, but easier to move between different revisions (no vcs checkout blah -r 234 when a client decides they prefered v02 more than v03)
Your question is interesting because your specifying that it be suitable for a small office. At the enterprise level, I would recommend something along the line of EMC Documentum's eRoom, but obviously thats going to be way more than you need, and more than you want cost-wise as well. I'm not sure of the licensing details on this but I've heard that if your office has MS Office, you have access to Sharepoint, which might work well for you. I'm also sure there are a lot of SAAS implementations of this kind of stuff, so you may want to look at that, keeping in mind that the servers will not be hosted by you, so if the material is extremely sensitive, thats obviously not the proper route.
You might want to consider using a Mac as your server and using Time Machine to backup your shared folders. Doing this gives you automatic backups and allows you to share through Samba so everyone can have a network drive on their computer. A Mac server is probably overkill. A Mac Mini would do for a small office or a repurposed desktop machine.
You might also consider Amazon's S3 service to do offline backups. Since it's a pay-as-you-go service this can scale with use, and if you feel you want to move to something else you can always download your data and take it somewhere else.
Windows Vista features local file versioning in its file system, which can be useful, but is limited in terms of teamwork. However, if somebody overwrites somebody else's file, a new version is stored as it should be.
Also consider KnowledgeTree. Have a look at it, some demos/screenshots are available at
http://www.knowledgetree.com/
It has a free open source Community Edition - so it's cost effective. We haven't tried it, but we chose this one over other systems for a small business looking for document versioning solution.