Image and File store - HBase, MongoDB or Cassandra - mongodb

Question

I want to built a distributed (across continents), fault-tolerant and fast image and file store. There would be a REST end-point in front of the storage which would serve the images and/or files.
The images or files are stored/inserted from a central location but served from a local intranet installed webserver which authenticates and authorises the user.
One object can have multiple sizes of the same image and probably files related to it. Using the mentioned storage gives me the ability to choose the column family and/or column qualifier to fetch the requested entity.
I did consider the FileSystem, however, to retrieve the requested entity I either need to know the correct path from the DB or the path should be intelligently designed. Which also means creating folders when a new year begins.
One entity can have the different sizes (thumbnail, grid, preview, etc.) for different years.
The request to get the image would look like -
entityId 123
year 2017
size thumbnail
The request to get all available image for a given entity for a year would look like -
entityId 123
year 2017
I am open for any other storage solution as long as the above are achievable. Thank you for your help and suggestions.

Answer 1

You could do as you suggest and build a filesystem table like th
cqlsh> use keyspace1;
cqlsh:keyspace1> create table filesystem(
... entitiyId int,
... year int,
... size text,
... payload blob,
... primary key (entitiyId, year, size));
cqlsh:keyspace1> insert into filesystem (entitiyId, year, size, payload) values (1,2017,'small',textAsBlob('payload'));
cqlsh:keyspace1> insert into filesystem (entitiyId, year, size, payload) values (1,2017,'big',textAsBlob('payload'));
cqlsh:keyspace1> insert into filesystem (entitiyId, year, size, payload) values (1,2016,'small',textAsBlob('payload'));
cqlsh:keyspace1> insert into filesystem (entitiyId, year, size, payload) values (1,2016,'big',textAsBlob('payload'));
cqlsh:keyspace1> insert into filesystem (entitiyId, year, size, payload) values (2,2016,'small',textAsBlob('payload'));
cqlsh:keyspace1>
cqlsh:keyspace1>
cqlsh:keyspace1> select * from filesystem where entitiyId=1 and year=2016;
entitiyid | year | size | payload
-----------+------+-------+------------------
1 | 2016 | big | 0x7061796c6f6164
1 | 2016 | small | 0x7061796c6f6164
(2 rows)
cqlsh:keyspace1>
and
cqlsh:keyspace1> select * from filesystem where entitiyId=1 and year=2016 and size='small';
entitiyid | year | size | payload
-----------+------+-------+------------------
1 | 2016 | small | 0x7061796c6f6164
(1 rows)
cqlsh:keyspace1>
What you cant do with this approach is selecting images for a specific size and id without specifying the year.
For related files you could build a list with the foreign entitiyIds or a seperate grouping table to keep them together.
But the cassandra blob type has a theoretically limit of 2GB but if you need performance the practial limit is about 1MB, in rare cases a few MB (performance degrades in many ways with bigger blobs). If that's no problem go ahead and just try out.
Another idea is using something like AWS S3 for storing the actual data with enabled cross region replication and cassandra for the metadata. But if someone goes to AWS - they also have EFS with cross region replication.
MongoDB is also easily deployed with cross region replication (https://docs.mongodb.com/manual/tutorial/deploy-geographically-distributed-replica-set/). In MongoDB you could keep all your data in one document and just query the relevant parts of it. In my opinion MongoDB requires more housekeeping than cassandra does (there is more config and planning necessary).