I'm planning to make web application which allows users to upload music/audio files and host them etc, i'm wondering what the best method would be to go about this, i have used cloudinary in previous projects for image hosting but nothing for audio.
What do companies like Soundcloud use if not there own service which i am assuming is the case.
What would you recommend? It will be vital when it comes to building a scalable and reliable service so I don't want to go into this project uneducated.
ps. I will be using meteor and mongodb to build the application.
I'd recommend getting started with edgee:slingshot in your app. It's much lighter on your Meteor server since uploads and downloads go straight to the storage system. There you have several choices including S3, Google Cloud Storage, and Rackspace Cloud. You could also use CollectionFS but slingshot seems architecturally better suited to this class of problem.
Related
THE PROBLEM
I'm writing a mobile app which will allow a user to log in, save some preferences that must be stored in a database, and display congressional bills to the user.
I've only written simple RESTful services with PHP and MySQL in the past. I'd like to take advantage of newer technologies, and am a little lost on general direction.
The bill data (formatted as JSON) can be gathered by running the scrapers found here. Using docker, I managed to set a working directory and download the files on my local machine.
I've designed a MySQL database for holding the relevant bill and user data.
I started to mess around in Google Cloud Platform, and read the doc that describes different models. I'm thinking of a few different ideas, but aren't familiar with GCP or what I can actually accomplish.
QUESTIONS
1) What are App Engine, Compute Engine, and Container Engine each for? I get the gist that Container Engine holds different instances of stuff you load up with docker, and that Compute Engine sets up a VM, but I don't really understand the relationships. How should I think of them?
2) When I run those scrapers from the shell, where are the files being stored, and how can I check on them? On my computer, I set a working directory, but how do directories work in GCP? Is it just a directory in the currently selected VM, or is this what Buckets are for?
IDEAS
1) Since my bill data already comes as JSON, should I skip the entire process of building a database for the bills and insert them into Firebase somehow? Is this even possible? If so, am I stuck using Firebase's NoSQL, or can I still set up a relational database?
2) I could schedule the scrapers to run periodically, detect new files, and run a script to parse the JSON and insert new bill data into my a database (PostgrSQL?/MySQL?). Then I would write an API.
3) Download the JSON files to a bucket, and write an API that reads from them. Not sure how the performance would compare to using a DB.
I'm open to other suggestions as well.
For your use case (stateless web application), App Engine is probably your best choice. The Google documentation has severalcomparisons of your computing options
You can use App Engine with PHP and cloud-hosted MySQL if you want, which could be a good way to get your toes wet without going in over your head.
I have a php app running on several instances of Google Compute Engine (GCE). The app allows users to upload images of various sizes, resizes the images and then stores the resized images (and their thumbnails) in the storage disk and their meta data in the database.
What I've been trying to find is a method for storing the images onto Google Cloud Storage (GCS) through the php app running on GCE instances. A similar question was asked here but no clear answer was given there. Any hints or guidance on the best way for achieving this is highly appreciated.
You have several options, all with pros and cons.
Your first decision is how users upload data to your service. You might choose to have customers upload their initial data to Google Cloud Storage, where your app would then fetch it and transform it, or you could choose to have them upload it directly to your service. Let's assume you choose the second option, and you want users to stream data directly to your service.
Your service then transforms the data into a different size. Great. You now have a new file. If this was video, you might care about streaming the data to Google Cloud Storage as you encode it, but for images, let's assume you want to process the whole thing locally and then store it in GCS afterwards.
Now we have to get a file into GCS. It's a PHP app, and so as you have identified, your main three options are:
Invoke the GCS JSON API through the Google API PHP client.
Invoke either the GCS XML or JSON API via custom code.
Use gsutil.
Using gsutil will be the easiest solution here. On GCE, it automatically picks up appropriate credentials for your service account, and it's got several useful performance optimizations and tuning that a raw use of the API might not do without extra work (for example, multithreaded uploads). Plus it's already installed on your GCE instances.
The upside of the PHP API is that it's in-process and offers more fine-grained, programmatic control. As your logic gets more complicated, you may eventually prefer this approach. Getting it to perform as well as gsutil may take some extra work, though.
This choice is comparable to copying files via SCP with the "scp" command line application or by using the libssh2 library.
tl;dr; Using gsutil is a good idea unless you have a need to handle interactions with GCS more directly.
How to auto sync the images and video from the app into the Api Web services.
Do we have any tutorials or source codes on this.
Thanks in advance!!!
There can be multiple approaches depending upon what you are trying to achieve and what backend you are using.
If you are using simple API web services, you need to write your own mechanism both on server as well as client side, which will identify records to be synced based on TimeStamp and few other columns in your DB
If you are open to use other Platforms like Parse.com, they provide in built APIs and libraries to take care of syncing logic
As you are trying to sync file(Images and Videos), you can also try DropBox, iCloud, or Google Drive. Here is a link to an article which i found in a quick search which uses CoreData and Dropbox
Let me know if this helps.
Do I need to implement my own sync methods in order to make an offline web app (html+css+js) stay up to date with changes made on the server (and viceversa)? I'm using MySQL on the server side.
I read Two-way sync between iPhone application and web application with some pointers but I think they're talking about native applications when they mention CFUUIDCreate and I wander if this is possible for the Web.
Does someone have some code to share or maybe can point me in the right direction?
Thank you!
P.S.: I hope my english is not that rusty ;)
To store static contents on the client-side, as Jethro Larson said, the Application Cache Manifest is the way to go to cache the static contents of your website (HTML, CSS, JS and images).
To handle dynamically generated contents offline, you can use javascript templates. There are several solutions for this.
To sync the two databases, there is a project called persistence.js (persistencejs.org) which is a javascript library which offers a unique API to work with WebSQL databases, Local Storage, etc. They have a plugin for this library called persistence.sync (persistencejs.org/plugin/sync) which syncs the remote database with the server's one. It consists of POST and GET requests to a specific url that you can configure (for example yourapp.dev/sync). They have an example back-end written in node.js and here is one for Rails. It's simple to understand and persistence.sync is well documented.
Look at the offline cache:
http://www.webreference.com/authoring/languages/html/HTML5-Application-Caching/
http://www.google.com/search?q=offline+cache+html5
http://www.slideshare.net/search/slideshow?q=offline+cache
I wanted to make a web application through which :
user(end user) can create his own
web application online.
which will be integrated with the
databases and application/web
servers
and user can also publish that
application withing that application
only...
Doing some research i come to know it goes under category of PaaS(Platform as a Service)
But i am confuse how can i do that ?
I want to build that application using Java/J2EE.
Can any one explain me from where should i start ?
Or which framework / technology is better to implement it in Java ?
Waiting for your helpful reply....
There are several options as far as choosing a PaaS - each PaaS vendor has its pros and cons, so you need to consider a range of issues:
Language support - Do you want just Java or maybe additional languages/frameworks?
Hosting - Are you going to use a cloud computing provider like Amazon Web Services? Or are you using your own hardware? Or do you want multiple options?
DB support - You mentioned using a central DB. Do you plan on installing/administering this DB yourself, or use a hosted DB service? Do you need the PaaS to support the particular DB you want? Perhaps you need multiple Databases to provide scalability and separation among clients?
There are several good comparisons of PaaS vendors out there. This might be a good place to start: Looking for PaaS providers recommendations
Full disclosure: I work for Cloudify, an Open-Source PaaS provider.
I think while choosing a PaaS you should think of the below points.
It should be easy to deploy, scale and manage
Scaling
Flexible topology
Data safety
No vendor lock in
Open software stack
I suggest you, before going to any big vendor try a new player in town Jelastic. I recently used their services and continue to do so and they are just awesome. Their web UI is best in class and has no drama of API, SDK or downloadable software. Let me know if you use them so that I can also have some feedback before I migrate all our apps to them.
Surya
As far as i understand, you are looking for a solution that Viravis is currenlty implemented.
But believe me it is not an easy task to build that kind of platform. i strongly suggest you to look at some PaaS providers to get involved and build a business relationship to achieve your goal. They are just exists for this purpose. Provide platform for people who just want to develop a single online application or some professional who want to build an SaaS business.
I think it is better to hire the platform rather than build it from scratch (ofcourse if you dont have to...)
A web application consists of language, framework, database and Web-server.
If you want to make a web application in java, you should try GRAILS framework.
Its fast and web development is very easy and you can use java code as it is, in it.
Secondly you can choose database from MySQL, PostgreSQL etc or if you want to use No-SQL dbs, you can choose from MOngoDB, CouchDB etc.
Web-Servers can be tomcat, jetty etc.
After creating you application, you would like to host your application on web, Here PaaS comes to play. It helps you to deploy and host your application on web. So what you have to do is only to focus on your application and then deploy your app on any PaaS providers.
There are many PaaS providers in market, you can choose any.