Mounting Google Cloud Storage volume in GCE instance - google-cloud-storage

Is it possible to mount Google Cloud Storage as a volume in a GCE instance for an application to access?

It is stil work in progress you can try s3fuse
Do not expect productional reliability just yet,
at least this was my experience about a month ago...

In addition to s3fuse mentioned in another answer, there are also gcsfs (based on s3fuse) and stormfs.
Note that I haven't tried any of them, so I can't speak as to how robust any of these solutions are.

Google has the official gcsfuse FS now. However, it's in the Beta and never will make it to stable release, since there are problems with the performance and stability which cannot be or are very hard to solve.
The performance degrades rapidly if you need to handle many small files. Also, scanning directories with many files (like find does) make take hours or days.
We have tried it for the production, but it has I/O errors occasionally. So now, we use it for the maintenance only.

Related

Is ExtensionContext.globalState a permanent storage?

I search over the internet, VSCode documentation and here in stack overflow, but I can not find a place explaining if ExtensionContext.globalState is a permanent storage solution or not.
With permanent I mean:
it survives across VSCode updates
it survives across extension updates
ideally it survives to extension uninstalls - installations
Which of the above points does the mentioned ExtensionContext.globalState fulfill?
Regards
In my experience, globalState does survive VSCode and extension updates. Even though it may not be documented, it's definitely intended to work that way, as you can see when looking at issues like this one where storage was being migrated due to some internal changes (however it sounds like it was possible to lose storage by skipping a version / going from 1.29 to 1.31 directly and due to some issues in the migration code).
Regarding extension uninstalls - this one is easy enough to test yourself, so I tested this on an extension I maintain that uses global storage. The global storage value was still present when after an un- and re-installation.

Which engine for hosting a perforce server on Google Cloud?

I'm looking for a place to run the perforce service on, since their Helix Cloud has been in beta since early 2015, and doesn't seem like it will leave said beta any time soon.
The problem is that I have little understanding of the whole gimmick. I basically need an engine that can run a service and can store data, so that I and my colleague can connect to it remotely and version our project. I've read their documentation regarding Big Data and Compute Engines, and I still can't figure it out. One seems massive, the other seems more appropriate for a gaming server. I suppose Storage would be the right one but I'm not sure that one can run services.
So do I run it on a VM and turn it off/on when done/using it? Or is there a better way at versioning with perforce in the cloud. Seems kinda pointless running a VM to turn it on and off, since that's what i'm currently doing with my local machine anyway. Unless there is a better solution.
I hope that's the right spot to ask about this, I was referred here by their support page.
PS: We are a small team,50-100 Gigs of versioning storage will last us for ages, please when answering don't assume I need something fancier than it needs to be.
After experimenting for a few days and reading the documentation, I have come to the conclusion that the Compute Engine is the only on on GCP that can run a service. Ended up configin a CentOS and keeping it there.

Google cloud VMs and Cloud SQL

Basically, I'm confused after looking at so many Google Cloud products. I'm starting up a new project that includes a website, an iOS app, and an android app. I've decided to move forward with the Compute Engine as I'll have the flexiblility to do a lot stuff.
I'm thinking of using Cloud SQL for database service. I know that I can install MySQL on my VM. But I'm not sure what's the pros and cons. I'm still researching on this but in the mean time some experts opinion would be greatly appreciated.
TL;DR: Go with managed Cloud SQL. Better than doing it yourself and it doesn't cost much.
I'm no expert but I can tell you from previous experience that a managed database solution feels like much less of a hassle than doing it from scratch. Installing and configuring MySQL isn't especially hard, but it can get tedious (especially for devs like me who have done this many times over).
Also, when your app begins to grow, it'll just be a matter of pushing a few sliders to make your DB respond better to all the traffic. Trust me, you can enjoy a higher quality of life with words like "sharding" and "replication" not being part of your technical vocabulary.
Lastly, I don't remember Cloud SQL to be very expensive.

High traffic site. >10 million user a day. VPS or dedicated server?

We're launching an iPhone app soon, and if everything goes well, we might reach up to tens of millions of user each day.
What server solution would you use for this? I guess a small VPS isn't enough. Is dedicated server a better choice? Is there any good hosting provider that can provide such servers?
I'm a newbie when It comes to servers, and would like some basic info about how to handle this.
Thanks in advance
Unfortunately, you are not really going to know the apps requirements until the app is launched. It all depends on how much the app needs to communicate with the server, and how often users are using the app. Depending on those variables and even more, a VPS might be enough, or you may need a dedicated box, or several. It also depends a lot on the performance of the VPS and dedicated boxes, furthermore it depends on how much access to the system you need.
Ultimately, it seems you may not even know how well the app is going to do, so I suggest you take the cheap/efficient route of using cloud computing. That way you will limit your expenses initially when you app has a small user base. Then your performance can amp up as quickly as your app requires (of course so will the price). That is the benefit of cloud computing, you will not be losing money in the beginning until you have the user base to use your server to its limit. Furthermore, you do not have downtime, etc when/if your server is no longer enough.
Check out Google's Cloud Computing to get a hint of what is possible. I personally like Google's cloud experience, but you have many more options with varying degrees of freedom that you will have to check out. Amazon of course is another possibility.

What are the pros and cons of DynamoDB with respect to other NoSQL databases?

We use MongoDB database add-on on Heroku for our SaaS product. Now that Amazon launched DynamoDB, a cloud database service, I was wondering how that changes the NoSQL offerings landscape?
Specifically for cloud based services or SaaS vendors, how will using DynamoDB be better or worse as compared to say MongoDB? Are there any cost, performance, scalability, reliability, drivers, community etc. benefits of using one versus the other?
For starters, it will be fully managed by Amazon's expert team, so you can bet that it will scale very well with virtually no input from the end user (developer).
Also, since its built and managed by Amazon, you can assume that they have designed it to work very well with their infrastructure so you can can assume that performance will be top notch. In addition to being specifically built for their infrastructure, they have chosen to use SSD's as storage so right from the start, disk throughput will be significantly higher than other data stores on AWS that are HDD backed.
I havent seen any drivers yet and I think its too early to tell how the community will react to this, but I suspect that Amazon will have drivers for all of the most popular languages and the community will likely receive this well - and in turn create additional drivers and tools.
Using MongoDB through an add-on for Heroku effectively turns MongoDB into a SaaS product as well.
In reality one would be comparing whatever service a chosen provider has compared to what Amazon can offer instead of comparing one persistance solution to another.
This is very hard to do. Each provider will have varying levels of service at different price points and one could consider the option of running it on their own hardware locally for development purposes a welcome option.
I think the key difference to consider is MongoDB is a software that you can install anywhere (including at AWS or at other cloud service or in-house) where as DynamoDB is a SaaS available exclusively as hosted service from Amazon (AWS). If you want to retain the option of hosting your application in-house, DynamoDB is not an option. If hosting outside of AWS is not a consideration, then, DynamoDB should be your default choice unless very specific features are of higher consideration.
There's a table in the following link that summarizes the attributes of DynamoDB and Cassandra:
http://www.datastax.com/dev/blog/amazon-dynamodb
Something that needs improvement on DynamoDB in order to become more usable is the possibility to index columns other than the primary key.
UPDATE 1 (06/04/2013)
On 04/18/2013, Amazon announced support for Local Secondary Indexes, which made DynamoDB f***ing great:
http://aws.amazon.com/about-aws/whats-new/2013/04/18/amazon-dynamodb-announces-local-secondary-indexes/
I have to be honest; I was very excited when I heard about the new DynamoDB and did attend the webinar yesterday. However it's so difficult to make a decision right now as everything they said was still very vague; I have no idea the functions that are going to be allowed / used through their service.
The one thing I do know is that scaling is automatically handled; which is pretty awesome, yet there are still so many unknowns that it's tough to really make a great analysis until all the facts are in and we can start using it.
Thus far I still see mongo as working much better for me (personally) in the project undertaking that I've been working on.
Like most DB decisions, it's really going to come down to a project by project decision of what's best for your need.
I anxiously await more information on the product, as for now though it is in beta and I wouldn't jump ship to adopt the latest and greatest only to be a tester :)
I think one of the key differences between DynamoDB and other NoSQL offerings is the provisioned throughput - you pay for a specific throughput level on a table and provided you keep your data well-partitioned you can always expect that throughput to be met. So as your application load grows you can scale up and keep you performance more-or-less constant.
Amazon DynamoDB seems like a pretty decent NoSQL solution. It is fast, and it is pretty easy to use. Other than having an AWS account, there really isn't any setup or maintenance required. The feature set and API is fairly small right now compared to MongoDB/CouchDB/Cassandra, but I would probably expect that to grow over time as feedback from the developer community is received. Right now, all of the official AWS SDKs include a DynamoDB client.
Pros
Lightning Fast (uses SSDs internally)
Really (really) reliable. (chances of write failures are lower)
Seamless scaling (no need to do manual sharding)
Works as webservices (no server, no configuration, no installation)
Easily integrated with other AWS features (can store the whole table into S3 or use EMR etc)
Replication is managed internally, so chances of accidental loss of data is negligible.
Cons
Very (very) limited querying.
Scanning is painful (I remember once a scanning through Java ran for 6 hours)
pre-defined throughput, which means sudden increase beyond the set throughput will be throttled.
throughput is partitioned as table is sharded internally. (which means if you had a throughput for 1000 and its partitioned in two and if you are reading only the latest data(from one part) then your throughput of reading is 500 only)
No joins, Limited indexing allowed (basically 2).
No views, triggers, scripts or stored procedure.
It's really good as an alternative to session storage in scalable application. Another good use would be logging/auditing in extensive system. NOT preferable for feature rich application with frequent enhancement or changes.