How can I build something like Amazon S3 in Perl? [closed] - perl

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am looking to code a file storage application in perl similar to amazon s3. I already have a amazon s3 clone that I found online called parkplace but its in ruby and is old also isn't built for high loads. I am not really sure what modules and programs I should use so id like some help picking them out. My requirements are listed below (yes I know there are lots but I could start simple then add more once I get it going):
Easy API implementation for client side apps. (maybe REST (?)
Centralized database server for the USERDB (maybe PostgreSQL (?).
Logging of all connections, bandwidth used, well pretty much everything to a centralized server (maybe PostgreSQL again (?).
Easy server side configuration (config file(s) stored on the servers).
Web based control panel for admin(s) and user(s) to show logs. (could work just running queries from the databases)
Fast
High Uptime
Low memory usage
Some sort of load distribution/load balancer (maybe a dns based or pound or perlbal or something else (?).
Maybe a cache of some sort (memcached or parlbal or something else (?).
Thanks in advance

Perhaps MogileFS may help?
MogileFS homepage
Contributing to MogileFS
Google code repo (however note sixapart repo in contributing link).
Also there was a recent discussion about MogileFS performance on the Google Groups / Mailing list which maybe of interest to you.
/I3az/

here I found a ruby impl
https://github.com/jubos/fake-s3
hope that helps
mike

I have created a super simple server, see the put routine in Photo::Librarian::Server.pm it supports the s3cmd put of a file, nothing more for now.
https://github.com/h4ck3rm1k3/photo-librarian-server
https://github.com/h4ck3rm1k3/photo-librarian-server/commit/837706542e57fbbed21549cd9e59257669d0220c

Related

GitHub vs Google Code for a hobby project [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
Note: I have seen this and tried to take as much from it as possible; but I believe my context is different.
I am working on a small-ish project. Call it Foobar. I'm wanting to get this done more organised..I've tried a few projects, mostly as an unorganised programming-as-a-light-hobby student. I'm trying to get more organised; 90% of those projects went after I either failed to document at all, or because I lost them.
As such, I've been thinking about getting version control/hosting going. Not only will it organise me more, but (a big if here) if it gets anywhere into a usable state, it will be easier for people to get.
The two places I'm considering are Google Code and GitHub. From the question I linked:
Google Code:
As with any Google page, the complexity is almost non-existent
Everyone (or almost everyone) has a Google account, which is nice if
people want to report problems using the issues system
GitHub:
May (or may not) be a little more complex (not a problem for me though) than Google's pages but...
...has a much prettier interface than Google's service
It needs people to be registered on GitHub to post about issues
I like the fact that with Git, you have your own revisions locally
From this I'm leaning towards GitHub, as Google Code doesn't look appealing to me.
For a small hobby project - basically making community features irrelevant - are there features that should take me over to one side or the other?
I prefer Google Code since it's just easier for my small personal projects. At the end of the day, for free projects, it's hard to steal time from family, friends or other commitments and the key to making small free projects a success is being realistic with your time. (Elsewise, you get the "80% done" problem.)
Google Code now has GIT support.
Biggest advantage of Google Code is that you don't need a website.
- The frontpage of the project is enough.
- You can add simple binary downloads in the Downloads section.
- In comparison, GITHUB's interface is REALLY confusing to non-programmers. Your frontpage is full of technobabble and so unless it's a coder's tool, you'll need a separate website.
- Marketing's really good- You get a good rank on Google and often you'll be picked up and sometimes reviewed by other download sites. There's no sense donating your time if no one can find your project.
If it is entirely a coder's tool (not just a handy IT tool), then perhaps GITHUB is better.
You say "I believe my context is different", but don't give any reasons why it is. As such, I can't offer you any specific suggestions other than the generic pros and cons, which are outlined in various documents and tutorials online.
My suggestion: pick a program first (git, Mercurial, or SVN) and use it. Find a hosting site that supports the software (at the time of this answer, GitHub for git, BitBucket or Google Code for Mercurial, Google Code for SVN) and use it. If you run into problems, switch to another one.
I've used all three, and typically the problem isn't the hosting, but the fact that you need to learn the program itself. All of the hosting providers listed here will suit you fine until you have a specific reason why it doesn't.
I would go with Github. The single reason for this is, that Google code shows your email and your full name (name only if you have google+ i think). And you cannot disable this at the moment.
Let's split the problem into two parts: for developers and for users.
In fact, if just terminal users are considered, both google code and gitbud has friendly interfaces, and as we all know, google is more well-known towards those who do not program.
But when we turn to programmers, git is more fashion and more comfortable(question?).
So, personally I will choose google code if I am planning an terminal user oriented product and github of course if I want to involve lots of potential collaborators of I was developing an complete programmers' product, like a API something.

Suggestions on project planning? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
This is not a programming question per se, but here it goes. I am a senior CS undergrad, and I started an internship this summer for a mid-sized software company. I've done a few freelancing jobs before, but it's the first time I've been officially (more or less) employed as a software developer.
I've been asked to code an internal website from scratch to be used by separate teams in the company, and I've been given a lot of flexibility in designing it. And therein lies the problem: we've had several meetings and design reviews and everyone seems to have an idea on a new feature, and even conflicting ideas on how things should work.
So far my initial prototype has survived all of this, which is something I was told not to expect - but I knew I had a solid design. While I am not behind schedule, the work is progressing significantly slower than I had predicted. A lot of this has to do with loose specs and constant features requests and changes.
I am to deploy an alpha in a couple of weeks, which I thought wouldn't be a problem, but the way things are going I am not sure how that's going to work out.
Does anyone have any ideas?
Thanks in advance
You are asking a timeless question about (software) project management. There are careers made writing books on the subject.
I agree generally with rockinthesixstring on this.
If you don't have an effective project manager who can filter the customters' requests and manage their expectations, and say "no", then that will have to be part of your job.
Sometimes there is an art to not saying "no". Sometimes you can say it more like "As you see in the schedule, version 1.1 is going alpha next week. The feature list for version 1.2 is already set. I'll add your new feature to the top of the list for 1.3. But if you like, I can call a meeting with the other teams to see if we can reprioritize the 1.2 features."
As to conflicting ideas, if there is no other "decider", than that becomes part of your job as well.
Understand that not everyone will get their way.
Without an approach that addresses these sorts of issues you simply won't succeed by any measure.
I'd start by locking into the features that have been agreed upon and place all feature requests into some sort of project planning software (OnTime Perhaps). Then roll out the Alpha release with the agreed upon specs before moving onto the "we'd like" and the "bells and whistles".
You need to prioritize and triage feature requests, possibly even some of the ones you have already agreed too.
It sounds like product ownership is not clear (as can be expected with internal projects with multiple teams). You should probably run some form of planning game. If you have multiple stakeholders, you might give them each 50 points to vote on all the features in an iteration.
You, as developer, decide how large each feature is. The features with most points/size get into the iteration. If some teams are more important, give them more points. You should also spend some points yourself.
I would like to express my approval of James McLeod's post. The only justification anybody needs for wanting a feature is "user x might try to...". The difficulty is resolving contradictions between their opinions and somebody else's. The feature with a higher 'priority' as assigned by your project management process is the one that gets implemented, at the expense of its competitors if necessary. Ask the people suggesting features to go away and put something down on paper explaining the reasoning behind the feature and the circumstances they think might preclude its inclusion. Letting everybody else see what limitations they think their approach has could help break a decision deadlock. The feature whose case is stated more thoroughly 'wins'.

Version control for InterSystems Ensemble/Caché [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I'm in a group which is starting to develop using InterSystems Ensemble (an integration framework built on top of InterSystems Caché).
InterSystems has not made the Ensemble Management Portal source-control-aware and this seems a source of problems for development team we would like to address.
I would like to know which version control system are you using for Ensemble/Caché and how are you structuring your development process around it.
I've found VC/m, a version control system designed for Caché.
Feel free to add your comments if you have had any experiences with it.
Another alternative seems TrackWare which is also designed specifically for Caché.
If you are not afraid by development work, you can make some development to hook studio to your current source control tool. There are hooks in place in Cache that allows you to detect modifications on files and to interact with your source control tool.
Here a link to a pdf the describes the basis :
Using the Studio Source Control Hooks
Of course with this solution you will have to do a lot of work on your side.
I'm using Mercurial and though I do use a Cache Studio source control hook (I'm not using ensemble) I think basically the same solution would work for you.
The key is that it's distributed source control. So all the hook does is, on a save, export the current file to a folder on my hard drive, and check it in to my local repository. When things are working right locally, I push it to the central repository - in other words, I just use distributed source control in a normal way.
It's nice to commit each save since this gives me a way to roll things back if I mess something up, but it isn't really necessary. You could write something that pushes the code out to your local repository when you call it from the Cache command prompt.
With distributed source control the fact that check-in and check-out features aren't supported doesn't matter, you handle those issues by merging when you push to the central repository (or however you decide to structure your repositories).
One warning - for Cache class definitions, they are exported as XML in a format you don't define. It includes a time stamp of when the file was generated, and a last modified date. These fool the source control system into thinking they have changed when they have not. So you will have to parse the XML at least enough to strip those out. I don't know of a flag to prevent them from being generated in the first place.
Late reply, but anyway - you can take a look at the CodeTools from Synerva. . CodeControl works as a Studio plugin
Caché Source Control
The Best Solution!
Good luck!
Synerva's CodeTools offer a pretty good solution for that. have been using that on several projects for quite a while.

Family Website CMS [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
I am looking for a CMS that would be incredibly user-friendly and would have the following features:
really simple message board (no login required)
family tree
story telling area
photo section
news section
Is there anything out there like this that is really easily configurable? I've already messed around with Mambo and Family Connects, but I didnt like either of those. In the past I've just programmed my own websites, for lack of easily implementable features. However, I've assuming there's something I need out there just like this, that I can't find. Thanks.
I don't want anyone to have to login, for one. The is for a family website, and much of my family really don't know what a website is, let alone how to use one. I want a super simple website with huge buttons and not a whole lot of distractions. Family Connects is a good example of what I want, except the photo album is horrible. I want people to post messages without logging in or signing up, and haven't seen that ability in mambo sites I've looked at.
I can understand your stipulation that your users (family) shouldn't have to sign up - but without a sign-in, your site will be a free-for-all for spammers, hackers and other bored Internet denizens.
That said, my suggestion is to use WordPress for a front end - register your family members yourself, and use a very basic template - or better yet, create one.
I have created a CMS for exactly what you are looking for. My family uses it all the time and the majority of them are not computer savy. The only downside is that it requires a login, but like other people have said, their really isn't a way around that if you want your information to be private.
Anyway, if you are still looking, try http://www.familycms.com/
I've been using http://www.myfamily.com/ and it fits all my needs. It includes:
Pictures (with option to order prints)
Discussion
Family Trees (free from ancestry.com)
Videos
Files
Events
I've setup CMS Made Simple a couple times now. It's all PHP and you can edit it to your heart's content. Give it a try.
CMS made simple seems to die according to this study about content management systems found on MytestBox.com
But if it's just for a family website...
maybe you can try other CMSs which any web hosting company provides (like Joomla or Wordpress).
These can be installed in several clicks (especially Wordpress - you can build a good site in Wordpress and it's very easy to maintain it).
For a family website I thiknk Wordpress is the best and enough (lots of plugins and skins can be found for it on the web.
If you're going for a family website you do have the option of removing the usernames/passwords/accounts by setting it up as an intranet site. Then you can browse it at home or from selective addresses.
I recommend geni.com. It's much better than Myfamily.com

Telligent's Community Server [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
The company I work for is wanting to add blog functionality to our website and they were looking to spend an awful amount of money to have some crap being built on top of a CMS they purchased (sitecore). I pointed them to Telligent's Community Server and we had a sales like meeting today to get the Marketing folks on board.
My question is if anyone has had issues working with Community Server, skinning it and extending it?
I wanted to explain a bit why I am thinking Community Server, the company is wanting multiple blogs with multiple authors. I want to be out of the admin part of this as much as possible and didn't think there were too many engines that having multiple blogs didn't mean db work. I also like the other functionality that Community Server provides and think the company will find it useful, particularly the media section as right now we have some really shotty way of dealing with whitepapers and stuff.
edit: We are actually using the Sitecore blog module for a single blog on our intranet (which is actually what the CMS is serving). Some reasoning for why I don't like it for our public site are they are on different servers, it doesn't support multiple authors, there is no built in syndication, it is a little flimsy feeling to me from looking at the source and I personally think the other features of Community Server make its price tag worth it.
another edit: Need to stick to .net software that run on sql server in my company's case, but I don't mind seeing recommendations for others. ExpressionEngine looks promising, will try it out on my personal box.
I've done quite a few projects using Community Server. If you're okay with the out-of-the-box functionality, or you don't mind sticking to the version you start with, I think you'll be very happy.
The times I've run into headaches using CS is when the client wants functionality CS does not provide, but also insists on keeping the ability to upgrade to the latest version whenever Telligent releases an update. You can mostly support that by making all of your changes either in a separate project or by only modifying aspx/ascx files (no codebehinds). Some kind of merge is going to be required though no matter how well you plan it out.
Community Server itself has been very solid for me, but if all you need is a blogging engine then it may be overkill. Skinning it, for example, is quite a bit of work (despite their quite powerful Chameleon theme engine).
I'd probably look closer at one of the dedicated blog engines out there, like BlogEngine.NET, dasBlog or SubText, if that's all you need. Go with Community Server if you think you'll want more "community-focused" features like forums etc.
You can also take a look at Telligent Graffiti CMS.
http://graffiticms.com/
It supports multiple blogs and authors.
Update: It's now open source and available at http://graffiticms.codeplex.com/
Community Server 2008.5 lets you add several members that can post articles. Also with
Community Server 2008.5 you now have wiki's along with forums and the blogs. This probably has one of the better web based admin control panel's I seen in a while. This let's you easily change several things including the site's theme (or skin). To me it is one of the most scalable applications I have seen in a while. We are using it for our site http://knowledgemgmtsolutions.com.
Skinning is pretty straightforward, and the sidebar widgets aren't very difficult to create (if you don't mind building controls in code). The widgets also allow options for the users to customize them in the control panel very easily. I doubt you'll find a strong community of widget builders for Community Server however. Nothing compared to the dev community for blogs like wordpress.
I recommend starting templates from scratch and adding in CS controls as needed, to get the markup you prefer for styling and to use only what you need.
Setting up different roles for users to post to different blogs is also very easy and requires no coding. You can have blog groups, and allow only certain users to post to certain blogs.
Sitecore's Forum module is powered by Community Server and integrated with Sitecore CMS.
Expression Engine with the Multi-Site Manager works great for that kind of situation.
Have you had a look at the Shared Source blog module for Sitecore?