Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
What is your method of "mapping out" an idea before creating it?
Say I wanted to take on a big project, for example at the scale of a site like Facebook or MySpace. What planning/design steps should I take before I start the actual work?
For example, should I map everything out page by page (their functionalities, data, etc.)?
For a large project first think of a one-liner to description of your site (try to not use any buzzwords here). Next think of three design maxims (rules your design Should never conflict with). Then draw a few views and think up a few user cases (1 day) then work in code for 2 weeks (this will be a throw away prototype so just work as fast as you can forget about bugs and details, don't worry about code smells or design patterns, just make as much as you can), then revaluate all the steps above and throw away your two week prototype, and begin your project in a serious manner applying solid engineering and design. After a month has gone by evaluate your(team) moral and get feedback. If it all seems to be going ok, continue, you got a long ride ahead, otherwise just give up, do a postmortem, and start over with new goals.
I always start with the user interface design. I figure out what the user should be able to do and what controls I will give them to do it. Once I get that laid out in a way I like it, then I start with the code "wiring".
Make a list of all the features that site have.
Make a list of nice to have features.
Make a list of the weakness of the site.
Order that list and prioritize the items that will be built first.
Identify what will be possible to do and what is not.
Meet with your customer and present these results.
Usually I do a mindmap of
problem I am trying to solve,
translated into exact requirements,
then mapping that to user workflows.
The cross linking features of mindmapping softwares make it lot easy. Since mindmapping is 'kind of freeform', I end up concentrating on the 'task' rather than 'representation' (e.g which type of UML diagram should I use to represent this) ?
Once initial ideas are clear then I can work on project plan, spec/design documents using UML for more low level details. This approach usually works well for me.
To see if it works for you or not, you can use FreeMind (opensource mindmaping software, good but currently limited functionality). Then You can try Mindmanager or iMindmap for mindmaping. Both integrate well with other Office products.
Usually I start out by grabbing my scratchbook and just start writing down what I want as in terms of features, this should be quite detailed. And can be quite messy with every thing scrambled together, if so, when you're done make an 'official version' of you're ideas on paper (REAL pen and paper works best for this in my opinion).
Then I start making some scetches of how the pages would look like, what information it must contain and translate that to a global database design. Then work that global design to a more advanced level where all pages come together, with relations between tables and stuff.
After that I build up the most important pages on a code framework (I always make use of a framework, if you don't then forget the framework part), and by 'most important pages' I mean in for example a blog that would be the posts. After that build the not-so-important pages, in case of a blog that could be an archive of posts.
If you have that done, put the code together with a design, or do that while coding if you do not seperate code from HTML/CSS/JS.
Oh and yes, do NOT expand your first idea along the way. Just write that down and implement that afterwards. So if, in case of the blog again, you think half way you want Youtube tags in you're BB-code, write it down. Add that later, offcourse before you're initial site releases.
That's my workflow, at least a basic basic, basic description of it.
Start with "paper prototypes", i. e. take a pencil and sketch each page very roughly. This lets you start from the user perspective, which I think is a good idea.
You can then use the sketches for a first hallway usability test and later as the basis for "wireframes" you would give a web designer to work from.
If you've gone through the complete site once, you probably have a good idea of what the backend should be able to do. You can now use your page sketches and compile a list of the actions a user can trigger by clicking on things. This is the raw material for designing the server-side API that the frontend can call.
Using the calls that need to be served, you can design the backend: What functionalities group nicely, what data needs to be fetched, what do you need to store between page calls (== Session variables) etc.
In this process, I have fared quite well by postponing technology decisions (frameworks, protocols etc.) and even class structure etc., until I've gone through the whole thing once in terms of "what things should do what to what other things" (I guess there's a better term).
I think I would start with an open-source SNS solution that comes close to what you need and then figure out how to add use-specific plug-ins, modules, and themes that achieve your purposes. There are a lot of em out there. Building from scratch is going to take a lot more effort and planning. Most SNS functionality is not worth re-inventing. Focus on what will make your site unique and build upward toward that.
I'm a fairly visual person when it comes to designing software so I sketch out dataflows, class hierarchies, UI and flow charts on whiteboards and paper first.
Butcher paper and colored pens can be particularly fun to use as it's 3 feet wide and comes in 100 foot rolls. When you've got a design that's satisfying or sufficiently complete, tear it off the roll and pin to the wall. Update as necessary.
That technique has worked for some large refactors as well as new projects.
You could start with something very simple and then add features a little at a time. You may reach a point where you want to start over, but the groundwork you did will be beneficial. Or you can try to do the whole thing at once, in which case you'll need the advice already given in the other replies.
One more idea: Specify those features you are not going to include, and other restrictions. These are called constraints, and are as important as the rest of the plan, as it gives you boundaries so you know when you're done planning!
If you work for the same company as this person, start by getting everything in writing so you aren't the one to take the fall when the inevitable happens...
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I come from a computer science. background, but I am now doing genomics.
My projects include a lot of bioinformatics typically involving: aligning sequences, comparing overlap, etc. between sequences and various genome-annotation-features, from different classes of biological samples, time-course data, microarray, high-throughput sequencing ("next-generation" sequencing, though it's the current generation actually) data, this kind of stuff.
The workflow with this kind of analyses is quite different from what I experienced during my computer science studies: no UML and thoughtfully designed objects shining with sublime elegance, no version management, no proper documentation (often no documentation at all), no software engineering at all.
Instead, what everyone does in this field is hacking out one Perl-script or AWK-one-liner after the other, usually for one-time usage.
I think the reason is that the input data and formats change so fast, the questions need to be answered so soon (deadlines!), that there seems to be no time for project organization.
One example to illustrate this: Let's say you want to write a raytracer. You would probably put a lot of effort into the software engineering first. Then program it, finally in some highly-optimized form. Because you would use the raytracer countless of times with different input data and would make changes to the source code over a duration of years to come. So good software engineering is paramount when coding a serious raytracer from scratch. But imagine you want to write a raytracer, where you already know that you will use it to raytrace one, single picture ever. And that picture is of a reflecting sphere over a checkered floor. In this case you would just hack it together somehow. Bioinformatics is like the latter case only.
You end up with whole directory trees with the same information in different formats until you have reached the one particular format necessary for the next step, and dozen of files with names like "tmp_SNP_cancer_34521_unique_IDs_not_Chimp.csv" where you don't have the slightest idea one day later why you created this file and what it exactly is.
For a while I was using MySQL which helped, but now the speed in which new data is generated and changes formats is such that it is not possible to do proper database design.
I am aware of one single publication which deals with these issues (Noble, W. S. (2009, July). A quick guide to organizing computational biology projects. PLoS Comput Biol 5 (7), e1000424+). The author sums the goal up quite nicely:
The core guiding principle is simple:
Someone unfamiliar with your project
should be able to look at your
computer files and understand in
detail what you did and why.
Well, that's what I want, too! But I am following the same practices as that author already, and I feel it is absolutely insufficient.
Documenting each and every command you issue in Bash, commenting it with why exactly you did it, etc., is just tedious and error-prone. The steps during the workflow are just too fine-grained. Even if you do it, it can be still an extremely tedious task to figure out what each file was for, and at which point a particular workflow was interrupted, and for what reason, and where you continued.
(I am not using the word "workflow" in the sense of Taverna; by workflow I just mean the steps, commands and programs you choose to execute to reach a particular goal).
How do you organize your bioinformatics projects?
I'm a software specialist embedded in a team of research scientists, though in the earth sciences, not the life sciences. A lot of what you write is familiar to me.
One thing to bear in mind is that much of what you have learned in your studies is about engineering software for continued use. As you have observed a lot of what research scientists do is about one-off use and the engineered approach is not suitable. If you want to implement some aspects of good software engineering you are going to have to pick your battles carefully.
Before you start fighting any battles, you are going to have to critically examine your own ideas to ensure that what you learned in school about general-purpose software engineering is valid for your current situation. Don't assume that it is.
In my case the first battle I picked was the implementation of source code control. It wasn't hard to find examples of all the things that go wrong when you don't have version control in place:
some users had dozens of directories each with different versions of the 'same' code, and only the haziest idea of what most of them did that was unique, or why they were there;
some users had lost useful modifications by overwriting them and not being able to remember what they had done;
it was easy to find situations where people were working on what should have been the same program but were in fact developing incompatibly in different directions;
etc etc etc
Once I had gathered the information -- and make sure you keep good notes about who said what and what it cost them -- it became relatively easy to paint a picture of a better world with source code control.
Next, well, next you have to choose your own next battle. But one of the seeds of doubt you have to sow in your scientist-colleagues minds is 'reproducibility'. Scientific experiments are not valid if they are not reproducible; if their experiments involve software (and they always do) then careful software engineering is essential for reproducibility. A lot of this is about data provenance, but that's a topic for another day.
Part of the issue here is the distinction between documentation for software vs documentation for publication.
For software development (and research plan) design, the important documentation is structural and intentional. Thus, modeling the data, reasons why you are doing something, etc. I strongly recommend using the skills you've learned in CS for documenting your research plan. Having a plan for what you want to do gives you a lot of freedom to multi-task while long analyses are running.
On the other hand, a lot of bioinformatics work is analysis. Here, you need to treat documentation like a lab notebook, and not necessarily a project plan. You want to be document what you did, maybe a brief comment why (e.g. when you are troubleshooting data), and what the outputs and results are.
What I do is fairly simple.
First, I start in a directory and create a git repo. Then, whenever I change some file, I commit it to the repo. As much as possible, I try to name data outputs in a way that I can drop then into my git ignore files.
Then, as much as possible, I work on a single terminal session for a project at a time, and when I hit a pause point (like when I've got a set of jobs sent up to the grid, I run 'history |cut -c 8-' and paste that into a lab notes file. I then edit the file to add comments for what I did, and remember, change the git add/commit lines to git checkout (I have a script that does this based on the commit messages). As long as I start it in the right directory, and my external data doesn't go away, this means that I can recreate the entire process later.
For any even slightly complex processing tasks, I write a script to do it, so that my notebook, as much as possible, looks clean. To an approximation, a helper script can be viewed as a subroutine in a larger project, and should be documented internally to at least that level.
Your question is about project management. Bad project management is not unique to bioinformatics. I find it hard to believe that the entire industry of bioinformatics is commited to bad software design.
About the presure... Again there are others in this world that have very challenging deadlines, and they are still using good software designs.
In many cases, following a good software design does not hold down the projects and may even speed its design and maintainance (at least on the long run).
Now to your real question... You can offer your manager to redesign small parts of the code that have no influence on the rest of the code as a proof of concept (POC), but it's really hard to stop a truck from keep on moving, so don't get upset if he feels "we worked this way for years - we know what we are doing, and we don't need a child to teach us how to do our work". Learn to work like the rest and when you will gain their trust, you could
do your thing once in a while (I hope you will have time and the devotion to do the right thing).
Good luck.
I'm writing an iPhone game and I am trying to write some requirements documents. I have never written requirements before so I got the book Software Requirements. I have not finished it yet, but I forsee some issues, as this book is targeted towards a business. My main question is I am the only person involved with this game and I feel the main purpose of the requirements document should be to nail out as many conceptual ideas of how the game works as I can before I am deep into design or construction. Does anyone have suggestions on how I should lay this out, should I still try to mimic the template provided in the book where it makes sense, or since I am both the sole developer and product owner, should I just stick to game concepts?
You're right that traditional SRS documents don't really fit games documentation all that well. Games instead have a general Game Design Document. It's usually created before any work on the game begins, and it's often edited as the development process goes to keep straight the intended end-result and specifics of the game.
While business software requirements documents are like contracts between a client and developer on what to produce, game design docs are more often specifications from the designer to the artists and programmers on what exactly they need to develop.
There is no specific layout to use. But you should consider who you're writing the document for. Is it for a class, for yourself, for peers after the project is done? The level of detail and the kind of things you include will be different depending on your audience. The format itself is very flexible, as long as it's coherent.
Brenda Brathwaite has a good blog entry on this subject which you might find helpful.
There is a semi-recent article from gamedev.net on the subject as well.
[Poor Jacob, you just read a book on the topic, and, collectively, the SO community writes another one for you, along with extra links, and probably with diverging views ;-) ]
Although I'm not familiar with the book you mention in the question, I think that the following suggestion may help you both take seriously, but also relax a bit, about the all too important question of requirements.
Being a "team of one", it is particularly important, and somewhat paradoxical, that you go through the effort of formalizing the requirements. However, rather than putting too much emphasis on the form, you may find an Agile approach to developement (and hence to requirements gathering) more appropriate. With regards to requirements, one of the main advantages of this approach, is flexibility, i.e. the understanding that while they should be formalized (with limited time/effort), requirements should be allowed to change (within limits) as part of an iterative process towards production of the target product.
In very broad terms, this generally go as follows:
write "user stories", these are individual "cards" (yes, physical cards, say 4 inches by 5 inches, are good, for you can then move then around, sort them etc.)
each story tells a particular feature of the application, here the game, from the end-user's perspective. You can/should start all cards with "As a user, I need the game to..." then follow with a particular feature, for example "... show my high score on the same page as the global high-scores are kept [because ... here optional reasons for why user may want this feature].
review each story and assign a rough estimation of the time involved in implementing it
review each story and assign a priority level (scale may vary, but something simple like "Must have from Version 1.0", "Should eventually be in there, for sure", "Would be nice to have" and "Maybe nice to have...")
organize releases, on the basis of what you can do within say 2 or 3 weeks, maximum. If a particular feature were to take too long, schedule it for a later release.
implement the features assigned to the current release
iterate through this release cycle, reviewing the requirements as you go, for the relative importance of features, and also the need of new features may become evident as with the insight provided by using the [incomplete/imperfect] intermediate releases.
Books like the one you describe are focused at a different audience, but there is value in the general concepts presented. Fully developed requirements documents are not as common as you might think. Don't let anyone think that you are a 'bad developer' for not having the most detailed requirements.
Requirements docs might be more important if you need to communicate the requirements with a co-developer.
If you are the sole developer I would strongly recommend that you spend your efforts on the design and implementation of the game, over requirements. If you have a good idea of what you build then let this flow as you build it.
Documentation can help you. The question is what is going to be most beneficial. Maybe design decisions are more critical than requirements for you but not for others. You'll maybe want to have a list of things that people have requested or ideas that you think of but cannot implement straight away. Sometimes a whiteboard can be handy for sketching out things, it's not just a tool for collaboration with other people.
Here's just a general approach...
Solidify the concept...write it in plain English first (ex: The game is a first person shooter. You kill zombies and hunt for treasure.)
Get a paper pad and pencil and draw out the general flow of the game and the main screens the users will encounter...main menu, options screen, help, etc. Make sure it makes sense.
Go to a site like mockingbird and create the detail wireframes for your screens...
Print these out and do some paper prototyping...i.e. put the printout in front of you and 'click' on a button...then bring up the appropriate screen...then click on another button, etc.
Once that makes sense, you can try to start coding your game.
Personally I believe you should use your own way to do this. The most commonly available one's will not match with your requirement. They might be suitable for a common commercial server application but not for a game. And since iPhone gaming is a new trend you may have to look in a different perspective.. You may not be able to fill a document with standard requirements and you may have different set of New type of requirements.
Just a suggestion... Sign up with Google Sites, and create a private site with documentation of the game, requirements, technical aspects, work log, etc... You can share it with select people, and it always keeps edit history.
I like it better than a Wiki because it is more structured, and just plain simple to use.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I was asked (at a place i just began working) to create simple specs for some new functionality that is going to be added to an existing Registration system. I need a little help since i've never done this before.
Here are two diagrams that show the current workflow and the new workflow.
Current Workflow: http://img80.imageshack.us/img80/102/currentworkflow.png
New Workflow http://img245.imageshack.us/img245/6748/newworkflow.png
I know they might be a bit vague but here's what's basically happening.
We are adding a new import form to an existing windows application.
We are modifying an existing form by adding a search button which will search
search and populate data read by an ocr.
I'm a new developer and i'm pretty bad at writing documents in general, but i would like to improve on this. Maybe some examples on how to go about writing something like this would be helpful. I've googled for some examples, but most of the ones i've found are on creating a brand new system. I need something that shows how to write one for modifying an exisiting system.
Here's my attempt at a specification. Maybe someone can critique it. At least then i will know what i need to improve. http://cid-ddb3f6a92ec2b97e.skydrive.live.com/self.aspx/.Public/Specs.docx
Thanks
I love writing specs (I'm a rare one in my company).
Diagrams are a good way to go, but for the more literally minded I start with a full specification template that has a ton of headings in it. For a new system, you'd generally have something to say for everyone. In your case you've specifically mentioned it's an existing app you're modifying, but the point is not to fill out all of the headings - the point is to think about them, and then delete them after due consideration. For example:
Business Requirements (short synopsis of the need, as explained to the business, non-technical users)
Use Cases (usually for bigger specs only)
Functional Requirements
Overview
Flowcharts etc.
Configuration
Error Reporting
Testing
Documentation
Training
Assumptions and Additional Constraints
Third-party Software Requirements
Internationalization
Expandability (e.g. for bits that might need to plug-in to others etc)
Customization
Questions (for questions that still need to be answered by someone to finish the spec)
Also if it's really technical then you might need an introduction sections for:
- Target Audience
- Terminology
- Examples
All of these is generally overkill for all but the largest of designs. But even for a modification, I'd go through every item and consider whether I need to write anything or not. I think this is where a lot of the value of writing a spec comes from - the process of creation. In other words, trying to be thorough and not miss too much. All the benefits that come afterwards - like being able to do estimates, being able to explain the functionality to others etc - are nice side-effects. As long as it doesn't end up completely garbled, and suits your company okay, I think that's more important than the specific appearance, format or contents of the spec.
EDIT: Comments on your specification
I think you've done a reasonable job here. Most developers should be able to take the spec and produce something sensible, and most business analysts should be able to look at the spec and work out what it does and how it works. In my comments below, keep in mind that there's always a trade-off between how detailed you want the spec and how much time you have. I tend to believe the more detailed the spec the more time everyone saves, but that's not the case for everyone.
If you want this to be clearly understood by a business user (e.g. the customer), then the Objective section could maybe contain a sentence or two describing the problem it solves. In other words not what it will achieve, but why.
It's worth explicitly naming the intermediate staging table here. At the very least it means if someone comes back to the spec a year from now, they know exactly where to look in the database.
Minor point: in my experience screenshots that contain unrealistic data are harder to understand. Instead of showing "My Sample Form", "Name", "Address" etc, it'd perhaps be easier to understand with some sensible data. Can still be fake to protect the customer's data, like "123 Fake St" etc. Not a huge deal though.
It's not clear what will happen when something goes wrong. Are there to be any checks that the data in the staging table is in a valid format? If not, is the user given an error message, or otherwise logged somewhere? One error per row of invalid data, or one for the whole batch? The form consists of a single button - something I think we can agree isn't the world's greatest UI, but I understand sometimes these things happen - perhaps it could be enhanced with a logging window to show the results of the import. The answers might be straightforward, but the developer needs to know what they are.
Perhaps not an issue depending on how much data there is, but if there was a lot and it will take a while, it might be worth having a progress bar. Or, mention if the data will be imported in stages.
Would it be worth mentioning the definition of the permanent table to which data is moved? Are all fields moved from the staging table to the permanent table, or only some? If only some, can you show what maps to what? If the permanent table has different data lengths - for example if Address Street is a Varchar(30) - what would happen if the data won't fit? Again, perhaps simple answers, but ones that would be very usefully answered here.
Perhaps worth mentioning if the data will be imported in a single transaction or not - if the data import fails partway through, if everything rolled back, or is half the imported data left imported?
If another developer will be doing this work, I think they're far more likely to get the work right if you mocked-up / draw the screens for them. Even if it's just a form with one button, and even though I can take a good guess at what your search pop-up form will look like, I would make no mistakes if I knew exactly what it's supposed to look like. Tools like Balsamiq Mockups (and see examples here) are wonderful for quick mocks, though the default "comic sans" look may not ride well with managers. I'd rather have a dirty mockup than none at all though. (Note: the free version of Balsamiq doesn't let you save images, but you can achieve the same with the export/import functionality. Also you can't save to an image file like PNG etc, but you can use a screen-capture program to take a picture of what you draw.)
Minor point: I try to avoid personal pronouns like "I", "we", "our", just to make it a little more professional and better for customers to read if necessary. I only noticed one "our", so you've mostly got it right in terms of tone here.
Minor point: are varchars enough or will there be non-standard characters in there that require unicode (i.e. nvarchar)?
It's less clear to me what's happening in the Voter Add/Update Form, but I don't have knowledge of your application - maybe everyone involved will say "oh right, I get it". For example I don't understand the relevance of "ImpRecord001" and "ImpRecord002" - would it be worth mentioning in the design what these batch codes actually mean in the real world?
Is the "Search Data" button the same as the "Search OCR" button?
For any document: first consider why you are writing it - who will read it, what do they need to know? How much detail is appropriate? Another couple of general ideas
If may be useful to then think about the sources of information that go into what you are writing. One result of that might be that you make sure that what you write can be verified. If for example an information source is a person, especially for IT docs it might be a non-IT person telling you stuff, then you may be quite careful about how your present some information so that the "source" can also understand what you are saying.
Also consider carefully what comes after the current document. For example might a test plan be written on the basis of what you write? This might lead you to present information in tables that quite naturally get expanded to test cases.
So to your specific question. What do you mean by "spec"? The workflow you give isn't enough for a user to look at and agree "Yes please, that's what I want". It's not enough for someone to write some code. I'm thinking you might need several documents.
1). Some kind of requirements doc. One format you might use is a storyboard. This focuses on what the user can see and do. Exactly what data is shown on each screen. If there are computations underlying what's displayed you may need to have appendices describing these. This doc is read by both users and developers. Powerpoint or Word could be used.
2). From that you might derive some explicit data models. Item-by-item, field-by-field. data types, sizes, validation etc. I might use date modelling tool, or UML or just a spreadsheet. Primary audience is developer, but ideally a user (or a business anlyst intermediary) could verify the details. [If you don't have a business analyst, you probably are the business analyst :-) ]
3). More technical, a spec for the developers referring to items 1&2. A decomposition of the implemntation. Names of modules, packages, classes or whatever you are using. Defintions of transformations, algorithms and calculations. A more technical doc. I would use UML, but any precise form of capture would do. This is where we might really drill down into what some of the detailed boxes in your workflow mean.
As has been observed, in general we also need to make sure the developer udnerstands the non-functional requirements, such as security and data volumes. In your situation this may be be implcitly understood, so possibly you may not need it now ... in some other life you may, so it might be a good idea to at least have a one liner in place to remind you for the future.
Those are an excellent start for a spec.
I would add to them by creating mock screen shots of what you want the windows application to look like.
On top of that you can add the details of each data field, and what the allowed values are.
Include details of any exceptions you can think of, and how you want errors reported.
You might also want to consider what sort of reporting, and security/auditing you need, as these will need to be included in the design.
Finally, it's worthwhile to sit down with the developer and talk them through the process, going through each step, as i'm sure further details will be needed.
Some of the steps down at the bottom are a bit wordy. Try splitting them up and make sure the word IF never appears. IFs should be designated by using a diamond and splitting out the flow paths based on the conditional.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 10 months ago.
Improve this question
I've never written functional specs, I prefer to jump into the code and design things as I go. So far its worked fine, but for a recent personal project I'm writing out some specs which describe all the features of the product, and how it should 'work' without going into details of how it will be implemented, and I'm finding it very valuable.
What are your thoughts, do you write specs or do you just start coding and plan as you go, and which practice is better?
If you're driving from your home to the nearest grocery store, you probably don't need a map. But...
If you're driving to a place you've never been before in another state, you probably do.
If you're driving around at random for the fun of driving, you probably don't need a map. But...
If you're trying to get somewhere in the most effective fashion (minimize distance, minimize time, make three specific stops along the way, etc.) you probably do.
If you're driving by yourself and can take as long as you like, stopping any time you see something interesting or to reconsider your destination or route, you may not need a map. But...
If you're driving as part of a convoy, and all need to make food and overnight lodging stops together, and need to arrive together, you probably do.
If you think I'm not talking about programming, you probably don't need a functional spec, story cards, narrative, CRCs, etc. But...
If you think I am, you might want to consider at least one of the above.
;-)
For someone who "jumps into the code" and "design[s] as they go", I would say writing anything including a functional spec is better than your current methods. A great deal of time and effort can be saved if you take the time to think it through and design it before you even start.
Requirements help define what you need to make.
Design helps define what you are planning on making.
User Documentation defines what you did make.
You'll find that most places will have some variation of these three documents. The functional spec can be lumped into the design document.
I'd recommend reading Rapid Development if you're not convinced. You truely can get work done faster if you take more time to plan and design.
Jumping "straight to code" for large software projects would almost surely lead to failure (as immediatley starting posing bricks to build a bridge would).
The guys at 37 Signals would say that is better to write a short document on paper than writing a complex spec. I'd say that this could be true for mocking up quickly new websites (where the design and the idea could lead better than a rigid schema), but not always acceptable in other real life situations.
Just think of the (legal, even) importance a spec document signed by your customer can have.
The morale probably is: be flexible, and plan with functional or technical specs as much as you need, according to your project's scenario.
For one-off hacks and small utilities, don't bother.
But if you're writing a serious, large application, and have demanding customers and has to run for a long time, it's a MUST. Read Joel's great articles on the subject - they're a good start.
I do it both ways, but I've learned something from Test Driven Development...
If you go into coding with a roadmap you will get to the end of the trip a helluva lot faster than you will if you just start walking down the road without having any idea of how it is going to fork in the middle.
You don't have to write down every detail of what every function is going to do, but define you basics so that way you know what you should get done to make everything work well together.
All that being said, I needed to write a series of exception handlers yesterday and I just dove right in without trying to architect it out at all. Maybe I should reread my own advice ;)
What a lot of people don't want to admit or realize is that software development is an engineering discipline. A lot can be learned as to how they approach things. Mapping out what your going to do in an application isn't necessarily vital on small projects as it is normally easier to quickly go back and fix your mistakes. You don't see how much time is wasted compared to writing down what the system is going to do first.
In reality in large projects its almost necessary to have road map of how the system works and what it does. Call it a Functional Spec if you will, but normally you have to have something that can show you why step b follows step a. We all think we can think it up on the fly (I am definitely guilty of this too), but in reality it causes us problems. Think back and ask yourself how many times you encountered something and said to yourself "Man I wish I would have thought of that earlier?" Or someone else see's what you've done, and showed you that you could have take 3 steps to accomplish a task where you took 10.
Putting it down on paper really forces you to think about what your going to do. Once it's on paper it's not a nebulous thought anymore and then you can look at it and evaluate if what you were thinking really makes sense. Changing a one page document is easier than changing 5000 lines of code.
If you are working in an XP (or similar) environment, you'll use stories to guide development along with lots of unit and hallway useability testing (I've drunk the Kool-Aid, I guess).
However, there is one area where a spec is absolutely required: when coordinating with an external team. I had a project with a large insurance company where we needed to have an agreement on certain program behaviors, some aspects of database design and a number of file layouts. Without the spec, I was wide open to a creative interpretation of what we had promised. These were good people - I trusted them and liked working with them. But still, without that spec it would have been a death march. With the spec, I could always point out where they had deviated from the agreed-to layout or where they were asking for additional custom work ($$!). If working with a semi-antagonistic relationship, the spec can save you from even worse: a lawsuit.
Oh yes, and I agree with Kieveli: "jumping right to code" is almost never a good idea.
I would say it totally "depends" on the type of problem. I tend to ask myself am I writing it for the sake of it or for the layers above you. I also had debated this and my personal experience says, you should since it keeps the project on track with the expectations (rather than going off course).
I like to decompose any non trivial problems loosely on paper first, rather than jumping in to code, for a number of reasons;
The stuff i write on paper doesn't have to compile or make any sense to a computer
I can work at arbitrary levels of abstraction on paper
I can add pictures and diagrams really easily
I can think through and debug a concept very quickly
If the problem I'm dealing with is likely to involve either a significant amount of time, or a number of other people, I'll write it up as an outline functional spec. If I'm being paid by someone else to develop the software, and there is any potential for ambiguity, I will add enough extra detail to remove this ambiguity. I also like to use this documentation as a starting point for developing automated test cases, once the software has been written.
Put another way, I write enough of a functional specification to properly understand the software I am writing myself, and to resolve any possibile ambiguities for anyone else involved.
I rarely feel the need for a functional spec. OTOH I always have the user responsible for the feature a phone call away, so I can always query them for functional requirements as I go.
To me a functional spec is more of a political tool than technical. I guess once you have a spec you can always blame the spec if you later discover problems with the implementation. But who to blame is really of no interest to me, the problem will still be there even if you find a scapegoat, better then to revisit the implementation and try to do it right.
It's virtually impossible to write a good spec, because you really don't know enough of either the problem or the tools or future changes in the environment to do it right.
Thus I think it's much more important to adapt an agile approach to development and dedicate enough resources and time to revisit and refactor as you go.
It's important not to write them: There's Nothing Functional about a Functional Spec
I'm at a point in my freelance career where I've developed several web applications for small to medium sized businesses that support things such as project management, booking/reservations, and email management.
I like the work but find that eventually my applications get to a point where the overhear for maintenance is very high. I look back at code I wrote 6 months ago and find I have to spend a while just relearning how I originally coded it before I can make a fix or feature additions. I do try to practice using frameworks (I've used Zend Framework before, and am considering Django for my next project)
What techniques or strategies do you use to plan out an application that is capable of handling a lot of users without breaking and still keeping the code clean enough to maintain easily?
If anyone has any books or articles they could recommend, that would be greatly appreciated as well.
Although there are certainly good articles on that topic, none of them is a substitute of real-world experience.
Maintainability is nothing you can plan straight ahead, except on very small projects. It is something you need to take care of during the whole project. In fact, creating loads of classes and infrastructure code in advance can produce code which is even harder to understand than naive spaghetti code.
So my advise is to clean up your existing projects, by continuously refactoring them. Look at the parts which were a pain to change, and strive for simpler solutions that are easier to understand and to adjust. If the code is even too bad for that, consider rewriting it from scratch.
Don't start new projects and expect them to succeed, just because your read some more articles or used a new framework. Instead, identify the failures of your existing projects and fix their specific problems. Whenever you need to change your code, ask yourself how to restructure it to support similar changes in the future. This is what you need to do anyway, because there will be similar changes in the future.
By doing those refactorings you'll stumble across various specific questions you can ask and read articles about. That way you'll learn more than by just asking general questions and reading general articles about maintenance and frameworks.
Start cleaning up your code today. Don't defer it to your future projects.
(The same is true for documentation. Everyone's first docs were very bad. After several months they turn out to be too verbose and filled with unimportant stuff. So complement the documentation with solutions to the problems you really had, because chances are good that next year you'll be confronted with a similar problem. Those experiences will improve your writing style more than any "how to write good" style guide.)
I'd honestly recommend looking at Martin Fowlers Patterns of Enterprise Application Architecture. It discusses a lot of ways to make your application more organized and maintainable. In addition, I would recommend using unit testing to give you better comprehension of your code. Kent Beck's book on Test Driven Development is a great resource for learning how to address change to your code through unit tests.
To improve the maintainability you could:
If you are the sole developer then adopt a coding style and stick to it. That will give you confidence later when navigating through your own code about things you could have possibly done and the things that you absolutely wouldn't. Being confident where to look and what to look for and what not to look for will save you a lot of time.
Always take time to bring documentation up to date. Include the task into development plan; include that time into the plan as part any of change or new feature.
Keep documentation balanced: some high level diagrams, meaningful comments. Best comments tell that cannot be read from the code itself. Like business reasons or "whys" behind certain chunks of code.
Include into the plan the effort to keep code structure, folder names, namespaces, object, variable and routine names up to date and reflective of what they actually do. This will go a long way in improving maintainability. Always call a spade "spade". Avoid large chunks of code, structure it by means available within your language of choice, give chunks meaningful names.
Low coupling and high coherency. Make sure you up to date with techniques of achieving these: design by contract, dependency injection, aspects, design patterns etc.
From task management point of view you should estimate more time and charge higher rate for non-continuous pieces of work. Do not hesitate to make customer aware that you need extra time to do small non-continuous changes spread over time as opposed to bigger continuous projects and ongoing maintenance since the administration and analysis overhead is greater (you need to manage and analyse each change including impact on the existing system separately). One benefit your customer is going to get is greater life expectancy of the system. The other is accurate documentation that will preserve their option to seek someone else's help should they decide to do so. Both protect customer investment and are strong selling points.
Use source control if you don't do that already
Keep a detailed log of everything done for the customer plus any important communication (a simple computer or paper based CMS). Refresh your memory before each assignment.
Keep a log of issues left open, ideas, suggestions per customer; again refresh your memory before beginning an assignment.
Plan ahead how the post-implementation support is going to be conducted, discuss with the customer. Make your systems are easy to maintain. Plan for parameterisation, monitoring tools, in-build sanity checks. Sell post-implementation support to customer as part of the initial contract.
Expand by hiring, even if you need someone just to provide that post-implementation support, do the admin bits.
Recommended reading:
"Code Complete" by Steve Mcconnell
Anything on design patterns are included into the list of recommended reading.
The most important advice I can give having helped grow an old web application into an extremely high available, high demand web application is to encapsulate everything. - in particular
Use good MVC principles and frameworks to separate your view layer from your business logic and data model.
Use a robust persistance layer to not couple your business logic to your data model
Plan for statelessness and asynchronous behaviour.
Here is an excellent article on how eBay tackles these problems
http://www.infoq.com/articles/ebay-scalability-best-practices
Use a framework / MVC system. The more organised and centralized your code is the better.
Try using Memcache. PHP has a built in extension for it, it takes about ten minutes to set up and another twenty to put in your application. You can cache whatever you want to it - I cache all my database records in it - for every application. It does wanders.
I would recommend using a source control system such as Subversion if you aren't already.
You should consider maybe using SharePoint. It's an environment that is already designed to do all you have mentioned, and has many other features you maybe haven't thought about (but maybe you will need in the future :-) )
Here's some information from the official site.
There are 2 different SharePoint environments you can use: Windows Sharepoint Services (WSS) or Microsoft Office Sharepoint Server (MOSS). WSS is free and ships with Windows Server 2003, while MOSS isn't free, but has much more features and covers almost all you enterprise's needs.