Yeah I know there are couple of questions which are related to syncing with iPhone and Web DB but none of them helped me.
I also did a lot googling but I rarely found informations about two-way-syncing. Maybe I just used the wrong keywords.
I'm building an App right now and I came up with the idea to add a two-sync to my App and my Web service.
My first thought were that it would be ridiculously easy but it turns out to be not that easy at all.
I found couple of problems and some solutions to my problems but I would like to hear from you guys if these soultions would create other problems or if these solutions are good or bad.
The idea of my App is helping me sync my notes which I will take on the go with my iPhone and at work or at home with a Web App.
Those two ends should always sync'd cause I don't know on any time which device (iPhone or Computer) I will use to take, edit or just read my notes.
What I have on both sides:
For my Web service (and web app) I will use rails and I think mysql on the DB side.
On the iPhone I will use a SQLite DB with a Objective-C wrapper (FMDB).
Both will exchange data via JSON (using a JSON framwork on iPhone side).
My ideas so far:
Primary key has to be unique on both sides
As a primary key I will use a UUID. I think it's a unique solution on both sides and it won't make any duplicates (at least I hope).
Revisions for changes of data
Each change will be saved as revision with a SHA1 key, which I will create from date + note data.
The revision object is also including information like:
date
which note object belongs to this revision
changes are made on which device?
what chaged? (atually I'm not sure about including this information)
My "solution" so far is I will track every modification (create, update, delete) on a histroy-table with revisions on both sides.
On the iPhone side I will first update my history-table from Web DB and then commit my changes to the Web DB.
This should work, right?
That sounds not that bad to me but my question here is how can I handle conflicts? I don't want to bother the user with messages how to handle the conflicts.
Roundup of my questions:
Is my "solution" good or bad? What should I change to make it better?
How can I handle change conflicts so the user don't notice them?
Do you have any resources I could read about two-way-sycing?
EDIT:
Thank you all for your answers. I know now that I'm not alone with this "problem" and there is no simple and all fitting solution for all Apps. I assume that I'm doing good with my ideas or solutions so far and I will try to come up with syncing rules.
My idea so far is: I will develop it simple as possible and will use it for my own needs. Solve problems I discovered while using and syncing. After that I will invite my friends to test and solve problems they have.
I think this way I can came up with real world rules for syncing my data with Web cause I see what people are actually doing and where problems are.
What you think?
"It depends."
Everyone loves that line in their answers.
Two way sync boils down fundamentally to conflict resolution. And only you as the application designer can come up with the rules for conflict resolution.
Without conflict, syncing is easy.
One way syncing is "easy" because it's just like two way sync, save that the rules for conflict always favor one party. "Make this look like that." Simple rule.
Fine grained two way syncing isn't that hard, you just need to record the specific changes that are made and when they are done, then when you sync, you take that log of changes from each party, combine them in to a single log, and then apply that log to each party starting with the last time they were in sync.
By specific changes I don't mean "record changed", as it's too coarse. Rather you want to know that "lastName" of record changed. It changed at 01/01/2011 12:23:45.
When party A says lastName changed to "Johnson" at 01/01/2011 12:22:45 and party B says lastName changed to "Smith" at 01/01/2011 12:22:46, then "Smith" is the right answer, since it's the latest.
But wait, did you see what happened there? I just pulled a rule out of thin air. "Latest wins". Maybe that doesn't work for you, maybe you have different rules. "It Depends".
So, really, it all comes down to the rules. You can make it as fine grained as you want. There will ALWAYS be conflicts. That's what the rules are for.
So you need to decide what those are for you application.
actually i consider that the only problem in any kind of two way syncing only happens when there are conflicts. Really. Take for example any version control system (svn, cvs, git, etc.). They solve this conflict more granularly because they split the file itself, and they are checking for line conflicts, so changes in two different parts of the file are not treated as conflicts.
However i suppose this solution would not be really feasible because it's a pain to implement it :) ...
If you decide to handle the conflicts at the level of notes, and not it's lines, then probably at the end of the day you need to come up with some business rule that defines what happens when there are changes that result in conflict.
Possibilities:
Use the last change. Override the older. This is easy.
A solution what Dropbox uses i've seen it a couple of times when we were changing the same document on multiple machines is, that it creates multiple files appending a suffix to let users know about the changes on the multiple machines. Something like this you could do easily with notes as well.
I'm not sure i've helped though ...
Moszi
Related
I am working with a client who has an ERP system in place, called M1, that they are looking to make custom changes to.
I have spent a little bit of time investigating the ERP system in terms of making customizations. Here is a list of what I have found with regards to custom changes:
Custom changes cannot be exported/imported. There is an option in the M1 Design Studio, however, they always appear to be disabled... I tried everything and I couldn't find a mention of it in the help documentation.
You can export a customizations change log (CSV, XML, Excel, HTML) that provides type, name, location and description. In essence, it is a read-only document that provides a list of changes you made. You cannot modify the contents of this log.
Custom form changes made, go into effect for all data sources (Test, Stage, LIVE). In other words, there does not appear an ability to limit the scope of a form change.
Custom field changes must be made in each data source (Test, Stage, LIVE). What's odd here is that if add a field in Test, adjust a grid to display it, subsequently change to LIVE, it detects that the field doesn't exist and negates the grid changes.
I'm unable to find documentation indicating that this application supports version control.
sigh
....
So...
How do I manage changes from an SDLC: ALM methodology and tools standpoint?
I could start by bringing in a change request system to manage pending and completed customizations. But then what? How should changes me managed and released? Put backups of application under source control and deploy when needed?
There might not be a good answer to this question since I'm unable to take advantage of version control and create a separation of environments, but I figured I'd ask in case anybody has had similar experience or worked with M1.
I take it from the lack of answers in two months that your question is unanswerable. SDLC is something you could write a textbook on, or read a textbook on, and not know enough about your environment, other than that probably in order to get hired at your shop, "SDLC" would be a bullet point on the hiring qualifications.
I have no experience with M1, but I am assuming that you're going to have to ask your peers at work for their ideas, because it sounds like you're asking a vertically closed (your shop, your tools, your practices) question that has no exact technical answer.
As for best practices; I suggest you investigate best practices outside your M1 ERP silo and apply them as makes sense to you.
The company I work for also uses M1 erp. We have similar issues regarding version control of the customisations. From what I can tell, all customisations are stored in the M1DD database. You could backup a copy of this database before any major development work as a basic revision control system.
I am familiar with the issue of all changes becoming immediately active in all datasets. This is particularly annoying when you are making changes to a commonly used modules as you don't know how live data will be affected during the development process. One technique I have found useful is to surround untested code with an if statement so it is only executed when I am logged in.
If App.UserID = "MYUSERNAME" Then
'new code here
End If
I would be interested in hearing how you solved this problem.
My company has 3 developers. Me, another guy, and a VP dev. I really want to implement source control, especially since our code seems to randomly change on it's own. We tend to develop on the server, live, etc.
I'm fine with having a copy of our database on my machine to work against, if necessary, as is the other guy. The VP dev doesn't want it. How can I work with him to change his mind, or make it work for him?
You have to make him think it's his idea.
Point out that with source control you not only have a built-in backup of everything, but you also have the previous versions - let him realize how much of a good thing that is.
Install SVN and tell the one that opposes it that "everybody does it" :)
And seriously - source control is a MUST even for a single developer, let alone for three.
As for the DB server - you can use one development server (it can be a regular machine). It is of course no problem if you use each a local copy, but you must have a strong database schema generation/synchronization tools.
You should have source control. There isn't much excuse for not having it. Source control will protect you against changes that will cause problems in your code. I would recommend putting the db schema and data (sample set) in your version control. This will allow independent changes to the db without screwing up what your users see live on the site.
Note that you're not really asking about source control here, but about where your development dataset resides. Local databases per-developer are best, if possible, but failing that, a reasonable alternative is to just have a virtual machine containing your source control server and a development database.
Putting things under source control is really easy - literally, 10 minutes from now you could have your source under source control. Rather than try and persuade him the benefits I would just go ahead and do it anyway.
Start simply by putting a copy of your source under source control - even if he doesn't use it just merge the changes from live into your source control repository on a regular basis. At least that way you have a revision history (and if you are him are the only people changing the source, it means that any changes you didnt make, he must have made)
With luck, slowly over time he will begin to see the benefits (him: oh no - everything just broke! you: Don't worry, I'll just look and see what has changed since the last working copy...)
It sounds like you need to convince him that it is
Necessary to solve a problem,
an appropriate solution (does exactly what you need) and
easy to use.
It sounds like you have the information to demonstrate #1: the last time the code, "changed on its own," on the server and you lost someone's work or mixed results poorly. Bam, there's your "problem." #3 is the next more difficult: you need to pick an SCM with a good set of tools and do a demo. The TortoiseX line of products (TortoiseHg, TortoiseSVN) are great for this, because they make it non-scary.
Item 2 is the hardest: to demonstrate that this is the appropriate solution. Perhaps, to convince him of this, you might refer to anecdotes of other programmers or by looking at Github, where you can look back at previous versions of a product. I'm clutching at straws, here, because I feel like his argument will be, "Ach, and that's when it's a huge headache, is when things break. It won't be worth it."
Obviously there are a large number of ways to deal with people (and for the most part you have a "people") problem.
The first thing I'd do is find out why he's so against source control. Often times people who don't like source control either don't like:
The extra work of committing
Don't always work next to an internet
See no extra value in it
There are different solutions to each of these problems. Obviously the third one is tricky, so I'll handle it last.
If they don't like the extra work of committing, some chron scripts will help them (or windows scheduler). Something that regularly commits in the background, or recursively goes through his files and adds them for the next commit. This will mean you'll do a little more work on your end to clean up extra files and deal with broken builds, but its a step. Alternatively if he's emailing you the code, a script that commits the emails works as well.
If he's not always working next to internet access, consider a system like GIT. The advantage of GIT (over something like SVN) is that it utilizes a pull model instead of a push. As a result you pull updates from other GIT users instead of pushing commits. If you are working on a plane and don't have internet access, this is a valuable feature.
Finally, demonstrating the importance of the system is tough. The best example is almost always: "My machine burned down." I suppose you could nuke his box, but for the moment let's look at ways that don't piss off your boss.
A good way to demonstrate the importance of a repository is a Daily Build. Having a daily build means you can readily integrate features and find bugs faster. Setting up a repository with a daily build will significantly improve your work conditions, and its likely to make a good impression.
These are just a few of the reasons that people don't like source control, but the key idea is finding what his reason is and adapting to it.
I recently joined the IT department of a big insurance company. Although the department's title is "IT", a lot of code gets written here; Java, JSP, JavaScript, COBOL and even some C++ from what I've heard. All the programs that allow insurers, brokers and the rest of the tie-wearing, white-collar workers to issue new contracts and deal with clients runs on the code produced by this department. I've been told that the department is pretty good by the parent company's standards and that we've even received an internal award or two. We're 17 people in the department, split in smaller groups of 2 or 3. As you might've guessed from the COBOL part further up, the average age is over 40 years (as a point of reference I'm 29 yo).
Right now, there is no version control system in place (there exists a general backup scheme though). When needed, files are passed around through shared folders. Usually there's one person in every group responsible for copying the "final" version of the files back to the production server. I find this absurd and even a bit dangerous.
How may I try to convince management that we should implement a VCS scheme in our department? I've never deployed a VCS myself but every other place I've worked at had one. I think I'll hit a "we've been OK until now, why bother" wall from the first step, coupled with the age of most of my co-workers that will feel this step is an unnecessary hurdle.
I know the basic advantages of VCS (traceability, granular backups, accountability etc). I'm looking to back my case with realistic cases and examples of real added value over the implementation costs, not just a "but-but-but, we must have a VCS you fools!" :-)
You don't necessarily need their permission.
Install svn on your machine, start using it, and then start convincing your fellow team members to use it too.
Then watch and see what happens.
Edits
The basic idea of this is that it's easier to show than to tell.
It's a great idea to support your ideas with a working implementation/solution.
Of course, if you succeed, and they want the system used department/company wide, you must be prepared to support the transition, know how the software is to be installed and used.
Going ahead and using something accepted in the industry is faster than having discussions on what system should be used.
There is a good change that this will get you noticed. You may also get your peers respect and support.
As suggested, the same approach can be made on other areas:
issue/bug tracking systems
quality tools
time tracking
continuous integration
a wiki for knowledge base, HOWTO's, guidelines, tutorials, presentations, screencasts
different IDEs and tools
build tools
automated deployment
various scripts that would save your team time
.. any item that will visibly add quality to your work, but doesn't (yet) disrupt existing methodologies and practices.
Joel Spolsky has an excellent article: Getting Things Done When You're Only a Grunt
Quote
Nobody on your team wants to use
source control? Create your own CVS
repository, on your own hard drive if
necessary. Even without cooperation,
you can check your code in
independently from everybody else's.
Then when they have problems that
source control can solve (someone
accidentally types rm * ~ instead of
rm *~), they'll come to you for help.
Eventually, people will realize that
they can have their own checkouts,
too.
Management? I will put bold the expressions and words you should use:
Your should display some examples how a VCS will prevent losing money to the company if some error/bugs or disaster happens. It will be faster to solve all problems, so maintaning the systens won't be so lazy and people become more productive.
You should also mention that implementing a VCS has no costs.
VCS will also give advantages for backup all the existing code. So, all the code will be safe.
My opinion on how to go about doing this, is that you should try to convince your fellow developers first. The way I see it, there are two ways this might go about:
You give the right arguments to the other developers (possibly only the head developers will suffice), they like the idea, and the suggest it to management. Management is easy to convince at that point, so everyone is happy.
You give the right arguments to management, who get all excited (great!) and mandate that version control has to be installed and used by everyone. Here's the thing: If at this point the other developers are not sold to the idea already, then (a) they might be hostile to an idea that management is forcing upon them, and (b) they might not like you for being the cause of it all.
So what are good arguments to convince fellow developers? As someone who uses subversion (which is the one I recommend in this case, by the way) even for his solo projects, here's a few advantages I can think of:
Using version control forces people to think of code modification in terms of a series of small, self-contained changes. This is an extremely beneficial way of working: where otherwise people would be inclined to make lots of changes all over the place, leaving the code in a mess, version control kinda forces them to change the code in bite-size, easy-to-swallow bits, keeping the code compiling at all times, easing the cost of integration with other modules, etc.
Version control makes it very easy to see what has changed in the code each time. This might sound trivial, but when you start modifying code it's easy to lose track after a while. But with VC it's all an "svn diff" (or equivalent) away, always.
Version control makes it very easy to see who has changed the code each time. So that, for example, when something breaks, you know who to blame. (It's not an accident that the subversion command which shows who last changed each line is called "svn blame".)
Version control makes it very easy to see why a piece of code was changed. Commit messages, if used properly, essentially provide continual documentation of the ongoing development process. Documentation that otherwise wouldn't be written.
Version control makes it very easy to track down regressions and see where they appeared. In the easy case, you just track down commit messages and spot the culprit. In the average case, you have to consult the diffs too. In the hard case, you have to do regression testing of previous versions using what amounts to binary search, which is still better than the no-VC case, where you simply have no clue.
This list is not exhaustive, of course, but these are the main benefits that come to mind right now. Obviously, as others have already mentioned, it's easier to show all this to your colleagues than to describe it to them, and setting it up for yourself first (but importing everyone's code, mind) sounds like a great idea.
As Joel points out on one of his articles, start using your own one man version control system and market its benefit on every opportunity you get. Show them the benefits of traceability, granular backups etc from your single instance. People will start realizing its benefits irrespective of their age.
I agree with the answer that are referring to the Joel Guerrilla article.
Install/Use some thing with a low overhead. Hg (Mercurial) is easy in a mixed eniroment and is good because you can bail out and use something else in an easy way.
You must share your things without making a fuzz about it. When someone needs your code, export it and use the "standard" corporate method (shared folder or whatever)
When you get code, always import it into a repos, if you think it is a new commit of a repos you already have, try to get it into that one.
Sooner or later you will have a code for several project and hopefully some commits on some repos. Then you can expose those with the mercurials webserver interface (hg serve -p XXXX).
When the times comes when someone don't know why something suddenly don't works as it should be and is trying to figure out why becase it was working last monday ... and you know that you have that code in a repos step up and ask if you can be of any assistance. Get the falty code, commit it into your repos and expose with hg serve. Look at it in the browser.
My point is that you must prove with real cases to your colleges that this stuff has a value.
If the haven't figured it out by themselves after some many years you have a mountain to climb but it can be done. You must be patience though. It could very well take a year to convert one man (old dog). If you have any younger coworkers try to do this together, the more code you can get hold on the better.
I would point out the hazards of not having one - lost code, developers over writing each other changes, ability to rollback problems, etc.
Also since Subversion and some others are free, point out there is no real cost to purchase, jsut the time to implement.
The biggest issue you will have as the new guy is that you will be seen as rockign the boat, if they had no issues to date they will be hard to convince. Perhaps start using it locally jsut for yourself and maybe they will like what they see and start to adopt it.
I would try small steps, maybe ask the others if they ever used one, point out the benefits, when an issue arises that a system would have prevented or aided in point it out delicately.
From a purely business perspective, and depending on the size and nature of your parent company an IT auditor may consider your lack of a VCS a finding (i.e. something that needs fixing). I believe you could improve your pitch to management by telling them that any CVS is a great way of showing that your department respects its resources and works in a structured way and efficient way, something auditors always like to see.
I don't know how your corporate culture works but I'd be careful about rolling out your own CVS since if it does see use it suddenly becomes your responsibility when things go wrong, even if you were not at fault. To cover your ass (and keep the aforementioned auditors even happier) I'd roll the system out with a full set of written procedures for its use and maintenance.
Finally while I myself am a big fan of initiative at any level of the enterprise don't expect people to remember to say thanks when they figure out how great it is. Some might, but for the most part you're doing this to make your own life easier and for your own karma.
Remember, there are plenty of version control systems that are absolutely free. And the amount of time spent installing and maintaining a version control system should be somewhere near 0 (they shouldn't require any maintenance). There isn't even a space penalty for most systems, as they can compress things internally.
You have listed some advantages, and there are others. But more importantly, I can't think of a single disadvantage.
I would also recommend starting with implementing VCS (Version Control System) for yourself first. I'd recommend using one of distributed VCS (Git, Mercurial, Bazaar) rather than centralized Subversion, because it would be easier to create central repository (or repositories) by cloning than moving your Subversion repository to central place. Distributed SCM can be also used in a smaller group to exchange ideas.
A few advantages of (modern) version control systems:
You can always revert (go back) to last working version of your code (provided that you follow some sane version control conventions, like at least tagging only tested code). With code shared via folders it might turn out that no one version works, backup copies were deleted to save space, and recovering code from backup is tedious / was never tested.
You can switch between working on some new feature (some experimental work), and working on urgent fix in currently deployed version (maintenace work) thanks to branches (and stash / shelve for uncommitted work).
If you follow good practices for version control (small and often commits, changes being about one single thing, writing good commit message describing change and whys of change) you would have much, much easier finding bugs, be it by bisecting history to find which change introduced bug, or by using version control system to look up who was responsible for given area of code (annotate / blame).
Start talking to the other developers about problems thay have had in the past as a way to get to know the system and how it evolved (sneaky, sneaky, sneaky, but hey this information will probably come in handy at some poitn even aside from the version control issue). You are bound to sooner or later find some wonderful examples of things that have already happened which would have been far less painful if you had version control. Use these examples when you present the idea to management.
I agree with the idea that you can probably start using your own version control and eventually will be able to help thm out of a fix, but I'd bet money they have been in some of those fixes already and if they already remember how painful the problme was before, it will help sell the new idea.
Look for another job.
Seriously.
There are way better jobs out there that don't require you to teach the existing staff.
Ones where you could go into work and just, y'know, work.
Also, keep in mind that 30 isn't far off. That's the age at which most people
stop suffering fools gladly.
Just a heads up.
EDIT
It's been suggested that quitting a bad job is for quitters.
Maybe so, but keep in mind that you're supposed to
put your employer to the Joel test before you accept the job, not after.
Automated merging isn't perfect. Just because there isn't a line-edit conflict doesn't mean there isn't a syntactic conflict, and that doesn't mean there isn't a semantic conflict.
Does anyone have strategies for authoring low-conflict changes? Is this something that falls out of TDD or other approaches (Certainly TDD will help catch them, but does it actually prevent)?
I've always found that the smaller my commits, the less likely they are to have merge conflicts. The folks who have big problems always seem to go off for days and work on things, then try to merge them all at once.
Right now I'm working on a 2 man team where we are right in the same codebase all the time. We each work in a personal branch and then integrate to a shared branch whenever we have something working. That's usually several times a day. We almost never have merge conflicts, and when we do they're pretty trivial.
So... get the latest code from the repository frequently. Work in your own branch, so you can commit your changes and merge other folks' work without affecting the rest of the team. Then push your own code up to the shared branch as frequently as possible so the changes will be as small as possible.
Also, talk to your team. If you know someone else is working in a specific file, you might want to wait until they get their work in before you jump in. Sometimes you can't help it, but communication at least lets you plan for a complicated merge rather than being surprised.
Classes that violate the single responsiblity principle are the hardest to merge. Finding a class that was difficult to merge probably is a sign that it needs to be refactored, probably in the direction of more parts.
First of all, your code base should be modular. Second, what you need is communication with the rest of your team. Everybody should know who is working on what. If there is a change in the internal API, it should be made clear to the whole team.
Also, before commiting, always fetch the last version, and if complex merging is needed, do it locally.
This is really a human problem, not a technical one. Source control doesn't replace proper communication channels. Your Project Manager should be on top of every changes, and he should realize when a change will span several people.
Also, common sence is needed. :)
Unit testing is of course a big help to catch the most elusive bugs that can come up when merging.
Talk to your fellow developers, and try to avoid sychronous editing of the same block of code wherever possible. Having a well-modularised architecture (small classes, decoupled functionality) makes this possible almost all the time.
If we ever do have a clash, we often resolve it by one of us switching to writing unit tests for untested code for a few minutes.
I use version control extensively. When I'm working by myself, I still use it, and find many good things about it. I know I'm 'supposed' to put in good messages etc, but find that usually the date of a commit and all the tools for checking diffs etc are enough. I often end up putting in junk messages like 'changes'.
I guess this is a weird question, but, what do others use as their log messages when they're making commits in repositories that only they are using? Is there any problem with not leaving messages?
I happen to use git, but this question is more general.
For me it depends on the nature of the fix. Sometimes, its just one word. "Backup", or "copy changes". However, if something that caused me a lot of grief, I'll document my changes a lot more extensively. If it is open source and I won't be there all that long, I document my changes very extensively. svn -diff ( and then document all my changes that way...:)
Bug fixes that are identified by a number in another system, need to be in the change log.
I'll, grant you that "Fixed bug" isn't very good in the change log, but if it a simple bug then maybe that will do.
I don't think there is a good and fast rule, but your entry should be proportional to the amount of time you spent doing the code. A copy change? spelling mistake? not that much of a message needed.
Did you spend 2 hours fixing a bug? Yep! Long commit message.
I'm a solo developer using version control as well. I recently started using an issue tracking system that monitors the messages, so mine have gotten better and at least reference an issue number when there is one. The rest of the time, I try to at least generally state what areas changed in a short sentence or two.
But every once in a while, I still get lazy (or am half asleep) and type in things like "fixed a bug".
You should put in messages as meaningful as those you would put into code that has multiple developers. There's usually little difference between someone else looking at your changes in a couple of days, and you looking at them 12 months down the track. There's a good chance, in both those situations, that the person looking will have no idea why the change was made :-)
I even go so far as to use proper change control, even for the stuff I do solo. That means every change to the code base has to have either a change request or a bug report (with full documentation).
That makes my life a lot easier when I need to understand why something was done. I've got better uses for my "wetware" than trying to remember every little change and why it was done. Far better to let the machine remember it - its memory is so much better.
And, in my opinion, if you can't be bothered doing it right, don't do it at all. Just revert to the cowboy-coder mentality and save yourself some effort.
Doing it right doesn't take that much extra effort and the rewards are substantial. It all comes down to a cost/benefit analysis.