I was working on something for a client today when I found a way to break some functionality in our program.
(The code is really legacy code, it's been in development for about 10 years and I've only been working here for about a year.)
It didn't cause an error, or cause the program to crash, but if a user was using the program and duplicated the behavior I'm pretty sure they'd be holding up their "WTF?" flag.
In our program we have named fields (textboxes) and static text (labels) that can be linked with the textboxes. When the textbox is not filled in the label(s) that were linked to them disappear.
The functionality that I broke was, when you change the name of a textbox that already has one label or more linked to it, and save the file, without re-associating the one or more labels associated with the textbox, the formerly-associated labels appear when the textbox is blank.
Now my thinking on the matter is that a simple observer pattern could have solved this problem in the first place, but then I didn't write the code.
I was thinking that if I could dig up more situations like this with the guys in my shop, that maybe I could talk them into considering unit testing, decoupling, applying patterns where they are called for and the like.
So for this reason I was wondering if anyone had any tips for finding broken (but not error causing) functionality in any sort of app (web-based, desktop, etc...)
For an app to fail usability, it has to have a defined set of expected behaviors.
"Is this textbox SUPPOSED to do nothing when the enter key is pressed?" Maybe it is, maybe it isn't. I've seen apps where a tester/reviewer reports something that they ASSUME should work another way, when in actuality the client specifically asked that they DON'T want the form submitted on a return key press, but only a submit button click.
So basically you have to define proper behaviour before you can determine incorrect behavior.
Hire some testers.
If it has an interface, then one of my favorite unconventional test is putting 5-10 year old children in front of it. You'd be surprised what they can come up with (especially the younger ones). While this may sound like a joke, it isn't -- it really works, because children don't have the mindset of only going through "mindset" paths.
And yeah, children are the experts in "breaking things" xP.
Code inspections, i.e. reading the source code: if you had taken time to read/inspect the source code, looking for "smells" or even just looking for code whose behaviour you don't immediately understand and agree with, you might have been holding up your "WTF?" flag too.
Test, test, test.
Do unexpected things. Start doing one task and switch another to see if anything goes haywire. Use the back button when you're not supposed to. Open it in two windows. Let it time out.
Test in all browsers, especially IE.
You can find database connections/sessions aren't released by:
working out the minimum number of connections you need to do something
setting resource limits to that minimum number
ensuring one "run" of the scenario that should use exactly that number (and release it afterwards)
then run it again a few times... do you run out of connections?
I used to work in a company where programmers regularly used to forget to de-allocate db connections. The standard answer was to reduce the resource to a minimum to see if there's a leak - and to try to work out where it is by restarting the system and running different scenarios repeatedly.
The first hour of code review, with the first reviewer, will do the most to find quality problems. But here's the thing: You don't need to convince people of quality problems. You need to convince them of the value of fixing bugs, and of rewriting only when the present quality absolutely justifies it.
I've dealt with some seriously bad code in my time. But you can't just rewrite. You need a spec before you can even tell if the rewrite is an improvement.
Sometimes, you have to infer the spec from the code and then check it against some human somewhere. But by the time you've done that, you understand the code as written and are now better prepared to repair than to rewrite -- most of the time.
Repair proceeds by a process of small behavior-preserving modifications that render the spec more clear in the code. Then, when you find something that looks wrong, you don't just change it. You ask around until you find the person responsible for that decision, and you get them to show you where in the spec it says that behavior X is correct. (This conversation can take many forms.) If you're lucky, they'll tell you that behavior X is in fact incorrect, and then you've earned your pay.
assert()
Also unit testing with coverage analysis.
This is particular to the Visual Studio IDE, although it probably also applies to others:
During testing, always at some point run in the debugger with "Break when an exception is thrown" turned on.
This can often help expose exceptions which are incorrectly being silently caught and which represent bugs, but otherwise may not be evident.
Code reviews should always also include reviews of the unit test code.
The problem is that with ad-hoc testing it's impossible to know how much or how well a developer has tested their code. So, you're at the mercy of different developers definition of the word "done".
If you include reviews of the unit test code at the same time you review the production code you should have a good idea of whether the code is really complete; in that "complete" includes "tested". Not just "Hey, I'll throw it over the wall to the testers!".
Related
In my app(unity5, il2cpp build), There is a function like
"GetScore()"
Unfortunately, i found the hacked version of my app in the black market. that hacked version's "GetScore()" function always returns 100. The original return value has to be under 10.
I guess this hacked App is repackaged by using like 'il2cppDumper' and changing my return value to 100;
Is there any way to prevent this problem?
Security is always a matter of making it harder for hackers - you can never make it impossible for them to tamper.
So here are some thoughts:
Obfuscation: GetScore() gets BananaJungle() - hackers cannot find the correct function without stepping through the source code for hours (hopefully)
Validate the score on multiple spots: the function calling GetScore() should do a sanity check: "is it below 10?"
In Addition to this: You may want to ignore scores above 10 to fool the hacker he succeeded. You may lock the app after 2 hours or so.
Add a ScoreLogger somewhere that logs the history of the score, so getScore() may return values <10 but someone might just overwrite the score in code to 999999. ScoreLogger will check the score history for jumps etc.
Validate Score with total playtime (approximately)
You won't ever keep hackers from hacking your games, even if it does indeed have a backing server. Just look at all the unofficial world of warcraft servers. You can keep things relatively safe if you have a server, you keep its source code secure, and your game is meaningless without its server (think Dota 2 with no multiplayer capabilities...). Even then, you can't actually validate the player's every move, unless it's a turn based game and you actually send every move the server to be processed (this works in Hearthstone, for example, but not in WoW, hence all the anti-cheating tools). EA couldn't do it, Rockstar couldn't do it, Activision couldn't do it, even the mighty Denuvo couldn't do it, you certainly can't do it.
However, you should stop and ask yourself why you want your game to be that secure. Out of every 1000 cheaters you stop, maybe one or two would actually pay. You should put in a moderate amount of effort on security (take KYL3R's advice), simply to keep honest people honest. Dishonest people will always find a way, so don't worry about them so much that you end up wasting time on (useless) security; time you could spend on making your game better.
Oh and by the way, that's also one way to keep hackers out: frequent updates to the game. They have no life, but they don't have enough time to keep making a hacked version of every game on the market every week.
I have a multiplayer project which has some forever loops with checking code inside of them.
The problem is, multiple computers might process this and change crabx or craby due to lag in the variables dvotes, uvotes, lvotes, or rvotes. Only one machine should change this, though.
This can be easily solved by giving each player an ID like many people do in SQL. I would just check if the ID is 1, and that would be the "operating machine". I would then do all of these checks on that one machine. It would do things a Scratch server would do if you could program it...
The problem with this is that there is no way to detect when a player leaves the game. There is no block that is called "on exit" or "on stop button pressed". How would I go about doing this? I have seen people have a button which people click to exit, but some people will not click it/not even see it.
Thanks in advance!
Option 1
I've never been especially successful with cloud data myself, but I've heard the theory on this before:
Essentially, each player gets a "counter". Their computer then constantly increases that counter. If the counter ever stops increasing (which will be detected by the other computers, who are all looking after one another), the project will know that the user has left and one of the computers will take care of removing their ID and other data.
Obviously, this is much easier said than done. (As I said, I've never gotten complex cloud data to work well for myself, but I've seen it done successfully and explained.)
Option 2
Alternatively, you might be better off taking advantage of this cloud api created by MegaApuTurkUltra. I find that stealing from others tends to be the best way of solving problems when it comes to code. ;)
So I have had this bug for a long time that I have been unable to track down. The problem is I cant recreate it often.
So far I have tracked the bug down to a specific process that basically goes like this.
Their are three simultaneous processes that go on that take different lengths of time. After each process is done it sets a boolean "key" to true, and then triggers a function that checks if all "keys" are in place. Basically once the last "key" is in it will actually start to do things.
Somewhere in the key setting process, or earlier, or even possibly after it crashes. Unfortunately it leaves a really cryptic error message, and when it crashes in Xcode it is "EXEC BAD ACESS" and thus just puts a breakpoint in the app delegate declaration.
I am sure I can easily fix this bug, I just dont know enough on how to fix this. Thankfully I have fabric which allows me to print to a text file that I can see when a user crashes. Each update I add new data to it (at the cost of a tad bit of latency) in order to better understand how it happened. Each new crash gets me closer. Unfortunately though I have slow adoption rates to new versions and the crashes just keep building up! I still dont know why.
Unfortunately because this crash only happens once in a blue moon (atleast on my device) on my device. And because of the low new version updates I have to collect the data myself. Which is really hard!
I have tried tons of methods of trying to get things to go wrong, or making the process that caused it happen rapidly, or even having auto pressing buttons. Still i cant get it to crash again! And when it does all I can do to track down the bug is add more println calls so I can see what is going on.
The freaky thing is for all I know I could have already fixed it because I usually try new tweaks ect. But I won't know because it won't consistently crashes. Honestly i'm fairly sure I fixed it (or at least took down the chances of it happening).
What would you do in a situation like this?
This sound like a race condition
I would use NSOperationQueue to make sure that all he tasks are done in order and that all are actually completed before the final one is performed.
I am struggling with this question since I noticed that many functional testing frameworks (like Selenium for the web or UISpec for iOS) actually simulate UI events while testing. I am asking: couldn't it be sufficient just to check for preconditions such as that, e.g., the target and selector for a button are set correctly and then fire the selector manually? Why do I need to simulate touches? This has the con that you have to know more about the UI elements you're testing (you have to know what makes them to behave correctly), but since I am the one writing the tests, maybe this doesn't matter?
Could anyone shed some light on this?
Simulating touches can be useful for determining crashes caused by obscure or unplanned user behaviour - a particularly common one is having two items pressed simultaneously. It also allows you to create potentially quite esoteric tests: for example, random user input for a sustained period of time to attempt to crash or break your application in ways you wouldn't expect. The level to which you'd do this would depend on your app, and how important it was to you.
Your alternative approach also has some disadvantages when it comes to multi-touch. Whilst it would be fairly straightforward to fire a button selector through some sort of automatic test rather than simulating user input, what happens if you have an app that deals with swiping, pinching, or other multiple input gestures? In those cases the desired result may not be as black and white as the on/off of the button: you may have many shades of grey and differing output that required validation.
Simulated UI testing actually has quite a long history - there's an interesting story (well, interesting to me) about the original MacPaint and how a random UI input test was able to assist in reproducing obscure or difficult crashes here: http://www.folklore.org/StoryView.py?story=Monkey_Lives.txt
In my current environment, we have a "clean" build machine, which has an exact copy of all committed changes, nothing more, nothing less.
And of course I have my own machine, with dozens of files in an "in-progress" state.
Often I need to build my application with only one change in place. For example, I've finished task ABC, and I want to build an EXE with only that change.
But of course I can't commit the change to the repository until it's tested.
Branching seems like overkill for this. What do you do in your environment to isolate changes for test builds and releases?
#Matt b: So while you wait for feedback on your change, what do you do? Are you always working on exactly one thing?
So you are asking how to handle working on multiple "tasks" at once, right? Except branching.
You can have multiple checkouts of the source on the local machine, suffixing the directory name with the name of the ticket you are working on. Just make sure to make changes in the right directory, depending on the task...
Mixing multiple tasks in one working copy / commit can get very confusing, especially if somebody needs to review your work later.
I prefer to make and test builds on my local machine/environment before committing or promoting any changes.
For your specific example, I would have checked out a clean copy of the source before starting task ABC, and after implementing ABC, created a build locally with that in it.
Something like that: git stash && ./bootstrap.sh && make tests :)
I try hard to make each "commit" operation represent a single, cohesive change. Sometimes it's a whole bug fix or whole feature, and sometimes it's a single small refactoring on the way to something bigger. There's no simple way to decide what a unit is here, just by gut feel. I also ask (beg!) my teammates to do the same.
When this is done well, you get a number of benefits:
You can write a high quality, detailed description for the change.
Reading the first line of the description of each change gives you a sense of the flow of the code.
The diffs of a change are easy to read & understand.
If a change introduces a bug / build break / other problem, it's easy to isolate, understand, and back out if necessary.
If I'm half-way through a change and decide to abort, I don't lose much.
If I'm not sure how to proceed next, I can spend a few minutes on each of several approaches, and then pick the one I like, discarding the others.
My coworkers pick up most of my changes sooner, dramatically simplifying the merge problem.
When I'm feeling stuck about a big problem, I can take a few small steps that I'm confident in, checking them in as I go, thereby making the big problem a little smaller.
Working like this can help reduce the need for small branches, since you take a small, confident step, validate it, and commit it, then repeat. I've talked about how to make the step small & confident, but for this to work, you also need to make validation phase go quickly. Having a strong battery of fast, fine-grained unit tests + high quality, fast application tests is key.
Teams that I have worked on before required code reviews before checking in; that adds latency, which interferes with my small-step work style. Making code reviews a high-urgency interrupt works; so does switching to pair programming.
Still, my brain seems to like heavy multitasking. To make that work, I still want multiple in-progress changes. I've used multiple branches, multiple local copies, multiple computers, and tools that make backups of pending changes. All of them can work. (And all of them are equivalent, implemented in different ways.) I think that multiple branches is my favorite, although you need a source control system that is good at spinning up new branches quickly & easily, without being a burden on the server. I've heard BitKeeper is good at this, but I haven't had a chance to check it out yet.