I would like to use hand-coded assembly language vertex and fragment shaders in order to program very optimized shaders on iPhone with OpenGL ES 2.0.
I googled around but I can't find an example or even if it is allowed by apple sdk.
On the iPhone, there is no way to hand tune shaders. It's worth noting that on the iPhone in particular, there are no optimizations that you can do that a compiler can't. That said, the GLSL compiler will probably beat or match your hand-tuned assembly.
On the PC, however, I personally do not have faith in the driver to know that a looping shader should use fewer registers and more instructions in order to achieve greater throughput through higher occupancy. Drivers simply don't have enough context to make the right choice all of the time. Data-specific compile-time optimizations are a great example of this problem.
As someone who has actually looked at the GLSL compiler's assembly output and tried to game the compiler's register allocation strategy, I can tell you that not having assembly access absolutely does hurt performance (on the PC, there are some publicly available tools from NVidia and AMD that allow you to do this). The trade-off to using assembly is that every shader needs to be hand tuned for every supported part in order to achieve the maximum possible performance. While this is a bit extreme, if I want to invest my own time into fine tuning a rendering back-end for each video card my products support, then I should be able to do so. A more practical example would be hand-tuning for low-end video cards, but letting the GLSL compiler do it's job on more high-end video cards.
Further, offline compilers provide a safety mechanism. Many video games today rely on drivers to emulate a lot of the functionality available in modern graphics APIs. As a game developer who works in the game-as-a-service space on the PC, I can tell you that it's extremely upsetting to get a call in the middle of the night because of a minor GLSL bug in a newly released graphics driver. Driver bugs severely impact the overall player experience. Most players simply think that your game is broken, and you can actually lose players as a result (and we probably have). Being able to compile once for each supported video card and hand-tune after the fact would be a huge win in this regard. It simply means that the driver would have to do less work. Code is evil, so the less code that executes is better =).
As a side note, I made the following demo using the 'compile'-'view assembly'-'modify'-'repeat' approach: http://www.youtube.com/watch?v=km0DpZUgvbg . I can tell you with 100% certainty that I could further improve the performance of this ray-tracer with assembly language, and AFAIK, it's the fastest voxel ray-tracer whose existance has been published (that was the case as of Mar 2012, but is likely no longer true). Unsurprisingly, each time a new driver would come out, I would see this demo's performance go from 125-130 fps down to 30 fps - all because the driver didn't know how to optimize my shader correctly. That means I'd have to repeat my optimization process each time a new driver came out, which caused me to simply mothball the project (ACK!). Even though my voxel raytracer can support a large variety of hardware in a performant manner, drivers are currently making it impossible to support this technology in a full product. I simply do not have the weight to put this technology in action because it would require driver vendors to know the ways in which they need to optimize my shader. How many other technologies would be possible if we simply had direct assembly shader access? This implies that lacking assembly access is actually a serious cost. For anyone else in this position, I recommend the following: Use NVidia's assembly language when possible and fallback to GLSL when it's not. If we show the advantage of assembly over GLSL, then hopefully we'll get first-class assembly support from all vendors =).
And finally, not to pick on another author, but I want to point out that the argument made by 'Nicol Bolas' is almost entirely fallacious (sorry Nicol, I have nothing against you, but I wanted to point out some popular arguments that simply don't hold up to an ethics test). Please note that a fallacious argument does not mean that a particular conclusion is incorrect -- just that the argument posited is simply fallacious.
"Why? You don't trust the compiler to do it's job? Do you really think that you know enough about the GPU in question to be able to consistently beat the compiler?"
This is not an argument. This is simply a question that a lot of people can't think of a real answer to. Therefor, they come to the conclusion that they should just trust the compiler. This prevents people from further investigating the ramifications of trusting the compiler, and prevents logical discourse of the actual pros and cons from taking place. Furthermore, the use of the word "really" in your second question implies that someone answering yes must be deluded. This also hints at what you think about yourself, Nicol -- it implies that you value your opinion above all others, and anyone who doesn't think like you must have something wrong with them (not that they are wrong, but that they have something wrong with them - big difference). That said, you might want to take some time to think about your thought process, your feelings, and your emotional state. Taking this approach will seriously limit your ability to learn, as you won't be challenging your own ideas with enough rigor. Please stop using this argument. It's not healthy or ethical.
"Ultimately, you're just going to have to trust the compiler made by the people who built your GPU. Nobody else has a problem with that these days."
I have a problem with it. Also, even if no one did have a problem with it, that still wouldn't matter. The fact is that there might still be some benefit to allowing assembly shaders. That said, a consensus does not mean correctness. Further, this argument is especially unethical because there is an implied "Nobody else has a problem with that these days so if you have a problem with it, you must be weird or out of date in your thinking". People have a natural desire to fit in, and using this argument is a way of dividing people on this issue and persecuting people that don't think like you. That said, this argument is especially insidious because of it's implications. Please don't use this argument.
Nicol, both of your fallacies imply that you are right and normal, and anyone who doesn't agree with you is wrong and has something wrong with them. These are extremely unhealthy viewpoints, and you should examine them rigorously for your own mental health and career.
For future reference: http://en.wikipedia.org/wiki/List_of_fallacies#Formal_fallacies
Thanks!
I would like to use hand-coded assembly language vertex and fragment shaders in order to program very optimized shaders on iphone with opengl es 2.0.
Why? You don't trust the compiler to do it's job? Do you really think that you know enough about the GPU in question to be able to consistently beat the compiler?
Anyway, you can't. Nor could you do it in desktop OpenGL. ARB assembly shaders aren't that much closer to the hardware than GLSL; they both go through compilation and optimization by an internal compiler.
Ultimately, you're just going to have to trust the compiler made by the people who built your GPU. Nobody else has a problem with that these days.
I understand you point very well : you want to see the generated assembly source code, and maybe modify it.
In fact, GLSL compilers are not optimized in comparison with HLSL ones. To convince yourself, just compare the generated assembly in ShaderAnalyzer for the same shader in GLSL and HLSL ; you will immediately see that they don't give the same optimization at all.
Even for trivial optimizations, like factorizing if() conditions, most of GLSL compilers don't do the job.
I really would like to see the generated ASM by Apple's compiler (especially for iOS platforms). If you know a way to get the assembly, I'm very interested in the process.
Related
I like C# 3.0 features especially lambda expressions, auto implemented properties or in suitable cases also implicitly typed local variables (var keyword), but when my boss revealed that I am using them, he asked me not to use any C# 3.0 features in work. I was told that these features are not standard and confusing for most developers and its usefulness is doubtful. I was restricted to use only C# 2.0 features and he is also considering forbidding anonymous methods.
Since we are targeting .NET Framework 3.5, I cannot see any reason for these restrictions. In my opinion, maybe the only disadvantage is that my few co-workers and the boss (also a programmer) would have to learn some basics of C# 3.0 which should not be difficult. What do you think about it? Is my boss right and am I missing something? Are there any good reasons for such a restriction in a development company where C# is a main programming language?
I have had a similar experience (asked not to use Generics, because the may be confusing to my colleagues).
The fact is, that we now use generics and non of my colleagues are having a problem with them. They may not have grasped how to create generic classes, but they sure do understand how to use them.
My opinion on that is that any developer can learn how to use these language features. They may seem advanced at first but as people get used to them the shock of newness lessens.
The main argument for using these features (or any new language features) is that this is a simple and easy way to help my colleagues advance their skills, rather than stagnating.
As for your particular problem - not using lambdas. Lots of the updates to the BCL have overloads that take delegates as parameters - these are in many cases most easily expressed as lambdas, not using them this way is ignoring some of the new and updated uses of the BCL.
In regards to the issues with your peers not being able to learn lambdas - I found that Jon Skeets C# in depth deals with how they evolved from delegates in a manner that was easy to follow and real eye opener. I would recommend you get a copy for your boss and colleagues.
You boss is going to need to understand that language (and other) improvements are designed to give developers more capabilities, and make them more efficient in completing the task at hand, and that if he is not going to allow them for unknown reasons then:
The development team isn't producing at its greatest potential.
The company isn't benefiting from increased efficiency/productivity.
like others have said developers aren't worth their salt if they can't keep up with some of the latest improvements in the language that they are using on a daily basis. I suspect your boss hasn't done much coding lately and it is his inability to understand the latest language improvements that has motivated this decision.
I was told that these features are not standard and confusing for most developers and its usefulness is doubtful. I was restricted to use only C# 2.0 features and he is also considering forbidding anonymous methods.
Presumably roughly translates to your boss meaning...
These features are confusing for me, and I don't find them useful because I don't understand them.
Which is fairly symptomatic of the Blub paradox (well, or just sheer laziness). Either way there's no merit in what he's saying, and you should start looking for another job if he continues down that road.
If the project is strictly C# 3+ from now on, then you would not break the build by including these items. However, before using them you should be aware of the following:
You can't use them if the project lead gets to make the decision and votes no.
Other than that, you should use them where it makes the code significantly easier to maintain.
You should not use them in ways that are confusing, or unnecessary in the sense that they do not significantly improve the maintainability of the code. This does mean you should not use them where the code is effectively the same or barely improved.
If Microsoft didn't define the standard and these were features that they added to a non-Microsoft language, I would say your boss might have a point. However, since Microsoft defines the language and uses these very features in implementing significant parts of .NET 3.5 (and 4.0), I'd say that you'd be foolish to ignore them. You may not choose to use some of them -- var, for instance, may not be acceptable in all environments due to coding standards -- but a blanket policy of avoiding new features seems unreasonable.
The trickier bit is when should you start using new features, because they can be confusing and may delay development. In general, I choose to use new language features and platform elements on new projects. I often avoid using them on projects that are currently in development when the feature/framework enhancement comes out, deferring until the next project. On a long project, I might introduce them at a significant milestone if the amount of rearchitecting is small or the feature is worth the changes. Normally, I'd wait until the project is due for significant changes anyway and then evaluate if refactoring to newer features is warranted.
The jury is still out on the long term consequences of some features, but if their main rationale is 'it is confusing to other developers' or something similar than I would be concerned about the quality of the talent.
I like C# 3.0 features especially
lambda expressions, auto implemented
properties or in suitable cases also
implicitly typed local variables (var
keyword), but when my boss revealed
that I am using them, he asked me not
to use any C# 3.0 features in work. I
was told that these features are not
standard and confusing for most
developers and its usefulness is
doubtful.
He's got a point.
Following that line of thought, let's make a rule against generic collections since List<T> doesn't make any sense (angle brackets? wtf?).
While we're at it, let's eliminate all interfaces (when are you ever gonna need a class without any implementation?).
Hell, let's go ahead eliminate inheritance since its so tricky these days (is-a? has-a? can't we all just be friends?).
And use of recursion is grounds for dismissal (Foo() invokes Foo()? Surely you must be joking!).
Errrm... back to reality.
Its not that C# 3.0 features are confusion to programmers, its that the features are confusing to your boss. He's familiar with one technology and stubbornly refuses to part with it. You're about to enter the Twilight Zone Blub Paradox:
Programmers get very attached to their
favorite languages, and I don't want
to hurt anyone's feelings, so to
explain this point I'm going to use a
hypothetical language called Blub.
Blub falls right in the middle of the
abstractness continuum. It is not the
most powerful language, but it is more
powerful than Cobol or machine
language.
And in fact, our hypothetical Blub
programmer wouldn't use either of
them. Of course he wouldn't program in
machine language. That's what
compilers are for. And as for Cobol,
he doesn't know how anyone can get
anything done with it. It doesn't even
have x (Blub feature of your choice).
As long as our hypothetical Blub
programmer is looking down the power
continuum, he knows he's looking down.
Languages less powerful than Blub are
obviously less powerful, because
they're missing some feature he's used
to. But when our hypothetical Blub
programmer looks in the other
direction, up the power continuum, he
doesn't realize he's looking up. What
he sees are merely weird languages. He
probably considers them about
equivalent in power to Blub, but with
all this other hairy stuff thrown in
as well. Blub is good enough for him,
because he thinks in Blub.
When we switch to the point of view of
a programmer using any of the
languages higher up the power
continuum, however, we find that he in
turn looks down upon Blub. How can you
get anything done in Blub? It doesn't
even have y.
C# 3.0 isn't hard. Sure you can abuse it, but it isn't hard or confusing to any programmer with more than week of C# 3.0 experience. Your boss's skills have just fallen behind and he wants to bring the rest of the team down to his level. DON'T LET HIM!
Continue using anonymous funcs, the var keyword, auto-properties, and what have you to your hearts content. You won't lose your job over it. If he gets pissy about it, laugh it off.
Like it or not, if you plan on using LINQ in any situation, you're going to have to utilize some of the C# 3.0 language specs.
Your boss is going to have to warm up to them if he wants to utilize the feature sets you get from 3.5, which are numerous and worth your time investing in.
Also, from my experience in leading teams, I've found that using the 3.0 specs actually has helped devs readability and understanding of the code base. There's about a weeks worth of time that is spent by the dev trying to understand what the syntax means, but once they get it they much prefer the new way over the old way.
Perhaps you can do a presentation once a week on each feature to everyone and get some of the developers on your side to help convince management of the benefits.
I recently moved from a bleeding edge C# house to a C# house that was running mostly on dot.Net 1.1 and some 2.0 projects, using mostly only 1.1 features. Luckily management stay away from the code. Most of the developers love all the new features in the newer frameworks, they just don't have the time or inclination to figure them out by themselves. Once I managed to show them how they can make their own lives easier they started using them by themselves and we have migrated several projects to gain the new language features and better tool advantages.
Some people are just afraid of change, because maybe you'll make them all look stupid using fancy new technologies. Could also be that your boss doesn't want the team learning new things instead of getting work done the old fasioned way.
The var keyword can certainly be abused, but in most cases reduces redundant code. LINQ is the main thing you want from .Net 3.5 because of the huge time saving in the amount of code you have to write. Your boss should be encouraging you to use it. Also the base class libraries now take delegates are parameters, so you will be limiting yourself a lot by not using them. Lambda's are just some fancy syntactic sugar to make delegates cleaner.
I would refer you to Effectively Integrating into Software Development Teams and Leading by Example. Two really great articles on how to deal with teams that are afraid of change.
Developer tools and software typically do not provide solution suggestions in error messages. This makes sense for compilers because they are supposed to tell precisely what went wrong.
There are "lint" tools to provide suggestions, but AFAIK, few developers use lint tools regularly or even at all.
There is a large set of developer-oriented software that would do well to have a "suggested solution(s)" part to error messages. This is one of the great features that IDEs like Eclipse have. But software like web application frameworks, standard/popular libraries, etc. do not have this helpful feature.
Is this something that is just lacking in user-friendly design (one can consider this unnecessary, given that Google is so good) or is there a good reason for it? Do any compilers, frameworks, platforms that you use provide error messages with solution suggestions, if none, why not?
What do you want to see?
Error: Null Pointer Exception (suggested solution: Set the object to something).
I mean, it's not the error writers job to educate you. I prefer simple error messages that point to the exact problem, so I myself can determine just what is causing it this time. For me, this certainly lines in the domain of 3rd party tools; perhaps the compilers could provide extensive context to them, to do their analysis, but it's not something I would really find valuable.
The primary thing I want from a compiler or runtime error is context - where did it happen and where was it called from when it failed.
I think most modern compilers and runtimes (Java, Ruby, Go) do a decent job there, with line numbers and stack traces you can find most bugs. Even the Javascript options getting good, it certainly beats the good old "alert()" approach to debugging.
Isn't it fair enough to leave suggested solutions to the IDEs?
But I do agree that I have seen frameworks/libraries that were very sparse with error messages, and "NullPointerException at line 264" deep inside some third-party library where you do not have the source code tells you very close to nothing.
If this is an issue, I think it is primarily restricted to third-party libraries. The "good reason" is presumably that it was developed in a hurry in somebody's spare time, and they did not put meaningful error messages very high on the priority list.
It's hard for the solution to an error being presented. There are so many possibilities and as #silky pointed out, some just cannot be diagnosed.
Warnings are a different beast. In many situations modern compilers use these to say "I think you meant X when you said Y; you might want to check that."
A programming language has the opportunity to be the ultimate in flexibility in terms of user interface. You can make the computer do anything you want. The flip side of the coin is that if you type so much as one character wrong, it might have no idea along which axis your mistake was made, or where.
Systems with less flexibility offer more opportunities for offering solutions to problems. If you type (a b c) to your Lisp compiler and it doesn't know what a is, it's so close to so many valid lines of code that it can't exactly suggest a single fix. If you misspell "IDENTIFICATION DIVISION" at the start of your COBOL program, it's relatively easy for the compiler to spot the error and help you out. Most other languages lie between these extremes.
Programmers tend to move, over their careers, from less powerful and more structured languages, into more powerful and flexible languages. (At least, that's what I saw happen before Javascript became such a hot newbie language.) This means their discipline improves to the point where they are able to use tools that offer power at the expense of being told what to do. The environments I've used that can tell me what to fix, tend to be those that I dislike using.
It's no different from any other art. Look at musicians or painters or martial artists or actors or writers or chefs or even people learning to speak Spanish: when they're young and inexperienced, they're put in a system where there's a lot of structure, and if they make a mistake somebody can easily correct them. As they become more skilled, they need and want less and less support. When they've become experts themselves, they need no support at all, but the flip side of the coin is that you can't as easily point out what's right or wrong. If your kid colors outside the lines, you can explain the issue, but if Picasso or Pollock makes a bad brushstroke, what would you say? Or if Philip Glass puts a note out of place, or Bruce Lee turns his body too far into a punch? And who would want to work in an art form that's so limited that profane things aren't possible? COBOL compilers still exist if anybody really wants them, but far more people pay money for awful paintings than masterful color-by-number prints.
More directly, there's a site, ErrorHelp (nee bug.gd), that lets you type in an error message and get a result, and it's older than SO but nobody uses it. I've tried. Unless you're in a context where there's only one possible answer, a simple problem-encountered to suggested-solution dictionary does not work, and therefore it's an utter failure in any creative field.
Most IDE's have their own compilers. This allows them to do partial compilation, code refactorings, and many other tricks. I find the error messages and suggestions quite useful. Just because the compiler isn't invoked on the command line, doesn't mean it's not a compiler.
(source: theeggeadventure.com)
I've worked on a number of products that make use of code generation. It seems to be the only way to achieve both a high degree of user-customizability and high execution speed.
The downside is that we are requiring users to install a compiler (primarily on MS Windows).
This has been an on-going headache, because vendors like MS keep obsoleting compilers, and some users tend to have more than one compiler installed.
We're considering using GNU C, and possibly C++, but even there, there are continual version issues.
I've considered possibly generating assembly language, in an effort to get off the compiler-version-treadmill, but assembly languages are all machine-specific.
Ideally there would be some way to produce generated code that would be flexible, run fast, and not expose us to the whims of third-party providers.
Maybe I'm overlooking something simple, like Java. Any ideas would be appreciated. Thanks.
If you're considering C and even assembler, take a look at LLVM first: http://llvm.org
I might be missing some context here, but could you just pin yourself to a specific version? E.g., .NET 2.0 can be installed side by side with .NET 1.1 and .NET 3.5, as well as other versions that will come out in the future. So as long as your code makes use of a specific version of a compiler, what's the problem?
I've considered possibly generating assembly language, in an effort to get off the compiler-version-treadmill, but assembly languages are all machine-specific.
That would be called a compiler :)
Why don't you stick to C90?
I haven't heard much of severe violations of standards from gcc's side, if you don't use extensions.
And you can always distribute a certain version of gcc along with your product, say, 4.3.2, giving an option to users to use their own compiler at their own risk.
As long as all code is generated by you (i. e. you don't embed your instructions into other's code), there shouldn't be any problems in testing against this version and using it to compile your libraries.
If you want to generate assembly language code, you may take a look at asmjit.
One option would be to use a language/environment that provides access to the compiler in code; For example, here is a C# example.
Why not ship a GNU C compiler with your code generator? That way you have no version issues, and the client can constantly generate code that is usable.
It sounds like you're looking for LLVM.
Start here: The Code Generation conference
In the spirit of "might not be to late to add my 2 cents" as in #Alvin's answer's case, here is something I'd think about: if your application is meant to last for some years, it is going to face several changes in how applications and systems work.
For instance, let's say you were thinking about this 10 years ago. I was watching Dexter back then, but I guess you actually have memories of how things were at that time. From what I can tell, multithreading was not much of an issue to developers of 2000, and now it is. So Moore's law broke for them. Before that people didn't even care about what will happen in "Y2K".
Speaking of Moore's law, processors are indeed getting quite fast, so maybe certain optimizations won't be even that necessary. And possibly the array of optimizations will be much bigger, some processors are getting optimizations for several server-centric stuff (XML, cryptography, compression and regex! I am surprised such things can get done on a chip) and also spend less energy (which is probably very important for warfare hardware...).
My point being that focusing on what exist today as a platform for tomorrow is not a good idea. Make it work today, and surely it will work tomorrow (backward-compatibility is especially valued by Microsoft, Apple is not bad it seems and Linux is very liberal about making it work as you want).
There is, yes, one thing that you can do. Attach your technology to something that just won't (likely) die, such as Javascript. I'm serious, Javascript VMs are getting terribly efficient nowdays and are just going to get better, plus everyone loves it so it's not going to dissappear suddenly. If needing more efficiency/features, maybe target the CRL or JVM?
Also I believe multithreading will become more and more of an issue. I have a gut feeling the number of processor cores will have a Moore's law of their own. And architectures are more than likely to change, from the looks of the cloud buzz.
PS: In any case, I belive C optimizations of the past are still quite valid under modern compilers!
I would stick to that language that you use for generating that language. You can generate and compile Java code in Java, Python code in Python, C# in C#, and even Lisp in Lisp, etc.
But it is not clear whether such languages are sufficiently fast for you. For top speed I would choose to generate C++ and use GCC for compilation.
Why not use something like SpiderMonkey or Rhino (JavaScript support in Java or C++). You can export your objects to JavaScript namespaces, and your users don't have to compile anything.
Embed an interpreter for a language like Lua/Scheme into your program, and generate code in that language.
I've heard a ton of complaining over the years about inherited projects that us developers have to work with. The WTF site has tons of examples of code that make me actually mutter under my breath "WTF?"
But have any of you actually been presented with code that made you go, "Holy crap this was well thought out!" or "Wow, I never thought of that!"
What inherited code have you had to work with that made you smile and why?
Long ago, I was responsible for the Turbo C/C++ run-time library. Tanj Bennett wrote the original 80x87 floating point emulator in 16-bit assembler. I hadn't looked closely at Tanj's code since it worked well and didn't require attention. But we were making the move to 32-bits and the task fell to me to stretch the emulator.
If programming could ever be said to have something in common with art this was it.
Tanj's core math functions managed to keep an 80-bit floating point temporary result in five 16-bit registers without having to save and restore them from memory. X86 assembly programmers will understand just what an accomplishment this was. Register space was scarce and keeping five registers as your temp while simultaneously doing complex math was a beautiful site to behold.
If it was only a matter of clever coding that would have been enough to qualify it as art but it was more than that. Tanj had carefully picked the underlying math algorithms that would be most suitable for keeping the temp in registers. The result was a blazing-fast floating point emulator which was an important selling point for many of our customers.
By the time the 386 came along most people who cared about floating-point performance weren't using an emulator but we had to support Intel's 386SX so the emulator needed an overhaul. I rewrote the instruction-decode logic and exception handling but left the core math functions completely untouched.
In my first job, I was amazed to discover a "safe ID" class in the codebase (c++), which was wrapping numerical IDs in a class templated with an empty tag class, that ensured that the compiler would complain if you tried for example to compare or assign a UserId into an OrderId.
Not only did I made sure that I had an equivalent Id class in all subsequent codebases I would be using, but it actually opened my eyes on what the compiler could do to guarantee correctness and help writing stronger code.
The code that impresses me the most, and which I try to emulate - is code that seems too simple and easy to understand.
It is damn difficult to write that kind of code. :-)
I have a funny story to tell here.
I was working on this Javaish application, filled with getters & setters that did nothing but get or set and interfaces and everything ever invented to make code unreadable. One day I stumbled upon some code which seemed very well crafted -- it was basically an algorithm implementation that looked very elegant = few lines of readable code, even though it respected every possible rule the project had to adhere to (it was checkstyled automatically).
I couldn't figure out who on the team could have written such code. I was dying to discuss with him and share thoughts. Thankfully, we had switched to subversion (from cvs) a few months earlier and I quickly ran am 'svn blame'. I loled all over the place, seeing my name next to the implementation.
I had heard stories about people not remembering code they wrote 6 months back, code that is a nightmare to maintain. I could not believe such a thing could happen: how can you forget code you wrote? Well, now I'm convinced it can happen. Thankfully the code was alright and easy to extend, so I've only experienced half of the story.
Some VB6 code by another programmer at my company I came across that handled the error conditions very well (whether it be deal with them directly or log them).
Along with some rather complex code that was well commented.
I know this will bring a lot of answers like,
"I've never find good code before I step in" and variations.
I think the real problem there is not that there isn't good coders or excellent projects out there, is that there's an excess of NIH syndrome and the fact that no body likes code from others. The latter is just because you have to make an intellectual effort to understand it, a much bigger effort than you need to understand you own code so that you dislike it (it's making you think and work after all).
Personally I can remember (as everyone I guess) some cases of really bad code but also I remember some pretty well documented, elegant code.
Currently, the project that most impressed me was a very potent, Dynamic Workflow Engine, not only by the simplicity but also for the way it is coded. I can remember some very clever snippets here and there, as well as a beautiful metaprogramming library based on a full IDL developed by some friends of mine (Aspl.es)
I inherited a large bunch of code that was SO well written I actually spent the $40 online to find the guy, I went to his house and thanked him.
I think Rocky Lhotka should get the credit, but I had to touch a CSLA.NET application recently {in my private practice on the side} and I was very impressed with the orderliness of the code. The app worked extremely well, but the client needed a few extensions. The original author had died tragically, and the new guy was unsophisticated. He didn't understand CSLA.NET's business object based approach, and he wanted to do it all over again in cut-and-dried VB.NET, without any fancy framework.
So I got the call. Looking at a working example of WinForm binding and CSLA.NET was pretty instructive about a lot of things.
Symbian OS - the old core bit of it anyway, the bit that dated back to the Psion days or those who even today keep that spirit alive.
And sitting right along side it and all over it is all the new crap created by the lowest bidders hired by the big phone corporations. It was startling, you could actually feel in your bones whether a bit of the code-base was old or new somehow.
I remember when I wrote my bachelor thesis on type inference, my Pascal-to-Pascal 'compiler' was an extension of a Parser my supervisor programmed (in Java). It had a pretty good structure as far as I can remember, and for me who had never done any serious Object-oriented programming, it was quite a revelation.
I've been doing a lot of Eclipse plug-in development and often had to debug into the actual Eclipse source code. While I haven't "inherited" it in the sense that I'm not continuing work on it, I've always been impressed with the design and quality of the early core.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed last month.
Improve this question
Summary:
Can I program a "thick
client" game in C without reinventing
wheels, or should I just bite the
bullet and use some library or SDK?
I'm a moderate C programmer and am not
afraid to work with pointers, data
structures, memory locations, etc. if
it will give me the control I need to
make a great "thick-client" game.
However, I'm thinking of eschewing
high-level languages & frameworks for
the sake of power and control, not
ease of use.
I'm interesting in tinkering with a 2D fighting/platforming game as a side project sometime. I'm primarily a Linux server-side programmer with experience in Python, Ruby and PHP. I know that there are excellent frameworks in some of these languages, like PyGame. I am also aware of the success people have had with stuff like Air and .NET... but I have some concerns:
Performance: Scripting languages are notoriously slow. If I'm making a real-time game, I want it to be as snappy as possible.
Huge binaries: Using frameworks like .NET or scripting languages like Ruby often result in big CLRs or libraries that you wouldn't otherwise need. The game I want to make will be small and simple--I don't want its CLR to be bigger than the game itself!
Extra stuff: Honestly, I just don't like the idea of inheriting some big game library's baggage if I can wrap my head around my own code better.
I'm asking this question because I know I'm very susceptible to Not Invented Here Syndrome. I always want to program it myself, and I'm sure it wastes a lot of time. However, this works out for me remarkably often--for example, instead of using Rails (a very big web project framework with an ORM and GUI toolkit baked in), I used an array of smaller Ruby tools like rack and sequel that fit together beautifully.
So, I turn to you, SO experts. Am I being naive? Here's how I see it:
Use C
Cons
Will probably make me hate programming
High risk of reinventing wheels
High risk of it taking so long that I lose interest
Pros
Tried & true - most A-list games are done in C (is this still true today?)
High level of control over memory management, speed, asset management, etc., which I trust myself to learn to handle
No cruft
Use framework or SDK
Cons
Risk of oversized deliverable
Dependent on original library authors for all facets of game development--what if there isn't a feature I want? I'll have to program it myself, which isn't bad, but partially defeats the purpose of using a high-level framework in the first place
High risk of performance issues
Pros
MUCH faster development time
Might be easier to maintain
No time wasted reinventing common paradigms
What else can I add to this list? Is it a pure judgment call, or can someone seal the deal for me? Book suggestions welcome.
I believe you are working under a fallacy.
There are several frameworks out there specifically for game programming --- written by people with much experience with the complication of game design, almost certainly more tha you do.
In other words, you have a "High risk of performance issues" if you DON'T use a framework.
My current thinking is:
If you want to learn to program, start making the game engine from the base elements upwards (even implementing basic data structures - lists, maps, etc). I've done this once, and while it was a learning experience, I made many mistakes, and I wouldn't do this a second time around. However for learning how to program as well as making something cool and seeing results I'd rate this highly.
If you want to make a proper game, use whatever libraries that you want and design all of the game infrastructure yourself. This is what I'm doing now, and I'm using all of the nice things like STL, ATL/WTL, Boost, SQLite, DirectX, etc. So far I've learnt a lot about the middle/game logic aspect of the code and design.
If you just want to make a game with artists and other people collaborating to create a finished product, use one of the existing engines (OGRE, Irrlicht, Nebula, Torque, etc) and just add in your game logic and art.
One final bit of wisdom I've learnt is that don't worry about the Not Invented Here syndrome. As I've come to realise that other libraries (such as STL, Boost, DirectX, etc) have an order of magnitude (or three) more man-hours of development time in them, far more than I could ever spend on that portion of the game/engine. Therefore the only reason to implement these things yourself is if you want to learn about them.
I would recomend you try pyglet.
It has good performance, as it utilizes opengl
Its a compact all-in-one library
It has no extra dependencies besides python
Do some tests, see if you can make it fast enough for you. Only if you prove to yourself that it's not move to a lower level. Although, I'm fairly confident that python + pyglet can handle it... at worst you'll have to write a few C extensions.
Today, I believe you are at a point where you can safely ignore the performance issue unless you're specifically trying to do something that pushes the limits. If your game is, say, no more complicated than Quake II, then you should choose tools and libraries that let you do the most for your time.
Why did I choose Quake II? Because running in a version compiled for .NET, it runs with a software renderer at a more than acceptable frame rate on a current machine. (If you like - compare MAME which emulates multiple processors and graphics hardware at acceptable rates)
You need to ask yourself if you are in this to build an engine or to build a game. If your purpose is to create a game, you should definitely look at an established gaming engine. For 2D game development, look at Torque Game Builder. It is a very powerful 2D gaming engine/SDK that will put you into production from day 1. They have plenty of tools that integrate with it, content packs, and you get the full source code if you want to make changes and/or learn how it works. It is also Mac OSX compatible and has Linux versions in the community.
If you are looking for something on the console side, they have that too.
I'm surprised nobody has mentioned XNA. Its a framework built around DirectX for doing managed DirectX programming while removing a lot of the fluff and verbosity of lower level DirectX programming.
Performance-wise, for most 2D and 3D game tasks, especially building something like a fighting game, this platform works very well. Its not as fast as if you were doing bare metal DirectX programming, but it gets you very close, and in a managed environment, no less.
Another cool benefit of XNA is that most of the code can be run on an Xbox 360 and can even be debugged over the network connection was the game runs on the Xbox. XNA games are now allowed to be approved by the Xbox Live team for distribution and sale on Xbox Live Arcade as well. So if you're looking to take the project to a commercial state, you might have am available means of distribution at your disposal.
Like all MS development tools, the documentation and support is first rate, and there is a large developer community with plenty of tutorials, existing projects, etc.
Do you want to be able to play your game on a console? Do you want to do it as a learning experience? Do you want the final product to be cross platform? Which libraries have you looked into so far?
For a 2d game I don't think performance will be a problem, I recommend going with something that will get you results on screen in the shortest amount of time. If you have a lot of experience doing Python then pyGame is a good choice.
If you plan on doing some 3d games in the future, I would recommend taking a look at Ogre (http://www.ogre3d.org). It's a cross platform 3d graphics engine that abstracts away the graphics APIs. However for a 2d project it's probably overkill.
The most common implementation language for A-list games today is C++, and a lot of games embed a scripting language (such as Python or Lua) for game event scripting.
The tools you'd use to write a game have a lot to do with your reasons for writing it, and with your requirements. This is no different from any other programming project, really. If it's a side project, and you're doing it on your own, then only you can assess how much time you have to spend on this and what your performance requirements are.
Generally speaking, today's PCs are fast enough to run 2D platformers written in scripting languages. Using a scripting language will allow you to prototype things faster and you'll have more time to tweak the gameplay. Again, this is no different than with any other project.
If you go with C++, and your reasons don't have to be more elaborate than "because I want to," I would suggest that you look at SDL for rendering and audio support. It will make things a little bit easier.
If you want to learn the underlying technologies (DirectX, or you want to write optimized blitters for some perverse reason) then by all means, use C++.
Having said all that, I would caution you against premature optimization. For a 2D game, you'll probably be better off going with Python and PyGame first. I'd be surprised if those tools will prove to be inadequate on modern PCs.
As to what people have said about C/C++/Python, I'm a game developer and my company encourages C. Not b/c C++ is bad, but because badly written C++ is poison for game development due to it's difficulty to read/debug compared to C. (C++ gives benefits when used properly, but let a junior guy make some mistakes with it and your time sink is huge)
As to the actual question:
If your purpose is to just get something working, use a library.
Otherwise, code it yourself for a very important reason: Practice
Practice in manipulating data structures. There WILL be times you need to manage your own data. Practice in debugging utility code.
Often libs do just what you want and are great, but sometimes YOUR specific use case is handled very badly by the lib and you will gain big benefits from writing you own. This is especially on consoles compared to PCs
(edit:) Regarding script and garbage collection: it will kill you on a console, on a recent game I had to rewrite major portions of the garbage collection on Unreal just to fill our needs in the editor portion. Even more had to be done in the actual game (not just by me) (to be fair though we were pushing beyond Unreal's original specs)
Scripting often good, but it is not an "I win" button. In general the gains disappear if you are pushing against the limits of your platform. I would use "percent of platforms CPU that I have to spare" as my evaluation function in deciding how appropriate script is
One consideration in favor of C/C++/obj-C is that you can mix and match various libraries for different areas of concern. In other words, you are not stuck with the implementation of a feature in a framework.
I use this approach in my games; using chipmunk for 2D physics, Lua as an embedded scripting language, and an openGL ES implementation from Apple. I write the glue to tie all of these together in a C language. The final product being the ability to define game objects, create instances of them, and handle events as they interact with each other in C functions exposed to Lua. This approach is used in many high performance games to much success.
If you don't already know C++, I would definitely recommend you go forward with a scripting language. Making a game from start to finish takes a lot of motivation, and forcing yourself to learn a new language at the same time is a good way to make things go slowly enough that you lose interest (although it IS a good way to learn a new language...).
Most scripting languages will be compiled to byte code anyway, so their biggest performance hit will be the garbage collection. I'm not experienced enough to give a definite description of how big a hit garbage collection would be, but I would be inclined to think that it shouldn't be too bad in a small game.
Also, if you use an existing scripting language library to make your game, most of the performance critical areas (like graphics) can be written in C++ anyway (hopefully by the game libraries). So 80% of the CPU might actually be spent in C++ code anyway, despite the fact that most of your project is written in, say Python.
I would say, ask yourself what you want more: To write a game from start to finish and learn about game development, or to learn a new language (C++). If you want to write a game, do it in a scripting language. If you want to learn a new language, do it in C++.
Yeah unless you just want to learn all of the details of the things that go into making a game, you definitely want to go with a game engine and just focus on building your game logic rather than the details of graphics, audio, resource management, etc.
Personally I like to recommend the Torque Game Builder (aka Torque 2D) from GarageGames. But you can probably find some free game engines out there that will suit your needs as well.
I'm pretty sure most modern games are done in C++, not C. (Every gaming company I ever interviewed with asked C++ questions.)
Why not use C++ and existing libraries for physics + collisions, sound, graphics engine etc. You still write the game, but the mundane stuff is taken care of.
There are alot of different solutions to the issue of abstracting and each deals with it in different ways.
My current project uses C#, DirectX 9, HLSL and SlimDX. Each of these offers a carefully calibrated level of abstraction. HLSL allows me to actually read the shader code I'm writing and SlimDX/C# allows me to ignore pointers, circular dependencies and handling unmanaged code.
That said, none of these technologies has any impact on the ease of developing my AI, lighting or physics! I still have to break out my textbooks in a way that I wouldn't with a higher-level framework.
Even using a framework like XNA, if most video games development concepts are foreign to you there's a hell of a lot still to take in and learn. XNA will allow you to neatly sidestep gimbal lock, but woe betide those who don't understand basic shading concepts. On the other hand, something like DarkBASIC won't solve your gimbal lock problem, but shading is mostly handled for you.
It's a sufficiently big field that your first engine will never be the one you actually use. If you write it yourself, you won't write it well enough. If you use third party libraries, there's certainly aspects that will annoy you and you'll want to replace.
As an idea, it might be worth taking various libraries/frameworks (definately make XNA one of your stops, even if you decide you don't want to use it, it's a great benchmark) and trying to build various prototypes. Perhaps a landscape (with a body of water) or a space physics demo.