I remember vaguely that i once saw an article about improving build times with the use of assembly files being built separatly and something about namespaces dividing the project to parts that will be compiled separatly.
again some about ASM files.
I can't find it anywhere. anyone know?
Build times are killing my pace, i need to speed up.
any help will be appreciated
Although you said build times. As you mentioned assembly definitions I will assume that you are referring to compile time and domain reload time.
Depending on your project Unity can generate up to 4 different assemblies for you. By dividing up your codebase into custom assemblies and not relying on the assemblies to be generated by Unity you can decrease the amount of code needed to recompile through incremental compilation. However this will not reduce domain reload time which is the usual culprit for high "compile times".
In order to reduce your domain reload time it is best to check what is causing your high domain reload times with the Editor Iteration Profiler. You can also start having a look at scripts that use InitializeOnLoad since that also gets called every time the domain is reloaded.
Some resources that expand upon what I mentioned:
https://gamejolt.com/p/tales-of-kulplex-devlog-6-attempting-to-decrease-compile-times-she9y7qf
https://johnaustin.io/articles/2020/domain-reloads-in-unity
Related
I've looked at Microsoft's MSDN and all around the web, but I still haven't been able to get a really good idea of what it is.
Does it mean the completed program loads DLLs at different times during its execution, as apposed to all at once upon launch?
Am I totally way off? :)
Linking involves packaging together all of the .obj files built from your source files, as well as any .lib files you reference, into your output (eg .exe or .dll).
Without incremental linking, this has to be done from scratch each time.
Incremental linking links your exe/dll in a way which makes it easier for the linker to update the existing exe/dll when you make a small change and re-compile.
So, incremental linking just makes it faster to compile and link your project.
The only runtime effect it might have is that it may make your exe/dll slightly bigger and slower, as decribed here:
http://msdn.microsoft.com/en-us/library/4khtbfyf.aspx
Edit: As mentioned by Logan, incremental linking is also incompatible with link time code generation - therefore losing a possible performance optimization.
You may want to use incremental linking for debug builds to speed development, but disable it for release builds to improve runtime performance.
Delay loaded DLLs may be what you are thinking of:
http://msdn.microsoft.com/en-us/library/151kt790.aspx
Also, quite importantly, incremental link is a prerequisite for Edit&Continue - possibily to edit your code and recompile it on the fly, without restarting.
So it is a good thing to have on debug builds, but not release builds.
I am building up some statistics after analyzing the source code in eclipse. But the overall process is too slow because i rebuild my model every time from scratch after each compilation.
I am looking for a way to get only the changed parts of the code (as ASTNodes) and to rebuild just that part of my model. I suppose that even the changed compilation units and not the exact code elements would be enough after the user compiles and still would be a nice optimization.
I am sure eclipse is capable of knowing what code elements are changed (and even to know their semantics), because when I use the subclipse plugin my changes are ordered by a code element (an import, a method, a variable declaration, etc). Well.. at least that plugin is capable of knowing that info.
Thanks in advance
The Eclipse builder infrastructure is created for exactly this reason. For a start I suggest the following article and FAQ entry.
I have an application that uses several configuration files (let just consider appli.properties here).
These files contain several values that depend on the environment. We can find some information such as:
server.port=${envi.server.port}
On other side, I have a set of properties files, one per environment (dev.properties, homolo.properties, etc.).
They contain the values for some properties in configuration files. We can find here this kind of properties:
envi.server.port=4242
My build is handled by Maven2. Everything is working fine.
However, I now need to import my project into Eclipse.
My main concern is about the configuration files filtering. Indeed, if I do not modify anything in my Eclipse parameter for my project (after a mvn eclipse:eclipse command), then all my configuration file will keep the property keys (i.e. ${envi.server.port}) instead of their values. And with such configuration files, my application will not run inside Eclipse...
So I tried two solutions:
A full-Maven solution, using m2eclipse plugin. I add a Maven Builder in the project configuration, and then, each time a build is made, the filtering is done on the files.
Ant (which is only used inside Eclipse). I've hardly defined a task that simulates the Maven2 filtering of files in Ant. This task is only dedicated to the filtering, no compilation.
The common problem of these two solutions is that the filtering is made at every operation (essentially saves on Java class edition), and then take time. The second solution is however quicker (3 seconds) than the first one (more than 10 seconds).
What do you think of my approach?
How would you do that, in a better way?
If the resources are not changed that often, you can set the Maven build to only run after a Clean build, then it won't interfere so much, this doesn't do anything to speed up the build however.
As far as making the filtering quicker, I don't know of any other simple mechanism that will help, as you've said you need either Ant or Maven to run the filtering, and they both take some time to set up before building, resulting in the slow down.
If this is causing you a lot of problems, you can write a custom Incremental Eclipse builder that performs the filtering on the deltas. This should be considerably quicker, but obviously a lot more effort to write.
In order to provide nice URLs between parts of our app we split everything up into several modules which are compiled independently. For example, there is a "manager" portion and an "editor" portion. The editor launches in a new window. By doing this we can link to the editor directly:
/com.example.EditorApp?id=1
The EditorApp module just gets the value for id and loads up the document.
The problem with this is ALL of the code which is common between the two modules is duplicated in the output. This includes any static content (graphics), stylesheets, etc.
And another problem is the compile time to generate JavaScript is nearly double because we have some complex code shared between both modules which has to be processed twice.
Has anyone dealt with this? I'm considering scrapping the separate modules and merging it all back into one compile target. The only drawback is the URLs between our "apps" become something like:
/com.example.MainApp?mode=editor&id=1
Every window loads the main module, checks the value of the mode parameter, and and calls the the appropriate module init code.
I have built a few very large applications in GWT, and I find it best to split things up into modules, and move the common code into it's own area, like you've done. The reason in our case was simple, we had some parts of our application that were very different to the rest, so it made sense from a compile size point of view. Our application compiled down to 300kb for the main section, and about 25-40kb for other sections. Had we just put them all in one the user would have been left with a 600kb download, which for us was not acceptable.
It also makes more sense from a design and re-usability point of view to seperate things out as much as possible, as we have since re-used a lot of modules that we built on this project.
Compile time is not something you should generally worry about, because you can actually make it faster if you have seperate modules. We use ant to build our project, and we set it to only compile the GWT that has changed, and during development to only build for one browser, typical compile times on our project are 20 seconds, and we have a lot of code. You can see and example of this here.
One other minor thing: I assume you know that you don't have to use the default GWT paths that it generates? So instead of com.MyPackage.Package you could just put it into a folder with a nice name like 'ui' or something. Once compiled GWT doesn't care where you put it, and is not sensitive to path changes, because it all runs from the same directory.
From my experience building GWT apps, there's a few things to consider when deciding on whether you want multiple modules (with or without entry points), or all in one: download time (Javascript bundle size), compile time, navigation/url, and maintainability/re-usability.
...per download time, code splitting pretty much obviates the need to break into different modules for performance reasons.
...per compile time, even big apps are pretty quick to compile, but it might help breaking things up for huge apps.
...per navigation/url, it can be a pain to navigate from one module to another (assuming different EntryPoints), since each module has it's own client-side state...and navigation isn't seamless across modules.
...per maintainability/re-usability, it can be helpful from an organization/structure perspective to split into separate modules (even if there's only one EntryPoint).
I wrote a blog post about using GWT Modules, in case it helps.
Ok. I really get the sense there really is no "right" answer because projects vary so much. It's very much dependent on the nature of the application.
Our main build is composed of a number of in-house modules and 3rd party modules. They are all managed in seperate projects. That makes sense since they are used in different places.
But having more than one module in a single project designed to operate as one complete application seems to have overcomplicated things. The original reason for the two modules was to keep the URL simple when opening different screens in a new window. Even though had multiple build targets they all use a very large common subset of code (including a custom XML/POJO marshalling library).
About size... for us, one module was 280KB and the other was just over 300KB.
I just got finished merging everything back into one single module. The new combined module is around 380KB. So it's actually a bit less to download since most everyone would use both screens.
Also remember there is perfect caching, so that 380KB should only ever downloaded once, unless the app is changed.
I run a rather complex project with several independent applications. These use however a couple of shared components. So I have a source tree looking something like the below.
My Project
Application A
Shared1
Shared2
Application B
Application C
All applications have their own MSBuild script that builds the project and all the shared resources it needs. I also run these builds on a CruiseControl controlled continuous integration build server.
When the applications are deployed they are deployed on several servers to distribute load. This means that it’s extremely important to keep track of what build/revision is deployed on each of the different servers (we need to have the current version in the DLL version, for example “1.0.0.68”).
It’s equally important to be able to recreate a revision/build that been built to be able to roll back if something didn’t work out as intended (o yes, that happends ...). Today we’re using SourceSafe for source control but that possible to change if we could present good reasons for that (SS it’s actually working ok for us so far).
Another principle that we try to follow is that it’s only code that been build and tested by the integration server that we deploy further.
"CrusieControl Build Labels" solution
We had several ideas on solving the above. The first was to have the continuous integration server build and locally deploy the project and test it (it does that now). As you probably know a successful build in CruiseControl generates a build label and I guess we somehow could use that to set the DLL version of our executables (so build label 35 would create a DLL like “1.0.0.35” )? The idea was also to use this build label to label the complete source tree. Then we probably could check out by that label and recreate the build later on.
The reason for labeling the complete tree is to include not only the actual application code (that’s in one place in the source tree) but also all the shared items (that’s in different places in the tree). So a successful build of “Application A” would label to whole tree with label “ApplicationA35” for example.
There might however be an issue when trying to recreate this build and setting the DLL version before deploying as we then don’t have access to the CruiseControl generated build label anymore. If all CrusieControl build labels were unique for all the projects we could use only the number for labeling but that’s not the case (both application A and B could at the same time be on build 35) so we have to include the application name in the label. Hence SourceSafe label “Application35”. How can I then recreate build 34 and set 1.0.0.34 to the DLL version numbers once we built build 35?
"Revision number" solution
Someone told me that Subversion for example creates a revision number for the entire source tree on every check in – is this the case? Has SourceSafe something similar? If this is correct the idea is then to grab that revision number when getting latest and build on the CruiseControl server. The revision number could then be used to set the DLL version number (to for example “1.0.0.5678”). I guess we could then get this specific revision for the Subversion if needed and that then would include that application and all the shared items to be able to recreate a specific version from the past. Would that work and could this also be achived using SourceSafe?
Summarize
So the two main requirements are:
Be able to track build/revision number of the build and deployed DLL.
Be able to rebuild a past revision/build, set the old build/revision number on the executables of that build (to comply with requirement 1).
So how would you solve this? What would be your preferred approach and how would you solve it (or do you have a totally different idea?)? **Pleased give detailed answers. **
Bonus question What are the difference between a revision number and a build number and when would one really need both?
Your scheme is sound and achievable in VSS (although I would suggest you consider an alternative, VSS is really an outdated product).
For your "CI" Build - you would do the Versioning take a look at MSBuild Community Tasks Project which has a "Version" tasks. Typically you will have a "Version.txt" in your source tree and the MSBuild task will increment the "Release" number while the developers control the Major.Minor.Release.Revision numbers (that's how a client of mine wanted it). You can use revision if you prefer.
You then would have a "FileUpdate" tasks to edit the AssemblyInfo.cs file with that version, and your EXE's and "DLL's" will have the desired version.
Finally the VSSLabel task will label all your files appropriately.
For your "Rebuild" Build - you would modify your "Get" to get files from that Label, obviously not execute the "Version" task (as you are SELECTING a version to build) and then the FileUpdate tasks would use that version number.
Bonus question:
These are all "how you want to use them" - I would use build number for, well the build number, that is what I'd increment. If you are using CI you'll have very many builds - the vast majority with no intention of ever deploying anywhere.
The major and minor are self evident - but revision I've always used for a "Hotfix" indicator. I intend to have a "1.3" release - which would in reality be a product with say 1.3.1234.0 version. While working on 1.4 - I find a bug - and need a hot fix as 1.3.2400.1. Then when 1.4 is ready - it would be say 1.4.3500.0
I need more space than responding as comments directly allows...
Thanks! Good answer! What would be the
difference, what would be better
solving this using SubVersion for
example?Richard Hallgren (15 hours
ago)
The problems with VSS have nothing to do with this example (although the "Labeling" feature I believe is implemented inefficiently...)
Here are a few of the issues with VSS
1) Branching is basically impossible
2) Shared checkout is generally not used (I know of a few people who have had success with it)
3) performance is very poor - it is exteremly "chatty"
4) unless you have a very small repository - it is completely unreliable, to the point for most shops it's a ticking timebomb.
For 4 - the problem is that VSS is implemented by the entire repository being represented as "flat files" in the file system. When the repository gets over a certain size (I believe 4GB but I'm not confident in that figure) you get a chance for "corruption". As the size increases the chances of corruption grow until it becomes an almost certainty.
So take a look at your repository size - and if you are getting into the Gigabytes - I'd strongly recommend you begin planning on replacing VSS.
Regardless - a google of "VSS Sucks" gives 30K hits... I think if you did start using an alterantive - you will realize it's well worth the effort.
Have CC.net label the successful builds
have each project in the solution link to a common solutioninfo.cs file which contains assembly and file version attributes (remove from each projects assemblyinfo.cs)
Before the build have cruise control run an msbuild regex replace (from msbuild community tasks) to update the version information using the cc.net build label (passed in as a parameter to the msbuild task)
build the solution, run tests, fx cop etc
Optionally revert the solution info file
The result is that all assemblies in the cc.net published build have the same version numbers which conform to a label in the source code repository
UppercuT can do all of this with a custom packaging task to split the applications up. And to get the version number of the source, you might think about Subversion.
It's also insanely easy to get started.
http://code.google.com/p/uppercut/
Some good explanations here: UppercuT