Since I first saw a dist/ directory in many open source projects, usually on GitHub, I've been wondering what it means.
With dist, vendor, lib, src, and many other folder names that we see quite often, I sometimes wonder how I should name my own folders.
Correct me if I'm wrong!
src: Contains the sources. Sometimes only the pure sources, sometimes with the minified version, depends on the project.
vendor: Contains other dependencies, like other open source projects.
lib: Good question, it's really close to vendor actually, depending on the project we can see one or another or both...
dist: From what I saw, it contains the "production" files, the one we should use if we want to use the library.
Why is open source so confusing? Isn't it possible to do things clearer? At least per language because some languages use specific names.
To answer your question:
/dist means "distributable", the compiled code/library.
Folder structure varies by build system and programming language. Here are some standard conventions:
src/: "source" files to build and develop the project. This is where the original source files are located, before being compiled into fewer files to dist/, public/ or build/.
dist/: "distribution", the compiled code/library, also named public/ or build/. The files meant for production or public use are usually located here.
There may be a slight difference between these three:
build/: is a compiled version of your src/ but not a production-ready.
dist/: is a production-ready compiled version of your code.
public/: usually used as the files runs on the browser. which it may be the server-side JS and also include some HTML and CSS.
assets/: static content like images, video, audio, fonts etc.
lib/: external dependencies (when included directly).
test/: the project's tests scripts, mocks, etc.
node_modules/: includes libraries and dependencies for JS packages, used by Npm.
vendor/: includes libraries and dependencies for PHP packages, used by Composer.
bin/: files that get added to your PATH when installed.
Markdown/Text Files:
README.md: A help file which addresses setup, tutorials, and documents the project. README.txt is also used.
LICENSE.md: any rights given to you regarding the project. LICENSE or LICENSE.txt are variations of the license file name, having the same contents.
CONTRIBUTING.md: how to help out with the project. Sometimes this is addressed in the README.md file.
Specific (these could go on forever):
package.json: defines libraries and dependencies for JS packages, used by Npm.
package-lock.json: specific version lock for dependencies installed from package.json, used by Npm.
composer.json: defines libraries and dependencies for PHP packages, used by Composer.
composer.lock: specific version lock for dependencies installed from composer.json, used by Composer.
gulpfile.js: used to define functions and tasks to be run with Gulp.
.travis.yml: config file for the Travis CI environment.
.gitignore: Specification of the files meant to be ignored by Git.
To answer your original question about the meaning of the dist folder:
The shortform dist stands for distributable and refers to a directory where files will be stored that can be directly used by others without the need to compile or minify the source code that is being reused.
Example: If I want to use the source code of a Java library that someone wrote, then you need to compile the sources first to make use of it. But if a library author puts the already compiled version into the repository, then you can just go ahead. Such an already compiled version is saved into the dist directory.
Something similar applies to JavaScript modules. Usually JavaScript code is minified and obfuscated for use in production. Therefore, if you want to distribute a JavaScript library, it's advisable to put the plain (not minified) source code into an src (source) directory and the minified and obfuscated version into the dist (distributable) directoy, so others can grab the minified version right away without having to minify it themselves.
Note: Some developers use names like target, build or dest (destination) instead of dist. But the purpose of these folders is identical.
Summary of the folders:
bin: binaries
contrib: contributions to the project
dist: -- see 1. and 2.
doc/s: documentation
include: headers (C/C++)
lib: libraries (C/C++)
man: short for man/manual pages (Unix/Linux), c.f. man(1)
src: source
"/dist means "distributable", the compiled code/library." ref.
"The shortform dist stands for distributable and refers to a directory where files will be stored that can be directly used by others without the need to compile or minify the source code that is being reused." ref.
Actually! "dist folder" is the result you get after modifying a source code with "npm run build" or "ng build" or "ng build --prod" for production.
Meanwhile! After getting "dist folder" there might still be few things that you still need to do depending on your project type ✌️
Related
I am interested in understanding whether bazel can handle "two stage builds", where dependencies are discovered based on the file contents and dependencies must be compiled before the code that depends on them (unlike C/C++ where dependencies are mostly header files that are not separately compiled). Concretely, I am building the Coq language which is like Ocaml.
My intuition for creating a build plan would use an (existing) tool (called coqdep) that reads a .v file and returns a list of all of its direct dependencies. Here's the algorithm that I have in mind:
invoke coqdep on the target file and (transitively) on each of its dependent files,
once transitive dependencies for a target are computed, add a rule to build the .vo from the .v that includes transitive dependencies.
Ideally, the calls to coqdep (in step 1) would be cached between builds and so only need to be re-computed when the file changes. And the transitive closure of the dependency information would also be cached.
Is it possible to implement this in bazel? Are there any pointers to setting up builds for languages like this? Naively, it seems to be a two-stage build and I'm not sure how this fits into bazel's compilation model. When I looked at the rules for Ocaml, it seemed like it was relying on ocamlbuild to satisfy the build order and dependency requirements rather than doing it "natively" in bazel.
Thanks for any pointers or insights.
(don't have enough rep to comment yet, so this is an answer)
#2 of Toraxis' answer is probably the most canonical.
gazelle is an example of this for Golang, which is in the same boat: dependencies for Golang files are determined outside a Bazel context by reading the import statements of source files. gazelle is a tool that writes/rewrites Golang rules in BUILD files according to the imports in source files of the Bazel workspace. Similar tools could be created for other languages that follow this pattern.
but the generated BUILD file will be in the output folder, not in the source folder. So you also have to provide an executable that copies the files back into the source folder.
Note that binaries run via bazel run have the environment variable BUILD_WORKSPACE_DIRECTORY set to the root of the Bazel workspace (see the docs) so if your tool uses this environment variable, it could edit the BUILD files in-place rather than generating and copying back.
(In fact, the generating-and-copying-back strategy would likely not be feasible, because purely-generated files would contain only Coq rules, and not any other types of rules. To generate a BUILD file with Coq rules from one with other types of rules, one would have to add the BUILD files themselves as dependencies - which would create quite the mess!)
I'm looking into similar questions because I want to build ReasonML with Bazel.
Bazel computes the dependencies between Bazel targets based on the BUILD files in your repository without accessing your source files. The only interaction you can do with the file system during this analysis phase is to list directory contents by using glob in your rule invocations.
Currently, I see four options for getting fine-grained incremental builds with Bazel:
Spell out the fine-grained dependencies in hand-written BUILD files.
Use a tool for generating the BUILD files. You cannot directly wrap that tool in a Bazel rule to have it run during bazel build because the generated BUILD file would be in the output folder, not in the source folder. But you can run rules that call coqdep during the build, and provide an executable that edits the BUILD file in the source folder based on the (cacheable) result of the coqdep calls. Since you can read both the source and the output folder during the build, you could even print a message to the user if they have to run the executable again. Anyway, the full build process would be bazel run //tools/update-coq-build-files && bazel build to reach a fixed point.
Have coarse-grained dependencies in the BUILD files but persistent workers to incrementally rebuild individual targets.
Have coare-grained dependencies in the BUILD files but generate a separate action for each target file and use the unused_inputs_list argument of ctx.actions.run to communicate to Bazel which dependencies where actually unused.
I'm not really sure whether 3 and 4 would actually work or how much effort would be involved, though.
Using the really easy to follow instructions for building a NuGet package for an assembly with an associated package of sources for the symbol server, found here on David Ebbo's blog "The easy way to publish NuGet packages with sources" I have indeed created a pair of packages: binary and sources.
However, the sources package is incomplete and the reason is that the sources come from two class library .csproj and I used ILMerge to combine the results of the second into the first. (*) So, using the minimal .nuspec described in this post and pointing nuget.exe at the .csproj for the "main" library, the binary package is fine, but of course the sources package only has sources for the "main" library, not also for the library that was ILMerged into the "main" library.
How do I fix this (and get the sources for both projects included in the symbol package but only the binary for the "main" project in the binary package)?
FYI, the actual nuget.exe command line was: nuget pack CommandLineLexing.csproj -Build -Symbols -Properties Configuration=Release.
(*) The reason I'm doing this, in case you're interested, is that the second library is a cut down version of my accumulated "C# utilities" library - you know, a bunch of extension methods and other helpers - cut down so it only has the bare minimum needed for this particular project. And so, since it is cut down, I don't want there to be separate assembly for it which might eventually get confused with the full assembly (having the same name, and not a strong name). So I used ILMerge to put the utilities methods into the main assembly (and also mark them internal).
Not going to be easy I'm afraid.
NuGet symbol packages are simply your regular package, with pdbs, augmented with source files.
Assuming you already know you can get a merged PDB with ILMerge/ILRepack (/debug), that part is probably working file, I'm assuming your issue is that only the source files from the current project gets included.
You could simply post-process your symbol.nupkg (which is a zip), and include the source files from your other (merged) project in the src folder (you can even try that manually).
Though if you run srctool.exe -r MyMergedAssembly.pdb, you'll see different root paths, where usually (for a non-merged project) they all have a common prefix.
It may work, if SymbolSource copes with having multiple path prefixes in your PDB, that I haven't tried.
I also failed to find any documentation irt to their processing of symbol packages. We can assume they use pdbstr.exe tool to update the PDB srcsrv section of the PDB file to redirect the symbol loading to their website, but one can only tell if they support multiple roots by testing it.
If you upload your augmented symbol package to symbolsource, you can download the updated PDB using a URL similar to http://srv.symbolsource.org/pdb/Public/Castle.Core.pdb/4C81FC30DF584853B9869EAB2FA7D9891/Castle.Core.pd_ (then unzip it to a pdb file)
Then you can use both srctool.exe file.pdb and pdbstr.exe -r -s:srcsrv -p:file.pdb to verify their work.
I've created a new foundation project using the foundation new myproject --libsass method.
Its just a simple static index.html (for the purpose of this question).
I've deployed it to a live server now, and I am wondering about the best way to structure this. I have omitted the node_modules & scss directories, and i'm left with the following:
bower_components/
css/
js/
index.html
bowerrc
bower.json
Gruntfile.js
humans.txt
package.json
README.md
bower_components is needed in its current form, unless I shuffle some files around, which is what I intend to do, but I'm checking if there is a better way of doing this, and that I haven't missed some magic terminal command to deploy to production.
As msturdy suggested in the comments, grunt is the way to go here. There are a lot of plugins out there, see a list on the official page.
If you want to have grunt "compile" your project into one specific folder which you can then for example push to a deployment server, you should do several things in grunt:
compile your scss
minify your javascript, that is making the files smaller, see jscompress for a demo of what it is. All your js-files from /bower_components which you include in your project should be in a vendor.js which is loaded first, and then a second js-file should contain your custom js from /js
save everything into a deployment folder.
(optional) automatically deploy to a server.
Take a look at these grunt tutorials for setting up your gruntfile.js, you'll want to have two tasks, one for just quickly compiling your scss, one for the whole deployment process:
grunt is not weird and hard
official tutorial
sitepoint tutorial
They show you how to do certain things and definitely how to write the gruntfile. Plugins you might want to use apart from your current libsass plugin are uglify and any plugin that lets you deploy your code via git, ftp or anything else. You can download all these plugins via npm by adding them to your package.json and doing npm install, refer to their websites for exact usage instruction.
I have a large project that is rather heterogenous - different languages and compilers are involved, altogether producing a build with the help of GNU make.
The project folder structure includes:
project
src
haxe --Haxe source code
graphics --Embeddable graphic resources
locale --Locale-specific resources
chinese --Chinese language resources
english --Generic English resources
build
china --Chinese market
debug --Debugging/Testing for developer(s)
release --Release
europe --European market
debug -- ...
release -- ...
All builds are produced by setting up and running 'make'. What I can't decide on is whether these makefiles should be put in 'src' directory as well? I generally consider original material I write by hand to be source code (since by my line of thought it originated from me and not produced by any program from some other input.) and I DO write my makefiles by hand. Another reason I consider this is because ONLY 'src' directory is a Git repository - I don't really need to version track anything else. Do I put all Makefiles into 'src'?
What's currently in your build directory structure? Is that the compiled output?
Intuitively (and I may be coming from a very different development world than you, so "intuitively" is a relative term) when I see a build folder adjoining a src folder, I expect the program's source code to be in the latter and the scripts/tools/etc. needed to build it to be in the former. The scripts (makefiles in this case, though potentially also to include other things) are themselves source code, as you state, but aren't the program's source. The distinction is that one is "what is being built" and the other is "how to build it."
If I understand you correctly, src is what's bound to your source control and build isn't? Under that circumstance, I would probably create a build (or builder or building or something of that nature) under src to house the scripts. It may be slightly unintuitive that it has to climb another folder before producing its output, but it should sit nicely alongside the resource folders you have there already.
Standard practice is to create a Makefile outside your src/ directory, which builds your project and within that src/ directory, another Makefile builds individual modules. That said, I think in your case it is sufficient to keep your Makefile solely in src/ though. I'm not sure if this applies, but you might want to look at the GNU autoconf package, it's used for exactly this type of thing.
in my Java Eclipse project that contains JUnit tests, I also have a package "resource" that contains all input data used for the tests. But when compiling JUnit tests, the Java compile also data available in resources, so I find the same data in the "bin" folder. Is there a way to avoid this?
thanks.
If you have a particular package within the source path you want to exclude (your resources folder for example), you can right click on the package and select: Build Path > Exclude.
This will tell Eclipse that you don't want to include that package as part of the build.
This is making a couple of assumptions: that you're using Eclipse Helios (because the option might be different in older versions), and that the resources are stored in the same folder as your regular java source files (because if resources is in a folder by itself, you can remove that entire folder from the build by using Build Path > Configure Build Path -> Source tab.
Update:
After the discussion in the comments regarding why you would or would not want to copy resources into the bin directory:
The contents of your bin directory should be ignored and not checked into to a version control system (when using CVS, bin should be an entry in the .cvsignore file)
The resources are only duplicated on your local machine, which is fast and hard discs are big. I'm not sure you should be worrying about this
If you're using Class.getResource to access those resources, they need to be on the classpath somewhere. The bin directory is as good a place as any
So, realistically (barring some unknown, like the files are hundreds of gigabytes or something), I don't think you need to be concerned about excluding these files from the build.