Using the really easy to follow instructions for building a NuGet package for an assembly with an associated package of sources for the symbol server, found here on David Ebbo's blog "The easy way to publish NuGet packages with sources" I have indeed created a pair of packages: binary and sources.
However, the sources package is incomplete and the reason is that the sources come from two class library .csproj and I used ILMerge to combine the results of the second into the first. (*) So, using the minimal .nuspec described in this post and pointing nuget.exe at the .csproj for the "main" library, the binary package is fine, but of course the sources package only has sources for the "main" library, not also for the library that was ILMerged into the "main" library.
How do I fix this (and get the sources for both projects included in the symbol package but only the binary for the "main" project in the binary package)?
FYI, the actual nuget.exe command line was: nuget pack CommandLineLexing.csproj -Build -Symbols -Properties Configuration=Release.
(*) The reason I'm doing this, in case you're interested, is that the second library is a cut down version of my accumulated "C# utilities" library - you know, a bunch of extension methods and other helpers - cut down so it only has the bare minimum needed for this particular project. And so, since it is cut down, I don't want there to be separate assembly for it which might eventually get confused with the full assembly (having the same name, and not a strong name). So I used ILMerge to put the utilities methods into the main assembly (and also mark them internal).
Not going to be easy I'm afraid.
NuGet symbol packages are simply your regular package, with pdbs, augmented with source files.
Assuming you already know you can get a merged PDB with ILMerge/ILRepack (/debug), that part is probably working file, I'm assuming your issue is that only the source files from the current project gets included.
You could simply post-process your symbol.nupkg (which is a zip), and include the source files from your other (merged) project in the src folder (you can even try that manually).
Though if you run srctool.exe -r MyMergedAssembly.pdb, you'll see different root paths, where usually (for a non-merged project) they all have a common prefix.
It may work, if SymbolSource copes with having multiple path prefixes in your PDB, that I haven't tried.
I also failed to find any documentation irt to their processing of symbol packages. We can assume they use pdbstr.exe tool to update the PDB srcsrv section of the PDB file to redirect the symbol loading to their website, but one can only tell if they support multiple roots by testing it.
If you upload your augmented symbol package to symbolsource, you can download the updated PDB using a URL similar to http://srv.symbolsource.org/pdb/Public/Castle.Core.pdb/4C81FC30DF584853B9869EAB2FA7D9891/Castle.Core.pd_ (then unzip it to a pdb file)
Then you can use both srctool.exe file.pdb and pdbstr.exe -r -s:srcsrv -p:file.pdb to verify their work.
Related
While debugging I can step into external dependencies and our own nuget packages, if they were compiled to symbol packages (MyProject.snupkg) and uploaded to the nuget server. This is how we do it for our own nuget packages on our private nuget server.
I discovered the Source Link tool, whose readme states
Source Link enables a great source debugging experience for your users, by adding source control metadata to your built assets
What is the difference between that and the setup I've described above? And is there a benefit to using both at the same time?
(I've found existing SO questions for configuring Source Link, but none about the difference to symbol packages.)
I consider Source Link as some kind of evolution from symbol packages.
With the symbol packages, you always had two files:
MyPackage.nupkg (contains the assembly itself)
MyPackage.snupkg (contains the debug symbols)
So basically the debug symbols are shipped as a separate NuGet package (the snupkg file) and the relationship to the underlying source code is broken.
Microsoft lists some more constraints (see here).
Now since a lot of source code is open anyway (e. g. on GitHub), why not link the NuGet package directly to the sources? That's what Source Link does: it bridges the gap between the referenced assembly and its source code, without relying on an intermediate file (like snupkg).
This enables true deterministic builds, i. e. one can verify which Git commit was used for an assembly of a NuGet package.
We are migrating over to using packages and NuGet for managing our dependencies on 3rd party components. This works well when referencing packages from within Visual Studio or building on the build server via msbuild.
However there are a number of files that we would like to access in our build scripts and installers. Previously these would be in source control with a well known path, now as the version of the package that we are consuming changes so the path to the package and hence the files is changing.
Is there a simple way I can get the path to a given package? The best solution I currently have is to search for all packages.config files and extract the package version from them.
Examples of the files that we need to access are
The NUnit console executable from the NUnit.Runners package for running unit tests.
License files from various packages that we redistribute with our installer.
Using the packages.config file is a pretty good solution. NuGet itself uses two approaches:
Reading the package information from the packages.config and using that to resolve to the packages path.
Enumerating all the directories in the packages directory.
You could use NuGet.Core to do either of the above if you do not want to write the code yourself. The classes that can be used are the DefaultPackagePathResolver, the PackageReferenceFile and LocalPackageRepository or SharedPackageRepository.
One problem with the second approach is that sometimes NuGet may occasionally leave behind NuGet packages that are not necessarily referenced by a project. In that case looking at the package directories may give you the incorrect information.
The only other approach I can think of might be to read the project files looking for the assembly references. Although that would not work for a solution level package such as NUnit.Runners.
Since I first saw a dist/ directory in many open source projects, usually on GitHub, I've been wondering what it means.
With dist, vendor, lib, src, and many other folder names that we see quite often, I sometimes wonder how I should name my own folders.
Correct me if I'm wrong!
src: Contains the sources. Sometimes only the pure sources, sometimes with the minified version, depends on the project.
vendor: Contains other dependencies, like other open source projects.
lib: Good question, it's really close to vendor actually, depending on the project we can see one or another or both...
dist: From what I saw, it contains the "production" files, the one we should use if we want to use the library.
Why is open source so confusing? Isn't it possible to do things clearer? At least per language because some languages use specific names.
To answer your question:
/dist means "distributable", the compiled code/library.
Folder structure varies by build system and programming language. Here are some standard conventions:
src/: "source" files to build and develop the project. This is where the original source files are located, before being compiled into fewer files to dist/, public/ or build/.
dist/: "distribution", the compiled code/library, also named public/ or build/. The files meant for production or public use are usually located here.
There may be a slight difference between these three:
build/: is a compiled version of your src/ but not a production-ready.
dist/: is a production-ready compiled version of your code.
public/: usually used as the files runs on the browser. which it may be the server-side JS and also include some HTML and CSS.
assets/: static content like images, video, audio, fonts etc.
lib/: external dependencies (when included directly).
test/: the project's tests scripts, mocks, etc.
node_modules/: includes libraries and dependencies for JS packages, used by Npm.
vendor/: includes libraries and dependencies for PHP packages, used by Composer.
bin/: files that get added to your PATH when installed.
Markdown/Text Files:
README.md: A help file which addresses setup, tutorials, and documents the project. README.txt is also used.
LICENSE.md: any rights given to you regarding the project. LICENSE or LICENSE.txt are variations of the license file name, having the same contents.
CONTRIBUTING.md: how to help out with the project. Sometimes this is addressed in the README.md file.
Specific (these could go on forever):
package.json: defines libraries and dependencies for JS packages, used by Npm.
package-lock.json: specific version lock for dependencies installed from package.json, used by Npm.
composer.json: defines libraries and dependencies for PHP packages, used by Composer.
composer.lock: specific version lock for dependencies installed from composer.json, used by Composer.
gulpfile.js: used to define functions and tasks to be run with Gulp.
.travis.yml: config file for the Travis CI environment.
.gitignore: Specification of the files meant to be ignored by Git.
To answer your original question about the meaning of the dist folder:
The shortform dist stands for distributable and refers to a directory where files will be stored that can be directly used by others without the need to compile or minify the source code that is being reused.
Example: If I want to use the source code of a Java library that someone wrote, then you need to compile the sources first to make use of it. But if a library author puts the already compiled version into the repository, then you can just go ahead. Such an already compiled version is saved into the dist directory.
Something similar applies to JavaScript modules. Usually JavaScript code is minified and obfuscated for use in production. Therefore, if you want to distribute a JavaScript library, it's advisable to put the plain (not minified) source code into an src (source) directory and the minified and obfuscated version into the dist (distributable) directoy, so others can grab the minified version right away without having to minify it themselves.
Note: Some developers use names like target, build or dest (destination) instead of dist. But the purpose of these folders is identical.
Summary of the folders:
bin: binaries
contrib: contributions to the project
dist: -- see 1. and 2.
doc/s: documentation
include: headers (C/C++)
lib: libraries (C/C++)
man: short for man/manual pages (Unix/Linux), c.f. man(1)
src: source
"/dist means "distributable", the compiled code/library." ref.
"The shortform dist stands for distributable and refers to a directory where files will be stored that can be directly used by others without the need to compile or minify the source code that is being reused." ref.
Actually! "dist folder" is the result you get after modifying a source code with "npm run build" or "ng build" or "ng build --prod" for production.
Meanwhile! After getting "dist folder" there might still be few things that you still need to do depending on your project type ✌️
I'm trying to create a NuGet package from a .csproj. I have successfully compiled the project and the output folder contains all of the necessary files (my assembly and all of its dependencies). However, NuGet only seems to be placing the assembly created by the .csproj into the package and not any of its dependencies. My command line looks like this:
nuget pack MyProject.csproj -Property Configuration=Release
and my resulting .nupkg file only has my assembly in the lib folder. I have successfully gotten NuGet to work for other projects, but it just so happens that this project is referencing the Enterprise Library logging block, but it was NOT retrieved via NuGet. I'm not sure if that could be related to my problem or not.
Any thoughts on why it's not picking up the dependenices?
If you need to keep your nuspec file up-to-date automatically, its really just an XML file (as I'm sure you know) so there are some very nice tools you can use from MSBuild to automate nuspec creation/updates. Out of the box, MSBuild provides a few tasks that can update or transform XML, and I've used MSBuild Community Tasks to customize the initial nuspec. For example, the default nuspec contains a few lines with broilerplate that I don't need, so I use XmlUpdate tasks to delete them.
Although I have not looked into scanning the csproj file for non-nuget references, I think its likely possible with a little research. Here are some links to blog entries describing my experiences with NuGet automation, they may help you get a head start:
Creating Packages with NuGet the MSBuild Way - This article includes some basic NuSpec updates because the package described is not that different from the type of package NuGet already knows how to automate.
Manage Your MEF Parts With Nuget - This article includes some more complex updates to support distributing MEF parts as runtime-only references.
If you plan on doing this alot, don't want to mess with MSBuild, or just want to get back the behavior you liked from the pre-1360 version of ProjectFactory.cs, NuGet supports third-party extension through MEF. You could go into the source control and grab the earlier code that you liked and create a custom command (for example custompack) that provides that behavior. Then you could use it from the command line like so:
nuget custompack MyProject.csproj -Property Configuration=Release
I think this is a really cool aspect of NuGet but I haven't played with it myself yet. Here is an article that explains how to do it:
Extend NuGet Command Line
So even though David mentioned that NuGet is not designed to support this scenario, if the scenario is correct for you then you can go this route to extend NuGet to meet your needs.
Say I have the following solution with multiple versions of the same code each targeting a different framework and I would like to generate a nuget package from it.
SharedLib.sln
SharedLib.Net35.csproj
packages.config
SharedLib.Net40.csproj
packages.config
SharedLib.Phone.csproj
packages.config
SharedLib.SL4.csproj
packages.config
The expected nupkg has the following structure
SharedLib.1.0.nupkg
lib/net35/SharedLib.dll
lib/net40/SharedLib.dll
lib/sl4-wp/SharedLib.dll
lib/sl4/SharedLib.dll
nuget.exe pack SharedLib.SL4.csproj will automatically determine that the target framework is SilverLight4 and place the binaries in lib/sl4
I know I can add a SharedLib.SL4.nuspec file with a <file> section to include binaries from the other projects but is there a way to make nuget automatically place the combined solution output into the proper structure (and also detect dependencies in packages.config from all projects?
No, there's currently no way to do this other than to write a custom build script that puts the files in the right place and then runs NuGet pack on them, or to take the .nuspec approach you described.
This is a feature we'd like to have, but haven't thought of a good way to do it. However, your post just gave me an idea.
Today, you can point nuget pack at a .csproj file.
We could consider an approach that allowed you to point it at a .sln file and if the project names follow some convention, we'd package all the projects into a single package.
If you really want this feature, consider logging an issue in the NuGet issue tracker. http://nuget.codeplex.com/workitem/list/basic