How do you determine the entire dependency tree for a given script in powershell? If your target script dot sources a number of files, which themselves dot source other files, what is the quickest way to identify and only deploy that tree?
Related
I am interested in understanding whether bazel can handle "two stage builds", where dependencies are discovered based on the file contents and dependencies must be compiled before the code that depends on them (unlike C/C++ where dependencies are mostly header files that are not separately compiled). Concretely, I am building the Coq language which is like Ocaml.
My intuition for creating a build plan would use an (existing) tool (called coqdep) that reads a .v file and returns a list of all of its direct dependencies. Here's the algorithm that I have in mind:
invoke coqdep on the target file and (transitively) on each of its dependent files,
once transitive dependencies for a target are computed, add a rule to build the .vo from the .v that includes transitive dependencies.
Ideally, the calls to coqdep (in step 1) would be cached between builds and so only need to be re-computed when the file changes. And the transitive closure of the dependency information would also be cached.
Is it possible to implement this in bazel? Are there any pointers to setting up builds for languages like this? Naively, it seems to be a two-stage build and I'm not sure how this fits into bazel's compilation model. When I looked at the rules for Ocaml, it seemed like it was relying on ocamlbuild to satisfy the build order and dependency requirements rather than doing it "natively" in bazel.
Thanks for any pointers or insights.
(don't have enough rep to comment yet, so this is an answer)
#2 of Toraxis' answer is probably the most canonical.
gazelle is an example of this for Golang, which is in the same boat: dependencies for Golang files are determined outside a Bazel context by reading the import statements of source files. gazelle is a tool that writes/rewrites Golang rules in BUILD files according to the imports in source files of the Bazel workspace. Similar tools could be created for other languages that follow this pattern.
but the generated BUILD file will be in the output folder, not in the source folder. So you also have to provide an executable that copies the files back into the source folder.
Note that binaries run via bazel run have the environment variable BUILD_WORKSPACE_DIRECTORY set to the root of the Bazel workspace (see the docs) so if your tool uses this environment variable, it could edit the BUILD files in-place rather than generating and copying back.
(In fact, the generating-and-copying-back strategy would likely not be feasible, because purely-generated files would contain only Coq rules, and not any other types of rules. To generate a BUILD file with Coq rules from one with other types of rules, one would have to add the BUILD files themselves as dependencies - which would create quite the mess!)
I'm looking into similar questions because I want to build ReasonML with Bazel.
Bazel computes the dependencies between Bazel targets based on the BUILD files in your repository without accessing your source files. The only interaction you can do with the file system during this analysis phase is to list directory contents by using glob in your rule invocations.
Currently, I see four options for getting fine-grained incremental builds with Bazel:
Spell out the fine-grained dependencies in hand-written BUILD files.
Use a tool for generating the BUILD files. You cannot directly wrap that tool in a Bazel rule to have it run during bazel build because the generated BUILD file would be in the output folder, not in the source folder. But you can run rules that call coqdep during the build, and provide an executable that edits the BUILD file in the source folder based on the (cacheable) result of the coqdep calls. Since you can read both the source and the output folder during the build, you could even print a message to the user if they have to run the executable again. Anyway, the full build process would be bazel run //tools/update-coq-build-files && bazel build to reach a fixed point.
Have coarse-grained dependencies in the BUILD files but persistent workers to incrementally rebuild individual targets.
Have coare-grained dependencies in the BUILD files but generate a separate action for each target file and use the unused_inputs_list argument of ctx.actions.run to communicate to Bazel which dependencies where actually unused.
I'm not really sure whether 3 and 4 would actually work or how much effort would be involved, though.
I am writing a PowerShell module that will perform various configurations on our product including some xml transformations.
So, I need to include the XDT files in my module.
My module manifest will include the FileList attribute with a list of all the transformation files.
But the question is, how do I reference these files in the functions in the module?
Didn't find much in the documentation except that the FileList is just an inventory.
I thought of creating a variable with a path to the files (dynamic of course, depending on where the module is installed) but there must be a way to use these files...
This is a script module, not binary module and intended for V5
Thanks!
I'm building a simple webserver using Swift3 and the Swift Package Manager. Running swift build in your project folder will build all of the sources found in the Sources directory and output an executable. However, I've found that if I create folders within the Sources directory to organize my code, my builds fail. It looks to me like the presence of folders causes swift to treat the source as different modules. Is this the cause, and if so, how can I work around it?
Yes. Putting directories directly under Sources will cause SwiftPM to interpret those as modules. This is described in the package manager reference.
To work around this, use another level of indirection: put a directory for your module inside Sources and your additional directories inside that directory:
Sources/YourApp/Stuff/Source1.swift
Sources/YourApp/Stuff/Source2.swift
Sources/YourApp/MoreStuff/Source3.swift
Sources/YourApp/MoreStuff/Source4.swift
I have a NAnt task create copies of MSBuild files. The copies are located in other directories than their originals.
The source files should remain where they are. Therefore, I'm using NAnt's <style> task to run an XSLT on the MSBuild files. It replaces the <Compile/> elements with <Compile><Link/></Compile> subtrees (attributes omitted for the sake of legibility).
I've encountered the following problem:
In the resulting MSBuild files, the Include attribute of the Compile elements should receive the relative path to the source file from the new location of the MSBuild files. As the original MSBuild files contain the relative paths to the files, what I am actually looking for is the relative path from the new MSBuild file location to the location of the original MSBuild files.
Workarounds that I know how to use, but which I'd like to avoid:
custom NAnt task
using substring to remove the mutual prefix of the directory paths (the NAnt base dir)
Is there any better way; possibly a NAnt function or an ingenious way to use several NAnt functions in conjunction to achieve this?
Trying to understand how EDE works by using it to generate Makefiles for
a project directory that contains several targets under a specific
hierarchy. I'm not having any luck, and the info pages don't seem to
answer my question.
My directory structure looks something like:
(The asterix (*) marks files containing main() functions)
research/
flow/
flow.cpp
flow.hpp
program.cpp *
samples/
sample1.yuv
utils/
yuvreader.cpp
yuvreader.hpp
tests/
yuvreader_test.cpp *
I want to create EDE project(s) with one or more subprojects; or maybe I
just want one or more targets...?
flow's program.cpp requires flow/ and utils/ sources, but yuvreader_test
only requires utils/ sources.
I did ede-new in the root directory, and all subdirectories. I also did ede-new-target in the root directory, but when adding source files in subdirectories, it does not recognize the target I created.
I would appreciate it if someone could point me to some more complicated
Project.ede files for something like I'm trying to do. You can guess
that I have more subdirectories containing class code files, some of
which have standalone programs that use that code; also I have more test
code under tests/. Any example files/command workflows would be
appreciated.
The EDE feature that will generate Makefile or Automakefiles has a few more constraints than either Make or Automake. For example, the files belonging to a target must be in the same directory as the project containing the target. In your example, you would probably have no projects in your root directory.
To bring multiple sources together into a single program, a simple mechanism is to have each subdirectory create an archive (.a) or a shared lib (.so) that is linked together in your program.
If that is too constraining, you can also write your own automake files, and EDE will read those directly, so you can have a more complex build procedure as needed.
The CEDET distribution uses EDE as its build process, so you could look at that as a complex example. It doesn't build C++ files though, so it may not be as helpful as you would like.