I have a build server with no internet access, and I need to resolve dependencies from both github.com and registry.npmjs.org. The build server has access to Artifactory, so I have created an NPM repo to proxy for registry.npmjs.org and that is working, and I just created a VCS repo to proxy for github.com.
How can I configure npm to resolve from both of these? Since the 2 repos are different types, I can't aggregate them into a single virtual repo. Can NPM be configured to resolve dependencies from both of these?
VCS repos have zero correlation to NPM dependencies. A VCS repo is just a gateway to a set of APIs on the remote git server that will help you cache source binaries (i.e a zip/tarball of a particular branch/tag or even a release). The npm client is not familiar with the REST endpoints that Artifactory exposes for such repos.
For NPM packages that reference github repos inside their package.json (see URLs as dependencies & Git URLs as Dependencies sections here), you want to look into Artifactory's dependency-rewrite mechanism.
Since your NPM client is running on a machine that has no access to the internet, your own package.json files should not depend directly on "github dependencies", since these make the client bypass the registry configuration inside your ~/.npmrc and go directly to github instead of Artifactory.
When the package.json of one of your project's dependencies uses github dependencies, and this package is resolved via Artifactory, the dependency rewrite mechanism modifies the package.json on the fly before returning it to the client, so that subsequent dependency requests to resolve such dependencies are attempted via Artifactory, and not via github -- this is perfect for use cases such as yours.
In summary, you should stick with NPM repositories on Artifactory specifically, but also utilize the dependency rewrite mechanism of the Virtual Repository in order to avoid direct resolution of dependencies via github.
HTH,
Related
I have a GitHub organisation set up and within that are two private repositories: Library and App. I have managed to configure a GitHub Action for the Library repository to create a NuGet package and push it to nuget.pkg.github.com. I added the Library package as a dependency to the App project and can fetch these locally using a configured Personal Access Token. I am now working on the GitHub Action to build and run unit tests on the App project.
I have added a step to the workflow to add the NuGet package source
- name: Set package source
run: dotnet nuget add source --username USERNAME --password ${{ secrets.GITHUB_TOKEN }} --store-password-in-clear-text --name github "https://nuget.pkg.github.com/REDACTED/index.json"
This is exactly the same line as is used when pushing the package, but when I reach the dotnet restore step later I get the following error:
/home/runner/work/App/App/App.Services/App.Services.csproj : error NU1101:
Unable to find package Library. No packages exist with this id in source(s): github, nuget.org [/home/runner/work/App/App/App.sln]
I understand that it is likely possible to work around this by adding a new Personal Access Token to the secrets of the App repository, but would prefer to rely on GITHUB_TOKEN if possible for both the security and management benefits.
According to GitHub, installing packages hosted with GitHub Packages requires minimal configuration using GITHUB_TOKEN. And by default, GITHUB_TOKEN has both read and write permissions for packages.
This question is different from this one as that relates to an issue with empty NuGet.config files which has since been resolved.
I hit the same problem, GITHUB_TOKEN can be used to push packages but not pull packages from other repos within the same organisation. This thread has a lot of people complaining about it: https://github.com/actions/setup-node/issues/49. I was unable to get any of the workarounds there working for me, and had no option in GitHub to grant specific permissions to repositories in the organisation for packages. Had to use a PAT in the end.
I'm using some third-party dependencies hosted on a corporate npm-registry (accessible publically) that fails my angular's app deployment
Is there a way to configure npm for Netfly ?
The solution to override the default npm-registry is to commit your local .npmrc config file in the repo
I'm looking to migrate from an existing private NuGet Gallery repo. We have Artifactory (v6.10.1 , will be upgrading soon) with a nuget virtual, nuget-local, and nuget-remote-cache repo.
I have configured the Artifactory remote repo to point to the NuGet Gallery server and as you can see from the screenshot below, it passes the connection test. However, when I try to nuget list from the remote repo, I get no packages returned. If I try the same call directly against the NuGet Gallery, I get lots of packages returned. My account has the "anything" permission in artifactory (Effective permissions shows all 4 checks) and no authentication is required to retrieve from either NuGet Gallery or the artifactory repo but I've set up my un/pw and apikey all the same. If I try to browse Packages, NuGet isn't even available. In Artifacts, the only thing that shows up is the package.json from the local repo. What is going on here?
The problem turned out to be that I had enabled "Global Offline mode" so Artifactory was refusing to even attempt to access the Remote repository. This was evident by the complete lack of log entries in the request.log. Disabling the setting allowed the list to list packages.
I still did not see any packages in the cache but after running nuget install <package>, it appeared in the cache. Apparently the caching only occurs on a pull of a package and does not function as a full replica of the remote repo.
Remote repositories in Artifactory serves as a caching proxy that the URL of the repository points to. To have the control over the storage of your hardware, the artifacts are being cached by demand of the users therefore it’s not being synced with the remote URL.
With this said, if the remote repository is pointing to other Artifactory repository you can set the pull replication feature of Artifactory to retrieve the artifacts from the remote Artifactory server and populate the remote repository cache and in this way to synchronize the repositories.
I have a build server with no internet access, and I need to resolve dependencies from both github.com and registry.npmjs.org. The build server has access to Artifactory(jfrog), so I have created an npm repo to proxy for registry.npmjs.org and that is working, and I just created dependency-rewrite mechanism under virtual repo for remote npm repo as mentioned in this link--Configure npm to resolve dependencies using artifactory as proxy for both npm registry and github
after configuring still I face the same issue:
node-sass#4.11.0 install /app/jenkins/workspace/uiwidget_smarthome1.0_dev/bwtk/node_modules/node-sass
node scripts/install.js
Downloading binary from https://github.com/sass/node-sass/releases/download/v4.11.0/linux-x64-47_binding.node
Cannot download "https://github.com/sass/node-sass/releases/download/v4.11.0/linux-x64-47_binding.node":
How can I configure npm to resolve from both of these? Since the 2 repos are different types, I can't aggregate them into a single virtual repo. Can npm be configured to resolve dependencies from both of these?
Yes, you need to pass the virtual repo url to the npm command. You can use ---registry virtual repo url or you can set the registry to using npm command
I’m trying to set up an operation where the only internet access to external repos is via artifactory server. I have followed the bower information on this web page http://www.jfrog.com/confluence/display/RTF/Bower+Repositories
I can successfully do the npm installs of bower-art-resolver as described (utilizing npm remote repository for npmjs in artifactory), but then trying to do the example bower install of bootstrap it fails because bower is attempting to find git://github.com/twbs/bootstrap.git and I don't have an access to github.com due to firewalls.
How do I make the full bower workflow work then if having the bower registry remote repository is not suffient to make the setup work? Is there some way that the artifactory VCS functions come into play? How would I make bower utilize that instead of trying to reach github.com?
This firewall scenario seems like a common use case for a repository server, so I'm sure I'm missing something.
Make sure you are doing the following:
(1) Create a remote repository in Artifactory proxying the Bower registry. Notice that Artifactory will need to access both the Bower registry and Github.
(2) Configure Bower to use the Artifactory repository you created in the previous step as the Bower registry. This should be done in the .bowerrc file, for example:
{
"registry": "http://localhost:8081/artifactory/api/bower/bower-repo"
}
(3) Use bower-art instead of bower when installing packages, for example:
bower-art install bootstrap