Yarn can't find private Github npm registry - github

I signed up for the Github private npm registry beta and followed their instruction: https://github.com/features/package-registry
Works great with npm but I'd prefer using yarn. And while npm has no issues finding the registered package, yarn can't find it at all.
yarn add #omniphx/adminite-adminite-ui-components outputs:
yarn add v1.19.0
info No lockfile found.
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
[1/4] πŸ” Resolving packages...
error Couldn't find package "#omniphx/adminite-ui-components" on the "npm" registry.
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.
After reading up on private repos with yarn, I thought the trick was due to yarn having a slightly different rc format. Unfortunately, that didn't work either and yarn is still unable to find the private registry.
.npmrc
registry=https://registry.npmjs.org
#omniphx:registry=https://npm.pkg.github.com/omniphx
.yarnrc
registry "https://registry.npmjs.org"
"#omniphx:registry" "https://npm.pkg.github.com/omniphx"
Also confirmed that my github token is set too with yarn config list:
yarn config v1.19.0
info yarn config
{
'version-tag-prefix': 'v',
'version-git-tag': true,
'version-commit-hooks': true,
'version-git-sign': false,
'version-git-message': 'v%s',
'init-version': '1.0.0',
'init-license': 'MIT',
'save-prefix': '^',
'bin-links': true,
'ignore-scripts': false,
'ignore-optional': false,
registry: 'https://registry.npmjs.org',
'strict-ssl': true,
'user-agent': 'yarn/1.19.0 npm/? node/v12.11.1 darwin x64',
email: 'mattjmitchener#gmail.com',
lastUpdateCheck: 1570679687836,
username: 'omniphx',
'#omniphx:registry': 'https://npm.pkg.github.com/omniphx'
}
info npm config
{
'//npm.pkg.github.com/:_authToken': 'fake12345',
registry: 'https://registry.npmjs.org',
'#omniphx:registry': 'https://npm.pkg.github.com/omniphx',
python: '/usr/bin/python'
}
Any idea?
Resolved
Changed "#myorg:registry" "https://npm.pkg.github.com/myorg"
To "#myorg:registry" "https://npm.pkg.github.com"

I've just run into a similar situation. It seemed that yarn was only looking in the main Yarn package registry for my organization's private package. I had copied the examples from GitHub's Packages documentation for constructing your .npmrc file directly to the .yarnrc file in the project that will be consuming the app, not knowing that the formats were different (I've never had to deal with .yarnrc files before).
However, after updating the .yarnrc file with the correct format that you've mentioned above (which I also found in googling around), yarn successfully found the private package and installed it correctly.
As a heads up, my yarn version: 1.17.3
Steps I Took
Start new terminal session
cd to the project
nvm use (if you have a specific node version to use)
Add the correctly-formatted .yarnrc file to the project. See below for what it looks like.
Manually add the package and version range to the package.json for my private package
Run npm login --registry=https://npm.pkg.github.com --scope=#MyOrg
See the note below on scope / org gotcha's
Run yarn
That worked for me.
.yarnrc
"#myorg:registry" "https://npm.pkg.github.com"
Note: See below for a note on the org / scope name gotcha's
Other Notes
I know that it appears that you don't have any issues with this, given your GH username / scope above, but for anyone else that comes here, the documentation on GH is a little sparse with regards to mapping your username / org name to a scope in the package name. Just remember these little gotcha's here:
The name of your package must always be scoped to your org (or username)
E.g., name: #johndturn/my-package
If your organization has capital letters in it, like MyOrg, just replace them in the name of the package in your package.json and your .yarnrc with lowercase
E.g., name: #myorg/my-package
Note: When authenticating with npm login, I still have kept the uppercase letters in the --scope= argument.
The name of your package doesn't have to be the same name of the repo.
E.g., for a repo called MyOrg/random-prefix.js-lib, you can have name: #myorg/js-lib in your package.json file for the project itself. Then, installing it in other projects will look something like #myorg/js-lib: 1.0.0.

The problem I had is slightly different.
After tried what John suggested I still can't add private registry packages with yarn (but perfectly fine with npm)
Then I realise two things:
For GitHub packages, npm is fine with either
registry=https://npm.pkg.github.com/my-org
or
#my-org:registry=https://npm.pkg.github.com
but yarn only allow the latter syntax.
Docs from Github website only show the first syntax which could cause problems for yarn users.
Another thing is that if you npm login to the private registry but use a .yarnrc file in your project, yarn can't really mix your npm credentials with it. Although it seems behave differently on different environment.
But it would seems to be a best practice to stick with either yarn login + .yarnrc, or npm login + .npmrc (you can still use yarn to manage your packages in both cases)

In Yarn v2+ the setup has changed quite a bit. ".yarnrc" is ignored and only ".yarnrc.yml" is used.
To setup a private registry with a scope and token from env, add something along these lines to the ".yarnrc.yml" file (fontawesome example):
npmScopes:
fortawesome:
npmRegistryServer: "https://npm.fontawesome.com"
npmAuthToken: ${FONTAWESOME_TOKEN}
Documentation: https://yarnpkg.com/configuration/yarnrc#npmScopes

I'm not an expert with npm/yarn so I might be misunderstanding what is happening here, but I don't think package proxying from the npm registry works with yarn yet. Could that be related? When package proxying was released for npm I remember reading comments on Twitter from people that tried it with yarn and it didn't work.
Found the Twitter thread here:
https://twitter.com/github/status/1171832034580451328
It doesn't work with Yarn. As soon as I change the registry url -> Couldn't find package.

Related

Yarn 2 init, add failing

I'm experimenting with yarn 2 and faceplanting.
I created a new folder: /projects/yarn2/
As per their install guide https://yarnpkg.com/getting-started I ran
cd /projects/yarn2/
yarn set version berry
yarn init
then (as per their guide https://yarnpkg.com/getting-started/usage )
yarn add react
and got the following error:
Usage Error: The nearest package directory (/projects/yarn2) doesn't seem to be part of the project declared in /projects.
- If the project directory is right, it might be that you forgot to list yarn2 as a workspace.
- If it isn't, it's likely because you have a yarn.lock or package.json file there, confusing the project root detection.
$ yarn add [--json] [-E,--exact] [-T,--tilde] [-C,--caret] [-D,--dev] [-P,--peer] [-O,--optional] [--prefer-dev] [-i,--interactive] [--cached] ...
What am I doing wrong?
You either don't have package.json or yarn.lock which confuses yarn if the package is added in the workspace or not. Just run the following command and I think your problem must be solved.
cd <folder_name>/<project_name>
touch yarn.lock
yarn
Check to see if you have a package.json or yarn.lock file in your /projects directory. If you do, clear it/them out and this should start working.
To add to the previous answers what worked for me was:
It seems to be that yarn.lock file is what's really required, so if you have started with yarn v1.x and did yarn init - then you will have a package.json file in the project dir already.
If you now switch over to yarn v2 by doing yarn set berry, and want to add a package by yarn add [whatever] it will fail. You can work around that by creating an empty file by: touch yarn.lock (or on windows just cat '' > yarn.lock) after that it should work fine.
I figured out an even better / faster way to do it now, just do this in an empty project folder:
yarn init -2
this will initialize a new yarn v2 project folder with .yarnrc.yml, package.json, and the .yarn folder, also initializes an empty .git folder and adds a proper .gitignore. check by yarn --version - should echo v2.x.x
Usage Error: The nearest package directory (/<path_to_folder>/<project_name>) doesn't seem to be part of the project declared in /<path_to_folder>.
Solution:
Search for yarn.lock and package.json files inside /<path_to_folder> and delete them!
you have to remove yarn.lock and package.json file in your root directory ,some times there may be .yarnrc yarnyml files in you root directory you have to remove them also.

Publishing Github packages for a monorepo as part of an organization

I have a Lerna monorepo on Github Enterprise which currently has two npm packages that I'm trying to publish to the Github package registry under the same repo.
For reference say they are:
github.com/mycompany/package-a
github.com/mycompany/package-b
I followed these instructions: https://help.github.com/en/github/managing-packages-with-github-packages/configuring-npm-for-use-with-github-packages#publishing-multiple-packages-to-the-same-repository
So now my 2 package.json files look like the following (trimmed for formatting purposes):
"name": "#mycompany/package-a",
"repository": {
"url": "ssh://git#github.com:mycompany/monorepo.git"
},
"publishConfig": {
"registry": "https://npm.pkg.github.com/"
},
"name": "#mycompany/package-b",
"repository": {
"url": "ssh://git#github.com:mycompany/monorepo.git"
},
"publishConfig": {
"registry": "https://npm.pkg.github.com/"
},
So you can notice they both have the same URL for the repository as recommended.
First problem: One is that my company already has repos called package-a and package-b. It seems that you can't have a naming collision with a package in a monorepo and a package outside the monorepo.ΒΉ
Second and more important problem: This doesn't seem to work for me at all. I renamed the package in their respective package.json files to avoid the naming collision to package-a-mono and package-b-mono which I don't really want to do but I'm just trying to get it to work. I get a 404 when trying to run either lerna publish or npm publish inside of the repos themselves. Like it's not actually trying to read that repository.url field in that it tells you to modify.
^ This turned out to be temporary or was never actually an issue, it was just because of the naming conflicts.
npm publish output:
npm ERR! code E404
npm ERR! 404 Not Found - PUT https://npm.pkg.github.com/#mycompany%2fpackage-a - The expected resource was not found.
lerna publish output:
lerna http fetch PUT 404 https://npm.pkg.github.com/mycompany/#mycompany%2fpackage-a 327ms
lerna ERR! E404 The expected resource was not found.
Has anyone run into this and found a solution?
ΒΉOn a somewhat worse note, for some reason the very first time I ran this it actually did publish a package into the monorepo for package-a. But from then on I get the error lerna ERR! E422 Package "package-a" is already associated with another repository. Nothing changed and I couldn't publish another version to the same repo.
Another possible cause of this error (discussed and ruled out in the original question body) is if any package's package.json's repository field does not match the git URL of the repo - for example, if you transferred the repo to a different organisation, or renamed the repo, but didn't update every package's package.json with the new URL.
The error message will report 404 on https://npm.pkg.github.com/#org/package-name even if the problem is with the repository URL.
(credit to jonas-reif's comment)
The issue here was that we had repos in the organization with the same name as the package being published and it didn't like that.
When I looped back around to solving this I renamed the packages to not have a collision with another existing repo in the organization and it worked as expected.
I ran into the same issue, I had to generate a new personal access token that had more privileges. Just read:packages and write:packages was not sufficient, you also needed repo.
I had the same problem and I had to add the following line to my NPM package's URL:
"repository": "https://github.com/ACCOUNT/REPOSITORY"
This repository was the root of my mono repository.
More on this here:
https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-npm-registry#publishing-multiple-packages-to-the-same-repository

Cannot install npm package in Github registry

I have published a package to the GitHub npm registry. I can see the package when I login onto GitHub, as per the screenshots below.
ScreenShot 1
ScreenShot 2
Locally, a different package has the former as a dependency. Therefore I have on my package.json:
"dependencies": {
"<#scope>/<package_name>": "1.2.0",
...
I also have, in-line with the documentation, a .npmrc file at the same directory as the package.json with the authToken. This authToken has all the permissions necessary (delete:packages, read:packages, repo, write:packages)
//npm.pkg.github.com/:_authToken=<my_auth_token>
registry=https://npm.pkg.github.com/OWNER
Lastly, I have on package.json the following two entries. Although I believe these are necessary to publish packages to the github registry, no to install them which is the current problem.
"publishConfig": {
"registry": "https://npm.pkg.github.com/"
},
"repository": {
"type": "git",
"url": "https://github.com/<OWNER>/<REPO>.git"
}
So, the error that I am having when I try to install the package is:
npm ERR! code E404
npm ERR! 404 Not Found: <#scope>/<package_name>#1.2.0
When I tried to understand what was going on with some helpful logs:
With npm install #<scope>/<package_name>#1.2.0 --loglevels verbose, I see:
....
npm http fetch GET 200 https://npm.pkg.github.com/OWNER/#<scope>%2f<package_name>
npm http fetch GET 404 https://npm.pkg.github.com/download/#<scope>/<package_name>/1.2.0/98e69e2adddec5c715a32c94352efac83ce586e4e6f3d06fef1fc1f82d238r34
...
The first http request is a success, which seems to indicate things are correctly wired up and authenticated. However, I have no idea why I am getting a 404 afterwards. The package is clearly there as we see on the image above.
Does anyone had a similar problem ?
Well, my own .npmrc (in my home directory) is configured as this:
//registry.npmjs.org/:_authToken=<token>
//npm.pkg.github.com/:_authToken=<token>
#myorg1:registry=https://npm.pkg.github.com
#myorg2:registry=https://registry.npmjs.org
I've not added my org in the github registry URL as you did, and specified the registry to use for each scope.
I just ran into the same issue. Removing package-lock.json fixed it for me. I suspect the problem was that I previously installed the offending package from a tarball and that the old installation information was somehow cached inside the package-lock.json
I had misread the checkboxes in my PAT. Make sure you've enabled read:packages.
My successful .npmrc file looks like this:
//npm.pkg.github.com/:_authToken=TOKEN_GOES_HERE

How to build Conda env on Mac using Windows yml file?

I'm creating Conda create environment from yml I generated on Windows' Miniconda install. I need to create same environment on OS X. Following the advise found here on SO I used the --no-builds option.
Also, the names of some packages under section ResolvePackageNotFound are clearly (many if not all) specific to Windows:
- m2w64-gmp=6.1.0
- m2w64-gcc-libs-core=5.3.0
- m2w64-gcc-libs=5.3.0
- vc=14.1
- vs2015_runtime=15.5.2
- msys2-conda-epoch=20160418
- menuinst=1.4.14
- icc_rt=2019.0.0
- m2w64-libwinpthread-git=5.0.0.4634.697f757
- pywinpty=0.5.5
- wincertstore=0.2
- m2w64-gcc-libgfortran=5.3.0
- win_inet_pton=1.1.0
- winpty=0.4.3
I removed all of these from the yml file. Even then it's stalled at the following screen:
(base) MacBook-Air:Anaconda.d xtian$ conda env create -f 32b-qb-2019-10-05.yml
Collecting package metadata (repodata.json): done
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abor|
Examining openssl: 10%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 29/279 [00:00<00:00, 3729.87it- ]
Comparing specs that have this dependency: 16%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 16/101 [05:53<31:19, 22.11s/it]
Finding shortest conflict path for openssl[version='>=1.0.2p,<1.0.3a']: 38%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Š | 6/16 [02:39<06:23, 38.32s/it]
This process is progressing at an astonishingly slow pace, and hasn't got past openssl ... 29/279. Should I wait and trust Conda can figure this all out?
Or,
Do I need another strategy--
I'm wondering if I can't remove the offending packages, each in turn, and create a series of yml files to install in order using, $ conda env update --prefix ./env --file environment.yml --prune, because whatever finally works here I know I'm going to need to use it on another machine so I can share the project env with a colleague.
Any other suggestions?
Short answer: Try deleting the packages that your system is getting stuck on from the .yml file. i.e., remove "openssl" from .yml file.
I have been running into the same issue trying to install a .yml file created in a Windows system to a Mac system. I basically followed the same procedure you did:
-Created yml file using the --no-builds option.
-Attempted to create environment on Mac system and had several windows specific packages left under ResolvePackageNotFound section (listed below)
m2w64-libwinpthread-git=5.0.0.4634.697f757
pyreadline=2.1
pywinpty=0.5.5
m2w64-gcc-libgfortran=5.3.0
vc=14
m2w64-gcc-libs-core=5.3.0
m2w64-gmp=6.1.0
wincertstore=0.2
icc_rt=2019.0.0
m2w64-gcc-libs=5.3.0
vs2015_runtime=14.15.26706
winpty=0.4.3
msys2-conda-epoch=20160418
-Deleted those from the yml file
-Attempted to create environment from updated yml file and received the following conflicts:
- Found conflicts! Looking for incompatible packages.
My system also got stuck trying to solve the "openssl" conflict along with a "_tflow_select". I ended up deleting those and was able to create my environment and run the code without too much trouble.

CMAKE FILE(COPY... adding permission not in FILE_PERMISSIONS list

To complete an install on Ubuntu necessitates adding postinst and prerm scripts to the package built with CMAKE. Adding the script files using SET(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA... was the easy part, however, when an install is attempted, Lintian complains about the quality of the package, specifically, the file permissions for postinst and prerm are not set properly to 755. After a considerable amount of searching for an answer one was found using FILE(COPY and setting the FILE_PERMISSIONS to OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE (755) before setting CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA. Now Lintian is still complaining about the package quality only this time the file permissions are set to 775 which includes GROUP_WRITE but that is not in the FILE_PERMISSIONS list. It is unclear how this is possible. The relevant snippet of CMAKE is posted below:
FILE(COPY ${installation_add_ins}/linux/postinst_in DESTINATION ${installation_add_ins}/linux/postinst
FILE_PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE
)
FILE(COPY ${installation_add_ins}/linux/prerm_in DESTINATION ${installation_add_ins}/linux/prerm
FILE_PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE
)
SET(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${installation_add_ins}/linux/postinst;${installation_add_ins}/linux/prerm;")
What's even more interesting is that if "Ingnore and install" is selected, the package works as expected. Ubuntu's package manager appears to fixup the file permissions and both install and uninstall work properly. But this fact is not relevant to the question.
Why is GROUP_WRITE being set on the resultant file when it is not specified in the permissions?