Powershell Parse Response String of Requirements.txt file - powershell

Question

Been struggling for a few hours. I'm trying to hit this link
That has these contents:
# Generated on: Python 3.6.12
# With snowflake-connector-python version: 2.4.6
asn1crypto==1.4.0
azure-common==1.1.27
azure-core==1.15.0
azure-storage-blob==12.8.1
boto3==1.17.98
botocore==1.20.98
certifi==2021.5.30
cffi==1.14.5
chardet==4.0.0
cryptography==3.4.7
dataclasses==0.8
idna==2.10
isodate==0.6.0
jmespath==0.10.0
msrest==0.6.21
oauthlib==3.1.1
oscrypto==1.2.1
pycparser==2.20
pycryptodomex==3.10.1
PyJWT==2.1.0
pyOpenSSL==20.0.1
python-dateutil==2.8.1
pytz==2021.1
requests==2.25.1
requests-oauthlib==1.3.0
s3transfer==0.4.2
six==1.16.0
urllib3==1.26.5
And pass each non-commented out line to Poetry (a command line tool for Python dependency management)
This is my first step
(iwr https://raw.githubusercontent.com/snowflakedb/snowflake-connector-python/v2.5.1/tested_requirements/requirements_36.reqs | Select-Object).Content > req.txt
Where I'm struggling is I've tried doing convertfrom-string with various delimiters, and .Split(), and I can't seem to parse out the pieces I need. Poetry needs to take as input just the "packagename==" although version number is optional. So I essentially want to ignore lines that start with a "#" and then pass each line through a pipe, or even save it as an array. It doesn't seem to respond to setting a delimited to "\t" or carriage return "`r".
So next I would do something like
foreach($package in $package_array){poetry add $package}
Any help would be appreciated.

Answer 1

AdminOfThings provided a good pointer in a comment, but let me try to put it all together:
$url = 'https://raw.githubusercontent.com/snowflakedb/snowflake-connector-python/v2.5.1/tested_requirements/requirements_36.reqs'
foreach ($pkgLine in (irm $url).Trim() -split '\r?\n' -notmatch '^\s*#') {
# Remove `Write-Host` to perform the actual poetry call.
Write-Host poetry add ($pkgLine -replace '=.*')
}
irm is the built-in alias for Invoke-RestMethod, which is a simpler alternative to Invoke-WebRequest (iwr) in this case, because it directly returns the text of interest, as a multi-line string.
As an aside: the | Select-Object in your code is effectively a no-op and can be omitted.
.Trim() trims a trailing newline (all trailing whitespace).
-split '\r?\n' splits the string into individual lines.
-notmatch '^\s*# filters out all lines that start with #, optionally preceded by whitespace.
-replace '=.* removes everything starting with = from each package line.