Split lines to words then save as a new file - powershell

Question

Say I have a text file test.txt in C drive.
On the face of things, we seem to be merely talking about text-based files, containing only
the letters of the English Alphabet (and the occasional punctuation mark).
On deeper inspection, of course, this isn't quite the case. What this site
offers is a glimpse into the history of writers and artists bound by the 128
characters that the American Standard Code for Information Interchange (ASCII)
allowed them. The focus is on mid-1980's textfiles and the world as it was then,
but even these files are sometime retooled 1960s and 1970s works, and offshoots
of this culture exist to this day.
I want to split all lines to words then save it as a new file. In the new file, each line only contains one word.
Thus the new file will be:
On
the
face
of
things
we
seem
to
....
The delimiter is a white space and please skip all punctuation marks.

Answer 1

You haven't even tried. The next time I'm voting for closed question. Powershell uses 99% of c# syntax and "all" .Net classes are available, so if you know c#, you will come far in PowerShell by using 5 minutes on google and trying some commands.
#create array
$words = #()
#read file
$lines = [System.IO.File]::ReadAllLines("C:\Users\Frode\Desktop\in.txt")
#split words
foreach ($line in $lines) {
$words += $line.Split(" ,.", [System.StringSplitOptions]::RemoveEmptyEntries)
}
#save words
[System.IO.File]::WriteAllLines("C:\Users\Frode\Desktop\out.txt", $words)
In PowerShell you could also do it like this:
Get-Content .\in.txt | ForEach-Object {
$_.Split(" ,.", [System.StringSplitOptions]::RemoveEmptyEntries)
} | Set-Content out.txt

Answer 2

$Text = #'
On the face of things, we seem to be merely talking about
text-based files, containing only the letters of the English Alphabet
(and the occasional punctuation mark). On deeper inspection, of
course, this isn't quite the case. What this site offers is a glimpse
into the history of writers and artists bound by the 128 characters
that the American Standard Code for Information Interchange (ASCII)
allowed them. The focus is on mid-1980's textfiles and the world as it
was then, but even these files are sometime retooled 1960s and 1970s
works, and offshoots of this culture exist to this day.
'#
[regex]::split($Text, ‘\W+’)

Answer 3

Here's a solution using regular expressions, that will:
Remove special characters
Parse words based on word boundaries (\b in regex)
Code:
$Text = #'
On the face of things, we seem to be merely talking about text-based files, containing only
the letters of the English Alphabet (and the occasional punctuation mark).
On deeper inspection, of course, this isn't quite the case. What this site
offers is a glimpse into the history of writers and artists bound by the 128
characters that the American Standard Code for Information Interchange (ASCII)
allowed them. The focus is on mid-1980's textfiles and the world as it was then,
but even these files are sometime retooled 1960s and 1970s works, and offshoots
of this culture exist to this day.
'#;
# Remove special characters
$Text = $Text -replace '\(|\)|''|\.|,','';
# Match words
$MatchList = ([Regex]'(?<word>\b\w+\b)').Matches($Text);
# Get just the text values of the matches
$WordList = $MatchList | % { $PSItem.Groups['word'].Value; };
# Examine the 'Count' of words
$WordList.Count
Result looks like:
$WordList[0..9];
On
the
face
of
things
we
seem
to
be
merely

Answer 4

I wouldn't bother splitting the string, since you write the result back to a file anyway. Just replace all punctuation (and perhaps parentheses as well) with spaces, replace all consecutive whitespace with newlines, and write everything back to a file:
$in = 'C:\test.txt'
$out = 'C:\test2.txt'
(Get-Content $in | Out-String) -replace '[.,;:?!()]',' ' -replace '\s+',"`r`n" |
Set-Content $out