Converting a text file with one column to multiple column [closed] - powershell

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I have a file with data in one column format. I need to use this file as an input file and the output file should be in a multi column format. I need help with a script that will do the conversion. It does not matter PowerShell or batch.
Input file content:input.txt
store1:
apple
orange
peach
THE END
store2:
Tree
Park
Pond
Bird
THE END
store3:
Building
Road
peach
store
Grocery
THE END
The output file should be:
store1:,store2:,store3:
apple, Tree, Building
orange, Park, Road
peach, Pond, peach
, Bird, store
, , Grocery

i know this is a gimmie, but i took it as a learning opportunity for myself, and since i have the code maybe someone else can learn from it
$text = gc C:\temp\input.txt
$groups = ($text | out-string) -split 'the end' | ? {$_ -notmatch '^(?:\s+)?$'}
$columns = $groups | % {$_.trim().split("`n")[0]}
$rows = $groups | % {$_.trim().Split("`n").count - 2} | sort -desc | select -f 1
$result = 0..$rows | % {
$row = $_
$obj = New-Object psobject
0..$($columns.Count-1) | % {
$column = $columns[$_]
$store = $groups[$_].trim().split("`n")
$item = $store[$row+1]
$obj | Add-Member -MemberType NoteProperty -Name $column.trim() -Value $(if ($item) {$item.trim()})
}
$obj
}
$result | epcsv C:\temp\input.csv -NoTypeInformation

Here is a pure batch-file solution:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem Define global settings here:
set "INFILE=input.txt"
set "OUTFILE=output.txt"
set "HEAD=^store[1-9][0-9]*:$"
set "FOOT=^THE END$"
set "DELIM=,"
set /A "COL=0, ROW=0, MAX=0"
for /F "delims=" %%L in ('
findstr /N /R "^" "%INFILE%"
') do (
set "LINE=%%L"
setlocal EnableDelayedExpansion
set "LINE=!LINE:*:=!"
if defined HEAD (
if !COL! EQU 0 set /A "ROW=-1"
cmd /V /C "echo^(!LINE!"| > nul findstr /R /C:"%HEAD%" ^
&& if !ROW! LSS 0 set /A "COL+=1, ROW=0"
if defined FOOT (
cmd /V /C "echo^(!LINE!"| > nul findstr /R /C:"%FOOT%" ^
&& set /A "ROW=-1" || if !COL! GTR 0 if !ROW! GEQ 0 set /A "ROW+=1"
) else (
if !COL! GTR 0 set /A "ROW+=1"
)
) else (
if defined FOOT (
if !ROW! EQU 0 set /A "COL+=1"
cmd /V /C "echo^(!LINE!"| > nul findstr /R /C:"%FOOT%" ^
&& set /A "ROW=0" || set /A "ROW+=1"
) else (
if !COL! EQU 0 set /A "COL=1"
if defined LINE (
set /A "ROW+=1"
) else (
if !ROW! GTR 0 set /A "COL+=1"
set /A "ROW=0"
)
)
)
if !MAX! LSS !ROW! set /A "MAX=!ROW!"
for /F "tokens=1-3 delims=;" %%I in ("!COL!;!ROW!;!MAX!") do (
endlocal
if %%I GTR 0 if %%J GTR 0 (
set "COLLECT[%%I_%%J]=%%L"
)
set /A "COL=%%I, ROW=%%J, MAX=%%K"
)
)
setlocal EnableDelayedExpansion
> "%OUTFILE%" (
for /L %%J in (1,1,%MAX%) do (
set "LINE="
for /L %%I in (1,1,%COL%) do (
if %%I GTR 1 set "LINE=!LINE!!DELIM!"
if defined COLLECT[%%I_%%J] (
set "LINE=!LINE!!COLLECT[%%I_%%J]:*:=!"
)
)
echo(!LINE!
)
)
endlocal
endlocal
exit /B
Basically this script collects the data in an array-like variable COLLECT[COL_ROW], where COL and ROW denote the column and row indexes, respectively. The code consists of two loops, where the first one walks through the given input file and assigns the line texts to the related array elements. The predefined header and footer strings (or, if both are not provided, any empty lines) control determination of the applicable COL and ROW indexes. MAX holds the greatest row index ROW, because the blocks of data might be of different size, for later padding. The second loop enumerates the collected data array, builds a line of text per each column and writes them to the specified output file.
The code section at the beginning marked with a remark rem defines the global settings of the script, like the input file (INFILE), the output file (OUTFILE), the header and the footer (HEAD and FOOT, respectively; both findstr-compatible regular expressions; either or both of them can be empty) and the delimiter (DELIM).
This approach has got 4 modes:
both header and footer are non-empty:
data is collected beginning at the first header string;
another header appearing after a footer starts a new data column;
everything between a footer and the next header is ignored;
a header appearing after another one and before a footer is treated as a normal field;
header texts are included in the returned data, footer texts are not;
empty lines are kept, meaning they result in an empty field;
header is non-empty but footer is empty:
data is collected beginning at the first header string;
another header starts a new data column;
header texts are included in the returned data;
empty lines are kept, meaning they result in an empty field;
footer is non-empty but header is empty:
data is collected beginning at the very first line;
a footer starts a new data column;
footer texts are not included in the returned data;
empty lines are kept, meaning they result in an empty field;
both header and footer are empty:
data is collected beginning at the first non-empty line;
a block of one or more consecutive empty lines starts a new data column;
empty lines are not included in the returned data;
Note:
Although the question lacks of information and attempts of or research by the poster, I decided to answer it, because the task at hand is a quite interesting challenge being solved with a batch-file.
EDIT: The code below is a solution from user Aacini (not from aschipfl) posted here after the original poster of this answer give me his kindly permission. I was forced to do this because the question is closed and I really wanted to post my code!
#echo off
setlocal EnableDelayedExpansion
rem Initialize data for first store
set /A max=0, lines=0, store=0
for /F "delims=" %%a in (input.txt) do (
if "%%a" neq "THE END" (
rem Process the next line of this store
set /A lines+=1
for %%l in (!lines!) do (
if not defined line[%%l] (
rem This store have more lines than previous ones: initialize new line
for /L %%i in (1,1,!store!) do set "line[%%l]=!line[%%l]! ,"
)
rem Append new data to this line
set "line[%%l]=!line[%%l]!%%a,"
)
) else (
rem This store ends: get the maximum number of lines
if !lines! gtr !max! (
set "max=!lines!"
) else (
rem Enlarge the additional lines of previous stores, if any
set /A lines+=1
for /L %%i in (!lines!,1,!max!) do set "line[%%i]=!line[%%i]! ,"
)
rem Pass to next store
set /A lines=0, store+=1
)
)
rem Output all result lines
(for /L %%i in (1,1,%max%) do echo !line[%%i]:~0,-1!) > output.txt
Output:
store1:,store2:,store3:
apple, Tree, Building
orange, Park, Road
peach, Pond, peach
, Bird, store
, , Grocery

You can pipe a text file into this PowerShell script. It uses PowerShell's dialect of CSV (which includes quote chars).
Begin {
# corresponds to (untransposed) records
$records = #()
# the current record
$this_record = #()
# maximum fields of any (untransposed) record
$max_fields = 0
}
Process {
If ($_ -eq "THE END") {
# Append the record to the array.
$records += ,$this_record
# Count the maximum number of fields (this will be the number of
# records when the data is transposed).
If ($this_record.Length -gt $max_fields) {
$max_fields = $this_record.Length
}
$this_record = #()
} ElseIf ($_.Trim() -eq "") {
# Ignore blank lines.
} Else {
# Append the field to the current record.
$this_record += $_
}
}
End {
# Transpose the fields
$objects = #()
For ($col=0; $col -lt $max_fields; $col+=1) {
# ConvertTo-CSV gets object properties. It doesn't implicitly
# operate on arrays the way we'd prefer.
$obj = New-Object PSCustomObject
For ($row=0; $row -lt $records.Length; $row+=1) {
# Create property names that sort lexically (zero-padded numbers).
$obj | Add-Member -MemberType NoteProperty `
-Name ("{00000}" -f $row) `
-Value $records[$row][$col]
}
$objects += $obj
}
# Convert to CSV, throw away the header
$objects | ConvertTo-CSV -NoTypeInformation | Select-Object -Skip 1
}
E.g. PowerShell -NoProfile -ExecutionPolicy Bypass -File xpose.ps1 < input.txt produces:
"store1:","store2:","store3:"
"apple","Tree","Building"
"orange","Park","Road"
"peach","Pond","peach"
,"Bird","store"
,,"Grocery"

Here's some code to help you out. Study this!
appendcolumn.bat
#echo off
set i=1
for /f "tokens=*" %%x in ('more') do (
call :app !i! %%x
set /a i += 1
)
exit /b
:app
set line%1=!line%1!,%2
exit /b

Related

Alternate-split CSV Column Using Powershell [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 1 year ago.
Improve this question
The task should be quite simple, yet I cannot figure out how to accomplish it.
Suppose I have a CSV file consisting of the following one-column data (there are actually hundreds of lines, not only six).
AAAAA
BBBBB
CCCCC
DDDDD
EEEEE
FFFFF
Using Powershell, I want to automate the creation of a new CSV file from the above file that splits the data into the following two tab-delimited columns.
AAAAA BBBBB
CCCCC DDDDD
EEEEE FFFFF
How can I achieve that? Thank You
What you show for the first file is not a CSV file, just a text file with values each on a new line.
The result you aparently want is a tab delimited file without headers.
In PowerShell, the easiest way to do that I think is to use an indexed loop like this:
$fileA = Get-Content -Path 'Path\To\The\File' # read the values as string array
$result = for ($i = 0; $i -lt $fileA.Count; $i += 2) { # loop through the lines in steps of 2
"{0}`t{1}" -f $fileA[$i], $fileA[$i + 1] # output the values delimited with a TAB character
}
# show in console window
$result
# write to file
$result | Set-Content -Path 'Path\To\The\New\File'
Output:
AAAAA BBBBB
CCCCC DDDDD
EEEEE FFFFF
If you want to create a real CSV file with headers, output objects instead of strings:
$fileA = Get-Content -Path 'Path\To\The\File' # read the values as string array
$result = for ($i = 0; $i -lt $fileA.Count; $i += 2) { # loop through the lines in steps of 2
# output an object with column headers
[PsCustomObject]#{ ColumnA = $fileA[$i]; ColumnB = $fileA[$i + 1] }
}
# show in console window
$result
# write to file
$result | Export-Csv -Path 'Path\To\The\New\File' -Delimiter "`t" -NoTypeInformation
Output:
ColumnA ColumnB
------- -------
AAAAA BBBBB
CCCCC DDDDD
EEEEE FFFFF
You used the batch-file tag, so I have to assume, a batch file solution is ok.
Read a line, remember it and print only every second run of the loop:
#echo off
setlocal enabledelayedexpansion
set "first="
for /f "delims=" %%a in (w.csv) do (
if not defined first (
set "first=%%a"
) else (
echo !first! %%a
set "first="
)
)
if defined first echo %first% -----
The last line takes care of odd line counts.
What follows is a batch file solution, which does things a little differently.
Essentially, if it's an odd line number, outputs the content followed by a horizontal tab, if its an even number, it outputs the line content followed by a carriage return and line feed. It also was written not to have issues outputting content beginning with a ; character, or containing ! characters.
Please adjust only your source, and destination files on lines 4 and 5 to your own absolute, or relative, filenames before testing:
#Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
For /F "Delims==" %%G In ('"(Set {) 2>NUL"') Do Set "%%G="
Set "{s}=C:\Users\Robith\Documents\input.csv"
Set "{d}=C:\Users\Robith\Desktop\output.csv"
If Not Exist "%{s}%" GoTo :EOF
For /F %%G In ('Copy /Z "%~f0" NUL') Do Set "{r}=%%G"
For /F "Delims==" %%G In ('%SystemRoot%\System32\wbem\WMIC.exe OS Call /?
^| %SystemRoot%\System32\find.exe "=="') Do Set "{t}=%%G"
(Set {n}=^
% 0x0A %
)
( For /F Delims^=^ EOL^= %%G In ('Type "%{s}%"
^| %SystemRoot%\System32\find.exe /V ""') Do (
Set /A {i} += 1, {m} = {i} %% 2
Set "{l}=%%G"
SetLocal EnableDelayedExpansion
If !{m}! Equ 1 (Set /P "=!{l}!%{t}:~-1%"<NUL
) Else Set /P "=!{l}!%{r}%!{n}!"<NUL
EndLocal)) 1>"%{d}%"
For /F "Delims==" %%G In ('"(Set {) 2>NUL"') Do Set "%%G="

How to get the folder name with highest number in folder name?

I have multiple folders with these names:
ABC_03_00_016_0
ABC_03_00_016_1
ABC_03_00_016_2
ABC_03_00_016_3
ABC_03_00_016_4
ABC_03_00_016_5
What I want to do is to retain the folder with largest number in the end of folder name, i.e. ABC_03_00_016_5 in above case using PowerShell or batch commands.
How to get the folder with greatest number?
Maybe this could be done more elegant, but it's probably working as you want. I'm stripping the last digits, converting & comparing them to then determine the highest one. As you can see, order does not matter:
$items =
"ABC_03_00_016_0",
"ABC_03_00_016_100",
"ABC_03_00_016_99"
[int]$highestNumberTillNow = 0
$highestitem = ""
foreach ($item in $items){
[int]$number = $item.substring($item.LastIndexOf("_")+1,$item.length-$item.LastIndexOf("_")-1)
if ($number -gt $highestNumberTillNow){
$highestNumberTillNow = $number
$highestitem = $item
}
}
write-host $highestitem
You can use this:
#echo off
setlocal enabledelayedexpansion
pushd "%~dp0"
for /f "tokens=4 delims=_" %%a in ('dir ABC_03_016_* /ad /b') do (
for /f "tokens=* delims= " %%b in ('dir ABC_03_016_* /ad /b') do (
set dir[%%a]=%%b
)
)
set dc=0
:loop
set /a dc=dc+1
if defined dir[%dc%] goto loop
goto break
:break
set /a dc=dc-1
echo The folder is !dir[%dc%]!
pause >nul
Assuming you could have more folders with a similar name in the root path like
ABC_03_00_016_0
ABC_03_00_016_1
ABC_03_00_016_2
ABC_03_00_016_3
ABC_03_00_016_4
ABC_03_00_016_5
DEF_03_00_016_0
DEF_03_00_016_1
DEF_03_00_016_10
Using PowerShell you can use something like below.
This will return the folder object(s) with the highest number at the end of the name:
$lastFolder = Get-ChildItem -Path 'D:\test' -Directory | Where-Object { $_.Name -match '(.+)_(\d+)$' } |
Group-Object -Property #{Expression={ $matches[1] }} | ForEach-Object {
$_.Group | Sort-Object -Property #{Expression={ [int]$matches[2] }} | Select-Object -Last 1
}
# for demo, just output the FullName property of the folders found
$lastFolder.FullName
Output:
D:\test\ABC_03_00_016_5
D:\test\DEF_03_00_016_10
Regex details:
( Match the regular expression below and capture its match into backreference number 1
. Match any single character that is not a line break character
+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
)
_ Match the character “_” literally
( Match the regular expression below and capture its match into backreference number 2
\d Match a single digit 0..9
+ Between one and unlimited times, as many times as possible, giving back as needed (greedy)
)
$ Assert position at the end of the string (or before the line break at the end of the string, if any)
If you're wanting to remove all of the directories except for the one ending with the largest number, then I'd suggest PowerShell too:
Get-ChildItem -Path 'C:\Users\Naqqash\Desktop' -Filter '*_*_*_*_*' -Directory |
Where-Object { $_.Name -Match '(.+)_(\d+)$' } |
Sort-Object -Property { [Int]$($_.Name).Split('_')[-1] } |
Select-Object -SkipLast 1 |
Remove-Item
Please remember to adjust the path on line 1 to that holding your directories.
The example above requires PowerShell v5.0
The first method can be used only if all folder names have the same length, i.e. leading zeros are used to make sure that all numbers in all folder names have same number of digits.
#echo off
set "LastFolder="
for /F "eol=| delims=" %%I in ('dir "%~dp0ABC_03_00_016_*" /AD /B /O-N 2^>nul') do set "LastFolder=%%I" & goto HaveFolder
echo Found no folder matching pattern ABC_03_00_016_* in "%~dp0".
goto :EOF
:HaveFolder
echo Last folder according to sorted folder names is: %LastFolder%
The task to get folder name with greatest last number is more difficult on number of digits differs on last number.
#echo off
set "LastFolder="
setlocal EnableExtensions EnableDelayedExpansion
set "FolderNumber=-1"
for /F "eol=| delims=" %%I in ('dir "%~dp0ABC_03_00_016_*" /AD /B 2^>nul') do (
for /F "eol=| tokens=5 delims=_" %%J in ("%%I") do (
if %%J GTR !FolderNumber! (
set "LastFolder=%%I"
set "FolderNumber=%%J"
)
)
)
endlocal & set "LastFolder=%LastFolder%"
if not defined LastFolder (
echo Found no folder matching pattern ABC_03_00_016_* in "%~dp0".
) else (
echo Last folder according to last number in name is: %LastFolder%
)
Note: The last number in folder name should have never leading zeros on using the second code above. A number with one or more leading 0 is interpreted as octal number which means 08, 09, 18, 19, etc. are invalid octal numbers and are interpreted for that reason with value 0 by command IF on making the integer comparison. There would be additional code necessary above the IF condition to first remove all leading 0 from number string before doing the integer comparison.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /? ... explains %~dp0 (drive and path of batch file ending with a backslash).
dir /?
echo /?
endlocal /?
for /?
if /?
set /?
setlocal /?
Read the Microsoft article about Using command redirection operators for an explanation of 2>nul. The redirection operator > must be escaped with caret character ^ on FOR command line to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded dir command line with using a separate command process started in background.

how to format my txt file output into excel in wincmd or powershell

I have these a .txt with the below output and need this in a excel with column name as "Name, ID, Location, Role" and repective details below in their rows.
Name: Murali
ID: 485
location: Trichy
Role: AS
Name: John
ID: 584
location: NY
Role: AS
Name: Oisce
ID: 358
location: NBC
Role: AS
Thanks in advance!!
The following is a generic PowerShell v3+ solution that works with any field names of any number (assuming that the lines repeat with the same field names, in the same order); it transforms the input directly into a CSV file that can be opened in Excel:
# Determine input and output file.
$inFile = 'file.txt'
$outFile = 'file.csv'
# Extract the headers from the input file, using an ordered hashtable.
# Lines are read, and each line's 1st field is added as a key to the hashtable
# until a duplicate value is found in the 1st field.
$headers = [ordered] #{}
foreach($line in (Get-Content $inFile)) {
$colName = ($line -split ': ', 2)[0]
if ($headers.Contains($colName)) { break }
$headers.$colName = $null
}
# Construct the header row and send it to the output file.
# Choose a suitable character encoding.
"`"$($headers.Keys -join '","')`"" | Out-File -Encoding Utf8 $outFile
#`# Now loop over all lines and write the data rows.
$numCols = $headers.Count
$outLine = ''
$sep = ''
$i = 0
foreach($line in (Get-Content $inFile)) {
++$i # Count lines starting at 1.
$val = ($line -split ': ', 2)[1]
# Enclose the value in "...", if necessary.
if ($val -match '[ ,"]') { $val = "`"$($val -replace '"', '\"')`"" }
#`# Add to the output line at hand.
$outLine += $sep + $val
if ($i % $numCols -eq 0) { # Last column value.
# Complete output line assembled, append it to the file.
# Note that the -Encoding value must match the one above.
$outLine | Add-Content -Encoding Utf8 $outFile
$sep = ''
$outLine = ''
} else { # 1st or interior column value
$sep = ','
}
}
Here is a pure batch-file solution; provide the text file to reformat as a command line argument:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "_FILES=%~1"
for /F "delims== eol==" %%E in ('2^> nul set $') do set "%%E="
for %%F in ("%_FILES%") do (
if /I "%%~xF"==".csv" (set "EXT=_NEW.csv") else set "EXT=.csv"
set "FLAG=#"
< "%%~F" set /P LONE=""
set "FILE=%%~F"
setlocal EnableDelayedExpansion
> "%%~dpnF!EXT!" (
for /F "tokens=1,* delims=: eol=:" %%K in ('type "!FILE!" ^& echo^(!LONE!') do (
endlocal
set "HEAD=%%K"
set "DATA=%%L"
if defined FLAG set "FLAG=,"
set "LINE=,"
setlocal EnableDelayedExpansion
if defined $!HEAD! (
for /F "tokens=1,* delims== eol==" %%G in ('set $') do (
endlocal
set "NAME=%%G"
set "ITEM=%%H"
setlocal EnableDelayedExpansion
for /F "tokens=1,* delims=: eol=:" %%E in ("!FLAG!"!NAME:*$^=!",:!LINE!"!ITEM!",") do (
endlocal
if defined FLAG set "FLAG=%%E"
set "LINE=%%F"
setlocal EnableDelayedExpansion
)
)
if defined FLAG (
echo(!FLAG:~1,-1!
echo(!LINE:~1,-1!
endlocal
set "FLAG="
) else (
echo(!LINE:~1,-1!
endlocal
)
for /F "delims== eol==" %%E in ('set $') do set "%%E="
setlocal EnableDelayedExpansion
)
if "!DATA:~,1!"==" " set "DATA=!DATA:~1!"
for /F "delims=" %%E in ("$!HEAD!=!DATA!") do (
endlocal
set "%%E"
setlocal EnableDelayedExpansion
)
)
)
endlocal
)
endlocal
exit /B
The output CSV file using your input file would look like:
"ID","location","Name","Role"
"485","Trichy","Murali","AS"
"584","NY","John","AS"
"358","NBC","Oisce","AS"
Update
Here is an improved version of the script that preserves the original order of fields as they first appear in the input file:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "_FILES=%~1"
set "_RSLTS=%~2"
rem // Clear all dynamic variables:
for /F "delims==" %%E in ('2^> nul set #') do set "%%E="
for /F "delims==" %%E in ('2^> nul set $') do set "%%E="
rem // Loop through the given file(s) to resolve its/their path(s):
for %%F in ("%_FILES%") do (
if /I "%%~xF"==".csv" (set "FEXT=_NEW.csv") else set "FEXT=.csv"
set "FLAG=#"
< "%%~F" set /P LONE=""
set "FILE=%%~F"
set "RSLT=%%~dpnF"
set /A "IDX=0"
setlocal EnableDelayedExpansion
if defined _RSLTS (set "RSLT=con") else set "RSLT=!RSLT!!FEXT!"
rem // Write to output file all at once:
> "!RSLT!" (
rem /* Read input file line by line, repeat first line once again finally, because
rem this contains an already present header for sure and therefore initiates
rem returning the previously collected last output data row: */
for /F "tokens=1,* delims=: eol=:" %%K in ('type "!FILE!" ^& echo^(!LONE!') do (
endlocal
set "HEAD=%%K"
set "DATA=%%L"
if defined FLAG set "FLAG=,"
set "LINE=,"
set /A "IDX+=1"
setlocal EnableDelayedExpansion
rem /* Check if current header is already present; if so, begin collecting a
rem new data row and return the previously collected one: */
if defined $!HEAD! (
rem // Read dynamic variable corresponding to current header indirectly:
for /F "tokens=1,* delims==" %%G in ('set #') do (
for /F "tokens=1,* delims=:" %%E in ("!FLAG!"%%H",:!LINE!"!$%%H!",") do (
endlocal
if defined FLAG set "FLAG=%%E"
set "LINE=%%F"
setlocal EnableDelayedExpansion
)
)
rem /* Return the currently collected data row, together with the column
rem header row in case it is the first time: */
if defined FLAG (
echo(!FLAG:~1,-1!
echo(!LINE:~1,-1!
endlocal
set "FLAG="
) else (
echo(!LINE:~1,-1!
endlocal
)
rem // Clear all dynamic variables:
for /F "delims==" %%E in ('set #') do set "%%E="
for /F "delims==" %%E in ('set $') do set "%%E="
set /A "IDX=1"
setlocal EnableDelayedExpansion
)
rem // Remove leading white-spaces from data value:
for /F "tokens=* eol= " %%E in ("!DATA!") do (
endlocal
set "DATA=%%E"
setlocal EnableDelayedExpansion
)
rem /* Assign name of dynamic variable corresponding with current header to a
rem numeric dynamic variable for preserving the order of (first) headers: */
set "IDX=000!IDX!"
for /F "delims=" %%E in ("#!IDX:~-4!=!HEAD!") do (
endlocal
set "%%E"
setlocal EnableDelayedExpansion
)
rem // Assign data value to dynamic variable corresponding with current header:
for /F "delims=" %%E in ("$!HEAD!=!DATA!") do (
endlocal
set "%%E"
setlocal EnableDelayedExpansion
)
)
)
endlocal
)
endlocal
exit /B
The output CSV file using your input file would look like:
"Name","ID","location","Role"
"Murali","485","Trichy","AS"
"John","584","NY","AS"
"Oisce","358","NBC","AS"
Without a need to format this in Excel is simply done by separating your fields with a character like , or ; (depending on your language settings in Excel) and renaming the file to have the externsion .csv, eg "data.csv".
When you open the file from explorer Excel will open this file and arrange your data in columns and rows. You could format the layout in Excel and save it as a XLSX file.
Here your data formatted as CSV with , as separator
Name, id, location, role
Murali, 485, Trichy, AS
John, 584, NY, AS
Oisce, 358, NBC, AS
EDIT: because of the challenge of mklement0 here a one line Ruby solution, NOT PART OF THE ANSWER, I'm sure not the shortest or best solution, just a proof of concept
%(
Name: Murali
ID: 485
location: Trichy
Role: AS
Name: John
ID: 584
location: NY
Role: AS
Name: Oisce
ID: 358
location: NBC
Role: AS
).scan(/(^\w+):( \w+)/).transpose.each_with_index.map{|r,i|i==0 ? r.uniq.collect{|x| %("#{x.strip}") }.join(',') : r.each_slice(4).map{|s|s.collect{|x| %("#{x.strip}") }.join(',')}}.join("\n")
here split up by operation with explanation
s.scan(/(^\w+):( \w+)/) # array of arrays (key, value), based on regular expression
.transpose.each_with_index.map{|r,i| # transpose to array of headers and values, use Enumerator
i==0 ? # first line = headers
r.uniq.collect{|x| %("#{x}") }.join(',') # unique values, surround with "", join with ","
: # rest of the lines = data
r.each_slice(4).map{|s| # pieces of 4 items
s.collect{|x| %("#{x.strip}") } # surround with ""
.join(',')}} # join with ","
.join("\n") # join with new line char
Or a shorter variant (s contains the string to parse)
(s.lines[0..3].join.scan(/(^\w+): (\w+)/).transpose.first + s.lines.join.scan(/(^\w+): (\w+)/).transpose.last).each_slice(4).map{|a| %("#{a.join('","')}")}
EDIT2:
Reading from a file is as simple as Fire.read("data.txt").scan(..) etc.
I can read files in memory GB's big if necessary but yes Ruby has the lazy method which reads only the part that is needed at the moment.
You can also use eg File.foreach('data.txt').first(10) which stops at reading at the 10th line.
Didn't want to make it too complicated though.
See eg http://blog.honeybadger.io/using-lazy-enumerators-to-work-with-large-files-in-ruby/

Replace certain characters from filenames in all files in a folder and his sub-folders

How can I change some character (#,%,&,$) in the filename of multiple files in one folder and his sub-folders?
Ravi Thapliyal answer how to do that with PowerShell but just for the files in a specific folder (Replace or delete certain characters from filenames of all files in a folder).
I need to generalize that for all files in all folders in the selected one.
Your own answer will only change one char, here is a solution with an array RegEx in one go, remove the chars you want to keep from $replace.
[char[]]$replace = '!##$%^&*(){}[]":;,<>/|\+=`~ '''
$regex = ($replace | % {[regex]::Escape($_)}) -join '|'
Get-ChildItem -recurse |
ForEach {
if ($_.Name -match $RegEx){
Ren $_.Fullname -NewName $($_.Name -replace $RegEx, '_') -whatif
}
}
If the output looks ok, remove the -whatif
Edit removed -File option from Get-ChildItem as it requires a recent Powershell version and wasn't necessary.
Edit2 I regularly state that Rename-Item accepts piped input, so here is a more straight forward version:
[char[]]$replace = '!##$%^&*(){}[]":;,<>/|\+=`~ '''
$regex = ($replace | % {[regex]::Escape($_)}) -join '|'
Get-ChildItem -recurse |
Where-Object { $_.Name -match $RegEx} |
Rename-Item -NewName {$_.Name -replace $RegEx, '_'} -whatif
I found this answer from Dustin Malone:
Get-ChildItem -recurse -name | ForEach-Object { Move-Item $_ $_.replace(" ", "_") }
It gives an error but do the trick.
The task can also be accomplished by a Windows cmd script (batch-file) -- however, there are some limitations, unless appropriate work-arounds are implemented.
The following script replaces #, %, &, $, ( and ) in file names. You can add more characters except ^, !, = and ~, which cannot be processed (an if query lets the replacement be skipped in order to avoid syntax errors). If you provide characters with a code greater than 127 = 0x7F, you may probably need to switch the code page adequately (see the chcp command). Note that the script does not rename any files but only displays them, until you remove the upper-case ECHO command:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "_ROOT=D:\path\to\root\dir" & rem /* (assign `%~1` here instead for the script to
rem accept the root dir. as command line argument) */
set _CHAR="#" "%%" "&" "$" "(" ")" & rem // (a `%` sign needs to be doubled!)
set "_REPL=_" & rem // (replacement character or string; this may also be empty)
if not defined _ROOT set "_ROOT=."
for /R "%_ROOT%" %%F in ("*.*") do (
set "FILE=%%~F"
set "NAME=%%~nxF"
setlocal EnableDelayedExpansion
for %%C in (!_CHAR!) do (
if not "%%~C"=="^" if not "%%~C"=="!" if not "%%~C"=="=" if not "%%~C"=="~" (
set "NAME=!NAME:%%~C=%_REPL%!"
)
)
ECHO ren "!FILE!" "!NAME!"
endlocal
)
endlocal
exit /B
To replace also the characters ^ and !, the script needs to be extended a bit:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "_ROOT=D:\path\to\root\dir" & rem /* (assign `%~1` here instead for the script to
rem accept the root dir. as command line argument) */
set _CHAR="#" "%%" "&" "$" "(" ")" "^" "!" & rem // (a `%` sign needs to be doubled!)
set "_REPL=_" & rem // (replacement character or string; this may also be empty)
if not defined _ROOT set "_ROOT=."
for /R "%_ROOT%" %%F in ("*.*") do (
set "FILE=%%~F"
set "NAME=%%~nxF"
setlocal EnableDelayedExpansion
for %%C in (!_CHAR!) do (
if "%%~C"=="^" (
set "NAME=!NAME:^=^^!"
call set "NAME=%%NAME:^=%_REPL%%%"
) else if "%%~C"=="!" (
call set "NAME=%%NAME:^!=%_REPL%%%"
) else if not "%%~C"=="=" if not "%%~C"=="~" (
set "NAME=!NAME:%%~C=%_REPL%!"
)
)
ECHO ren "!FILE!" "!NAME!"
endlocal
)
endlocal
exit /B
To replace also the characters = and ~, the script needs to be extended much more:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "_ROOT=D:\path\to\root\dir" & rem /* (assign `%~1` here instead for the script to
rem accept the root dir. as command line argument) */
set _CHAR="#" "%%" "&" "$" "(" ")" "^" "!" "=" "~" & rem // (a `%` sign needs to be doubled!)
set "_REPL=_" & rem // (replacement character or string; this may also be empty)
set "$LEN="
if not defined _ROOT set "_ROOT=."
for /R "%_ROOT%" %%F in ("*.*") do (
set "FILE=%%~F"
set "NAME=%%~nxF"
setlocal EnableDelayedExpansion
for %%C in (!_CHAR!) do (
if "%%~C"=="^" (
set "NAME=!NAME:^=^^!"
call set "NAME=%%NAME:^=%_REPL%%%"
) else if "%%~C"=="!" (
call set "NAME=%%NAME:^!=%_REPL%%%"
) else if "%%~C"=="~" (
call :FAST NAME NAME "%%~C" "%_REPL%"
) else if "%%~C"=="=" (
call :SLOW NAME NAME "%%~C" "%_REPL%"
) else (
set "NAME=!NAME:%%~C=%_REPL%!"
)
)
ECHO ren "!FILE!" "!NAME!"
endlocal
)
endlocal
exit /B
:FAST rtn_string ref_string val_char val_repl
rem // This works for `~` and `*`, but NOT for `=`!
setlocal DisableDelayedExpansion
set "STR="
set "CHR=%~3"
if not defined CHR goto :FQUIT
setlocal EnableDelayedExpansion
set "CHR=!CHR:~,1!"
set "STR=!%~2!"
:FLOOP
if defined STR (
set "END=!STR:*%~3=!"
if not "!END!"=="!STR!" (
for /F "eol=%CHR% delims=%CHR%" %%L in ("!STR!") do (
for /F "delims=" %%K in (^""!END!"^") do (
endlocal
set "STR=%%L%~4%%~K"
setlocal EnableDelayedExpansion
)
)
goto :FLOOP
)
)
:FQUIT
for /F "delims=" %%K in (^""!STR!"^") do (
endlocal
set "STR=%%~K"
)
endlocal & set "%~1=%STR%"
exit /B
:SLOW rtn_string ref_string val_char val_repl
rem // This works even for `=`.
setlocal DisableDelayedExpansion
set "STR="
set "CHR=%~3"
set "RPL=%~4"
if not defined CHR goto :SQUIT
if not defined $LEN call :LEN $LEN RPL
setlocal EnableDelayedExpansion
set "CHR=!CHR:~,1!"
set "STR=!%~2!"
set /A "IDX=0"
:SLOOP
set /A "NXT=IDX+1"
if defined STR (
set "POS=!STR:~%IDX%,1!"
if defined POS (
if "!POS!"=="!CHR!" (
set "STR=!STR:~,%IDX%!!RPL!!STR:~%NXT%!"
set /A "NXT=IDX+$LEN"
)
set /A "IDX=NXT"
goto :SLOOP
)
)
:SQUIT
for /F "delims=" %%K in (^""!STR!"^") do (
endlocal
set "STR=%%~K"
set "$LEN=%$LEN%"
)
endlocal & set "%~1=%STR%" & set "$LEN=%$LEN%"
exit /B
:LEN rtn_length ref_string
setlocal EnableDelayedExpansion
set "STR=!%~2!"
if not defined STR (set /A LEN=0) else (set /A LEN=1)
for %%L in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
if defined STR (
set "INT=!STR:~%%L!"
if not "!INT!"=="" set /A LEN+=%%L & set "STR=!INT!"
)
)
endlocal & set "%~1=%LEN%"
exit /B

Use PowerShell to determine length of a line in a text file

I'm new to PowerShell, but I would like to use it, because there isn't a easy way to get the length of a string in Windows batch.
I need to write a piece of code in PowerShell that go through each line in a .txt file and determine the character length of that line. If the character length is over 250 then....etc.
The ....etc part is not important at the moment :)
In Windows batch I would write it like this:
FOR /F %%A IN ("C:\TestFile.txt") DO (
SET LINE=%%A
If LINE > 250 characters then ( ' This line is made up
....etc
)
How can I do it?
The following will do what you want:
$data = Get-Content "C:\TestFile.txt"
foreach($line in $data)
{
if ($line.Length -gt 250) {
Write-Host "A match"
}
}
Try this:
:: Not fully tested:
for /f "delims=" %%s in (C:\TestFile.txt) do (
set "x=%%s" & set /A y+=1
setlocal enabledelayedexpansion
for /f "skip=1 delims=:" %%i in ('"(set x&echo()|findstr /o ".*""') do set/a n=%%i-4
if !n! gtr 250 echo Line !y! Length !n!
endlocal
)
Was looking for this today and found an elegant solution here: https://softwarerecs.stackexchange.com/questions/38934/finding-the-longest-line-of-a-document/38936
GC "c:\folder\file.txt" | Measure -Property length -Maximum | Select Maximum
GC "c:\folder\file.txt" | Sort -Property length | Select -last 1
Important: credit goes to Pimp Juice IT from the link above, I'm just copy/pasting : )