I am writing a Powershell script that compares the number of rows in 2 parquet files that are created each hour to monitor the number of rows etc.
I have found the Parquet.NET project, but since all I want to do is query the count of rows, I wonder if this is overkill. Is there a module for parquet?
Related
I have thousands of csv files and they basically have 2 formats. One type of 2 formats is that in those csv files there are 100 rows and 2 columns. The other type of csv files has 50 columns and 5 rows. The numbers are given just to provide an example.
What I want to do is to write a Matlab code that will extract the complete second row of the csv files with the first format and make it the first row of the csv files with the second format. The number of the csv files with the first and second format is equal.
Any help is appreciated.
I'm trying to figure out is there a way to read CSV file in Powershell
and import in cell 1 column 1 the data -"machine name" to get the
monitor serial number ?
Right now I have this code in which I manually replace the machine name (Station01) with the monitor serial number and then I copy the serial number and paste it into the csv file.
I'm hoping the that Csv file can be used to record multiple monitors for each serial number on the columns.
Example, 4 or 5 lines one for each machine containing the serial numbers 2 or 3 monitors.
This I want to achieve using a Powershell script -
Get input list of machine name on csventer code here run the report and see the serial number on it
Earlier we were using vb script that used to work.
But it doesn't work for some reason now.
Please help me with such Powershell script?
I have to combine a lot of files , mostly CSV, already have code to combine however I need first to trim the desired csv files so I can get the data that I want. Each CSV has first 2 columns of 8 rows which contain data that I want. and then just below those there is a row that generates 8 columns. I am only having issue grabbing data from the first 8 rows of the 2 columns.
Example of the csv first 3 rows:
Target Name: MIAW
Target OS: Windows
Last Updated On: June 27 2019 07:35:11
This is the data that I want, the first 3 rows are like this, with 2 columns. My idea is to store the 3 values of the 2nd column each into a variable and then use it with the rest of my code.
As I only have a problem extracting the data, and since the way the CSV are formated there is no header at all, it is hard to come up with an easy way to read the 2nd column data. Below is an example, this of course will be used to process several files so it will be a foreach, but I want to come up first with the simple code for 1 file so I can adapt it myself to a foreach.
$a = MIAW-Results-20190627T203644Z.csv
Write-Host "$a[1].col2"
This would work if and only if I had a header called col2, I could name it with the first value on the 2nd column but the issue is that that value will change for CSV file. So the code I tried would not work for example if I were to import several files using:
$InFiles = Get-ChildItem -Path $PSScriptRoot\ *.csv -File |
Where-Object Name -like *results*
Each CSV will have a different value for the first value on the 2nd column.
Is there an easier way to just grab the 3 rows of the second column that I need? I need to grab each one and store each in a different variable.
dumping a Postgres table out by sections is yielding sections that are 30GB+ in size. The files are landing on a windows 2008 server. I'm trying to count the rows in the csv to ensure I have a row count that I expect (22,725,303 to be exact). I can count the rows in the section that I expect to dump - but I am not sure if I'm getting them all.
It's a 190M row table so sections of table is the way to go.
so how can I count the rows so I know I've got the full section?
In a PL/pgSQL function, you can get the count of rows processed by the last command - since Postgres 9.3 including COPY - with:
GET DIAGNOSTICS x = ROW_COUNT;
Get the count of rows from a COPY command
I am trying to write/learn a simple Perl parser for some CSV files that I have and I need some help.
In a directory I have a series of date-indexed CSV files in the form of Text-Date.csv. The date is in the form of Month-DD-YYYY (ex., January-07-2011). For each weekday there is a CSV file generated.
The Perl script should open each file look for a particular row that matches a user-entered criteria and return that row. Each row is stock price data with different stocks in different rows. What the script should do is return the price of a particular stock (ex., IBM) across all dates that CSVs are generated.
I have the parser working for a specific CSV/date that I choose, but I want to be able to pluck out the row in all CSVs. Also when I print the IBM price for each dated CSV I want to display the date next to the price (ex., January-07-2011 IBM 147.93).
Can you help me get this done?
If your question is how to crawl a bunch of files and run some function on each one, you probably want File::Find. To parse CSV, definitely use Text::xSV and not a custom parser. There is more to parsing CSV than calling split(",").
To parse CSV files, use the Text::CSV module.
It is more complex to decide how you are going to apply the criteria - you'll need to determine what the user specifies and work out how to translate that into Perl code that evaluates the condition correctly.