I am trying to get the list of documents older than X date from my collection. However my query is not working for datetime.
I have a mongo atlas cluster with multiple databases and collections. I want to lookup for a specific collection and delete the documents under that collection that are older than X date and time. I could get the list of records and apply the filter on a powershell object but when I use the query while retrieving data, no results are returning.
$connstring = "mongodb+srv://User:pwd#clus56tg-a0ov.azure.mongodb.net"
Connect-mdbc -ConnectionString $connstring
$global:Server = $Server
Write-Host "Server `$Server $($Server.Settings.Server)"
$allDatabases = ($Server.GetDatabaseNames()).Where{($_ -notmatch 'local') -and ($_ -notmatch 'admin') -and ($_ -notmatch 'config')}
$date = "9/16/2019 0:00:00 PM"
$query = New-MdbcQuery DateTime -LTE $date
#printing the query looks like this
#{ "DateTime" : { "$lte" : "9/16/2019 0:00:00 PM" } }
$settings = New-Object MongoDB.Driver.MongoCollectionSettings
$settings.GuidRepresentation = 2
foreach($dbname in $allDatabases)
{
$database = $Server.GetDatabase($dbname)
$collections = $database.GetCollection('consumerstate', $settings)
$collections.Count()
foreach($coll in $collections)
{
Write-Output("Database Name: $coll.Database.Name, Collection Name: $coll.Name")
###This code works###
$objs = Get-MdbcData -As PS -Collection $coll
$objs.Where{$_.DateTime -le '9/16/2019 0:00:00 PM'}
#or this works
Get-MdbcData(New-MdbcQuery _id -EQ c33af1-acef-4g6a-ai05-aase56d5)
########
###This does not return any results###
Get-MdbcData -As PS -Collection $coll -Query $query
#or this
Get-MdbcData -Collection $coll -Query $query
return
}
}
The record output looks like this.
Sendfeedlooppacket {LatestRequestStatusTypeId, LatestRequestStatusDate, DisplayId, CId...}
_id 9f8a7-53vva-4xdr-b45c-a34hsd22e282
TenantId 12sf3484c-e349-43a9-9951-80345sf268452
Consumer RequestService.MessageConsumers.RequestResponseConsumer
DateTime 9/19/2019 5:39:04 PM
I have gone through the tests(https://github.com/nightroman/Mdbc/blob/master/Tests/New-MdbcQuery.test.ps1)and noticed some with datetime and the interpreted query looked like this.
test { New-MdbcQuery Name -In $date } '{ "Name" : { "$in" : [ISODate("2011-11-11T00:00:00Z")] } }'
But in my case the date is not wrapped around ISODate.
What I was missing is casting the datatype in the query.
$date = "9/16/2019 0:00:00 PM"
$query = New-MdbcQuery DateTime -LTE ([datetime]$date).
I need to do the same for guid and other datatypes.
Thanks to #nightroman for his help.
Related
I want to extract text from a .txt file. The way the file is layed out is in this format (below first block). Optimally, I would like for the powershell script to take the content of username and votecount and output them side by side. With an integer of 25>= add the letter D beside it. With the output adding itself to a pre-existing output file. Say this week is week 1. And testuser voted 25 times. They should have the output "testuser" 25D. But say in week 2 they voted 24 times. Then it should be "testuser" 49D. However say they had 25 again. Output should then be "testuser" 50DD or 50D2?.. I have what I think should work as an initial baseline for the script which in itself doesn't work.. But combining an output with a pre existing output is beyond my capability. This needs to parse an entire txt file of some 100+ people. So imagine there's like an extra 100 users..
{
"username": "testuser",
"votecount": "42",
"votesclaimed": "0",
"lastvotetime": "2022-11-04 09:08:29",
"steamid": "00000000000000000000"
}
Below is what I am working with.
Get-Content -Raw C:\Users\--------\Desktop\votes.txt |
ConvertFrom-txt |
ForEach-Object {
[pscustomobject] #{
UserName = $_.username
VoteCount = '{0}{1}' -f $_.votecount, ('', 'D')[[int] $_.votecount -gt 25]
}
} |
Export-Csv -NoTypeInformation -Encoding utf8 C:\Users\---------\Desktop\outvotes.csv
Try following :
$match = Select-String -Path "c:\temp\test.txt" -Pattern '^\s*"(?<key>[^"]+)"\s*:\s*"(?<value>[^"]+)'
$table = [System.Collections.ArrayList]::new()
foreach( $row in $match.Matches )
{
$key = $row.Groups["key"].Value
$value = $row.Groups["value"].Value
if($key -eq "username") {
$newRow = New-Object -TypeName psobject
$table.Add($newRow) | Out-Null
}
$newRow | Add-Member -NotePropertyName $key -NotePropertyValue $value
}
$table | Format-Table
$groups = $table | Group-Object {$_.username}
I have a DataTable ($dt = New-Object system.Data.datatable) which contains entries as below:
My objective is :
Find servers which same names ,ie , from ServerID column trim the part after underscore (_) (which I achieved via Split()) and then compare with rest of the rows.
If the Server Name is same, check the value of all respective "Status" column
If none of the columns have "IN PROCESS" in them for the respective server, then print the ServerID.
This is what I came up with but got stuck since values are not returned correctly:
foreach($backupid in ($dt.'ServerID' | %{foreach ($y in $_){$y.Split('_')[0]}} | sort -Unique)){
foreach ($row in $dt){
if ($row.'ServerID ' -match "^$backupid" -and $row.Status -ne "IN PROCESS" ){
$row.'ServerID '
}
}
}
Just use a hash table to check whether a server id is (not) IN PROCESS, like:
$dt = ConvertFrom-Csv #'
Server,Status
abc_123,"IN PROCESS"
abc_345,"INACTIVE"
abc_546,"INACTIVE"
xyz_123,"INACTIVE"
xyz_457,"INACTIVE"
xyz_230,"INACTIVE"
'#
$InProcess = #{}
$dt | Foreach-Object {
$Id = $_.Server.Split('_')[0]
if (!$InProcess.Contains($Id)) { $InProcess[$Id] = $False }
if ($_.Status -eq 'IN PROCESS') { $InProcess[$Id] = $True }
}
$dt | Foreach-Object {
$Id = $_.Server.Split('_')[0]
if ($InProcess[$Id] -eq $False) { $_ }
}
Server Status
------ ------
xyz_123 INACTIVE
xyz_457 INACTIVE
xyz_230 INACTIVE
Instead of nested loops, try Group-Object!
# Group objects by the first part of the Server ID
$dt |Group { $_.ServerID.Split('_')[0] } |Where-Object {
# Then find only the groups with no objects where Status is IN_PROGRESS
$_.Group.Status -notcontains 'IN_PROGRESS'
} |ForEach-Oject -MemberName ServerID # Output just the Server value
Running a sql script for a list of databases.
The output on powershell is not separated. How could I separate them based on server name.
$SERVERS = gc "C:\Users\listOfServers.txt"
foreach ($SERVER in $SERVERS) {
$InvokeParams = #{
Server = $SERVER
Database = 'test'
Username = 'admin'
Password = 'testpassword'
InputFile = 'C:\business.sql'
}
Invoke-SqlCmd #InvokeParams
}
Right now my output looks like this :
ValueDate: 1/30/2019 12:00:00 AM
PrevValueDate: 1/29/2019 12:00:00 AM
Count:100
ValueDate: 3/30/2019 12:00:00 AM
PrevValueDate: 3/29/2019 12:00:00 AM
Count:200
ValueDate: 4/30/2019 12:00:00 AM
PrevValueDate: 4/29/2019 12:00:00 AM
Count:2100
ValueDate: 11/30/2019 12:00:00 AM
PrevValueDate: 11/29/2019 12:00:00 AM
Count:12200
Goal is : Server 1 (output server 1) server 2 (output server 2)
I would like to add a parameter that gives the server name for each output- or sort like an Id to separate them.
Goal is to export the output into an Excel sheet - not working at the moment
$out = Invoke-SqlCmd #InvokeParams | Format-Table
$path = 'C:\Users\test1.csv'
$out | Export-Csv -Path $path
Invoke-Item -Path $path
You can use calculated properties to achieve this:
$SERVERS = gc "C:\Users\listOfServers.txt"
$out = foreach ($SERVER in $SERVERS) {
$InvokeParams = #{
Server = $SERVER
Database = 'test'
Username = 'admin'
Password = 'testpassword'
InputFile = 'C:\business.sql'
}
Invoke-SqlCmd #InvokeParams | Select-Object -Property *, #{L='Server'; E={$SERVER}}
}
$path = 'C:\Users\test1.csv'
$out | Export-Csv -Path $path
Invoke-Item -Path $path
I am trying to select items by date in a CSV file using PowerShell. The format in the CSV file for the date is 1/8/2018 10:04:00 AM. When I run this I get no data although I know that data exists.
$events = Import-Csv c:\normtest\server2_perf.csv | foreach {
New-Object PSObject -prop #{
Date = [DateTime]::Parse($_.Date);
CPULoad = $_.CPULoad;
MemLoad = $_.Memload
}
}
$events | Where { $_.Date -eq (Get-Date).AddDays(-4) }
As you have a time part to your date, this will only work for exactly 4 days from now (i.e. where time of day = right now).
Assuming this part works correctly: Date = [DateTime]::Parse($_.Date);, you can do this:
$start = (Get-Date).Date.AddDays(4)
$fin = $start.AddDays(1) # assuming 1 day window
$events |
Where {$_.Date -gt $start -and $_.Date -lt $fin}
Alternatively, you could treat the date field as string:
$events = Import-Csv c:\normtest\server2_perf.csv |
Where {$_.Date -like "$(Get-Date).AddDays(-4).ToString("M/d/yyyy"))*" }
Assuming your date format is "M/d/yyyy"
I am working on query user command in PowerShell to filter the content to get the users who wer disconnected for more than 2 days on the server.
This is my result:
USERNAME SESSIONNAME ID STATE IDLE TIME LOGON TIME
a_admin 2 Disc 20+16:56 19.08.2015
b_admin 3 Disc . 10.12.2015
c_admin 4 Disc 5+22:33 24.08.2015
d_admin 5 Disc 17:47 17.12.2015
e_admin 6 Disc 101+18:58 02.09.2015
f_admin 7 Disc 1+01:27 14.12.2015
The problem is the query user don't retrieve the data as an object format, so I can't select any column from these data, can any one help me to find a way to filter this content? Also, I am having a problem in the content of the idle time. It seems weird!?
I tried to put the output in a text file then get the content back and do some filtration, but the result is the same (USERNAME with empty records).
query user produces string output. You can't convert that to objects by piping it into Format-Table. And Select-Object won't do with the output of Format-Table what you seem to expect anyway.
Use a regular expression match to transform the string output into a list of objects:
$server = 'servername'
$re = '(\w+)\s+?(\S*)\s+?(\d+)\s+Disc\s+(\S+)\s+(\d+\.\d+\.\d+)'
query user /server:$server | Where-Object { $_ -match $re } | ForEach-Object {
New-Object -Type PSCustomObject -Property #{
'Username' = $matches[1]
'SessionID' = $matches[3]
'IdleTime' = $matches[4]
'LogonTime' = $matches[5]
}
} | Select-Object Username, IdleTime
This will give you everything as string values, though. Since you want to filter on the idle time you may want to convert the values to appropriate types. Using a more elaborate regular expression (with named groups) will help with that.
$server = 'servername'
$re = '(?<username>\w+)\s+?' +
'(\S*)\s+?' +
'(?<session>\d+)\s+' +
'Disc\s+' +
'(?:(?:(?<days>\d+)\+)?(?<hours>\d+):)?(?<minutes>\d+)\s+' +
'(?<logon>\d+\.\d+\.\d+)'
query user /server:$server | Where-Object { $_ -match $re } | ForEach-Object {
New-Object -Type PSCustomObject -Property #{
'Username' = $matches['username']
'SessionID' = [int]$matches['session']
'IdleTime' = if ($matches['days']) {
New-TimeSpan -Days $matches['days'] -Hours $matches['hours'] -Minutes $matches['minutes']
} elseif ($matches['hours']) {
New-TimeSpan -Hours $matches['hours'] -Minutes $matches['minutes']
} else {
New-TimeSpan -Minutes $matches['minutes']
}
'LogonTime' = [DateTime]::ParseExact($matches['logon'], 'dd\.MM\.yyyy', [Globalization.CultureInfo]::InvariantCulture)
}
} | Where-Object {
$_.IdleTime.TotalDays -gt 2
} | Select-Object Username, IdleTime