Postgres copy to insert data into table - postgresql

I have a text file in the below format. The first row is the header. The second row is the data.
"A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M" "N" "O" "P" "Q" "R" "S" "T" "U" "V" "W" "X" "Y" "Z" "AA" "BB" "CC" "DD" "EE" "FF" "GG" "HH" "II" "JJ" "KK" "LL" "MM" "NN" "OO"
"1019311288593" "3" "3" "3421D" "1" "2" "3" "" "1500188" "201802" "" "" "11" "" "" "" "" "9710" "121598630" "1336332" "60" "20180213" "" "" "" "1" "" "00209500188021318" "" "" "" "" "1215630" "" "" "" "" "" 20180222 "green" 1
I am trying to use the following command to insert this data into a table -
\COPY service(<All columns listed above>) FROM 'test.txt' DELIMITER E'\t' CSV HEADER
But I am getting this error -
`Missing data for column B.`

Related

Digit's Values not getting extracted from Dictionary

I have a dictionary that contains Letters and Digits and i can get the values of the letter keys just fine but not the ones from digits even though they're enclosed in " . It isn't important for the question but just in case you're wondering what the bluscream.ahk file that's included looks like: https://github.com/Bluscream/ahk-scripts/blob/master/Lib/bluscream.ahk
I tried what can be seen in the code. I have no idea what to do
; Version 1
; Date 02/02/2019
#Include <bluscream>
#SingleInstance Force
Process Priority,, Below Normal
SetWorkingDir %A_ScriptDir%
#Warn
#Persistent
SetKeyDelay, 150
game_name := "LEGO" ; LEGO Jurassic World
game_title := "ahk_class TTalesWindow" ; ahk_exe LEGOJurassicWorld_DX11.EXE
chars := { "B" : "{Up}", "C" : "{Up 2}", "D" : "{Up 3}", "E" : "{Up 4}", "F" : "{Up 5}", "G" : "{Up 6}", "H" : "{Up 7}", "J" : "{Up 8}", "K" : "{Up 9}", "L" : "{Up 10}", "M" : "{Up 11}", "N" : "{Up 12}", "O" : "{Up 13}", "P" : "{Up 14}", "Q" : "{Up 15}", "R" : "{Up 16}", "S" : "{Down 18}", "T" : "{Down 17}", "U" : "{Down 16}", "V" : "{Down 15}", "W" : "{Down 14}", "X" : "{Down 13}", "Y" : "{Down 12}", "Z" : "{Down 11}", "0" : "{Down 10}", "1" : "{Down 9}", "2" : "{Down 8}", "3" : "{Down 7}", "4" : "{Down 6}", "5" : "{Down 5}", "6" : "{Down 4}", "7" : "{Down 3}", "8" : "{Down 2}", "9" : "{Down}" }
file := "codes.txt"
global noui := false
scriptlog("Started logging here...")
FileRead, LoadedText, %file%
codes := StrSplit(LoadedText, "`n", "`r")
Loop, % codes.MaxIndex()
{
if !(WinActive(game_title)) {
TrayTip, AutoHotKey, Bringing %game_name% to front to enter code...
Sleep, 1000
WinWaitActive, %game_title%
}
code := StrStrip(codes[A_Index])
length := StrLen(code)
FormatTime, timestamp, A_Now, hh:mm:ss
scriptlog("[" . timestamp . "] Now processing code: " . code . " [" . length . "] (`r`n", "", true)
splitted_code := StrSplit(code)
for i, char in splitted_code {
tosend := chars[char]
scriptlog("i:" . i . " char:" . char . " tosend:" . tosend . "`r`n", "", true)
if (tosend){
SendEvent, % tosend
}
if (i < length)
SendInput, {Right}
}
scriptlog(")`r`n","",true)
SendInput, {Enter}
}
Expected results:
[11:17:43] Started logging here...
[11:17:46] Now processing code: 28SPSR [6] (
i:1 char:2 tosend:{Down 8}
i:2 char:8 tosend:{Down 2}
i:3 char:S tosend:{Down 18}
i:4 char:P tosend:{Up 14}
i:5 char:S tosend:{Down 18}
i:6 char:R tosend:{Up 16}
)
Actual Results:
[11:17:43] Started logging here...
[11:17:46] Now processing code: 28SPSR [6] (
i:1 char:2 tosend:
i:2 char:8 tosend:
i:3 char:S tosend:{Down 18}
i:4 char:P tosend:{Up 14}
i:5 char:S tosend:{Down 18}
i:6 char:R tosend:{Up 16}
)
Looks like a bug of the object declaration. A problem about string vs numbers.
Using numbers without quotes removes the problem.
But also, doing this after declaring the object fixes the problem too:
for k,v in chars
chars[k] := v
You could fill a bug on the official board: https://www.autohotkey.com/boards/

mongodb $sum all values in object with variable key name

we have the following document
{
"_id" : "F80BBEDE6395",
"up" : {
"10" : NumberLong(1916327),
"11" : NumberLong("4557798933"),
"12" : NumberLong(1057250763),
"13" : NumberLong(10167577),
"14" : NumberLong(9464393),
"15" : NumberLong(10082616),
"16" : NumberLong(9982689),
"17" : NumberLong(8612066),
"18" : NumberLong(193879404),
"19" : NumberLong("4692628921"),
"20" : NumberLong(541674158),
"21" : NumberLong(10482839),
"22" : NumberLong(9671272)
},
"down" : {
"10" : NumberLong(1880514),
"11" : NumberLong("19698709594"),
"12" : NumberLong("28850509901"),
"13" : NumberLong(10073361),
"14" : NumberLong(9283009),
"15" : NumberLong(9533924),
"16" : NumberLong(9605101),
"17" : NumberLong(1398871165),
"18" : NumberLong("34124867465"),
"19" : NumberLong("180468996228"),
"20" : NumberLong("18472208011"),
"21" : NumberLong(10066752),
"22" : NumberLong(9288118)
},
"monthUp" : NumberLong("11113611958"),
"monthDown" : NumberLong("283073893143")
}
up and down are objects, one by day with a counter (day 10 of month, day 11, and so on).
The problem we get is how to in an update, sum all "up" values to "monthUp" and all "down" values to "monthDown"
In this example, the monthUp and monthDown are not ok, since both are not the sum of up / down objects.
This document is build from a hadoop process that updates every item on "up" and "down", since the hadoop job updates the document and we want to avoid a new job to make the "monthDown" and "monthUp" sum, will be very effective and useful for us to do the sum on the same update operation.
Thanks in advance

How do you Sed to replace the content of a string, instead of just append

I am trying to modify a file content, which has a specific structure:
{
"name" : "thename",
"value" : {
"a" : "123",
"b" : "456"
}
}
I would like to replace the content of a or b; but I can only append to the values, not replace what is in the quotes. For example instead of 123 for a, I would like to put 999.
sed -i 's/"a":*/"a": "999"/g' myfile.txt
But what I get is
{
"name" : "thename",
"value" : {
"a" : "999" "123",
"b" : "456"
}
}
How do I tell sed that I want to replace what is in the quote after that string, and not append to it?
Try:
$ sed 's/"a" *:.*/"a" : "999"/' myfile.txt
{
"name" : "thename",
"value" : {
"a" : "999"
"b" : "456"
}
}
The issue is that sed uses regular expressions, not globs. As a consequence, "a":* matches "a" followed by zero or more colons. Since, in your sample file above, "a" is followed by a space, this matches just "a".
In the code above, "a":* is replaced with "a" *:.*/ which matches "a" followed by zero or more spaces, followed by a colon, and .* which matches everything that follows the colon. In a regular expression, . matches any character and .* therefore matches zero or more of any character. This will match the entirety of the line that follows the colon.

MongoDB bulk insert failing for array of strings

Given the following array:
arr = ["{'myId': 'myVal1'}","{'myId': 'myVal2'}"]
I can insert the items one by one, e.g.
db.collection.insert(arr[0])
But it fails when I try to insert the whole array (like it says in http://docs.mongodb.org/manual/reference/method/db.collection.insert/ )
db.collection.insert(arr)
Error: not an object src/mongo/shell/collection.js:179
I'm using MongoDB 2.2.4
How can I make this work?
You are trying to insert an array of strings - mongo expects an array of json documents.
"{'foo':'bar'}" is a string.
{'foo':'bar'} is an object - a json document with one key value pair.
It looks to you like the insert is succeeding when you do
db.collection.insert("{'foo':'bar'}") but it's not doing what you think it is.
> db.collection.findOne()
{
"_id" : ObjectId("51941c94c12b0a7bbd416430"),
"0" : "{",
"1" : "'",
"2" : "f",
"3" : "o",
"4" : "o",
"5" : "'",
"6" : ":",
"7" : "'",
"8" : "b",
"9" : "a",
"10" : "r",
"11" : "'",
"12" : "}",
"trim" : function __cf__12__f__anonymous_function() {
return this.replace(/^\s+|\s+$/g, "");
},
"ltrim" : function __cf__13__f__anonymous_function() {
return this.replace(/^\s+/, "");
},
"rtrim" : function __cf__14__f__anonymous_function() {
return this.replace(/\s+$/, "");
},
"startsWith" : function __cf__15__f__anonymous_function(str) {
return this.indexOf(str) == 0;
},
"endsWith" : function __cf__16__f__anonymous_function(str) {
return (new RegExp(RegExp.escape(str) + "$")).test(this);
}
}
I already put a pointer to this question/answer in your other question which asks how to convert this string.

Test empty string in mongodb and pymongo

Here is my data structure.
[{
"name": "David",
"lastname": "",
},
{
"name": "Angela"
}]
"lastname" is sometimes present and sometimes not and sometime is "".
I want to get all rows that have lastname not equal to "". But this does not work. It returns both the rows when lastname is "" and when lastname is not present at all. in the example above I want to only get the David node.
db.collection.find( {"lastname": {"$ne": ""}} )
db.collection.find({"lastname" : {"$exists" : true, "$ne" : ""}})
In the mongo shell (id's omitted to save space)
> db.collection.find()
{ "name" : "Angela" }
{ "name" : "David", "lastname" : "" }
{ "name" : "Kyle", "lastname" : "Test" }
{ "name" : "John", "lastname" : null }
> db.collection.find({"lastname" : {"$exists" : true, "$ne" : ""}})
{ "name" : "Kyle", "lastname" : "Test" }
{ "name" : "John", "lastname" : null }
In case you also want to filter out matches against null values you need to adjust the criteria as follows (we can also get rid of $exists as "$ne": null takes care of this)
> db.collection.find({$and:[{"lastname": {"$ne": null}}, {"lastname": {"$ne": ""}}]})
{ "name" : "Kyle", "lastname" : "Test" }
Facing this problem I thought in another solution:
db.collection.find({"lastname": {"$gte": " "}})
With this I could get only the not empty strings, also ignoring null and not existent field, because any printable value (ASCII) has a greater value than space (32).
https://en.wikipedia.org/wiki/ASCII
You can use a regex query:
db.test.find({ "lastname": /(.|\s)*\S(.|\s)*/ })
This regex matches strings beginning or ending with 0 or N whitespaces (.|\s) and it have to be one or more non-whitespaces \S in the middle.
I'm not sure if this helps, but it has worked for me. The regex .+ returns anything that contains more than 1 character, whereas .* returns 0 or more. So it won't return anything less than 0 characters.
In terms of strings containing only whitespace, I don't think this solution can handle that.