What is the difference between these two commands?
$ cat volcanoes.txt | wc > islands.txt
results:
$ cat islands.txt
17 26 204
second command:
$ cat volcanoes.txt | wc | cat > islands.txt
results:
$ cat islands.txt
17 26 204
We can see two results are same.
I am confused about the second commands.Why it has another cat in the commands.
Thank you for the help:)
The cat command just copies its input to its output verbatim.
So even
cat volcanoes.txt | cat | cat | wc | cat | cat | cat | cat > islands.txt
would lead to the same content of islands.txt.
Even without cat you can get the same result:
wc < volcanoes.txt > islands.txt
This one being the most efficient version, starting only one process.
Related
We have the following line of text:
|  |  |  |
As you can see, the line of text simply consists of three similar phrases, which can be matched and changed (separately) using the following sed expression:
sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
If we had just one phrase (instead of the given three), the result would be the following:
$ echo '|  |' | sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
|  |
But when we have two or thee phrases, the result always points to the last matched phrase:
Here's an example with two matches:
$ echo '|  |  |' | sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
|  |
And here's an example with three matches:
$ echo '|  |  |  |' | sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
|  |
Why does this happening?
Is there a way to force sed to print the result only for the very first match?
The expected behavior? I though the following command would print something similar to this (just the first match):
$ echo '|  |  |  |' | sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
|  |
or this (all matches):
$ echo '|  |  |  |' | sed -n 's#| !\[.*\](\(\/img\/\)\([0-9]*\/[0-9]*\/[0-9]*\)\.\(.*\)\.\(.*\)) |#|  |#p'
|  |  |  |
What's happening is that | !\[.*\] matches the longest possible match. That is, the first phrase, up to the beginning of the last phrase. If you want to match only the first phrase you must be more specific. For instance with:
sed 's#| !\[\]\(([^.]*\.\([^.]*\)\.[^)]*)\) |.*#| ![\2]\1 |#'
I do not fully understand the question, but, you can try this sed
$ sed 's#\([^[]*.\)\([^\.]*.\([^\.]*\)[^)]*.\)#\1\3\2#' input_file
This will print all 3 matches but will only substitute into the first match
$ sed 's#\([^[]*.\)\([^\.]*.\([^\.]*\)[^)]*.\)#\1\3\2#' input_file
|  |  |  |
To target all 3, the g flag can be added
sed 's#\([^[]*.\)\([^\.]*.\([^\.]*\)[^)]*.\)#\1\3\2#g' input_file
|  |  |  |
You could also target just #2 for example
$ sed 's#\([^[]*.\)\([^\.]*.\([^\.]*\)[^)]*.\)#\1\3\2#2' input_file
|  |  |  |
we are using kafka in production and I try to push the adoption and usage of KSQL in the same direction. But I already failed with one simple table-table join. I’ve tried with our production data first and ran in an issue. So I thought I missed something and moved back to the example from the confluent docs and ran in the same problem.
I will explain my issue with the example data https://docs.confluent.io/current/ksql/docs/tutorials/basics-docker.html#table-table-join
When I have created both tables and try to join the data it works, but as soon as I try to alter or add something I get new entries in my table. From every example I found at confluent or even at the youtube videos this is not suppose to happen.
Creating records
docker run --interactive --rm --network tutorials_default \
confluentinc/cp-kafkacat \
kafkacat -b kafka:39092 \
-t warehouse_location \
-K: \
-P <<EOF
1:{"warehouse_id":1,"city":"Leeds","country":"UK"}
2:{"warehouse_id":2,"city":"Sheffield","country":"UK"}
3:{"warehouse_id":3,"city":"Berlin","country":"Germany"}
EOF
docker run --interactive --rm --network tutorials_default \
confluentinc/cp-kafkacat \
kafkacat -b kafka:39092 \
-t warehouse_size \
-K: \
-P <<EOF
1:{"warehouse_id":1,"square_footage":16000}
2:{"warehouse_id":2,"square_footage":42000}
3:{"warehouse_id":3,"square_footage":94000}
EOF
Creating tables
CREATE TABLE WAREHOUSE_LOCATION (WAREHOUSE_ID INT, CITY VARCHAR, COUNTRY VARCHAR)
WITH (KAFKA_TOPIC='warehouse_location',
VALUE_FORMAT='JSON',
KEY='WAREHOUSE_ID');
CREATE TABLE WAREHOUSE_SIZE (WAREHOUSE_ID INT, SQUARE_FOOTAGE DOUBLE)
WITH (KAFKA_TOPIC='warehouse_size',
VALUE_FORMAT='JSON',
KEY='WAREHOUSE_ID');
Creating a joined table:
CREATE TABLE WH_U AS SELECT WL.WAREHOUSE_ID, WL.CITY, WL.COUNTRY, WS.SQUARE_FOOTAGE
FROM WAREHOUSE_LOCATION WL
LEFT JOIN WAREHOUSE_SIZE WS
ON WL.WAREHOUSE_ID=WS.WAREHOUSE_ID;
With this I get the expected results:
1 | Leeds | UK | 16000.0
2 | Sheffield | UK | 42000.0
3 | Berlin | Germany | 94000.0
But when I add or chnage records, this happens:
1566375174496 | 1 | 1 | Leeds | UK | 16000.0
1566375174496 | 2 | 2 | Sheffield | UK | 42000.0
1566375174496 | 3 | 3 | Berlin | Germany | 94000.0
1566375595372 | 4 | 4 | London | UK | null
1566375641291 | 4 | 4 | London | UK | 94000.0
1566375641291 | 1 | 1 | Leeds | UK | 1.0
I expected:
1566375174496 | 1 | 1 | Leeds | UK | 1.0
1566375174496 | 2 | 2 | Sheffield | UK | 42000.0
1566375174496 | 3 | 3 | Berlin | Germany | 94000.0
1566375641291 | 4 | 4 | London | UK | 94000.0
What am I missing?
SOLVED
The reason for this behaviour was a simple env in ksql server. KSQL_CACHE_MAX_BYTES_BUFFERING was set to 0
Suppose below sed command:
$ seq 4 | sed 'p;n;'
1
1
2
3
3
4
I couldn't understand why 2 and 4 are printed once while
The "n" command will print out the current pattern space...
and p; prints current pattern space earlier than n;.
Let me show you my thoughts (O: output, PS: pattern space):
+------------+---------+-----------+
| Current PS | `p;` | `n;` |
+------------+---------+-----------+
| 1 | O=1 | O=1 PS=2 |
+------------+---------+-----------+
| 2 | O=2 | O=2 PS=3 |
+------------+---------+-----------+
| 3 | O=3 | O=3 PS=4 |
+------------+---------+-----------+
| 4 | O=4 | O=4 PS=4 |
+------------+---------+-----------+
What am I missing in definition of n here that I expect 2 and 4 to be output twice as well?
This is what happens:
1 is read into PS.
p : 1 is printed.
n : 1 is printed again, 2 is read into PS.
End of iteration, 2 is printed.
3 is read into PS.
p : 3 is printed.
etc.
Modify the string to see why it's being printed:
$ seq 4 | sed 'p;s/$/ n command/;n;s/$/ end/'
1
1 n command
2 end
3
3 n command
4 end
\watch command is great in Postrges.
It appends an output of a query, being watched over and over.
For example, when I previously run a query SELECT id, nickname FROM users;, and entered \watch, in case of no new data, I get the same output over and over:
my_db=# SELECT id, nickname FROM users;
id | nickname
----+----------
1 | AntonAL
(1 row)
my_db=# \watch
четверг, 31 августа 2017 г. 13:23:26 (every 2s)
id | nickname
----+----------
1 | AntonAL
(1 row)
четверг, 31 августа 2017 г. 13:23:28 (every 2s)
id | nickname
----+----------
1 | AntonAL
(1 row)
It there any option to clear screen output, between watch executions?
I want to get dashboard-like experience, when a set of rows is displayed and refreshed, not appended over and over to the Postgres console output.
I had this problem, and with a suggestion from MatheusOl on #postgresql, the following works for me:
\C `clear`
SELECT * FROM pg_tables \watch 10
The \C [value] command sequence lets you set a title, and you can set that (shelling out with backticks) to the ANSI escapes returned by clear.
I don't think psql has any way to clear the terminal. Maybe do with bash?
while true;
clear;
psql -c "SELECT id, nickname FROM users"
sleep 2;
done
Similar to the last answer but a little simpler with the "watch" command (on linux) :
watch 'psql -c "SELECT id, nickname FROM users"'
... this will execute the query with psql every 2 seconds (default interval) and you can also add different parameters to this command :
$ watch --help
Usage:
watch [options] command
Options:
-b, --beep beep if command has a non-zero exit
-c, --color interpret ANSI color and style sequences
-d, --differences[=<permanent>]
highlight changes between updates
-e, --errexit exit if command has a non-zero exit
-g, --chgexit exit when output from command changes
-n, --interval <secs> seconds to wait between updates
-p, --precise attempt run command in precise intervals
-t, --no-title turn off header
-x, --exec pass command to exec instead of "sh -c"
-h, --help display this help and exit
-v, --version output version information and exit
For more details see watch(1).
I like to use this to monitor sessions activity with pg_stat_activity :
$ cat sessions.sql
select pid as process_id,
usename as username,
datname as database_name,
client_addr as client_address,
application_name,
backend_start,
state,
state_change
from pg_stat_activity;
$ watch -d psql -f sessions.sql
... and we get this output refreshed every 2 seconds :
Every 2.0s: psql -f sessions.sql Sun Sep 29 17:38:20 2019
process_id | username | database_name | client_address | application_name | backend_start | state | state_change
------------+----------+---------------+----------------+------------------+-------------------------------+--------+-------------------------------
1240 | postgres | | | | 2019-09-29 15:40:36.697364-04 | |
1236 | | | | | 2019-09-29 15:40:36.697681-04 | |
14025 | postgres | postgres | | psql | 2019-09-29 17:38:20.508996-04 | active | 2019-09-29 17:38:20.511069-04
1233 | | | | | 2019-09-29 15:40:36.69497-04 | |
1232 | | | | | 2019-09-29 15:40:36.695851-04 | |
1234 | | | | | 2019-09-29 15:40:36.690183-04 | |
(6 rows)
I've got a huge table for R which should not be exported. So I added a #TAGS: noexport above the table, but that doesn't seem to have any effect. So how do I tag a table?
You cannot add tags to a table, only to a header.
Also, the #+TAGS directive is used to declare a group of tags specific to the whole document, not for tagging a table.
If you surround the table within a comment block, it will not be exported. For example:
#+begin_comment
#+tblname: very_long_table
| col1 | col2 |
|------+------|
| 1 | 22 |
| 2 | 23 |
| 3 | 24 |
| 4 | 25 |
| 5 | 26 |
#+end_comment
#+begin_src sh :var tbl=very_long_table :exports both
echo $tbl
#+end_src
#+RESULTS:
: 1 22 2 23 3 24 4 25 5 26