Dataframe merge creates duplicate records in pandas (0.7.3) - merge

When I merge two CSV files, of the format (date, someValue), I see some duplicate records.
If I reduce the records to half the problem goes away. However, if I double the size of both the files it worsens. Appreciate any help!
My code:
i = pd.DataFrame.from_csv('i.csv')
i = i.reset_index()
e = pd.DataFrame.from_csv('e.csv')
e = e.reset_index()
total_df = pd.merge(i, e, right_index=False, left_index=False,
right_on=['date'], left_on=['date'], how='left')
total_df = total_df.sort(column='date')
(Note: the dupulicate records for 11/15, 11/16, 12/17, 12/18.)
In [7]: total_df
Out[7]:
date Cost netCost
25 2012-11-15 00:00:00 1 2
26 2012-11-15 00:00:00 1 2
31 2012-11-16 00:00:00 1 2
32 2012-11-16 00:00:00 1 2
37 2012-11-17 00:00:00 1 2
2 2012-11-18 00:00:00 1 2
5 2012-11-19 00:00:00 1 2
8 2012-11-20 00:00:00 1 2
11 2012-11-21 00:00:00 1 2
14 2012-11-22 00:00:00 1 2
17 2012-11-23 00:00:00 1 2
20 2012-11-24 00:00:00 1 2
23 2012-11-25 00:00:00 1 2
29 2012-11-26 00:00:00 1 2
35 2012-11-27 00:00:00 1 2
0 2012-11-28 00:00:00 1 2
3 2012-11-29 00:00:00 1 2
6 2012-11-30 00:00:00 1 2
9 2012-12-01 00:00:00 1 2
12 2012-12-02 00:00:00 1 2
15 2012-12-03 00:00:00 1 2
18 2012-12-04 00:00:00 1 2
21 2012-12-05 00:00:00 1 2
24 2012-12-06 00:00:00 1 2
30 2012-12-07 00:00:00 1 2
36 2012-12-08 00:00:00 1 2
1 2012-12-09 00:00:00 2 2
4 2012-12-10 00:00:00 2 2
7 2012-12-11 00:00:00 2 2
10 2012-12-12 00:00:00 2 2
13 2012-12-13 00:00:00 1 2
16 2012-12-14 00:00:00 2 2
19 2012-12-15 00:00:00 2 2
22 2012-12-16 00:00:00 2 2
27 2012-12-17 00:00:00 1 2
28 2012-12-17 00:00:00 1 2
33 2012-12-18 00:00:00 1 2
34 2012-12-18 00:00:00 1 2
i.csv
date,Cost
2012-11-15 00:00:00,1
2012-11-16 00:00:00,1
2012-11-17 00:00:00,1
2012-11-18 00:00:00,1
2012-11-19 00:00:00,1
2012-11-20 00:00:00,1
2012-11-21 00:00:00,1
2012-11-22 00:00:00,1
2012-11-23 00:00:00,1
2012-11-24 00:00:00,1
2012-11-25 00:00:00,1
2012-11-26 00:00:00,1
2012-11-27 00:00:00,1
2012-11-28 00:00:00,1
2012-11-29 00:00:00,1
2012-11-30 00:00:00,1
2012-12-01 00:00:00,1
2012-12-02 00:00:00,1
2012-12-03 00:00:00,1
2012-12-04 00:00:00,1
2012-12-05 00:00:00,1
2012-12-06 00:00:00,1
2012-12-07 00:00:00,1
2012-12-08 00:00:00,1
2012-12-09 00:00:00,2
2012-12-10 00:00:00,2
2012-12-11 00:00:00,2
2012-12-12 00:00:00,2
2012-12-13 00:00:00,1
2012-12-14 00:00:00,2
2012-12-15 00:00:00,2
2012-12-16 00:00:00,2
2012-12-17 00:00:00,1
2012-12-18 00:00:00,1
e.csv
date,netCost
2012-11-15 00:00:00,2
2012-11-16 00:00:00,2
2012-11-17 00:00:00,2
2012-11-18 00:00:00,2
2012-11-19 00:00:00,2
2012-11-20 00:00:00,2
2012-11-21 00:00:00,2
2012-11-22 00:00:00,2
2012-11-23 00:00:00,2
2012-11-24 00:00:00,2
2012-11-25 00:00:00,2
2012-11-26 00:00:00,2
2012-11-27 00:00:00,2
2012-11-28 00:00:00,2
2012-11-29 00:00:00,2
2012-11-30 00:00:00,2
2012-12-01 00:00:00,2
2012-12-02 00:00:00,2
2012-12-03 00:00:00,2
2012-12-04 00:00:00,2
2012-12-05 00:00:00,2
2012-12-06 00:00:00,2
2012-12-07 00:00:00,2
2012-12-08 00:00:00,2
2012-12-09 00:00:00,2
2012-12-10 00:00:00,2
2012-12-11 00:00:00,2
2012-12-12 00:00:00,2
2012-12-13 00:00:00,2
2012-12-14 00:00:00,2
2012-12-15 00:00:00,2
2012-12-16 00:00:00,2
2012-12-17 00:00:00,2
2012-12-18 00:00:00,2

This does seem like a bug with pandas 0.7.3 or numpy 1.6. This only happens if the column being merged on is a date (internally converted to numpy.datetime64). My solution was to convert date into a string-
def _DatetimeToString(datetime64):
timestamp = datetime64.astype(long)/1000000000
return datetime.fromtimestamp(timestamp).strftime('%Y-%m-%d')
i = pd.DataFrame.from_csv('i.csv')
i = i.reset_index()
i['date'] = i['date'].map(_DatetimeToString)
e = pd.DataFrame.from_csv('e.csv')
e = e.reset_index()
i['date'] = i['date'].map(_DatetimeToString)
total_df = pd.merge(i, e, right_index=False, left_index=False,
right_on=['date'], left_on=['date'], how='left')
total_df = total_df.sort(column='date')

This issue/bug came up for me as well. I was not merging on a datetime series, however, I did have a datetime series in the left dataframe. My solution was to de-dupe:
len(pophist)
2347
pop_merged = pd.merge(left=pophist, right=df_labels, how='left',
left_on ='candidate', right_on ='Slug', indicator = True)
pop_merged.shape
3303
pop_merged2 = pop_merged.drop_duplicates() #note dedupping is required due to issue in how pandas handles datetime dtypes on merge.
len(pop_merged2)
2347

Related

my-sql query to fill out the time gaps between the records

I want to write an optimized query to fill out the time gaps between the records with the stock value that is most recent to date.
The requirement is to have the latest stock value for every group of id_warehouse, id_stock, and date. The table is quite large (2 million records) and hence I would like to optimize the query that I have added below and the table grows.
daily_stock_levels:
date
id_stock
id_warehouse
new_stock
is_stock_avaible
2022-01-01
1
1
24
1
2022-01-01
1
1
25
1
2022-01-01
1
1
29
1
2022-01-02
1
1
30
1
2022-01-06
1
1
27
1
2022-01-09
1
1
26
1
Result:
date
id_stock
id_warehouse
closest_date_with_stock_value
most_recent_stock_value
2022-01-01
1
1
29
1
2022-01-02
1
1
30
1
2022-01-03
1
1
30
1
2022-01-04
1
1
30
1
2022-01-05
1
1
30
1
2022-01-06
1
1
27
1
2022-01-07
1
1
27
1
2022-01-07
1
1
27
1
2022-01-09
1
1
26
1
2022-01-10
1
1
26
1
..
..
..
..
..
..
..
..
..
..
..
..
..
..
..
..
..
..
..
..
2022-08-08
1
1
26
1
SELECT
sl.date,
sl.id_warehouse,
sl.id_item,
(SELECT
s.date
FROM
daily_stock_levels s
WHERE s.is_stock_available = 1
AND sl.id_warehouse = s.id_warehouse
AND sl.id_item = s.id_item
AND sl.date >= s.date
ORDER BY s.date DESC
LIMIT 1) AS closest_date_with_stock_value,
(SELECT
s.new_stock
FROM
daily_stock_levels s
WHERE s.is_stock_available = 1
AND sl.id_warehouse = s.id_warehouse
AND sl.id_item = s.id_item
AND sl.date >= s.date
ORDER BY s.date DESC
LIMIT 1) AS most_recent_stock_value
FROM
daily_stock_levels sl
GROUP BY sl.id_warehouse,
sl.id_item,
sl.date

Is there a way to do a selective sum using a time interval in Postgres?

I have two tables, the first table has columns: id, start_time, and end_time. The second table has columns: id, timestamp, value. Is there a way to make a sum of table 2 based on the conditions in table 1?
Table 1:
id
start_date
end_date
5
2000-01-01 01:00:00
2000-01-05 02:45:00
5
2000-01-10 01:00:00
2000-01-15 02:45:00
6
2000-01-01 01:00:00
2000-01-05 02:45:00
6
2000-01-11 01:00:00
2000-01-12 02:45:00
6
2000-01-15 01:00:00
2000-01-20 02:45:00
Table 2:
id
timestamp
value
5
2000-01-01 05:00:00
1
5
2000-01-01 06:00:00
2
6
2000-01-01 05:00:00
1
6
2000-01-11 05:00:00
2
6
2000-01-15 05:00:00
2
6
2000-01-15 05:30:00
2
Desired result:
id
start_date
end_date
Sum
5
2000-01-01 01:00:00
2000-01-05 02:45:00
3
5
2000-01-10 01:00:00
2000-01-15 02:45:00
null
6
2000-01-01 01:00:00
2000-01-05 02:45:00
1
6
2000-01-11 01:00:00
2000-01-12 02:45:00
2
6
2000-01-15 01:00:00
2000-01-20 02:45:00
4
Try this :
SELECT a.id, a.start_date, a.end_date, sum(b.value) AS sum
FROM table1 AS a
LEFT JOIN table2 AS b
ON b.id = a.id
AND b.timestamp >= a.start_date
AND b.timestamp < a.end_date
GROUP BY a.id, a.start_date, a.end_date

Returns the value in the next empty rows

Input
Here is an example of my input.
Number
Date
Motore
1
Fri Jan 01 00:00:00 CET 2021
Motore 1
2
Motore 2
3
Motore 3
4
Motore 4
5
Fri Feb 01 00:00:00 CET 2021
Motore 1
6
Motore 2
7
Motore 3
8
Motore 4
Expected Output
Number
Date
Motore
1
Fri Jan 01 00:00:00 CET 2021
Motore 1
2
Fri Jan 01 00:00:00 CET 2021
Motore 2
3
Fri Jan 01 00:00:00 CET 2021
Motore 3
4
Fri Jan 01 00:00:00 CET 2021
Motore 4
5
Fri Feb 01 00:00:00 CET 2021
Motore 1
6
Fri Feb 01 00:00:00 CET 2021
Motore 2
7
Fri Feb 01 00:00:00 CET 2021
Motore 3
8
Fri Feb 01 00:00:00 CET 2021
Motore 4
I tried to use the TmemorizeRows component but without any result, the second line is valorized but the others are not. Could you kindly help me.
You can solve this with a simple tMap with 2 inner variables (using the "var" array in the middle of the tMap)
Create two variables :
currentValue : you put in it the value of your input column date (in my example "row1.data").
updateValue : check whether currentValue is null or not : if null then you do not modify updateValue field . If not null then you update the field value. This way "updateValue" always contains not null data.
In output, just use "updateValue" variable.

I am trying to set up mariadb on a raspberry pi and keep getting an error message

The error that I keep receiving is:
Errors were encountered while processing:
/tmp/apt-dpkg-install-XXe4L1/1-mariadb-server-10.0_10.0.28-2+b1_armhf.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
When I run the command
sudo apt-get install mariadb-server-10.0
I used to be able to log into mariadb, and now it won't let me log in and even enter mysql safe mode, so that I can change my root#localhost password.
I ran
sudo cat /etc/mysql/debian.cnf
and received the output
# Automatically generated for Debian scripts. DO NOT TOUCH!
[client]
host = localhost
user = root
password =
socket = /var/run/mysqld/mysqld.sock
[mysql_upgrade]
host = localhost
user = root
password =
socket = /var/run/mysqld/mysqld.sock
basedir = /usr
So I went into the file and added a password for both positions and I am still having difficulties executing the command to allow me into mariadb
I then ran ps -ef Like the first suggestion bellow and am not sure what process needs to be halted. Here is the lengthy list. Can someone please guide me to what process I need to kill.
dan#natcoCo:/ $ ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 Dec29 ? 00:00:42 /sbin/init splash
root 2 0 0 Dec29 ? 00:00:00 [kthreadd]
root 3 2 0 Dec29 ? 00:00:00 [rcu_gp]
root 4 2 0 Dec29 ? 00:00:00 [rcu_par_gp]
root 8 2 0 Dec29 ? 00:00:00 [mm_percpu_wq]
root 9 2 0 Dec29 ? 00:00:05 [ksoftirqd/0]
root 10 2 0 Dec29 ? 00:01:10 [rcu_sched]
root 11 2 0 Dec29 ? 00:00:00 [rcu_bh]
root 12 2 0 Dec29 ? 00:00:00 [migration/0]
root 13 2 0 Dec29 ? 00:00:00 [cpuhp/0]
root 14 2 0 Dec29 ? 00:00:00 [cpuhp/1]
root 15 2 0 Dec29 ? 00:00:00 [migration/1]
root 16 2 0 Dec29 ? 00:00:00 [ksoftirqd/1]
root 19 2 0 Dec29 ? 00:00:00 [cpuhp/2]
root 20 2 0 Dec29 ? 00:00:00 [migration/2]
root 21 2 0 Dec29 ? 00:00:00 [ksoftirqd/2]
root 24 2 0 Dec29 ? 00:00:00 [cpuhp/3]
root 25 2 0 Dec29 ? 00:00:00 [migration/3]
root 26 2 0 Dec29 ? 00:00:00 [ksoftirqd/3]
root 29 2 0 Dec29 ? 00:00:00 [kdevtmpfs]
root 30 2 0 Dec29 ? 00:00:00 [netns]
root 34 2 0 Dec29 ? 00:00:00 [khungtaskd]
root 35 2 0 Dec29 ? 00:00:00 [oom_reaper]
root 36 2 0 Dec29 ? 00:00:00 [writeback]
root 37 2 0 Dec29 ? 00:00:00 [kcompactd0]
root 38 2 0 Dec29 ? 00:00:00 [crypto]
root 39 2 0 Dec29 ? 00:00:00 [kblockd]
root 41 2 0 Dec29 ? 00:00:00 [watchdogd]
root 42 2 0 Dec29 ? 00:00:00 [rpciod]
root 43 2 0 Dec29 ? 00:00:00 [kworker/u9:0-hci0]
root 44 2 0 Dec29 ? 00:00:00 [xprtiod]
root 47 2 0 Dec29 ? 00:00:04 [kswapd0]
root 48 2 0 Dec29 ? 00:00:00 [nfsiod]
root 59 2 0 Dec29 ? 00:00:00 [kthrotld]
root 60 2 0 Dec29 ? 00:00:00 [iscsi_eh]
root 62 2 0 Dec29 ? 00:00:00 [dwc_otg]
root 64 2 0 Dec29 ? 00:00:00 [DWC Notificatio]
root 65 2 0 Dec29 ? 00:00:05 [vchiq-slot/0]
root 66 2 0 Dec29 ? 00:00:02 [vchiq-recy/0]
root 67 2 0 Dec29 ? 00:00:00 [vchiq-sync/0]
root 68 2 0 Dec29 ? 00:00:00 [vchiq-keep/0]
root 69 2 0 Dec29 ? 00:00:00 [SMIO]
root 71 2 0 Dec29 ? 00:00:00 [irq/38-brcmstb_]
root 72 2 0 Dec29 ? 00:00:17 [irq/39-mmc1]
root 74 2 0 Dec29 ? 00:00:00 [irq/39-mmc0]
root 75 2 0 Dec29 ? 00:00:00 [mmc_complete]
root 78 2 0 Dec29 ? 00:00:12 [jbd2/mmcblk0p7-]
root 79 2 0 Dec29 ? 00:00:00 [ext4-rsv-conver]
root 81 2 0 Dec29 ? 00:00:00 [ipv6_addrconf]
root 111 1 0 Dec29 ? 00:00:06 /lib/systemd/systemd-journald
root 143 1 0 Dec29 ? 00:00:01 /lib/systemd/systemd-udevd
root 209 2 0 Dec29 ? 00:00:00 [cfg80211]
root 214 2 0 Dec29 ? 00:00:00 [brcmf_wq/mmc1:0]
root 217 2 0 Dec29 ? 00:00:06 [brcmf_wdog/mmc1]
root 221 2 0 Dec29 ? 00:00:00 [SMIO]
root 232 2 0 Dec29 ? 00:00:00 [mmal-vchiq]
root 234 2 0 Dec29 ? 00:00:42 [v3d_bin]
root 235 2 0 Dec29 ? 00:00:00 [mmal-vchiq]
root 236 2 0 Dec29 ? 00:00:42 [v3d_render]
root 237 2 0 Dec29 ? 00:00:00 [v3d_tfu]
root 238 2 0 Dec29 ? 00:00:00 [mmal-vchiq]
root 239 2 0 Dec29 ? 00:00:00 [v3d_csd]
root 241 2 0 Dec29 ? 00:00:00 [v3d_cache_clean]
message+ 327 1 0 Dec29 ? 00:00:05 /usr/bin/dbus-daemon --system --
avahi 334 1 0 Dec29 ? 00:01:45 avahi-daemon: running [natcoCo.l
root 344 1 0 Dec29 ? 00:00:00 /usr/lib/udisks2/udisksd
nobody 347 1 0 Dec29 ? 00:00:51 /usr/sbin/thd --triggers /etc/tr
avahi 354 334 0 Dec29 ? 00:00:00 avahi-daemon: chroot helper
root 356 1 0 Dec29 ? 00:00:00 /usr/sbin/alsactl -E HOME=/run/a
dan 364 5719 0 Dec30 ? 00:00:18 /usr/lib/chromium-browser/chromi
root 367 1 0 Dec29 ? 00:00:00 /usr/sbin/cron -f
root 374 1 0 Dec29 ? 00:00:01 /sbin/wpa_supplicant -u -s -O /r
root 378 1 0 Dec29 ? 00:00:03 /lib/systemd/systemd-logind
root 385 1 0 Dec29 ? 00:00:00 /sbin/dhcpcd -q -b
root 396 1 0 Dec29 ? 00:00:00 /usr/sbin/rsyslogd -n -iNONE
root 402 1 0 Dec29 ? 00:00:15 /usr/sbin/rngd -r /dev/hwrng
ntp 480 1 0 Dec29 ? 00:00:24 /usr/sbin/ntpd -p /var/run/ntpd.
root 482 1 0 Dec29 ? 00:00:00 /usr/lib/policykit-1/polkitd --n
root 485 1 0 Dec29 ? 00:00:00 /usr/bin/python3 /usr/share/unat
root 499 1 0 Dec29 ? 00:00:00 /usr/sbin/lightdm
root 523 1 0 Dec29 ? 00:00:02 wpa_supplicant -B -c/etc/wpa_sup
Debian-+ 918 1 0 Dec29 ? 00:00:00 /usr/sbin/exim4 -bd -q30m
root 965 1 0 Dec29 ? 00:00:00 /usr/bin/hciattach /dev/serial1
root 966 2 0 Dec29 ? 00:00:00 [kworker/u9:2-hci0]
root 971 1 0 Dec29 ? 00:00:00 /usr/lib/bluetooth/bluetoothd
root 973 1 0 Dec29 ? 00:00:00 /usr/bin/bluealsa
root 986 2 0 Dec29 ? 00:00:00 [krfcommd]
dan 1003 5719 0 Dec30 ? 00:00:06 /usr/lib/chromium-browser/chromi
dan 1079 5719 0 Dec30 ? 00:00:21 /usr/lib/chromium-browser/chromi
ken 1147 1 0 Dec29 ? 00:00:00 /lib/systemd/systemd --user
ken 1150 1147 0 Dec29 ? 00:00:00 (sd-pam)
ken 1170 1147 0 Dec29 ? 00:00:00 /usr/bin/dbus-daemon --session -
ken 1204 1147 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfsd
ken 1209 1147 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfsd-fuse /run/us
ken 1231 1 0 Dec29 ? 00:00:00 /usr/bin/ssh-agent -s
ken 1277 1147 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfs-udisks2-volum
ken 1281 1147 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfs-mtp-volume-mo
ken 1285 1147 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfs-afc-volume-mo
ken 1290 1147 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfs-gphoto2-volum
ken 1294 1147 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfs-goa-volume-mo
ken 1365 1204 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfsd-trash --spaw
root 1372 2 0 Dec29 ? 00:00:00 [jbd2/mmcblk0p8-]
root 1373 2 0 Dec29 ? 00:00:00 [ext4-rsv-conver]
ken 1378 1147 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfsd-metadata
dan 1542 5719 0 Dec30 ? 00:00:17 /usr/lib/chromium-browser/chromi
root 3390 1 0 Dec29 ? 00:00:12 /usr/sbin/nmbd --foreground --no
root 4172 1 0 Dec29 ? 00:00:02 /usr/sbin/smbd --foreground --no
root 4174 4172 0 Dec29 ? 00:00:00 /usr/sbin/smbd --foreground --no
root 4175 4172 0 Dec29 ? 00:00:00 /usr/sbin/smbd --foreground --no
root 4176 4172 0 Dec29 ? 00:00:01 /usr/sbin/smbd --foreground --no
dan 4866 5719 0 Dec30 ? 00:00:36 /usr/lib/chromium-browser/chromi
root 5266 499 0 Dec29 tty7 00:18:54 /usr/lib/xorg/Xorg :0 -seat seat
root 5315 499 0 Dec29 ? 00:00:00 lightdm --session-child 14 23
dan 5326 1 0 Dec29 ? 00:00:00 /lib/systemd/systemd --user
dan 5329 5326 0 Dec29 ? 00:00:00 (sd-pam)
dan 5340 5315 0 Dec29 ? 00:00:04 /usr/bin/lxsession -s LXDE-pi -e
dan 5349 5326 0 Dec29 ? 00:00:00 /usr/bin/dbus-daemon --session -
dan 5374 5719 0 Dec30 ? 00:00:10 /usr/lib/chromium-browser/chromi
dan 5446 5719 1 Dec30 ? 00:13:28 /usr/lib/chromium-browser/chromi
dan 5508 5340 0 Dec29 ? 00:00:00 /usr/bin/ssh-agent x-session-man
dan 5518 5326 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfsd
dan 5523 5326 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfsd-fuse /run/us
dan 5534 5340 0 Dec29 ? 00:00:20 openbox --config-file /home/dan/
dan 5536 5340 0 Dec29 ? 00:00:01 lxpolkit
dan 5538 5340 0 Dec29 ? 00:01:39 lxpanel --profile LXDE-pi
dan 5540 5340 0 Dec29 ? 00:00:26 pcmanfm --desktop --profile LXDE
dan 5548 1 0 Dec29 ? 00:00:00 /bin/sh /usr/lib/raspi-config/cm
dan 5552 1 0 Dec29 ? 00:00:00 /usr/bin/ssh-agent -s
dan 5555 5548 0 Dec29 ? 00:00:00 xcompmgr -aR
dan 5586 1 0 Dec29 ? 00:00:00 /usr/lib/menu-cache/menu-cached
dan 5591 5326 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfs-udisks2-volum
dan 5597 5326 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfs-mtp-volume-mo
dan 5601 5326 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfs-afc-volume-mo
dan 5606 5326 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfs-gphoto2-volum
dan 5610 5326 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfs-goa-volume-mo
dan 5682 5518 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfsd-trash --spaw
dan 5696 5538 1 Dec29 ? 00:52:59 /usr/lib/chromium-browser/chromi
dan 5717 5696 0 Dec29 ? 00:00:00 /usr/lib/chromium-browser/chromi
dan 5719 5717 0 Dec29 ? 00:00:03 /usr/lib/chromium-browser/chromi
dan 5745 5696 1 Dec29 ? 00:38:15 /usr/lib/chromium-browser/chromi
dan 5749 5696 0 Dec29 ? 00:10:22 /usr/lib/chromium-browser/chromi
dan 5897 5719 0 Dec30 ? 00:00:23 /usr/lib/chromium-browser/chromi
dan 5909 5719 0 Dec29 ? 00:02:27 /usr/lib/chromium-browser/chromi
dan 5941 5719 0 Dec29 ? 00:00:03 /usr/lib/chromium-browser/chromi
dan 6017 5745 0 Dec29 ? 00:00:00 /usr/lib/chromium-browser/chromi
dan 6075 5719 0 Dec29 ? 00:11:19 /usr/lib/chromium-browser/chromi
dan 6092 5719 0 Dec29 ? 00:15:44 /usr/lib/chromium-browser/chromi
dan 6308 5719 0 Dec30 ? 00:00:42 /usr/lib/chromium-browser/chromi
dan 6792 5719 0 Dec30 ? 00:00:32 /usr/lib/chromium-browser/chromi
dan 7097 5719 0 Dec30 ? 00:00:04 /usr/lib/chromium-browser/chromi
root 7208 2 0 Dec30 ? 00:00:01 [kworker/2:2H-kblockd]
dan 7326 5719 0 Dec30 ? 00:00:09 /usr/lib/chromium-browser/chromi
dan 7471 5719 0 Dec30 ? 00:00:22 /usr/lib/chromium-browser/chromi
dan 8941 5326 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfsd-metadata
dan 8969 5540 0 Dec29 ? 00:00:03 gpicview /home/dan/2019-12-29-14
dan 8982 5540 0 Dec29 ? 00:00:04 gpicview /home/dan/2019-12-29-14
root 9491 2 0 Dec29 ? 00:00:06 [kworker/3:1H-kblockd]
root 9881 1 0 Dec29 ? 00:00:00 sudo mysqld_safe --skip-grant-ta
root 9886 9881 0 Dec29 ? 00:00:00 /bin/sh /usr/bin/mysqld_safe --s
mysql 10003 9886 0 Dec29 ? 00:03:04 /usr/sbin/mysqld --basedir=/usr
root 10004 9886 0 Dec29 ? 00:00:00 logger -t mysqld -p daemon error
dan 12831 5719 0 08:16 ? 00:00:09 /usr/lib/chromium-browser/chromi
dan 13405 5719 0 Dec29 ? 00:01:30 /usr/lib/chromium-browser/chromi
dan 13419 5719 0 Dec29 ? 00:01:32 /usr/lib/chromium-browser/chromi
dan 13721 5719 0 Dec29 ? 00:02:43 /usr/lib/chromium-browser/chromi
root 13797 32525 0 08:20 pts/2 00:00:00 sudo mysql_secure_installation
root 13802 13797 0 08:20 pts/2 00:00:00 /bin/sh /usr/bin/mysql_secure_in
dan 13904 5719 0 08:25 ? 00:01:24 /usr/lib/chromium-browser/chromi
dan 13933 26204 0 08:26 pts/3 00:00:00 bash
root 14028 2 0 08:32 ? 00:00:00 [kworker/1:1-cgroup_destroy]
dan 14209 5518 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfsd-network --sp
dan 14220 5518 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfsd-smb-browse -
dan 14222 5326 0 Dec29 ? 00:00:00 /usr/lib/dconf/dconf-service
dan 14236 5518 0 Dec29 ? 00:00:00 /usr/lib/gvfs/gvfsd-dnssd --spaw
root 14881 2 0 08:56 ? 00:00:02 [kworker/0:1H-mmc_complete]
dan 16533 5719 7 09:25 ? 00:08:01 /usr/lib/chromium-browser/chromi
dan 17777 5719 3 10:22 ? 00:01:49 /usr/lib/chromium-browser/chromi
dan 17810 5719 0 10:22 ? 00:00:30 /usr/lib/chromium-browser/chromi
root 17821 2 0 10:22 ? 00:00:00 [kworker/1:1H-kblockd]
dan 17822 5719 0 10:22 ? 00:00:12 /usr/lib/chromium-browser/chromi
dan 18115 5719 0 10:28 ? 00:00:21 /usr/lib/chromium-browser/chromi
dan 18130 5719 0 10:28 ? 00:00:00 /usr/lib/chromium-browser/chromi
root 18146 2 0 10:29 ? 00:00:00 [kworker/2:2-events]
root 18190 1 0 10:30 ? 00:00:00 /usr/sbin/apache2 -k start
www-data 18191 18190 0 10:30 ? 00:00:00 /usr/sbin/apache2 -k start
www-data 18192 18190 0 10:30 ? 00:00:00 /usr/sbin/apache2 -k start
www-data 18193 18190 0 10:30 ? 00:00:00 /usr/sbin/apache2 -k start
www-data 18194 18190 0 10:30 ? 00:00:00 /usr/sbin/apache2 -k start
www-data 18195 18190 0 10:30 ? 00:00:00 /usr/sbin/apache2 -k start
root 18383 1 0 10:33 ? 00:00:00 nginx: master process /usr/sbin/
www-data 18385 18383 0 10:33 ? 00:00:00 nginx: worker process
www-data 18386 18383 0 10:33 ? 00:00:00 nginx: worker process
www-data 18387 18383 0 10:33 ? 00:00:00 nginx: worker process
www-data 18388 18383 0 10:33 ? 00:00:00 nginx: worker process
root 18460 2 0 10:38 ? 00:00:00 [kworker/u8:2-events_unbound]
root 19112 2 0 10:41 ? 00:00:00 [kworker/3:2H]
root 19205 2 0 10:53 ? 00:00:00 [kworker/1:0-mm_percpu_wq]
root 19238 2 0 10:58 ? 00:00:00 [kworker/u8:1-events_unbound]
root 19243 2 0 10:58 ? 00:00:00 [kworker/0:0-events_power_effici
root 19249 2 0 10:59 ? 00:00:00 [kworker/2:1H]
root 19283 2 0 11:05 ? 00:00:00 [kworker/1:0H]
root 19292 2 0 11:05 ? 00:00:00 [kworker/3:2-mm_percpu_wq]
root 19300 2 0 11:07 ? 00:00:00 [kworker/0:0H-kblockd]
root 19303 2 0 11:08 ? 00:00:00 [kworker/0:1-events]
root 19371 2 0 11:09 ? 00:00:00 [kworker/2:0-cgroup_destroy]
root 19377 2 0 11:10 ? 00:00:00 [kworker/3:1-events_freezable]
root 19387 2 0 11:12 ? 00:00:00 [kworker/0:2H]
root 19396 2 0 11:13 ? 00:00:00 [kworker/0:2-mm_percpu_wq]
root 19411 2 0 11:14 ? 00:00:00 [kworker/2:0H]
dan 19417 13933 0 11:16 pts/3 00:00:00 ps -ef
dan 19563 5719 0 Dec30 ? 00:00:10 /usr/lib/chromium-browser/chromi
dan 20209 5719 0 Dec30 ? 00:03:28 /usr/lib/chromium-browser/chromi
dan 20260 5719 0 Dec30 ? 00:00:13 /usr/lib/chromium-browser/chromi
dan 21898 5719 0 Dec30 ? 00:01:42 /usr/lib/chromium-browser/chromi
dan 21937 5719 0 Dec30 ? 00:00:27 /usr/lib/chromium-browser/chromi
dan 21948 5719 0 Dec30 ? 00:00:01 /usr/lib/chromium-browser/chromi
dan 21962 5719 0 Dec30 ? 00:01:52 /usr/lib/chromium-browser/chromi
dan 23333 5719 1 Dec30 ? 00:16:12 /usr/lib/chromium-browser/chromi
dan 23378 5719 0 Dec30 ? 00:01:14 /usr/lib/chromium-browser/chromi
dan 23499 5719 0 Dec30 ? 00:00:29 /usr/lib/chromium-browser/chromi
dan 24888 5719 0 Dec30 ? 00:00:20 /usr/lib/chromium-browser/chromi
dan 26204 5538 0 Dec30 ? 00:01:20 lxterminal
dan 26210 26204 0 Dec30 pts/1 00:00:00 bash
dan 26741 5719 0 Dec30 ? 00:00:33 /usr/lib/chromium-browser/chromi
root 28781 1 0 Dec30 ? 00:00:00 dbus-launch --autolaunch 9092ccb
root 28782 1 0 Dec30 ? 00:00:00 /usr/bin/dbus-daemon --syslog-on
dan 32525 26204 0 Dec30 pts/2 00:00:00 bash
dan 32596 5719 0 Dec30 ? 00:00:14 /usr/lib/chromium-browser/chromi
The file you showed (/etc/mysql/debian.cnf) looks exactly the same on my system, and I have a working instrallation with a root password. It appears to be normal that the password line is empty.
Important note: only the root user is able to login to the mariadb root account.
To login, try $ sudo mysql -uroot -p
As mentioned before, try stopping the mysql process (in your list nr. 10003).
I tried to write this as a comment, but my reputation wasn't high enough.

How to get rows between time intervals

I have delivery slots that has a from column (datetime).
Delivery slots are stored as 1 hour to 1 hour and 30 minute intervals, daily.
i.e. 3.00am-4.30am, 6.00am-7.30am, 9.00am-10.30am and so forth
id | from
------+---------------------
1 | 2016-01-01 03:00:00
2 | 2016-01-01 04:30:00
3 | 2016-01-01 06:00:00
4 | 2016-01-01 07:30:00
5 | 2016-01-01 09:00:00
6 | 2016-01-01 10:30:00
7 | 2016-01-01 12:00:00
8 | 2016-01-02 03:00:00
9 | 2016-01-02 04:30:00
10 | 2016-01-02 06:00:00
11 | 2016-01-02 07:30:00
12 | 2016-01-02 09:00:00
13 | 2016-01-02 10:30:00
14 | 2016-01-02 12:00:00
I’m trying to get all delivery_slots between the hours of 3.00am - 4.30 am. Ive got the following so far:
SELECT * FROM delivery_slots WHERE EXTRACT(HOUR FROM delivery_slots.from) >= 3 AND EXTRACT(MINUTE FROM delivery_slots.from) >= 0 AND EXTRACT(HOUR FROM delivery_slots.from) <= 4 AND EXTRACT(MINUTE FROM delivery_slots.from) <= 30;
Which kinda works. Kinda, because it is only returning delivery slots that have minutes of 00.
Thats because of the last where condition (EXTRACT(MINUTE FROM delivery_slots.from) <= 30)
To give you an idea, of what I am trying to expect:
id | from
-------+---------------------
1 | 2016-01-01 03:00:00
2 | 2016-01-01 04:30:00
8 | 2016-01-02 03:00:00
9 | 2016-01-02 04:30:00
15 | 2016-01-03 03:00:00
16 | 2016-01-03 04:30:00
etc...
Is there a better way to go about this?
Try this: (not tested)
SELECT * FROM delivery_slots WHERE delivery_slots.from::time >= '03:00:00' AND delivery_slots.from::time <= '04:30:00'
Hope this helps.
Cheers.
The easiest way to do this, in my mind, is to cast the from column as a type time and do a where >= and <=, like so
select * from testing where (date::time >= '3:00'::time and date::time <= '4:30'::time);