Related
I am having a problem with Google's SMTP servers. They reject my SMTP server's IP for no valid reason. My server complies with all the necessary rules to deliver the message but Google rejects it by IP without giving me details and I cannot find support either. I have written to postmaster#gmail.com but got no response. My server sends notifications to the users of the system, it does not do SPAM and apparently everything is correct. I don't see a valid reason for this crash, and I can't find a way to fix it. I would like to know if the same thing happened to someone and how they could solve it. Next I copy the console with the delivery attempt and additionally some DNS checks that show that apparently there is no problem on my server:
root#venabili:~# hostname -f
venabili.tecnologica.com.ar
root#venabili:~# id
uid=0(root) gid=0(root) grupos=0(root)
root#venabili:~# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 200.69.236.179 netmask 255.255.255.0 broadcast 200.69.236.255
inet6 fe80::f816:3eff:fe27:b8e1 prefixlen 64 scopeid 0x20<link>
ether fa:16:3e:27:b8:e1 txqueuelen 1000 (Ethernet)
RX packets 302342269 bytes 32703331063 (30.4 GiB)
RX errors 0 dropped 17 overruns 0 frame 0
TX packets 75025298 bytes 12670842456 (11.8 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
root#venabili:~# nslookup
> server 8.8.8.8
Default server: 8.8.8.8
Address: 8.8.8.8#53
> set type=mx
> tecnologica.com.ar
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
tecnologica.com.ar mail exchanger = 10 venabili.tecnologica.com.ar.
Authoritative answers can be found from:
> set type=a
> venabili.tecnologica.com.ar
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: venabili.tecnologica.com.ar
Address: 200.69.236.179
> set type=aaaa
> venabili.tecnologica.com.ar
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: venabili.tecnologica.com.ar
Address: fe80::f816:3eff:fe27:b8e1
> set type=ptr
> 200.69.236.179
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
179.236.69.200.in-addr.arpa name = venabili.tecnologica.com.ar.
Authoritative answers can be found from:
> set type=txt
> tecnologica.com.ar
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
tecnologica.com.ar text = "v=spf1 a mx ip4:200.69.236.179 ip6:fe80::f816:3eff:fe27:b8e1 ~all"
Authoritative answers can be found from:
> set type=txt
> default._domainkey.tecnologica.com.ar
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
default._domainkey.tecnologica.com.ar text = "v=DKIM1; t=s; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAz+GrX8vxp9W51ehJuixhL5AbjmCgcN2h7KqMiLI8LMUfmpWPP1GIhxlWCieFUVMOAQGlQrImuFE3kk/qLOgyumzUTRBwxlNX+7tix7dlBclXAWq8SjB9SbbAcPKkTBAq0pvXvp4l4qTCFnfVXAs1g/lCywlJrbfAFVVXWdN44ElFz+bD4YRYsXSmz//L1uFU7YE" "zkFUvbMtwBOL1xRvjAFXH4xQ7/vkHX6+OIxnm47vO/a2CqFVXok0FhAj44BmlBT+Py0x0SP8jsm+xhnLc238ZIsGylTwCb0Zbl3DR9bKGBy9FqXoUyRIzWKEkAtwaKq7qeBO3oRT4kQOKEOog2QIDAQAB"
Authoritative answers can be found from:
> _dmarc.tecnologica.com.ar
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
_dmarc.tecnologica.com.ar text = "v=DMARC1; p=reject; rua=mailto:postmaster#tecnologica.com.ar; ruf=mailto:postmaster#tecnologica.com.ar"
Authoritative answers can be found from:
> _smtp._tls.tecnologica.com.ar
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
_smtp._tls.tecnologica.com.ar text = "v=TLSRPTv1; rua=mailto:postmaster#tecnologica.com.ar"
Authoritative answers can be found from:
> _mta-sts.tecnologica.com.ar
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
_mta-sts.tecnologica.com.ar text = "v=STSv1; id=20200918192500"
Authoritative answers can be found from:
> set type=cname
> mta-sts.tecnologica.com.ar
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
mta-sts.tecnologica.com.ar canonical name = venabili.tecnologica.com.ar.
Authoritative answers can be found from:
> exit
root#venabili:~# openssl s_client -connect mta-sts.tecnologica.com.ar:443
CONNECTED(00000004)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = mta-sts.tecnologica.com.ar
verify return:1
---
Certificate chain
0 s:CN = mta-sts.tecnologica.com.ar
i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
1 s:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
i:O = Digital Signature Trust Co., CN = DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFbTCCBFWgAwIBAgISBC7Vr1sblDEJHYgxBtfro6xcMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0yMDA5MTgyMzM4MjdaFw0y
MDEyMTcyMzM4MjdaMCUxIzAhBgNVBAMTGm10YS1zdHMudGVjbm9sb2dpY2EuY29t
LmFyMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA5glA7vEmuS0EzKhn
w9IfLPHHertYYOYx54t/2zr8G1pSmkCjrOlfV+YHwIuWeehxDUz/ezy1JXyK11kd
raDd71MoBd9mV12fjITO99oDpkYu9g09TxJ9rpIEkU8LRGkgjfDGBBtf5X5Za+Ah
gWbJ4JudVoykVZ00riQsZzbfJ8FNZcwEgErWCZCBL45tR7tfeFlXTJGFhyszqQN+
r4gS3C/JpsObIMaoqmkJrO5YUGcfGvqDOG3INthp2L9/t743OfYUGTqRFu09Rq5S
xwEYxR8GksaLwzMWOIRelMP4W5F9ce7hhdxU68cK4j9rHXaaeNXLySeMs7K9C2vt
p1nnowIDAQABo4ICcDCCAmwwDgYDVR0PAQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsG
AQUFBwMBBggrBgEFBQcDAjAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBQ1A7mXXvtb
eS6ZYRtw4EA7WhJ3/zAfBgNVHSMEGDAWgBSoSmpjBH3duubRObemRWXv86jsoTBv
BggrBgEFBQcBAQRjMGEwLgYIKwYBBQUHMAGGImh0dHA6Ly9vY3NwLmludC14My5s
ZXRzZW5jcnlwdC5vcmcwLwYIKwYBBQUHMAKGI2h0dHA6Ly9jZXJ0LmludC14My5s
ZXRzZW5jcnlwdC5vcmcvMCUGA1UdEQQeMByCGm10YS1zdHMudGVjbm9sb2dpY2Eu
Y29tLmFyMEwGA1UdIARFMEMwCAYGZ4EMAQIBMDcGCysGAQQBgt8TAQEBMCgwJgYI
KwYBBQUHAgEWGmh0dHA6Ly9jcHMubGV0c2VuY3J5cHQub3JnMIIBBQYKKwYBBAHW
eQIEAgSB9gSB8wDxAHcA5xLysDd+GmL7jskMYYTx6ns3y1YdESZb8+DzS/JBVG4A
AAF0o8xFcQAABAMASDBGAiEAlc3Q8FEkd5ZUayxVfeorQv9+Jt1OSgOOP+jqVw5K
sskCIQCE0Hx7YDWcJ9o4Cck50WP7UtA/qRgmfn1fVIHyEZMftwB2AAe3XBvlfWj/
8bDGHSMVx7rmV3xXlLdq7rxhOhpp06IcAAABdKPMRZcAAAQDAEcwRQIgBVUd9xxC
DVX/kr0+308ra9YOqJC/yvBJnktxWg6YUVMCIQCR14/ouxc+Ydm4tePVBjBnmpRq
QzQEDYrd+4R7JjpiCjANBgkqhkiG9w0BAQsFAAOCAQEAlCnRwvP7wNRHTzB5N9/N
2cBPuVxKbvf1rHwusXYzQBVlyM6grEvjTgd5V/C+we1ueDUp991tP3z5AAksIGWM
/ssDlK0VSxv9PIzC1LxDK39Sl+P8xkuGj0zSDYg321sFjFyZLashP+xFYyzx3QdD
9h65J0uB/BigWOIN9sK1vLVhIB4yqV190vucyMamJBZ1dvSaYCi52dCgEEsAxM3v
LksETYgpSqB4+6zB2ae3/FN2/m3vyWCdFv1fL19hIidKjwIYhIKVOLKTQCYvCd+p
srhlTtnRf8i+P8Dni9xKmKSFwSGc8L+AS0AfTh6M/+IEPOO2zJsp1R/PBWgiy42v
fw==
-----END CERTIFICATE-----
subject=CN = mta-sts.tecnologica.com.ar
issuer=C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3203 bytes and written 403 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-CHACHA20-POLY1305
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-CHACHA20-POLY1305
Session-ID: 38FDD25BCB4C28F5364CA3418C05D13F279E62882E594190276850599BB67EAB
Session-ID-ctx:
Master-Key: 41AEF29EC1545AAE8C53958032EFF464E237232D8AA9D22CF9513297DFFBA40645226685FE58FEC782DA20CDFFEB5EDB
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - 5f 78 fa 6f 3a 54 9c c2-f0 8e b2 b7 70 13 05 d4 _x.o:T......p...
0010 - 4b 39 84 62 c7 cb 96 f8-89 5c 27 09 6c 2b aa ad K9.b.....\'.l+..
0020 - 52 22 36 d6 a3 0b 8b 5e-ed 4b 78 f5 49 61 47 69 R"6....^.Kx.IaGi
0030 - c9 e7 41 88 c2 e6 29 86-b5 52 a9 8f 56 3f 79 3e ..A...)..R..V?y>
0040 - 14 bd b5 24 ca c5 f7 a4-ab b9 f9 26 dc 1c 71 7a ...$.......&..qz
0050 - ab 5a a2 a9 76 df 61 70-a5 91 5f 69 36 bc 64 69 .Z..v.ap.._i6.di
0060 - 02 b5 4b ba 79 e0 c9 a7-3b e2 a6 30 9b 2b 34 33 ..K.y...;..0.+43
0070 - 02 af 1e 3c 82 90 bc ca-32 b3 57 5b e0 b6 33 b0 ...<....2.W[..3.
0080 - a6 4c dc a8 c9 01 29 cf-98 ba 7c 40 3a ae 4b 04 .L....)...|#:.K.
0090 - 95 66 2f 96 b2 b9 5b f1-b0 f2 b0 6c e4 61 6f d0 .f/...[....l.ao.
00a0 - 98 a2 67 06 c9 22 ef a3-f5 ec 24 ac a2 b1 5f 4e ..g.."....$..._N
Start Time: 1600571394
Timeout : 7200 (sec)
Verify return code: 0 (ok)
Extended master secret: yes
---
GET /.well-known/mta-sts.txt HTTP/1.1
Host: venabili.tecnologica.com.ar
HTTP/1.1 200 OK
Server: nginx/1.14.2
Date: Sun, 20 Sep 2020 03:10:22 GMT
Content-Type: text/plain
Content-Length: 77
Last-Modified: Sat, 19 Sep 2020 14:07:46 GMT
Connection: keep-alive
ETag: "5f6610b2-4d"
Accept-Ranges: bytes
version: STSv1
mode: testing
mx: venabili.tecnologica.com.ar
max_age: 604800
^C
root#venabili:~# openssl s_client -starttls smtp -connect venabili.tecnologica.com.ar:587 -crlf -ign_eof
CONNECTED(00000004)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = venabili.tecnologica.com.ar
verify return:1
---
Certificate chain
0 s:CN = venabili.tecnologica.com.ar
i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
1 s:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
i:O = Digital Signature Trust Co., CN = DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFbzCCBFegAwIBAgISBAyLUnN1zyR0tAMF7k1G3023MA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0yMDA5MTgyMjMxMjZaFw0y
MDEyMTcyMjMxMjZaMCYxJDAiBgNVBAMTG3ZlbmFiaWxpLnRlY25vbG9naWNhLmNv
bS5hcjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL/svQH9FsoQZyGh
krGZLrN3GAzwuAV2IEsYdAixVrVXGPPjA4oTxQDWncBPVh8x6nMstJqq4xP19aBr
pw5P2rcsMvLT9RJ14wGuNzzDcx9uOmR709ZrPpPzZJ4JJnZxQP5dE2jJMrPQ7tnI
3AudscEaAMW8qBLj91jPDiv4LbRHL4xU+w46yZbB6szgb4u567SSKFaVlRQ8tmCt
msIkYdA5mP5/1Z0QN6PgtAzCWECEkn/eRpuTUHliMjgCocwQ3RYkofGrNQ1a4H7F
RgdaQzjCjQ7JdT/pZTYWrejr3alMsFAukRfIH/FMI5v8ojaxzlQOhz46WJiw3qNH
rmjb+G0CAwEAAaOCAnEwggJtMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggr
BgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQU91GfsO41
iPZN1SROqwqnCPSN9yIwHwYDVR0jBBgwFoAUqEpqYwR93brm0Tm3pkVl7/Oo7KEw
bwYIKwYBBQUHAQEEYzBhMC4GCCsGAQUFBzABhiJodHRwOi8vb2NzcC5pbnQteDMu
bGV0c2VuY3J5cHQub3JnMC8GCCsGAQUFBzAChiNodHRwOi8vY2VydC5pbnQteDMu
bGV0c2VuY3J5cHQub3JnLzAmBgNVHREEHzAdght2ZW5hYmlsaS50ZWNub2xvZ2lj
YS5jb20uYXIwTAYDVR0gBEUwQzAIBgZngQwBAgEwNwYLKwYBBAGC3xMBAQEwKDAm
BggrBgEFBQcCARYaaHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcwggEFBgorBgEE
AdZ5AgQCBIH2BIHzAPEAdwDnEvKwN34aYvuOyQxhhPHqezfLVh0RJlvz4PNL8kFU
bgAAAXSjjusOAAAEAwBIMEYCIQDOwhtVW/tkJzexdCpcX2gLXX7axA8Fw4bDURrr
tu/ecQIhAKpdzqz+iWGw9I7pkKfYNjyyLdWojRlT+IYQfR7qxQc4AHYAB7dcG+V9
aP/xsMYdIxXHuuZXfFeUt2ruvGE6GmnTohwAAAF0o47rMwAABAMARzBFAiBUUOSO
S+8ueu2ACk+862qxbAsXylwZAfHhg/7HmEeEQQIhAKq3jqMMSIjTo47t2JXrU58d
PP6E3NwNHOgGCNje4R/nMA0GCSqGSIb3DQEBCwUAA4IBAQAua/T4drS/7xpNmGDV
Xp2iFhSaVsbWeHh5+40POEAy9NvaZ8O4CeNjV3NeV84QI2Jh1PjehebM72K85rCT
7jEOqaJY5vmKrLBVji6MsgryyGLm4Y368nMR3JrXDLUkATgart+jLdsxLbaRsI1t
Gf+BLc6TYb4roGiI7TTjEQDAy2SOGqfOWbAEdVKIEr9P3IBrlOSOl2o8gd97PXQ/
vEmklVM/5zr5l5paInHOidBL8y8qVk2s9AkaQNAS2QTX5+iFl9D1u8z5mz49eKPb
sb9GSFVTm/qe0KHl02qga4/1ANRoNCuKoaE5xxhqyboJwy58qp7opbzipqxbQI0T
N8e/
-----END CERTIFICATE-----
subject=CN = venabili.tecnologica.com.ar
issuer=C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3389 bytes and written 432 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
250 CHUNKING
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
Protocol : TLSv1.3
Cipher : TLS_AES_256_GCM_SHA384
Session-ID: D8A0F5C48FBFE47FAEF6482AC696CBCAB01EB030F683053A41539DCD5B91E593
Session-ID-ctx:
Resumption PSK: 3CACF3832764ACCF8A00FFDAFA9A8771E62C04FCDF1429A85EB2A3AC0F39733F642045E4602CE73F62AEC75745B51392
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 7200 (seconds)
TLS session ticket:
0000 - 30 00 51 89 4c df 15 62-da 50 55 37 92 60 65 f2 0.Q.L..b.PU7.`e.
0010 - 05 72 39 3d fb 1e 8a 05-2f 95 6b f0 cd 33 29 b0 .r9=..../.k..3).
0020 - b7 38 89 4f 2f 32 91 66-f7 59 2f 80 43 fc 81 f7 .8.O/2.f.Y/.C...
0030 - c6 53 68 3d d8 69 0d 10-6c 4c 62 9c 81 d9 ec 60 .Sh=.i..lLb....`
0040 - 9f ef a0 95 46 d3 e0 10-29 09 20 ab 48 3b 07 34 ....F...). .H;.4
0050 - 82 d3 0e 39 3d 11 18 e8-9e 44 b7 69 b9 d0 45 2e ...9=....D.i..E.
0060 - 97 4f da 69 aa 89 27 2c-e5 9c 63 39 4b f8 3f 54 .O.i..',..c9K.?T
0070 - 23 1a db 73 ac 7e 78 20-76 f5 67 b9 8f e1 c3 34 #..s.~x v.g....4
0080 - d7 f0 b5 0b 0d c7 dc 80-f6 40 0c 20 3f 8d 16 b7 .........#. ?...
0090 - 3a c2 e2 a9 c1 b6 fd 84-65 7f a5 1c 16 81 60 5a :.......e.....`Z
00a0 - 53 12 3a bf d0 4a 0c 0e-a2 3b 57 ce ad 63 89 e6 S.:..J...;W..c..
00b0 - a7 58 ea 21 f9 2e 04 00-ff 6f a7 40 9d 2c bf 39 .X.!.....o.#.,.9
00c0 - 8d d9 19 c9 e1 05 a6 19-a4 60 06 75 8d 3e 95 89 .........`.u.>..
Start Time: 1600567929
Timeout : 7200 (sec)
Verify return code: 0 (ok)
Extended master secret: no
Max Early Data: 0
---
read R BLOCK
ehlo venabili.tecnologica.com.ar
250-venabili.tecnologica.com.ar
250-PIPELINING
250-SIZE
250-VRFY
250-ETRN
250-AUTH PLAIN LOGIN
250-AUTH=PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250-SMTPUTF8
250 CHUNKING
auth login
334 VXNlcm5hbWU6
bm8tcmVzcG9uZGVyQHRlY25vbG9naWNhLmNvbS5hcg==
334 UGFzc3dvcmQ6
!!!
235 2.7.0 Authentication successful
mail from: no-responder#tecnologica.com.ar
250 2.1.0 Ok
rcpt to: jmouriz#gmail.com
250 2.1.5 Ok
data
354 End data with <CR><LF>.<CR><LF>
from: root <no-responder#tecnologica.com.ar>
to: Juan Manuel Mouriz <jmouriz#gmail.com>
subject: Mensaje de prueba
Hola, este es un mensaje de prueba
.
250 2.0.0 Ok: queued as 5E50844853
quit
221 2.0.0 Bye
closed
root#venabili:~# grep 5E50844853 /var/log/mail.log
Sep 19 23:18:54 venabili postfix/submission/smtpd[7994]: 5E50844853: client=venabili.tecnologica.com.ar[200.69.236.179], sasl_method=login, sasl_username=no-responder#tecnologica.com.ar
Sep 19 23:20:35 venabili postfix/cleanup[8543]: 5E50844853: message-id=<20200920021854.5E50844853#venabili.tecnologica.com.ar>
Sep 19 23:20:35 venabili postfix/qmgr[5677]: 5E50844853: from=<no-responder#tecnologica.com.ar>, size=554, nrcpt=1 (queue active)
Sep 19 23:20:35 venabili postfix/cleanup[8543]: 6F3BE4485D: message-id=<20200920021854.5E50844853#venabili.tecnologica.com.ar>
Sep 19 23:20:35 venabili amavis[26666]: (26666-19) Passed CLEAN {RelayedOutbound}, ORIGINATING LOCAL [200.69.236.179]:55833 [200.69.236.179] <no-responder#tecnologica.com.ar> -> <jmouriz#gmail.com>, Queue-ID: 5E50844853, Message-ID: <20200920021854.5E50844853#venabili.tecnologica.com.ar>, mail_id: bcEyyqxUQvNK, Hits: -1, size: 554, queued_as: 6F3BE4485D, dkim_new=default:tecnologica.com.ar, 329 ms
Sep 19 23:20:35 venabili postfix/smtp[8676]: 5E50844853: to=<jmouriz#gmail.com>, relay=127.0.0.1[127.0.0.1]:10026, delay=115, delays=115/0.02/0.01/0.32, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10027): 250 2.0.0 Ok: queued as 6F3BE4485D)
Sep 19 23:20:35 venabili postfix/qmgr[5677]: 5E50844853: removed
root#venabili:~# grep 6F3BE4485D /var/log/mail.log
Sep 19 23:20:35 venabili postfix/smtpd[8679]: 6F3BE4485D: client=localhost[127.0.0.1]
Sep 19 23:20:35 venabili postfix/cleanup[8543]: 6F3BE4485D: message-id=<20200920021854.5E50844853#venabili.tecnologica.com.ar>
Sep 19 23:20:35 venabili postfix/qmgr[5677]: 6F3BE4485D: from=<no-responder#tecnologica.com.ar>, size=1625, nrcpt=1 (queue active)
Sep 19 23:20:35 venabili amavis[26666]: (26666-19) Passed CLEAN {RelayedOutbound}, ORIGINATING LOCAL [200.69.236.179]:55833 [200.69.236.179] <no-responder#tecnologica.com.ar> -> <jmouriz#gmail.com>, Queue-ID: 5E50844853, Message-ID: <20200920021854.5E50844853#venabili.tecnologica.com.ar>, mail_id: bcEyyqxUQvNK, Hits: -1, size: 554, queued_as: 6F3BE4485D, dkim_new=default:tecnologica.com.ar, 329 ms
Sep 19 23:20:35 venabili postfix/smtp[8676]: 5E50844853: to=<jmouriz#gmail.com>, relay=127.0.0.1[127.0.0.1]:10026, delay=115, delays=115/0.02/0.01/0.32, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10027): 250 2.0.0 Ok: queued as 6F3BE4485D)
Sep 19 23:20:37 venabili postfix/smtp[8680]: 6F3BE4485D: to=<jmouriz#gmail.com>, relay=gmail-smtp-in.l.google.com[172.217.192.26]:25, delay=1.7, delays=0.02/0.02/0.88/0.8, dsn=5.7.1, status=bounced (host gmail-smtp-in.l.google.com[172.217.192.26] said: 550-5.7.1 [200.69.236.179] The IP you're using to send mail is not authorized to 550-5.7.1 send email directly to our servers. Please use the SMTP relay at your 550-5.7.1 service provider instead. Learn more at 550 5.7.1 https://support.google.com/mail/?p=NotAuthorizedError a44si4686321qtk.87 - gsmtp (in reply to end of DATA command))
Sep 19 23:20:37 venabili postfix/bounce[8681]: 6F3BE4485D: sender non-delivery notification: 32EF64485E
Sep 19 23:20:37 venabili postfix/qmgr[5677]: 6F3BE4485D: removed
root#venabili:~#
The domain in question is tecnologica.com.ar and I leave a link to some verifications where it is seen that the RR SPF, DKIM, DMARC, _SMTP._TLS, _MTA-STS, MTA-STS, CAA and PTR are correct and it is not a open relay:
https://mxtoolbox.com/domain/tecnologica.com.ar
The certificates are signed by Let's Encrypt. The host name matches the certificate. The classes are correct. And finally, I do not SPAM. I would greatly appreciate a help because at this point I am completely disoriented and can't find what else to do.
I am not looking for a solution but rather where I can find it.
Thank you very much for your help
Additional info
This is a report domain for DMARC:
<?xml version="1.0" encoding="UTF-8" ?>
<feedback>
<report_metadata>
<org_name>google.com</org_name>
<email>noreply-dmarc-support#google.com</email>
<extra_contact_info>https://support.google.com/a/answer/2466580</extra_contact_info>
<report_id>7762784093316082866</report_id>
<date_range>
<begin>1603756800</begin>
<end>1603843199</end>
</date_range>
</report_metadata>
<policy_published>
<domain>tecnologica.com.ar</domain>
<adkim>r</adkim>
<aspf>r</aspf>
<p>reject</p>
<sp>reject</sp>
<pct>100</pct>
</policy_published>
<record>
<row>
<source_ip>200.69.236.179</source_ip>
<count>2</count>
<policy_evaluated>
<disposition>none</disposition>
<dkim>pass</dkim>
<spf>pass</spf>
</policy_evaluated>
</row>
<identifiers>
<header_from>tecnologica.com.ar</header_from>
</identifiers>
<auth_results>
<dkim>
<domain>tecnologica.com.ar</domain>
<result>pass</result>
<selector>default</selector>
</dkim>
<spf>
<domain>tecnologica.com.ar</domain>
<result>pass</result>
</spf>
</auth_results>
</record>
</feedback>
And this is a report domain for TLS:
{"organization-name":"Google Inc.","date-range":{"start-datetime":"2020-10-26T00:00:00Z","end-datetime":"2020-10-26T23:59:59Z"},"contact-info":"smtp-tls-reporting#google.com","report-id":"2020-10-26T00:00:00Z_tecnologica.com.ar","policies":[{"policy":{"policy-type":"sts","policy-string":["version: STSv1\r","mode: testing\r","mx: venabili.tecnologica.com.ar\r","max_age: 604800\r","\r"],"policy-domain":"tecnologica.com.ar"},"summary":{"total-successful-session-count":1,"total-failure-session-count":0}}]}
Hi i'm using sigar very usefully,
but some critical problem was occurred.
in Windows 10 & jdk11 sigar lib makes JVM crash(append logs below) at all method
but, It works fine when using jdk8 under same environment.
It seems to my sigar-amd64-winnt.dll isn't compatible with jdk11 on windows 10.
so i update my sigar binary to 1.6.4 download from https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/magelan/hyperic-sigar-1.6.4.zip
Unfortunately problem isn't resolved.
is any resolution about this problem?
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000010014ed4, pid=28016, tid=5076
#
# JRE version: Java(TM) SE Runtime Environment (11.0.6+8) (build 11.0.6+8-LTS)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (11.0.6+8-LTS, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C [sigar-amd64-winnt.dll+0x14ed4]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
--------------- S U M M A R Y ------------
Command Line: org.ngrinder.NGrinderAgentStarter --mode=agent --command=run -o
Host: Intel(R) Core(TM) i7-7700 CPU # 3.60GHz, 8 cores, 15G, Windows 10 , 64 bit Build 17134 (10.0.17134.753)
Time: Thu Feb 13 18:15:10 2020 ¢¥eCN©öI¡¾©ö C¡ÍA¨ª¨öA elapsed time: 0 seconds (0d 0h 0m 0s)
--------------- T H R E A D ---------------
Current thread (0x00000219633f8800): JavaThread "main" [_thread_in_native, id=5076, stack(0x000000256b800000,0x000000256b900000)]
Stack: [0x000000256b800000,0x000000256b900000], sp=0x000000256b8fee50, free space=1019k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [sigar-amd64-winnt.dll+0x14ed4]
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j org.hyperic.sigar.ProcState.gather(Lorg/hyperic/sigar/Sigar;J)V+0
j org.hyperic.sigar.ProcState.fetch(Lorg/hyperic/sigar/Sigar;J)Lorg/hyperic/sigar/ProcState;+11
j org.hyperic.sigar.Sigar.getProcState(J)Lorg/hyperic/sigar/ProcState;+2
j org.hyperic.sigar.Sigar.getProcState(Ljava/lang/String;)Lorg/hyperic/sigar/ProcState;+6
j org.ngrinder.NGrinderAgentStarter.checkDuplicatedRun(Ljava/lang/String;)V+26
j org.ngrinder.NGrinderAgentStarter.main([Ljava/lang/String;)V+227
v ~StubRoutines::call_stub
siginfo: EXCEPTION_ACCESS_VIOLATION (0xc0000005), reading address 0x0000000009811d78
Register to memory mapping:
RIP=0x0000000010014ed4 sigar-amd64-winnt.dll
RAX=0x0000000009811c40 is an unknown value
RBX={method} {0x000002197fe1ab58} 'gather' '(Lorg/hyperic/sigar/Sigar;J)V' in 'org/hyperic/sigar/ProcState'
RCX=0x00000219633f8b40 points into unknown readable memory: 90 5d 23 16 ff 7f 00 00
RDX=0x000000256b8ff0b0 is pointing into the stack for thread: 0x00000219633f8800
RSP=0x000000256b8fee50 is pointing into the stack for thread: 0x00000219633f8800
RBP=0x000000256b8ff080 is pointing into the stack for thread: 0x00000219633f8800
RSI=0x0000000000200021 is an unknown value
RDI=0x0000000000000268 is an unknown value
R8 =0x0000000000000032 is an unknown value
R9 =0x00000007107f73c0 is an oop: org.hyperic.sigar.Sigar
{0x00000007107f73c0} - klass: 'org/hyperic/sigar/Sigar'
R10=0x0000000000000010 is an unknown value
R11=0x00007fff162b0188 jvm.dll
R12=0x0 is NULL
R13=0x000002197fe1ab48 is pointing into metadata
R14=0x000000256b8ff0b8 is pointing into the stack for thread: 0x00000219633f8800
R15=0x00000219633f8800 is a thread
Registers:
RAX=0x0000000009811c40, RBX=0x000002197fe1ab50, RCX=0x00000219633f8b40, RDX=0x000000256b8ff0b0
RSP=0x000000256b8fee50, RBP=0x000000256b8ff080, RSI=0x0000000000200021, RDI=0x0000000000000268
R8 =0x0000000000000032, R9 =0x00000007107f73c0, R10=0x0000000000000010, R11=0x00007fff162b0188
R12=0x0000000000000000, R13=0x000002197fe1ab48, R14=0x000000256b8ff0b8, R15=0x00000219633f8800
RIP=0x0000000010014ed4, EFLAGS=0x0000000000010202
Top of Stack: (sp=0x000000256b8fee50)
0x000000256b8fee50: 00000219633f8b40 000000256b8ff0b0
0x000000256b8fee60: 00000219633f8800 0000000000000000
0x000000256b8fee70: 0000000009811c40 0000000000000000
0x000000256b8fee80: 000000256b8ff0b8 000000001002113b
0x000000256b8fee90: 00000219633f8b40 000000256b8ff0b0
0x000000256b8feea0: 0000000000200021 0000000000000268
0x000000256b8feeb0: 0000000000000000 0000000000000000
0x000000256b8feec0: 0000000000000000 0000000000000000
0x000000256b8feed0: 0000000000000000 000002190875c9b0
0x000000256b8feee0: 00007fff160ce510 000002190860a550
0x000000256b8feef0: 000000037f73c000 00007fff15f009e5
0x000000256b8fef00: 000002197fe1ab50 00000219633f8800
0x000000256b8fef10: 0000000100000004 00007fff00000003
0x000000256b8fef20: 00000219633f9160 0000002800000003
0x000000256b8fef30: 00000219633f8800 000000256b8ff0b8
0x000000256b8fef40: 000002197fe1ab48 00007fff159c880f
Instructions: (pc=0x0000000010014ed4)
0x0000000010014eb4: 7c 24 20 00 75 15 48 8d 15 df 58 04 00 48 8b 4c
0x0000000010014ec4: 24 40 e8 45 00 00 00 33 c0 eb 32 48 8b 44 24 20
0x0000000010014ed4: 83 b8 38 01 00 00 00 74 1f 48 8b 44 24 20 44 8b
0x0000000010014ee4: 80 38 01 00 00 48 8b 54 24 20 48 8b 4c 24 40 e8
.
.
. (skip)
.
.
Stack slot to memory mapping:
stack at sp + 0 slots: 0x00000219633f8b40 points into unknown readable memory: 90 5d 23 16 ff 7f 00 00
stack at sp + 1 slots: 0x000000256b8ff0b0 is pointing into the stack for thread: 0x00000219633f8800
stack at sp + 2 slots: 0x00000219633f8800 is a thread
stack at sp + 3 slots: 0x0 is NULL
stack at sp + 4 slots: 0x0000000009811c40 is an unknown value
stack at sp + 5 slots: 0x0 is NULL
stack at sp + 6 slots: 0x000000256b8ff0b8 is pointing into the stack for thread: 0x00000219633f8800
stack at sp + 7 slots: 0x000000001002113b sigar-amd64-winnt.dll
--------------- S Y S T E M ---------------
OS: Windows 10 , 64 bit Build 17134 (10.0.17134.753)
CPU:total 8 (initial active 8) (4 cores per cpu, 2 threads per core) family 6 model 158 stepping 9, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, ht, tsc, tscinvbit, bmi1, bmi2, adx, fma
Memory: 4k page, system-wide physical 16287M (6175M free)
TotalPageFile size 23455M (AvailPageFile size 6527M)
current process WorkingSet (physical memory assigned to process): 74M, peak: 74M
current process commit charge ("private bytes"): 355M, peak: 355M
vm_info: Java HotSpot(TM) 64-Bit Server VM (11.0.6+8-LTS) for windows-amd64 JRE (11.0.6+8-LTS), built on Dec 11 2019 09:17:57 by "mach5one" with MS VC++ 15.5 (VS2017)
END.
My application sends an email to an Exchange mail server, mail server is configured with a third party application where it routes email to agent and agent replies to that email. Application reads agent reply from the mailbox which is used to send the email.
Email sending code is below;
Message mimeMessage = new MimeMessage(session);
mimeMessage.setFrom(new InternetAddress(from));
mimeMessage.addRecipient(Message.RecipientType.TO, new InternetAddress(to));
mimeMessage.setSubject(subject);
mimeMessage.setContent(emailText,"text/plain");
mimeMessage.setReplyTo(replyToAddress);
Transport.send(mimeMessage);
This works perfectly. When agent reply is received, Application read it as;
if (message.isMimeType("multipart/MIXED")) {
logger.info("Email MIME Type is: multipart/MIXED");
MimeMultipart multipart =(MimeMultipart)message.getContent();
logger.info("Content type = "+multipart.getContentType());
int count = multipart.getCount();
}
The content type is "multipart/mixed" but the count is 0 means there are no parts in this emails.
I need to set System property,
System.setProperty("mail.mime.multipart.allowempty", "true");
if it is not set, multipart.getCount() throws "missingBoundryException".
Why it is so ?
I can see that the agent's reply is not empty.
The email was sent with content type as text/plain, why reply type is multipart/mixed?
Is this due to any invalid formatting of email by third party application, what is the workaround?
Below is the snap of agent reply.
Below is the raw MIME content,
Received: from sociaminer.host (192.168.1.29) by thirdpartHost
(192.168.1.53) with Microsoft SMTP Server (TLS) id 14.1.218.12; Thu, 19 Jan
2017 17:06:26 +0500
To: hafiz <hafiz#bla.bla>
Message-ID: <hassan.MESSAGEID#bla.bla>
In-Reply-To: <CF72F94#bla.bla>
References: <CF72F945A#bla.bla>
Subject: Re: 1122+50
Content-Type: multipart/mixed;
boundary="----=_Part_127_14151461.1484827604583"
From: <reply#bla.bla>
Return-Path: reply#bla.bla
Date: Thu, 19 Jan 2017 17:06:26 +0500
X-MS-Exchange-Organization-AuthSource: bla.bla
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 06
X-Originating-IP: [SocialMinerIP]
MIME-Version: 1.0
------=_Part_127_14151461.1484827604583
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 7bit
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">Reply to 50<br>
<blockquote><hr>
<b>From:</b> hafiz <hafiz#bla.bla><br><b>Sent:</b> Thursday, January 19, 2017 5:05 PM<br><b>To:</b> testing2 <testing2#bla.bla><br><b>Subject:</b> 1122+50<br>
<html dir="ltr">
<head>
<style type="text/css" id="owaParaStyle"></style>
</head>
<body fpstyle="1" ocsi="0">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">Testing 50</div>
</body>
</html>
</blockquote>
------=_Part_127_14151461.1484827604583--
JavaMail debug output looks like below,
DEBUG: setDebug: JavaMail version 1.4.7
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle]
DEBUG IMAP: mail.imap.fetchsize: 16384
DEBUG IMAP: mail.imap.ignorebodystructuresize: false
DEBUG IMAP: mail.imap.statuscachetimeout: 1000
DEBUG IMAP: mail.imap.appendbuffersize: -1
DEBUG IMAP: mail.imap.minidletime: 10
DEBUG IMAP: disable AUTH=PLAIN
DEBUG IMAP: enable STARTTLS
DEBUG IMAP: trying to connect to host "Echange IP", port 143, isSSL false
* OK The Microsoft Exchange IMAP4 service is ready.
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 LOGINDISABLED STARTTLS UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+
A0 OK CAPABILITY completed.
DEBUG IMAP: protocolConnect login, host=192.168.1.53, user=hafiz#bla.bla, password=<non-null>
A1 STARTTLS
A1 OK Begin TLS negotiation now.
A2 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+
A2 OK CAPABILITY completed.
DEBUG IMAP: AUTH: NTLM
DEBUG IMAP: AUTH: GSSAPI
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: AUTHENTICATE NTLM command trace suppressed
DEBUG NTLM: type 1 message: 4E 54 4C 4D 53 53 50 00 01 00 00 00 03 A2 00 00 00 00 00 00 23 00 00 00 03 00 03 00 20 00 00 00 31 39 32
DEBUG NTLM: type 3 message: 4E 54 4C 4D 53 53 50 00 03 00 00 00 18 00 18 00 68 00 00 00 18 00 18 00 80 00 00 00 00 00 00 00 40 00 00 00 22 00 22 00 40 00 00 00 06 00 06 00 62 00 00 00 00 00 00 00 98 00 00 00 01 82 00 00 68 00 61 00 66 00 69 00 7A 00 40 00 65 00 66 00 6C 00 61 00 62 00 2E 00 6C 00 6F 00 63 00 61 00 6C 00 31 00 39 00 32 00 3B 5E 2B 86 67 49 E3 01 C9 9E F2 CA ED 54 21 11 81 89 94 C6 EC E0 26 E3 BA DB E7 5A F4 CA 28 17 7C 0E 8A 08 18 B5 5A 4E 72 4F C5 7F 52 64 FA 76
DEBUG IMAP: AUTHENTICATE NTLM command result: A3 OK AUTHENTICATE completed.
A4 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+
A4 OK CAPABILITY completed.
DEBUG IMAP: AUTH: NTLM
DEBUG IMAP: AUTH: GSSAPI
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: connection available -- size: 1
A5 SELECT INBOX
* 40 EXISTS
* 0 RECENT
* FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
* OK [PERMANENTFLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)] Permanent flags
* OK [UNSEEN 39] Is the first unseen message
* OK [UIDVALIDITY 436] UIDVALIDITY value
* OK [UIDNEXT 46] The next unique identifier value
A5 OK [READ-WRITE] SELECT completed.
A6 SEARCH UNSEEN ALL
* SEARCH 39
A6 OK SEARCH completed.
A7 SEARCH UNSEEN ALL
* SEARCH 39
A7 OK SEARCH completed.
main INFO emailToSms.EmailReader - 1 unread emails read from inbox.
A8 STORE 39 +FLAGS (\Seen)
* 39 FETCH (FLAGS (\Seen))
A8 OK STORE completed.
A9 FETCH 39 (BODY.PEEK[HEADER])
* 39 FETCH (BODY[HEADER] {851}
MIME-Version: 1.0
Received: from HOST (IP) by HOST
(192.168.1.53) with Microsoft SMTP Server (TLS) id 14.1.218.12; Thu, 19 Jan
2017 17:06:26 +0500
To: hafiz <hafiz#bla.bla>
Message-ID: <hassan.B69E3DD110000159000004A73F57FEE3.1484827604448.cisco-ccp#bla.bla>
In-Reply-To: <CF72F945A1ED2E438A53A11DA9415F65A0E981#Expert.bla.bla>
References: <CF72F945A1ED2E438A53A11DA9415F65A0E981#Expert.bla.bla>
Subject: Re: 1122+50
Content-Type: multipart/mixed;
boundary="----=_Part_127_14151461.1484827604583"
From: <testing2#bla.bla>
Return-Path: testing2#bla.bla
Date: Thu, 19 Jan 2017 17:06:26 +0500
X-MS-Exchange-Organization-AuthSource: Expert.bla.bla
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 06
X-Originating-IP: [IP]
)
A9 OK FETCH completed.
A10 FETCH 39 (ENVELOPE INTERNALDATE RFC822.SIZE)
* 39 FETCH (ENVELOPE ("Thu, 19 Jan 2017 17:06:26 +0500" "Re: 1122+50" ((NIL NIL "testing2" "bla.bla")) NIL NIL (("hafiz" NIL "hafiz" "bla.bla")) NIL NIL "<CF72F945A1ED2E438A53A11DA9415F65A0E981#Expert.bla.bla>" "<hassan.B69E3DD110000159000004A73F57FEE3.1484827604448.cisco-ccp#bla.bla>") INTERNALDATE "19-Jan-2017 17:06:26 +0500" RFC822.SIZE 1250)
A10 OK FETCH completed.
A11 FETCH 39 (BODYSTRUCTURE)
* 39 FETCH (BODYSTRUCTURE ("multipart" "mixed" ("boundary" "----=_Part_127_14151461.1484827604583") NIL NIL 7BIT 0 NIL NIL NIL NIL))
A11 OK FETCH completed.
DEBUG IMAP: IMAPProtocol noop
A12 NOOP
A12 OK NOOP completed.
This is a bug in Microsoft Exchange. Report this bug to Microsoft and upgrade to a newer version or newer service pack if possible in case they've already fixed it.
Exchange is returning the BODYSTRUCTURE information for the message as if it were a single part message when in fact it is a multipart message. This is a violation of the IMAP protocol spec.
You can use the workaround in the JavaMail FAQ.
Also, you might want to upgrade to a newer version of JavaMail - 1.4.7 is pretty old, the current version is 1.5.6.
This is happening on JBoss EAP 6.1, for single user it works fine, But as soon as 2 or 3 concurrent user start interacting with the application, JVM gets crashed.
The JVM is crashing with EXCEPTION_ACCESS_VIOLATION; the full trace is below:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x024c8266, pid=5136, tid=8736
#
# JRE version: 6.0_45-b06
# Java VM: Java HotSpot(TM) Client VM (20.45-b01 mixed mode windows-x86 )
# Problematic frame:
# J org.infinispan.util.concurrent.jdk8backported.LongAdder.add(J)V
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread (0x6602dc00): JavaThread "http-ds-7071bc90200f..corp.in/10.112.70.75:8080-6" daemon [_thread_in_Java, id=8736, stack(0x663a0000,0x663f0000)]
siginfo: ExceptionCode=0xc0000005, reading address 0x8ddb6987
Registers:
EAX=0x00000002, EBX=0xffffffff, ECX=0x00000001, EDX=0x0ddb6988
ESP=0x663ee3c0, EBP=0x663eea58, ESI=0x0ddb60a0, EDI=0xffffffff
EIP=0x024c8266, EFLAGS=0x00010293
Top of Stack: (sp=0x663ee3c0)
0x663ee3c0: 0ddb4830 00000000 663eea58 024d2b44
0x663ee3d0: 0ddb1610 0ddb1610 663eea58 024d7368
0x663ee3e0: 0ddb4830 00000000 0ddb1610 0ddb6be0
0x663ee3f0: 22239910 0ddb1610 2223aff0 22239910
0x663ee400: 0ddb60a0 2223c338 2127c660 24bb6ca8
0x663ee410: 24c1f6e0 0ddb6070 00000002 0ddb6988
0x663ee420: 4ac892b3 0ddb1610 663eea58 024d5bf4
0x663ee430: 00000000 00000000 663eea58 024c5a0c
Instructions: (pc=0x024c8266)
0x024c8246: 0c 00 00 00 b8 01 00 00 00 8b f7 e9 5b 00 00 00
0x024c8256: 8b bc 24 80 00 00 00 8b 9c 24 84 00 00 00 3b 02
0x024c8266: f2 0f 10 82 ff ff ff 7f 66 0f 7e c6 66 0f 73 d0
0x024c8276: 20 66 0f 7e c0 8b d6 8b c8 03 d7 13 cb 89 34 24
Register to memory mapping:
EAX=0x00000002 is an unknown value
EBX=0xffffffff is an unknown value
ECX=0x00000001 is an unknown value
EDX=
[error occurred during error reporting (printing register info), id 0xc0000005]
Stack: [0x663a0000,0x663f0000], sp=0x663ee3c0, free space=312k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J org.infinispan.util.concurrent.jdk8backported.LongAdder.add(J)V
j org.infinispan.CacheImpl.put(Ljava/lang/Object;Ljava/lang/Object;JLjava/util/concurrent/TimeUnit;JLjava/util/concurrent/TimeUnit;Ljava/util/EnumSet;Ljava/lang/ClassLoader;)Ljava/lang/Object;+24
j org.infinispan.CacheImpl.put(Ljava/lang/Object;Ljava/lang/Object;JLjava/util/concurrent/TimeUnit;JLjava/util/concurrent/TimeUnit;)Ljava/lang/Object;+12
j org.infinispan.CacheSupport.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+17
j org.infinispan.AbstractDelegatingCache.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+6
j org.infinispan.spring.provider.SpringCache.put(Ljava/lang/Object;Ljava/lang/Object;)V+6
j com.hmtp.security.server.util.SessionTokenValidator.refreshAccessTime(Ljava/lang/String;)V+25
j com.hmtp.security.server.util.SessionTokenValidator.checkAndRenewToken([Ljava/lang/String;)Lorg/springframework/security/web/authentication/rememberme/PersistentRememberMeToken;+174
j com.hmtp.security.server.auth.MultiTenantRememberMeServices.processAutoLoginCookie([Ljava/lang/String;Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)Lorg/springframework/security/core/userdetails/UserDetails;+5
j org.springframework.security.web.authentication.rememberme.AbstractRememberMeServices.autoLogin(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)Lorg/springframework/security/core/Authentication;+64
j com.hmtp.common.security.server.BrownstoneRememberMeAuthenticationFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+20
J org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
j org.springframework.security.web.FilterChainProxy.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+40
j org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(Ljavax/servlet/Filter;Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+5
j org.springframework.web.filter.DelegatingFilterProxy.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+71
J org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
j org.apache.catalina.core.StandardContextValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+200
j org.jboss.as.web.session.ClusteredSessionValve.handleRequest(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;Lorg/jboss/servlet/http/HttpEvent;Z)V+61
j org.jboss.as.web.session.ClusteredSessionValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+26
j org.jboss.as.web.session.JvmRouteValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+33
j org.jboss.as.web.session.LockingValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+43
j org.jboss.as.web.security.SecurityContextAssociationValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+402
J org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V
j org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+42
j org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V+188
j org.apache.coyote.http11.Http11Processor.process(Ljava/net/Socket;)Lorg/apache/tomcat/util/net/JIoEndpoint$Handler$SocketState;+349
j org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Ljava/net/Socket;)Lorg/apache/tomcat/util/net/JIoEndpoint$Handler$SocketState;+65
j org.apache.tomcat.util.net.JIoEndpoint$Worker.run()V+128
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
V [jvm.dll+0xfb88b]
V [jvm.dll+0x18d551]
V [jvm.dll+0xfba31]
V [jvm.dll+0xfba8b]
V [jvm.dll+0xb5e89]
V [jvm.dll+0x119b74]
V [jvm.dll+0x14217c]
C [msvcr71.dll+0x9565] endthreadex+0xa0
C [kernel32.dll+0x4ee6c] BaseThreadInitThunk+0x12
C [ntdll.dll+0x6399b] RtlInitializeExceptionChain+0xef
C [ntdll.dll+0x6396e] RtlInitializeExceptionChain+0xc2
VM state:synchronizing (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x00dd8360] Safepoint_lock - owner thread: 0x01a68c00
[0x00dd83c8] Threads_lock - owner thread: 0x01a68c00
Heap
def new generation total 400384K, used 175418K [0x03ae0000, 0x1ed50000, 0x1ed80000)
eden space 355904K, 47% used [0x03ae0000, 0x0df2c4f0, 0x19670000)
from space 44480K, 16% used [0x1c1e0000, 0x1c8e2540, 0x1ed50000)
to space 44480K, 0% used [0x19670000, 0x19670000, 0x1c1e0000)
tenured generation total 889536K, used 111605K [0x1ed80000, 0x55230000, 0x552e0000)
the space 889536K, 12% used [0x1ed80000, 0x25a7d7f0, 0x25a7d800, 0x55230000)
compacting perm gen total 102144K, used 102123K [0x552e0000, 0x5b6a0000, 0x652e0000)
the space 102144K, 99% used [0x552e0000, 0x5b69ad60, 0x5b69ae00, 0x5b6a0000)
No shared spaces configured.
Code Cache [0x01ae0000, 0x02cb8000, 0x03ae0000)
total_blobs=9369 nmethods=9146 adapters=163 free_code_cache=14876096 largest_free_block=384
VM Arguments:
jvm_args: -Dprogram.name=standalone.bat -Xms1303M -Xmx1303M -XX:MaxPermSize=256M -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n -Dorg.jboss.boot.log.file=C:\jboss-eap-6.1\standalone\log\server.log -Dlogging.configuration=file:C:\jboss-eap-6.1\standalone/configuration/logging.properties
java_command: C:\jboss-eap-6.1\jboss-modules.jar -mp C:\jboss-eap-6.1\modules -jaxpmodule javax.xml.jaxp-provider org.jboss.as.standalone -Djboss.home.dir=C:\jboss-eap-6.1
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=C:\Program Files\Java\jdk1.6.0_45
PATH=C:\ProgramData\Oracle\Java\javapath;C:\Program Files\Java\jdk1.6.0_45\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\apache-maven-3.2.2\bin;C:\Program Files\TortoiseSVN\bin;D:\apache-ant-1.9.4\bin
USERNAME=neeraj.ar
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 6 Model 23 Stepping 10, GenuineIntel
--------------- S Y S T E M ---------------
OS: Windows 7 Build 7601 Service Pack 1
CPU:total 2 (2 cores per cpu, 1 threads per core) family 6 model 23 stepping 10, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1
Memory: 4k page, physical 3338764k(929876k free), swap 15625020k(11901304k free)
vm_info: Java HotSpot(TM) Client VM (20.45-b01) for windows-x86 JRE (1.6.0_45-b06), built on Mar 26 2013 13:40:03 by "java_re" with MS VC++ 7.1 (VS2003)
time: Thu Jul 23 18:41:10 2015
elapsed time: 256 seconds
The crash happens in C1-compiled code on the instruction
f20f1082ffffff7f movsd xmm0, qword ptr [edx+0x7fffffff]
which represents a volatile long load that has not been patched.
I believe this is a JVM bug JDK-6965570 or its duplicate JDK-7004258.
It has been fixed a long ago in JDK 6u60, but you seem to use very old version of JDK.
I have been using MongoDB as my primary storage for 1.5Tb+ of data since last year. Everything was fine, but recently i decided to execute some map-reduce against 14 000 000 documents collection and my production instance got down.
Please take a look at details:
My config:
Ubuntu 12.04.5 LTS, MongoDB 2.6.4, LVM (2 HDD, 1.5TB+ free of 3TB+ total), 24GB RAM (almost all free)
Mongo config is default (except logpath and dbpath parameters)
Mongo log:
2014-08-28T07:33:41.147+0400 [DataFileSync] flushing mmaps took 16177ms for 777 files
2014-08-28T07:33:44.004+0400 [conn13] M/R: (1/3) Emit Progress: 9920300
2014-08-28T07:33:47.178+0400 [conn13] M/R: (1/3) Emit Progress: 9928100
2014-08-28T07:33:50.004+0400 [conn13] M/R: (1/3) Emit Progress: 9967800
2014-08-28T07:33:53.115+0400 [conn13] M/R: (1/3) Emit Progress: 10007800
2014-08-28T07:33:56.009+0400 [conn13] M/R: (1/3) Emit Progress: 10048800
2014-08-28T07:33:59.050+0400 [conn13] M/R: (1/3) Emit Progress: 10091200
2014-08-28T07:34:02.530+0400 [conn13] M/R: (1/3) Emit Progress: 10102300
2014-08-28T07:34:05.510+0400 [conn13] M/R: (1/3) Emit Progress: 10102400
2014-08-28T07:34:08.932+0400 [conn13] SEVERE: Invalid access at address: 0x7cc8b2fe70b4
2014-08-28T07:34:08.983+0400 [conn13] SEVERE: Got signal: 7 (Bus error).
Backtrace:0x11e6111 0x11e54ee 0x11e55df 0x7f5a7031ecb0 0xf29cad 0xf32f28 0xf32770 0x8b601f 0x8b693a 0x982885 0x988485 0x9966d8 0x9a3355 0xa2889a 0xa29ce2 0xa2bea6 0xd5dd6d 0xb9fe62 0xba1440 0x770aef
mongod(_ZN5mongo15printStackTraceERSo+0x21) [0x11e6111]
mongod() [0x11e54ee]
mongod() [0x11e55df]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f5a7031ecb0]
mongod(_ZN5mongo16NamespaceDetails5allocEPNS_10CollectionERKNS_10StringDataEi+0x1bd) [0xf29cad]
mongod(_ZN5mongo19SimpleRecordStoreV111allocRecordEii+0x68) [0xf32f28]
mongod(_ZN5mongo17RecordStoreV1Base12insertRecordEPKcii+0x60) [0xf32770]
mongod(_ZN5mongo10Collection15_insertDocumentERKNS_7BSONObjEbPKNS_16PregeneratedKeysE+0x7f) [0x8b601f]
mongod(_ZN5mongo10Collection14insertDocumentERKNS_7BSONObjEbPKNS_16PregeneratedKeysE+0x22a) [0x8b693a]
mongod(_ZN5mongo2mr5State12_insertToIncERNS_7BSONObjE+0x85) [0x982885]
mongod(_ZN5mongo2mr5State14reduceInMemoryEv+0x175) [0x988485]
mongod(_ZN5mongo2mr5State35reduceAndSpillInMemoryStateIfNeededEv+0x148) [0x9966d8]
mongod(_ZN5mongo2mr16MapReduceCommand3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0xcc5) [0x9a3355]
mongod(_ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x3a) [0xa2889a]
mongod(_ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x1042) [0xa29ce2]
mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x6c6) [0xa2bea6]
mongod(_ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x22ed) [0xd5dd6d]
mongod() [0xb9fe62]
mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x580) [0xba1440]
mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x9f) [0x770aef]
After my first run of that map-reduce, i made db.repairDatabase(), but after second attempt to map-reduce (after repairing) the same crash happened again. Now, i have no idea how to get my m/r done
Any ideas, folks?
Having issue investigated, i recently came up with a couple of things:
As it was suggested in comments, i took a look at mongo jira ticket SERVER-12849
and double checked my logs.
/var/log/syslog says:
kernel: [1349503.760215] ata6.00: exception Emask 0x0 SAct 0x3 SErr 0x0 action 0x0
Aug 28 08:18:41 overlord kernel: [1349503.760253] ata6.00: irq_stat 0x40000008
Aug 28 08:18:41 overlord kernel: [1349503.760281] ata6.00: failed command: READ FPDMA QUEUED
Aug 28 08:18:41 overlord kernel: [1349503.760318] ata6.00: cmd 60/08:00:10:48:92/00:00:84:00:00/40 tag 0 ncq 4096 in
Aug 28 08:18:41 overlord kernel: [1349503.760318] res 41/40:08:10:48:92/00:00:84:00:00/00 Emask 0x409 (media error)
Aug 28 08:18:41 overlord kernel: [1349503.760411] ata6.00: status: { DRDY ERR }
Aug 28 08:18:41 overlord kernel: [1349503.760437] ata6.00: error: { UNC }
Aug 28 08:18:41 overlord kernel: [1349503.788325] ata6.00: configured for UDMA/133
Aug 28 08:18:41 overlord kernel: [1349503.788340] sd 5:0:0:0: [sdb] Unhandled sense code
Aug 28 08:18:41 overlord kernel: [1349503.788343] sd 5:0:0:0: [sdb]
Aug 28 08:18:41 overlord kernel: [1349503.788345] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Aug 28 08:18:41 overlord kernel: [1349503.788348] sd 5:0:0:0: [sdb]
Aug 28 08:18:41 overlord kernel: [1349503.788350] Sense Key : Medium Error [current] [descriptor]
Aug 28 08:18:41 overlord kernel: [1349503.788353] Descriptor sense data with sense descriptors (in hex):
Aug 28 08:18:41 overlord kernel: [1349503.788355] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
Aug 28 08:18:41 overlord kernel: [1349503.788365] 84 92 48 10
Aug 28 08:18:41 overlord kernel: [1349503.788370] sd 5:0:0:0: [sdb]
Aug 28 08:18:41 overlord kernel: [1349503.788373] Add. Sense: Unrecovered read error - auto reallocate failed
Aug 28 08:18:41 overlord kernel: [1349503.788376] sd 5:0:0:0: [sdb] CDB:
Aug 28 08:18:41 overlord kernel: [1349503.788377] Read(10): 28 00 84 92 48 10 00 00 08 00
Aug 28 08:18:41 overlord kernel: [1349503.788387] end_request: I/O error, dev sdb, sector 2224179216
Aug 28 08:18:41 overlord kernel: [1349503.788434] ata6: EH complete
looks like /dev/sdb is culprit, let's check SMART status (as suggested in jira)
SMART Error Log Version: 1
ATA Error Count: 135 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 135 occurred at disk power-on lifetime: 11930 hours (497 days + 2 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: WP at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
61 00 08 ff ff ff 4f 00 49d+12:01:35.512 WRITE FPDMA QUEUED
60 00 08 ff ff ff 4f 00 49d+12:01:33.380 READ FPDMA QUEUED
ea 00 00 00 00 00 a0 00 49d+12:01:33.294 FLUSH CACHE EXT
61 00 00 ff ff ff 4f 00 49d+12:01:33.292 WRITE FPDMA QUEUED
ea 00 00 00 00 00 a0 00 49d+12:01:33.153 FLUSH CACHE EXT
Error 134 occurred at disk power-on lifetime: 11930 hours (497 days + 2 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: WP at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
61 00 08 ff ff ff 4f 00 49d+11:17:00.189 WRITE FPDMA QUEUED
61 00 10 ff ff ff 4f 00 49d+11:17:00.189 WRITE FPDMA QUEUED
61 00 28 ff ff ff 4f 00 49d+11:17:00.188 WRITE FPDMA QUEUED
61 00 08 ff ff ff 4f 00 49d+11:17:00.188 WRITE FPDMA QUEUED
61 00 08 ff ff ff 4f 00 49d+11:17:00.188 WRITE FPDMA QUEUED
Error 133 occurred at disk power-on lifetime: 11930 hours (497 days + 2 hours)
When the command that caused the error occurred, the device was active or idle.
so, as we can see there are errors on /dev/sdb, let's do the final check - copy entire data to another host and try to run original map-reduce script there.
Result is success.
So mongo is ok in my case. It seems (Bus Error) log entries in mongo log signal that it is time to check your hardware.