I try to reply a rest response to the user as a simple string (reply message). The authentication against an rest api works and i get a json formatted reponse (called "token"). I've declared this attribute as an attribute in raddb/dictionary file.
My question is: How can i access this attribute in authentication or post-authentication section?
Below my config (raddb/sites-available/default):
authenticate {
#
# REST authentication
Auth-Type rest {
rest {
updated = 1
}
if (updated) {
%{Token}
ok
}
}
I tried all possibilities like &Token "%{Token}" &rest:Token
See below my Debug-Output:
Ready to process requests,
(0) Received Access-Request Id 96 from 127.0.0.1:49260 to 127.0.0.1:1812 length 50,
(0) User-Name = "Arya Stark",
(0) User-Password = "verySecret",
(0) # Executing section authorize from file /etc/freeradius/sites-enabled/default,
(0) authorize {,
(0) [preprocess] = ok,
(0) [chap] = noop,
(0) [mschap] = noop,
(0) [digest] = noop,
(0) suffix: Checking for suffix after "#",
(0) suffix: No '#' in User-Name = "Arya Stark", looking up realm NULL,
(0) suffix: No such realm "NULL",
(0) [suffix] = noop,
(0) eap: No EAP-Message, not doing EAP,
(0) [eap] = noop,
(0) files: users: Matched entry DEFAULT at line 154,
(0) [files] = ok,
(0) [expiration] = noop,
[logintime] = noop,
Not doing PAP as Auth-Type is already set.,
(0) [pap] = noop,
(0) } # authorize = ok,
Found Auth-Type = rest,
(0) # Executing group from file /etc/freeradius/sites-enabled/default,
(0) Auth-Type rest {,
rlm_rest (rest): Reserved connection (0),
(0) rest: Expanding URI components,
(0) rest: EXPAND https://172.16.0.5,
(0) rest: --> https://172.16.0.5,
(0) rest: EXPAND /identityprovider/auth/passwordlogin,
(0) rest: --> /identityprovider/auth/passwordlogin,
(0) rest: Sending HTTP POST to "https://172.16.0.5/provider/auth/login",
(0) rest: EXPAND { "Username": "%{User-Name}", "Password":"%{User-Password}" },
(0) rest: --> { "Username": "Arya Stark", "Password":"verySecret" },
(0) rest: Processing response header,
(0) rest: Status : 200 (OK),
(0) rest: Type : json (application/json),
(0) rest: Parsing attribute "token",
(0) rest: EXPAND eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJBcnlhIFN0YXJrIiwiaWF0IjoxNTgwMTI5......1DuIVMzCI4a1UWUThAce0xnA,
(0) rest: --> eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJBcnlhIFN0YXJrIiwiaWF0IjoxNTgwMTI5......1DuIVMzCI4a1UWUThAce0xnA,
(0) rest: Token := "eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJBcnlhIFN0YXJrIiwiaWF0IjoxNTgwMTI5......1DuIVMzCI4a1UWUThAce0xnA",
rlm_rest (rest): Released connection (0),
Need 5 more connections to reach 10 spares,
rlm_rest (rest): Opening additional connection (5), 1 of 27 pending slots used,
rlm_rest (rest): Connecting to "https://172.16.0.5",
(0) [rest] = updated,
(0) if (updated) {,
(0) if (updated) -> TRUE,
(0) if (updated) {,
(0) EXPAND %{Token},
(0) --> ,
(0) [ok] = ok,
(0) } # if (updated) = ok,
(0) } # Auth-Type rest = ok,
(0) # Executing section post-auth from file /etc/freeradius/sites-enabled/default,
(0) post-auth {,
(0) if (session-state:User-Name && reply:User-Name && request:User-Name && (reply:User-Name == request:User-Name)) {,
(0) if (session-state:User-Name && reply:User-Name && request:User-Name && (reply:User-Name == request:User-Name)) -> FALSE,
(0) update {,
(0) No attributes updated for RHS &session-state:,
(0) } # update = noop,
(0) [exec] = noop,
(0) policy remove_reply_message_if_eap {,
(0) if (&reply:EAP-Message && &reply:Reply-Message) {,
(0) if (&reply:EAP-Message && &reply:Reply-Message) -> FALSE,
(0) else {,
(0) [noop] = noop,
(0) } # else = noop,
(0) } # policy remove_reply_message_if_eap = noop,
} # post-auth = noop,
(0) Sent Access-Accept Id 96 from 127.0.0.1:1812 to 127.0.0.1:49260 length 0,
(0) Finished request,
Waking up in 4.9 seconds.,
(0) Cleaning up request packet ID 96 with timestamp +17,
Ready to process requests,
The issue here is that by default any attributes specified in the JSON response are inserted into the reply list.
This can be fixed in two ways, either change your JSON blob to specify a list qualifier:
{ 'request:Token': '<token value>' }
or access the attribute in the reply list:
%{reply:Token}
Related
I tried to implement a simple read-preferred read-write lock using 2 mutexes (using redis.lock.Lock), like what is described in this link (https://en.wikipedia.org/wiki/Readers%E2%80%93writer_lock)
In the [End Read] steps, I encountered this problem:
If b = 0, unlock g #(a write lock).
As, this READ process is not the one that acquired the lock, so the system throws an error. I think it has some token stored somewhere, and I can get it to use for the lock release, but I am not sure.
Could someone give me a hint? Thanks.
from enum import Enum
from redis import StrictRedis, lock
# data in Redis cache:
# "read_counter_lock_name" : 0
# "read_lock_lock_name" -> read lock, protect "read_counter_lock_name"
# "write_lock_lock_name" -> write lock, protect write data section
class Prefix(Enum):
READ = 'read_lock_'
WRITE = 'write_lock_'
READ_COUNTER = 'read_counter_'
class RedisLockParams(Enum):
TIMEOUT = 60 # maximum life for the lock in seconds = 60 seconds
SLEEP_TIME = 0.1 # the amount of time to sleep in seconds per loop iteration
# in redis lock's acquire() - sleep then retry
BLOCKING = True # acquire() should block until the lock has been acquired
BLOCKING_TIMEOUT = None # maximum amount of time in seconds to spend trying
# to acquire the lock
class ReadWriteLock:
def __init__(self, lock_name: str, redis_host: str, redis_port: int, redis_key: str):
self.__read_lock_name = Prefix.READ.value + lock_name
self.__write_lock_name = Prefix.WRITE.value + lock_name
self.__read_counter_key = Prefix.READ_COUNTER.value + lock_name
# self.__redis_host = redis_host
# self.__redis_port = redis_port
# self.__redis_key = redis_key
self.__cache = StrictRedis(host = redis_host,
port = redis_port,
db=0, # up to 16 logical database
password = redis_key,
ssl=True)
print(f'ping return:{self.__cache.ping()}')
# set the read counter to 0, if it does not exist.
self.__cache.setnx(self.__read_counter_key, 0)
# init the read lock
self.__read_lock = lock.Lock(self.__cache,
self.__read_lock_name,
RedisLockParams.TIMEOUT.value,
RedisLockParams.SLEEP_TIME.value,
RedisLockParams.BLOCKING.value,
RedisLockParams.BLOCKING_TIMEOUT.value)
# init the write lock
self.__write_lock = lock.Lock(self.__cache,
self.__write_lock_name,
RedisLockParams.TIMEOUT.value,
RedisLockParams.SLEEP_TIME.value,
RedisLockParams.BLOCKING.value,
RedisLockParams.BLOCKING_TIMEOUT.value)
def acquire_read_lock(self) -> bool:
write_lock_acquired = False
self.__read_lock.acquire()
read_counter = self.__cache.incr(self.__read_counter_key)
if (read_counter == 1):
write_lock_acquired = self.__write_lock.acquire() # acquire write lock
self.__read_lock.release()
return write_lock_acquired
def release_read_lock(self):
read_lock_acquired = self.__read_lock.acquire()
read_counter = self.__cache.decr(self.__read_counter_key)
if read_counter == 0 and read_lock_acquired:
self.__write_lock.release() # release the write lock-> issue!!!
self.__read_lock.release()
def acquire_write_lock(self) -> bool:
return self.__write_lock.acquire()
def release_write_lock(self):
self.__write_lock.release()
I am having the same issue.
"LockNotOwnedError("Cannot release a lock"\nredis.exceptions.LockNotOwnedError: Cannot release a lock that's no longer owned"
redis = Redis.from_url(redis_url)
try:
with redis.lock(name, timeout=timeout, blocking_timeout=blocking_timeout) as redis_lock:
yield redis_lock
except RedisLockError as e:
logger.warning("Cannot acquire lock", name=name, timeout=timeout, blocking_timeout=blocking_timeout)
raise LockError(f"Cannot acquire lock {name}") from e
Can someone add some hints?
I have figured out how to release the not-owned redis lock by taking a look at the redis' python library source code. Below is the modified version of the multiread-single-write lock class.
# read_write_lock.py
from enum import Enum
from redis import StrictRedis, lock
# data in Redis cache:
# "read_counter_lock_name" : 0
# "read_lock_lock_name" -> read lock, protect "read_counter_lock_name"
# "write_lock_lock_name" -> write lock, protect write data section
class Prefix(Enum):
READ = 'read_lock_'
WRITE = 'write_lock_'
READ_COUNTER = 'read_counter_'
class RedisLockParams(Enum):
TIMEOUT = 60 # maximum life for the lock in seconds = 60 seconds
SLEEP_TIME = 0.1 # the amount of time to sleep in seconds per loop iteration
# in redis lock's acquire() - sleep then retry
BLOCKING = True # acquire() should block until the lock has been acquired
BLOCKING_TIMEOUT = None # maximum amount of time in seconds to spend trying
# to acquire the lock
class ReadWriteLock:
def __init__(self, lock_name: str, redis_host: str, redis_port: int, redis_key: str):
self.__read_lock_name = Prefix.READ.value + lock_name
self.__write_lock_name = Prefix.WRITE.value + lock_name
self.__read_counter_key = Prefix.READ_COUNTER.value + lock_name
self.__cache = StrictRedis(host = redis_host,
port = redis_port,
db=0, # up to 16 logical database
password = redis_key,
ssl=True)
# set the read counter to 0, if it does not exist.
self.__cache.setnx(self.__read_counter_key, 0)
# init the read lock
self.__read_lock = lock.Lock(self.__cache,
self.__read_lock_name,
RedisLockParams.TIMEOUT.value,
RedisLockParams.SLEEP_TIME.value,
RedisLockParams.BLOCKING.value,
RedisLockParams.BLOCKING_TIMEOUT.value)
# init the write lock
self.__write_lock = lock.Lock(self.__cache,
self.__write_lock_name,
RedisLockParams.TIMEOUT.value,
RedisLockParams.SLEEP_TIME.value,
RedisLockParams.BLOCKING.value,
RedisLockParams.BLOCKING_TIMEOUT.value)
def acquire_read_lock(self) -> bool:
write_lock_acquired = False
self.__read_lock.acquire()
read_counter = self.__cache.incr(self.__read_counter_key)
if (read_counter == 1):
write_lock_acquired = self.__write_lock.acquire() # acquire write lock
self.__read_lock.release()
return write_lock_acquired
def release_read_lock(self):
read_lock_acquired = self.__read_lock.acquire()
if read_lock_acquired:
read_counter = self.__cache.decr(self.__read_counter_key)
if read_counter == 0:
if self.__write_lock.owned():
self.__write_lock.release()
else: # if the lock was not owned, just take its token and override
write_lock_token = self.__cache.get(self.__write_lock_name)
self.__write_lock.local.token = write_lock_token
self.__write_lock.release()
self.__read_lock.release()
def acquire_write_lock(self) -> bool:
return self.__write_lock.acquire()
def release_write_lock(self) -> None:
self.__write_lock.release()
Hi I am getting an error from the api response bases on the error code I have to show the error popup.
Error Domain=com.ca.mailina.targetAPI:ErrorDomain Code=1854 "The data
couldn’t be read because it isn’t in the correct format." UserInfo=
{mailinaInfoHeaderInfoKey=<CFBasicHash 0x600000bff000 [0x7fff8004b340]>{type =
immutable dict, count = 8,
entries =>
0 : Pragma = no-cache
1 : x-up-err = 0001854
2 : Content-Type = <CFString 0x6000011c63a0 [0x7fff8004b340]>{contents = "application/json;charset=UTF-8"}
3 : x-mail-err = 100
4 : x-uxp-err = 30102
6 : Date = <CFString 0x6000011c6370 [0xgghyd654fx40]>{contents = "Tue, 17 Aug 2021 10:37:19 GMT"}
10 : Content-Length = 907
11 : Cache-Control = no-store
}, NSLocalizedDescription=The data couldn’t be read because it isn’t in the correct format., status-code=401}
I have to check the condition based on 4 : x-uxp-err = 30102 which is on the fourth position of error response now the problem is I am not able to access the 4th passion of response, guide me to get the solution, example
if x-uxp-err == 30102 {
print("open popup 1")
} else {
print("open popup 2")
}
I'm trying to authenticate Radius Requests against Restful API. My Virtual Server configuration as below:
authorize {
filter_username
filter_password
preprocess
auth_log
if (User-Password) {
update control {
Auth-Type := rest
}
}
}
authenticate {
rest
}
My radius -X output is:
(0) Received Access-Request Id 202 from 127.0.0.2:10708 to 127.0.0.2:1812 length 73
(0) User-Name = "bob"
(0) User-Password = "hello"
(0) NAS-IP-Address = 127.0.0.2
(0) NAS-Port = 1
(0) Message-Authenticator = 0xcd622e98255234964d081be2513a0a9c
(0) # Executing section authorize from file /usr/local/etc/raddb/sites-enabled/testserver
(0) authorize {
(0) policy filter_username {
(0) if (&User-Name) {
(0) if (&User-Name) -> TRUE
(0) if (&User-Name) {
(0) if (&User-Name =~ / /) {
(0) if (&User-Name =~ / /) -> FALSE
(0) if (&User-Name =~ /#[^#]*#/ ) {
(0) if (&User-Name =~ /#[^#]*#/ ) -> FALSE
(0) if (&User-Name =~ /\.\./ ) {
(0) if (&User-Name =~ /\.\./ ) -> FALSE
(0) if ((&User-Name =~ /#/) && (&User-Name !~ /#(.+)\.(.+)$/)) {
(0) if ((&User-Name =~ /#/) && (&User-Name !~ /#(.+)\.(.+)$/)) -> FALSE
(0) if (&User-Name =~ /\.$/) {
(0) if (&User-Name =~ /\.$/) -> FALSE
(0) if (&User-Name =~ /#\./) {
(0) if (&User-Name =~ /#\./) -> FALSE
(0) } # if (&User-Name) = notfound
(0) } # policy filter_username = notfound
(0) policy filter_password {
(0) if (&User-Password && (&User-Password != "%{string:User-Password}")) {
(0) EXPAND %{string:User-Password}
(0) --> hello
(0) if (&User-Password && (&User-Password != "%{string:User-Password}")) -> FALSE
(0) } # policy filter_password = notfound
(0) [preprocess] = ok
(0) auth_log: EXPAND /antikor/log/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/auth-detail-%Y%m%d
(0) auth_log: --> /antikor/log/radacct/127.0.0.2/auth-detail-20170429
(0) auth_log: /antikor/log/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/auth-detail-%Y%m%d expands to /antikor/log/radacct/127.0.0.2/auth-detail-20170429
(0) auth_log: EXPAND %t
(0) auth_log: --> Sat Apr 29 19:46:26 2017
(0) [auth_log] = ok
(0) if (User-Password) {
(0) if (User-Password) -> TRUE
(0) if (User-Password) {
(0) update control {
(0) Auth-Type := rest
(0) } # update control = noop
(0) } # if (User-Password) = noop
(0) } # authorize = ok
(0) Found Auth-Type = rest
(0) # Executing group from file /usr/local/etc/raddb/sites-enabled/testserver
(0) authenticate {
rlm_rest (rest): Reserved connection (0)
(0) rest: Expanding URI components
(0) rest: EXPAND http://127.0.0.1:8902
(0) rest: --> http://127.0.0.1:8902
(0) rest: EXPAND /test.php?action=authenticate
(0) rest: --> /test.php?action=authenticate
(0) rest: Sending HTTP POST to "http://127.0.0.1:8902/test.php?action=authenticate"
(0) rest: EXPAND {"username":"%{User-Name}", "password":"%{User-Password}"}
(0) rest: --> {"username":"bob", "password":"hello"}
(0) rest: Processing response header
(0) rest: Status : 200 (OK)
(0) rest: Type : json (application/json)
(0) rest: Parsing attribute "control:Cleartext-Password"
(0) rest: EXPAND hello
(0) rest: --> hello
(0) rest: Cleartext-Password := "hello"
(0) rest: Parsing attribute "request:User-Password"
(0) rest: EXPAND hello
(0) rest: --> hello
(0) rest: User-Password := "hello"
(0) rest: Parsing attribute "reply:Reply-Message"
(0) rest: EXPAND Hello bob
(0) rest: --> Hello bob
(0) rest: Reply-Message := "Hello bob"
rlm_rest (rest): Released connection (0)
Need 5 more connections to reach 10 spares
rlm_rest (rest): Opening additional connection (5), 1 of 27 pending slots used
rlm_rest (rest): Connecting to "http://127.0.0.1:8902/test.php"
(0) [rest] = updated
(0) } # authenticate = updated
(0) Failed to authenticate the user
(0) Login incorrect: [bob/hello] (from client antikor-l2tp port 1)
(0) Using Post-Auth-Type Reject
(0) # Executing group from file /usr/local/etc/raddb/sites-enabled/testserver
(0) Post-Auth-Type REJECT {
(0) attr_filter.access_reject: EXPAND %{User-Name}
(0) attr_filter.access_reject: --> bob
(0) attr_filter.access_reject: Matched entry DEFAULT at line 11
(0) [attr_filter.access_reject] = updated
(0) [eap] = noop
(0) policy remove_reply_message_if_eap {
(0) if (&reply:EAP-Message && &reply:Reply-Message) {
(0) if (&reply:EAP-Message && &reply:Reply-Message) -> FALSE
(0) else {
(0) [noop] = noop
(0) } # else = noop
(0) } # policy remove_reply_message_if_eap = noop
(0) } # Post-Auth-Type REJECT = updated
(0) Delaying response for 1.000000 seconds
Waking up in 0.3 seconds.
Waking up in 0.6 seconds.
(0) Sending delayed response
(0) Sent Access-Reject Id 202 from 127.0.0.2:1812 to 127.0.0.2:10708 length 33
(0) Reply-Message = "Hello bob"
I added both control:Cleartext-Password and request:User-Password variables to test.php json reply. tried one by one. But still authentication step fails. JSON output is as below:
{"control:Cleartext-Password":"hello", "request:User-Password":"hello","reply:Reply-Message":"Hello bob"}
I wonder that if the JSON response is wrong and how should it be for Authentication reply?
Thanks.
The authorize method rlm_rest module acts like other datastore modules like rlm_sql, rlm_redis and rlm_couchbase.
It is mainly for retrieving AVPs from a remote source, it can be used as an authentication module, but not in the way you were calling it above (example at the bottom of this answer).
With the way you're calling rlm_rest, In order for the user to be accepted, you'll need to list another module that can look at the attributes in the request, look at what you got back from your rest API, and figure out what type of authentication to perform. If you're doing plaintext authentication (i.e. no EAP) then you can use the pap module.
Your server config would then look something like
authorize {
rest
pap
}
authenticate {
pap
}
rest.authorize retrieves control:Cleartext-Password which gives the server the "good" password to compare against the password the user sent.
pap.authorize checks to see if request:User-Password exists, and if it does, sets control:Auth-Type pap.
pap.authenticate compares control:Cleartext-Password with request:User-Password and if they match returns ok or reject depending on whether they do or not.
The other way of authenticating a plaintext userr in this case is by using HTTP BasicAuth and rlm_rest's authenticate method. The policy for that would look something like this:
authorize {
if (&User-Password) {
update control {
Auth-Type := 'rest'
}
}
}
authenticate {
rest
}
I have a Go1.5.1 process/app. When I run /usr/sbin/lsof -p on that process, I see a lot of "can't identify protocol".
monitor_ 13105 root 101u sock 0,6 0t0 16960100 can't identify protocol
monitor_ 13105 root 102u sock 0,6 0t0 21552427 can't identify protocol
monitor_ 13105 root 103u sock 0,6 0t0 17565091 can't identify protocol
monitor_ 13105 root 104u sock 0,6 0t0 18476870 can't identify protocol
proc status/limit/fd
[root#Monitor_q ~]# cat /proc/13105/status
Name: monitor_client
State: S (sleeping)
Tgid: 13105
Pid: 13105
PPid: 13104
TracerPid: 0
Uid: 0 0 0 0
Gid: 0 0 0 0
Utrace: 0
FDSize: 16384
Groups:
...
[root#Monitor_q ~]# cat /proc/13105/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 10485760 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 3870 3870 processes
Max open files 9999 9999 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 3870 3870 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
[root#Monitor_q ~]# ll /proc/13105/fd/
lrwx------ 1 root root 64 Dec 7 00:15 8382 -> socket:[52023221]
lrwx------ 1 root root 64 Dec 7 00:15 8383 -> socket:[51186627]
lrwx------ 1 root root 64 Dec 7 00:15 8384 -> socket:[51864232]
lrwx------ 1 root root 64 Dec 7 00:15 8385 -> socket:[52435453]
lrwx------ 1 root root 64 Dec 7 00:15 8386 -> socket:[51596071]
lrwx------ 1 root root 64 Dec 7 00:15 8387 -> socket:[52767667]
lrwx------ 1 root root 64 Dec 7 00:15 8388 -> socket:[52090632]
lrwx------ 1 root root 64 Dec 7 00:15 8389 -> socket:[51739068]
lrwx------ 1 root root 64 Dec 7 00:15 839 -> socket:[22963529]
lrwx------ 1 root root 64 Dec 7 00:15 8390 -> socket:[52023223]
lrwx------ 1 root root 64 Dec 7 00:15 8391 -> socket:[52560389]
lrwx------ 1 root root 64 Dec 7 00:15 8392 -> socket:[52402565]
...
but there is no similar output in netstat -a.
What are these sockets and how can I find out what they do?
monitor_client.go
package main
import (
"crypto/tls"
"encoding/json"
"fmt"
"log"
"net"
"net/http"
nurl "net/url"
"strconv"
"strings"
"syscall"
"time"
)
type Result struct {
Error string `json:"error"`
HttpStatus int `json:"http_status"`
Stime time.Duration `json:"http_time"`
}
//http://stackoverflow.com/questions/20990332/golang-http-timeout-and-goroutines-accumulation
//http://3.3.3.3/http?host=3.2.4.2&servername=a.test&path=/&port=33&timeout=5&scheme=http
func MonitorHttp(w http.ResponseWriter, r *http.Request) {
var host, servername, path, port, scheme string
var timeout int
u, err := nurl.Parse(r.RequestURI)
if err != nil {
log.Fatal(err)
return
}
if host = u.Query().Get("host"); host == "" {
host = "127.0.0.0"
}
if servername = u.Query().Get("servername"); servername == "" {
servername = "localhost"
}
if path = u.Query().Get("path"); path == "" {
path = "/"
}
if port = u.Query().Get("port"); port == "" {
port = "80"
}
if scheme = u.Query().Get("scheme"); scheme == "" {
scheme = "http"
}
if timeout, _ = strconv.Atoi(u.Query().Get("timeout")); timeout == 0 {
timeout = 5
}
//log.Printf("(host)=%s (servername)=%s (path)=%s (port)=%s (timeout)=%d", host, servername, path, port, timeout)
w.Header().Set("Content-Type", "application/json")
res := httptool(host, port, servername, scheme, path, timeout)
result, _ := json.Marshal(res)
fmt.Fprintf(w, "%s", result)
}
func httptool(ip, port, servername, scheme, path string, timeout int) Result {
var result Result
startTime := time.Now()
host := ip + ":" + port
transport := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
DisableKeepAlives: true,
}
dialer := net.Dialer{
Timeout: time.Duration(timeout) * time.Second,
KeepAlive: 0 * time.Second,
}
transport.Dial = func(network, address string) (net.Conn, error) {
return dialer.Dial(network, address)
}
client := &http.Client{
Transport: transport,
}
rawquery := ""
url := fmt.Sprintf("%s://%s%s%s", scheme, host, path, rawquery)
req, err := http.NewRequest("GET", url, nil)
if err != nil {
result.HttpStatus = -1
errs := strings.Split(err.Error(), ": ")
result.Error = errs[len(errs)-1]
result.Stime = time.Now().Sub(startTime) / time.Millisecond
return result
}
req.Header.Set("User-Agent", "monitor worker")
req.Header.Set("Connection", "close")
req.Host = servername
resp, err := client.Do(req)
//https://github.com/Basiclytics/neverdown/blob/master/check.go
if err != nil {
nerr, ok := err.(*nurl.Error)
if ok {
switch cerr := nerr.Err.(type) {
case *net.OpError:
switch cerr.Err.(type) {
case *net.DNSError:
errs := strings.Split(cerr.Error(), ": ")
result.Error = "dns: " + errs[len(errs)-1]
default:
errs := strings.Split(cerr.Error(), ": ")
result.Error = "server: " + errs[len(errs)-1]
}
default:
switch nerr.Err.Error() {
case "net/http: request canceled while waiting for connection":
errs := strings.Split(cerr.Error(), ": ")
result.Error = "timeout: " + errs[len(errs)-1]
default:
errs := strings.Split(cerr.Error(), ": ")
result.Error = "unknown: " + errs[len(errs)-1]
}
}
} else {
result.Error = "unknown: " + err.Error()
}
result.HttpStatus = -2
result.Stime = time.Now().Sub(startTime) / time.Millisecond
return result
}
resp.Body.Close()
result.HttpStatus = resp.StatusCode
result.Error = "noerror"
result.Stime = time.Now().Sub(startTime) / time.Millisecond //spend time (ms)
return result
}
func setRlimit() {
var rLimit syscall.Rlimit
err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &rLimit)
if err != nil {
log.Printf("Unable to obtain rLimit", err)
}
if rLimit.Cur < rLimit.Max {
rLimit.Max = 9999
rLimit.Cur = 9999
err = syscall.Setrlimit(syscall.RLIMIT_NOFILE, &rLimit)
if err != nil {
log.Printf("Unable to increase number of open files limit", err)
}
}
}
func main() {
setRlimit()
s := &http.Server{
Addr: ":59059",
ReadTimeout: 7 * time.Second,
WriteTimeout: 7 * time.Second,
}
http.HandleFunc("/http", MonitorHttp)
log.Fatal(s.ListenAndServe())
}
There are couple of points here.
I was unable to reproduce your behavior, anyway, can't identify protocol is usually tied to sockets not being properly closed.
Some commenters suggested you don't have to create http client inside each handler - that's true. Simply create it once and reuse.
Second, I'm not sure why are you creating your own http.Client struct and why you're disabling keepalives. Can't you just go with http.Get ? Simpler code is easier to debug.
Third, not sure why are you overwriting transport.Dial function. Even if you must do it, the documentation (for Go 1.9.2) says:
% go doc http.transport.dial
type Transport struct {
// Dial specifies the dial function for creating unencrypted TCP
connections.
//
// Deprecated: Use DialContext instead, which allows the transport
// to cancel dials as soon as they are no longer needed.
// If both are set, DialContext takes priority.
Dial func(network, addr string) (net.Conn, error)
That comment about deprecation and lack of dials reuse may point to the source of your problems.
To sum up, when in your shoes, I'd do two things:
move client creation to the code which executes once, or just use default client with http.Get
I'd clean up this thing with overwriting default transport fields, if you must do it then I'd use DialContext as suggested.
Good luck.
I couldn't reproduce the issue. But here are my 2 cents (no pun intended)
Simmilar issue was found in SockJS-node noticed in an article https://idea.popcount.org/2012-12-09-lsof-cant-identify-protocol/ according to this issue was observed on FreeBSD. But the issue was "websockets are not bieng properly cleaned up"
Another test test I would like ou to do if you still have hands on same environment. If possible post wireshark logs. just to confirm there are not subtle things in network frames which may have caused this.
I am sorry I cann't install Go 1.5.1 just to reproduce this issue.
Hope this was helpful.
I want to log a activity which starts in the scope of a different class and ends in the scope of another class. Using TrackingLogger.getCurrentActivity() throws an exception as there is no activity in memory.Now if I try to store the activity in static HashMap in singleton Class and retrieve it in the other class, the snapshot of activity is showing confusing values. Here is what I am trying to do :
In first Class :
TrackingLogger tracker = TrackingLogger.getInstance(this.getClass()) ;
TrackingActivity activity = tracker.newActivity(); //
activity.start();
CacheUtil.addTrackingActivity("logger",activity); // Store it in static Hashmap in singleton class
In Second Class :
TrackingLogger tracker = TrackingLogger.getInstance(this.getClass());
TrackingActivity activity = CacheUtil.getTrackingActivity("logger");
activity.stop();
tracker.tnt(activity);
this is the snapshot generated :
18:37:39,780 INFO [Nastel TNT4J] {status: 'END' | time: '2014-07-30 13:07:39.778000 UTC' | sev: 'INFO' | type: 'ACTIVITY' | name: 'NOOP' | usec: '72827000' | wait.usec: '21000' | start.time: '2014-07-30 18:36:26.196000 IST' | end.time: '2014-07-30 18:37:39.023000 IST' | pid: '8092' | tid: '90' | id-count: '0' | snap-count: '6' | source: 'APPL=Nastel TNT4J#JVM=8092#FACH13140035#SERVER=FACH13140035#NETADDR=172.25.19.28#DATACENTER=default#GEOADDR=unknown' | track-id: '09d708c4-a6a9-4200-a70f-25c1da838c11'
Snapshot(CPU#Java) {
Count: 4
TotalCpuUsec: 46800.3
TotalCpuUserUsec: 46800.3
}
Snapshot(Thread#Java) {
Count: 69
DaemonCount: 46
StartedCount: 85
PeakCount: 69
BlockedCount: 7
WaitedCount: 0
BlockedUsec: 21000
WaitUsec: 0
}
Snapshot(Memory#Java) {
MaxBytes: 532742144
TotalBytes: 512499712
FreeBytes: 216341712
UsedBytes: 296158000
Usage: 57
}
Snapshot(Copy#GarbageCollector) {
Count: 477
Time: 1940
isValid: true
}
Snapshot(MarkSweepCompact#GarbageCollector) {
Count: 13
Time: 4652
isValid: true
}
Snapshot(Activity#Java) {
TotalCpuUsec: -1778411.4
SlackUsec: 74584411
WallUsec: -1757411.4
BlockedCount: -24
WaitedCount: -1
BlockedUsec: 21000
WaitUsec: 0
OverheadUsec: 24290.215
}}
Am I missing something?? Pls help ..
TrackingLogger.getCurrentActivity() will only return a valid activity if executed within the same thread. The activity handle is stored in thread local. so you most likely calling TrackingLogger.getCurrentActivity() from a different thread and therefore get an exception.
I am not exactly clear where the confusing values are. Can you elaborate?