OWASP corerulet warning "invalid http request line" triggered by CONNECT method - owasp

Summary:
I have setup a basic WAF with mod-security and the OWASP coreruleset 3.3.2. When using the WAF I see lots of warnings in modsec_audit.log regarding the CONNECT method, which trigger crs rule 920100:
Message: Warning. Match of "rx
^(?i:(?:[a-z]{3,10}\s+(?:\w{3,7}?://[\w\-\./](?::\d+)?)?/[^?#](?:\?[^#\s])?(?:#[\S])?|connect
(?:\d{1,3}\.){3}\d{1,3}\.?(?::\d+)?|options
\)\s+[\w\./]+|get /[^?#](?:\?[^#\s])?(?:#[\S])?)$" against
"REQUEST_LINE" required. [file
"/etc/httpd/modsecurity.d/owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"]
[line "63"] [id "920100"] [msg "Invalid HTTP Request Line"] [data
"CONNECT oneofmy.longer.hostname.here.abcde.com:443 HTTP/1.1"]
[severity "WARNING"] [ver "OWASP_CRS/3.3.2"] [tag "application-multi"]
[tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"]
[tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/210/272"]
Details regarding setup:
I set up the WAF with mod_security 2.9.2 and httpd. I used the crs-setup.conf.example provided by crs and only modified these two settings regarding threshhold and allowing the CONNECT http method. (CONNECT method is used in our setup for proxy purposes).
SecAction \
"id:900110,\
phase:1,\
nolog,\
pass,\
t:none,\
setvar:tx.inbound_anomaly_score_threshold=5,\
setvar:tx.outbound_anomaly_score_threshold=4"
SecAction \
"id:900200,\
phase:1,\
nolog,\
pass,\
t:none,\
setvar:'tx.allowed_methods=GET HEAD POST OPTIONS CONNECT'"
What I have tried so far:
Google a lot about crs and mod-security rules. As per my current, admittedly limited mod-security/crs understanding, basically the regex of the rule 920100 does not match against the request line CONNECT oneofmy.longer.hostname.here.abcde.com:443 HTTP/1.1
Tried to find more details how the regex for 920100 is constructed.
looked up how the request line for a CONNECT string is defined in RFC. To me, the request line looks fine as per the RFC.
tried to debug the regexp with a site like regex101 and my request line. I did not succeed in debugging.

Looking at the part of the regular expression for rule 920100 that deals with the CONNECT method:
connect (?:\d{1,3}\.){3}\d{1,3}\.?(?::\d+)?
we can see that it's looking for an IPv4 address. For example, it would match on (and hence not raise an alert for) a request line like the following:
CONNECT 1.2.3.4:80 HTTP/1.1
As you say, the RFC seems to allow for domain names to be used with the CONNECT method, however, rule 920100 is not currently written to allow for this. This seems to be the cause of your issue.
With further investigation and testing, the rule can probably be expanded in a future CRS release to allow for domain names in CONNECT requests.
What can you do here and now?
To address this issue right now, you can add a rule exclusion to your Apache configuration. There are two ways you can do this:
Remove the offending rule completely by excluding it at configure time, like so:
#
# -- CRS Rule Exclusion: 920100 - Invalid HTTP Request Line
#
# Turn off rule due to issues with CONNECT requests
#
SecRuleRemoveById 920100
As a configure time rule exclusion, this should be placed after the inclusion of the Core Rule Set in your Apache configuration (because the rule that's being removed must be added/included in the first place before it can then be removed!).
Remove the offending rule conditionally, only for CONNECT requests, by excluding it at runtime, like so:
#
# -- CRS Rule Exclusion: 920100 - Invalid HTTP Request Line
#
# Turn off rule for CONNECT requests, which cause issues
#
SecRule REQUEST_METHOD "#streq CONNECT" \
"id:1000,\
phase:1,\
pass,\
nolog,\
ctl:ruleRemoveById=920100"
As a runtime time rule exclusion, this should be placed before the inclusion of the CRS in your Apache configuration (because the rule that's being modified must be modified before it fires off during execution).
A small word of warning
I can't find any mention of anyone else using ModSecurity with the CRS in this way, i.e. fielding CONNECT requests. The CRS rules aren't written to accommodate having a domain name in the request line, so you may find that you run into further false positives as a result. You would need to deal with those in a similar way to that described above.
(You can find an excellent tutorial on the subject of writing rule exclusions here: https://www.netnea.com/cms/apache-tutorial-8_handling-false-positives-modsecurity-core-rule-set/).
Hopefully this helps.

Related

ModSecurity not putting some transactions to audit log

We are facing a problem when in certain cases ModSecurity is not tracking the blocking in the audit log despite we have set it as a default action and the rule is not having any logging property set. In the error log, we can see only the rule which was triggered due to the anomaly score has been reached but nothing about the rules which actually counted the score. In some other cases, this information is visible.
In modsecurity.conf we have logging of rules enabled
SecAuditLogParts ABCIJDFHKZ
In crs-setup.conf we have default logging set to capture everything in both error and audit log.
SecDefaultAction "phase:1,log,auditlog,pass"
SecDefaultAction "phase:2,log,auditlog,pass"
We have also tried other combinations like using this setup
SecDefaultAction "phase:2,pass, nolog,auditlog"
But notning helped.
The only thing we can see in the log is this part from the error log and when we need more details we need to enable debug and redo the action. In the example below I have changed the IP addresses for security reasons.
2022/03/09 08:32:25 [error] 1149#1149: *13832 [client 1.1.1.1] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `15' ) [file "/etc/nginx/owasp-modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "139"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 15)"] [data ""] [severity "2"] [ver "OWASP_CRS/3.4.0-dev"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "10.1.0.1"] [uri "/favicon.ico"] [unique_id "1646811145"] [ref ""], client: 1.1.1.1, server: example.com, request: "GET /favicon.ico HTTP/1.1", host: "example.com", referrer: "http://example.com/?q=%22%3E%3Cscript%3Ealert(1)%3C/script%3E%27"
We need to find a way how to make sure every attempt will be logged into the audit log.
Thanks in advance for any help.
Peter
It is a bug that is presented in ModSecurity 3.0.6 when used along with Nginx.
The resolution is either not to use custom error_page in Nginx configuration or to recompile the current solution with this fix https://github.com/SpiderLabs/ModSecurity-nginx/pull/273

Using custom REQUEST_HEADER on whitelist action

I'm trying to use a custom request header on a ModSecurity whitelist rule so I can remove specific rules.
SecRule REQUEST_URI "^/admin/index\.html$" \
"id:1004,\
phase:2,\
pass,\
nolog,\
ctl:ruleRemoveById=%{request_headers.x-rules-excluded}"
However, when I try to do it I get the following error on NGINX:
Mar 20 16:18:42 develop-node ip[16035]: nginx: [emerg] "modsecurity_rules_file" directive Rules error. File: /tmp/test.modsec.conf. Line: 1062. Column: 96. Expecting an action, got: ctl:ruleRemoveById=%{request_headers.x-rules-excluded}" in /tmp/test.nginx.conf:28
I wonder if this is even possible to do...
Thanks!
It is not possible. You are trying to perform macro expansion, and it is not supported in ctl actions. You can only pass an integer or a range in ctl:ruleRemoveById.

Overriding CORS on Kubernetes leads to a connection error

I am trying a test where I enable CORS for all domains (to be restricted after I get the use-case working correctly).
As far as I can tell, I should be adding --cors-allowed-origins=["http://*"] to my kube-apiserver.manifest
When I try to do that however:
spec:
containers:
- command:
- /bin/sh
- -c
- /usr/local/bin/kube-apiserver --address=127.0.0.1 --admission-control=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,Priority,ResourceQuota
--allow-privileged=true --anonymous-auth=false --apiserver-count=1 --authorization-mode=AlwaysAllow
--basic-auth-file=/srv/kubernetes/basic_auth.csv --client-ca-file=/srv/kubernetes/ca.crt
--cloud-provider=aws --etcd-servers-overrides=/events#http://127.0.0.1:4002
--etcd-servers=http://127.0.0.1:4001 --insecure-port=8080 --kubelet-preferred-address-types=InternalIP,Hostname,ExternalIP
--proxy-client-cert-file=/srv/kubernetes/apiserver-aggregator.cert --proxy-client-key-file=/srv/kubernetes/apiserver-aggregator.key
--requestheader-allowed-names=aggregator --requestheader-client-ca-file=/srv/kubernetes/apiserver-aggregator-ca.cert
--requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group
--requestheader-username-headers=X-Remote-User --secure-port=443 --service-cluster-ip-range=100.64.0.0/13
--storage-backend=etcd2 --tls-cert-file=/srv/kubernetes/server.cert --tls-private-key-file=/srv/kubernetes/server.key
--token-auth-file=/srv/kubernetes/known_tokens.csv --v=2 2>&1 | /bin/tee -a
--cors-allowed-origins=["https://*"]
I get the following error when trying to use kubectl get pods:
The connection to the server 127.0.0.1 was refused - did you specify the right host or port?
What exactly am I doing wrong for the setup? How do I add CORS domains to Kubernetes?
EDIT: I am now trying this:
- kube-apiserver
--cors-allowed-origins=["https://*"]
My pods no longer crash, however I still have CORS issues in my application.
I am getting errors like this:
[Error] Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin.
[Error] XMLHttpRequest cannot load https://example.com/auth/realms/name/protocol/openid-connect/token due to access control checks.
[Error] Failed to load resource: Origin http://localhost:8080 is not allowed by Access-Control-Allow-Origin. (token, line 0)
[Error] Preflight response is not successful
[Error] XMLHttpRequest cannot load https://example.com/api/v1/users/me/profile? due to access control checks.
Looks like you are trying to access your api-server with the http protocol, try:
--cors-allowed-origins=["http://*"]
As per documentation:
--cors-allowed-origins: List of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled.
I have tested this settings (and also http//*) against API and it works:
--cors-allowed-origins=example.com,example2.com
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, X-Requested-With, If-Modified-Since
Access-Control-Allow-Methods: POST, GET, OPTIONS, PUT, DELETE, PATCH
Access-Control-Allow-Origin: http://example.com
Those errors looks like problem with settings on your serverside app.
Access-Control-Allow-Origin: specifies the authorized domains to make cross-domain request (you should include the domains of your REST clients or “*” if you want the resource public and available to everyone – the latter is not an option if credentials are allowed during CORS requests)
hope this help.

How to disable a specific mod_security rule in a specific directory?

I want to disable this rule:
[file "/usr/share/modsecurity-crs/activated_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "159"] [id "981173"] [rev "2"] [msg "Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded"] [data "Matched Data: - found within ARGS:customize_changeset_uuid: a507417f-75f3-434e-ac8c-90b21b3b164d"] [ver "OWASP_CRS/2.2.9"] [maturity "9"] [accuracy "8"] [tag "OWASP_CRS/WEB_ATTACK/SQL_INJECTION"]
For the directory
/var/www/romanpastu/wp-admin
Where should the exception go? In my .htaccess? or the vhost file?
Cause nothing seems to be working.
I'm using apache2
I've tried adding the following in my virtualhost file. However, its still not working
<LocationMatch "/wp-admin/update.php">
<IfModule security2_module>
SecRuleRemoveById 981173
</IfModule>
</LocationMatch>
<LocationMatch "/wp-admin/customize.php">
<IfModule security2_module>
SecRuleRemoveById 981173
</IfModule>
</LocationMatch>
It should be added after the rule is defined in your config. So if you are defining your vhost (including removing above rule) and then load your ModSecurity rules later on in your config then that will not work - it needs to be the other way around.
Also, unless ModSecurity was compiled with the --enable-htaccess-config setting (which is not by default), then you cannot alter ModSecurity rules in htaccess files.
Additionally putting rules within LocationMatch can cause issues. And also wp-admin is exactly the sort of page you want to use ModSecuritg to protect against so carefully consider if you really want to turn off these rules. Please see also my answer to this question for some more information: Apache LocationMatch wildcard for ModSecurity on wordpress site.
So I would suggest using the following instead to only turn this rule off for this argument that's causing you issues:
SecRuleUpdateTargetById 981173 !ARGS:'customize_changeset_uuid'
Note you may need to add similar exceptions if any other arguments cause problems.

JENKINS Authentication Fails

I am getting the following error while trying to trigger Jenkins job from any REST Client
Authentication required
<!-- You are authenticated as: anonymous
Groups that you are in:
Permission you need to have (but didn't):
hudson.model.Hudson.Read
... which is implied by: hudson.security.Permission.GenericRead
... which is implied by: hudson.model.Hudson.Administer
-->
</body> </html>
The request is getting triggered while using curl from terminal
I am using the following syntax
http://user:apiToken#jenkins.yourcompany.com/job/your_job/build?token=TOKEN
[ref :https://wiki.jenkins-ci.org/display/JENKINS/Authenticating+scripted+clients]
ie. curl -X POST http://user:apiToken#jenkins.yourcompany.com/job/your_job/build?token=TOKEN
Check this "This build is parameterized " , select the credentials parameter from drop down.
Use this
curl -X POST http://jenkins.rtcamp.com/job/Snapbox/buildWithParameters --user "username:password"
It solved my authentication problem.
I hope it will help others too.
My development team's configuration settings were matrix-based security so I had to find my group and give my group workspace access.
1.Click on Manage Jenkins .
2.Click on Configure Global Security .
3.in matrix-based security change:
Overall - Read
Job - Build
Job - Read
Job - Workspace
Then
POST jobUrl/buildWithParameters HTTP/1.1
Host: user:token
Authorization: Basic dWdlbmxpazo4elhjdmJuTQ==
Cache-Control: no-cache
Content-Type: application/x-www-form-urlencoded
Branch=develop
For me
https://user:password#jenkins.mycompany.org/job/job_name/build?token=my_token
in https://jenkins.mycompany.org/configureSecurity
disable CORS
hope this help
Try using the -u parameter to specify the credentials:
curl -u user:apiToken -X POST http://jenkins.yourcompany.com/job/your_job/build?token=TOKEN
I provided header Authorization parameter with value :
BASIC base_64encoded(username:password) and it worked fine.
Authorization Basic bmltbWljdjpqZX*********
Simply disable "CSRF Protection" in the global Security Options, because those URLs don't send post data identification.
focal point :
username:password#
curl -u user:apiToken -X POST http://username:password#jenkins.yourcompany.com/job/your_job/build?key1=value1&key2=value2 ...
If you are encountering this problem with jenkins api client in ruby.
I figured Jenkins is blocking all the get request, instead use api_post_request.
Also, you have to generate api token because normal password is not working anymore.
#client = JenkinsApi::Client.new(
server_url: "",
username: '',
password: ""
)
SITE_FILE_PATH = 'artifact/target/site'.freeze
#jenkins_uri=''
#jenkins_job_name=''
def latest_test_file_path
"/job/#{#jenkins_job_name}/job/master/lastSuccessfulBuild/#{SITE_FILE_PATH}/Test-Results/test-results.html"
end
puts #client.api_post_request(latest_test_file_path,{},true).body
you can set the parameter true if you want the raw response.
default parameter or passing false will just return response code.
Also make sure to construct the right prefix.
You can refer to the above snipped.