Strange behavior of AEM Dispatcher: - aem

Question

We are using AEM 6.2 with Dispatcher 4.2 and after each page refresh the cache is not considered for response.
There are Secure content which are secured by /auth_checker and cached by /allowAuthorized.
The URL of content is as follows,
https://www.securedcontent.com/content/sample.html
On load of above page, it requests via Ajax following request.
https://www.securedcontent.com/content/sample/some.query.json/sample/input.json
Both above URL result gets cached by dispatcher.
But if the page is refreshed (sample.html), only page html (sample.html) will be fetched from cache. input.json will not be fetched from cache but from publish server.
This is consistent and can be reproduced with another Ajax call to another URL like following.
https://www.securedcontent.com/content/sample/some.query.json/sample/another/input.json
But subsequent request to (input.json) would fetch from cache, until the page (sample.html) is refreshed.
Details of our setup is as follows:
Dispatcher configuration:
/aemsecuredcontent
{
/auth_checker
{
/url "/bin/permissioncheck"
/filter
{
/0000
{
/glob "*"
/type "deny"
}
/0001
{
/glob "/content/securedcontent/*"
/type "allow"
}
}
/headers
{
/0000
{
/glob "*"
/type "deny"
}
/0001
{
/glob "Set-Cookie:*"
/type "allow"
}
}
}
/clientheaders{ "*" }
/filter
{
/2401 { /type "deny" /glob "*" }
/2417 { /type "allow" /glob "*query.json*" }
}
/cache
{
/statfileslevel "1"
/allowAuthorized "1"
/rules
{
/0000
{
/type "allow" /glob "*"
}
/0001
{
/glob "*.nocache.html*"
/type "deny"
}
}
/invalidate
{
/0000
{
/glob "*"
/type "deny"
}
/0001
{
/glob "*.html"
/type "allow"
}
}
}
}
Request Header of URLs:
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cache-Control: no-cache
Connection: keep-alive
Cookie: WT_FPC=id=39823rjfngfeihe9jwe; JSESSIONID=sjerht93qu4fg; login-token=48039qi4ig4joacrx.default
Host: www.securedcontent.com
Pragma: no-cache
Referer: https://www.securedcontent.com/content/sample.query.json/sample/input.json
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36
X-Requested-With: XMLHttpRequest
Response Header of URLs:
Accept-Ranges: bytes
Cache-Control: max-age=2592000
Connection: Keep-Alive
Content-Encoding: gzip
Content-Length: 8303
Content-Type: application/json
Date: Thu, 24 May 2018 06:53:12 GMT
Expires: Sat, 23 Jun 2018 06:53:12 GMT
Keep-Alive: timeout=5, max=98
Proxy-Server: something-proxywcm
Server: Apache
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Accept-Encoding
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-UA-Compatible: IE=edge
X-XSS-Protection: 1; mode=block
Dispatcher Log after every page refresh (not from cache):
[D] [pid 23365 (tid 139886375950080)] checking [/cache-dir/content/sample.query.json/sample/input.json]
[D] [pid 23365 (tid 139886375950080)] cachefile does not exist: /cache-dir/content/sample.query.json/sample/input.json
[D] [pid 23365 (tid 139886375950080)] try to create new cachefile: /cache-dir/content/sample.query.json/sample/input.json
[D] [pid 23365 (tid 139886375950080)] cache-action for [/cache-dir/content/sample.query.json/sample/input.json]: CREATE
Dispatcher Log of second Ajax request of the same URL (cached response):
[D] [pid 23365 (tid 139886302521088)] checking [/cache-dir/content/sample.query.json/sample/input.json]
[D] [pid 23365 (tid 139886302521088)] Authorization checker: connected to backend rend01
[D] [pid 23365 (tid 139886302521088)] cache-action for [/cache-dir/content/sample.query.json/sample/input.json]
[I] [pid 23365 (tid 139886302521088)] "GET /cache-dir/content/sample.query.json/sample/input.json" - -