SharePoint Search REST API with bearer token returns wrong number of items - rest

Question

I have an EXTREMELY strange issue with the search.
I am doing a query by using a GET on
https://sonar-sandbox.gredspdev.loc/_api/search/query?querytext='DMSSonarDocId:5042aa1f-b3a4-4577-8e21-8a47ca27c243 OR DMSSonarDocId:1401144b-bd3d-429a-a386-5061ecc714e1'&sourceid='a0f4d450-e701-4f2a-888a-8d871002752d'&trimduplicates=false&rankingmodelid='05289DBE-73E9-4665-BF69-EE68274176EB'&rowlimit=9000&enablestemming=false&enablesorting=false&selectproperties='DMSSonarDocId,<...>'
I am authenticating using a bearer token generated for my user. This query returns 7 items. Then I am executing THE SAME URL in my browser with my user (NTLM) and it returns 10 items. That is not all. I generate the token for my user one more time. Paste it to the previous GET request with a bearer token and it returns 10 items... I am waiting few seconds, lets say 30... GET one more time and I have 7 items returned (always the same)! And this is 100% replicable. After another GET from the browser and regeneration of the token 10 items, after some time on the same token 7 items....
Update. I have found difference in logs in ULS:
When working correct:
Context has no SMTP/UPN claims. IdentityContext: '{"nameid":"s-1-5-21-2843295230-2675739751-2774624307-1482","nii":"urn:office:idp:activedirectory","upn":"kowalj#spdev.loc","userId":"0#.w|spdev\\kowalj","appliesTo":"https:\/\/sonar-sandbox.spdev.loc\/"}'
When not working correct:
Context has no SMTP/UPN claims. IdentityContext: '{"nameid":"s-1-5-21-2843295230-2675739751-2774624307-1482","nii":"urn:office:idp:activedirectory","upn":"spdev\\kowalj","userId":"0#.w|spdev\\kowalj","appliesTo":"https:\/\/sonar-sandbox.spdev.loc\/"}'
ANOTHER FINDINGS:
Missing items are those which are assigned to me directly - not through group resolved by our custom claims provider - yes, we have a custom claims provider which worked ok for a long time (we were using only NTLM authorization).
We are sending those claims:
new Claim[]
{
new Claim("nameid", sid),
new Claim("nii", Constants.Auth.Token.IdentityIssuer)
};
ANOTHER FINDINGS:
When everything work correctly, executing this code in the SP farm solution in some REST proxy: ((ClaimsIdentity)HttpContext.Current.User?.Identity).Claims.FirstOrDefault(c => c.ClaimType.EqualsIgnoreCase(ClaimTypes.Upn)) returns upn.
When the search is not working, the same code returns null... And as I said, I can refresh the page and at the beginning the upn is there and after some time it is not...

Answer 1

I have found a work around. Not very good but I do not see any other option for now.
We have started Claims to Windows Token Service and if user does some requests to our app, we do (from time to time) requests to our custom proxy placed in the SharePoint farm solution to simulate using SharePoint by that user by using normal windows authentication:
public void RefreshUpn()
{
WindowsImpersonationContext _wic = null;
try
{
string login = HttpContext.Current.User.Identity.Name;
login = login.Substring(login.LastIndexOf('|') + 1);
string[] loginParts = login.Split('\\');
string loginForUpnLogon = Culture.Invariant($"{loginParts[1]}#{loginParts[0]}");
WindowsIdentity wi = S4UClient.UpnLogon(loginForUpnLogon);
if(wi == null)
{
throw new InvalidOperationException(Culture.Invariant($"Could not impersonate user '{HttpContext.Current.User.Identity.Name}'."));
}
_wic = wi.Impersonate();
using (var wc = new WebClient())
{
wc.UseDefaultCredentials = true;
var requestUrl = HttpContext.Current.Request.Url;
wc.DownloadString(requestUrl.Scheme + "://" + requestUrl.Host + "/_api/web/currentuser");
}
}
finally
{
_wic?.Undo();
}
}
After such request, SharePoint responses to us correctly for around 150 seconds.