Custom STS SSO failing - single-sign-on

Question

I'm working on a roll-your-own Federation implementation. There are two RPs. SSO between the RPs does not work (erroneously). I suspect it has to do with the cookie that the STS is creating. The STS is writing a fedauth cookie for itself. From my understanding, it should be writing a Forms Authentication cookie?
When hitting the STS for the second time from the other RP I can see in the ClaimsPrincipal that IsAuthenticated=True, yet the user is prompted to login and not automatically redirected back to the RP.
It's worth noting that SSO did work previously, auto redirect and all, but the RPs on the load balancer couldn't share cookies as it was using the machine key (and no sticky sessions). I fixed this by implementing a custom SessionSecurityTokenHandler that utilizes the certificate (code below). It's at this point that the STS started writing FedAuth cookies and SSO started failing.
The sts token is being written with:
FederatedAuthentication.SessionAuthenticationModule.WriteSessionTokenToCookie(token);
Token handler:
var sessionTransforms = new List<CookieTransform>(new CookieTransform[]
{
new DeflateCookieTransform(),
new RsaEncryptionCookieTransform(federationConfiguration.ServiceCertificate),
new RsaSignatureCookieTransform(federationConfiguration.ServiceCertificate)
});
var sessionHandler = new SessionSecurityTokenHandler(sessionTransforms.AsReadOnly());
federationConfiguration.IdentityConfiguration.SecurityTokenHandlers.AddOrReplace(sessionHandler);

Answer 1

The STS writes its own cookie. It POSTs the security token to your application. Your application will typically respond by writing a session authentication cookie, which it will use until it expires (and then it goes back to the STS)
If you work in a web farm then there is an out of the box support for this using WIF configuration :
Of course, the machines in the web farm should then share the same machine key. You can of course use your own mechanism but that seldom makes sense.
Next, each RP should therefore write its own "session" cookie that proves your authentication. If two RP's live in the same domain then they should use a different cookie name.