Question
I'm running into a strange issue with SSL certs not being trusted when running in IIS from the Admin and Admin API applications. I'm using locally generated certificates from mkcert
which are fully trusted by my browser. I have the root certificate installed and things seem fine when examining from the browser side. However, when I attempt to hit the Admin site, I get exceptions and errors because of SSL connection issues with the security token service.
System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
Checking the health checks returns Unhealthy
as a result: https://admin.identity.development.com/health
{"status":"Unhealthy","totalDuration":"00:00:00.1316345","entries":{"ConfigurationDbContext":{"data":{},"duration":"00:00:00.0351978","status":"Healthy","tags":[]},"PersistedGrantsDbContext":{"data":{},"duration":"00:00:00.0349855","status":"Healthy","tags":[]},"IdentityDbContext":{"data":{},"duration":"00:00:00.0350268","status":"Healthy","tags":[]},"LogDbContext":{"data":{},"duration":"00:00:00.0352917","status":"Healthy","tags":[]},"AuditLogDbContext":{"data":{},"duration":"00:00:00.0351420","status":"Healthy","tags":[]},"DataProtectionDbContext":{"data":{},"duration":"00:00:00.0352220","status":"Healthy","tags":[]},"Identity Server":{"data":{},"description":"The SSL connection could not be established, see inner exception.","duration":"00:00:00.0749101","exception":"The SSL connection could not be established, see inner exception.","status":"Unhealthy","tags":[]},"ConfigurationDb":{"data":{},"duration":"00:00:00.0199259","status":"Healthy","tags":[]},"PersistentGrantsDb":{"data":{},"duration":"00:00:00.0199690","status":"Healthy","tags":[]},"IdentityDb":{"data":{},"duration":"00:00:00.0199584","status":"Healthy","tags":[]},"LogDb":{"data":{},"duration":"00:00:00.0199091","status":"Healthy","tags":[]},"AuditLogDb":{"data":{},"duration":"00:00:00.0144659","status":"Healthy","tags":[]},"DataProtectionDb":{"data":{},"duration":"00:00:00.0144428","status":"Healthy","tags":[]}}}
I tried to regenerate the certificate multiple times, including multiple machine restarts but nothing seems to resolve that issue and I'm not sure what to try next. I created a small application to use HttpClient
to see if I can fetch the openid-configuration document manually and that works fine without any SSL errors.
Oddly it also works fine if I run the Admin client in IIS Express (from within Visual Studio) https://localhost:44303/health
{"status":"Healthy","totalDuration":"00:00:00.1320430","entries":{"ConfigurationDbContext":{"data":{},"duration":"00:00:00.0372028","status":"Healthy","tags":[]},"PersistedGrantsDbContext":{"data":{},"duration":"00:00:00.0372451","status":"Healthy","tags":[]},"IdentityDbContext":{"data":{},"duration":"00:00:00.0438421","status":"Healthy","tags":[]},"LogDbContext":{"data":{},"duration":"00:00:00.0373719","status":"Healthy","tags":[]},"AuditLogDbContext":{"data":{},"duration":"00:00:00.0309596","status":"Healthy","tags":[]},"DataProtectionDbContext":{"data":{},"duration":"00:00:00.0394936","status":"Healthy","tags":[]},"Identity Server":{"data":{},"duration":"00:00:00.0820682","status":"Healthy","tags":[]},"ConfigurationDb":{"data":{},"duration":"00:00:00.0285621","status":"Healthy","tags":[]},"PersistentGrantsDb":{"data":{},"duration":"00:00:00.0340030","status":"Healthy","tags":[]},"IdentityDb":{"data":{},"duration":"00:00:00.0256599","status":"Healthy","tags":[]},"LogDb":{"data":{},"duration":"00:00:00.0094582","status":"Healthy","tags":[]},"AuditLogDb":{"data":{},"duration":"00:00:00.0095660","status":"Healthy","tags":[]},"DataProtectionDb":{"data":{},"duration":"00:00:00.0039096","status":"Healthy","tags":[]}}}
I'm not sure why it would operate any differently when hitting the discovery document from IIS vs IIS Express or just a regular HttpClient
. Is this something that you've run into before? I'm not sure where to go next and hoping this is just some configuration issue that I may have missed.
Let me know what you think. Appreciate any suggestions you may have.
Thanks!
-Rondel
Relevant parts of the log file
2022-02-23 07:52:38.342 -05:00 [INF] Executing endpoint 'Health checks'
2022-02-23 07:52:38.343 -05:00 [INF] Start processing HTTP request GET "https://identity.development.com/.well-known/openid-configuration"
2022-02-23 07:52:38.343 -05:00 [INF] Sending HTTP request GET "https://identity.development.com/.well-known/openid-configuration"
...
2022-02-23 07:52:38.351 -05:00 [ERR] Health check Identity Server with status "Unhealthy" completed after 8.6405ms with message 'null'
System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid because of errors in the certificate chain: UntrustedRoot
at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
at System.Net.Security.SslStream.CompleteHandshake(SslAuthenticationOptions sslAuthenticationOptions)
at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at HealthChecks.IdSvr.IdSvrHealthCheck.CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken)
2022-02-23 07:52:38.354 -05:00 [INF] Executed DbCommand (1ms) [Parameters=[], CommandType='"Text"', CommandTimeout='30']
SELECT 1
2022-02-23 07:52:38.356 -05:00 [INF] Executed endpoint 'Health checks'
2022-02-23 07:52:38.357 -05:00 [INF] Request finished HTTP/2 GET https://admin.identity.development.com/health - - - 503 - application/json 14.9348ms