The microdot's discuss from gigya

Does it load balance?

Team,
Do you any documentation over Load balancing? If not, could you please give a high level flow.

Thanks

Error CS8002: Referenced assembly 'Gigya.ServiceContract, Version=2.5.5.0, Culture=neutral, PublicKeyToken=null' does not have a strong name.

Can you strongly type your assemblies?

Service Discovery: Per-method call and cache timeouts

Problem: There might be a need to change cache TTLs for specific methods in deployed services (e.g. production) via configuration, or to disable them outright. After modifying the Caching proxy to support TTL per method, there isn't any way to configure it.

Problem: Similarly, devs want to be able to define individual timeouts per service method. For example, methods that rely on DB will be allowed longer timeouts.

Proposed solution: Add a new configuration that allows changing both the request and caching TTLs and disabling the caching for any specific method. This configuration should reside close to service discovery configuration. This implies we could define per-client TTLs.

ServiceProxyProvider

looking at our stats looks like the network time is unjustified long, for example, a random query of one minute:

target.service	target.method	MAX(stats.network.time)
UserIdService	TryGetUserIds	665.7730102539062
PolicyService	GetEffectiveSitePolicy	132.62899780273438
SitesService	GetSite	230.61199951171875

other stats of the service doesn't indicate any issues so I was searching for some time-consuming logic between
RequestStartTimestamp marking and the actually httpClient.PostAsync and I have some concerns about this piece of code:

microdot/Gigya.Microdot.ServiceProxy/ServiceProxyProvider.cs

Lines 177 to 223 in 9bdf40c

 private (HttpClient httpClient, bool isHttps) GetHttpClient(ServiceDiscoveryConfig serviceConfig, DiscoveryConfig defaultConfig, bool tryHttps, string hostname, int basePort) 

 { 

 var forceHttps = serviceConfig.UseHttpsOverride ?? (ServiceInterfaceRequiresHttps || defaultConfig.UseHttpsOverride); 

 var useHttps = tryHttps || forceHttps; 

 string securityRole = serviceConfig.SecurityRole; 

 var verificationMode = serviceConfig.ServerCertificateVerification ?? 

 defaultConfig.ServerCertificateVerification; 

 var supplyClientCertificate = (serviceConfig.ClientCertificateVerification ?? defaultConfig.ClientCertificateVerification) 

 == ClientCertificateVerificationMode.VerifyIdenticalRootCertificate; 

 var httpKey = new HttpClientConfiguration(useHttps, securityRole, serviceConfig.RequestTimeout, verificationMode, supplyClientCertificate); 

 lock (HttpClientLock) 

 { 

 if (LastHttpClient != null && LastHttpClientKey.Equals(httpKey)) 

 return ( httpClient: LastHttpClient, isHttps: useHttps); 

 // In case we're trying HTTPs and the previous request on this instance was HTTP (or if this is the first request) 

 if (Not(forceHttps) && httpKey.UseHttps && Not(LastHttpClientKey?.UseHttps ?? false)) 

 { 

 var now = DateTime.Now; 

 if (now - _lastHttpsTestTime > _httpsTestInterval) 

 { 

 _lastHttpsTestTime = now; 

 RunHttpsAvailabilityTest(httpKey, hostname, basePort); 

 } 

 httpKey = new HttpClientConfiguration( 

 useHttps: false, 

 securityRole: null, 

 timeout:httpKey.Timeout, 

 verificationMode:httpKey.VerificationMode, 

 supplyClientCertificate: httpKey.SupplyClientCertificate); 

 } 

 if (!(LastHttpClientKey?.Equals(httpKey) ?? false)) 

 { 

 var messageHandler = _httpMessageHandlerFactory(httpKey); 

 var httpClient = CreateHttpClient(messageHandler, httpKey.Timeout); 

 LastHttpClient = httpClient; 

 LastHttpClientKey = httpKey; 

 _httpMessageHandler = messageHandler; 

 } 

 return (httpClient: LastHttpClient, isHttps: httpKey.UseHttps); 

 } 

 }

in case of ServerCertificateVerification or ClientCertificateVerification configured, this lock contains inside the certificate loading which can take time, and we are syncly blocking any attempts to call the service which probably the intention, but what happens to the thread during this lock?

microdot/Gigya.Microdot.ServiceProxy/HttpsAuthenticator.cs

Line 35 in 9bdf40c

clientCert = CertificateLocator.GetCertificate("Client");

microdot/Gigya.Microdot.SharedLogic/Security/WindowsStoreCertificateLocator.cs

Lines 79 to 80 in 9bdf40c

var store = new X509Store(storeName, storeLocation);

store.Open(OpenFlags.OpenExistingOnly | OpenFlags.ReadOnly);

is it possible that async cache refresh is "syncly" waiting for this lock?

microdot/Gigya.Microdot.ServiceProxy/Caching/AsyncCache.cs

Lines 300 to 314 in 9bdf40c

 existingItem.RefreshTask = ((Func<Task>)(async () => 

  { 

  try 

  { 

  var getNewValue = WrappedFactory(false); 

  await getNewValue.ConfigureAwait(false); 

  existingItem.CurrentValueTask = getNewValue; 

  existingItem.NextRefreshTime = DateTime.UtcNow + policy.RefreshTime; 

  MemoryCache.Set(new CacheItem(key, existingItem), policy); 

  } 

  catch 

  { 

  existingItem.NextRefreshTime = DateTime.UtcNow + policy.FailedRefreshDelay; 

  } 

  })).Invoke();

in case of an unreachable error or any type of 'HTTP request exception', we are repeating the process but with tryHttps=false, I'm not sure I completely understood the flow, this is what I see:

creating a client for HTTPS:

microdot/Gigya.Microdot.ServiceProxy/ServiceProxyProvider.cs

Line 194 in 9bdf40c

if (Not(forceHttps) && httpKey.UseHttps && Not(LastHttpClientKey?.UseHttps ?? false))

after the request is failing, creating a client for HTTP:

microdot/Gigya.Microdot.ServiceProxy/ServiceProxyProvider.cs

Line 462 in 9bdf40c

tryHttps = false;

microdot/Gigya.Microdot.ServiceProxy/ServiceProxyProvider.cs

Lines 211 to 214 in 9bdf40c

 if (!(LastHttpClientKey?.Equals(httpKey) ?? false)) 

 { 

 var messageHandler = _httpMessageHandlerFactory(httpKey); 

 var httpClient = CreateHttpClient(messageHandler, httpKey.Timeout);

in the next call, we are trying HTTPS again and creating a new client:

microdot/Gigya.Microdot.ServiceProxy/ServiceProxyProvider.cs

Line 374 in 9bdf40c

bool tryHttps = GetConfig().TryHttps ?? discoveryConfig.TryHttps;

if the time fits we are creating two more clients for the reachability checks

microdot/Gigya.Microdot.ServiceProxy/ServiceProxyProvider.cs

Lines 194 to 200 in 9bdf40c

 if (Not(forceHttps) && httpKey.UseHttps && Not(LastHttpClientKey?.UseHttps ?? false)) 

 { 

 var now = DateTime.Now; 

 if (now - _lastHttpsTestTime > _httpsTestInterval) 

 { 

 _lastHttpsTestTime = now; 

 RunHttpsAvailabilityTest(httpKey, hostname, basePort);

after the request is failing, creating a new client for HTTP, and so on and so on

what am I missing here?
are we performing every HTTP request twice?
are we creating two new HTTP clients for every request?
are we disposing all of these clients and handlers?
this happens every reachability check, plus to calls attempts?
what will happen when we are running parallel requests to the same service, even without the certificate, creating an HTTP client and handler is not the cheapest code, and do it each request and within a lock statement sounds expensive.

** see creating multiple HTTP clients effect

questions/discussions

Where is the best place to ask about microdot?

Should add more documentation about Service Discovery mechanism

Looks like it a bit confusing and not self-discoverable, see gigya/microdot-samples#2

Status on .NET Core

Hello!

I wanted to know whether you plan supporting .NET Core soon.

Thanks!

the best way

How to avoid hot grain? 10,000 request/s

Do you have a plan to add wiki pages?

As the title says.

[rl-reuse_tool-1] Violation against OSS Rules of Play

A violation against the OSS Rules of Play has been detected.

Rule ID: rl-reuse_tool-1
Explanation: Does README mention REUSE? No

Find more information at: https://sap.github.io/fosstars-rating-core/oss_rules_of_play_rating.html

Fix compiler warnings

Currently when compiling Microdot, the compiler give 26 warnings. These should be fixed or suppressed.

Replace copy-pasted DispatchProxy with official NuGet

The DispatchProxy class was copied from the .NET Core sources into the Gigya.Microdot.ServiceProxy project a long time ago, and is no longer synchronized with the latest release by Microsoft.

The official NuGet is available here:

https://www.nuget.org/packages/System.Reflection.DispatchProxy

Spelling Mistake in description

Kindly correct the word "Class" in the description:

.NET core porting

Hi,
We have an up coming requirement in doing MicroServices in .NET core 2.0. What is the time line by which the .NET core port will be available.

Thanks and regards
Venkatesh

Cannot build Microdot Solution

I downloaded source code of Microdot and I tried to compile solution.
I got exception while paket tries to download Gigya.OrleansDashboard.NetStandard 3.0.8.

Here is message:

Downloading Gigya.OrleansDashboard.NetStandard 3.0.8
Source 'http://nuget.gigya.net/nugetForVS/nuget' exception: System.Exception: Could not retrieve data from 'http://nuget.gigya.net/nugetForVS/nuget/FindPackagesById()?semVerLevel=2.0.0&id='Gigya.OrleansDashboard.NetStandard'' ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The remote name could not be resolved: 'nuget.gigya.net'
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
at [email protected](Exception _arg1)
at [email protected](AsyncParams1 args) --- End of inner exception stack trace --- at [email protected](Exception _arg3) at [email protected](AsyncParams1 args)
Something went wrong while downloading Gigya.OrleansDashboard.NetStandard 3.0.8
Message: Could not download Gigya.OrleansDashboard.NetStandard 3.0.8.

I was able to work with version 3.0.5. My currently downloaded version is 3.1.6.

[rl-assigned_teams-2] Violation against OSS Rules of Play

A violation against the OSS Rules of Play has been detected.

Rule ID: rl-assigned_teams-2
Explanation: Does it have an admin team on GitHub? No

Find more information at: https://sap.github.io/fosstars-rating-core/oss_rules_of_play_rating.html

About the value of ConsulEndPoint.HostName

the namespace Gigya.Microdot.ServiceDiscovery
class ConsulClient
line: 494
var endpoints = nodes.Select(ep => new ConsulEndPoint { HostName = ep.Node.Name, Port = ep.Service.Port, Version = GetEndpointVersion(ep) }).OrderBy(x => x.HostName).ThenBy(x => x.Port).ToArray();
why HostName = ep.Node.Name and not ep.Node.Address or ep.Service.Address?

[rl-assigned_teams-3] Violation against OSS Rules of Play

A violation against the OSS Rules of Play has been detected.

Rule ID: rl-assigned_teams-3
Explanation: Does it have enough admins on GitHub? No

Find more information at: https://sap.github.io/fosstars-rating-core/oss_rules_of_play_rating.html

how to use consul in microdot?

I have downloaded the GpuService sample, and then change some Environments, but got some error running GpuService.Client

// an empty file loadPaths.json
Environment.SetEnvironmentVariable("GIGYA_CONFIG_ROOT", @"D:\test\dotnet\microdot-samples\GpuService\");
Environment.SetEnvironmentVariable("ENV", "dev");
// consul
Environment.SetEnvironmentVariable("CONSUL", "127.0.0.1:8500");

consul service defination

{
  "service": {
    // can not find a way to change service name(default to GpuService-)
    "name": "GpuService-dev",
    "port": 10000
  }
}

errors

System.AggregateException: One or more errors occurred. ---> Gigya.Common.Contra
cts.Exceptions.EnvironmentException: Query not exists on Consul. See tags for de
tails.; requestedService=GpuService-dev, consulAddress=http://127.0.0.1:8500/, r
equestTime=0001-01-01 00:00:00.000, requestLog=, responseLog=, queryDefined=Fals
e, consulError=

is there any documents i can refer to?

how to publish on docker

like the title

Cannot build

I have tried to build the package with VS2015 and VS2017.
In both cases many errors.
What I am doing wrong?

Help with inventory sample

I am new to microdot. I am trying to run sample inventory service.
I managed it after a while. Now I am trying to run the inventory service client, but I got exception at startup. I have no idea why.
Here is what I got:
System.NullReferenceException
HResult=0x80004003
Message=Object reference not set to an instance of an object.
Source=Gigya.Microdot.ServiceProxy
StackTrace:
at Gigya.Microdot.ServiceProxy.ServiceProxyProvider.GetConfig()
at Gigya.Microdot.ServiceProxy.ServiceProxyProvider.d__80.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Gigya.Microdot.ServiceProxy.ServiceProxyProvider.d__79.MoveNext()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at InventoryService.Client.FortuneCookieTrader.d__10.MoveNext() in D:\Limos\Downloads\microdot-samples-master\microdot-samples-master\InventoryService\InventoryService.Client\FortuneCookieTrader.cs:line 31

This exception was originally thrown at this call stack:
[External Code]
InventoryService.Client.FortuneCookieTrader.Start() in FortuneCookieTrader.cs

	private (HttpClient httpClient, bool isHttps) GetHttpClient(ServiceDiscoveryConfig serviceConfig, DiscoveryConfig defaultConfig, bool tryHttps, string hostname, int basePort)
	{
	var forceHttps = serviceConfig.UseHttpsOverride ?? (ServiceInterfaceRequiresHttps \|\| defaultConfig.UseHttpsOverride);
	var useHttps = tryHttps \|\| forceHttps;
	string securityRole = serviceConfig.SecurityRole;
	var verificationMode = serviceConfig.ServerCertificateVerification ??
	defaultConfig.ServerCertificateVerification;
	var supplyClientCertificate = (serviceConfig.ClientCertificateVerification ?? defaultConfig.ClientCertificateVerification)
	== ClientCertificateVerificationMode.VerifyIdenticalRootCertificate;
	var httpKey = new HttpClientConfiguration(useHttps, securityRole, serviceConfig.RequestTimeout, verificationMode, supplyClientCertificate);

	lock (HttpClientLock)
	{
	if (LastHttpClient != null && LastHttpClientKey.Equals(httpKey))
	return ( httpClient: LastHttpClient, isHttps: useHttps);

	// In case we're trying HTTPs and the previous request on this instance was HTTP (or if this is the first request)
	if (Not(forceHttps) && httpKey.UseHttps && Not(LastHttpClientKey?.UseHttps ?? false))
	{
	var now = DateTime.Now;
	if (now - _lastHttpsTestTime > _httpsTestInterval)
	{
	_lastHttpsTestTime = now;
	RunHttpsAvailabilityTest(httpKey, hostname, basePort);
	}

	httpKey = new HttpClientConfiguration(
	useHttps: false,
	securityRole: null,
	timeout:httpKey.Timeout,
	verificationMode:httpKey.VerificationMode,
	supplyClientCertificate: httpKey.SupplyClientCertificate);
	}

	if (!(LastHttpClientKey?.Equals(httpKey) ?? false))
	{
	var messageHandler = _httpMessageHandlerFactory(httpKey);
	var httpClient = CreateHttpClient(messageHandler, httpKey.Timeout);

	LastHttpClient = httpClient;
	LastHttpClientKey = httpKey;
	_httpMessageHandler = messageHandler;
	}

	return (httpClient: LastHttpClient, isHttps: httpKey.UseHttps);
	}
	}

	var store = new X509Store(storeName, storeLocation);
	store.Open(OpenFlags.OpenExistingOnly \| OpenFlags.ReadOnly);

	existingItem.RefreshTask = ((Func<Task>)(async () =>
	{
	try
	{
	var getNewValue = WrappedFactory(false);
	await getNewValue.ConfigureAwait(false);
	existingItem.CurrentValueTask = getNewValue;
	existingItem.NextRefreshTime = DateTime.UtcNow + policy.RefreshTime;
	MemoryCache.Set(new CacheItem(key, existingItem), policy);
	}
	catch
	{
	existingItem.NextRefreshTime = DateTime.UtcNow + policy.FailedRefreshDelay;
	}
	})).Invoke();

gigya / microdot Goto Github PK

microdot's Issues

Recommend Projects

Recommend Topics

Recommend Org