FormatException and OutOfRangeException in the constructor about coordinatesharp HOT 35 CLOSED

This happens in a 4.6 console project where I tried the library. Right now I don't actually need celestial calculations so it's not a critical issue so the workaround is my working solution, but I will try to reproduce it again with the other branch as soon as i can and report here.

from coordinatesharp.

Ok, so the good news is that Next Games hasn't seen the issue present. With their user base that's a good indicator that the issue is most likely Unit Test related, which puts my mind at ease for people currently using the library in production environments at least.

from coordinatesharp.

This may be a problem in my xUnit tests. I rewrote them today and they all pass reliably. I'm going to try to reproduce a minimal failing tests and see what's going on.

from coordinatesharp.

@beppe9000 @NickKellett @beppe9000 Thank you everyone I finally isolated the issue (although maybe someone could give me an explanation as to why).

The heavy load was they key, which explains why unit tests were failing, but no one could replicate. I threw a boatload of requests at the test project @njbrown09 made and was able to determine that the values list was being modified by other web requests that were NOT from the originating request.

The values list is global (if that's the right word) to the SolarEclipseClass for no reason (a left over from the NASA code port).

By removing its declaration here

And changing the values list creation from.

to

List<string> values = new List<string>();

The server seems to stop being confused on which list it's updating for which call.

I am not sure why it's the values list specifically as other variables like this exist, but I can't seem to get it to throw any more exceptions.

@njbrown09 I updated the Development Branch with the change. Would you be able to download it and test on your end. I would like to get multiple eyes on as your test is different then mine.

from coordinatesharp.

Thank you for providing the thread information for me.

There is a call in the Eclipse calculations that needs to convert a string back to a DateTime. I am perplexed as to why it's causing this issue and I can't seem to replicate though. Could you do me a favor and provide me with a DateTime string formatted in your current culture or let me know what culture you are using. I want to see if I can force this error.

Edit: sorry I read your post again and saw you’re using invariant.

from coordinatesharp.

Is this for an ASP.NET project?

and

Do you get this error if you initialize with a DateTime? Example:

Coordinate c = new Coordinate(41.8532527957705, 12.5651266486892, new DateTime(1999,1,1));

from coordinatesharp.

I can't replicate, but I believe I know where you are getting the error. I honestly need help debugging though, if it's something you would be able to assist with.

If you are up for it, remove the nuget package from your project, download the .cs files in the Invariant Culture Branch and add them to your project instead. If you don't get the error great, if you do though you will be able to see the exact line of code and the value passed that's causing issue.

I hate when people have problems I can't replicate or solve, so any assistance would be huge.

from coordinatesharp.

I had to push an update a couple days ago to address a possible validation issue with one of the CoordinatePart constructors. The Invariant Culture branch parts pushed with it. With that said, before you download the files, you can just update the Nuget package in your console app, turn EagerLoading back on and see if the error still presents itself.

If it's still occurring then we will need to investigate further. I specified the culture in the string to DateTime conversion in hopes that it fixes the issue by specifying culture (if that's what the underlying issue is).

Something I want to note is that when the string is formatted during eclipse calculations the final result is in the following format yyyy-MMM-dd (ex 2018-Mar-03). That format works for Invariant which is why I am so confused about the issue. What country settings are your operating system set too? I wonder if the library is ignoring your apps specified culture and grabbing it from your OS settings.

from coordinatesharp.

My OS locale is italian (the date string is exactly reversed) and I set thread culture like this (vbnet):
Globalization.CultureInfo.DefaultThreadCurrentCulture = Globalization.CultureInfo.InvariantCulture

from coordinatesharp.

Well I am at a loss. I tried changing thread cultures in the library directly to both Italian, and Invariant to try and get the exception to present itself, but it won't. This is making me insane!

I even confirmed everything was executing in the specified culture by tracing the culture used during function calls. I'm afraid I am out of options for now. I eventually want to change the code to avoid using strings, but for the time being I am going to change from Convert.ToDateTime' to 'DateTime.ParseExact in hopes that is addresses the issue.

Do you still get the exception with the the update to 1.1.2.8 if you turn EagerLoading back on? If not, then the previous fix I put in there is sufficient for the time being.

Let me know what you find out.

Thanks for helping.

from coordinatesharp.

Going to close, as conversions are exact parse in 1.1.3.1 which should resolve any cultural issues. If it still presents I will reopen.

from coordinatesharp.

I think I may be experiencing this error (I have a version running against .NET Core 2.).

Result StackTrace:	
at System.DateTime.ParseExact(String s, String format, IFormatProvider provider)
   at CoordinateSharp.SunCalc.CalculateSolarEclipse(DateTime date, Double lat, Double longi, Celestial c) in C:\Dev\External Repos\CoordinateSharp\CoordinateSharpCore\Celestial.SunCalculations.cs:line 255
   at CoordinateSharp.SunCalc.CalculateSunTime(Double lat, Double longi, DateTime date, Celestial c, Double offset) in C:\Dev\External Repos\CoordinateSharp\CoordinateSharpCore\Celestial.SunCalculations.cs:line 78
   at CoordinateSharp.Celestial.CalculateCelestialTime(Double lat, Double longi, DateTime date) in C:\Dev\External Repos\CoordinateSharp\CoordinateSharpCore\Celestial.cs:line 321
   at CoordinateSharp.Celestial..ctor(Double lat, Double longi, DateTime geoDate) in C:\Dev\External Repos\CoordinateSharp\CoordinateSharpCore\Celestial.cs:line 47
   at CoordinateSharp.Coordinate..ctor(Double lat, Double longi) in C:\Dev\External Repos\CoordinateSharp\CoordinateSharpCore\Coordinate.cs:line 78
   at DeploySoftware.LaunchPad.Shared.Domain.GeographicLocation..ctor() in C:\Dev\deploy-solutions\launchpad-shared\src\DeploySoftware.LaunchPad.Shared\Domain\Entities\Impl\GeographicLocation.cs:line 98
   at DeploySoftware.LaunchPad.Shared.Tests.GeographicLocationTests.Should_Have_Valid_Elevation_Number_When_Instantiated() in C:\Dev\deploy-solutions\launchpad-shared\tests\DeploySoftware.LaunchPad.Shared.Tests\GeographicLocationTests.cs:line 38
Result Message:	System.FormatException : String was not recognized as a valid DateTime.

When I change Celestial.SunCalculations line 255 to TryParseExact it solves the invalid DateTime string but then generates the IndexOutofRangeException mentioned above:

DateTime ld;
DateTime.TryParseExact(values[0], "yyyy-MMM-dd", 
                    CultureInfo.InvariantCulture,
                    DateTimeStyles.None, 
                    out ld);

When I remove Celestial from EagerLoading it works. I'm happy to do some debugging for you if you have some suggestions.

from coordinatesharp.

Darn, this bug is elusive! Thank you for bringing it to my attention. This is something I have been unable to replicate, so any information you can provide is most welcome.

1. Is this happening during initialization?
2. Is this happening with specific dates or all dates?
3. What culture are you running in?
4. Can you provide a code sample?

It's strange to me that a 'TryParseExact()' would fail at this point. It seems to be effecting only certain people so any help isolating the cause would be very appreciated.

from coordinatesharp.

1. I have a GeographicLocation class which now contains a Coordinate property called EarthCoordinate. The error pops up in the GeographicLocation constructor whenever it is instantiated.
2. I don't pass in any date at all, so I imagine it would use the default geodate:
   this.geoDate = new DateTime(1900, 1, 1, 0, 0, 0, DateTimeKind.Utc);
3. I don't think I have a specific culture set since I am getting this error from xUnit testing and not from a web page. :
4. Code sample (I can provide the whole class if you want)
   public GeographicLocation() { // We will set the elevation, latitude, and longitude of Greenwich _elevation = 46; _latitude = 51.476852; _longitude = -0.000500; EagerLoad load = new EagerLoad // I just added EagerLoad to stop the error { Celestial = false }; EarthCoordinate = new Coordinate(_latitude, _longitude, load); }

from coordinatesharp.

Thanks for the assistance. I still can't replicate the error (this is driving me insane).

First, can you throw in a break point at line 255 and see what values[0] is outputting.

Second, do you mind passing a couple dates to the Coordinate constructor to see if the error still occurs on both?

EarthCoordinate = new Coordinate(_latitude,_longitude, new DateTime(2000,1,1));
EarthCoordinate = new Coordinate(_latitude,_longitude, new DateTime(1800,2,1));

from coordinatesharp.

This bug is nuts. I spent half the day troubleshooting it and I'm not 100% sure the issue is your code after all (sorry for driving you insane!). Bizarrely, I get the error I reported when I enable Eager Loading for Celestial and then Run All tests in xunit runner, but when I run them individually (with or without the extra lines you asked for) they each pass. So there may be a dependency order error of some kind in my test code, which is surfacing in your code (but if so I have no idea what the error is).

1. Eager Loading false, Run All Tests = Pass
2. Eager Loading true, Run Selected Tests = Pass
3. Eager Loading true, Run All Tests = Fail

When it fails at line 255, values[0] is "P" - hence the format error. Why it is P I don't yet know, but it could definitely be culture- or OS-related. My culture is English Canada and we do dates a bit differently than the US (probably we use the same as @beppe9000 ).

from coordinatesharp.

Seriously thank you for even finding this much out. Nothing is worse then not being to fix a bug, because you can't replicate. This information actually helps a lot.

So "P" is being returned to values at index 0. It should always be at index 1, which means GetDate(double[] elements, double[] circumstances) may be doing something funky. This function is line 632 in the SolarEclipseCalc.cs file and it's being called from line 75. At least we know this much now.

Can you throw a breakpoint in at line line [75](

It should even matter, because even if GetDate() even returned null it would still populate a null value into values. It's almost as if the function is being skipped all together and the first index gets added by the line 79 `values.Add("P");'. This is so confusing. I wonder if @beppe9000 has issues only during unit tests as well now?

At least we have a direction now.

Laslty can you throw the following above line 255 in SunCalculations.cs to force skip bad parses and Run All Tests again. If it passes are you able to view the SolarEclipse times in the Celestial class?

DateTime ld;
if(!DateTime.TryParseExact(values[0], "yyyy-MMM-dd",  CultureInfo.InvariantCulture, DateTimeStyles.None, out ld){continue;}

This solution isn't really ideal IMO, but considering it works when running selected test, I don't know of any other options. If this works for you however, I will run it against the TestProj to see if we still get all the required SolarEclipseEvents. I would only want to implement this if we can't narrow down a cause.

I really appreciate the assistance.

from coordinatesharp.

Happy to help, you built a great library. The values in line 75 are always 12656 and 41 (doubles). With the continue code included, I still end up failing in SolarEclipseDetails line 35 when values[0] is "P" and I run all tests:
System.FormatException occurred HResult=0x80131537 Message=The string was not recognized as a valid DateTime. There is an unknown word starting at index 0. Source=<Cannot evaluate the exception source> StackTrace: at System.DateTimeParse.Parse(String s, DateTimeFormatInfo dtfi, DateTimeStyles styles) at System.Convert.ToDateTime(String value) at CoordinateSharp.SolarEclipseDetails..ctor(List1 values) in C:\Dev\External Repos\CoordinateSharp\CoordinateSharpCore\Celestial.Assistant.cs:line 1047
at CoordinateSharp.SunCalc.CalculateSolarEclipse(DateTime date, Double lat, Double longi, Celestial c) in C:\Dev\External Repos\CoordinateSharp\CoordinateSharpCore\Celestial.SunCalculations.cs:line 270
at CoordinateSharp.SunCalc.CalculateSunTime(Double lat, Double longi, DateTime date, Celestial c, Double offset) in C:\Dev\External Repos\CoordinateSharp\CoordinateSharpCore\Celestial.SunCalculations.cs:line 80
at CoordinateSharp.Celestial.CalculateCelestialTime(Double lat, Double longi, DateTime date) in C:\Dev\External Repos\CoordinateSharp\CoordinateSharpCore\Celestial.cs:line 321
at CoordinateSharp.Celestial..ctor(Double lat, Double longi, DateTime geoDate) in C:\Dev\External Repos\CoordinateSharp\CoordinateSharpCore\Celestial.cs:line 47
at CoordinateSharp.Coordinate..ctor(Double lat, Double longi, DateTime date, EagerLoad eagerLoad) in C:\Dev\External Repos\CoordinateSharp\CoordinateSharpCore\Coordinate.cs:line 180
at DeploySoftware.LaunchPad.Shared.Domain.GeographicLocation..ctor()
at DeploySoftware.LaunchPad.Shared.Tests.GeographicLocationTests.Should_Have_Valid_Elevation_Number_When_Instantiated()
`

from coordinatesharp.

You're too kind. It certainly helps when good people like yourself help contribute! Ok next steps.

Line 264 is causing the same issue then which must mean every pass in the loop is returning "P". I was hoping it was just a single pass during initialization.

Can you put a break point in SolarEclipseCalc line 77 and tell me what values (line 75) count is and if it has a count what is the value at index 0. It should at least have values[0] set as null which means "P" should be going to values[1]. But the issue described makes it seem like it's skipping the call at 75 all together.

from coordinatesharp.

Count 1 (or more), values [0] = "1902-Oct-31" (or some other date). It doesn't ever seem to be null. I'm not sure if it would help you, but I'm happy to do a skype or something and show my screen.

from coordinatesharp.

Ok so to recap:

1. The values arrays are populated at call List<List<string>> se = SolarEclipseCalc.CalculateSolarEclipse(date, latR, longR); - SunCalculations Line 243
2. As you just confirmed values[0] is always a date and never empty or null when it's populated via SolarEclipseCalc Line 77.
3. With populated values that you confirmed have correct dates at index 0: A parse on index 0 is attempted, but we get an exception because it equals "P", even though it equaled a date previously - SunCalculations Line 255

WTF?!?!?!?

Is values[0] being removed by the unit test? This makes absolutely no sense. At least we know it's not a culture issue.

A screen share may be a good call at this point, but I won't be able to until next week. I would like to try and replicate the unit test. You said you are using xUnit right? I'm not familiar with it, but I see its available as a Nuget package. Could you pass me a minimal code sample of a Test that fails so I can download and replicate the test and see if I can get it to throw the error on my end.

from coordinatesharp.

Also, I just shot an email to Next Games as they are the biggest user of the library. I want to see if they are seeing any issues regarding this. I am hoping its a test environment issue only.

from coordinatesharp.

Hi, I'm having trouble reproducing. I think the issue may have been related to my xUnit, but I'm not sure. I don't think the problem is in your code however - you're welcome to close the bug as far as I'm concerned, and thanks for investigating it.

from coordinatesharp.

That's so strange. I'm a little concerned as you are the second person who has had this issue though, but at least we have narrowed it down a bit more. Thank you for helping me debug!!!

from coordinatesharp.

Im getting this bug now. https://gyazo.com/d3a96b03e8309f4b5681e8f601a68ec6

Asp.NET Core 2.1

from coordinatesharp.

Its also weird because its very random. When i stress test my server, 95% of the time it works, but the other 5% of the time it doesnt even with the same request data.

from coordinatesharp.

@njbrown09 could it be a time-based problem?

from coordinatesharp.

Im doing DateTime.UtcNow for the time. It only appears in really high load scenarios. The longer the high load is the more often i get the bug.

from coordinatesharp.

SON OF A **** I'm just going to leave this open until we actually figure this out once and for all.

Any ideas on why a higher server load would cause this exception to throw. Going through the code it seems impossible for the string to be empty. @beppe9000 have you experienced the issue since we first addressed it?

@njbrown09 what version are you using?

from coordinatesharp.

The "P" string throwing the exception should always be values[1] not values[0]. Some how the following call is being dropped from the logic OR the Array. This is the first value that goes into the list.

How on earth does it just disappear. If the function that populates the value threw an exception, then that's where the exception would throw right? But it completes, the the value is dropped from the list OR it's being skipped all together. What are we missing?

from coordinatesharp.

CoordinateSharpTest.zip

Stress test YOURIP:90/test/Coordinate with loadimpact.com and within 3 minutes you should get this bug.

from coordinatesharp.

My firewall is blocking the test I think. I'm just getting HTTP request errors.

EDIT: I'm going to try and recreate locally. I'm wondering if it's some type of memory or garbage collection issue considering the array element just disappears.

from coordinatesharp.

I never experienced 'production' problems after the troubleshooting done duding the original report (ended up disabling celestial calculations which I didn't need anyway). I have not used this library for other projects after that, but I had tried the many solutions suggested by tronald with no success.
This is one hell of a sneaky ***** bug...

from coordinatesharp.

@njbrown09 Thank you so much! Using your program I was able to create a local impactor to throw at it and I finally got it to throw the exception. Now I can finally troubleshoot. I will get back to you.

from coordinatesharp.

Fix pushed to master

from coordinatesharp.