C implementation that accommodates 10-byte and 16-byte floating point data types about ryu HOT 7 CLOSED

Hi,

First and most simple step is to implement 'long double' on MSVC, as it has the exact same size and stricture as 'double'. ;)

#include "ryu/d2s_intrinsics.h"

You will need to copy this file to say "e2s_intrinsic.h" and "dd2s_intrinsics.h" for the binary80 and binary128 correspondingly. (e for extended dd for double double. Did not use l for long because that may be 80 or 128, depending on the CPU and OS.) The file may need to be extended with 256 bit operations, I am not sure. If no changes or extensions are needed I would still make the file with another name and include "d2s_instrinsic.h" through that one, with a comment that the same intrinsics are use.

// Include either the small or the full lookup tables depending on the mode.
#if defined(RYU_OPTIMIZE_SIZE)
#include "ryu/d2s_small_table.h"
#else
#include "ryu/d2s_full_table.h"
#endif

The functions and tables above have to be extended for the larger types on platforms that support 128 bit unsigned integers, and probably new functions added for don't have 128 bit ints. You will see while working on d2d I think.

#define DOUBLE_MANTISSA_BITS 52
#define DOUBLE_EXPONENT_BITS 11
#define DOUBLE_BIAS 1023

The above constants should be changed to the ones corresponding to the 128 or 80 bit float. you are working on (and their names changed). In this comment I will call the x87 80-bit extended binary format as binary80 for simplicity. IEEE-754 does not define a binary80 format.

I would start changing the code with the 16 bytes one (binary128), as it is a "move up" that you already have an example for, from IEEEE-754 binary32 (float) f2s.c to binary64 (double) d2s.c. If you find a computer that actually supports binray128. I assume it is simpler to go back from binary128 by removing stuff than doing binary80 and then binary128.

I would also start with make it correct first, then and only then try to make it faster. The fixed code contains example on 128 bit fast division. The 64 bit shortest-representation code contains example of 64 bit fast division, as well as handling certain small integer values, which handling can be extended to the 128 bit type. Binary128 is well documented just like 64 on the Unix compilers like Linux. Binary80 (extended double precision FP) is a bit harder to achieve, but not impossible. The first step would be to try to find a compiler that gives you that binary80 as one of its built-in types. Either long double or some __double80 . @StephanTLavavej might have suggestions there on type name or whom to ask. MSVC does not support embedded assembly in 64 bit mode, which would be another way to access such types. Every Intel CPU supports binary80, but I do not remember any "trick" on how to make them visible to C or C++ code. So with binary80 this would be the big deal, finding an "oracle" that makes sure whatever value you printed using Ryu is actually the value of the binary80. So you would like to be able to go from C floating point literal to binary80 using proven means, meaning the compiler or maybe assembler from long double. Otherwise you would first have to test the code you wrote to create binary80...

Once the "how do I get reliable values" part is done I would start by defining the typedef struct floating_decimal_128 (and typedef struct floating_decimal_80 structures. Interestingly the definition would start by defining first the exponent range, then the mantissa size. The mantissa will be an integer that is "biased" as the exponent can go from the largest absolute subnormal decimal exponent (most negative exponent, -324 for double, the exponent of DBL_TRUE_MIN) to the largest positive exponent, which is the exponent of DBL_MAX. Note that while the exponent of DBL_MAX is the same as DBL_MAX_10_EXP, the exponent of DBL_MIN is not DBL_MIN_10_EXP, but one less! Why? XXX_MIN_10_EXP is the exponent value where every decimal number (every possible mantissa) can be represented as a normal number. But 1e-308 cannot.

While designing the typedef struct floating_decimal_128 type remember that it will store positive values on, so its mantissa will always be positive.

Once you know the value ranges of the mantissa and exponent of floating_decimal_128/80 you will need to chose the data type for them.: an unsigned type for the mantissa, and a signed type for the exponent. I think you will have no issue choosing an exponent type as every compiler nowadays supports up to 64 but signed types directly. Your issue will come with the longer-than-64-bit mantissa. Some compilers do support some 128 bit types but their speed of operation and even reliability (if they are newly implemented) may be questionable. I suggest to build the simplest solution first, then go from there to make it faster. There are 128 bit examples already in d2fixed.c. I have no idea if they are the fastest or how well tested they are, but they are good for a starting point.

Once you are done with the typedef struct floating_decimal_128 you will need to extend the d2d function. Here is where you can use the 128 bit operation functions from d2fixed.c. Compare the d2d function in f2s.cpp with the one in d2s.cpp. Follow the logic how it was extended. Test your code as you progress by inputting values and verifying the result in floating_decimal_128 format.

Once done with d2d, you can extend to_chars just how to_chars was extended between f2s.c and d2s.c. You can continue using your test by now verifying the outout of both d2d and then to_chars. At this point you need to calculate the maximum size of the character output from the values you have calculated for the floating_decimal_128. At this point the numbers are all positive, so we do not need to add space for a negative sign.

The next is to modify the small_int function to work. I would change what is line 433 in my copy of d2s from if (e2 < -52) to if (e2 < -DOUBLE_MANTISSA_BITS). The code is well commented, follow it, changing types and numbers as needed. Keep testing the code once it compiles! Changes your test case from testind d2d and to_chars to call small_int just as d2s does.

Now come d2s_buffered_n(double f, char* result), where we just need to change type names and the function names(d2d should be d2e or d2dd, same for the prefix of d2d_small_int).

The d2s_buffered function needs no change, d2s needs its malloc argument recalculated from the boundary values of binary80 and binary128. The calculation is simple, as we need to calculate the maximum length of the scientific format only, because anything longer will just fall back to the scientific format. So the end result is: 1 + XXX_DECIMAL_DIG + 1 + 1 + 1 + DIGITS10(XXX_MAX_10_EXP) + 1. The one's are: optional negative sign, the optional decimal point, the 'e' for the exponent, the exponent's sign (not optional in printf formats). The last + 1 is only necessary for the malloc in e2s or dd2s, or the _buffered variant, for the closing null character.

A very good start at understanding the different values and structures are:

Standard formats (also the 128 bit binary)
https://en.wikipedia.org/wiki/IEEE_754

The x87 80 bit extended format:
https://en.wikipedia.org/wiki/Extended_precision#x86_extended_precision_format

The C <limits.h> header (unfortunately the page is half done) right now:
https://en.cppreference.com/w/c/types/limits
The same page for C++, shows all values:
https://en.cppreference.com/w/cpp/types/climits

The C++ numeric_limits page:
https://en.cppreference.com/w/cpp/types/numeric_limits/max
Not sure if I have helped or not, I hop I did.

from ryu.

Thank you @cppguru for the detailed reply.
(Incidentally, I have no need to be concerned with Microsoft compilers.),

I was hoping there might have been something ready made sitting around somewhere, but I can see that I have some work to do.

Thanks again for the links and pointers.

I intend to close this issue in a few days from now unless a good reason to keep it open is presented.
I suspect that raising this topic as an "Issue" here might actually amount to (mild) abuse of this platform, but I couldn't think of a more appropriate place to ask the questions.

from ryu.

BTW if you look at the generic_128.* files there is a generalized implementation in the ryu directory (in C), but I think it was kind of half-hearted compared to the rest. By that I mean it is not optimized to the end of the world and I could not even compile it with every C compiler I had to support. So it is not "as Ryu" as the rest of the sources, if I make any sense by saying that. But it may be good enough. I don't recall how well that code was tested, and that is like 95% of the work. :)

from ryu.

No problems in compiling that source with the compilers I use, so long as:

1. the __uint128_t type is available;
2. the compiler is operating in C99 mode.

For non-C99 compilers it was a simple task to rewrite the offending sections of the source into a form that was acceptable.

I suppose that a good place to start would be to extend that source to accommodate those compilers where the absence of the __uint128_t type is the issue.
With that done, I'll probably be better equipped to move on to the handling of extended precision long double and IEEE-754 long double/__float128 types.

As stated earlier, I'm now also closing this issue.
Further comments are, of course, still welcome.

from ryu.

Heh ... true to form, I've only just now noticed the -DRYU_ONLY_64_BIT_OPS option - which will probably simplify the task of getting this source to build in environments where the __uint128_t type is unavailable ;-)

from ryu.

hi guys, :-) - cookbook recipe needed ...

you are writing on a high level about a thing i'd like to have up and running but didn't get. i managed to get the libryu.a solution from the ryu subdir to work, but now need to convert 80-bit long doubles. ( debian style linux, gcc, 'C' mode ) .
Starting with guessing which header file to use ( generic_128.h or ryu_generic_128.h, and or additional, how to check availability of _uint128_t, and how to compile, install, where to place, what to call and if or with which parameters ... i'm a little stuck.
Can someone provide a working example with not too much 'may be you need to'? just a collection of files managing the conversion of a 80-bit long double? Would be great, i think also for others. Once I have seen something running I normally can adapt it to my needs.

TIA! for any help, b.

from ryu.

answering myself, @Pro's: pls. improve, @newbie's: hope it helps you:

1. understand that's a 2-step process,
2. fetch a copy of the ryu sorce, 'git clone ... '
3. for an easy start work in the ryu/ryu subdirectory,
4. in this directory fire 'make', assumed a 'normal' gcc development environment available you get a library 'libryu.a',
5. place the program below in that directory, name as you like, i'd choose 'convert_long_double_to_string.c'
6. i needed to add '#include stdbool.h' to generic_128.h,
7. compile with: 'gcc convert_long_double_to_string.c -o convert_long_double_to_string -lryu', evtl. add ' -ldfp',
8. run with 'convert_long_double_to_string'
9. get an output like below: good, if not: investigate acc. errors you get thrown.
10. @Newbies: i hope it helped,
11. @Pros: i know! below is cruel ... pls improve ... there once was a culture in the linux world to provide a 'howto' which helped users to do the first steps and then understand / look up the options ...
12. i got lot's!!! of compiler complaints, warnings and errors, but it works somehow, @Pros: pls. improve ...
13. program:

// Build with: 'gcc convert_long_double_to_string.c -o convert_long_double_to_string -lryu 
// evtl. -ldfp is needed,  

#define __STDC_WANT_DEC_FP__ 1 
#include <stdio.h>      // reg. e.g. printf, 
#include <stdlib.h>     // reg. e.g. gentenv, 
#include "ryu.h"        // reg. e.g. d2s, 
#include "generic_128.h"        // test, reg. e.g. generic_binary_to_decimal?, 
#include "ryu_generic_128.h"        // test, reg. e.g. generic_binary_to_decimal?, 
#include <stdbool.h>    // reg. bool in generic128.h?, 
#include <quadmath.h>   // reg. e.g. float128, 
 
typedef struct floating_decimal_128 t_fd128; 

int main (const int argc, const char * const argv[]) 
{ 
    long double xL = 1.0471975511965977461L; 
    char* buffer[55]; 
    t_fd128 test_fd128; 

    test_fd128 = long_double_to_fd128( xL );
    generic_to_chars( test_fd128, buffer );
    printf( "buffer:  . . . . . . . . . . . . . . .  %s\n", buffer ); 
    printf( " \n"); 

    xL = 1.04719755119659790151L;
    test_fd128 = long_double_to_fd128( xL );
    generic_to_chars( test_fd128, buffer );
    printf( "buffer:  . . . . . . . . . . . . . . .  %s\n", buffer ); 
    printf( " \n"); 

    xL = 1.04719755119659790152L;
    test_fd128 = long_double_to_fd128( xL );
    generic_to_chars( test_fd128, buffer );
    printf( "buffer:  . . . . . . . . . . . . . . .  %s\n", buffer ); 
    printf( " \n"); 

    xL = 1.04719755119659790156L;
    test_fd128 = long_double_to_fd128( xL );
    generic_to_chars( test_fd128, buffer );
    printf( "buffer:  . . . . . . . . . . . . . . .  %s\n", buffer ); 

return 0;
}```
-------------------------------------------------------------------------------------------
14. output: 

buffer: . . . . . . . . . . . . . . . 1.0471975511965977461E0

buffer: . . . . . . . . . . . . . . . 1.0471975511965979015E0

buffer: . . . . . . . . . . . . . . . 1.0471975511965979015E0

buffer: . . . . . . . . . . . . . . . 1.0471975511965979016E0

above may be changed / improved by any experienced dev. 

from ryu.