incorrect benchmark for floating points about turbopfor-integer-compression HOT 4 CLOSED

Thanks for reporting. I've made some changes now to "icapp.c"
You can use the option "-v5" to display the first entries from the input file.

Test file downloaded from : hctsa_timeseriesdata.csv

./icapp -Fcd hctsa_timeseriesdata.csv -v5

or for lossy compression:

./icapp -Fcd -g.001 hctsa_timeseriesdata.csv

from turbopfor-integer-compression.

Right so this is something similar to what I attempted to do and I ended up with a ratio of 100%. I pulled in your change and I got a ratio of 100% as well

./icapp -Ftd -f8  test.txt -v5
dfmt=12,size=-8
reading text lines. pre=1.00, col=1, sep=
'7.654084'->7.654084  '7.284062'->7.284062  '19.871333'->19.871333  '18.276295'->18.276295  '19.889258'->19.889258  '13.973003'->13.973003  '11.984927'->11.984927  '4.745301'->4.745301  '17.609446'->17.609446  '8.408568'->8.408568  '19.196000'->19.196000  '3.445371'->3.445371  '13.123649'->13.123649  '11.500248'->11.500248  '17.432656'->17.432656  '9.408568'->9.408568  '2.794389'->2.794389  '7.454506'->7.454506  '13.276589'->13.276589  '6.092409'->6.092409  '2.287832'->2.287832  '16.712406'->16.712406  '11.911262'->11.911262  '18.825086'->18.825086  '17.371417'->17.371417  '7.347464'->7.347464  '9.900287'->9.900287  '4.764600'->4.764600  '17.462729'->17.462729  '1.029691'->1.029691  '7.159808'->7.159808  '14.733583'->14.733583  '8.781528'->8.781528  '1.340957'->1.340957  '8.705878'->8.705878  '4.984214'->4.984214  '19.049450'->19.049450  '9.714655'->9.714655  '4.928462'->4.928462  '17.908039'->17.908039  '7.451029'->7.451029  '11.457274'->11.457274  '5.477730'->5.477730  '11.739603'->11.739603  '8.182033'->8.182033  '6.346988'->6.346988  '10.744302'->10.744302  '15.401736'->15.401736  '14.084666'->14.084666  '7.982885'->7.982885  '13.010926'->13.010926  '11.500662'->11.500662  '4.764194'->4.764194  '9.641864'->9.641864  '3.155587'->3.155587  '19.200254'->19.200254  '10.120186'->10.120186  '12.002349'->12.002349  '3.987833'->3.987833  '12.366019'->12.366019  '10.464250'->10.464250  '4.378455'->4.378455  '17.126242'->17.126242  '17.906185'->17.906185  '6.587847'->6.587847  '4.784348'->4.784348  '12.248120'->12.248120  '3.984555'->3.984555  '10.798502'->10.798502  '17.155995'->17.155995  '5.050986'->5.050986  '4.364953'->4.364953  '13.737133'->13.737133  '13.277931'->13.277931  '13.267108'->13.267108  '3.312669'->3.312669  '18.795482'->18.795482  '6.961895'->6.961895  '17.526652'->17.526652  '5.656442'->5.656442  '4.349281'->4.349281  '18.599398'->18.599398  '14.020364'->14.020364  '6.367115'->6.367115  '18.656983'->18.656983
   E MB/s     size     ratio    D MB/s  function
     0.00        681 100.15%     0.00 p4nenc64         	test.txt
     0.00        681 100.15%     0.00 p4nenc128v64     	test.txt
     0.00        649  95.44%     0.00 p4ndenc64        	test.txt
     0.00        649  95.44%     0.00 p4nd1enc64       	test.txt
     0.00        591  86.91%     0.00 p4nzenc64        	test.txt
     0.00        671  98.68%     0.00 bitnpack64       	test.txt
     0.00        671  98.68%     0.00 bitnpack128v64   	test.txt
     0.00        682 100.29%     0.00 bitndpack64      	test.txt
     0.00        682 100.29%     0.00 bitnd1pack64     	test.txt
     0.00        598  87.94%     0.00 bitnzpack64      	test.txt
     0.00        681 100.15%     0.00 vbenc64          	test.txt
     0.00        680 100.00%     0.00 vbzenc64         	test.txt
     0.00        765 112.50%     0.00 vsenc64          	test.txt

Some sample values from test.txt

7.654084
7.284062
19.871333
18.276295
19.889258
13.973003
11.984927
4.745301
17.609446
8.408568
19.196000
3.445371
13.123649
11.500248
17.432656
9.408568
2.794389
7.454506
13.276589
6.092409
2.287832
16.712406
11.911262
18.825086
17.371417
7.347464
9.900287
4.764600
17.462729
1.029691

from turbopfor-integer-compression.

If you have a fixed number of decimal digits, it is better to convert your numbers to integers.
You can do this directly with icapp:

./icapp -Ftl.6 test.txt -v5

dfmt=12,size=8
reading text lines. pre=1000000.00, col=1, sep=
'7.654084'->7654084  '7.284062'->7284062  '19.871333'->19871333  '18.276295'->18276295  '19.889258'->19889258  '13.973003'->13973003  '11.984927'->11984927  '4.745301'->4745301  '17.609446'->17609446  '8.408568'->8408568  '19.196000'->19196000  '3.445371'->3445371  '13.123649'->13123649  '11.500248'->11500248  '17.432656'->17432656  '9.408568'->9408568  '2.794389'->2794389  '7.454506'->7454506  '13.276589'->13276589  '6.092409'->6092409  '2.287832'->2287832  '16.712406'->16712406  '11.911262'->11911262  '18.825086'->18825086  '17.371417'->17371417  '7.347464'->7347464  '9.900287'->9900287  '4.764600'->4764600  '17.462729'->17462729  '1.029691'->1029691  
   E MB/s     size     ratio    D MB/s  function
     0.00         95  39.58%     0.00 p4nenc64         	test.txt
     0.00         95  39.58%     0.00 p4nenc128v64     	test.txt
     0.00        182  75.83%     0.00 p4ndenc64        	test.txt
     0.00        182  75.83%     0.00 p4nd1enc64       	test.txt
     0.00         96  40.00%     0.00 p4nzenc64        	test.txt
     0.00         95  39.58%     0.00 bitnpack64       	test.txt
     0.00         95  39.58%     0.00 bitnpack128v64   	test.txt
     0.00        237  98.75%     0.00 bitndpack64      	test.txt
     0.00        237  98.75%     0.00 bitnd1pack64     	test.txt
     0.00         96  40.00%     0.00 bitnzpack64      	test.txt
     0.00        129  53.75%     0.00 vbenc64          	test.txt
     0.00        130  54.17%     0.00 vbzenc64         	test.txt
     0.00        120  50.00%     0.00 vsenc64          	test.txt
     0.00        120  50.00%     0.00 vszenc64         	test.txt
     0.00        135  56.25%     0.00 bvzzenc64        	test.txt
     0.00        127  52.92%     0.00 bvzenc64         	test.txt
     0.00        104  43.33%     0.00 fpgenc64         	test.txt
     0.00         99  41.25%     0.00 fpzzenc64        	test.txt
   240.00         95  39.58%   240.00 fpfcmenc64       	test.txt
   240.00         99  41.25%   240.00 fpdfcmenc64      	test.txt
   240.00         99  41.25%   240.00 fp2dfcmenc64     	test.txt
    40.00        192  80.00%     0.00 trle             	test.txt
    34.29        222  92.50%     0.00 trlez            	test.txt
     0.00        240 100.00%     0.00 srle64           	test.txt
     0.00        240 100.00%     0.00 srlez64          	test.txt
     0.00        240 100.00%     0.00 memcpy          

from turbopfor-integer-compression.

I actually wanted to see if it's feasible to store/compress double precision values as is, but limiting the decimal digits is not a bad option. Thanks for your prompt response!

from turbopfor-integer-compression.