I am using this library to write an SPSS file. The data I am using is a very reasonabl

Writer is taking a very long time to write a reasonable amount of data. about spss HOT 1 CLOSED

tiamo commented on June 10, 2024

Writer is taking a very long time to write a reasonable amount of data.

from spss.

Comments (1)

commented on June 10, 2024

I gathered some data here. I wanted to see when the execution time starts to skyrocket and it appears that the code hangs at some limit and the exciton time does not gradually build up. I did some tests and for my data the execution time explodes at exactly 47 variables.

[2019-04-23 20:31:19] local.INFO: VARIABLES: 40 - EXECUTION TIME: 0.13907388448715 MINUTES || 8.3444330692291 SECONDS  
[2019-04-23 20:32:55] local.INFO: VARIABLES: 41 - EXECUTION TIME: 0.13721104860306 MINUTES || 8.2326629161835 SECONDS  
[2019-04-23 20:33:30] local.INFO: VARIABLES: 42 - EXECUTION TIME: 0.13726431528727 MINUTES || 8.2358589172363 SECONDS  
[2019-04-23 20:33:58] local.INFO: VARIABLES: 43 - EXECUTION TIME: 0.13848913113276 MINUTES || 8.3093478679657 SECONDS  
[2019-04-23 20:34:26] local.INFO: VARIABLES: 44 - EXECUTION TIME: 0.13936993281047 MINUTES || 8.3621959686279 SECONDS  
[2019-04-23 20:34:52] local.INFO: VARIABLES: 45 - EXECUTION TIME: 0.16820962031682 MINUTES || 10.092577219009 SECONDS  
[2019-04-23 20:35:24] local.INFO: VARIABLES: 46 - EXECUTION TIME: 0.16949793497721 MINUTES || 10.169876098633 SECONDS  
[2019-04-23 20:35:24] local.INFO: VARIABLES: 47 - TIMES OUT AT 5 MINUTES
[2019-04-23 20:42:35] local.INFO: VARIABLES: 46 - EXECUTION TIME: 0.16918338139852 MINUTES || 10.151002883911 SECONDS  
[2019-04-23 20:35:24] local.INFO: VARIABLES: 47 - TIMES OUT AT 5 MINUTES

As you can see I did 46 and 47 variables twice just to be sure and it's clear that I've hit some limit. I would be happy to work with someone to help diagnose the problem. I'm not 100% sure on what the best way forward is.

from spss.

Recommend Projects