Bug Report - The latest version of Macos and Windows is stuck when importing large CSV files, and the program interface is white. about compass HOT 6 CLOSED

Hi @yjn140 - we recently did some changes in our import backend which likely caused this issue.

I've created a jira ticket to track it to reproduce/fix: https://jira.mongodb.org/browse/COMPASS-6617
Thanks for reporting it.

from compass.

@yjn140 Can you perhaps send us a sample of the file? Just the first few lines including the header row. We didn't autodetect the CSV separator at all until recently and it would just always pick ,. Which delimiter did it auto-detect? Which delimiter is the file using? How many columns does the file have? Does it show the preview table with the columns before you import? Some screenshots from before it crashes could also be helpful.

Also as a test, please try importing with the Stop on errors checkbox checked and let us know what it does.

from compass.

I use a leak database, his format is：

id phone
234342355 13133334566
2343324355 13133344566

The separator is a space

The amount of data is 700 million lines, and the size of the file is 12GB

I think you can use a program to generate such a txt data file for testing. I want to describe the details of the error. When I choose the CSV file that I can import, he recognizes the separator, and I click to import. The maximum number of rows displayed at the beginning of the program is not the actual number of rows. The picture, the program is blank. There is also no upload of data in the background. It is just that the program occupies LeCPU and memory, and there is no operation to read.

I should be able to reproduce, because when my MacOS version was updated to the latest M1/M2 version, this problem occurred. Then I chose to install the latest version on Windows, which is the same. After that, I installed the official version of the 3.5, and the software of the two platforms was available. About 2 hours later, I will upload a screenshot

from compass.

from compass.

There was a bug/memory leak in the code that lists the csv fields for the preview table that just got merged. We'll get a new release out with this fix ASAP #4178

from compass.

Another issue that will likely affect you is that the analysis of a 12GB file to detect the field types can take a very long time and is probably not going to be that useful for a two column file anyway. The new release will be containing that, unfortunately. We'll be limiting the preview size or time that takes in a follow-up as well.

from compass.