Comments (8)
I don't think asan would show anything different than massif
already does.
The reason why I used valgrind's massif
in the first place is because valgrind's memcheck
does not show any leaks so my assumption was the memory gets ultimately freed but only at exit. That's why I'm monitoring runtime memory use and not memory that was not freed at exit.
Anyway I found the issue:
Lines 1245 to 1249 in fa83255
This logic is wrong as far as I can tell. Calculated rounded
req_size
is assigned to m->email_max
which will always be selected in above MAX()
and further increased by 25
due to increment and it will only keep on growing by 25
with each call. Since it is called per each email and every call allocates 25
more of sizeof(void*) + sizeof(int)
which on my machine happens to be 12b
, with 108260
emails that I have in big mailbox gives 108260*25*12=32478000
. The 30MB
leak that I'm seeing.from neomutt.
Damn! You're good!
Thanks for the debugging.
I'll create a fix after I've done my chores -- lots of things need merging.
from neomutt.
I not sure that NeoMutt is leaking, but it does seem to be using quite a lot of memory.
When you open NeoMutt, obviously it allocates memory for all the mailboxes in the sidebar and all the emails in the open mailbox.
Even mailboxes that aren't open take up memory.
I did quick test whilst watching the memory in top
(low-tech, I know).
I created two macros:
macro index <f5> "<change-folder>-<enter>"
macro index <f6> "<f5><f5><f5><f5><f5><f5><f5><f5><f5><f5>"
Then, I opened a small folder, then opened a very large folder.
Changing folder to -
means open the previous folder.
I hit <f6> five times and waited.
That should open each folder 25 times.
The memory usage fluctuated as folders were opened and closed, but crucially it didn't creep upwards.
At least, not by any obvious amount.
I'll leave this open and keep thinking
from neomutt.
Maybe some more data to compare:
Memory use after opening initial small mailbox only without opening big one is negligible and less than 2MB.
Memory use after opening small mx -> big mx -> small mx
99.45% (34,998,419B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->92.71% (32,624,676B) 0x518404: mutt_mem_realloc (memory.c:131)
| ->61.58% (21,669,400B) 0x43033F: mx_alloc_memory (mx.c:1260)
| | ->61.58% (21,669,400B) 0x4B6E46: maildir_move_to_mailbox (shared.c:95)
| | ->61.58% (21,669,400B) 0x4B1E40: maildir_read_dir (maildir.c:731)
| | ->61.58% (21,669,400B) 0x4B1FEA: maildir_mbox_open (maildir.c:1127)
| | | ->61.58% (21,669,400B) 0x431563: mx_mbox_open (mx.c:388)
| | | ->61.58% (21,669,400B) 0x43BCAC: change_folder_mailbox (dlg_index.c:696)
| | | | ->61.58% (21,669,400B) 0x44035F: op_main_change_folder (functions.c:1156)
| | | | ->61.58% (21,669,400B) 0x4428F3: index_function_dispatcher (functions.c:3323)
| | | | ->61.58% (21,669,400B) 0x43C782: dlg_index (dlg_index.c:1343)
| | | | ->61.58% (21,669,400B) 0x40C7B6: main (main.c:1392)
| | | |
| | | ->00.00% (0B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.00% (0B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->30.79% (10,834,700B) 0x430350: mx_alloc_memory (mx.c:1261)
| | ->30.79% (10,834,700B) 0x4B6E46: maildir_move_to_mailbox (shared.c:95)
| | ->30.79% (10,834,700B) 0x4B1E40: maildir_read_dir (maildir.c:731)
| | ->30.79% (10,834,700B) 0x4B1FEA: maildir_mbox_open (maildir.c:1127)
| | | ->30.79% (10,834,700B) 0x431563: mx_mbox_open (mx.c:388)
| | | ->30.79% (10,834,700B) 0x43BCAC: change_folder_mailbox (dlg_index.c:696)
| | | | ->30.79% (10,834,700B) 0x44035F: op_main_change_folder (functions.c:1156)
| | | | ->30.79% (10,834,700B) 0x4428F3: index_function_dispatcher (functions.c:3323)
| | | | ->30.79% (10,834,700B) 0x43C782: dlg_index (dlg_index.c:1343)
| | | | ->30.79% (10,834,700B) 0x40C7B6: main (main.c:1392)
| | | |
| | | ->00.00% (0B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.00% (0B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->00.34% (120,576B) in 1+ places, all below ms_print's threshold (01.00%)
Memory use after opening small mx -> big mx -> big mx -> small mx:
99.71% (67,434,423B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->96.26% (65,101,648B) 0x518404: mutt_mem_realloc (memory.c:131)
| ->64.05% (43,320,800B) 0x43033F: mx_alloc_memory (mx.c:1260)
| | ->64.05% (43,320,800B) 0x4B6E46: maildir_move_to_mailbox (shared.c:95)
| | ->64.05% (43,320,800B) 0x4B1E40: maildir_read_dir (maildir.c:731)
| | ->64.05% (43,320,800B) 0x4B1FEA: maildir_mbox_open (maildir.c:1127)
| | | ->64.05% (43,320,800B) 0x431563: mx_mbox_open (mx.c:388)
| | | ->64.05% (43,320,800B) 0x43BCAC: change_folder_mailbox (dlg_index.c:696)
| | | | ->64.05% (43,320,800B) 0x44035F: op_main_change_folder (functions.c:1156)
| | | | ->64.05% (43,320,800B) 0x4428F3: index_function_dispatcher (functions.c:3323)
| | | | ->64.05% (43,320,800B) 0x43C782: dlg_index (dlg_index.c:1343)
| | | | ->64.05% (43,320,800B) 0x40C7B6: main (main.c:1392)
| | | |
| | | ->00.00% (0B) in 1+ places, all below ms_print's threshold (01.00%)
| | |
| | ->00.00% (0B) in 1+ places, all below ms_print's threshold (01.00%)
| |
| ->32.03% (21,660,400B) 0x430350: mx_alloc_memory (mx.c:1261)
| | ->32.03% (21,660,400B) 0x4B6E46: maildir_move_to_mailbox (shared.c:95)
| | ->32.03% (21,660,400B) 0x4B1E40: maildir_read_dir (maildir.c:731)
| | ->32.03% (21,660,400B) 0x4B1FEA: maildir_mbox_open (maildir.c:1127)
| | | ->32.03% (21,660,400B) 0x431563: mx_mbox_open (mx.c:388)
| | | ->32.03% (21,660,400B) 0x43BCAC: change_folder_mailbox (dlg_index.c:696)
| | | | ->32.03% (21,660,400B) 0x44035F: op_main_change_folder (functions.c:1156)
| | | | ->32.03% (21,660,400B) 0x4428F3: index_function_dispatcher (functions.c:3323)
| | | | ->32.03% (21,660,400B) 0x43C782: dlg_index (dlg_index.c:1343)
| | | | ->32.03% (21,660,400B) 0x40C7B6: main (main.c:1392)
So it's seems pretty linear with 30MB allocated more per big mx open and matches pretty well allocation of 157MB after opening big mx 5 times that was shown in initial comment.
from neomutt.
OK, thanks.
If you've got the time, please could you build with the Address Sanitizer
It tracks all memory allocations / frees.
If there are any leaks, it prints out a report on exit telling us exactly where the allocations occurred.
from neomutt.
I guess the growing should occur only if original req_size
(as present in function invocation) is higher than m->email_max
. Not always.
from neomutt.
Made an attempt to fix the issue in #4089.
Memory use in situation from description (smal mx -> 5x big mx -> small mx)
- before fix: ~157MB
- after fix: 3.5MB
Although use when having big mx open is possibly still concerning: 500MB. To be analyzed independently though.
I will create one more PR for reducing allocation overhead a little.
from neomutt.
Thanks for diving in; I'll have a play in a second.
when having big mx open is possibly still concerning: 500MB
Long ago, I had an idea to create a :memory
page, like firefox's about:memory
I'd be very interested in knowing where it's all being used.
from neomutt.
Related Issues (20)
- Periodical crashing with segfault 11 on macOS 12.7.1. HOT 1
- Latest release hangs on startup (macOS 12.7.4) HOT 8
- Newline in attribution no longer properly handled HOT 2
- Compose Menu - top bar not updated after attaching files from file browser HOT 3
- Subject with non-printable characters or new line is shown with `?` HOT 30
- `Today` conditional date broken HOT 4
- Lag on half-down/half-up scrolling on macOS HOT 5
- Do we need to handle <stddef.h> specially? HOT 1
- conditional format strings broken in 2024-04-25 HOT 8
- Duplicate gpg keys in key selection menu for message encryption HOT 18
- segfault while opening a mailbox in from the sidebar HOT 2
- Doubts about paranoic defensive programming style HOT 3
- C dialect HOT 2
- tag-prefix doesn't show-up in attach HOT 5
- Doubts about some `*const` pointers HOT 5
- How to use the notmuch command <modify-labels> to tag emails with tags containing spaces?
- Is it possible to edit the automatic title when forwarding or replying a message? HOT 1
- Progress for postponing a message with remote (IMAP)
- autocrypt breaks command line argument `-e "push foo"` HOT 4
- How to forward an email thread to someone else together with attachments ? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from neomutt.