Request: Support for per-thread function stats about yappi HOT 5 CLOSED

You mean the thread stats retrieved by get_thread_stats() or per-thread function stats? pstats is a specific class designed for function profiling stats and thus have fields for those, our current thread statistics return information that is irrelevant(such as sched_cnt...) to a pstats object.

from yappi.

I meant can we get per-thread function stats in pstats format(as we get by calling yappi.convert2pstats(yappi.get_func_stats())). Right now, per thread information is limited.

My code is something like this

def start_job():
       yappi.start()
       main_func()         #main thread
       func_pstats = yappi.convert2pstats(yappi.get_func_stats())
       func_pstats.dump_stats(funcfile)
       thstats = yappi.get_thread_stats()

def main_func():
     # do some processing
     # spawns n threads. Each thread processes items from queue pushed by main thread
     # push items to main thread q.put()
     # threads.join()

I am looking for detailed function stats per thread to identlify where most of the time is spent by them.

Updated subject to make it clearer

from yappi.

I believe this has been implemented while ago: after getting func_stats(), you can filter on a specific thread id. yappi.get_func_stats(filter={"ctx_id":<thread id>, })

from yappi.

The documentation is incomplete for this and there are other problems, too like no example is given on how to obtain context id of a thread and the reason why we don't use thread_id in the filter. We will look into those but in the mean time you can use set_context_id_callback() function to set/get thread id on the internal context object then use that id to filter on the threads.

from yappi.

I have updated the documentation. https://github.com/sumerc/yappi/blob/master/doc/api.md
You can especially look at the get_func_stats() implementation.

The answer to your question is you can do following:
func_pstats = yappi.convert2pstats(yappi.get_func_stats(filter={'ctx_id': }))

To get the context id of a thread:
threads = yappi.get_thread_stats()
Now in threads you can both look at the thread_id,name given by OS whatever you want(I do not know how you identify your thread), and get the associated context id.

from yappi.