Request: solving the lack of incremental reporting in loops / functions about pytorch_memlab HOT 2 OPEN

For functions, yes, @profile_every(1) does the trick, thank you! And would it be possible to add a limit? Say my function runs 100 times, and I want to see it's 1st, 2nd, and last time and not deal with a huge output which is not needed? Perhaps @profile_these_times(1,2,-1) ? Or if it's too complicated, perhaps just adding a stop as in @profile_every(1,3) == profile every run and stop after 3 times.

for loops, no, we want to do the same as the function, so that the body of the loop could be profiled. That is we write it symbolically:

     @profile_every(1) # not legal python
     for _ in range(10): 
        x = net(x)

output:

        x = net(x)  # first hit
        x = net(x)  # 2nd hit
        x = net(x)  # 3rd hit
        [...]
        x = net(x)  # 10th hit

I hope I was able to explain in the OP how one doesn't get the incremental info when the loop is repeated multiple times.

One workaround would be to turn the body of the loop into a function and profile it, but that may require significant alternations to the user's code.

from pytorch_memlab.

Hi Stas,

Thanks for your detailed feature description.

I would like to propose a sample to make sure I get the point.

statement unrolling:

Suppose I have such a funtion:

@profile
def func():
    net = torch.nn.Linear(5，5)
    x = torch.Tensor(5,5)
    for _ in range(10):
        x = net(x)

The expected output should be:

def func():
    for _ in range(10):  # first hit
        x = net(x)
    for _ in range(10):  # second hit
        x = net(x)
    for _ in range(10):  # third hit (probably)
        x = net(x)
    # ...

function unrolling

If you want to profile such a function like inner:

def inner(x):
    x = net(x)
    return x

def outter(x):
    for _ in range(10):
        x = inner(x)

you can simply add @profile_every to the inner for memory stats in every iteration,
which prints something like:

def inner(x):
    x = net(x)
    return x
def inner(x):
    x = net(x)
    return x
def inner(x):
    x = net(x)
    return x
def inner(x):
    x = net(x)
    return x

from pytorch_memlab.