GPU vs CPU which one is best? about llama-chat HOT 2 OPEN

this does't make sense to me as i have seen the opposite, cpu is super slow and only gpu can truly speed things up. llamaccp however is the faster cpu version but my query times talking to my own embedded data is still 1-3 mins per query

from llama-chat.

I have a question on memory (and I also observe very slow response using a GPU). On my Dell Precision 7780 workstation laptop with the 7B model I see the Nvidia GPU mem goes up to about 3GB out of the available 6GB. I assume this is due to 4-bit quantization? BUT, the CPU memory shoots up to almost 95% of my 32GB so it appears it is both shipping to GPU and CPU at the same time? Looking at the code I see in model.py these lines:

    for layer in tqdm(self.layers, desc="flayers", leave=True):
        if use_gpu:
            move_parameters_to_gpu(layer)
        h = layer(h, start_pos, freqs_cis, mask)
        if use_gpu:
            **move_parameters_to_cpu(layer)**

Why whould it move parameters to CPU in that if use_gpu statement?

from llama-chat.