christy1206 / ststnet Goto Github PK

Hi , could you please the link of the data you used. Other link is broken. I cannot wait to run and see the results. Thank you.

为什么我用给出的光流代码得到的光流图再进行实验，得到的UF1和UAR只有0.68左右呢？SAMM数据集我经过了裁剪，得到的光流特征没有已知数据集的光流特征更明显，请问是经过了什么处理吗?

May I ask why Python main.py -- train False appears after running it
X_ Train=torch Tensor (X_train). permute (0,3,1,2)
RuntimeError: number of dims don't match in permute
This is the result of the operation：
D:\STSTNet-master> python main.py --train False
lr=0.000050, epochs=800, device=cpu

Subject: 006
Traceback (most recent call last):
File "main.py", line 233, in
main(config)
File "main.py", line 153, in main
X_train = torch.Tensor(X_train).permute(0,3,1,2)
RuntimeError: number of dims don't match in permute

Dear author, it is my great honor to star your project. I would like to ask you how to use your project to recognize micro-expressions in videos or pictures

hi,where can I get the code of D&C-RoIs?

在进行LOSO交叉验证时，由于测试集比较小，得到的结果会不会存在偶然性呢？

When I use Google GPU to run Python code, I encounter this problem. How can I solve it? Hope to get your reply

lr=0.000050, epochs=800, device=cuda

Subject: sub25
Predicted : [0, 0, 0, 2, 2]
Ground Truth : [0, 0, 0, 2, 2]
Evaluation until this subject:
UF1: 1.0 | UAR: 1.0
Subject: sub26
Predicted : [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Ground Truth : [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Evaluation until this subject:
UF1: 0.641 | UAR: 0.6667
Subject: sub20
Predicted : [0, 0]
Ground Truth : [0, 0]
Evaluation until this subject:
UF1: 0.6444 | UAR: 0.6667
Subject: sub21
Predicted : [1]
Ground Truth : [0]
Evaluation until this subject:
UF1: 0.6344 | UAR: 0.6444
Subject: sub24
Predicted : [2, 0, 0]
Ground Truth : [2, 0, 0]
Evaluation until this subject:
UF1: 0.6381 | UAR: 0.6471
Subject: sub22
Predicted : [0, 0]
Ground Truth : [0, 0]
Evaluation until this subject:
UF1: 0.641 | UAR: 0.6491
Subject: sub16
Predicted : [0, 1, 0]
Ground Truth : [1, 1, 0]
Evaluation until this subject:
UF1: 0.746 | UAR: 0.7333
Subject: sub19
Predicted : [2, 2, 2, 2, 2, 0, 0, 0, 1, 1, 1]
Ground Truth : [2, 2, 2, 2, 2, 0, 0, 0, 1, 1, 1]
Evaluation until this subject:
UF1: 0.8611 | UAR: 0.8427
Subject: sub17
Predicted : [2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1]
Ground Truth : [2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1]
Evaluation until this subject:
UF1: 0.9018 | UAR: 0.8903
Subject: sub23
Predicted : [0, 0, 0, 0, 0, 1, 0, 0]
Ground Truth : [0, 0, 0, 0, 0, 0, 0, 1]
Evaluation until this subject:
UF1: 0.8801 | UAR: 0.87
Subject: sub15
Predicted : [0, 2, 0]
Ground Truth : [1, 2, 0]
Evaluation until this subject:
UF1: 0.8695 | UAR: 0.8565
Subject: sub14
Predicted : [0, 1, 1]
Ground Truth : [1, 1, 1]
Evaluation until this subject:
UF1: 0.8722 | UAR: 0.8587
Subject: sub12
Predicted : [0, 0, 1, 0, 0, 1, 1, 2, 2, 2, 2]
Ground Truth : [0, 0, 0, 0, 0, 1, 1, 2, 2, 2, 2]
Evaluation until this subject:
UF1: 0.8757 | UAR: 0.8663
Subject: sub08
Predicted : [1]
Ground Truth : [0]
Evaluation until this subject:
UF1: 0.8672 | UAR: 0.8611
Subject: sub13
Predicted : [0, 1]
Ground Truth : [1, 1]
Evaluation until this subject:
UF1: 0.864 | UAR: 0.8563
Subject: sub09
Predicted : [0, 0, 0, 0, 0, 1, 0, 1, 1, 1]
Ground Truth : [0, 0, 0, 0, 0, 1, 1, 1, 1, 1]
Evaluation until this subject:
UF1: 0.8754 | UAR: 0.8672
Subject: sub07
Predicted : [1, 0, 0, 0, 0]
Ground Truth : [0, 0, 0, 0, 0]
Evaluation until this subject:
UF1: 0.8707 | UAR: 0.8643
Subject: sub06
Predicted : [2, 2, 0, 0]
Ground Truth : [2, 2, 1, 0]
Evaluation until this subject:
UF1: 0.8647 | UAR: 0.8569
Subject: sub11
Predicted : [0, 0, 0, 0]
Ground Truth : [0, 0, 0, 0]
Evaluation until this subject:
UF1: 0.8666 | UAR: 0.8584
Subject: sub05
Predicted : [2, 2, 0, 2, 2, 1]
Ground Truth : [2, 2, 2, 2, 2, 1]
Evaluation until this subject:
UF1: 0.8601 | UAR: 0.8463
Subject: sub04
Predicted : [0, 1]
Ground Truth : [0, 0]
Evaluation until this subject:
UF1: 0.8545 | UAR: 0.8427
Subject: sub03
Predicted : [0, 0, 0, 0, 2]
Ground Truth : [0, 0, 0, 0, 2]
Evaluation until this subject:
UF1: 0.8567 | UAR: 0.8449
Subject: s20
Traceback (most recent call last):
File "main.py", line 232, in
main(config)
File "main.py", line 152, in main
X_train = torch.Tensor(X_train).permute(0,3,1,2)
RuntimeError: number of dims don't match in permute

Hi!
I tried to repeat data preprocessing according to one in your paper. I've failed to get equal results.
I worked with CASME2 dataset and took cropped faces from CASME2_preprocessed_small_Li Xiaobai/Cropped.
I used the following code mainly from https://github.com/genbing99/SoftNet-SpotME repository:

import numpy as np
import pandas as pd
import cv2

def pol2cart(rho, phi): #Convert polar coordinates to cartesian coordinates for computation of optical strain
    x = rho * np.cos(phi)
    y = rho * np.sin(phi)
    return (x, y)

def computeStrain(u, v):
    u_x= u - pd.DataFrame(u).shift(-1, axis=1)
    v_y= v - pd.DataFrame(v).shift(-1, axis=0)
    u_y= u - pd.DataFrame(u).shift(-1, axis=0)
    v_x= v - pd.DataFrame(v).shift(-1, axis=1)
    os = np.array(np.sqrt(u_x**2 + v_y**2 + 1/2 * (u_y+v_x)**2).ffill(1).ffill(0))
    return os

def resize(img, target_size=(28,28)):
    rimg = cv2.resize(img, (*target_size,), interpolation=cv2.INTER_LINEAR)
    return rimg

def perChannelNormalize(img):
    res = img.copy()
    for chnl in range(img.shape[2]):
        channel = res[:,:,chnl]
        res[:,:,chnl] = 255.*(channel - channel.min())/(channel.max() - channel.min())
    return res.astype(np.uint8)

def get_optical_flow(img1, img2):
    #Compute Optical Flow Features
    assert img1.shape == img2.shape
    # optical_flow = cv2.DualTVL1OpticalFlow_create() #Depends on cv2 version
    optical_flow = cv2.optflow.DualTVL1OpticalFlow_create(
        tau = 0.25, 
        #lambda_ = 0.15,
        theta = 0.3,
        nscales = 5, 
        warps=5,
        epsilon=0.01, 
        innnerIterations=30,
        outerIterations=10, 
        scaleStep=0.5,
        gamma=0.1,
        medianFiltering=5,
        useInitialFlow=False
    )
    flow = optical_flow.calc(img1, img2, None)
    magnitude, angle = cv2.cartToPolar(flow[..., 0], flow[..., 1]) 
    u, v = pol2cart(magnitude, angle)
    os = computeStrain(u, v)
                
    #Features Concatenation
    final = np.zeros((*img1.shape[0:2],3))
    final[:,:,0] = os #B
    final[:,:,1] = v #G
    final[:,:,2] = u #R

    return final

if __name__ == "__main__":
    path1 = "img1.jpg"
    path2 = "img2.jpg"
    img1 = cv2.imread(path1, 0)
    img2 = cv2.imread(path2, 0)
    opfl = get_optical_flow(img1, img2) # get (u,v,os)
    opfl = resize(opfl) # resize to 28x28
    opfl = perChannelNormalize(opfl) # normalize each channel to 0..255 

Help me please to get correct results!

Hi, I am wondering how to preprocess the raw data and transform them into norm_u_v_os.
Can you give me some advice or the preprocess code to this?