A Textual-Visual-Entailment-based Unsupervised Algorithm for Cheapfake Detection

Requirements

python 3.7
cuda 11.1
pytorch 1.9.0
torchvision 0.10.0 + torchtext 0.4.0
spacy
COSMOS (included in the code, just install the requirements.txt)
OFA (included in the code, just install the requirements.txt)
detectron2 (included in the code, just install the requirements.txt)
GPU with minimum 10GB memory

Note: We recommend using Docker with GPU for easy installation.

Install with Docker

Build the image

docker build -t acmmmcheapfakes:submission .

or Pull from dockerhub

docker pull tqtnk2000/acmmmcheapfakes:submission

Run the image

docker run -v  <path to folder containing the hidden test split file test.json>:/acmmmcheapfakes/  --gpus all acmmmcheapfakes:submission > <output file>

Input / Output:

Input:

The input includes an annotation file and an image folder from the COSMOS dataset. The annotation file should include the captions and the relative path to the raw image.

Output:

The output is a dataframe with predicted labels (0 for NOOC, 1 for OOC) and a confusion matrix. The dataframe also includes a field to indicate which method is used to predict the label. Please modify the code in "acmmm.py" to save this dataframe for further inspection.

How the code run:

The main code is in the file "acmmm.py". Each of the components is written separatedly after the command print("Runnning method")

Dataset:

The COSMOS dataset:

The COSMOS dataset is not public. Please visit https://detecting-cheapfakes.github.io/ or fill in this form to get access.

Dataset folder structure:

The folder containing the test split file should look like this (for both tasks):

data
├── images_task_2            
│   ├── 2.jpg                
│   ├── 20.jpg        
│   ├── 58.jpg      
│   └── ...      
├── public_test_mmsys          
│   ├── 0.jpg
│   ├── 1.jpg
│   ├── 2.jpg  
│   └── ...          
├── task_2.json 
└── test_data.json

Please note that a JSON file containing annotation for Task 1 and 2 must be named test_data.json and task_2.json respectively.

No module named 'keras'

Hi,
I followed the readme file and pulled the image from dockerhub with: docker pull tqtnk2000/acmmmcheapfakes:submission. When I try to run the image I have the following problem:

Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/transformers/utils/import_utils.py", line 1076, in _get_module
return importlib.import_module("." + module_name, self.name)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.8/dist-packages/transformers/models/deberta/modeling_tf_deberta.py", line 32, in
from ...modeling_tf_utils import (
File "/usr/local/lib/python3.8/dist-packages/transformers/modeling_tf_utils.py", line 39, in
from keras.saving.hdf5_format import save_attributes_to_hdf5_group
ModuleNotFoundError: No module named 'keras'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "acmmm.py", line 15, in
classifier = pipeline("text-classification", model = "microsoft/deberta-xlarge-mnli", device=torch.device('cuda',0))
File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/init.py", line 727, in pipeline
framework, model = infer_framework_load_model(
File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/base.py", line 233, in infer_framework_load_model
_class = getattr(transformers_module, f"TF{architecture}", None)
File "/usr/local/lib/python3.8/dist-packages/transformers/utils/import_utils.py", line 1067, in getattr
value = getattr(module, name)
File "/usr/local/lib/python3.8/dist-packages/transformers/utils/import_utils.py", line 1066, in getattr
module = self._get_module(self._class_to_module[name])
File "/usr/local/lib/python3.8/dist-packages/transformers/utils/import_utils.py", line 1078, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.models.deberta.modeling_tf_deberta because of the following error (look up to see its traceback):
No module named 'keras'

I also tried to install keras and the version '2.10.0' seems to be correctly installed. Could you tell me if there is a way to solve it?
Thanks in advance

pwnyniche / acmmmcheapfake2022 Goto Github PK

acmmmcheapfake2022's Introduction

A Textual-Visual-Entailment-based Unsupervised Algorithm for Cheapfake Detection

Requirements

Install with Docker

Build the image

or Pull from dockerhub

Run the image

Input / Output:

Input:

Output:

How the code run:

Dataset:

The COSMOS dataset:

Dataset folder structure:

acmmmcheapfake2022's People

Contributors

Stargazers

Watchers

Forkers

acmmmcheapfake2022's Issues

Recommend Projects

Recommend Topics

Recommend Org