Comments (1)
chaunceywx
commented on June 12, 2024
Hi, thanks for the great work and the comprehensive evaluation framework. We recently wanted to add your work for comparison but found that there seems to be a problem in evaluating the quality of generated images. Please tell me if there is any misunderstanding:
As line 233 in scripts/pose2img_metrics.py shows, we saved the images as a concatenation of four images: [generated_image, pose_image, text_image, original_image]. However, in your implementation of the quality evaluation in utils/metrics/evalute_metrics_of_each_category.py, you seem to load the entire image for evaluation rather than splitting out the generated image. Please refer to line 78. We believe this will lower down the quality scores.
We tried to revise the codes in evalute_metrics_of_each_category.py by adding three lines of codes between line 77 and line 78:
img = np.array(img).reshape(512,4,512,3) \\ img = cv2.cvtColor(img[:,0,:,:], cv2.COLOR_BGR2RGB) \\ img = Image.fromarray(img)The codes above basically extracts the generated image from the saved image. Having revised this, we are able to achieve a FID of about 10 using your pretrained checkpoint (much lower than reported).Thanks and I'm looking forward to further discussion :)
Do you calculate metrics through:
python scripts/pose2img_metrics.py --outdir outputs/metrics --config utils/metrics/metrics.yaml --ckpt humansd_data/checkpoints/humansd-v1.ckpt
??
I calculate metrics in this way but encountered this error:
Loading model from humansd_data/checkpoints/higherhrnet_w48_humanart_512x512_udp.pth
Traceback (most recent call last):
File "E:\code\T2F\HumanSD-main\scripts\pose2img_metrics.py", line 260, in <module>
main()
File "E:\code\T2F\HumanSD-main\scripts\pose2img_metrics.py", line 151, in main
model = load_model_from_config(config, f"{opt.ckpt}")
File "E:\code\T2F\HumanSD-main\scripts\pose2img_metrics.py", line 57, in load_model_from_config
model = instantiate_from_config(config.model)
File "F:\conda_env\envs\HumanSD_21\lib\site-packages\omegaconf\dictconfig.py", line 353, in __getattr__
self._format_and_raise(
File "F:\conda_env\envs\HumanSD_21\lib\site-packages\omegaconf\base.py", line 190, in _format_and_raise
format_and_raise(
File "F:\conda_env\envs\HumanSD_21\lib\site-packages\omegaconf\_utils.py", line 821, in format_and_raise
_raise(ex, cause)
File "F:\conda_env\envs\HumanSD_21\lib\site-packages\omegaconf\_utils.py", line 719, in _raise
raise ex.with_traceback(sys.exc_info()[2]) # set end OC_CAUSE=1 for full backtrace
File "F:\conda_env\envs\HumanSD_21\lib\site-packages\omegaconf\dictconfig.py", line 351, in __getattr__
return self._get_impl(key=key, default_value=_DEFAULT_MARKER_)
File "F:\conda_env\envs\HumanSD_21\lib\site-packages\omegaconf\dictconfig.py", line 438, in _get_impl
node = self._get_node(key=key, throw_on_missing_key=True)
File "F:\conda_env\envs\HumanSD_21\lib\site-packages\omegaconf\dictconfig.py", line 470, in _get_node
raise ConfigKeyError(f"Missing key {key}")
omegaconf.errors.ConfigAttributeError: Missing key model
full_key: model
object_type=dict
It looks like configs in the metrics.yaml are incomplete. What configs should I add to utils/metrics/metrics.yaml? Can you give me some advice?
from humansd.
Related Issues (20)
- After clicking the 'Run' button, the resutls are not obtained and the demo seems running. Can you give me some guidances? HOT 3
- Which model ckpt of SD used in the paper for finetune, and how to load the model ckpt of SD in the main.py HOT 4
- About GHI dataset. HOT 1
- ValueError: 'cuda' is not a valid DistributedType HOT 2
- pose_ckpt HOT 1
- Memory issue when calculating FID HOT 1
- About the data quality HOT 2
- huggingface connect error HOT 3
- How to correctly install and use ldm
- 'cuda' is not a valid DistributedType HOT 1
- How to generate poses for custom images. Please guide. HOT 4
- laion human dataset download HOT 1
- How to run the task about (img+pose+text)2img HOT 2
- Questions on Evaluation Metrics HOT 5
- LAION-Human Dataset HOT 4
- About the heatmap usage HOT 1
- Downloading LAION-Human Dataset HOT 1
- questions about metrics
- About the text annotations in Human-SD
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from humansd.