Comments (4)
First, you need to extract the video features and then feed these features to the model.
from rmn.
About extracting video features, I have extracted i3d features for my video.
Which layer of InceptionResnetV2 should be used for extracting frame features?
Also for BUTD, I referred the issue and it says the script is modified, so is the new script sufficient for usage?
from rmn.
- IRV2 extractor:
class AppearanceEncoder_inceptionresnetv2(nn.Module):
def __init__(self):
super(AppearanceEncoder_inceptionresnetv2, self).__init__()
IRV2 = InceptionResNetV2(num_classes=1001)
# print('IRV2:\n', IRV2)
IRV2.load_state_dict(torch.load(opt.IRV2_checkpoint))
modules = list(IRV2.children())[:-1] # delete the last fc layer.
self.IRV2 = nn.Sequential(*modules)
# print('IRV2:\n', self.IRV2)
def forward(self, images):
"""Extract feature vectors from input images."""
with torch.no_grad():
features = self.IRV2(images)
features = features.reshape(features.size(0), -1)
# print(features.size())
return features
Image preprocessing:
image -= np.array([0.5, 0.5, 0.5])
image /= np.array([0.5, 0.5, 0.5])
2.About BUTD, just feed your image to the pre-trained BUTD model and you will get 36 region features. What I modifed is preprossing MSRVTT and MSVD, and saving the extracted features to h5 file, the core code is remained.
from rmn.
Hi tgc
Great work! Thanks for sharing
Would like to ask if you could share your codes on feature extraction using i3d, irv2 and butd? Im not sure how to extract these video features.
Thank you
from rmn.
Related Issues (20)
- hi! would like to know how to resolve the following issue
- hi! would like to know how to get these
- the mismatch error happened when using the pretarined model you provide. HOT 5
- a bug report HOT 4
- result reproduce for msr-vtt dataset HOT 9
- TypeError: h5py objects cannot be pickled
- Spatial Feats HOT 3
- What's the range of cider score?
- POS
- The link of visual and text features cannot be opened HOT 1
- a problem about region_feature file HOT 3
- A refinement report
- text feature processing HOT 1
- Would I ask one question?
- When I tried to run evaluatie.py it reported the function incorrectly HOT 2
- a problem about msr-vtt_model.pth HOT 1
- a problem about sample.py
- a problem about features HOT 2
- problems about feature extraction models
- Problem about Feature Extraction HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rmn.