Hi Author，
Could you please give me a detailed introduction on how to visualization?

look forward to your reply.

I have installed a Conda environment with the dependencies as written on the install page. However, when I try to run a test I get these errors:

(openmmlab) root@xyz:/FB-BEV# python tools/test.py --config log/weights/fbocc-r50-cbgs_depth_16f_16x4_20e.py --checkpoint log/weights/fbocc-r50-cbgs_depth_16f_16x4_20e.pth --out test/test.pkl
Traceback (most recent call last):
File "tools/test.py", line 303, in
main()
File "tools/test.py", line 155, in main
cfg = Config.fromfile(args.config)
File "/opt/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmcv/utils/config.py", line 340, in fromfile
cfg_dict, cfg_text = Config._file2dict(filename,
File "/opt/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmcv/utils/config.py", line 208, in _file2dict
mod = import_module(temp_module_name)
File "/opt/anaconda3/envs/openmmlab/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 843, in exec_module
File "", line 219, in _call_with_frames_removed
File "/tmp/tmp2fp83puh/tmpcfk_b6pr.py", line 1, in
NameError: name 'false' is not defined
Exception ignored in: <function _TemporaryFileCloser.del at 0x7f251c1f88b0>
Traceback (most recent call last):
File "/opt/anaconda3/envs/openmmlab/lib/python3.8/tempfile.py", line 440, in del
self.close()
File "/opt/anaconda3/envs/openmmlab/lib/python3.8/tempfile.py", line 436, in close
unlink(self.name)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp2fp83puh/tmpcfk_b6pr.py'

Do you have any idea what the solution could be?
This is my pip list:

Package Version Editable project location

absl-py 1.4.0
addict 2.4.0
anyio 4.0.0
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.2.3
asttokens 2.4.0
async-lru 2.0.4
attrs 23.1.0
Babel 2.12.1
backcall 0.2.0
beautifulsoup4 4.12.2
black 23.9.1
bleach 6.0.0
cachetools 5.3.1
ccimport 0.4.2
certifi 2023.7.22
cffi 1.15.1
charset-normalizer 3.2.0
click 8.1.7
cmake 3.27.4.1
comm 0.1.4
contourpy 1.1.0
cumm-cu114 0.4.11
cycler 0.11.0
debugpy 1.7.0
decorator 5.1.1
defusedxml 0.7.1
descartes 1.1.0
exceptiongroup 1.1.3
executing 1.2.0
fastjsonschema 2.18.0
filelock 3.12.3
fire 0.5.0
flake8 6.1.0
fonttools 4.42.1
fqdn 1.5.1
google-auth 2.23.0
google-auth-oauthlib 1.0.0
grpcio 1.58.0
idna 3.4
imageio 2.31.3
importlib-metadata 6.8.0
importlib-resources 6.0.1
iniconfig 2.0.0
ipykernel 6.25.2
ipython 8.12.2
ipython-genutils 0.2.0
ipywidgets 8.1.0
isoduration 20.11.0
jedi 0.19.0
Jinja2 3.1.2
joblib 1.3.2
json5 0.9.14
jsonpointer 2.4
jsonschema 4.19.0
jsonschema-specifications 2023.7.1
jupyter 1.0.0
jupyter_client 8.3.1
jupyter-console 6.6.3
jupyter_core 5.3.1
jupyter-events 0.7.0
jupyter-lsp 2.2.0
jupyter_server 2.7.3
jupyter_server_terminals 0.4.4
jupyterlab 4.0.5
jupyterlab-pygments 0.2.2
jupyterlab_server 2.24.0
jupyterlab-widgets 3.0.8
kiwisolver 1.4.5
lark 1.1.7
lit 16.0.6
llvmlite 0.36.0
lyft-dataset-sdk 0.0.8
Markdown 3.4.4
MarkupSafe 2.1.3
matplotlib 3.5.2
matplotlib-inline 0.1.6
mccabe 0.7.0
mistune 3.0.1
mmcls 0.25.0
mmcv-full 1.5.2
mmdet 2.24.0
mmdet3d 1.0.0rc4 /FB-BEV
mmsegmentation 0.24.0
mpmath 1.3.0
mypy-extensions 1.0.0
nbclient 0.8.0
nbconvert 7.8.0
nbformat 5.9.2
nest-asyncio 1.5.7
networkx 2.2
ninja 1.11.1
notebook 7.0.3
notebook_shim 0.2.3
numba 0.53.0
numpy 1.20.3
nuscenes-devkit 1.1.10
nvidia-cublas-cu11 11.10.3.66
nvidia-cuda-cupti-cu11 11.7.101
nvidia-cuda-nvrtc-cu11 11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11 8.5.0.96
nvidia-cufft-cu11 10.9.0.58
nvidia-curand-cu11 10.2.10.91
nvidia-cusolver-cu11 11.4.0.1
nvidia-cusparse-cu11 11.7.4.91
nvidia-nccl-cu11 2.14.3
nvidia-nvtx-cu11 11.7.91
oauthlib 3.2.2
opencv-python 4.8.0.76
overrides 7.4.0
packaging 23.1
pandas 2.0.3
pandocfilters 1.5.0
parso 0.8.3
pathspec 0.11.2
pccm 0.4.8
pexpect 4.8.0
pickleshare 0.7.5
Pillow 10.0.0
pip 23.2.1
pkgutil_resolve_name 1.3.10
platformdirs 3.10.0
plotly 5.16.1
pluggy 1.3.0
plyfile 1.0.1
portalocker 2.7.0
prettytable 3.9.0
prometheus-client 0.17.1
prompt-toolkit 3.0.39
protobuf 4.24.3
psutil 5.9.5
ptyprocess 0.7.0
pure-eval 0.2.2
pyasn1 0.5.0
pyasn1-modules 0.3.0
pybind11 2.11.1
pycocotools 2.0.7
pycodestyle 2.11.0
pycparser 2.21
pyflakes 3.1.0
Pygments 2.16.1
pyparsing 3.1.1
pyquaternion 0.9.9
pytest 7.4.2
python-dateutil 2.8.2
python-json-logger 2.0.7
pytz 2023.3.post1
PyWavelets 1.4.1
PyYAML 6.0.1
pyzmq 25.1.1
qtconsole 5.4.4
QtPy 2.4.0
referencing 0.30.2
requests 2.31.0
requests-oauthlib 1.3.1
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rpds-py 0.10.2
rsa 4.9
scikit-image 0.19.3
scikit-learn 1.3.0
scipy 1.10.1
Send2Trash 1.8.2
setuptools 68.0.0
Shapely 1.8.5
six 1.16.0
sniffio 1.3.0
soupsieve 2.5
spconv-cu114 2.3.6
stack-data 0.6.2
sympy 1.12
tenacity 8.2.3
tensorboard 2.14.0
tensorboard-data-server 0.7.1
termcolor 2.3.0
terminado 0.17.1
terminaltables 3.1.10
threadpoolctl 3.2.0
tifffile 2023.7.10
tinycss2 1.2.1
tomli 2.0.1
torch 2.0.1
torchvision 0.15.2
tornado 6.3.3
tqdm 4.66.1
traitlets 5.9.0
trimesh 2.35.39
triton 2.0.0
typing_extensions 4.7.1
tzdata 2023.3
uri-template 1.3.0
urllib3 1.26.16
wcwidth 0.2.6
webcolors 1.13
webencodings 0.5.1
websocket-client 1.6.3
Werkzeug 2.3.7
wheel 0.38.4
widgetsnbextension 4.0.8
yapf 0.40.1
zipp 3.16.2

How do you calculate the latency of FB-BEV, F-VTM + B-VTM + FRPN？Thanks!

Will this open source 3D detection code and config?

Thanks for your excellent works!

When I test with dist_test.sh, the memory continues to increase, causing the evaluation to stop.
The same thing happens when fbocc.py's pred_occcupancy_category = pred_occcupancy_category.cpu().numpy() is performed without changing to cpu.
I wonder if there were any problems like this, do you know how to solve them?

An error was reported when I ran the occ model given by o official, Loading error, detailed as follows:
run : python ./tools/test.py /zz/code/FB-BEV/occupancy_configs/fb_occ/fbocc-r50-cbgs_depth_16f_16x4_20e.py /zz/code/FB-BEV-main/fbocc-r50-cbgs_depth_16f_16x4_20e.pth

raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for FBOCC:
Unexpected key(s) in state_dict: "img_sem_seg_head.seg_net.0.weight", "img_sem_seg_head.seg_net.0.bias", "img_sem_seg_head.seg_net.3.weight", "img_sem_seg_head.seg_net.3.bias".

Hi, I want to why we need to apply the below transformations in LoadOccupancy function?

# to BEVDet format
occupancy = occupancy.permute(2, 0, 1)  # to (16, 200, 200)
occupancy = torch.rot90(occupancy, 1, [1, 2])
occupancy = torch.flip(occupancy, [1])
occupancy = occupancy.permute(1, 2, 0)

Although the comment says they are used to transform the occupancy to BEVDet format, they are not exist in the BEVDet codebase.

Can anybody help to understand the purposes of these operations?

Is the simulator that turns this source code on the nvidia drive sim? Is the nvidia drive sim the only thing that needs to be installed separately?

Hello,

I tried to run the inference code and faced this error "ModuleNotFoundError: No module named 'spconv'"
although I did the installation steps successfully. I would appreciate your input for solving this issue.

Thank you in advance for your help!

Anyone else having problems with the evaluation?

my results are really bad:
others,barrier,bicycle,bus,car,construction_vehicle,motorcycle,pedestrian,traffic_cone,trailer,truck,driveable_surface,other_flat,sidewalk,terrain,manmade,vegetation,Overall
0.0,0.01,0.0,0.0,0.0,0.04,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,3.78,0.0,0.23

I had to upgrade the mmcv-full to 1.6.0 because of the following error:

File "./tools/test.py", line 303, in <module>
    main()
  File "./tools/test.py", line 272, in main
    outputs = custom_multi_gpu_test(model, data_loader, args.tmpdir,
  File "/FB-BEV/mmdet3d/apis/test.py", line 164, in custom_multi_gpu_test
    result = model(return_loss=False, rescale=True, **data)
  File "/root/miniconda3/envs/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/miniconda3/envs/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1008, in forward
    output = self._run_ddp_forward(*inputs, **kwargs)
  File "/root/miniconda3/envs/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 969, in _run_ddp_forward
    return module_to_run(*inputs[0], **kwargs[0])
  File "/root/miniconda3/envs/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/miniconda3/envs/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 140, in new_func
    output = old_func(*new_args, **new_kwargs)
  File "/FB-BEV/mmdet3d/models/detectors/base.py", line 62, in forward
    return self.forward_test(**kwargs)
  File "/FB-BEV/mmdet3d/models/fbbev/detectors/fbocc.py", line 487, in forward_test
    if num_augs==1 and not img_metas[0][0
].get('tta_config', dict(dist_tta=False))['dist_tta']:
TypeError: 'DataContainer' object is not subscriptable

Here is my Dockerfile, maybe it helps:
Dockerfile.txt

I would like to express my gratitude for your excellent research. I had always been curious about whether using both forward and backward projections for BEV representation would enhance its performance. I had contemplated taking this up as a research topic. I was particularly astounded by the improvement in performance when a module for depth was added during the backward projection.

One aspect I'm curious about is the experiment with ResNet-50 mentioned in your paper. From what I observed, the FLOPS are lower compared to when BEVDepth is reproduced. I anticipate that to achieve this, one cannot use the BEVBackbone + neck portion utilized in BEVDepth. Could you confirm if my assumption is correct?

Thank you.

Thanks for great work，how to use tools/ vis_occupancy.py to show the result? it looks like has bug. I don't know where the problem is.
the inputs shape is 200 * 200 * 18

Can you share a visual script?
thanks

what does "fix_void" mean? In config, fix_void=num_class==19. And in model pred_occupancy = pred_occupancy[..., 1:] when fix_void is True. So what does it mean?

Hello,
I have completed the model training phase , but an error occurred during the evaluation. The specific information is as follows：

warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
/opt/conda/envs/fbocc/lib/python3.8/site-packages/torch/utils/checkpoint.py:25: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
/opt/conda/envs/fbocc/lib/python3.8/site-packages/torch/utils/checkpoint.py:25: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
[                                                  ] 4/6019, 0.4 task/s, elapsed: 9s, ETA: 14005s/opt/conda/envs/fbocc/lib/python3.8/site-packages/torch/utils/checkpoint.py:25: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 6084/6019, 21.4 task/s, elapsed: 284s, ETA:    -2sWARNING:torch.distributed.elastic.multiprocessing.api:Sending process 795432 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 795433 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 795434 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 3 (pid: 795435) of binary: /opt/conda/envs/fbocc/bin/python
Traceback (most recent call last):
  File "/opt/conda/envs/fbocc/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/envs/fbocc/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/root/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/root/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/root/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 317, in run_module
    run_module_as_main(options.target, alter_argv=True)
  File "/root/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 238, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/root/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/opt/conda/envs/fbocc/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/opt/conda/envs/fbocc/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/opt/conda/envs/fbocc/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/opt/conda/envs/fbocc/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run
    elastic_launch(
  File "/opt/conda/envs/fbocc/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/opt/conda/envs/fbocc/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
tools_mm/test.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-08-30_01:11:31
  host      : fd-taxi-f-houjiawei-1691719963348-985c0990-3609275187
  rank      : 3 (local_rank: 3)
  exitcode  : -9 (pid: 795435)
  error_file: <N/A>
  traceback : Signal 9 (SIGKILL) received by PID 795435
============================================================

The evaluation run 6084 samples rather than 6019 samples and closed unexpectly.

Has anyone tried to export any of the FB-OCC or FB-BEV model to ONNX format? Is it even possible?

Is this model able to convert to tensorrt engine?

The result is awesome, I'm wondering when the code of FB-BEV will be release? thank you

May I know the detailed results of the FB-OCC on the occ task with InternImage-H backbone?
Actually, the Version-K in your report, could u provide the result without test-time augmentation?
This will be very helpful to me, thank you!

请问【LoadBEVMask】 具体是如何实现的

gt-mask画在128 x 128 的bev网格时，由于small-gt 在画mask时需要将float类型的bbox取整，出现偏移的问题，这个偏移对小目标而言是较大的。

请问作者是如何克服这个问题的。

Hello,
When trying to evaluate the model I get the following error:

ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 887) of binary: /opt/conda/bin/python
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/run.py", line 755, in run
    )(*cmd_args)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 247, in launch_agent
    failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
./tools/test.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-01-16_18:10:10
  host      : adas-server
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 887)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

As far as I know, this error happens due to insufficient memory in a computer. Right now, I have around 45 GBs of memory available and am not able to increase it for the time being, but I read it is possible to avoid it by changing the batch size. Unfortunately, I cannot seem to figure out how exactly to do that.
Any help would be greatly appreciated!

File "FB-BEV/mmdet3d/models/fbbev/detectors/fbocc.py", line 486, in forward_test
if num_augs==1 and not img_metas[0][0].get('tta_config', dict(dist_tta=False))['dist_tta']:
TypeError: 'DataContainer' object is not subscriptable

FB-OCC (39.1 mIoU)所有类别的IoU可以分享下吗？谢谢~

Thank you for your great work! I noticed that all 3d heads are forced to use fp32. Will there be performance drop if we use amp instead?

Thanks so much for sharing!

I would like to ask if the resnet50 used in Table 1 is the imagenet pre-trained or other data? How many epochs were trained for the models in Table 1?

Dear Author,

Thank you for your great work and sharing.
I have followed all the preparation steps and run the code through python tools/train.py ./configs/occupancy/bevdet4d-r50-cbgs_depth_cool.py and I got the following error message: KeyError: 'NewBEV4D is not in the models registry'

Which config files should I run?

Thank you.

I have tried exporting the onnx file of FB-OCC, but I face the following error during tracing at the custom op of QuickCumsumCuda specifically when torch.onnx.export while the feed-forward inference of the model does not have any issue:

File "/FB-BEV/mmdet3d/ops/bev_pool_v2/bev_pool.py", line 102, in forward_dummy
    x = QuickCumsumCuda.apply(depth, feat, ranks_depth, ranks_feat, ranks_bev, bev_feat_shape, interval_starts, interval_lengths)
RuntimeError: _Map_base::at

This is how the error can be reproduced:

1. I had isolated the custom op QuickCumsumCuda in a separate class function as showin the following for the ease of reproducibility:

class Bev_Pool_v2(torch.nn.Module):
    def __init__(self):
        super(Bev_Pool_v2, self).__init__()
    def forward(self, depth, feat, ranks_depth, ranks_feat, ranks_bev, bev_feat_shape, interval_starts, interval_lengths):
        x = QuickCumsumCuda.apply(depth, feat, ranks_depth, ranks_feat, ranks_bev, bev_feat_shape, interval_starts, interval_lengths)
        x = x.permute(0, 4, 1, 2, 3).contiguous()
        return x
    def forward_dummy(self, data):
        depth, feat, ranks_depth, ranks_feat, ranks_bev, bev_feat_shape, interval_starts, interval_lengths = data
        x = QuickCumsumCuda.apply(depth, feat, ranks_depth, ranks_feat, ranks_bev, bev_feat_shape, interval_starts, interval_lengths)    
        x = x.permute(0, 4, 1, 2, 3).contiguous()
        return x

2. I generate/feed-forward the random inputs, which does not yield any issue during model inference.

# Random Generations of Inputs
depth = torch.rand(1, 6, 80, 16, 44).cuda()
feat = torch.rand(1, 6, 80, 16, 44).cuda()
ranks_depth = torch.randint(0, 337522, (206988, )).to(dtype=torch.int32).cuda()
ranks_feat = torch.randint(0, 4223, (206988, )).to(dtype=torch.int32).cuda()
ranks_bev = torch.randint(0, 79972, (206988, )).to(dtype=torch.int32).cuda()
bev_feat_shape = (1, 8, 100, 100, 80)
interval_starts = torch.randint(0, 79972, (52815, )).to(dtype=torch.int32).cuda()
interval_lengths = torch.randint(0, 213, (52815, )).to(dtype=torch.int32).cuda()

# Define the model and the input
model = Bev_Pool_v2().eval().cuda()
model.forward = model.forward_dummy
input_ = [depth, feat, ranks_depth, ranks_feat, ranks_bev, 
    bev_feat_shape, interval_starts, interval_lengths]

# Feed the input to the model
model(input_)
print('feed-forward inference is done without errors.')

3. Yet, the error mentioned above appears when exporting the model.

with torch.no_grad():
    torch.onnx.export(
        model,
        input_,
        'bev_pool_v2_USE.onnx',
        # export_params=True,
        keep_initializers_as_inputs=True,
        do_constant_folding=False,
        verbose=True,
        opset_version=13
    )

Despite exploring various solutions, I have yet to resolve this error.

In backward projection, context is used as the transform key. Is there a reason to use context instead of image feature? I don't know what the difference is between image feature and context.

Thanks for sharing this wonderful work.

I wonder why setting do_history=False in the configs. Does this mean we have not used the temporal fusion strategy? And I also have tried to set do_history=True and train the same setting, the experimental results are almost the same. I don't know why.

Could anyone help me?

Following the install instructions, numpy>=1.24.0 will be installed at this point. Which is somehow incompatible with numba==0.53.0.
Install earlier numpy version can solve this problem.

Hello zhiqi. How to train a depth pretrained r50+depthnet that is used in your occupancy model?

I conducted an ablation study where I set 'backward_projection' to None (while keeping other configurations at their default settings). The results showed that the availability of this module did not have a significant impact on the results. Here are the results for epochs 2 and 4:

Can you please share any ablation results you may have for this module?

Traceback (most recent call last):
File "./tools/test.py", line 15, in
from mmdet3d.apis import single_gpu_test
File "/home/jl/FB-BEV/mmdet3d/apis/init.py", line 2, in
from .inference import (convert_SyncBN, inference_detector,
File "/home/jl/FB-BEV/mmdet3d/apis/inference.py", line 12, in
from mmdet3d.core import (Box3DMode, CameraInstance3DBoxes, Coord3DMode,
File "/home/jl/FB-BEV/mmdet3d/core/init.py", line 4, in
from .evaluation import * # noqa: F401, F403
File "/home/jl/FB-BEV/mmdet3d/core/evaluation/init.py", line 4, in
from .kitti_utils import kitti_eval, kitti_eval_coco_style
File "/home/jl/FB-BEV/mmdet3d/core/evaluation/kitti_utils/init.py", line 2, in
from .eval import kitti_eval, kitti_eval_coco_style
File "/home/jl/FB-BEV/mmdet3d/core/evaluation/kitti_utils/eval.py", line 5, in
import numba
File "/home/jl/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/numba/init.py", line 43, in
from numba.np.ufunc import (vectorize, guvectorize, threading_layer,
File "/home/jl/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/numba/np/ufunc/init.py", line 3, in
from numba.np.ufunc.decorators import Vectorize, GUVectorize, vectorize, guvectorize
File "/home/jl/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/numba/np/ufunc/decorators.py", line 3, in
from numba.np.ufunc import _internal
SystemError: initialization of _internal failed without raising an exception
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 34725) of binary: /home/jl/anaconda3/envs/open-mmlab/bin/python
Traceback (most recent call last):
File "/home/jl/anaconda3/envs/open-mmlab/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/jl/anaconda3/envs/open-mmlab/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/jl/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in
main()
File "/home/jl/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/home/jl/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/home/jl/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run
elastic_launch(
File "/home/jl/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/jl/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

./tools/test.py FAILED

Congratulations and thanks for sharing this great work!
I wonder when the code will be requested! Could you please estimate the release date?
Thanks

Can the author provide a log file for training the model so that we can correspond to the training situation at each stage？