Comments (3)
snake-biscuits
commented on July 17, 2024
re.match also makes the list here, along side StaticProp initialisation
Likely coming from shared.Entities.__init__ parsing the entity lumps
We might be able to avoid fully parsing ENTITIES by checking the raw lump string for { & } (curly brace) indices
Pre-emptively slicing up the lump for later parsing (indexing by integer or slice should be quite fast & low cost)
Though bsp.ENTITIES.search() would then end up parsing the entire lump on first call (which is the more common indexing)
Again, I would like to defer costs as much as possible so users only pay for what they use.
SpecialLumpClasses in their current implementation do not follow this philosophy.
We should really take this approach when building .from_bytes methods for SpecialLumpClasses (see #68)
But we will first need to resolve #23 for the sake of stability (stability first, performance second)
from bsp_tool.
snake-biscuits
commented on July 17, 2024
After e9cd7ea
We no longer create ApexSPRP when loading a .bsp!
Loading is also much faster!
This only applies to respawn.titanfall2 & respawn.apex_legends rn
>>> import cProfile, pstats
>>> cProfile.run("; ".join(["import bsp_tool",
... "bsp = bsp_tool.load_bsp('.../ApexLegends/season3/3dec19/maps/mp_rr_desertlands_64k_x_64k.bsp')",
... "del bsp;", "s3we.prof")
>>> pstats.Stats("s3we.prof").sort_stats("cumulative").print_stats(20)
Tue May 9 14:51:47 2023 s3we.prof
5877186 function calls (5877138 primitive calls) in 3.456 seconds
Ordered by: cumulative time
List reduced from 271 to 20 due to restriction <20>
ncalls tottime percall cumtime percall filename:lineno(function)
2/1 0.000 0.000 3.456 3.456 {built-in method builtins.exec}
1 0.000 0.000 3.456 3.456 <string>:1(<module>)
1 0.000 0.000 3.456 3.456 bsp_tool\__init__.py:48(load_bsp)
1 0.000 0.000 2.759 2.759 bsp_tool\respawn.py:158(__init__)
1 0.000 0.000 2.759 2.759 bsp_tool\valve.py:23(__init__)
1 0.000 0.000 2.759 2.759 bsp_tool\base.py:26(__init__)
1 0.001 0.001 2.759 2.759 bsp_tool\respawn.py:163(_preload)
6 0.508 0.085 2.106 0.351 bsp_tool\branches\shared.py:68(from_bytes)
950037 0.348 0.000 1.023 0.000 Python310\lib\re.py:187(match)
68 0.654 0.010 0.654 0.010 {built-in method nt.stat}
2 0.000 0.000 0.644 0.322 Python310\lib\genericpath.py:16(exists)
1355876 0.390 0.000 0.564 0.000 Python310\lib\re.py:288(_compile)
128 0.000 0.000 0.510 0.004 bsp_tool\valve.py:26(_preload_lump)
405707 0.156 0.000 0.480 0.000 Python310\lib\re.py:197(search)
148 0.458 0.003 0.458 0.003 {method 'read' of '_io.BufferedReader' objects}
950037 0.279 0.000 0.279 0.000 {method 'match' of 're.Pattern' objects}
1 0.000 0.000 0.249 0.249 bsp_tool\lumps.py:354(__init__)
7 0.218 0.031 0.218 0.031 {built-in method io.open}
1357273 0.174 0.000 0.174 0.000 {built-in method builtins.isinstance}
405707 0.157 0.000 0.157 0.000 {method 'search' of 're.Pattern' objects}
<pstats.Stats object at 0x0000025E5712E8C0>
>>> quit()from bsp_tool.
snake-biscuits
commented on July 17, 2024
MegaTest duration reduced from 14hrs to 1hr
Big success
Comparing the cProfile runs, the load time have improved by 50x (takes ~1/50th of the time)
With some luck, we won't need to make any new performance fixes for a while
(So long as the BspLump._changes cache doesn't get too messy)
from bsp_tool.
Related Issues (20)
- `RespawnBsp.save_as` dumps `.bsp_lump` copies in working directory HOT 1
- `RespawnBsp.save_as()` test cases HOT 1
- Quake II Rerelease increased lightmap resolution HOT 2
- Usage of ContentsMask in source.py HOT 3
- Need integer vectors to preserve accuracy
- (extensions.convert.respawn) r1 -> r2 for `.ain` generation HOT 2
- (MegaTest) download & catalogue dukeworld maps
- Standardising geometry extraction & processing HOT 9
- extensions should add new methods to branches HOT 2
- how use x360.py? HOT 9
- Consider `sprp` version part of "branch id"
- Split related files handler out of `BspClass._preload`
- cannot load CS:O2 `aim_dust.bsp` with `nexon.cso2_2018` HOT 1
- Catalogue Genesis3D Engine HOT 1
- Need a common Image LumpClass base type
- [Test Maps] `genesis3d`
- Configure `pytest` to remove naming convention HOT 2
- Create a `utils` submodule to pull tools out of `branches` HOT 1
- (extensions.editor) `.save_as(filename: str)` method for all `MapFile` classes
- (extensions.editor) MetaPattern string conversion is an ugly mess
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bsp_tool.