gothickit / zenkit Goto Github PK

View Code? Open in Web Editor NEW

37.0 3.0 8.0 10.3 MB

A re-implementation of file formats used by the early 2000's ZenGin

Home Page: http://zk.gothickit.dev/

License: MIT License

CMake 1.06% C++ 98.94%

file-format-library file-formats game game-engine gothic gothic2 zengin gothic1 library

zenkit's Introduction

The ZenKit Project

ZenKit has recently been updated to version 1.3! For information about how to migrate your projects, see the documentation.

Important

NEW: C# and Java bindings for ZenKit are now available! See the NuGet Package Gallery and Maven Central.

The ZenKit project aims to re-implement file formats used by the ZenGin made by Piranha Bytes for their early-2000s games Gothic and Gothic II. It is heavily based on ZenLib which is used as a reference implementation for the different file formats used.

ZenKit includes parsers and basic datastructures for most file formats used by the ZenGin as well as a type-safe VM for Daedalus scripts and supporting infrastructure like Gothic II class definitions. Tools for inspecting and converting ZenGin files can be found in phoenix studio.

To get started, take a look in the Reference Documentation. Don't hesitate to open a discussion thread over in Discussions if you have a question or need help. Please open an issue for any bug you encounter!

You can also contact me directly on Discord, ideally by pinging me (@lmichaelis) in the GMC Discord in the tools channel.

Supported File Formats

Currently, the following file formats are supported.

Format	Extension	Description	ZenKit Class Name
Model Animation	`.MAN`	Contains animations for a model	`ModelAnimation`
Model Hierarchy	`.MDH`	Contains skeletal information for a model	`ModelHierarchy`
Model Mesh	`.MDM`	Contains the mesh of a model	`ModelMesh`
Model	`.MDL`	Contains a mesh and a hierarchy which make up a model	`Model`
Morph Mesh	`.MMB`	Contains a morph mesh with its mesh, skeleton and animation data	`MorphMesh`
Multi Resolution Mesh	`.MRM`	Contains a mesh with LOD information	`MultiResolutionMesh`
Mesh	`.MSH`	Contains mesh vertices and vertex features like materials	`Mesh`
Daedalus Script	`.DAT`	Contains a compiled Daedalus script	`DaedalusScript`
Texture	`.TEX`	Contains texture data in a variety of formats	`Texture`
Font	`.FNT`	Contains font data	`Font`
ZenGin Archive	`.ZEN`	Contains various structured data (general object persistence).	`ReadArchive`
Text/Cutscenes	`.BIN`, `.CSL`, `.DAT`, `.LSC`	Contains text and cutscene data	`CutsceneLibrary`
Model Script	`.MDS`, `.MSB`	Contains model animation script data and associated hierarchy and mesh information	`ModelScript`
Virtual File System	`.VDF`	Contains a directory structure containing multiple files; similar to TAR.	`Vfs`

Contributing

If you'd like to contribute, please read Contributing first.

Building

ZenKit is currently only tested on Linux and while Windows should be supported you might run into issues. If so, feel free to create an issue or open a merge request. You will need

A working compiler which supports C++17, like GCC 9
CMake 3.10 or above
Git

To build ZenKit from scratch, just open a terminal in a directory of your choice and run

git clone --recursive https://github.com/GothicKit/phoenix
cd phoenix
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build

You will find the library in build/.

Using

#include <zenkit/Vfs.hh>
#include <zenkit/Model.hh>
#include <zenkit/Stream.hh>

int main(int, char**) {
	// Open the VDF file for reading
	zenkit::Vfs vfs {};
	vfs.mount_disk("Models.VDF");
	
	// Find the MyModel.MDL within the VDF
	auto entry = vdf.find("MyModel.MDL");
	
	if (entry == nullptr) {
		// MyModel.MDL was not found in the VDF
		return -1;
	}
	
	// Open MyModel.MDL for reading
	auto buf = entry->open_read();
	
	// One could also memory-map a normal file from disk:
	//   auto buf = zenkit::Read::from("/path/to/file");

	// Or if you have a vector of data:
	//   std::vector<uint8_t> data { /* ... */ };
	//   auto buf = zenkit::Read::from(std::move(data));
	
	// Parse the model
	zeknit::Model mdl {};
	mdl.load(buf.get());
	
	// Do something with mdl ...
	
	return 0;
}

ZenKit also provides a VM implementation for the Daedalus scripting language used by ZenGin:

#include <zenkit/Stream.hh>
#include <zenkit/DaedalusScript.hh>
#include <zenkit/DaedalusVm.hh>

#include <iostream>
#include <string>

enum class MyScriptEnum : int {
    FANCY = 0,
    PLAIN = 1,
};

// Declare a class to be bound to members in a script. This is used in `main`.
struct MyScriptClass : public zenkit::DaedalusInstance {
    // Declare the members present in the script class.
    // Supported types are:
    //   * int
    //   * float
    //   * std::string
    //   * `enum` types with sizeof(enum) == 4
    // and their C-Style array versions.
    
    std::string myStringVar;
    int someIntegers[10];
    float aFloat;
    MyScriptEnum anEnum;
};

// Define a function to be bound to an external definition in a script. This is used in `main`.
// Supported parameter types are:
//   * int
//   * float
//   * bool
//   * std::string_view
//   * std::shared_ptr<instance> or any subclass of instance
// Supported return types are:
//   * int (or anything convertible to int32_t)
//   * float (or anything convertible to float)
//   * bool
//   * void
//   * std::shared_ptr<instance> or any subclass of instance
bool MyExternalFunction(std::string_view param1, int param2, std::shared_ptr<MyScriptClass> param3) {
    std::cout << "Calling MyExternalFunction(" << param1 << ", " << param2 << ", " << param3->symbol_index() << ")\n";
    return true;
}

// Define a function to be bound to an internal definition in a script. This is used in `main`.
// Supported parameter and return types are the same as for external functions. 
std::string MyInternalFunction(int param1) {
    return std::to_string(param1);
}

int main(int, char**) {
    // Open a buffer containing the script.
    auto buf = zenkit::Read::from("MyScript.DAT");
    
    // Create the VM instance
    zenkit::DaedalusScript script {};
    script.load(buf.get());
    
    zenkit::DaedalusVm vm {std::move(script)};
    
    // You can register Daedalus -> C++ shared classes. The `register_member` function will automatically
    // validate that the definitions match at runtime.
    vm.register_member("MyScriptClass.myStringVar", &MyScriptClass::myStringVar);
    vm.register_member("MyScriptClass.someIntegers", &MyScriptClass::someIntegers);
    vm.register_member("MyScriptClass.aFloat", &MyScriptClass::aFloat);
    vm.register_member("MyScriptClass.anEnum", &MyScriptClass::anEnum);
    
    // You could also have differing member and/or class names:
    //   vm.register_member("SomeOtherClass.fancyness", &MyScriptClass::anEnum);
    
    // phoenix supports registering external script functions to a C++ function. The function signature is
    // validated at runtime to match the definition of the function in the script file.
    vm.register_external("MyExternalFunction", &MyExternalFunction);
    
    // You can also register a function to be called if an external is not registered:
    vm.register_default_external([](std::string_view name) {
        std::cout << "External " << name << " not registered\n";
    });
    
    // phoenix allows you to override internal script functions as well. The signature of the function
    // is also validated at runtime.
    vm.override_function("MyInternalFunction", &MyInternalFunction);
    
    // You can call the instance initializer like this:
    auto myNpc = vm.init_instance<MyScriptClass>("MyInstance");
    
    // Alternatively, you can also provide a pointer to the instance instead of having it be allocated
    // automatically:
    //
    //   auto ptr = std::make_shared<MyScriptClass>();
    //   vm.init_instance(ptr, "MyInstance");
    
    // Calling internal script function is also easy:
    std::cout << "Result of MyInternalFunction(10): " 
              << vm.call_function<std::string>("MyInternalFunction", 10) << "\n";
    
    // If you'd like to avoid passing a string to this function, you can also fetch the
    // function symbol beforehand and pass it instead:
    //
    //   auto* functionSym = vm.find_symbol_by_name("MyInternalFunction");
    //   auto result = vm.call_function<std::string>(functionSym, 10);

    // Sometimes it is required to set the HERO, SELF, OTHER, ITEM, or VICTIM  global instance variables.
    // This can be done like this:
    auto oldValue = vm.global_other()->get_instance();
    vm.global_other()->set_instance(myNpc);

    // The other global variables can be accessed using:
    //   * vm.global_self()
    //   * vm.global_other()
    //   * vm.global_victim()
    //   * vm.global_hero()
    //   * vm.global_item()
    
    // No special clean-up logic is required. All initialized instances will be
    // valid even after the script is destructed because they are shared_ptrs.
    
    return 0;
}

For more examples on how to use ZenKit, take a look into the examples directory and phoenix studio repository.

Licensing

While the source code of ZenKit is licensed under the MIT license, the ZenKit logo is licensed under CC BY-NC 4.0.

zenkit's People

Contributors

Stargazers

Watchers

Forkers

thokkat dadummy lisembart try jucanandreidaniel thielhater bogu9821 polymeilex

zenkit's Issues

Parsing zCCSCamera keyframes

There are multiple warnings in command line:

[phoenix] vob_tree: encountered unknown VOb [% § 0 25125]
[phoenix] vob_tree: encountered unknown VOb [% § 0 25126]
[phoenix] vob_tree: encountered unknown VOb [% § 0 25127]
[phoenix] vob_tree: encountered unknown VOb [% § 0 25128]
[phoenix] vob_tree: encountered unknown VOb [% § 0 25129]
[phoenix] vob_tree: encountered unknown VOb [% § 0 25130]

AFAIR § is a special way to refer keyframe in cameras

Stabilize and generify the API

The API needs to be stabilized and made more intuitive by making all classes and features behave in a similar way. Right now, for example, one can create a daedalus::script from a buffer or by passing a std::string_view which reads a file. Ignoring for now, that it should take a std::filesystem::path instead, this API should either be available for all classes or be removed in favour of just always passing a buffer.

[implemented in 7e7aff] Another painpoint is the requirement of passing a buffer& to basically all user-facing functions which require one. While it is useful to have this feature within phoenix, users would almost never care about the buffer but they have to keep it around anyways. User-facing functions should always be able to take a buffer or a buffer&& for the user's convenience.

Issues with TheChroniclesOfMyrtana mod

I'm looking into KoM startup routine, to see if it's possible to have adequate support for this mod.
Before phoenix, I was able to react stating location and play tutorial segment on a ship.

Issues so far:

Crash in symbol::set_int
C++ side:

void symbol::set_int(std::int32_t value, std::size_t index, const std::shared_ptr<instance>& context) {
	...
	if (is_member()) {
		...
	} else {
		// _m_value is null
		std::get<std::unique_ptr<std::int32_t[]>>(_m_value)[index] = value;
	}
}

Script call-stack:

"HOOKENGINEF"
"INIT_FEATURES_GAMESTART"
"INIT_GAMESTART"
"INIT_GLOBAL"

Apparently it's related to assignment to HOOKENGINEF.FUNCTION (function override?)

No error handling in opcode::div and opcode::mod

If script divides by zero - whole application crashes

Crash in allocate_instance, when called for "$INSTANCE_HELP"
Most likely not intended by script - happens late in initialize routines, and everything is supper broke there, due to no DMA support.

auto* parent = find_symbol_by_index(sym->parent()); // parent ==nullptr here
while (parent->type() != datatype::class_) {
	parent = find_symbol_by_index(parent->parent());
	}

illegal mutable access of const symbol

[phoenix] vm: internal exception: illegal mutable access of const symbol WIN_GETLASTERROR.CALL
[phoenix] vm: internal exception: illegal mutable access of const symbol KMLIB_INITIALIZEGAMESTART._KMLIB_INITIALIZEGAMESTART_PTR
[phoenix] vm: internal exception: illegal mutable access of const symbol CALLINT_RETVALISFLOAT
[phoenix] vm: internal exception: illegal mutable access of const symbol CALLINT_PUTRETVALTO
[phoenix] vm: internal exception: illegal mutable access of const symbol CALLINT_NUMPARAMS

Proposed fix: introduce vm_allow_const_cast_access, flag similar to vm_allow_null_instance_access

Minor issue with de-compiled sources:
Local variables were displayed as global. I'm aware, that Daedalus has no real stack-variables, so just readability issue.

// CATINV_NEXTNONACTIVEITEM.* should be local variables
var int CATINV_NEXTNONACTIVEITEM.I = 0;
instance CATINV_NEXTNONACTIVEITEM.L(ZCLISTSORT)
instance CATINV_NEXTNONACTIVEITEM.ITM(C_ITEM)
func int CATINV_LASTNONACTIVEITEM(var int LIST, var int MAX) {
    ...
}

Allow for receiving symbols directly
This is required to implement control-flow functions in ikarus. Basically my plan is to:
vm.override_function("repeat", [this](phoenix::symbol* var, int count){ repeat(var,count); })
And access(and memorize) var as a reference to loop-counter.

Proposed solution: remove check for phoenix::symbol in check_external_params, rest seem to work as-is

Some stack introspection?
Followup to repeat. All together makes workflow:

// script
repeat(i, 32)
...
end; // variable-token

// c++
void Ikarus::repeat(phoenix::symbol* var, int count) {
  auto end = vm.find_symbol_by_name("end");
  if(end==nullptr)
    return;
  auto& called_from = ...; // need to get parent context
  for(uint32_t i=called_from.pc(); i<vm.size(); ) {
    auto inst = vm.instruction_at(i);
    if(inst.op==phoenix::opcode::pushv && inst.symbol==end->address()) {
      // inject jump-back to have a loop
      break;
      }
    i += inst.size;
    }
  }

PS:
Probably it's better to have one ticket instead of million small ones.

Improve tool CLI

The command-line interface for the tool scripts can be a bit fiddly with regards to argument placement. Ideally, their CLI would be generified as well to reduce the number of surprises for the user. Most tools also need more options for specifying file output locations, file types and more.

[v2.0.0] Investigate `§`-VObs

These VObs are empty though they seem to be related to zCCSCamera keyframes. ZenLib calles them zReference but does not use them anywhere. Reverse-engineering of the original game binary might be required.

Leads:

Looking through the Gothic II binary, zCArchiverBinSafe::ReadObject@0x51fb50 contains a check against 0xA7 at 0x51fbde
Error at 0x51fc63 contains string "D: zArchiver(::ReadObject): objRef, illegal object index: "

[v1.1.0] Make `symbol::set_string` take a `std::string_view` instead of a `const std::string&`

This makes sense because the string is copied internally anyways. Additionally, defaulting to taking std::string_view provides more opportunities for copy elision. This is needed as a consequence to #30.

Ubuntu build failures

OpenGothic CI: https://ci.appveyor.com/project/Try/opengothic/builds/45436817/job/vljogctckhdu2sh0

https://github.com/lmichaelis/phoenix/blob/529137d516615d1a2e62fe380e5690f8cf078a7c/source/world/bsp_tree.cc#L60

[ 41%] Building CXX object lib/phoenix/CMakeFiles/phoenix.dir/source/world/bsp_tree.cc.o
/home/appveyor/projects/opengothic/lib/phoenix/source/world/bsp_tree.cc:60:3: error: extra ‘;’ [-Werror=pedantic]
   60 |  };
      |   ^
cc1plus: all warnings being treated as errors
make[3]: ***

Procedure entry point failure at runtime when linking phoenix

Hi team,

I want to set up phoenix as linked dependency to my C++ project.

Unfortunately calling a phoenix function leads to runtime exception:

The procedure entry point _ZNKSt7codecvtlwc9_MbstatetE10do_unshiftERS0_PcS3_RS3_ could not be located in the dynamic link library [...]\build\test_lib.exe

I checked repositories phoenix-studio and OpenGothic for some references how to handle but without success.

What am I missing?

I'm posting my setup here to reproduce:

main.cpp

#include <iostream>
#include <phoenix/vdfs.hh>

int main(int argc, char** argv) {
    auto vdf = phoenix::vdf_file::open("speech_babe_speech_engl.VDF");
    return 0;
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.10)
set(CMAKE_CXX_STANDARD 17)
project(test_lib VERSION 0.1.0)

add_subdirectory(phoenix)
add_executable(test_lib main.cpp)
target_link_libraries(test_lib phoenix)

build commands
(cmake version = 3.25.2)

cmake -G "MinGW Makefiles" -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build

--> executing test_lib.exe

arm64: AddressSanitizer: container-overflow on address

Run into this issue, when testing latest OpenGothic build on M1.
Apart from it, game runs fine, if detect_container_overflow is disabled. Failure is quite consistent in trigger_list parse code, yet I wasn't able to spot any suspicius code in relevant places.

Full log is below:

GPU = Apple M1
Depth format = 13 Shadow format = 13
[phoenix] world: parsing object [MeshAndBsp % 0 0]
[phoenix] bsp_tree: parsing chunk C000
[phoenix] bsp_tree: parsing chunk C010
[phoenix] bsp_tree: parsing chunk C040
[phoenix] bsp_tree: parsing chunk C045
[phoenix] bsp_tree: parsing chunk C050
[phoenix] bsp_tree: parsing chunk C0FF
[phoenix] mesh: 1 bytes remaining in section 0xB020
[phoenix] world: parsing object [VobTree % 0 0]
=================================================================
==89736==ERROR: AddressSanitizer: container-overflow on address 0x0002cdc006a0 at pc 0x0001051a1c60 bp 0x0002c142faa0 sp 0x0002c142f258
WRITE of size 24 at 0x0002cdc006a0 thread T8
    #0 0x1051a1c5c in __asan_memcpy+0x240 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x3dc5c)
    #1 0x1014a0cdc in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) string:1999
    #2 0x1014a0c2c in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) string:2000
    #3 0x1014a0a88 in phoenix::vobs::trigger_list::target::target(phoenix::vobs::trigger_list::target&&) trigger.hh:95
    #4 0x1014a0994 in phoenix::vobs::trigger_list::target::target(phoenix::vobs::trigger_list::target&&) trigger.hh:95
    #5 0x1014a0928 in phoenix::vobs::trigger_list::target* std::__1::construct_at<phoenix::vobs::trigger_list::target, phoenix::vobs::trigger_list::target, phoenix::vobs::trigger_list::target*>(phoenix::vobs::trigger_list::target*, phoenix::vobs::trigger_list::target&&) construct_at.h:37
    #6 0x10149fe14 in void std::__1::allocator_traits<std::__1::allocator<phoenix::vobs::trigger_list::target> >::construct<phoenix::vobs::trigger_list::target, phoenix::vobs::trigger_list::target, void, void>(std::__1::allocator<phoenix::vobs::trigger_list::target>&, phoenix::vobs::trigger_list::target*, phoenix::vobs::trigger_list::target&&) allocator_traits.h:298
    #7 0x10149f260 in void std::__1::vector<phoenix::vobs::trigger_list::target, std::__1::allocator<phoenix::vobs::trigger_list::target> >::__construct_one_at_end<phoenix::vobs::trigger_list::target>(phoenix::vobs::trigger_list::target&&) vector:948
    #8 0x1014945c0 in phoenix::vobs::trigger_list::target& std::__1::vector<phoenix::vobs::trigger_list::target, std::__1::allocator<phoenix::vobs::trigger_list::target> >::emplace_back<phoenix::vobs::trigger_list::target>(phoenix::vobs::trigger_list::target&&) vector:1706
    #9 0x101494218 in phoenix::vobs::trigger_list::parse(phoenix::vobs::trigger_list&, phoenix::archive_reader&, phoenix::game_version) trigger.cc:60
    #10 0x1015c9f00 in phoenix::parse_vob_tree(phoenix::archive_reader&, phoenix::game_version) vob_tree.cc:180
    #11 0x1015cc840 in phoenix::parse_vob_tree(phoenix::archive_reader&, phoenix::game_version) vob_tree.cc:251
    #12 0x1015cc840 in phoenix::parse_vob_tree(phoenix::archive_reader&, phoenix::game_version) vob_tree.cc:251
    #13 0x101566cec in phoenix::world::parse(phoenix::buffer&, phoenix::game_version) world.cc:88
    #14 0x1006374b8 in World::World(GameSession&, std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<void (int)>) world.cpp:75
    #15 0x100638020 in World::World(GameSession&, std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<void (int)>) world.cpp:64
    #16 0x1002d4418 in GameSession::GameSession(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) gamesession.cpp:62
    #17 0x1002d4ffc in GameSession::GameSession(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) gamesession.cpp:55
    #18 0x1004e25d0 in MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0::operator()(std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) const mainwindow.cpp:866
    #19 0x1004e251c in decltype(static_cast<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0&>(fp)(static_cast<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >>(fp0))) std::__1::__invoke<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0&, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > >(MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0&, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) type_traits:3918
    #20 0x1004e24c4 in std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > std::__1::__invoke_void_return_wrapper<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >, false>::__call<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0&, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > >(MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0&, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) invoke.h:30
    #21 0x1004e2464 in std::__1::__function::__alloc_func<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0, std::__1::allocator<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0>, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>::operator()(std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) function.h:178
    #22 0x1004e0e88 in std::__1::__function::__func<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0, std::__1::allocator<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0>, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>::operator()(std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) function.h:352
    #23 0x100338994 in std::__1::__function::__value_func<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>::operator()(std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) const function.h:505
    #24 0x1003386a8 in std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>::operator()(std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) const function.h:1182
    #25 0x100338308 in Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0::operator()() const gothic.cpp:417
    #26 0x100338244 in decltype(static_cast<Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0>(fp)()) std::__1::__invoke<Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0>(Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0&&) type_traits:3918
    #27 0x1003381e0 in void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0>&, std::__1::__tuple_indices<>) thread:287
    #28 0x1003378e0 in void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0> >(void*) thread:298
    #29 0x1861a94e8 in _pthread_start+0x90 (libsystem_pthread.dylib:arm64e+0x74e8)
    #30 0x1861a42cc in thread_start+0x4 (libsystem_pthread.dylib:arm64e+0x22cc)

0x0002cdc006a0 is located 96 bytes inside of 128-byte region [0x0002cdc00640,0x0002cdc006c0)
allocated by thread T8 here:

    #0 0x1051b0bd8 in wrap__Znwm+0x74 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x4cbd8)
    #1 0x10149d8fc in void* std::__1::__libcpp_operator_new<unsigned long>(unsigned long) new:235
    #2 0x10149d6ec in std::__1::__libcpp_allocate(unsigned long, unsigned long) new:261
    #3 0x1014a2da4 in std::__1::allocator<phoenix::vobs::trigger_list::target>::allocate(unsigned long) allocator.h:108
    #4 0x1014a2768 in std::__1::allocator_traits<std::__1::allocator<phoenix::vobs::trigger_list::target> >::allocate(std::__1::allocator<phoenix::vobs::trigger_list::target>&, unsigned long) allocator_traits.h:262
    #5 0x1014a2378 in std::__1::__split_buffer<phoenix::vobs::trigger_list::target, std::__1::allocator<phoenix::vobs::trigger_list::target>&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<phoenix::vobs::trigger_list::target>&) __split_buffer:315
    #6 0x1014a18f0 in std::__1::__split_buffer<phoenix::vobs::trigger_list::target, std::__1::allocator<phoenix::vobs::trigger_list::target>&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<phoenix::vobs::trigger_list::target>&) __split_buffer:314
    #7 0x10149f71c in void std::__1::vector<phoenix::vobs::trigger_list::target, std::__1::allocator<phoenix::vobs::trigger_list::target> >::__emplace_back_slow_path<phoenix::vobs::trigger_list::target>(phoenix::vobs::trigger_list::target&&) vector:1687
    #8 0x101494644 in phoenix::vobs::trigger_list::target& std::__1::vector<phoenix::vobs::trigger_list::target, std::__1::allocator<phoenix::vobs::trigger_list::target> >::emplace_back<phoenix::vobs::trigger_list::target>(phoenix::vobs::trigger_list::target&&) vector:1709
    #9 0x101494218 in phoenix::vobs::trigger_list::parse(phoenix::vobs::trigger_list&, phoenix::archive_reader&, phoenix::game_version) trigger.cc:60
    #10 0x1015c9f00 in phoenix::parse_vob_tree(phoenix::archive_reader&, phoenix::game_version) vob_tree.cc:180
    #11 0x1015cc840 in phoenix::parse_vob_tree(phoenix::archive_reader&, phoenix::game_version) vob_tree.cc:251
    #12 0x1015cc840 in phoenix::parse_vob_tree(phoenix::archive_reader&, phoenix::game_version) vob_tree.cc:251
    #13 0x101566cec in phoenix::world::parse(phoenix::buffer&, phoenix::game_version) world.cc:88
    #14 0x1006374b8 in World::World(GameSession&, std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<void (int)>) world.cpp:75
    #15 0x100638020 in World::World(GameSession&, std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<void (int)>) world.cpp:64
    #16 0x1002d4418 in GameSession::GameSession(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) gamesession.cpp:62
    #17 0x1002d4ffc in GameSession::GameSession(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) gamesession.cpp:55
    #18 0x1004e25d0 in MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0::operator()(std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) const mainwindow.cpp:866
    #19 0x1004e251c in decltype(static_cast<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0&>(fp)(static_cast<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >>(fp0))) std::__1::__invoke<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0&, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > >(MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0&, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) type_traits:3918
    #20 0x1004e24c4 in std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > std::__1::__invoke_void_return_wrapper<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >, false>::__call<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0&, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > >(MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0&, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) invoke.h:30
    #21 0x1004e2464 in std::__1::__function::__alloc_func<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0, std::__1::allocator<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0>, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>::operator()(std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) function.h:178
    #22 0x1004e0e88 in std::__1::__function::__func<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0, std::__1::allocator<MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >)::$_0>, std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>::operator()(std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) function.h:352
    #23 0x100338994 in std::__1::__function::__value_func<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>::operator()(std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) const function.h:505
    #24 0x1003386a8 in std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>::operator()(std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&) const function.h:1182
    #25 0x100338308 in Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0::operator()() const gothic.cpp:417
    #26 0x100338244 in decltype(static_cast<Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0>(fp)()) std::__1::__invoke<Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0>(Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0&&) type_traits:3918
    #27 0x1003381e0 in void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0>&, std::__1::__tuple_indices<>) thread:287
    #28 0x1003378e0 in void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0> >(void*) thread:298
    #29 0x1861a94e8 in _pthread_start+0x90 (libsystem_pthread.dylib:arm64e+0x74e8)

Thread T8 created by T0 here:

    #0 0x10519df58 in wrap_pthread_create+0x54 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x39f58)
    #1 0x100337880 in std::__1::__libcpp_thread_create(_opaque_pthread_t**, void* (*)(void*), void*) __threading_support:421
    #2 0x100337638 in std::__1::thread::thread<Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0, void>(Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0&&) thread:314
    #3 0x100322e80 in std::__1::thread::thread<Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0, void>(Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>)::$_0&&) thread:306
    #4 0x100322c64 in Gothic::implStartLoadSave(std::__1::basic_string_view<char, std::__1::char_traits<char> >, bool, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>) gothic.cpp:411
    #5 0x100322dc0 in Gothic::startLoad(std::__1::basic_string_view<char, std::__1::char_traits<char> >, std::__1::function<std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> > (std::__1::unique_ptr<GameSession, std::__1::default_delete<GameSession> >&&)>) gothic.cpp:393
    #6 0x1004d3888 in MainWindow::startGame(std::__1::basic_string_view<char, std::__1::char_traits<char> >) mainwindow.cpp:864
    #7 0x1004d2ce8 in MainWindow::MainWindow(Tempest::Device&) mainwindow.cpp:75
    #8 0x1004d43d8 in MainWindow::MainWindow(Tempest::Device&) mainwindow.cpp:35
    #9 0x1004cc964 in main main.cpp:107
    #10 0x104ca50f0 in start+0x204 (dyld:arm64e+0x50f0)

HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.

SUMMARY: AddressSanitizer: container-overflow (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x3dc5c) in __asan_memcpy+0x240
Shadow bytes around the buggy address:
  0x007059ba0080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x007059ba0090: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x007059ba00a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x007059ba00b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x007059ba00c0: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
=>0x007059ba00d0: 00 00 00 00[fc]fc fc fc fa fa fa fa fa fa fa fa
  0x007059ba00e0: 00 00 00 00 00 00 fc fc fc fc fc fc fc fc fc fa
  0x007059ba00f0: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x007059ba0100: 00 00 00 00 00 00 00 fa fa fa fa fa fa fa fa fa
  0x007059ba0110: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x007059ba0120: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==89736==ABORTING

[v2.0.0] Uniform parse syntax

Instead of defining a parse static member function for each entity, there should instead be a uniform

namespace phoenix {
    template <typename T>
    T parse(buffer&);

    template <typename T>
    T parse(buffer&&);

    template <typename T>
    T parse(archive_reader&);
}

This can only be implemented if a heuristic for detecting the Gothic version within worlds can be found. One idea would be to search through the archive's key-map to find object keys only present in Gothic 2 VObs, like visualAniMode (probably unlikely to yield good results since it's in the base zCVob which is most likely packed), canMove or fadeOutSky. It might still be a good idea, however, to keep the explicit version around somehow, so consumers of the library can prevent loading the wrong version anyways.

This alteration should also apply to dumping function (if implemented):

namespace phoenix {
    template <typename T>
    buffer dump(const T&);
}

[v1.1.0] Unified parse function for MDS files

Currently there are two functions model_script::parse and model_script::parse_binary. Since we can detect whether we're dealing with a binary file or not by reading the first two bytes and checking whether the resulting uint16_t is in range of any of the chunk IDs, there should only be one parse function.

Suspicious call to destructor

https://github.com/lmichaelis/phoenix/blob/87235e137823460bebf2769a51fd6665ce3856ed/source/vm.cc#L266-L270

Found this while debugging "Chronicles of Myrthana" mod.
Relevant piece of code: https://github.com/lmichaelis/tcom-decompilation/blob/c993234e0fd8520401491674c8ababe461859d09/Source/Raw/526.d

func int SPELL_TELEKINESIS_FOCUS_REMOVER() {
    NPC = HLP_GETNPC(0x71b); // returns null. This is query for PC_HERE, but PC_HERE is not yet initialized
    NPC.FOCUS_VOB = 0; // invalid access, causing explicit destructor call 
    STAGE = 0;
    return FALSE;
}

Shortly after executing opcode::movi VM crashes with memory error.

[v1.1.0] Allow VM external to take raw instance pointers

In cases where the pointer is not used outside of the external function, this can improve performance by not having to increment the shared_ptr's refcount.

[v1.1.0] Make internal associative contains use the `std::less<>` comparator

This allows for find-ing keys of type std::string using std::string_view which makes everything faster.

Properly Document the API

The API for phoenix as a whole needs to be properly documented for the end-user. Additionally, examples should be added for each feature.

[v1.0.0-rc2] Add allocate_instance VM operation

[v2.0.0] Switch to in-house MDS parser

phoenix currently uses lexy for parsing model script files. This is only a temporary solution and should be replaced by a self-built parser which can properly take into account the broken scripts originally written by the team at PiranhaBytes (missing parentheses and such; see b317ad).

string_view version of `script::find_symbol_by_name` and few others

Current syntax:
symbol* script::find_symbol_by_name(const std::string& name) forces client to bake std::string into interface.
for find function this is especially unnecessary, since it does string copy immediatly after:

	symbol* script::find_symbol_by_name(const std::string& name) {
		std::string up {name}; // <-- fine, but no need to query name as string
		std::transform(up.begin(), up.end(), up.begin(), ::toupper);

		if (auto it = _m_symbols_by_name.find(up); it != _m_symbols_by_name.end()) {
			return find_symbol_by_index(it->second);
		}

		return nullptr;
	}

In general it's probably would be good to pass read-only strings as string_view in all cases. If implementation then uses this string as key to has-map or similar - solve this locally inside the function.

Other places:

R vm::call_function(const std::string& name, P... args)
vm::init_instance(const std::string& name)
vm::allocate_instance(const std::string& name)
void vm::push_string(const std::string& value)
void register_external(const std::string& name, const std::function<R(P...)>& callback)
void override_function(const std::string& name, const std::function<R(P...)>& callback)
const way_point* way_net::waypoint(const std::string& name) const
const message_block* messages::block_by_name(const std::string& name) const

Dialogs parsing performance

GameScript::loadDialogOU() become noticeably slower with phenix backend.

I haven't done proper profiling, but issue seem to be rooted in large amount of small allocations, while population hash-map.
Lazy profiling(debug break a few times) shows, that this place can be an issue:

	bool archive_reader_binsafe::read_object_begin(archive_object& obj) {
		...
		std::stringstream ss {line.substr(1, line.size() - 2)}; // substr creates a copy; and well stringstream known to be slow
		ss >> obj.object_name >> obj.class_name >> obj.version >> obj.index;
		return true;
	}

`archive_reader_binsafe::read_object_begin` performance

Followup to: #28
I've tested std::stringstream performance today.
Change:

    char name[128] = {};
    char cls [128] = {};
    sscanf_s(line.c_str(), "[%s %s %u %u]",
             name, rsize_t(sizeof(name)),
             cls,  rsize_t(sizeof(cls)),
             &obj.version, &obj.index);
    obj.object_name = name;
    obj.class_name  = cls;

Benchmark (world = oldworld.zen)

    auto buf = entry->open();
    auto time = Tempest::Application::tickCount();
    auto world = phoenix::world::parse(buf, version().game == 1 ? phoenix::game_version::gothic_1
                                                                : phoenix::game_version::gothic_2);
    time = Tempest::Application::tickCount() - time;
    Tempest::Log::d("parse time = ", time);

While change is rough one, and hopefully there is a better way than sscanf, as prototype looking quite good:

// Before (time - milliseconds)
parse time = 2371
parse time = 2769
parse time = 2713
parse time = 2447

// After (roughly 40% improvement)
parse time = 1618
parse time = 1441
parse time = 1225
parse time = 1559

MDS file parsing stops early

In OpenGothic the size of p.animations is smaller for some mds files than if using a pre-phoenix build. E.g. for HumanS.mds only the first 279 of 864 entries are loaded resulting in stuck npc behavior.

This is only a issue for G1, in G2 there's no difference.

Animation::Animation(phoenix::model_script &p, std::string_view name, const bool ignoreErrChunks) {
  ref = std::move(p.aliases);

  for(auto& ani : p.animations) {

https://github.com/Try/OpenGothic/blob/fc1c1343028fb92dbb7b5c1f7ed67b4294b800b3/game/graphics/mesh/animation.cpp#L33

[v1.0.0] make sure the stack is kept intact when executing instructions fails

This is most important in the opcode::be handler where the return value of the external is not pushed if it is not registered. Other opcodes might benefit from such a system as well. The most practical implementation would probably be an RAII helper class.

[v1.0.0-rc2] Rename `camera_lock_mode` to `sprite_alignment`

And camera_lock_mode should be renamed to sprite_alignment_mode

Building with MSVC / Visual Studio 2019 (compile errors)

Hi,

i tried building this with CMake 3.24.1 Visual Studio 2019 project generator.
Cmake Output:

Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19042.
The C compiler identification is MSVC 19.29.30146.0
The CXX compiler identification is MSVC 19.29.30146.0
Detecting C compiler ABI info
Detecting C compiler ABI info - done
Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
Detecting C compile features
Detecting C compile features - done
Detecting CXX compiler ABI info
Detecting CXX compiler ABI info - done
Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
Detecting CXX compile features
Detecting CXX compile features - done
GLM: Version 0.9.9.9
Module support is disabled.
Version: 9.1.1
Build type: 
CXX_STANDARD: 20
Required features: cxx_variadic_templates
CMake Warning (dev) at C:/Program Files/CMake/share/cmake-3.24/Modules/FetchContent.cmake:1264 (message):
  The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
  not set.  The policy's OLD behavior will be used.  When using a URL
  download, the timestamps of extracted files should preferably be that of
  the time of extraction, otherwise code that depends on the extracted
  contents might not be rebuilt if the URL changes.  The OLD behavior
  preserves the timestamps from the archive instead, but this is usually not
  what you want.  Update your project to the NEW behavior or specify the
  DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
  robustness issue.
Call Stack (most recent call first):
  vendor/CMakeLists.txt:12 (FetchContent_Declare)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Deprecation Warning at build/_deps/libsquish-src/CMakeLists.txt:14 (CMAKE_MINIMUM_REQUIRED):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


Configuring done
Generating done
CMake Error: Cannot open file for write: C:/Users/User/Eigene Repositories/Renderer/phoenix/build/tools/zdump/phoenix::tools::zinfo.sln.tmp77450
CMake Error: : System Error: Invalid argument
CMake Error: Cannot open file for write: C:/Users/User/Eigene Repositories/Renderer/phoenix/build/tools/zmodel/phoenix::tools::zmodel.sln.tmp07b13
CMake Error: : System Error: Invalid argument
CMake Error: Cannot open file for write: C:/Users/User/Eigene Repositories/Renderer/phoenix/build/tools/zscript/phoenix::tools::zscript.sln.tmp17120
CMake Error: : System Error: Invalid argument
CMake Error: Cannot open file for write: C:/Users/User/Eigene Repositories/Renderer/phoenix/build/tools/ztex/phoenix::tools::ztex.sln.tmp44674
CMake Error: : System Error: Invalid argument
CMake Error: Cannot open file for write: C:/Users/User/Eigene Repositories/Renderer/phoenix/build/tools/zvdfs/phoenix::tools::zvdfs.sln.tmpdf74c
CMake Error: : System Error: Invalid argument

No idea whats up with the CMake errors at the end there, i ignored them.
Then to fix project setup i needed to change:

Properties (all configurations) -> C++/Command Line/Additional Options:

squish: Remove -Wno-unused-parameter
phoenix, phoenix-tests: Remove -Wextra, -Werror, -Wconversion -Wno-conversion
zdump, ztex, zmodel, zvdfs, zscript: Remove -Wno-conversion

I think these flags do only apply to certain compilers and should not be set if MSVC is used.
After that i can build much more than before, however I still get a lot of compile errors (Debug x64):

Severity	Code	Description	Project	File	Line
Error	C2665	'std::from_chars': none of the 14 overloads could convert all the argument types	phoenix	C:\Users\User\Eigene Repositories\Renderer\phoenix\source\archive\archive_ascii.cc	146
Error	C2039	'base': is not a member of 'std::_String_iterator<std::_String_val<std::_Simple_types<_Elem>>>'	phoenix	C:\Users\User\Eigene Repositories\Renderer\phoenix\source\archive\archive_ascii.cc	140
Error	C3536	'beg_it': cannot be used before it is initialized	phoenix	C:\Users\User\Eigene Repositories\Renderer\phoenix\source\archive\archive_ascii.cc	146
Error	C2665	'std::from_chars': none of the 14 overloads could convert all the argument types	phoenix	C:\Users\User\Eigene Repositories\Renderer\phoenix\source\archive\archive_ascii.cc	147
Error	C2665	'std::from_chars': none of the 14 overloads could convert all the argument types	phoenix	C:\Users\User\Eigene Repositories\Renderer\phoenix\source\archive\archive_ascii.cc	148
Error	C2665	'std::from_chars': none of the 14 overloads could convert all the argument types	phoenix	C:\Users\User\Eigene Repositories\Renderer\phoenix\source\archive\archive_ascii.cc	149
Error	C2039	'base': is not a member of 'std::_String_iterator<std::_String_val<std::_Simple_types<_Elem>>>'	phoenix	C:\Users\User\Eigene Repositories\Renderer\phoenix\source\archive\archive_ascii.cc	163
Error	C3536	'beg_it': cannot be used before it is initialized	phoenix	C:\Users\User\Eigene Repositories\Renderer\phoenix\source\archive\archive_ascii.cc	166
Error	C2665	'std::from_chars': none of the 14 overloads could convert all the argument types	phoenix	C:\Users\User\Eigene Repositories\Renderer\phoenix\source\archive\archive_ascii.cc	166
Error	C2440	'initializing': cannot convert from 'int' to 'phoenix::alpha_function'	phoenix	C:\Users\User\Eigene Repositories\Renderer\phoenix\include\phoenix\material.hh	87
Error	C2440	'initializing': cannot convert from 'int' to 'phoenix::alpha_function'	phoenix	C:\Users\User\Eigene Repositories\Renderer\phoenix\include\phoenix\material.hh	87
Error	LNK1104	cannot open file '....\lib\Debug\phoenix.lib'	zscript	C:\Users\User\Eigene Repositories\Renderer\phoenix\build\tools\zscript\LINK	1
Error	LNK1104	cannot open file '....\lib\Debug\phoenix.lib'	zmodel	C:\Users\User\Eigene Repositories\Renderer\phoenix\build\tools\zmodel\LINK	1
Error	LNK1104	cannot open file '....\lib\Debug\phoenix.lib'	zdump	C:\Users\User\Eigene Repositories\Renderer\phoenix\build\tools\zdump\LINK	1
Error	LNK1104	cannot open file '....\lib\Debug\phoenix.lib'	zvdfs	C:\Users\User\Eigene Repositories\Renderer\phoenix\build\tools\zvdfs\LINK	1
Error	LNK1104	cannot open file '....\lib\Debug\phoenix.lib'	ztex	C:\Users\User\Eigene Repositories\Renderer\phoenix\build\tools\ztex\LINK	1
Error	C2079	'array' uses undefined class 'std::arraystd::byte,2'	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	259
Error	C2666	'phoenix::buffer::get': 3 overloads have similar conversions	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	260
Error	C2672	'std::equal': no matching overloaded function found	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	264
Error	C2780	'bool std::equal(_ExPo &&,const _FwdIt1,const _FwdIt1,const _FwdIt2,const _FwdIt2) noexcept': expects 5 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	264
Error	C2780	'bool std::equal(const _InIt1,const _InIt1,const _InIt2,const _InIt2)': expects 4 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	264
Error	C2780	'bool std::equal(_ExPo &&,const _FwdIt1,const _FwdIt1,const _FwdIt2,const _FwdIt2,_Pr) noexcept': expects 6 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	264
Error	C2780	'bool std::equal(const _InIt1,const _InIt1,const _InIt2,const _InIt2,_Pr)': expects 5 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	264
Error	C2780	'bool std::equal(_ExPo &&,const _FwdIt1,const _FwdIt1,const _FwdIt2) noexcept': expects 4 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	264
Error	C2780	'bool std::equal(const _InIt1,const _InIt1,const _InIt2)': expects 3 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	264
Error	C2780	'bool std::equal(_ExPo &&,const _FwdIt1,const _FwdIt1,const _FwdIt2,_Pr) noexcept': expects 5 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	264
Error	C2780	'bool std::equal(const _InIt1,const _InIt1,const _InIt2,_Pr)': expects 4 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	264
Error	C2079	'array' uses undefined class 'std::arraystd::byte,2'	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	274
Error	C2664	'void phoenix::buffer::get(uint64_t,std::spanstd::byte,18446744073709551615) const': cannot convert argument 2 from 'int' to 'std::spanstd::byte,18446744073709551615'	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	275
Error	C2672	'std::equal': no matching overloaded function found	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	278
Error	C2780	'bool std::equal(_ExPo &&,const _FwdIt1,const _FwdIt1,const _FwdIt2,const _FwdIt2) noexcept': expects 5 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	278
Error	C2780	'bool std::equal(const _InIt1,const _InIt1,const _InIt2,const _InIt2)': expects 4 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	278
Error	C2780	'bool std::equal(_ExPo &&,const _FwdIt1,const _FwdIt1,const _FwdIt2,const _FwdIt2,_Pr) noexcept': expects 6 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	278
Error	C2780	'bool std::equal(const _InIt1,const _InIt1,const _InIt2,const _InIt2,_Pr)': expects 5 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	278
Error	C2780	'bool std::equal(_ExPo &&,const _FwdIt1,const _FwdIt1,const _FwdIt2) noexcept': expects 4 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	278
Error	C2780	'bool std::equal(const _InIt1,const _InIt1,const _InIt2)': expects 3 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	278
Error	C2780	'bool std::equal(_ExPo &&,const _FwdIt1,const _FwdIt1,const _FwdIt2,_Pr) noexcept': expects 5 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	278
Error	C2780	'bool std::equal(const _InIt1,const _InIt1,const _InIt2,_Pr)': expects 4 arguments - 2 provided	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	278
Error	C2664	'void phoenix::buffer::get(uint64_t,std::spanstd::byte,18446744073709551615) const': cannot convert argument 2 from 'int' to 'std::spanstd::byte,18446744073709551615'	phoenix-tests	C:\Users\User\Eigene Repositories\Renderer\phoenix\tests\test_buffer.cc	281

Looks similar when building in release mode. The projects are configured with ISO C++20 Standard (/std:c++20).

I might try again with a later version of Visual Studio, but for Visual Studio 19 it seems to much work to get it running. Maybe not all used C++ 20 features are supported by MSVC, i don't know.

[v1.1.0] `vobs::trigger_mover` keyframes are not parsed correctly

Keyframes consist of a position and a rotation stored as a quaternion. Currently the rotation is parsed as a glm::mat4 and then converted to a glm::quat which is not correct.

https://github.com/lmichaelis/phoenix/blob/ffe0c0fa28a7e36181dd93d072b037ed4630b67f/source/vobs/trigger.cc#L38-L41

It should be parsed as a quaternion directly. The component order is w, x, y, z.

Source: Try/OpenGothic#362

Gothic1: T_DIALOGGESTURE_** are not loaded

Noticed at top level testing in OpenGothic, presumable those were not loaded due to potential error in model-script parsing.
Do not have much more info at this point, can do extra testing, if needed.

Version: Steam
Parser: text based (lexy)

[v2.0.0] Implement dump function for all data types

This is something which has been on my mind for a long time. phoenix should also support writing the files it parses so it can be used as a modern modding tool.

This requires merging the feature/flexible-buffers branch first and then implementing an archive_writer. After that is done, dump functions can be implemented for each file type.

Danger: especially for the VOb tree, it is critical that the correct object version numbers are saved!

Add GitHub actions to automagically build and test phoenix

Self-explanatory. phoenix used to have CI over on GitLab but since moving the project I've not set it up again. CI should cover clang-format and clang-tidy compliance as well as running the tests¹.

CI would ideally build on the latest Ubuntu release as well as Windows to ensure compatibility. Since MacOS is unix-like, phoenix should run on it without problems.

Some tests can currently not run in CI due to copyright issues. There might be a way to use different files, though. ↩

Issue with loading Union related archives

More FYI, rather than actual issues. When OpenGothic load *.vdf from file system, game crashes with exception:

 for(auto& i:archives)
    inst->gothicAssets.merge(phoenix::vdf_file::open(i.name), false); // buffer_underflow here

Callstack:

0x00007ff67c54ed76: dbg::call_stack<64u>::collect(unsigned int) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c3cf1dc: CrashLog::dumpStack(char const*) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c3cf738: terminateHandler() in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007fff739263f6: ZN10__cxxabiv111__terminateEPFvvE in C:\Users\mrsta\Downloads\opengothic_win\libstdc++-6.dll
0x00007fff73a22d43: ZSt9terminatev in C:\Users\mrsta\Downloads\opengothic_win\libstdc++-6.dll
0x00007fff73a2bf56: _cxa_throw in C:\Users\mrsta\Downloads\opengothic_win\libstdc++-6.dll
0x00007ff67c430819: phoenix::buffer::slice(unsigned long long, unsigned long long) const in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c47a368: phoenix::vdf_entry::read(phoenix::buffer&, unsigned int) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c47aa5b: phoenix::vdf_file::open(phoenix::buffer&) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c47ac87: phoenix::vdf_file::open(std::filesystem::__cxx11::path const&) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe
0x00007ff67c3b7c39: Resources::loadVdfs(std::vector<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> >, std::allocator<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> > > > const&) in C:\Users\mrsta\Downloads\opengothic_win\Gothic2Notr.exe

I've added simple try-catch handler to workaround issue, so here is the list of files that are failed to load:

unable to load archive: "D:/Games/Gothic II/Data/$Templates$/asmcl.dll"
unable to load archive: "D:/Games/Gothic II/Data/Union.vdf"
unable to load archive: "D:/Games/Gothic II/Data/$Templates$/ZUNIONUTILS.DLL"
unable to load archive: "D:/Games/Gothic II/Data/$Templates$/ZBINKFIX.DLL"
unable to load archive: "D:/Games/Gothic II/Data/$Templates$/ZMOUSEFIX.DLL"

Proposed change:
Can you add new dedicated type of exception, if *.vdf file signature does not match VDF_SIGNATURE_G* ?
Thanks!

[v2.0.0] Allow removal of deprecated symbols using preprocessor directives and CMake options

Currently, deprecated symbols are always included in the build but there should be a way of excluding them at compile time. This can be done using a pre-processor symbol and a set of #ifndefs.

Some Gothic 1 ModelScript files can not be parsed

Some model script files don't follow proper syntax which leads to the parser failing. The parser fails when trying to parse any statement which is missing a closing parenthesis like this:

// ...
aniEnum {
    ani("aniName1" 111 "aniNext1" 4.2 0.5 MI "aniModel1" F 221 331 FPS:25 CVS:0.2   // <- missing parenthesis!

    // ...

The ZenGin itself seems to be able to parse these files just fine, so the parser in phoenix has to be adjusted so that it treats closing round parenthesis (and possible curly braces too) as whitespace.

Assetion in `std::string buffer::get_line(bool skip_whitespace)`, with MSVC build

sorry for awful screenshot :)

Offended code:

	std::string buffer::get_line(bool skip_whitespace) {
		...
		if (skip_whitespace) {
			auto count = mismatch([](char chr) { return !std::isspace(chr); }); // <--
			if (count == -1) {
				position(limit()); // the end of the buffer has been reached
			} else {
				position(position() + count);
			}
		}

		return tmp;
	}

char can be signed on some C++ implementations, and have negative values.
Issue was hit, when parsing world mesh in steam-eng version of gothic2

[v1.1.0] Allow usage of submodules instead of FetchContent

In isolated build envionments it might not be possible to access the network while in the CMake configure step. Ideally, it would be possible to include phoenix' dependences as Git submodules instead, like it was before c30bba8. There is one dependency which is not available as a Git submodule, however: libsquish. It can be downloaded as a ZIP from SourceForge but it is to my knownlege not possible to add it as a Git submodule.

It would require making a fork of libsquish here on GitHub, GitLab or another Git hosting provider. Ideally, that fork would automatically be updatd as well though that might not be possible on GitHub due to SourceForge using Subversion.

I have made a clone of libsquish at https://github.com/lmichaelis/phoenix-libsquish.

Origin: Try/OpenGothic#352

[v1.1.0] Add heuristic for automatic world version detection

One idea would be to search through the archive's key-map to find object keys only present in Gothic 2 VObs, like visualAniMode (probably unlikely to yield good results since it's in the base zCVob which is most likely packed), canMove or fadeOutSky. It might still be a good idea, however, to keep the explicit version around somehow, so consumers of the library can prevent loading the wrong version anyways.

Improve error reporting

At the moment, phoenix only reports some failures using custom exception classes. Ideally, any failure which needs to be reported to the user should be wrapped in a custom exception class.

There is also a discussion to be had about using exceptions at all. It might be better to report failures using std::optional or a custom std::expected structure than throwing exceptions since some codebases don't allow exceptions at all.

Since this is a parser library, failure can occur more often than usual thus performance is also a point to keep in mind when using exceptions to report parsing failures.

[v1.1.0] Original Savegame Support

phoenix should support the original save-game format.

[v1.0.0] Mmap fails on empty files

    Hit runtime exception, at loading "Chronicles of Myrtana" mod:

...\TheChroniclesOfMyrtana\_work\Data\Scripts\_compiled\Fight.dat - is empty file, that causes mmap to fail internally.

// _deps\mio-src\include\mio\detail\mmap.ipp
inline mmap_context memory_map(const file_handle_type file_handle, const int64_t offset,
    const int64_t length, const access_mode mode, std::error_code& error)
{
...
    char* mapping_start = static_cast<char*>(::MapViewOfFile(
            file_mapping_handle,
            mode == access_mode::read ? FILE_MAP_READ : FILE_MAP_WRITE,
            win::int64_high(aligned_offset),
            win::int64_low(aligned_offset),
            length_to_map));
    if(mapping_start == nullptr) // <-- file exists, but mapping is null
    {
...

Originally posted by @Try in Try/OpenGothic#271 (comment)

Incorrect parsing on dragonisland.zen

Hi, I've found an issue with level parsing. Offended object:

This is invisible ore nugget, that triggers the big bridge, after fire-dragon. focus_override is needed, so player can aim it with bow.

focus_override value is wrongly false, with current main branch.
When debugging locally, I've found that problem is in read_bool routine:

bool archive_reader_binsafe::read_bool() {
	ensure_entry_meta(bs_bool);
-	return input.get_uint() != 1;
+	return input.get_uint() != 0;
	}

Proposed change is to replace all read_bool variants with !=0 - what is similar to how code would behave, if original game uses implicit case.

[v2.0.0] Allow phoenix itself to be linked (and installed) dynamically

Disable fsanitize on MnGW builds

https://github.com/lmichaelis/phoenix/blob/ff1ab4e7575050c8e2ae9fa36709b35823e8ecbf/CMakeLists.txt#L11-L22

MinGW has no -fsanitize support, should be disabled:

- else()
- elif (NOT WIN32)

[v1.1.0] Make the VM's stack more efficient

A std::stack does not have any relevant performance requirements, so it might be unsuitable for a semi high-performance application like the VM. Replacing it with a wrapper around a std::vector, for example should improve performance.

Multiple "not fully parsed" messages in Gothic 1

There are multiple "not fully parsed" error messages while running OpenGothic with the German version of Gothic 1.

To reproduce these, you don't have to do anything too difficult. Just open up OpenGothic with Gothic 1 and start a new game. No interaction with Diego is necessary.

Complete log:

OpenGothic v1.0 dev
no *.ini file in path - using default settings
GPU = NVIDIA GeForce RTX 2070
Depth format = Depth32F Shadow format = Depth16
[phoenix] world: parsing object [MeshAndBsp % 0 0]
[phoenix] bsp_tree: parsing chunk c000
[phoenix] bsp_tree: parsing chunk c010
[phoenix] bsp_tree: parsing chunk c040
[phoenix] bsp_tree: parsing chunk c045
[phoenix] bsp_tree: parsing chunk c050
[phoenix] bsp_tree: parsing chunk c0ff
[phoenix] world: parsing object [VobTree % 0 0]
[phoenix] world: parsing object [WayNet % 0 0]
[phoenix] world: parsing object [EndMarker % 0 0]
unable to load animation sequence: "PILLAR_7M-s_S1.MAN"
unable to load animation sequence: "PILLAR_7M-t_S1_2_S0.MAN"
unable to load sound fx: WOOD_NIGHT2
[phoenix] messages: oCMsgConversation("DIA_BaalTaran_Lehre_05_04") not fully parsed
[phoenix] messages: zCCSAtomicBlock("DIA_BaalTaran_Lehre_05_04") not fully parsed
[phoenix] messages: oCMsgConversation("DIA_Jarvis_Rest_08_01") not fully parsed
[phoenix] messages: zCCSAtomicBlock("DIA_Jarvis_Rest_08_01") not fully parsed
[phoenix] messages: oCMsgConversation("DIA_Mud_Nerve_14_07_00") not fully parsed
[phoenix] messages: zCCSAtomicBlock("DIA_Mud_Nerve_14_07_00") not fully parsed
[phoenix] messages: oCMsgConversation("DIA_Mud_Nerve_18_07_00") not fully parsed
[phoenix] messages: zCCSAtomicBlock("DIA_Mud_Nerve_18_07_00") not fully parsed
[phoenix] messages: oCMsgConversation("Info_Bloodwyn_PayDay_PayNoMore_08_01") not fully parsed
[phoenix] messages: zCCSAtomicBlock("Info_Bloodwyn_PayDay_PayNoMore_08_01") not fully parsed
[phoenix] messages: oCMsgConversation("Info_Diego_JoinAnalyze_Whistler_11_01") not fully parsed
[phoenix] messages: zCCSAtomicBlock("Info_Diego_JoinAnalyze_Whistler_11_01") not fully parsed
[phoenix] messages: oCMsgConversation("Info_GorNaToth_ARMOR_H_11_03") not fully parsed
[phoenix] messages: zCCSAtomicBlock("Info_GorNaToth_ARMOR_H_11_03") not fully parsed
[phoenix] messages: oCMsgConversation("Info_Jackal_Hello_WhatIf_07_02") not fully parsed
[phoenix] messages: zCCSAtomicBlock("Info_Jackal_Hello_WhatIf_07_02") not fully parsed
[phoenix] messages: oCMsgConversation("Info_Kirgo_Charge_Beer_15_02") not fully parsed
[phoenix] messages: zCCSAtomicBlock("Info_Kirgo_Charge_Beer_15_02") not fully parsed
[phoenix] messages: oCMsgConversation("Info_Vlk_1_EinerVonEuchWerden_01_01") not fully parsed
[phoenix] messages: zCCSAtomicBlock("Info_Vlk_1_EinerVonEuchWerden_01_01") not fully parsed
[phoenix] messages: oCMsgConversation("Info_Xardas_DANGER_14_03") not fully parsed
[phoenix] messages: zCCSAtomicBlock("Info_Xardas_DANGER_14_03") not fully parsed
[phoenix] messages: oCMsgConversation("Info_Xardas_LOADSWORD01_14_04") not fully parsed
[phoenix] messages: zCCSAtomicBlock("Info_Xardas_LOADSWORD01_14_04") not fully parsed
[phoenix] messages: oCMsgConversation("KDF_402_Corristo_KREIS4_Info_14_04") not fully parsed
[phoenix] messages: zCCSAtomicBlock("KDF_402_Corristo_KREIS4_Info_14_04") not fully parsed
[phoenix] messages: oCMsgConversation("KDF_402_Corristo_WANNBEKDF_Info_14_05") not fully parsed
[phoenix] messages: zCCSAtomicBlock("KDF_402_Corristo_WANNBEKDF_Info_14_05") not fully parsed
[phoenix] messages: oCMsgConversation("Org_819_Drax_Creatures_Fell_06_02") not fully parsed
[phoenix] messages: zCCSAtomicBlock("Org_819_Drax_Creatures_Fell_06_02") not fully parsed
[phoenix] messages: oCMsgConversation("Org_859_Aidan_Creatures_Fell_13_02") not fully parsed
[phoenix] messages: zCCSAtomicBlock("Org_859_Aidan_Creatures_Fell_13_02") not fully parsed
[phoenix] messages: oCMsgConversation("Sit_2_PSI_Yberion_BringFocus_WO_12_02") not fully parsed
[phoenix] messages: zCCSAtomicBlock("Sit_2_PSI_Yberion_BringFocus_WO_12_02") not fully parsed
comb not found: t_FIREPLACE_Stand_2_S0 -> c_FP_STAND_2_S0_(c_FP_STAND_2_S0_2)
comb not found: t_FIREPLACE_S0_2_Stand -> c_FP_STAND_2_S0_(c_FP_STAND_2_S0_2)
alias not found: t_Walk_2_WalkBL -> t_walkL_2_Walk
alias not found: t_SwimF_2_Dive -> t_swim_2_dive
alias not found: t_SwimF_2_Dive -> t_swim_2_dive
[phoenix] vm: accessing member "C_NPC.ATTRIBUTE" without an instance set
alias not found: r_Roam1 -> r_Scratch
alias not found: t_FallenB_2_Stand -> t_Fallen_2_Stand	
inserNpc: invalid waypoint
inserNpc: invalid waypoint
alias not found: t_FistWalkBL_2_FistWalk -> t_FistWalk_2_FistWalkL
comb not found: t_FIREPLACE_Stand_2_S0 -> c_FP_STAND_2_S0_(c_FP_STAND_2_S0_2)
comb not found: t_FIREPLACE_S0_2_Stand -> c_FP_STAND_2_S0_(c_FP_STAND_2_S0_2)
alias not found: t_Walk_2_WalkBL -> t_walkL_2_Walk
comb not found: t_FIREPLACE_Stand_2_S0 -> c_FP_STAND_2_S0_(c_FP_STAND_2_S0_2)
comb not found: t_FIREPLACE_S0_2_Stand -> c_FP_STAND_2_S0_(c_FP_STAND_2_S0_2)
room not found: NLHU03
room not found: NLHU05
room not found: NLHU06
room not found: NLHU09
room not found: NLHU08
room not found: NLHU07
room not found: NLHU04
room not found: NLHU02
room not found: NLHU01
not implemented call [WLD_SETOBJECTROUTINE]
room not found: H�TTE5
room not found: H�TTE34
[phoenix] vm: accessing member "C_NPC.ID" without an instance set
[phoenix] vm: accessing member "C_NPC.NAME" without an instance set
not implemented call [WLD_EXCHANGEGUILDATTITUDES]
room not found: NLHU03
room not found: NLHU05
room not found: NLHU06
room not found: NLHU09
room not found: NLHU08
room not found: NLHU07
room not found: NLHU04
room not found: NLHU02
room not found: NLHU01
room not found: H�TTE5
room not found: H�TTE34
[phoenix] vm: accessing member "C_NPC.ID" without an instance set
[phoenix] vm: accessing member "C_NPC.NAME" without an instance set
[phoenix] world: parsing object [VobTree % 0 0]
[phoenix] world: parsing object [EndMarker % 0 0]
not implemented call [mdl_applyrandomani]
not implemented call [mdl_applyrandomanifreq]
not implemented call [NPC_HASREADIEDWEAPON]
not implemented call [AI_LOOKAT]
not implemented call [NPC_SETKNOWSPLAYER]

Build error on Ubuntu/GCC

https://github.com/lmichaelis/phoenix/blob/0c29d03205a7bbf0ca5200a683a30be0157e43b8/include/phoenix/mesh.hh#L45
Last good revision: 32770f8
Blamed: 1957cbb

-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
...
[ 40%] Building CXX object lib/phoenix/CMakeFiles/phoenix.dir/source/animation.cc.o
In file included from /home/appveyor/projects/opengothic/lib/phoenix/include/phoenix/animation.hh:6,
                 from /home/appveyor/projects/opengothic/lib/phoenix/source/animation.cc:3:
/home/appveyor/projects/opengothic/lib/phoenix/include/phoenix/mesh.hh:45:49: error: ‘bool phoenix::polygon_flags::operator==(const phoenix::polygon_flags&) const’ cannot be defaulted
   45 |   bool operator==(const polygon_flags&) const = default;
      |                                                 ^~~~~~~
make[3]: *** [lib/phoenix/CMakeFiles/phoenix.dir/build.make:76: lib/phoenix/CMakeFiles/phoenix.dir/source/animation.cc.o] Error 1
make[2]: *** [CMakeFiles/Makefile2:709: lib/phoenix/CMakeFiles/p

Also this operator appears to be unused, probably it would be good to remove this line.

[v2.0.0] Decouple dropping LOD-polygons from loading MRMs

Currently, when loading MRMs (or phoenix::meshes), a secondary parameter for only including certain polygons can be provided. This is used mostly by phoenix::world::parse to exclude level-of-detail polygons from the world mesh.

This functionality should

Not be forced onto the user when loading worlds
Be a function to call on a mesh, instead of being passed to parse.

[v2.0.0] Update the CMake build (and library code) for easier distribution

Currently, the CMake build script is pretty bare-bones and does not allow for easy, clean distribution of the library code. This is due to being statically linked as well as pulling in 3rd-party libraries directly using target_link_libraries(... PUBLIC ...) which causes CMake's install process to malfunction by including 3rd-party library code in the installation.

This is unacceptable for installing phoenix into systems directly since the extra files might pollute existing installations of those libraries or could cause the package manager to break.

Things to do

Fix the install step of CMake build
(potentially) Depend on shared 3rd-party libraries instead of the statically linked ones wherever possible
#48

[v1.1.0] Check whether the VOb field `focusOverride` is actually an enum

Currently, focusOverride is parsed as a bool but that might not be correct. A look into the disassembly might be helpful in determining whether is is actually a bool or might be an enum instead.

Related: #44

Build order polution with libsquish

After building OpenGothic, sub-module OpenGothic\lib\phoenix\vendor\libsquish becomes modifyed:

HEAD detached at b28220d
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
  (commit or discard the untracked or modified content in submodules)
        modified:   vendor/libsquish (modified content)
...
HEAD detached at ae7054a
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   squish_export.h

I'm not sure, if this header usefull at all. Also part of that: libsquish is only to implement texture::as_rgba8, right? Can it be removed eventually?