Problem when try to build a DLL for the API about hololens2forcv HOT 18 OPEN

@aal-rvr Dll and Unity build have to be ARM64. I am loading the the rm API with LoadLibraryA("ResearchModeApi") as well and #include "researchmode/ResearchModeApi.h" it in my header file. I'm also using a VS 2019 DLL (Universal Windows) Project template.
I defined m_pSensorDevice as global and static:

extern "C"
{
	class DllExports
	{
	public:
		DllExport static void InitSensors();
		...
	};

	//##### Global Vars #####
	// static vars for research mode init
	static IResearchModeSensorDevice* s_SensorDevice;
	static std::vector<ResearchModeSensorDescriptor> s_SensorDescriptorsList;

	// static vars for sensor access
	static IResearchModeSensor* s_AccSensor;
	static IResearchModeSensor* s_GyroSensor;
	static IResearchModeSensor* s_MagnetSensor;
	static size_t s_AccSampleBufferSize;
	static size_t s_GyroSampleBufferSize;
	static size_t s_MagnetSampleBufferSize;
}

And use similar code as @whirwind to init the sensors.
I tested my DLL outside of Unity first, with a normal C++ VS UWP Project. There you have to set 'Content' to 'Yes' (in the DLL's properties) otherwise the DLL will not be copied to the HL2 and therefore cannot be found. In Unity this should not be a problem. But you have to modify the .appxmanifest of the Unity build before you deploy it to your HL2.
If you can't see any probems in your project environment, maybe you can use GetLastError() inside your DLL to get some more infos about your error.

from hololens2forcv.

@dakesse Exactly. I was working for this past few days. I return a Byte* pointer in C++ and receive it in C# using IntPtr, then use marshal.Copy. But today I found that app will have crashing problem. I guess it is because I did something stupid while passing the pointer and lead to some kinds of memory leak.

1. If I get the 'IntPtr Ptr=DLL.GetImage()', and do 'marshal.copy', then call a 'ReleaseMemory(Ptr)'. The app will crash immediately at beginning.
2. If I just get the IntPtr and do marshal.copy without 'ReleaseMemory(Ptr)', the app will run for a short time and crash eventually.

Do you have any advice about this? Thanks.

@agroenenberg Sorry for the late reply. It is clear in the init() function in samples. It is some thing like this:
`

HMODULE hrResearchMode = LoadLibraryA("ResearchModeAPI");
if (hrResearchMode)
{
    typedef HRESULT(__cdecl* PFN_CREATEPROVIDER) (IResearchModeSensorDevice** ppSensorDevice);
    PFN_CREATEPROVIDER pfnCreate = reinterpret_cast<PFN_CREATEPROVIDER>(GetProcAddress(hrResearchMode, "CreateResearchModeSensorDevice"));
    if (pfnCreate)
    {
        hr = pfnCreate(&m_pSensorDevice);
    }
    else
    {
        hr = E_INVALIDARG;
    }
}


m_pSensorDevice->QueryInterface(IID_PPV_ARGS(&m_pSensorDeviceConsent));



m_pSensorDevice->DisableEyeSelection();

m_pSensorDevice->GetSensorCount(&sensorCount);
m_sensorDescriptors.resize(sensorCount);

m_pSensorDevice->GetSensorDescriptors(m_sensorDescriptors.data(), m_sensorDescriptors.size(), &sensorCount);`

from hololens2forcv.

Hi!

I am having a similar problem. I had the same error trying to build a DLL for my Unity app.
I followed the sample code - making an explicit call to load the DLL, using LoadLibraryA("ResearchModeAPI") and then doing the following:

typedef HRESULT(__cdecl* PFN_CREATEPROVIDER) (IResearchModeSensorDevice** ppSensorDevice);
        PFN_CREATEPROVIDER pfnCreate = reinterpret_cast<PFN_CREATEPROVIDER>(GetProcAddress(hrResearchMode, "CreateResearchModeSensorDevice"));
        if (pfnCreate)
        {
            winrt::check_hresult(pfnCreate(&m_pSensorDevice));
        }
        else
        {
            winrt::check_hresult(E_INVALIDARG);
        }

I got the DLL to build and was able to call it through my C# program but the line: HMODULE hrResearchMode = LoadLibraryA("ResearchModeAPI"); returns NULL.

Is there a way to make an implicit call like suggested in the documentation.
What would be the correct way to load the ResearchModeAPI Library?
Does the plugin DLL being called from the C# process cause this problem? Do I need to do this from a separate process? and if so, how do we share m_pSensorDevice pointer across processes?

Thanks.

from hololens2forcv.

I would not try to return research mode interfaces into unity. Just export entry points that call into the rm apis. Basically initialize open, close and getbuffer exports from your dll, Also if your Unity app is not arm64 the RM dll will not load. We are working on adding arm support.

from hololens2forcv.

Hi @aal-rvr

It seems that you are using windows runtime component template to build the project? My project template in VS2019 is DLL(Universal Windows). I just followed sample in the API Doc, so my code is extremely simple .

CreateResearchModeSensorDevice(&m_pSensorDevice); // Factory function
m_pSensorDevice->GetSensorCount(&sensorCount);
m_pSensorDevice->GetSensorDescriptors(m_sensorDescriptors.data(), m_sensorDescriptors.size(), &sensorCount);

for (const auto& sensorDescriptor : m_sensorDescriptors)
{
    m_pSensorDevice->GetSensor(sensorDescriptor.sensorType, &pSensor);
    pSensor->GetSampleBufferSize(&sampleBufferSize);
    pSensor->OpenStream();
    for (UINT i = 0; i < 4; i++)
    {
        pSensor->GetNextBuffer(&pSensorFrame);
        ProcessFrame(pSensor, pSensorFrame, i);
    }
    pSensor->CloseStream();
    pSensor->Release();
}

Hi @dorinung
Yes, exactly. I don't return any research mode interfaces. I just plan to return some general parameters such as resolution, data buffer, or status. So I think this is not the reason for that error. The error I guess is something about my project environment. I am still looking for my mistake.

Best,
whirwind

from hololens2forcv.

@dakesse Thanks for your advice, I successfully get a resolution parameter from my DLL in Unity. Although more work need to do to get the raw sensor data, initial problem has been solved. Here is what I learn:

1. About the initial problem, just follow sample code and use LoadLibraryA("ResearchModeAPI") to call 'CreateResearchModeSensorDevice' function
2. Once the Unity build is done, follow sample projects and modify 'Package' and 'Capability' in Package.appxmanifest
3. If the implementation of the code in the document is different from the sample, try to follow the sample first.

from hololens2forcv.

@whirwind did you manage to marshal the data buffer of
sensorAccelFrame->GetCalibratedAccelarationSamples(&accelBuffer, &bufferOutLength);
to use it in C#?

from hololens2forcv.

@whirwind , I get the exact same error as you did in the beginning. How did you call the 'CreateResearchModeSensorDevice' function via LoadLibraryA("ResearchModeApi") ? I can't seem to get it working. Thanks 👍

from hololens2forcv.

@whirwind
I first tried it with marshalling but due to multiple assembly errors on C#- and (memory) access errors on C++-side I tried using 'unsafe'. It's surely not the best way to do it but it's working for accelero and gyro values. But somehow accessing the magnetometer causes a crash. I have to take a deeper look on that. But here is how I did it:

On the C++ side I added a new struct because I had problems to access the array of the sensor values inside the respective struct array. So I just splitted them up into 3 single values.

struct SensorDataStruct {
		uint64_t VinylHupTicks;
		uint64_t SocTicks;
		float SensorValueX;
		float SensorValueY;
		float SensorValueZ;
		float temperature; };

My functions then return a pointer to such a struct array.
DllExport static SensorDataStruct* GetAccDataBatched();
Inside these functions I did the following.

	// init the return array
	SensorDataStruct* data = new SensorDataStruct[ACCBATCHSIZE];

        // get the sensor Frame and next buffer
        ...

	if (SUCCEEDED(hr)) {
		// read sensor values ...
		hr = sensorAccelFrame->GetCalibratedAccelarationSamples(&accelBuffer, &bufferOutLength);
		…

		// … and copy them to return array
		for (UINT i = 0; i < bufferOutLength; i++) {
			data[i].VinylHupTicks = accelBuffer[i].VinylHupTicks;
			data[i].SocTicks = accelBuffer[i].SocTicks;
			data[i].SensorValueX = accelBuffer[i].AccelValues[0];
			data[i].SensorValueY = accelBuffer[i].AccelValues[1];
			data[i].SensorValueZ = accelBuffer[i].AccelValues[2];
			data[i].temperature = accelBuffer[i].temperature;
		}
		// release resources
	        ...

        // and return the data
	return data;

On the C# side I have the same struct with corresponding C# types that I just use inside a 'unsafe' block to get the data from my C++ DLL.

    public unsafe struct SensorDataStruct {
        public ulong VinylHupTicks;
        public ulong SocTicks;
        public float SensorValueX;
        public float SensorValueY;
        public float SensorValueZ;
        public float temperature; };

By declaring the imported function as 'unsafe' you can just use pointers as if you where working with C++.

[DllImport("newDll.dll", EntryPoint = "?GetAccDataBatched@DllExports@@SAPEAUSensorDataStruct@@XZ", CallingConvention = CallingConvention.Cdecl)]
public unsafe static extern SensorDataStruct* getAccDataBatched();

 // init sensor
 …

unsafe {
	// get data from dll ...
	SensorDataStruct* dataPointer = getAccDataBatched(); ;

	for (int i = 0; i < ACCBATCHSIZE; i++) {
	// … and save it in a safe struct
	}
}

To work with the retreived data everywhere else in Unity i copy them into another safe struct.

from hololens2forcv.

Okay I've been trying to get this DLL working on the HoloLens 2 for days now and it's driving me to insanity. I tried every way I could find, the testfunction works perfectly on Unity but I keep getting this error on the HL2:

DLLTestRun.exe' (Win32): Loaded 'U:\USERS\DefaultAccount\AppData\Local\DevelopmentFiles\Template3DVS.Debug_ARM64.Annabel\DLLResearchMode.dll'. Symbols loaded.
'DLLTestRun.exe' (Win32): Unloaded 'U:\USERS\DefaultAccount\AppData\Local\DevelopmentFiles\Template3DVS.Debug_ARM64.Annabel\DLLResearchMode.dll'
Exception thrown at 0x00007FFD50CD39EC in DLLTestRun.exe: Microsoft C++ exception: Il2CppExceptionWrapper at memory location 0x000000CFB72FCA50.
DllNotFoundException: Unable to load DLL 'DLLResearchMode': The specified module could not be found.
  at CallTest.TestFunction (System.Int32 a, System.Int32 b) [0x00000] in <00000000000000000000000000000000>:0

I did everything as described, built the DLL for x64 and ARM64, put them in the right folders in Unity(ARM64 for UWP and WSAplayer only), x64 for every platform except WSAplayer. Added DLL to project and set content to "Yes". I tried everything with each time a different component. (only ARM64, only x64, build via Unity, not adding to project). Everthing built and configured in Debug Mode. It should just give me (int a + int b) in the VS terminal after uploading to HL2. Please can anyone help me? @dakesse @whirwind How did you guys get it working on the HL2?

from hololens2forcv.

@aal-rvr Dll and Unity build have to be ARM64. I am loading the the rm API with LoadLibraryA("ResearchModeApi") as well and #include "researchmode/ResearchModeApi.h" it in my header file. I'm also using a VS 2019 DLL (Universal Windows) Project template.
I defined m_pSensorDevice as global and static:

extern "C"
{
	class DllExports
	{
	public:
		DllExport static void InitSensors();
		...
	};

	//##### Global Vars #####
	// static vars for research mode init
	static IResearchModeSensorDevice* s_SensorDevice;
	static std::vector<ResearchModeSensorDescriptor> s_SensorDescriptorsList;

	// static vars for sensor access
	static IResearchModeSensor* s_AccSensor;
	static IResearchModeSensor* s_GyroSensor;
	static IResearchModeSensor* s_MagnetSensor;
	static size_t s_AccSampleBufferSize;
	static size_t s_GyroSampleBufferSize;
	static size_t s_MagnetSampleBufferSize;
}

And use similar code as @whirwind to init the sensors.
I tested my DLL outside of Unity first, with a normal C++ VS UWP Project. There you have to set 'Content' to 'Yes' (in the DLL's properties) otherwise the DLL will not be copied to the HL2 and therefore cannot be found. In Unity this should not be a problem. But you have to modify the .appxmanifest of the Unity build before you deploy it to your HL2.
If you can't see any probems in your project environment, maybe you can use GetLastError() inside your DLL to get some more infos about your error.

Hi @dakesse ,

While trying to make the DLL work for a C++ UWP app, I am running into an error: 'A dependent dll was not found'.
I tried adding the .dll file to the Project and set the 'Content' to 'Yes', but that still gives the same error. I have tried other things like, adding the .lib file to references and including the header file (which is what I have always done to add a DLL to a project).

Do you know what else needs to be changed?

Thanks!

from hololens2forcv.

I found the solution to my problem and maybe for yours too @aal-rvr . I installed Dependency Walker to check if my DLL was dependent on any other dlls that were not included. After some checks and research this is what solved it: When building the DLL itself, I changed C/C++> Code Generation> Runtime Library from MDd to MTd and now it works! Even if yours doesn't work, try https://www.dependencywalker.com/ to check your dll on dependencies.

from hololens2forcv.

@aal-rvr Dll and Unity build have to be ARM64. I am loading the the rm API with LoadLibraryA("ResearchModeApi") as well and #include "researchmode/ResearchModeApi.h" it in my header file. I'm also using a VS 2019 DLL (Universal Windows) Project template.
I defined m_pSensorDevice as global and static:

extern "C"
{
	class DllExports
	{
	public:
		DllExport static void InitSensors();
		...
	};

	//##### Global Vars #####
	// static vars for research mode init
	static IResearchModeSensorDevice* s_SensorDevice;
	static std::vector<ResearchModeSensorDescriptor> s_SensorDescriptorsList;

	// static vars for sensor access
	static IResearchModeSensor* s_AccSensor;
	static IResearchModeSensor* s_GyroSensor;
	static IResearchModeSensor* s_MagnetSensor;
	static size_t s_AccSampleBufferSize;
	static size_t s_GyroSampleBufferSize;
	static size_t s_MagnetSampleBufferSize;
}

And use similar code as @whirwind to init the sensors.
I tested my DLL outside of Unity first, with a normal C++ VS UWP Project. There you have to set 'Content' to 'Yes' (in the DLL's properties) otherwise the DLL will not be copied to the HL2 and therefore cannot be found. In Unity this should not be a problem. But you have to modify the .appxmanifest of the Unity build before you deploy it to your HL2.
If you can't see any probems in your project environment, maybe you can use GetLastError() inside your DLL to get some more infos about your error.

Hi @dakesse ,

While trying to make the DLL work for a C++ UWP app, I am running into an error: 'A dependent dll was not found'.
I tried adding the .dll file to the Project and set the 'Content' to 'Yes', but that still gives the same error. I have tried other things like, adding the .lib file to references and including the header file (which is what I have always done to add a DLL to a project).

Do you know what else needs to be changed?

Thanks!

Hi @aal-rvr ,
I'm not sure if I had the exact same error. But I remember that at some point I had to add the .lib file as well and set 'Content' to 'Yes' in its settings.
It seemes a bit random to me because I just tested it again with my C++ UWP app and it still worked after excluding the .lib file. I'm sorry not being a big help with that.

from hololens2forcv.

I found the solution to my problem and maybe for yours too @aal-rvr . I installed Dependency Walker to check if my DLL was dependent on any other dlls that were not included. After some checks and research this is what solved it: When building the DLL itself, I changed C/C++> Code Generation> Runtime Library from MDd to MTd and now it works! Even if yours doesn't work, try https://www.dependencywalker.com/ to check your dll on dependencies.

Nice that you found a solution.
My setting for this is '/MD'. But perhaps there are some other differences in our project setting that caused the errors.
Dependency Walker is always worth a try.

from hololens2forcv.

I found the solution to my problem and maybe for yours too @aal-rvr . I installed Dependency Walker to check if my DLL was dependent on any other dlls that were not included. After some checks and research this is what solved it: When building the DLL itself, I changed C/C++> Code Generation> Runtime Library from MDd to MTd and now it works! Even if yours doesn't work, try https://www.dependencywalker.com/ to check your dll on dependencies.

Thanks for the advice. My setting is /MDd by default. My original DLL might have had some differences as it was created in VS2017 (although the properties match my new DLL exactly), After quickly making a UWP DLL in VS 2019 it works as expected for C++ UWP Blank App. For this I only added the .lib file and didn't have to add the DLL.
I don't know what was wrong with my previous DLL. I could have missed a property somewhere.
Thanks!

from hololens2forcv.

Did anyone actually use a Windows Runtime Component instead of a DLL? I'm trying to record/ save 1 pointcloud from the AHAT sensor so I can process that on my desktop and send some data back to the HL2. I don't want/need any frames of the data on my Unity app, I just want to save a pcloud via my app that I can then access.
Unfortunately, the DLL format doesn't allow for using winrt::Windows::Something so the StartRecording & StopRecording functions can't be incorporated in the DLL... Would building it in a Windows Runtime Component be an option?

from hololens2forcv.

@dakesse Hi there, thanks for your detailed instructions.
I see only ACCEL, GYRO, MAG sensor types on your code. Have you ever tested camera sensors (VLC or depth) with the way of extern "C"?

from hololens2forcv.

@dakesse Hi there, thanks for your detailed instructions.
I see only ACCEL, GYRO, MAG sensor types on your code. Have you ever tested camera sensors (VLC or depth) with the way of extern "C"?

Sorry, unfortunately not. I was only interested in the IMU data.

from hololens2forcv.