Git Product home page Git Product logo

advanceddlsupport's Introduction

AdvancedDLSupport

Alternative approach to your usual P/Invoke!

Join the chat at https://discord.gg/fDy5Vhb

Use C# interfaces to bind to native code - quick and easy usage of C API in C# code, on any platform. Gone are the days of broken DllImport and annoying workarounds for the different runtimes.

Fully compatible with Mono, .NET Framework, .NET Core, and .NET Standard. Compatible with Mono DLL mapping on all platforms and runtimes. Configurable and adaptible.

Why use ADL?

  1. Modern API - no more static classes, no more extern. Use your native API as if it were first-class objects.
  2. Flexibility - Inject your native API into your classes, change the library scanning logic, mix your managed and native code.
  3. Speed - ADL is blazing fast, and gives your native interop an edge. See performance increases that are at least 2 and up to 8 times faster than other, existing solutions.
  4. Easy to use - Not only is ADL simple to set up and get working, it's a breeze to maintain, and reduces clutter in your codebase.

Read the Docs, or install via NuGet and get started.

Features

  • Supports all the typical P/Invoke patterns and constructs
  • Seamlessly mix native functions and managed code
  • Use more complex types, such as Nullable<T> and string without any extra code
  • Select library architectures at runtime
  • Select library names at runtime
  • Swappable native library search algorithms
  • Import global variables
  • Optional lazy loaded symbols
  • Optional Mono DllMap support

Basic Usage

  1. Declare your interface

    public interface IMyNativeLibrary
    {
    	long MyNativeGlobal { get; set; }
    	int MyNativeMultiply(int a, int b);
    	void MyOtherNativeFunction(MyStruct strct, ref MyStruct? maybeStruct);
    }
  2. Activate it

    const string MyLibraryName = "MyLibrary";
    
    var activator = new NativeLibraryBuilder();
    var library = activator.ActivateInterface<IMyNativeLibrary>(MyLibraryName);
  3. Use it

    library.MyNativeGlobal = 10;
    
    var result = library.MyNativeMultiply(5, 5);
    
    var myStruct = new MyStruct();
    MyStruct? myOtherStruct = null;
    
    library.MyOtherNativeFunction(myStruct, ref myOtherStruct);

See the Quickstart for more information.

Installation

Get it on NuGet!

Support me

Become a Patron Buy Me a Coffee at ko-fi.com

License

If the library's license doesn't fit your project or product, please contact us. Custom licensing options are available, and we are always open to working something out that fits you - be it modified, commercial, or otherwise.

AdvancedDLSupport's public release is licensed under the GNU Lesser General Public License, Version 3 (LGPLv3). See the LICENSE for details. Without the support of the open-source movement, it would never have existed.

advanceddlsupport's People

Contributors

john-h-k avatar jvbsl avatar nihlus avatar perksey avatar realvictorprm avatar sunkin351 avatar sylveon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

advanceddlsupport's Issues

"Symbol loading failed" when trying to use a method with ref struct argument

Description

I have successfully used this project for a small test, but I somehow can't seem to get things working when it gets complex.

My use case is mainly to optimize and simplify the usage of Win32 APIs like User32 and Kernel32 etc.

I started with:

    public interface IUser32
    {
        /// <summary>
        /// Retrieves a handle to the desktop window. The desktop window covers the entire screen. The desktop window is the area on top of which other windows are painted.
        /// See <a href="https://msdn.microsoft.com/en-us/library/windows/desktop/ms633504(v=vs.85).aspx">GetDesktopWindow function</a>
        /// </summary>
        /// <returns>the return value is a handle to the desktop window.</returns>
        IntPtr GetDesktopWindow();
    }

And generate native code with:

        public static IUser32 GenerateUser32()
        {
            var activator = new NativeLibraryBuilder(ImplementationOptions.UseIndirectCalls);
            return activator.ActivateInterface<IUser32>("user32");
        }

This works, and the performance improvement compated to DllImport (measured with BenchmarkDotNet) is almost 2 fold. This is very cool!

Now I wanted to make the test more complex, using ref / structs, and got into a problem.
The P/Invoke for the example can be found here: https://www.pinvoke.net/default.aspx/kernel32.getversionex

Repro steps

Please provide the steps required to reproduce the problem

  1. Create an interface for the Kernel32 lib
    public interface IKernel32
    {
        /// <summary>
        /// See <a href="https://msdn.microsoft.com/en-us/library/windows/desktop/ms724451(v=vs.85).aspx">GetVersionEx function</a>
        /// See also <a href="https://www.pinvoke.net/default.aspx/kernel32.getversionex">pinvoke getversionex</a>
        /// </summary>
        /// <param name="osVersionInfo">ref to OsVersionInfoEx</param>
        /// <returns>bool</returns>
        bool GetVersionEx(ref OsVersionInfoEx osVersionInfo);
    }

 [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
    public struct OsVersionInfoEx
    {
        public int dwOSVersionInfoSize;
        public int dwMajorVersion;
        public int dwMinorVersion;
        public int dwBuildNumber;
        public int dwPlatformId;
        [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 128)]
        public string szCSDVersion;
        public ushort wServicePackMajor;
        public ushort wServicePackMinor;
        public ushort wSuiteMask;
        public byte wProductType;
        public byte wReserved;
    }

  1. Generate the native code
        public static IKernel32 GenerateKernel32()
        {
            var activator = new NativeLibraryBuilder(ImplementationOptions.UseIndirectCalls);
            return activator.ActivateInterface<IKernel32>("kernel32");
        }
  1. Call the code
            OsVersionInfoEx osVersionInfoEx = new OsVersionInfoEx();
            if (!_kernel32.GetVersionEx(ref osVersionInfoEx))
            {
                throw new NotSupportedException("Whaaaat?");
            }

Expected behavior

No exception, and the structure osVersionInfoEx should be filled.

Actual behavior

I get an exception:

System.Reflection.TargetInvocationException: Ein Aufrufziel hat einen Ausnahmefehler verursacht. ---> AdvancedDLSupport.SymbolLoadingException: Symbol loading failed. Symbol name: GetVersionEx ---> System.ComponentModel.
Win32Exception: Die angegebene Prozedur wurde nicht gefunden (The supplied function was not found)
bei AdvancedDLSupport.NativeLibraryBuilder.ActivateClass[TClass,TInterface](String libraryPath)
bei AdvancedDLSupport.NativeLibraryBuilder.ActivateInterface[TInterface](String libraryPath)

Related information

  • Operating system - Windows 7 & Windows 10
  • What version of the library - NuGet v1.2.2
  • What runtime - .NET Framework 4.7
  • Workarounds - None known

Any tipps & hints? I probably overlooked something..

Feature Request: Custom symbol loading logic

I'm wish to use ADL for loading OpenGL. Current logic uses wglGetProcAddress (Exported from the main library) as a backup when the primary method for loading a symbol fails. Can I replicate this with ADL? (This lets me support OGL extensions which are not exported from the main library)

Implementation of less accessible types

Is your feature request related to a problem? Please describe.

internal interface IA
{
   void DoStuff();
}
public abstract class X : NativeLibraryBase, IA
{
...
}

In my opinion it should create an implementation like that:

public class X_asdf : X
{
   void IA.DoStuff()
   {
   }
}

while it does create more of something like this:

public class X_asdf : X
{
   public void DoStuff()
   {
   }
}

Describe the solution you'd like
First we should rename the functions to [InterfaceName].[MethodName] for all types with less accessibility for consistency sake(with C#, IL itself can be named like you want afaik):
https://github.com/Firwood-Software/AdvanceDLSupport/blob/master/AdvancedDLSupport/Pipeline/ImplementationPipeline.cs#L202

Secondly this one should use "Private" instead of "Public" Flag for all types with less accessibility:
https://github.com/Firwood-Software/AdvanceDLSupport/blob/master/AdvancedDLSupport/Pipeline/ImplementationPipeline.cs#L203

I noticed this one because in OpenTK some interfaces are internal while in my opinion should be public.

Marshaling ref struct for return value

Linux OS: Arch Linux (Latest Update Nov-27-2018)
Dotnet Version: 2.1.500
Clang: 7.0.0
Compile CLI for C: clang -std=c99 -fPIC -shared -olibChapSix.so ChapSix.c
ADL Version: 2.3.0 (From Nuget)

The problem is that Ref Struct does not seem to be supported for return value when defined in interface:

C File: (ChapSix.c -> libChapSix.so via Clang)

typedef struct
{
    char StringData[128];
} ByValString;

ByValString* Initialize()
{
    return (ByValString*)malloc(sizeof(ByValString));
}

C# File:

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct ByValString
{
    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 128)]
    public string String_ByValTStr;
}

public unsafe interface IChapSixLib
{
    ref ByValString Initialize();
}

public class Program
{
    static void Main(string[] args)
    {
        var lib = NativeLibraryBuilder.Default.ActivateInterface<IChapSixLib>("ChapSix");
        var result = lib.Initialize();
    }
}

CSProj File Content (Including Build Target):

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.1</TargetFramework>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="AdvancedDLSupport" Version="2.3.0" />
  </ItemGroup>
  <Target Name="CompileCProject" AfterTargets="AfterBuild">
    <exec Command="clang -std=c99 -shared -fPIC -olibChapSix.so ChapSix.c" />
    <Copy SourceFiles="libChapSix.so" DestinationFolder="$(OutDir)" />
  </Target>
</Project>

The stacktrace:

Unhandled Exception: System.Runtime.InteropServices.MarshalDirectiveException: Cannot marshal 'return value': Invalid managed/unmanaged type combination.
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateClass(String libraryPath, Type baseClassType, Type[] interfaceTypes) in /home/jarl/Programming/AdvanceDLSupport/AdvancedDLSupport/NativeLibraryBuilder.cs:line 338
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateClass[TClass,TInterface](String libraryPath) in /home/jarl/Programming/AdvanceDLSupport/AdvancedDLSupport/NativeLibraryBuilder.cs:line 257
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateInterface[TInterface](String libraryPath) in /home/jarl/Programming/AdvanceDLSupport/AdvancedDLSupport/NativeLibraryBuilder.cs:line 191
   at Program.Main(String[] args) in /home/centi/Projects/Tutorials/ChapterSix/Program.cs:line 48

Support for loading exported symbols from main executable.

I'd like to host a .NET Core runtime inside of a native C++ executable. My main exe exports some symbols.

I'd like this project to be able to load symbols from the main executable.

I'd understand not having first-class support for this, but maybe consider making things public instead of internal, particularly this:

https://github.com/Firwood-Software/AdvanceDLSupport/blob/e3d5ce6485a269712e00e0a19bd2dbc8ec4b5d76/AdvancedDLSupport/NativeLibraryBase.cs#L54

Then, I could provide my own IPlatformLoader that does this.

Introduce Fody to do everything you do right now, but at compile time

No error, just an idea for your "roadmap":
Currently the code is generated at runtime, I myself don't have a reason for that and it feels to me like it would be more logical if the code is generated as part of the build. This should be possible by using https://github.com/Fody/Fody

I am considering this myself, for a library I wrote. It should make the library usable on most platforms, as code generation at runtime is an issue for some of them.

Maybe there are use cases which I don't know of, just felt like discussing this with you.

P.S.
There is also a Pluralsight course for it, if you happen to be a member:
https://app.pluralsight.com/library/courses/automatic-dotnet-code-weaving-fody/table-of-contents
Although it's a bit older...

attempting to implement an inaccessible interface

Please provide a minimal compilable example of

just a simple native code

typedef void(*ActionCallback)();

__declspec(dllexport) void RunCallback(ActionCallback callback)
{
	callback();
}
using System;

namespace Invoker
{
    using AdvancedDLSupport;

    class Program
    {
        public delegate void ActionCallback();

        public interface ITest
        {
            void RunCallback(ActionCallback callback);
        }

        static void Main(string[] args)
        {
            const string MyLibraryName = "native.dll";
            var library = NativeLibraryBuilder.Default.ActivateInterface<ITest>(MyLibraryName);
            library.RunCallback(() => Console.WriteLine("Callback ran!"));
        }
    }
}
dotnet Invoker.dll

Unhandled Exception: System.TypeLoadException: Type 'Generated_NativeLibraryBase
_a6553129_a2dc_4c8c_a872_7c2dc3c740bd' from assembly 'DLSupportDynamicAssembly,
Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' is attempting to implemen
t an inaccessible interface.
   at System.Reflection.Emit.TypeBuilder.TermCreateClass(RuntimeModule module, I
nt32 tk, ObjectHandleOnStack type)
   at System.Reflection.Emit.TypeBuilder.CreateTypeNoLock()
   at System.Reflection.Emit.TypeBuilder.CreateTypeInfo()
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateClass(String libraryPath, T
ype baseClassType, Type[] interfaceTypes)
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateClass[TClass,TInterface](St
ring libraryPath)
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateInterface[TInterface](Strin
g libraryPath)
   at Invoker.Program.Main(String[] args) in d:\Invoker\Program.cs:line 26

Desktop (please complete the following information):

  • OS: windows 8.1 64bit
  • Runtime .NET Core
  • Version 2.2

Additional context
Add any other context about the problem here.

Explain type matrix

From what I can tell, there isn't a description of how to map unions to .NET types. This is a problem for me since I have an API that uses unions and I'd like to bind it using this library. It would also be nice if there was something like a reference sheet that described both simple and complex types in C mapped to the associated simple and complex types in .NET. According to this SO question, the StructLayout attribute combined with the FieldOffset can simulate this with structs, but I don't know how well this would map across the FFI boundary. The library I'm binding has the following structs and associated union:

struct syz_AutomationPoint {
  int interpolation_type;
  double values[6];
  unsigned long long flags;
};

struct syz_AutomationAppendPropertyCommand {
  int property;
  struct syz_AutomationPoint point;
};

struct syz_AutomationClearPropertyCommand {
  int property;
};

struct syz_AutomationSendUserEventCommand {
  unsigned long long param;
};

union syz_AutomationCommandParams {
  struct syz_AutomationAppendPropertyCommand append_to_property;
  struct syz_AutomationClearPropertyCommand clear_property;
  struct syz_AutomationSendUserEventCommand send_user_event;
};

struct syz_AutomationCommand {
  syz_Handle target;
  double time;
  int type;
  unsigned int flags;
  union syz_AutomationCommandParams params;
};

SYZ_CAPI syz_ErrorCode syz_createAutomationBatch(syz_Handle *out, syz_Handle context, void *userdata,
                                                 syz_UserdataFreeCallback *userdata_free_callback);
SYZ_CAPI syz_ErrorCode syz_automationBatchAddCommands(syz_Handle batch, unsigned long long commands_len,
                                                      const struct syz_AutomationCommand *commands);
SYZ_CAPI syz_ErrorCode syz_automationBatchExecute(syz_Handle batch);

Therefore, one possible implementation of this translation might be:

using System.Runtime.InteropServices;

// ...
public struct AutomationPoint {
    public int InterpolationType;
    public double[6] Values;
    public ulong Flags;
}

public struct AutomationAppendPropertyCommand {
    public nint Property;
    public AutomationPoint Point;
}

public struct AutomationClearPropertyCommand {
    public int Property;
}

public struct AutomationSendUserEventCommand {
    public uint param;
}

[StructLayout(LayoutKind.Explicit, Pack=1)]
public struct AutomationCommandParams {
    [FieldOffset(0)]
    public AutomationAppendPropertyCommand AppendToProperty;
    [FieldOffset(56)]
    public AutomationClearPropertyCommand ClearProperty;
    // ...
}

The only major problem with this is that its majorly error-prone. I could use the sizeof operator but I'm not quite sure how I'd use that since FieldOffset requires a contiguous range. Is there a much better way of doing this?

Native callback binding to C# events and overall performance

Hello,

I have a native C++ application that is running time critical algorithms and is optimized for low-latency. As of right now, it is running on an optimized linux machine and I am very happy with the performance so far.

On the other hand, I want to write higher-level stuff of the application in C#, so I need some kind of interop between the two languages. I looked at a lot of IPC related stuff, but since there is .NET Core running on linux, my idea was to wrap the API of the native application into a shared library and access it from a C# executable with an interface and by using this awesome library.

The native API will be very simple and it basically has methods for starting and stopping the processing. This will setup a couple of threads and do the calculation intensive stuff natively and asynchronously. Implementing this methods and accessing them by AdvanceDLSupport or P/Invoke seems not a problem.

One critical thing I also need on the C# side, is to asynchronously receive statistical data from the native processing side. The statistic does not include big data (only a couple of floats and integer) and will be generated every 1-5 seconds.

My idea was to declare a callback on the C++ side (std::function mayby) that gets called whenever the native side wants to report.

This leads to the following questions:

How can I bind the native callback on the C++ side to their implementations on the C# side. Can I use standard C# events in the interface for this? And is this safe when using the GC at all?

Will the performance of the native part be degraded because the whole thing runs in a managed process with a GC? Even though the native code does its calculations asynchronously on multiple pthreads, I fear that the GC will interfer with it. It would be great if some of you could share your experiences with such implementations.

I am gratefull for any replies. Thank you
Philipp

Add support for delegate <-> function pointer interop

Description

Currently there is no way to pass a C# callback function as a parameter to a C function.
Typically this is done with regular pinvoke using something like...

    [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
    public delegate void ReadFileCallback(string path, string contents, string error);

    [DllImport("solc")]
    public static extern int compileStandard(string input, bool optimize, ReadFileCallback readCallback);

Actual behavior

Tried specifying the parameter as an IntPtr and passing it the result of Marshal.GetFunctionPointerForDelegate(MyMethod) but the C code ends up with something like nullptr.

Related information

  • Workarounds (if any)
    Can't find any workaround.

dylib not found on Mac.

Description

The dylib isn't found, even with DYLD_LIBRARY_PATH set correctly.

Repro steps

const string myLibraryName = "libQtNetCoreQml"; // or libQtNetCoreQml.dylib or QtNetCoreQml
var activator = new NativeLibraryBuilder();
var library = activator.ActivateInterface<IMyNativeLibrary>(myLibraryName);

Expected behavior

Expected it to find the library.

Actual behavior

It didn't.

Related information

I am using dotnet version 2.1.301 on OSX.

I did a little debugging and found the culprit. See here:

https://github.com/Firwood-Software/AdvanceDLSupport/blob/93201d655195279a65e2c0952099bc7f93e02b9b/AdvancedDLSupport/PathResolvers/MacOSPathResolver.cs#L55

The Path.Combine results in a valid path, but the Path.GetFullPath appends that current path to the working directory. So, it is checking if a file exists at /current-working-dir//some-path-in-ld/file.dylib when it should really be checking for a file at/some-path-in-ld/file.dylib.

Add support for NuGet package cache loading.

Is your feature request related to a problem? Please describe.
Recently, when trying to distribute a package containing native packages with the intention of invoking them using ADL, I noticed that ADL would not find the library on a non-RID-specific build but DllImport can.

I then found out that DllImport also checks the NuGet cache for native assemblies, but ADL doesn't.

Describe the solution you'd like
We need to investigate the best way to search across multiple paths similar to CoreFX. This could be done internally, or we could expose an API similar to .NET Core 3's new NativeLibrary API and use DllImportSearchPaths.

Either way, we need to find out what paths we need to be looking in and then we can go from there.

Additional context

  • CoreFX doesn't copy native libraries unless you build with a RID specified.

Multiple Implementations of Method

Describe the bug
I don't actually know if it is a problem with AdvanceDLSupport or the current code in OpenTK.
But I tried https://github.com/Nihlus/opentk/tree/al-adl with a really simple OpenAL test project which just tries to create ALContext at runtime.

Problem is that
https://github.com/Firwood-Software/AdvanceDLSupport/blob/1167a435759046c325d5e5f49923aff80433c367/AdvancedDLSupport/NativeLibraryBuilder.cs#L226
classType.GetInterfaces()
returns 7 Interfaces for me:
[0] = {RuntimeType} "System.IDisposable"
[1] = {RuntimeType} "OpenTK.OpenAL.Interfaces.IALC"
[2] = {RuntimeType} "OpenTK.OpenAL.Interfaces.IContext"
[3] = {RuntimeType} "OpenTK.OpenAL.Interfaces.IContextDevices"
[4] = {RuntimeType} "OpenTK.OpenAL.Interfaces.IContextErrors"
[5] = {RuntimeType} "OpenTK.OpenAL.Interfaces.IContextExtensions"
[6] = {RuntimeType} "OpenTK.OpenAL.Interfaces.IContextState"

which kinda does make sense, with following class structure:
public abstract class ALContext : NativeLibraryBase, IALC
where
internal interface IALC : IContext, IContextDevices, IContextErrors, IContextExtensions, IContextState

so we get a flat structure of all the interfaces. But after that we will read all methods per interface in a flat structure as well, meaning IALC will yield all methods, contained in implemented interfaces like IContext.
https://github.com/Firwood-Software/AdvanceDLSupport/blob/1167a435759046c325d5e5f49923aff80433c367/AdvancedDLSupport/NativeLibraryBuilder.cs#L578
e.g. CreateContext method will be created. Later on after that the IContext interface is searched and we'll find the CreateContext method again and will create it again as well with completly the same signature. Which obviously is not allowed in IL and throws an Exception:

System.TypeLoadException: Method 'CreateContext' on type 'Generated_ALContext_ada0976a_f17a_4000_b25f_76adbcb712db' from assembly 'DLSupportDynamicAssembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' is overriding a method that has been overridden.

Edit: Actually I found out that the problem was introduced with following commit:
14af9f8#diff-985c2f29072740d4022c57beb36d066dR574

I think the solution should be either to either not flattenHierarchy in GetIntrospectiveMethods call by passing false. Or not iterating over the given interfaces but using classType.GetIntrospectiveMethods(true) instead.
While I think the fist solution should be the better one.
https://github.com/Firwood-Software/AdvanceDLSupport/blob/1167a435759046c325d5e5f49923aff80433c367/AdvancedDLSupport/NativeLibraryBuilder.cs#L578

To Reproduce
Steps to reproduce the behavior:

  1. Clone https://github.com/Nihlus/opentk/
  2. and switch to al-adl branch
  3. Simple test project with following code in main:
    ALContext context = ALContext.GetAPI();

@Nihlus I'm cheeky enough to not post a minimal example, because I guess you are working on OpenTK as well and it should be much easier to debug it directly inside OpenTK...

Expected behavior
Creating the class successfully at runtime.

Desktop:

  • OS: Linux 64 Bit
  • Runtime NET Core
  • Version 2.1.403

Lazy Binding

Allows symbols to be loaded only when on demand.

Implement a way to marshal arrays as return types

Typically, arrays cannot be automatically marshalled as return types from native code, due to them being raw pointers without range information. This is a limitation in traditional P/Invoke, and one that is not usually solved on a binding level.

Describe the solution you'd like
Having ADL support a scenario where the length of an array could be retrieved from another member of the same type (a method, or a property) and then used to marshal the return pointer would be an interesting and useful solution to the problem.

Conceivably, an attribute such as [LengthSource(string memberName)] could be provided, which when applied to a return parameter would invoke or read the given member in order to retrieve the length of the array in question. As an example,

interface ILibrary
{
    UIntPtr GetNumNames();

    [return: LengthSource(nameof(GetNumNames))]
    string[] GetNames();
}

would be valid syntax.

The signature of the length retrieval method would be limited to predictable scenarios; initially, only parameterless methods returning a numeric type would be considered valid. Methods that do not fit the constraints would result in a bind-time exception.

The same would hold for properties - numeric properties only, and an accessible getter must be present.

Describe alternatives you've considered
At the moment, alternative options is to implement this type of behaviour yourself in a mixed-mode class.

Additional context
OpenTK would benefit from this features, as it has a number of signatures following this pattern.

Can't bind an interface that is internal.

It makes sense. But maybe you know some magic?

TypeLoadException: Type 'CallbacksIterop_a1fefae9_927a_498b_af14_5c370ec8783d' from assembly 'DLSupportDynamicAssembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' is attempting to implement an inaccessible interface.
internal interface IQGuiApplicationInterop
{
    [NativeSymbol(Entrypoint = "qguiapplication_create")]
    IntPtr Create();
}

I'd like to prevent people from poking at my guts.

Clarification on strings and how they are free'd?

Consider this line of code.

Q_DECL_EXPORT LPCSTR qt_getenv(LPCSTR name)
{
    QByteArray result = qgetenv(name);
    if(result.isNull()) {
        return nullptr;
    }
    char* resultString = new char[static_cast<rsize_t>(result.size() + 1)];
    strcpy_s(resultString, static_cast<rsize_t>(result.size() + 1), result.data());
    return resultString;
}
internal interface IQtInterop
{
    [NativeSymbol(Entrypoint = "qt_putenv")]
    bool PutEnv([MarshalAs(UnmanagedType.LPStr)]string name [MarshalAs(UnmanagedType.LPStr)]string value);
    [NativeSymbol(Entrypoint = "qt_getenv")]
    [return: MarshalAs(UnmanagedType.LPStr)]string GetEnv(string name);
}

Who is going to delete the buffer returned from qt_getenv? Also, if null (IntPtr.Zero) is returned, I assume that the free'in stuff doesn't happen?

The local path resolver uses the wrong base path

When resolving local paths, the local path resolver uses the location of the ADL assembly as its base path, and not the current directory.

See LocalPathResolver.cs#40.

As a temporary workaround, prefix the library name with "./" or ".", depending on your system.

Using UseIndirectCalls with a void method containing a boolean results in an InvalidProgramException

Description
With ImplementationOptions.UseIndirectCalls enabled, using a void method with boolean parameters you get an InvalidProgramException

To Reproduce
Steps to reproduce the behavior:

  1. Get a NativeLibraryBuilder field with ImplementationOptions.UseIndirectCalls
  2. Activate an interface containing a method returning void and a boolean parameter
  3. Call the method
  4. See error

Please provide a minimal compilable example of

Expected behavior
That the method gets run

Desktop:

  • OS: Linux
  • Runtime: Mono 4.6.2, .NET Core 2.1
  • Version 2.2

Additional context
See: https://github.com/morguldir/ADLTest

AnyCpu - how to pass path of both 32 and 64 bit dll

In the case of AnyCpu how to pass paths of both 32 bit as welll as 64 bit, when it only allows one path. It is currently invoking default 32-bit dll on 64 bit machine too. And on passing 64 bit dll path in AnyCpu it throws an exception:

{"Library loading failed."}

Use of default calling convention on core 2.0 causes GC issues.

Something I noticed occasionally when testing this on .net core 2.0 was I would occasionally get gc crashes while in native calls. I noticed the restriction on using this in 32 bit mode on core 2.0, and that got me thinking and looking through the source code. Apparently, using the default managed calling convention does work on 64 bit, however it doesn't so the housekeeping unmanaged Calli does, which can lead to stability issues, especially with blocking and long running native calls (which I have several). I would recommend this functionality be removed, or at least hidden behind a flag that even 64 bit core 2.0 is unstable.

https://github.com/dotnet/coreclr/issues/19997

NativeSymbolAttributes inside interfaces inherited by a mixed mode class don't work.

Describe the bug
NativeSymbol only works inside classes when using a mixed mode class, not inside interfaces

To Reproduce
Steps to reproduce the behavior:

  1. Create an interface implementing a function with a different name than the entrypoint.
  2. Add a NativeSymbol attribute containing the actual entrypoint.
  3. Create an abstract class inheriting from the interface and create an abstract equivalant function.
  4. Activate the class.

See https://github.com/Firwood-Software/AdvanceDLSupport/blob/master/docs/mixed_mode_classes.md

Expected behavior
The entrypoint is the same as the one in NativeSymbolAttribute inside interfaces.

Desktop (please complete the following information):

  • OS: Linux
  • Runtime: .NET Core
  • Version: 2.2.0

Implement solution for marshalling collection-like types

C# has multiple types and ways of expressing a linear collection of finite data (that is, a type that contains n instances of some type T). Passing this collection to native code typically requires the user to contain it in an array, pin it, and acquire a pointer to the first element of the array.

Traditional P/Invoke allows a user to pass an array directly, and performs these operations under the hood. ADL does the same.

However, in both cases, the user must supply the length of the array themselves in some other parameter to the function. This parameter is typically placed directly before or after the pointer parameter, and is a numeric type.

Describe the solution you'd like
ADL could simplify this by allowing collections to be passed on their own, without requiring an additional parameter, and instead generate this parameter automatically. This behaviour would be opt-in by way of a parameter attribute, which would also contain additional hinting information about the generation scheme. As an example,

interface ILibrary
{
    void DoSomethingWithArray
    (
        [CollectionMarshalling(typeof(UIntPtr), LengthParameterPosition.Before)]
        string[] myArray
    );
}

would be valid syntax, and generate a native signature matching

void DoSomethingWithArray(size_t collectionLength, char** myArray);

Conceivably, this could also extend to any type implementing ICollection<T> (which would firstly construct an array from the type, or use GetPinnableReference to pin it), as well as Span<T> and Memory<T>.

Describe alternatives you've considered
At present, the methods described in the preamble are the only available alternatives.

Additional context
OpenTK would benefit from this features, as it has a number of signatures following this pattern.

This issue supersedes #71, as it incorporates that feature request.

Span<T> support

It can be very useful to allow using Span as argument for native methods.
There is an article with some examples and information.

ld.so.cache parsing is incorrect

The library's current parsing of ld.so.config on Linux systems isn't very good and can produce incorrect results.

Proposed solutions:

  • Improve parser (figure out the actual format?)
  • Drop use of platform path resolvers and just try loading the library if we can't find anything in the local paths
  • Something else?

Stack Overflow Exception in StdCallMangler

The Garbage Collector messes in someway with ADL and causes the application to crash with a stack overrun exception. I cannot find the direct cause of this crash but I've attached a project with 100% crash reproduction rate.

To Reproduce
Using the attached source:

  1. Build the project.
  2. Run the executable.
  3. Press any key -- This will call GC.Collect()
  4. The program has crashed (use eventvwr.exe to see the crash infomations)

To test using PInvoke uncomment the #define PINVOKE at the top of NativeLibrary.cs

Environment
The project uses .NET Framework 4.7.3 and NuGet ADL version 2.3.1

System.Runtime.InteropServices.MarshalDirectiveException: Cannot marshal 'parameter #1': Non-blittable generic types cannot be marshaled

@Nihlus I have a C API that has a lot of void* pointers. It also uses callbacks in certain places. Pretty much all of the void* parameters are nullable. But the API also accepts arbitrary float* and other typed arrays, with a separate length parameter. However, I want to use Span<T> and Memory<T> instead of raw arrays if at all possible. However, whenever I try to activate the interface, I get this error:

  Failed TestDoubleRefcount [112 ms]
  Error Message:
   System.Runtime.InteropServices.MarshalDirectiveException : Cannot marshal 'parameter #1': Non-blittable generic types cannot be marshaled.
  Stack Trace:
     at AdvancedDLSupport.NativeLibraryBuilder.ActivateClass(String libraryPath, Type baseClassType, Type[] interfaceTypes)
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateClass[TClass,TInterface](String libraryPath)
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateInterface[TInterface](String libraryPath)
   at Synthizer.Tests.Utils.FFIActivator.ActivateFFIInterface() in /home/ethin/source/SynthizerSharp/SynthizerSharp.Tests/Utils.cs:line 35
   at SynthizerSharp.Tests.UnitTest1.TestDoubleRefcount() in /home/ethin/source/SynthizerSharp/SynthizerSharp.Tests/UnitTest1.cs:line 38

I'm confused because according to many resources that I've found, including the MS docs themselves, they imply that Span<T> and Memory<T> should be able to be passed to unmanaged code or across the FFI boundary. However, this doesn't appear to be the case. (In many places, I'm doing things like Memory<byte>? and Span<byte>? to indicate that these can be null.) Am I misusing these types? And, if so, how do I use them properly? Should I just revert to normal arrays?

Support for LPUTF8Str

Is your feature request related to a problem? Please describe.
I'm trying to do some interop with a Go library that exports some functions as C functions.

package main

import "C"

func main() {}

//export Verify
func Verify() *C.char {
    return C.CString("Test! ๐ŸŽ‰")
}

cgo outputs the following function definition for that:

extern char* Verify();

Describe the solution you'd like
This is what I would like to use:

internal interface IVerify : IDisposable
{
    [return: MarshalAs(UnmanagedType.LPUTF8Str)]
    public string Verify();
}

However, I'm getting an exception on the activation of this interface instead:

System.ArgumentOutOfRangeException
  HResult=0x80131502
  Message=The unmanaged type wasn't a recognized string type. (Parameter 'unmanagedType')
  Source=AdvancedDLSupport
  StackTrace:
   at AdvancedDLSupport.ImplementationGenerators.StringMarshallingWrapper.SelectUnmanagedToManagedTransformationMethod(UnmanagedType unmanagedType)
   at AdvancedDLSupport.ImplementationGenerators.StringMarshallingWrapper.EmitEpilogue(ILGenerator il, PipelineWorkUnit`1 workUnit)
   at AdvancedDLSupport.ImplementationGenerators.CallWrapperBase.<GenerateImplementation>d__2.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at AdvancedDLSupport.Pipeline.ImplementationPipeline.ConsumeDefinitions[T](IEnumerable`1 definitions, IReadOnlyList`1 pipeline)
   at AdvancedDLSupport.NativeLibraryBuilder.ConstructMethods(ImplementationPipeline pipeline, Type classType, Type[] interfaceTypes)
   at AdvancedDLSupport.NativeLibraryBuilder.GenerateInterfaceImplementationType(Type classType, Type[] interfaceTypes)
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateClass(String libraryPath, Type baseClassType, Type[] interfaceTypes)
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateClass[TClass,TInterface](String libraryPath)
   at AdvancedDLSupport.NativeLibraryBuilder.ActivateInterface[TInterface](String libraryPath)
   at TestApp.Program.Main() in C:\Projects\TestApp\TestApp\Program.cs:line 38

Describe alternatives you've considered
I'm currently using this instead:

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
internal readonly struct NativeUtf8String
{
    private readonly IntPtr String;

    public static explicit operator string(NativeUtf8String native) =>
        Marshal.PtrToStringUTF8(native.String);
}

Additional context
I believe StringMarshallingWrapper needs to be updated to support LPUTF8Str, however I'm wondering whether there are any runtime-specific details I'm not aware of.

Also: Is there a way to add support for LPUTF8Str without adjusting StringMarshallingWrapper, in case direct support is not of interest for this library?

Use of 'in' keyword in mapping interface throws

Describe the bug
The ref and out keywords seem to map just fine between the C# interface and the C API, but the in keyword triggers a System.TypeLoadException.

To Reproduce
Steps to reproduce the behavior:

  1. Create an interface that maps an in parameter to a const T* C parameter.
  2. Attempt to activate that interface.
  3. System.TypeLoadException is thrown.

Please provide a minimal compilable example of

[NativeSymbols(Prefix = "gl")]
public interface IOpenGl : IDisposable
{
    [NativeSymbol("DeleteBuffers")]
    void DeleteBuffersByRef(int n, ref uint buffers);

    [NativeSymbol("DeleteBuffers")]
    void DeleteBuffersByOut(int n, out uint buffers);

    // This one throws. If I comment it out, no throw.
    [NativeSymbol("DeleteBuffers")]
    void DeleteBuffersByIn(int n, in uint buffers);
}

class Program
{
    static void Main(string[] args)
    {
        using (var gl = NativeLibraryBuilder.Default.ActivateInterface<IOpenGl>("GL"))
        {

        }
    }
}

Expected behavior
It would be wonderful to be able to map in parameters to const T* parameters.

Desktop (please complete the following information):

  • OS: Arch Linux (Antergos)
  • Runtime: .NET Core
  • Version: 2.2

Additional context
N/A

No way to use inherited interfaces

The library doesn't support interfaces inheriting from interfaces, as it cannot infer which should be included in the method generation.

Calls to native code silently fail when using inherited interfaces.

It's hard to describe, and I can't reproduce it in a minimal example. Maybe if I describe it, something will light up.

To begin with, this only happens after upgrading to version 2.3.0.

What I'm seeing is that some methods are calling native code, while some methods are ignoring native code completely. But, it only happens when I have a parent interface that implements a bunch of other interfaces.

See the ICombined inteface here, and it's usage here.

If I manually build one of the child interfaces, the call to native works as expected.

// This works
builder.ActivateInterface<INetVariantListInterop>("QmlNet").Create()
// This doesn't work
((INetVariantListInterop)builder.ActivateInterface<ICombined>("QmlNet")).Create()

I'm using .NET Core 2.1.500 on Linux.

Humanizer dependency?

I know what the library is, but I'm not sure what it is used for.

Is it needed? It produces a lot of satellite assemblies.

Event Binding for Function Pointer

Allows C Function Pointer to bind with C# Event.

C program calls FunctionPointer, the FunctionPointer resolves to C# emitted method that invokes Event with parameters passed from C program.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.