Git Product home page Git Product logo

clr-loader's Introduction

pythonnet - Python.NET

Join the chat at https://gitter.im/pythonnet/pythonnet stackexchange shield

gh shield

license shield

pypi package version conda-forge version python supported shield

nuget preview shield nuget release shield

Python.NET is a package that gives Python programmers nearly seamless integration with the .NET Common Language Runtime (CLR) and provides a powerful application scripting tool for .NET developers. It allows Python code to interact with the CLR, and may also be used to embed Python into a .NET application.

Calling .NET code from Python

Python.NET allows CLR namespaces to be treated essentially as Python packages.

import clr
from System import String
from System.Collections import *

To load an assembly, use the AddReference function in the clr module:

import clr
clr.AddReference("System.Windows.Forms")
from System.Windows.Forms import Form

By default, Mono will be used on Linux and macOS, .NET Framework on Windows. For details on the loading of different runtimes, please refer to the documentation.

.NET Core

If .NET Core is installed in a default location or the dotnet CLI tool is on the PATH, loading it instead of the default (Mono/.NET Framework) runtime just requires setting either the environment variable PYTHONNET_RUNTIME=coreclr or calling pythonnet.load explicitly:

from pythonnet import load
load("coreclr")

import clr

Embedding Python in .NET

  • You must set Runtime.PythonDLL property or PYTHONNET_PYDLL environment variable starting with version 3.0, otherwise you will receive BadPythonDllException (internal, derived from MissingMethodException) upon calling Initialize. Typical values are python38.dll (Windows), libpython3.8.dylib (Mac), libpython3.8.so (most other Unix-like operating systems).
  • Then call PythonEngine.Initialize(). If you plan to use Python objects from multiple threads, also call PythonEngine.BeginAllowThreads().
  • All calls to python should be inside a using (Py.GIL()) {/* Your code here */} block.
  • Import python modules using dynamic mod = Py.Import("mod"), then you can call functions as normal, eg mod.func(args).
  • Use mod.func(args, Py.kw("keywordargname", keywordargvalue)) or mod.func(args, keywordargname: keywordargvalue) to apply keyword arguments.
  • All python objects should be declared as dynamic type.
  • Mathematical operations involving python and literal/managed types must have the python object first, eg. np.pi * 2 works, 2 * np.pi doesn't.

Example

static void Main(string[] args)
{
    PythonEngine.Initialize();
    using (Py.GIL())
    {
        dynamic np = Py.Import("numpy");
        Console.WriteLine(np.cos(np.pi * 2));

        dynamic sin = np.sin;
        Console.WriteLine(sin(5));

        double c = (double)(np.cos(5) + sin(5));
        Console.WriteLine(c);

        dynamic a = np.array(new List<float> { 1, 2, 3 });
        Console.WriteLine(a.dtype);

        dynamic b = np.array(new List<float> { 6, 5, 4 }, dtype: np.int32);
        Console.WriteLine(b.dtype);

        Console.WriteLine(a * b);
        Console.ReadKey();
    }
}

Output:

1.0
-0.958924274663
-0.6752620892
float64
int32
[  6.  10.  12.]

Resources

Information on installation, FAQ, troubleshooting, debugging, and projects using pythonnet can be found in the Wiki:

https://github.com/pythonnet/pythonnet/wiki

Mailing list

https://mail.python.org/mailman/listinfo/pythondotnet

Chat

https://gitter.im/pythonnet/pythonnet

.NET Foundation

This project is supported by the .NET Foundation.

clr-loader's People

Contributors

bneumann avatar br-sk avatar c-sellers avatar daredevildenis avatar filmor avatar koubaa avatar leedonggeon1996 avatar lostmsu avatar m-rossi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

clr-loader's Issues

Missing TargetFrameworkName from AppDomain.SetupInformation

When the .Net code is launched from pythonnet, the TargetFrameworkName property of the AppDomain.SetupInformation is not configured / provided.

Here is what AppDomain setup information returns from the normally executed .Net code:

AppDomain setup information:
	AppDomainManagerAssembly = NULL
	AppDomainManagerType = NULL
	ApplicationBase = D:\_WorkRoot\Clients\...\bin\Debug\
	ConfigurationFile = D:\_WorkRoot\Clients\...\bin\Debug\UPSTest.exe.Config
	TargetFrameworkName = .NETFramework,Version=v4.8
	DynamicBase = NULL
	DisallowPublisherPolicy = False
	DisallowBindingRedirects = False
	DisallowCodeDownload = False
	DisallowApplicationBaseProbing = False
	ApplicationName = UPSTest.exe
	PrivateBinPath = NULL
	PrivateBinPathProbe = NULL
	ShadowCopyDirectories = NULL
	ShadowCopyFiles = NULL
	CachePath = NULL
	LicenseFile = NULL
	LoaderOptimization = NotSpecified
	SandboxInterop = False

Here is how it is setup when called from PythonNet:

AppDomain setup information:
	AppDomainManagerAssembly = NULL
	AppDomainManagerType = NULL
	ApplicationBase = C:\Program Files\Python310\
	ConfigurationFile = C:\Program Files\Python310\python.exe.Config
	TargetFrameworkName = NULL
	DynamicBase = NULL
	DisallowPublisherPolicy = False
	DisallowBindingRedirects = False
	DisallowCodeDownload = False
	DisallowApplicationBaseProbing = False
	ApplicationName = python.exe
	PrivateBinPath = NULL
	PrivateBinPathProbe = NULL
	ShadowCopyDirectories = NULL
	ShadowCopyFiles = NULL
	CachePath = NULL
	LicenseFile = NULL
	LoaderOptimization = NotSpecified
	SandboxInterop = False

Why is it important? The below is the copy of the .Net 4.7.2 code that uses that moniker inside AppContext class (system\AppContext\AppContextDefaultValues.cs) (added in .Net 4.6.2).

    internal static partial class AppContextDefaultValues
    {
        public static void PopulateDefaultValues()
        {
            string platformIdentifier, profile;
            int version;

            ParseTargetFrameworkName(out platformIdentifier, out profile, out version);

            // Call into each library to populate their default switches
            PopulateDefaultValuesPartial(platformIdentifier, profile, version);
        }

        /// <summary>
        /// We have this separate method for getting the parsed elements out of the TargetFrameworkName so we can
        /// more easily support this on other platforms.
        /// </summary>
        private static void ParseTargetFrameworkName(out string identifier, out string profile, out int version)
        {
            string targetFrameworkMoniker = AppDomain.CurrentDomain.SetupInformation.TargetFrameworkName;

            // If we don't have a TFM then we should default to the 4.0 behavior where all quirks are turned on.
            if (!TryParseFrameworkName(targetFrameworkMoniker, out identifier, out version, out profile))
            {
#if FEATURE_CORECLR
                if (CompatibilitySwitches.UseLatestBehaviorWhenTFMNotSpecified)
                {
                    // If we want to use the latest behavior it is enough to set the value of the switch to string.Empty.
                    // When the get to the caller of this method (PopulateDefaultValuesPartial) we are going to use the 
                    // identifier we just set to decide which switches to turn on. By having an empty string as the 
                    // identifier we are simply saying -- don't turn on any switches, and we are going to get the latest
                    // behavior for all the switches
                    identifier = string.Empty;
                }
                else
#endif
                {
                    identifier = ".NETFramework";
                    version = 40000;
                    profile = string.Empty;
                }
            }
        }

The AppContext switches control behavior of number of .Net components (between versions). For example, one of such switches controls behavior of asymmetric encryption. It may mean that any code that calls .Net API impacted by any of those switches will behave differently inside Python than what is expected by the .Net code author.

It creates uncertainties for the .Net code that calls such .Net Framework API. For example, we are trying to detect which level of compatibility we are currently under to call our encryption code accordingly. That determination includes TargetFramework of the entry assembly or supportedRuntime element in the config file. And our code might not have a proper recovery strategy to go all the way back to .Net 4.0 compatibility (yet having installed all the latest .Net and compiled using latest .Net Framework).

Some switches are explictely controllable via configuration file but missing framework in the AppDomain setup just opens the mine field of discrepancies.

get_coreclr exception on Windows 32 bit

Using clr-loader 0.1.6 with pythonnet 3.0.0.dev1.

When running on 32 bit Windows, when my code calls:

config_path = get_coreclr(r'C:\MyApp.runtimeconfig.json')

It throws exception:

Traceback (most recent call last):
  File "MyApp.py", line 10, in <module>
    config_path = get_coreclr(r'C:\MyApp.runtimeconfig.json')
  File "C:\Program Files\Python38-32\lib\site-packages\clr_loader\__init__.py", line 38, in get_coreclr
    dotnet_root = find_dotnet_root()
  File "C:\Program Files\Python38-32\lib\site-packages\clr_loader\util\find.py", line 19, in find_dotnet_root
    dotnet_root = os.path.join(prog_files, "dotnet")
  File "C:\Program Files\Python38-32\lib\ntpath.py", line 78, in join
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType

This is because clr_loader/util/find.py tries to read environment variable "ProgramFiles(x86)" but it doesn't exist on 32 bit Windows:

def find_dotnet_root() -> str:
    dotnet_root = os.environ.get("DOTNET_ROOT", None)
    if dotnet_root is not None:
        return dotnet_root

    if sys.platform == "win32":
        # On Windows, the host library is stored separately from dotnet.exe for x86
        if sys.maxsize > 2 ** 32:
            prog_files = os.environ.get("ProgramFiles")
        else:
            prog_files = os.environ.get("ProgramFiles(x86)")    <--- doesn't exist on 32 bit Windows

        dotnet_root = os.path.join(prog_files, "dotnet")   <--- exception

The __version__ attribute is not defined

🐞 Problem
The __version__ attribute is not defined anywhere in the source code. The dynamic field is declared in the pyproject.toml but there is no version specified in the __init__.py top file.

💡 Idea
Declare this variable in the __init__.py file. Note the value of the __version__ needs to be updated every time a release is made.

Deadlock calling pythonnet.set_runtime twice

With the update to delay the load of the runtime from 0.25 to 0.26 a deadlock occurs with following code

import pythonnet
pythonnet.set_runtime("coreclr")
pythonnet.set_runtime("coreclr")

This is explicit a problem of the core clr. We did not see that with .NET FF.

We tracked it down to

res = dll.hostfxr_initialize_for_runtime_config(

this function deadlocks the second time called if

res = dll.hostfxr_get_runtime_delegate(

was not called before.

Mono 'The 'ExeConfigFilename' argument cannot be null.'

Hi all,
We've recently updated QuantConnect/pythonnet#47 to be aligned with https://github.com/pythonnet/pythonnet and started getting the following error

The 'ExeConfigFilename' argument cannot be null.
  at System.Configuration.ExeConfigurationHost.CheckFileMap (System.Configuration.ConfigurationUserLevel level, System.Configuration.ExeConfigurationFileMap map) [0x0001b] in <65bca3ec97b145069173491f1a974cbf>:0 
  at System.Configuration.ExeConfigurationHost.InitForConfiguration (System.String& locationSubPath, System.String& configPath, System.String& locationConfigPath, System.Configuration.Internal.IInternalConfigRoot root, System.Object[] hostInitConfigurationParams) [0x00030] in <65bca3ec97b145069173491f1a974cbf>:0 
  at System.Configuration.InternalConfigurationSystem.InitForConfiguration (System.String& locationConfigPath, System.String& parentConfigPath, System.String& parentLocationConfigPath) [0x00000] in <65bca3ec97b145069173491f1a974cbf>:0 
  at System.Configuration.Configuration..ctor (System.Configuration.InternalConfigurationSystem system, System.String locationSubPath) [0x0001f] in <65bca3ec97b145069173491f1a974cbf>:0 
  at System.Configuration.InternalConfigurationFactory.Create (System.Type typeConfigHost, System.Object[] hostInitConfigurationParams) [0x0000d] in <65bca3ec97b145069173491f1a974cbf>:0 
  at System.Configuration.ConfigurationManager.OpenExeConfigurationInternal (System.Configuration.ConfigurationUserLevel userLevel, System.Reflection.Assembly calling_assembly, System.String exePath) [0x000ef] in <65bca3ec97b145069173491f1a974cbf>:0 
  at System.Configuration.ClientConfigurationSystem.get_Configuration () [0x0000e] in <65bca3ec97b145069173491f1a974cbf>:0 

Steps to reproduce:

from pythonnet import load
from clr import AddReference
AddReference("RestSharp")
from RestSharp import *
client = RestClient("https://www.google.com/")
restsharpResponse = client.Execute(RestRequest())
print(restsharpResponse.ErrorException.InnerException)

I suspect it's related to this new clr-loader, seen some old related issues pythonnet/pythonnet@37a4ea5 & https://stackoverflow.com/questions/35872504/pythonnet-exception-the-execonfigfilename-argument-cannot-be-null .

  • Seems mono_domain_set_config isn't being called anymore? should it be called automatically internally (as before I believe) or we are missing some call for it in? Or is it maybe we are screwing it up in the way we consume it/missing some step

Thanks!

Could not find a suitable hostfxr library

Environment

  • Pythonnet version: 3.0.0.post1
  • Python version: 3.8
  • Operating System: macOS 12.6
  • .NET Runtime: .NETCore 6
  • Architecture: arm64 (Apple M1)

Details

import os
import sys
from clr_loader import get_coreclr
from pythonnet import set_runtime

runtime_config = os.path.join("/Users/gabloe/dotnetconfig", "pythonnettest.runtimeconfig.json")
rt = get_coreclr(runtime_config=runtime_config)
set_runtime(rt)

import clr

Traceback

Traceback (most recent call last):
...
    rt = get_coreclr(runtime_config=runtime_config)
  File "/Users/gabloe/Library/Caches/pypoetry/virtualenvs/test-lQu4lGUY-py3.8/lib/python3.8/site-packages/clr_loader/__init__.py", line 122, in get_coreclr
    impl = DotnetCoreRuntime(runtime_config=runtime_config, dotnet_root=dotnet_root)
  File "/Users/gabloe/Library/Caches/pypoetry/virtualenvs/test-lQu4lGUY-py3.8/lib/python3.8/site-packages/clr_loader/hostfxr.py", line 20, in __init__
    self._dll = load_hostfxr(self._dotnet_root)
  File "/Users/gabloe/Library/Caches/pypoetry/virtualenvs/test-lQu4lGUY-py3.8/lib/python3.8/site-packages/clr_loader/ffi/__init__.py", line 31, in load_hostfxr
    raise RuntimeError(f"Could not find a suitable hostfxr library in {dotnet_root}")
RuntimeError: Could not find a suitable hostfxr library in /usr/local/share/dotnet
Exception ignored in: <function Runtime.__del__ at 0x7f8609306820>
Traceback (most recent call last):
  File "/Users/gabloe/Library/Caches/pypoetry/virtualenvs/test-lQu4lGUY-py3.8/lib/python3.8/site-packages/clr_loader/types.py", line 137, in __del__
    self.shutdown()
  File "/Users/gabloe/Library/Caches/pypoetry/virtualenvs/test-lQu4lGUY-py3.8/lib/python3.8/site-packages/clr_loader/hostfxr.py", line 111, in shutdown
    if self._handle is not None:
AttributeError: 'DotnetCoreRuntime' object has no attribute '_handle'

Learning to use clr-loader

I am learning to use clr-loader because my environment does not allow non-pure python libraries.

If I were using pythonnet I would something like:

>>> import clr
>>> clr.AddReference('System.Data')
<System.Reflection.RuntimeAssembly object at 0x00000229C0EC3288>
>>> from System.Data import DataTable
>>> DataTable()
<System.Data.DataTable object at 0x00000229C0EC3288>

What would be the equivalent in clr-loader?

FYI, my goal is to call the Server constructor of this.

dotnet_root detection broke in ubuntu 22.04

basically dotnet_root detection is returning /etc/alternatives as root, because this is the folder where /usr/bin/dotnet is linking to
but the actual dotnet_root folder is /usr/lib/dotnet/dotnet6-6.0.108
so I believe the detection function should be updated to account for this.

Note: Setting DOTNET_ROOT environment variable to /usr/lib/dotnet/dotnet6-6.0.108 is a valid workaround, so this is not an urgent issue.

Tests fail to run if a newer runtime than 6.0 is installed

Since all tests currently run within one process, running the "explicit" test that uses a net60 runtime config and afterwards running the "automatic" one (picking net70) will fail as only a single runtime version can be loaded into a process. This is a general issue with the way tests are set up right now, maybe multiprocessing can help.

Cannot set parameters for .net core start. AppContext.BaseDirectory cannot be set.

in order to set AppContext.BaseDirectory correctly, one has to provide this during startup.
The answer seems to be the following:

pythonnet.load("coreclr", properties={"APP_CONTEXT_BASE_DIRECTORY":r"c:\dev\ppp\buildOutput\binaries\x64-debug\playground"})

However, this does not actually work. While the lib has the needed function call to hostfxr_set_runtime_property_value() available, it is not actually usuable. Consider this function in hostfxr.py:

class DotnetCoreRuntime(Runtime):
    def __init__(self, runtime_config: Path, dotnet_root: Path, **params: str):
        self._handle = None

        if _IS_SHUTDOWN:
            raise RuntimeError("Runtime can not be reinitialized")

        self._dotnet_root = Path(dotnet_root)
        self._dll = load_hostfxr(self._dotnet_root)
        self._is_initialized = False
        self._handle = _get_handle(self._dll, self._dotnet_root, runtime_config)
        self._load_func = _get_load_func(self._dll, self._handle)

        for key, value in params.items():
            self[key] = value

The only place where hostfxr_set_runtime_property_value() can be called is between _get_handle and _get_load_func. Try before and the dll is not ready, try afterwards and the runtime is already loaded. Also, the for loop at the end is misguided since it can only lead to an "runtime already loaded" error.
A second problem is that the "params" section of the init parameters cannot be used, as the call to this constructor is hard-coded without parameter forwarding.

Suggest action: Move the for loop to between _get_handle and _get_load_func, make the call to the constructor forward the parameters.

Add a `.info` attribute to runtimes and drop the wrappers where possible

The runtime wrapper object is a very thin layer on top of the actual runtime objects, we can probably do without it and instead use a base class.

An .info attribute (or function) would be very useful for debugging, it should contain all parameters that were used in initialisation and maybe a bit of internal information if available.

Example notebook code fails to run due to missing runtime config file

I'm having a go at getting this up and running so I can look into compatibility with .net core self-contained mode (I have a feeling it will 'just work' with a small tweak).

But when I try to run the example code from the example notebook, it seems to be missing example.runtimeconfig.json file:

T h e s p e c i f i e d r u n t i m e c o n f i g . j s o n [ e x a m p l e / o u t / e x a m p l e . r u n t i m e c o n f i g . j s o n ] d o e s n o t e x i s t

The code I try to run is:

from clr_loader import hostfxr
from clr_loader.ffi import ffi

core = hostfxr.HostFxr("example/out/example.runtimeconfig.json")

clr_loader.ffi.load_hostfxr might not load the latest version of hostfxr

Environment

  • Pythonnet version: 3.0.0.post1
  • Python version: 3.7.7
  • Operating System: Windows 10 21H2
  • .NET Runtime: 5.0.17

Details

Depending on the contents of C:\Program Files\dotnet\host\fxr (or wherever the dotnet root is), clr_loader might pick up an older version of hostfxr based on the logic in def load_hostfxr(dotnet_root: Path)

e.g. if the contents of the folder are

2.1.7
3.1.26
5.0.5
5.0.17

then it doesn't pick up 5.0.17, but picks up 5.0.5 because Python's lexicographic sort will order the folder as ['2.1.7', '3.1.26', '5.0.17', '5.0.5'].

This is admittedly very minor, but we just ran into this and wanted to give a heads up :)
Thank you!

Error loading clr into non-root domain

Environment

  • Pythonnet version: 3.0.1
  • Python version: 3.10
  • Operating System: Windows Server 2019
  • .NET Runtime: .Net Framework 4.8

Details

  • Describe what you were trying to get done.

In order to configure a specific .Net configuration file, I had to run set_runtime prior of clr load.

import os
from pathlib import Path
config_file = r"...valid path...\sctlib.py.config"
config_path = Path(config_file)
from pythonnet import set_runtime
set_runtime("netfx", domain="sctlib_py", config_file=config_path)
#set_runtime("netfx", domain="sctlib_py")
import clr

The config file is valid as it is borrowed verbatim (with exception of renaming) from the working console application.

"import clr" statement never succeeded:

Same script running with default domain works without an issue (but of course, does not recognize the config file which I need to have to target proper .Net Framework)

  • If there was a crash, please include the traceback here.
Traceback (most recent call last):
  File "D:\_WorkRoot\Tests\Python_ASR\Diarization\_Examples\Enrichment\sctlib.py", line 9, in <module>
    import clr
  File "C:\Program Files\Python310\lib\site-packages\clr.py", line 6, in <module>
    load()
  File "C:\Program Files\Python310\lib\site-packages\pythonnet\__init__.py", line 143, in load
    if func(b"") != 0:
  File "C:\Program Files\Python310\lib\site-packages\clr_loader\types.py", line 64, in __call__
    return self._callable(ffi.cast("void*", buf_arr), len(buf_arr))
RuntimeError: cannot call null pointer pointer from cdata 'int(*)(void *, int)'

" initializer for ctype 'char *' must be a bytes or list or tuple, not str" error

When I was trying to setup configuration file for the py script I am running to call my .Net assembly, I've got this error:

  Traceback (most recent call last):
   File "C:\Program Files\Python310\lib\site-packages\pythonnet\__init__.py", line 73, in _create_runtime_from_spec
     return clr_loader.get_netfx(**params)
   File "C:\Program Files\Python310\lib\site-packages\clr_loader\__init__.py", line 168, in get_netfx
     impl = NetFx(domain=domain, config_file=_maybe_path(config_file))
   File "C:\Program Files\Python310\lib\site-packages\clr_loader\netfx.py", line 27, in __init__
     self._domain = _FW.pyclr_create_appdomain(domain_s, config_file_s)
 TypeError: initializer for ctype 'char *' must be a bytes or list or tuple, not str

As it appears the pyclr_create_appdomain end point expects LPUTF8Str and not LPStr.

That raises number of issues on its own - such as what if my config file is located inside the Unicode named path - but immediate problem was in the code:

        initialize()
        if config_file is not None:
            config_file_s = str(config_file) #.encode()
        else:
            config_file_s = ffi.NULL

        self._domain_name = domain
        self._config_file = config_file
        if domain is not None:
            domain_s = domain #.encode()
        else:
            domain_s = ffi.NULL
        self._domain = _FW.pyclr_create_appdomain(domain_s, config_file_s)

As you could see I made a local fix adding .encode() call before passing parameters to the end point.

Questions:

  1. Am I doing something wrong? Here is my code below:
import os
from pathlib import Path
config_file = r"D:\_WorkRoot\Tests\Python_ASR\Diarization\_Examples\Enrichment\sctlib.py.config"
#os.environ["PYTHONNET_NETFX_CONFIG_FILE"] = config_file
config_path = Path(config_file)
from pythonnet import set_runtime
#set_runtime("netfx", domain="sctlib_py", config_file=config_path)
set_runtime("netfx", domain="sctlib_py")
  1. It took me awhile to recognize I don't need to construct Path inside my script and passing string from my end works too. But in your code, you take my string, convert to Path only to convert it back to string. Maybe to validate the path exists? Just a comment.

  2. I made a local change adding .encode() call to by-pass the error. What you want me to do? Abandon that - since you already addresses that, keep my local changes (and remember maintaining that on different machines) or to apply the fix to the repo (I have not done such contributions before, so I simply don't know the customary procedures).

Thanks.

RuntimeInfo.version '<undefined>'

Environment

  • Pythonnet version: 3.0.1
  • Python version: 3.7.1
  • Operating System: Windows 10
  • .NET Runtime: .NET Framework 4.8 is latest version installed.

Details

  • Describe what you were trying to get done.
    Determine the .NET Framework runtime version pythonnet is using.
import pythonnet
import clr
pythonnet.get_runtime_info()
# Prints the following: 
# RuntimeInfo(kind='.NET Framework', version='<undefined>', initialized=True, shutdown=False)

Why does this show '<undefined>' as the RuntimeInfo version? Is this a bug?

macOS: get_coreclr fails if run in an already open Terminal window after a fresh dotnet install. Works on Windows.

On macOS with pythonnet 3.0.0a2 installed (clr-loader 0.1.7) but without .NET core installed:

  • Open a terminal window
  • Download and install the correct .NET runtime (by double clicking the installer in Finder)
  • In the same terminal window, run a Python script that has:
    get_coreclr(C:\MyApp.runtimeconfig.json)

This results in the following exception:

  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/MyPackage/my_script.py", line 28, in <module>
    runtime_config_path = get_coreclr(r'C:\MyApp.runtimeconfig.json')
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/clr_loader/__init__.py", line 42, in get_coreclr
    dotnet_root = find_dotnet_root()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/clr_loader/util/find.py", line 22, in find_dotnet_root
    raise RuntimeError("Can not determine dotnet root")
RuntimeError: Can not determine dotnet root

If a NEW Terminal window is opened then this error does not occur. However we should not have to open a new Terminal window for it to find the dotnet executable. Performing the same steps on Windows works fine - we don't need to open a new command window.

My application has the prerequisites that pythonnet and .NET runtime are first installed. I am getting reports from people that they have installed the prerequisites but it is still failing. They don't realise that they need to open a new Terminal window to get it working.

The reason for the exception seems to be in clr-loader/util/find.py there is a difference how it finds the "dotnet" executable between Windows and macOS:

  • Windows just looks in C:\Program Files\dotnet
  • On macOS however, the line dotnet_path = shutil.which("dotnet") doesn't find anything because dotnet is not in the path for this Terminal window.
def find_dotnet_root() -> str:
    dotnet_root = os.environ.get("DOTNET_ROOT", None)
    if dotnet_root is not None:
        return dotnet_root

    if sys.platform == "win32":
        # On Windows, the host library is stored separately from dotnet.exe for x86
        prog_files = os.environ.get("ProgramFiles")
        dotnet_root = os.path.join(prog_files, "dotnet")
        if os.path.isdir(dotnet_root):
            return dotnet_root

    # Try to discover dotnet from PATH otherwise
    dotnet_path = shutil.which("dotnet")
    if not dotnet_path:
        raise RuntimeError("Can not determine dotnet root")

To fix this, could macOS just look in /usr/local/share/dotnet ?
(I'm don't have Linux so I'm not sure where dotnet gets installed to on Linux)

Suggestion: ability to add binary folders to app domain

Instead of relying on the overburdening the PATH environment variable, if the non-root domain is requested (which is important to manage base directory and config file etc.), the list of locations to search for DLLs could be provided as well.

Using a relocated Mono

Environment

  • Pythonnet version: 2.5.2
  • Python version: 3.9.2
  • Operating System: CentOS 7
  • .NET Runtime: 4.5

Details

  • I relocated the mono I built from source (in preparation for embedding mono in a product) and got this
The assembly mscorlib.dll was not found or could not be loaded.
It should have been installed in the `/opt/mono-6.8.0.123/lib/mono/4.5/mscorlib.dll' directory.

My understanding is mono can be relocated, but it requires invoking their C API to set the assembly paths, etc...
It would be nice if clr-loader could enable this by calling the right functions from libmono-2.so

I found this problem after relocating mono by running

echo "import clr" | python3

Build issue when using dotnet-sdk v6

I just tested creating a recipe for conda-forge and seem to get some issues with dotnet-sdk v6:

"System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Users\USERNAME\miniforge3\conda-bld\clr_loader_1637417796354\_build_env\dotnet\sdk-manifests'

dotnet-sdk v5 seems to work fine. I tried to find out which dotnet-sdk is used in your github-actions, but failed. Which version you use?

If you are using v6 this may be related to the build process in the conda-forge-recipe: https://github.com/conda-forge/dotnet-feedstock

Cannot find dotnet root on x64 darwin

The latest version of clr-loader finds the dotnet root with literal path string. It causes the problem on x64 darwin system, which dotnet root is /usr/local/share/dotnet/x64

Before on 0.1.7, it finds with which(dotnet) so there was no problem.
Could you have a look?

elif sys.platform == "darwin":
dotnet_root = Path("/usr/local/share/dotnet")

---Edited

With noted on Python documentation, I'd like to suggest to check whether the system is 64-bits or not.
https://docs.python.org/3/library/platform.html#cross-platform

    if:
        # Something
        pass
    elif sys.platform == "darwin":
        if sys.maxsize > 2**32: # is_64bits
            dotnet_root = Path("/usr/local/share/dotnet/x64")
        else:
            dotnet_root = Path("/usr/local/share/dotnet")

How to import namespace of assembly built by .net 5.0.4

Excuse me, is it possible to import a namespace from an assembly built by .net 5.0.4?
I could execute these lines

import os
import sys
from clr_loader import get_coreclr
from pythonnet import set_runtime

runtime_config = os.path.join(r"Z:\SharFTrade\pythonnettest\pythonnettest\bin\Release\net5.0", "pythonnettest.runtimeconfig.json")
rt = get_coreclr(runtime_config)
set_runtime(rt)

import clr

without any errors, however I am unable to import namespace defined in the dll...

RuntimeError: cannot call null pointer pointer from cdata 'int(*)(void *, int)'

[2023-01-11 18:21:28] Traceback (most recent call last):
  File "main.py", line 69, in <module>
  File "webview\__init__.py", line 122, in start
  File "webview\guilib.py", line 101, in initialize
  File "webview\guilib.py", line 62, in try_import
  File "webview\guilib.py", line 52, in import_winforms
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "webview\platforms\winforms.py", line 31, in <module>
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "clr.py", line 6, in <module>
  File "pythonnet\__init__.py", line 143, in load
  File "clr_loader\types.py", line 64, in __call__
RuntimeError: cannot call null pointer pointer from cdata 'int(*)(void *, int)'

Suggestion: manage base directory when creating non-root domain

The code that launches a non-root domain expects to use the base directory of the original root domain - which is essentially the location of python.exe.

The suggestion is to allow specifying the following:

  • inherit from root domain
  • current directory
  • directory of the script importing clr
  • specific folder

clr-loader failed to load dll if the assembly path contain Chinese

Environment

  • Pythonnet version: 3.0.1
  • Python version: 3.8.10
  • Operating System: Windows 11
  • .NET Runtime:

Details

I find that everything just works fine if Python path doesn't contains any Chinese characters. But if the path contain some Chinese chars, let's said, start a python interactive shell via the following command:

D:\中文测试\CCInsight\python-3.8.10-embed-amd64\python.exe

And then when I run import clr, the following error will be raised.

>>> import clr
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\中文测试\CCInsight\python-3.8.10-embed-amd64\lib\site-packages\clr.py", line 6, in <module>
    load()
  File "D:\中文测试\CCInsight\python-3.8.10-embed-amd64\lib\site-packages\pythonnet\__init__.py", line 143, in load
    if func(b"") != 0:
  File "D:\中文测试\CCInsight\python-3.8.10-embed-amd64\lib\site-packages\clr_loader\types.py", line 64, in __call__
    return self._callable(ffi.cast("void*", buf_arr), len(buf_arr))
RuntimeError: cannot call null pointer pointer from cdata 'int(*)(void *, int)'

I delivery a software that provide a GUI depends on pythonnet, this become an annoy bug as user may choose their own installation location, which may contain Chinese chars. Hope to find a fix or a work around.

Update:

I try to narrow down the scope of code to

https://github.com/pythonnet/clr-loader/blob/master/clr_loader/netfx.py#L36-L44

It looks to me like the _get_callable failed to get the right callabe object by the following args

assembly_path:  d:\中文测试\CCInsight\python-3.8.10-embed-amd64\lib\site-packages\pythonnet\runtime\Python.Runtime.dll
typename:          Python.Runtime.Loader
funcname:          Initialize

COR_E_INVALIDOPERATION when running release branch

Environment

  • Pythonnet version: release branch
  • Python version: 3.7.14
  • Operating System: Debian Bullseye (running under Windows Subsystem for Linux 2, Ubuntu 20.04, on Windows 10 21H2)
  • .NET Runtime: .NET Core 3.1.29

Details

I am trying to load the CoreCLR with the following statements:

from pythonnet import load  
load("coreclr", runtime_config="pythonnet.json")     

where pythonnet.json contains the following:

{
  "runtimeOptions": {
   "tfm": "netcoreapp3.1",
   "framework": {
    "name": "Microsoft.NETCore.App",
    "version": "3.1.29"
    }
   }
}

The error message I get is as follows (note - the same happens if I try using .NET 6.0 instead of .NET Core 3.1):

Python 3.7.14 (default, Sep 13 2022, 02:30:55)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pythonnet import load
>>> load("coreclr", runtime_config="pythonnet.json")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/pythonnet/pythonnet/__init__.py", line 137, in load
    func = assembly.get_function("Python.Runtime.Loader.Initialize")
  File "/usr/local/lib/python3.7/site-packages/clr_loader/types.py", line 94, in get_function
    return ClrFunction(self._runtime, self._path, name, func)
  File "/usr/local/lib/python3.7/site-packages/clr_loader/types.py", line 58, in __init__
    self._callable = runtime._get_callable(assembly, typename, func_name)
  File "/usr/local/lib/python3.7/site-packages/clr_loader/hostfxr.py", line 101, in _get_callable
    check_result(res)
  File "/usr/local/lib/python3.7/site-packages/clr_loader/util/__init__.py", line 42, in check_result
    raise error
clr_loader.util.clr_error.ClrError: 0x80131509: COR_E_INVALIDOPERATION => An operation is not legal in the current state.

Attached the docker file I am using to build PythonNet and subsequently run the above statements in it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.