juliapy / pythoncall.jl Goto Github PK
View Code? Open in Web Editor NEWPython and Julia in harmony.
Home Page: https://juliapy.github.io/PythonCall.jl/stable/
License: MIT License
Python and Julia in harmony.
Home Page: https://juliapy.github.io/PythonCall.jl/stable/
License: MIT License
e.g. we could have a macro which transforms ordinary Julia code to track which variables are Python objects and decrefs them as appropriate. These Python objects could just be pointers, as if we were writing CPython directly. We'd need to ensure that any Julia code mixed in could not raise uncaught exceptions.
Originally PyRef
was intended to be the single place to safely hold a reference to a Python object, so that other wrapper types (PyObject
, PyList
, etc.) could each simply have a PyRef
field.
It would actually be simpler to just have these types hold a PyPtr
and have a generic finalizer that decrefs whatever is in the ptr
field (i.e. re-use pyref_finalize!
).
Export pynull, pycopy and pyisnull. Rename to unsafe_pycopy. Document these in a section about writing packages.
This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml
to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix
on this issue.
I'll open a PR within a few hours, please be patient!
This works:
(@v1.8) pkg> add PythonCall#master # with or without master, while I get an error for 1.7, likely bad installation of Julia.
[Do you have any idea what might be happening? And since works in 1.6, an issue for Julia not your package?]
But doesn't work in 1.8 nor beta2 (and likely not in beta3 either):
julia> @time using PythonCall
ERROR: InitError: type DataType has no field mutable
Stacktrace:
[1] getproperty
@ ./Base.jl:37 [inlined]
[2] pointer_from_obj(o::Vector{String})
@ PythonCall ~/.julia/packages/PythonCall/r2eLx/src/utils.jl:236
[3] PyJuliaValue_SetValue(__o::Ptr{PythonCall.CPython.PyObject}, v::Vector{String})
@ PythonCall.CPython ~/.julia/packages/PythonCall/r2eLx/src/cpython/juliabase.jl:70
[4] PyJuliaValue_New(t::Ptr{PythonCall.CPython.PyObject}, v::Vector{String})
@ PythonCall.CPython ~/.julia/packages/PythonCall/r2eLx/src/cpython/juliabase.jl:89
[5] PyJuliaVectorValue_New
@ ~/.julia/packages/PythonCall/r2eLx/src/cpython/juliavector.jl:244 [inlined]
[6] PyJuliaValue_From
@ ~/.julia/packages/PythonCall/r2eLx/src/cpython/juliavector.jl:245 [inlined]
[7] PyObject_From
@ ~/.julia/packages/PythonCall/r2eLx/src/cpython/object.jl:82 [inlined]
[8] exec_impl(f::PythonCall.var"#245#263", code::PythonCall.PyCode, globals::PythonCall.PyDict{String, PythonCall.PyObject}, locals::Nothing, extrakeys::Tuple{PythonCall.PyInternedString, PythonCall.PyInternedString}, extravals::Tuple{Vector{String}, Bool})
@ PythonCall ~/.julia/packages/PythonCall/r2eLx/src/eval.jl:38
[9] macro expansion
@ ~/.julia/packages/PythonCall/r2eLx/src
┌ Error: Pkg.precompile error
│ exception =
│ SystemError: mktemp: Permission denied
│ Stacktrace:
│ [1] systemerror(p::Symbol, errno::Int32; extrainfo::Nothing)
│ @ Base ./error.jl:174
│ [2] #systemerror#68
│ @ ./error.jl:173 [inlined]
│ [3] systemerror
│ @ ./error.jl:173 [inlined]
│ [4] mktemp(parent::String; cleanup::Bool)
[ Info: Precompiling PythonCall [6099a3de-0909-46bc-b1f4-468b9a2dfc0d]
ERROR: InitError: type DataType has no field mutable
Stacktrace:
[1] getproperty
@ ./Base.jl:37 [inlined]
[2] pointer_from_obj(o::Vector{String})
@ PythonCall ~/.julia/packages/PythonCall/r2eLx/src/utils.jl:236
[3] PyJuliaValue_SetValue(__o::Ptr{PythonCall.CPython.PyObject}, v::Vector{String})
@ PythonCall.CPython ~/.julia/packages/PythonCall/r2eLx/src/cpython/juliabase.jl:70
[4] PyJuliaValue_New(t::Ptr{PythonCall.CPython.PyObject}, v::Vector{String})
@ PythonCall.CPython ~/.julia/packages/PythonCall/r2eLx/src/cpython/juliabase.jl:89
[5] PyJuliaVectorValue_New
@ ~/.julia/packages/PythonCall/r2eLx/src/cpython/juliavector.jl:244 [inlined]
[6] PyJuliaValue_From
@ ~/.julia/packages/PythonCall/r2eLx/src/cpython/juliavector.jl:245 [inlined]
[7] PyObject_From
@ ~/.julia/packages/PythonCall/r2eLx/src/cpython/object.jl:82 [inlined]
[8] exec_impl(f::PythonCall.var"#245#263", code::PythonCall.PyCode, globals::PythonCall.PyDict{String, PythonCall.PyObject}, locals::Nothing, extrakeys::Tuple{PythonCall.PyInternedString, PythonCall.PyInternedString}, extravals::Tuple{Vector{String}, Bool})
@ PythonCall ~/.julia/packages/PythonCall/r2eLx/src/eval.jl:38
[9] macro expansion
@ ~/.julia/packages/PythonCall/r2eLx/src/eval.jl:156 [inlined]
[10] (::PythonCall.var"#243#261")()
@ PythonCall ~/.julia/packages/PythonCall/r2eLx/src/init.jl:203
[11] with_gil(f::PythonCall.var"#243#261", c::Bool)
@ PythonCall ~/.julia/packages/PythonCall/r2eLx/src/gil.jl:10
[12] with_gil(f::Function)
@ PythonCall ~/.julia/packages/PythonCall/r2eLx/src/gil.jl:9
[13] init()
@ PythonCall ~/.julia/packages/PythonCall/r2eLx/src/init.jl:195
[14] (::PythonCall.var"#275#276")()
@ PythonCall ~/.julia/packages/Requires/035xH/src/init.jl:11
[15] __init__()
@ PythonCall ~/.julia/packages/Requires/035xH/src/init.jl:18
[16] _include_from_serialized(path::String, depmods::Vector{Any})
@ Base ./loading.jl:768
[17] _require_from_serialized(path::String)
@ Base ./loading.jl:821
[18] _require(pkg::Base.PkgId)
@ Base ./loading.jl:1130
[19] require(uuidkey::Base.PkgId)
@ Base ./loading.jl:1013
[20] require(into::Module, mod::Symbol)
@ Base ./loading.jl:997
[21] top-level scope
@ timing.jl:210
during initialization of module PythonCall
... or nested UnionAll of DataType.
Split any Unions into their parts and pass each part separately into the rule.
Any TypeOfBottom can be ignored.
Hi there,
Just wondering - is there a way to reload the code from a Python script without restarting the current Julia session?
Thanks in advance! :)
It is basically unused, increases memory usage and (for some reason I don't understand) slows down pyptr
even on objects that are not lazy.
The only instances of it are _none
, _true
and _false
.
Rename the PyCall page to include both the comparison and some migration tips.
Bit of a strange request, but there is a use case. A minimal example is:
import pickle
from juliacall import Base
pickle.dumps(Base)
# Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# TypeError: cannot pickle 'juliacall.ModuleValue' object
This would be useful for defining functions that are passed around in Python. I initially ran into this when using ray, so a slightly more complex example that has the same behaviour:
import ray
ray.init()
from juliacall import Base
@ray.remote
def v(x):
Base.println(x)
x = ray.get(v.remote(1))
I had a look at PyCall as well, but they've got a similar issue here JuliaPy/PyCall.jl#863, with a few comments on how to fix it.
Any ideas? I'll try to have a look into this further, but any help getting this working would be appreciated.
e.g. _type_ub(Vector{Vector{T}} where {T<:Int}) == Vector{Vector{Int}}, when the intention is that it is Vector{Vector{T} where {T<:Int}}
pynew, pydel!, pycopy!, pyisnull, unsafe_pynext
I first noticed this because I tried to import juliacall
from a plain Python script, running in a conda environment. I then got this error:
> python -c "import juliacall"
fatal: error thrown and no exception handler available.
InitError(mod=:Sys, error=ErrorException("could not load library "libpcre2-8"
The specified module could not be found. "))
jl_errorf at /cygdrive/c/buildbot/worker/package_win64/build/src\rtutils.c:77
jl_load_dynamic_library at /cygdrive/c/buildbot/worker/package_win64/build/src\dlload.c:277
jl_get_library_ at /cygdrive/c/buildbot/worker/package_win64/build/src\runtime_ccall.cpp:52
jl_get_library_ at /cygdrive/c/buildbot/worker/package_win64/build/src\runtime_ccall.cpp:41 [inlined]
jl_load_and_lookup at /cygdrive/c/buildbot/worker/package_win64/build/src\runtime_ccall.cpp:63
jlplt_pcre2_compile_8_32718.clone_1 at c:\users\nardi\appdata\local\programs\julia-1.6.1\lib\julia\sys.dll (unknown line)
compile at .\pcre.jl:124
compile at .\regex.jl:79
match at .\regex.jl:297
match at .\regex.jl:297 [inlined]
match at .\regex.jl:316 [inlined]
splitdrive at .\path.jl:38
joinpath at .\path.jl:256
abspath at .\path.jl:410 [inlined]
__init_build at .\sysinfo.jl:125
__init__ at .\sysinfo.jl:117
jfptr___init___38133.clone_1 at c:\users\nardi\appdata\local\programs\julia-1.6.1\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
jl_module_run_initializer at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:72
_julia_init at /cygdrive/c/buildbot/worker/package_win64/build/src\init.c:794
jl_init_with_image__threading at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:74 [inlined]
jl_init_with_image__threading at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:63 [inlined]
jl_init__threading at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:90
DllCanUnloadNow at C:\tools\miniconda3\envs\hgo_3.7\DLLs\_ctypes.pyd (unknown line)
DllCanUnloadNow at C:\tools\miniconda3\envs\hgo_3.7\DLLs\_ctypes.pyd (unknown line)
DllCanUnloadNow at C:\tools\miniconda3\envs\hgo_3.7\DLLs\_ctypes.pyd (unknown line)
DllCanUnloadNow at C:\tools\miniconda3\envs\hgo_3.7\DLLs\_ctypes.pyd (unknown line)
unknown function (ip: 00007ffaa94e6de9)
PyObject_FastCallKeywords at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyMethodDef_RawFastCallKeywords at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyMethodDef_RawFastCallKeywords at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalCodeWithName at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyImport_Import at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyImport_Import at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyMethodDef_RawFastCallDict at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PySlice_New at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalCodeWithName at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyMethodDef_RawFastCallKeywords at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyMethodDef_RawFastCallKeywords at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyMethodDef_RawFastCallKeywords at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyMethodDef_RawFastCallKeywords at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyFunction_FastCallDict at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyObject_CallMethodIdObjArgs at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyObject_CallMethodIdObjArgs at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyMember_SetOne at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyImport_ImportModuleLevelObject at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyObject_LengthHint at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalFrameDefault at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalCodeWithName at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalCodeEx at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyEval_EvalCode at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyArena_Free at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyRun_StringFlags at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyRun_SimpleStringFlags at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
Py_Main at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
Py_Main at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
PyBytes_AsString at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
Py_Main at C:\tools\miniconda3\envs\hgo_3.7\python37.dll (unknown line)
unknown function (ip: 00007ff6b69e1267)
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
It looks like it can't load libpcre, which seems weird, because the conda environment should only add to the paths/environment variables, right?
On the other hand, using PythonCall from Julia was working fine, and it was using the Python executable from the environment as well. However, when I switch it to the conda environment explicitly using ENV["JULIA_PYTHONCALL_EXE"] = "CONDA:..."
it also crashes:
julia> @pyv `1+1`
ERROR: UndefRefError: access to undefined reference
Stacktrace:
[1] Py_CompileString(code::String, filename::String, mode::Int64)
@ PythonCall.CPython C:\Users\Nardi\Programming\PythonCall.jl\src\cpython\fundamentals.jl:64
[2] pyptr
@ C:\Users\Nardi\Programming\PythonCall.jl\src\PyCode.jl:27 [inlined]
[3] eval_impl(::Type{PyObject}, code::PyCode, globals::PyDict{String, PyObject}, locals::Nothing, extrakeys::Tuple{}, extravals::Tuple{})
@ PythonCall C:\Users\Nardi\Programming\PythonCall.jl\src\eval.jl:3
[4] top-level scope
@ REPL[14]:1
I am running on Windows 10, and the conda environment has Python 3.7 installed. However, running julia
when the environment is activated works fine, and regexes (so libpcre) also work. The only thing I can think of is that there is a mismatch between the libpcre version inside and outside the conda environment. But I'm not still not sure why that only shows up when setting up the Julia runtime from "inside" Python.
Python traceback
objects are not picklable, and hence PyException
cannot always be serialised.
As a fix, serialise the b
field of a PyException
as None
. Or possibly always serialize traceback
to None
? Or to Unpicklable()
?
Optional extra: if we detect that tblib
is installed, allow pickling of traceback
.
Document this behaviour in the compat section.
Let's consider the code fragment
import numpy as np
X = np.ones((10,10))
Y = np.ones((10,10))
x, y = jl.MyModule.compute_something(X, Y, -1.0, 1.0, -1.0, 1.0)
where I call a Julia function defined in the MyModule
module. I am getting the error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Julia: MethodError: no method matching compute_something(::PythonCall.PyArray{Float64, 2, Float64, true, false}, ::PythonCall.PyArray{Float64, 2, Float64, true, false}, ::Float64, ::Float64, ::Float64, ::Float64)
Closest candidates are:
compute_something(!Matched::Matrix{T}, !Matched::Matrix{T}, ::T, ::T, ::T, ::T) where T<:Real at /opt/julia/compute_something:362
Does this mean that I need to define a new method for compute_something()
with ::PythonCall.PyArray
arguments? Can I avoid doing that? From the documentation it is not clear what to do in this situation.
Serialisation does not support NULL Py objects. Support it by serialising the value to nothing
instead of pickling the object (and check for nothing
when deserialising).
The current behaviour lazily loads C API function/object pointers, requiring a check each time one is accessed.
Instead, load them all at init.
Hi, I CAN install your package on a clean install (not otherwise, see below), but as you know startup is rather slow.
A.
I added precompile statements and got down to (still higher than what I would like to see):
$ ~/julia-1.6.0-rc2/bin/julia -O0 --compile=min --startup-file=no -q
julia> @time using PythonCall
2.384556 seconds (2.29 M allocations: 143.823 MiB, 1.21% gc time)
Still rather bad:
$ ~/julia-1.6.0-rc2/bin/julia -O0 --startup-file=no
julia> @time using PythonCall
17.399345 seconds (44.16 M allocations: 2.541 GiB, 6.10% gc time, 0.02% compilation time)
and even worse:
$ ~/julia-1.6.0-rc2/bin/julia --startup-file=no -q
julia> @time using PythonCall
26.929030 seconds (44.16 M allocations: 2.541 GiB, 3.96% gc time, 0.02% compilation time)
compared to your package as is:
julia> @time using PythonCall
31.716683 seconds (55.21 M allocations: 3.177 GiB, 5.03% gc time, 0.02% compilation time)
$ ~/julia-1.6.0-rc2/bin/julia -O0 --compile=min --startup-file=no -q
julia> @time using PythonCall
6.888381 seconds (19.04 M allocations: 1.109 GiB, 9.42% gc time)
I used a naive way, not SnoopCompile.jl, so maybe it's better. I'm also thinking, do you need to do this much at init? Can you postpone some things, and do later lazily?
B.
The above worked with the new Julia 1.6.0-rc2, but with my original .julia folder I get (on that version and later, including 1.53):
(@v1.6) pkg> add PythonCall.jl
Updating registry at `~/.julia/registries/General`
ERROR: refusing to add package `PythonCall [6099a3de]`: package `PythonCall = "6099a3de-0909-46bc-b1f4-468b9a2dfc0d"` already exists as a direct dependency
Do you have any idea what it means (this is before my work with precompile statements)?
At least I thought you would like to know, as your users might encounter this, and I have no idea where to begin to debug this. Except, while it doesn't seem to strictly be about your package (or is it?), only an interaction with my (broken?) setup:
(@v1.6) pkg> st
Status `~/.julia/environments/v1.6/Project.toml`
[7d9fca2a] Arpack v0.5.1
[1dc0ca97] ArrayTools v0.1.0 `https://github.com/emmt/ArrayTools.jl#master`
[77e5a97a] AsyPlots v0.2.1
[c52e3926] Atom v0.12.30
[b6338580] BPFnative v0.1.0 `https://github.com/jpsamaroo/BPFnative.jl.git#master`
[6e4b80f9] BenchmarkTools v0.5.0
[7cffe744] BetterExp v0.1.0 `https://github.com/oscardssmith/BetterExp.jl#master`
[7f725544] BinaryBuilderBase v0.5.0 `https://github.com/JuliaPackaging/BinaryBuilderBase.jl.git#master`
[ad839575] Blink v0.12.4
[336ed68f] CSV v0.8.3
[159f3aea] Cairo v1.0.5
[49dc2e85] Calculus v0.5.1
[710b7bb7] ClearStacktrace v0.2.2
[aaaa29a8] Clustering v0.14.2
[da1fd8a2] CodeTracking v1.0.5
[944b1d66] CodecZlib v0.7.0
[35d6a980] ColorSchemes v3.10.2
[5ae59095] Colors v0.12.6
[8f4d0f93] Conda v1.5.0
[1b08a953] Dash v0.1.3
[03207cf0] DashBase v0.1.0 `https://github.com/plotly/DashBase.jl.git#master`
[1b08a953] DashCoreComponents v1.12.0
[1b08a953] DashHtmlComponents v1.1.1
[1b08a953] DashTable v4.10.1
[9024f26f] Dashboards v0.2.8
[a93c6f00] DataFrames v0.22.5 `https://github.com/JuliaData/DataFrames.jl.git#main`
[1313f7d8] DataFramesMeta v0.6.0
[864edb3b] DataStructures v0.17.20
[55939f99] DecFP v1.1.0
[85a47980] Dictionaries v0.3.7
[a1bb12fb] Electron v3.1.1
[d872a56f] ElectronDisplay v1.0.1-DEV `https://github.com/queryverse/ElectronDisplay.jl.git#master`
[c04bee98] ExcelReaders v0.11.0
[53c48c17] FixedPointNumbers v0.8.4
[186bb1d3] Fontconfig v0.4.0
[5752ebe1] GMT v0.29.0
[28b8d3ca] GR v0.55.0
[c91e804a] Gadfly v1.3.1
[4b11ee91] Gaston v1.0.2 `https://github.com/mbaz/Gaston.jl.git#master`
[c43c736e] Genie v1.15.1
[bc5e4493] GitHub v5.4.0
[dc211083] Gnuplot v1.3.0
[4c0ca9eb] Gtk v1.1.6
[cd3eb016] HTTP v0.8.19
[60101457] ICOADSDict v0.1.0
[7073ff75] IJulia v1.23.2
[e8efc688] ImPlot v0.1.1
[5903a43b] Infiltrator v0.3.0
[1c8ee90f] IterableTables v1.0.0
[82899510] IteratorInterfaceExtensions v1.0.0
[4138dd39] JLD v0.12.1
[682c06a0] JSON v0.21.1
[2535ab7d] JSON2 v0.3.2
[0f8b85d8] JSON3 v1.7.0
[aa1ae85d] JuliaInterpreter v0.8.9
[75827d1f] JuliaTutor v0.1.0 `https://github.com/caseykneale/JuliaTutor.jl#master`
[e5e0dc1b] Juno v0.8.4
[fc18253b] LazyJSON v0.2.2
[194296ae] LibPQ v1.6.2
[23992714] MAT v0.10.1
[10e44e05] MATLAB v0.8.0
[1914dd2f] MacroTools v0.5.6
[739be429] MbedTLS v1.0.3
[c03570c3] Memoize v0.4.4
[13650a8a] NicePipes v0.1.3
[47be7bcc] ORCA v0.5.0
[5fb14364] OhMyREPL v0.5.8 `~/.julia/dev/OhMyREPL`
[3ad7be9e] OpenCV v0.1.1 `https://github.com/TakekazuKATO/OpenCV.jl#master`
[bac558e1] OrderedCollections v1.4.0
[ace2c81b] PETSc v0.1.0 `https://github.com/JuliaParallel/PETSc.jl#master`
[06a5ddd6] PGPlot v0.1.0 `https://github.com/emmt/PGPlot.jl#master`
[9b87118b] PackageCompiler v1.2.5
[eadc2687] Pandas v1.4.0 `https://github.com/PallHaraldsson/Pandas.jl#master`
[f5117550] PandasLite v0.1.7
[69de0a69] Parsers v1.0.15
[32113eaa] PkgBenchmark v0.2.10
[f3e62ec7] PkgCleanup v0.1.0 `https://github.com/giordano/PkgCleanup.jl#main`
[58dd65bb] Plotly v0.3.0
[a03496cd] PlotlyBase v0.3.1
[f0f68f2c] PlotlyJS v0.13.1
[91a5bcdd] Plots v1.10.6
[8e5c59b7] PlusPlus v0.1.0
[c3e4b0f8] Pluto v0.12.21
[c46f51b8] ProfileView v0.6.9
[438e738f] PyCall v1.92.2 `https://github.com/JuliaPy/PyCall.jl.git#master`
[d330b81b] PyPlot v2.9.0 `~/.julia/dev/PyPlot`
[6099a3de] Python v0.1.0 `https://github.com/cjdoris/Python.jl#master`
[ce6b1742] RDatasets v0.7.4
[295af30f] Revise v3.1.12
[22415677] RobotOS v0.7.2
[5e6a0ad9] SafeREPL v0.0.1 `https://github.com/rfourquet/SafeREPL.jl#master`
[88634af6] SaferIntegers v2.5.1
[1277b4bf] ShiftedArrays v1.0.0
[01919df6] SimpleTypePrint v0.1.2
[aa65fe97] SnoopCompile v2.5.2
[90137ffa] StaticArrays v0.12.5
[4acbeb90] Stipple v0.6.0
[30ddb3f0] StippleCharts v0.4.1
[ec984513] StipplePlotly v0.1.0 `https://github.com/GenieFramework/StipplePlotly.jl#main`
[a3c5d34a] StippleUI v0.2.4 `https://github.com/GenieFramework/StippleUI.jl.git#master`
[88034a9c] StringDistances v0.8.0
[69024149] StringEncodings v0.3.3
[b9c42661] SwapLiterals v0.1.0 `https://github.com/rfourquet/SafeREPL.jl:SwapLiterals#master`
[382cd787] TableTraitsUtils v1.0.1
[05f542c5] TestGtkMegaJLL v0.1.0 `~/.julia/dev/TestGtkMegaJLL`
[db00978d] TightBinding v0.1.3
[37f6aa50] TikzPictures v3.3.1 `https://github.com/JuliaTeX/TikzPictures.jl.git#master`
[30578b45] URIParser v0.4.1
[112f6efa] VegaLite v2.4.1
[81def892] VersionParsing v1.2.0
[5e69872b] Xcb v0.2.0 `https://gitlab.com/poposca/Xcb.jl.git#master`
[77ec8976] GTK3_jll v3.24.11+5
[2e76f6c2] HarfBuzz_jll v2.6.1+10
[9c32591e] Poppler_jll v0.87.0+2
[ab825dc5] SDL2_jll v2.0.12+2
[502467ad] UBPF_jll v0.0.1+0
[3f19e933] p7zip_jll
If I do:
using Python
using PyCall
the latter segfaults (at least on 1.6-rc1). For the reverse order I do get an error.
I don't personally need both to work, just curious if they could (probably not). That's not really the end-goal, but some packages do have PyCall as a dependency, e.g. PyPlot.jl and Pandas.jl. Is there a good way to support such, as it seems this package has a similar API as PyCall?
i.e. put juliacall on the same footing.
Ensure the parent object is preserved
juliacall.AnyValue.__copy__
and __deepcopy__
could call Julia's copy
and deepcopy
.
Conversely, copy(::Py)
and deepcopy_internal(::Py, ::IdDict)
could call Python's copy.copy
and copy.deepcopy
.
Is rewrite complete now ?
If finished rewrite , can you release a version as soon as possible?
First of all, thanks for working on this, I really appreciate these improvements from PyCall.
Seems like calling pyimport
from the top level of a module in precompiled code would create issues.
In PyCall, the following approach is recommended:
const scipy_opt = PyNULL()
function __init__()
copy!(scipy_opt, pyimport_conda("scipy.optimize", "scipy"))
end
end
I think the equivalent in PythonCall would be
const scipy_opt = pynew()
function __init__()
pycopy!(scipy_opt, pyimport("scipy.optimize"))
end
It would be worth documenting this somewhere. Obviously a lot of folks are going to be coming from PyCall (and will know to be mindful of this) but it would be better to make this fact explicit, or indicate that this isn't required.
I once wanted to use a logo in colors of python in order to highlight that my project is intended to be used with python (it is essentially python bindings to some third-party code), but feeled like they can be trademarked.
I have contacted the email and asked for clarifiaction and permission. I was told that currently they don't intend to trademark python colors and that if I start using my logo, I'd "be" a my trademark (damn this legal shit, I wish there would be no trademark laws at all!). But the colors are already a de-facto trademark.
I asked if they can either give me a digitally signed text that the use of the colors in my logo is not considered trademark violation, or change their trademark policy to clarify which uses of colors are explicitly allowed. I was answered that this is the decision which is up to the board, and the board won't do that for me unless maybe for a lot of money.
IMHO this FUD situation is a bit annoying.
https://pyright.blogspot.com/2011/02/svg-xml-and-python-logo.html
Nuitka/Nuitka-website#19
PyAV-Org/PyAV#728
derb12/mcetl#13
https://github.com/ReagentX/Logria-py/issues/147
python-trio/trio#1828
amcc1996/symbeam#7
#3
fatiando/pooch#222
aiortc/aiortc#442
MatthijsKamstra/haxepython#3
It might make handling conversion of arrays between Julia and Python easier - or at least provide a consistent interface to override to control the behaviour.
Hey! Found this repo, looks cool! I'm looking into it as a possible new backing for https://github.com/SciML/diffeqpy . Specifically the issue is whether overloads in Julia for ModelingToolkit.jl could work well enough from Python. They do in diffeqr via JuliaCall, so it seems like it's just a wrapper issue with pyjulia. SciML/diffeqpy#57 (comment) in particular highlights such an issue. SciML/diffeqpy#78 is the same thing. If there's a good answer here, then I think got a new lead!
Hi, interesting project
I'm getting this error even when using a fresh Julia installation (1.6.3). OS: Mac OS 10.13.6.
julia> using PythonCall
ERROR: InitError: MethodError: no method matching project_file_name_uuid(::Bool, ::String)
Closest candidates are:
project_file_name_uuid(::String, ::String) at loading.jl:405
Stacktrace:
[1] init_context()
@ PythonCall.C ~/.julia/packages/PythonCall/YSeRU/src/cpython/context.jl:30
[2] __init__()
@ PythonCall.C ~/.julia/packages/PythonCall/YSeRU/src/cpython/CPython.jl:19
[3] _include_from_serialized(path::String, depmods::Vector{Any})
@ Base ./loading.jl:696
[4] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
@ Base ./loading.jl:782
[5] _require(pkg::Base.PkgId)
@ Base ./loading.jl:1020
[6] require(uuidkey::Base.PkgId)
@ Base ./loading.jl:936
[7] require(into::Module, mod::Symbol)
@ Base ./loading.jl:923
during initialization of module C
I've tried both:
pkg> add PythonCall
and
pkg> add PythonCall#master
We embed PyInternedString
s and PyLazyObject
s into the AST, but accessing them still does a NULL check and IncRef each time.
Instead, we could keep a list of all the lazy objects used, and a bool indicating whether or not they have all been checked, and check this once at the start of the DSL block. So the first time, all these objects are initialised, and thereafter we can just extract the pointer value directly.
Currently about 7 seconds, 80% compilation
This can be done by having a custom Pickler
and Unpickler
classes which take a Julia Serializer
on construction which is used to serialize any Julia objects it sees in persistent_id
. These classes are used when serializing a Python object.
Similarly we can have a custom Serializer
type which takes a Python Pickler
and Unpickler
on construction which is used to (de-)serialize any Python objects it sees. This class is used when pickling/unpickling a Julia object.
There is a considerable overhead from calling a C function (~15ns).
In the case of Py_IncRef
/Py_DecRef
we could modify the ref count directly from Julia, only calling Py_DecRef
once the ref count is <2.
When running the following example code:
using PythonCall
using Tables
@pyg `
import numpy as np
import pandas as pd
$x = pd.DataFrame(np.array((100, 100)))`
# Make sure the columns are strings, or it will fail in any case.
@pyg `$x.columns = map(str, $x.columns)`
y = PyPandasDataFrame(x)
Tables.columns(y)
I get the error:
ERROR: LoadError: map is not defined on sets
Stacktrace:
[1] error(s::String)
@ Base .\error.jl:33
[2] map(f::Type, #unused#::Base.KeySet{String, PyPandasDataFrame})
@ Base .\abstractarray.jl:2326
[3] columns(x::PyPandasDataFrame)
@ PythonCall [,,,]\.julia\packages\PythonCall\MBkDb\src\PyPandasDataFrame.jl:155
This could be fixed by changing the map call on line 155 to map(Symbol, collect(keys(x)))
instead.
Consider a dummy function inside my custom package EOS:
function dummy(x::AbstractVector)
@show typeof(x)
end
From Python, I call it as follows
jl.EOS.dummy(np.array([1.1, 2.3]))
and the following error occurs
TypeError: Julia: MethodError: no method matching dummy(::PythonCall.PyIterable{PythonCall.Py})
Closest candidates are:
dummy(!Matched::AbstractVector{T} where T) at /home/shuhua/Work/opt/EOS/src/EOS.jl:26
According to Conversion to Julia, I expect that a numpy array is converted to a Julia array automatically. Did I miss anything?
Hi! Thanks for this project. It would be nice if the README could highlight the difference between PythonCall and PyCall, so users can easily decide which package to use; or if both can be used together for particular tasks. Thanks!
I am trying to call a Julia function defined in a module (say MyModule, defined in MyModule.jl
) in a Python script. I did something along the lines of
from juliacall import Main as jl
jl.seval('push!(LOAD_PATH, "<path to MyModule.jl>")')
jl.seval("using MyModule")
ret = jl.MyModule.some_function(arg1,arg2,arg3)
however I am getting an error:
juliacall.Error: ArgumentError: Package MyModule not found in current path:
- Run `import Pkg; Pkg.add("MyModule")` to install the MyModule package.
What am I doing wrong?
Py<Wrapper>
PyIterable
PyList
PyDict
PySet
PyBuffer
PyArray
__array__
, __array_interface__
).PyPandasDataFrame
pandas.core.indexes.range.RangeIndex
to a StepRange{Int}
?PyIO
PyObjectArray <: AbstractArray{PyObject}
, represented as an array of CPyPtr
, used by default as output of pyjulia(::AbstractArray).__array__()
.pyjulia
__array_interface__
)__array__
) returning PyObjectArray
by default unless __array_interface__
already supported.pytypestrformat
to work on any type.pyjlraw
from pyjl
BufferedIterator
which iterates over 1024 (say) values at a time and saves them as a buffer of PyObject
.pyconvert
__array__
)np.float32
etc)_repr_MIME_
methods from ipython)QT_PLUGIN_PATH
hack@pyeval
, @pyexec
, py"..."
for calling pyeval
and pyexec
with Julia value interpolation?some_python_function
should show the function name and signaturepygiloff
, pygilon
, etc)sys.stdout
to go the the kernelA declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.