Question: What hinders a macOS/Windows port? about mirage HOT 19 OPEN

With some patches, I managed launch Mirage on Windows 8.1 and Windows 10. I guess everything is working fine. Any problem you tell me. I guess these changes will be merged once testing phase passes.

Getting Mirage to run

A note: do not use --user option when running pip, as we are going to copy the entire python dist later.

1. Install MSYS2

   • We will be using MINGW64 environment from now on, so start shell/cmd using MINGW64 script

2. Install dependencies: pacman -S base-devel git mingw-w64-x86_64-{toolchain,cmake,qt5,libjpeg-turbo,libtiff,libwebp,openjpeg2,python-pip,zstd,cairo}

3. Clone, build and install my fork of pyotherside

   • Should build and install with no error
   • Tip: do not use -jN option of make
   • After installing, run mv -v /mingw64/share/qt5/qml/io/thp/pyothersideplugin{1,}.dll if needed. It should not contain the 1 on its name.
   • My fork contains fixes for building on mingw and correcting runtime issues.

4. Clone and build olm (do not install)

   • Use clang to build, as make fails:

   $ cmake . -Bbuild -G "MinGW Makefiles"
   $ cmake --build build
   

   • After building the shared library, install the python binding:

   $ cd python
   $ pip install -Ur requirements.txt
   $ make olm-python3
   $ pip install .
   

   • If for some case you get failed to create process ... when running the pip command, reinstall python and pip: pacman -S mingw-w64-x86_64-python-{,-pip}
   • No errors are expected in this step.

5. Clone and install my fork of matrix-nio

   • It contains fixes for managing file names on Windows.
   1. git clone https://github.com/vSLG/matrix-nio.git
   2. pip install .

6. Clone and build my fork of Mirage (should be merged soon)

   • git clone https://github.com/vSLG/mirage -b win --recursive && cd mirage
   • First of all, run cp -v /mingw64/lib/libz{,lib}.a, as lxml native extension fails to build because it uses -lzlib instead of -lz
   1. Install python dependencies: pip install -Ur requirements.txt
   2. Run qmake mirage.pro
   3. Run make (do not use -jN option), it will generate a single .exe file in a directory called release.
   4. Copy Olm dll we built before: cp <olm_source>/build/libolm.dll release
   • At this point, you can run mirage by typing PYTHONHOME=/mingw64 ./release/mirage. If you desire to open Mirage with a double click, follow down the instructions for packaging:
   5. Copy QT dlls to release directory: windeployqt.exe --qmldir . ./release/mirage.exe.
   6. Copy leftover dlls: cp -v $(ldd ./release/mirage.exe | cut -d ' ' -f 3 | grep mingw64) release
   7. Copy python dll: cp -v /mingw64/bin/libpython3.8.dll release
   8. Create a lib directory and copy python dist with every package we need:

   $ mkdir release/lib
   $ cp -r /mingw64/lib/python3.8 release/lib
   

   9. Cleanup python cache: find release -type d -name "__pycache__" -exec rm -vrf {} \;
   • Now the release directory may be suitable for packaging. You can copy it to you program files or AppData and create a shortcut to your desktop.
   • If you desire to get rid of the cmd that opens with Mirage:
   1. Edit mirage.pro
   2. Find the line with CONFIG += ... and remove console from it.
   3. Run make clean (will not delete release directory)
   4. Run qmake
   5. And finally run make.

from mirage.

I just had some time on my hand and got Mirage up and running on macOS and thought I'd share what I did. And... it was insanely easy! :)

1. Install the macOS command line tools (xcode-select --install)
2. Install the Homebrew package manager.
3. Install the various libraries listed in the "Manual Install" section. (brew search to find them, brew install to install. I had all of them already installed.)
4. Install libolm.
5. install Qt 5: brew install qt5
6. Either add the Qt setup to your $PATH or just use the full path for QMake (/usr/local/opt/qt/bin/qmake)
7. Install PyOtherSide manually
8. Build Mirage.

After make, there is a mirage.app. This is the app that ultimatively is able to run the app. However, I haven't figured out where and how the app has it's Python files and I am not so much a snake-y guy...

I believe that if you built a Python 3 venv, statically compiled Qt5 libraries and dependencies and then also statically built the few native dependencies, all you would have to do is figure out a way to ship a relocateable venv. And that'd be it. Well, on macOS you could rely on the fact that most applications are at /Applications - and in this case, /Applications/mirage.app.

So this will be fun to figure out. :) A small shell-script should be able to automate this build as well. Just static-build everything into something like /opt/mirage (prefix), then build Mirage and link "as static as posible", then strip that and pack it into a relocateable Python 3 venv. That should yield a redistributable package for macos. :)

from mirage.

From these lines of your output:

      /usr/include/python3.8/Python.h:44:10: schwerwiegender Fehler: crypt.h: No such file or directory
         44 | #include <crypt.h>
            |          ^~~~~~~~~

It seems like you're missing libcrypt-devel, which should be a dependency of python-devel.

from mirage.

• Windows: I did manage to start Mirage in it a long time ago, haven't tested since. The only real trouble is the pip olm library, having to go through the pain of installing visual studio to compile it, unless there's a simpler way I'm not aware of
• Mac: I haven't had any success yet trying to setup a VM for it, but some people have already managed to run it and posted in the mirage room about it.

Either way, if someone can build it on non-Linux systems, an INSTALL.md instruction addition PR would be greatly appreciated.

from mirage.

Also as for Windows: You could just use a minimal llvm/mingw setup. They use just about the same headers and its much faster. Chocolatey can be a big help as well!

My PC's PSU exploded on saturday, so I can't currently check back on my Win10 install - sorry.

from mirage.

After make, there is a mirage.app. This is the app that ultimatively is able to run the app. However, I haven't figured out where and how the app has it's Python files and I am not so much a snake-y guy...

The built binary, unless qmake has been run with CONFIG+=dev, uses the Qt Resource System to have all the python and QML files self-contained.

When running qmake, mirage.pro takes care of generating the qrc file that lists what should go in the binary here.

Concerning deploying, I found some info about a tool that should take care of the annoying stuff:
https://doc.qt.io/qt-5.9/osx-deployment.html#the-mac-deployment-tool
https://www.ics.com/blog/your-guide-deploying-qt-applications-macos

from mirage.

Also as for Windows: You could just use a minimal llvm/mingw setup. They use just about the same headers and its much faster. Chocolatey can be a big help as well!

Recently got Mirage running on Windows with MSYS2/MinGW environment. But I am having trouble with pyotherside currently. Everything else is fine, I hope.

Some notes:

• For building pyotherside, QT should not be statically compiled
• QT can be either self compiled or obtained from MSYS MinGW repository
• Python must be from MSYS repository if you do not want to use MSVC, as Python downloaded from python.org is compiled with MSVC and anything installed with pip with C extensions requires MSVC to compile

from mirage.

I would appreciate it very much if Mirage gets ported to Windows.

from mirage.

@vSLG: 87fcb0a should get rid of the problem with PyOtherSide.

from mirage.

Would love to see this working on Windows! Wish I could figure out how to do this myself. ^^;

from mirage.

It would be really greate to have a working Mirage on Windows. If its any help i could to build but i am no coder so i would need more instructions.

from mirage.

so i try to build with msys. But its my first time so i may ask some noob questions. i try to install pyotherside manually according to the INSTALL.md but make clean is not possible. libolm worked after bit of errors and now i try to build mirage itself:
i got big error about pip-wheel as far as i understood. i pasted the error here
i hope this i the right place to ask my questions.

from mirage.

I missed to install python-devel. but now i'm hanging at pillow; i installed also the packaged version but i found nothing about how use this instead of building.
The error for pillow:
I guess i miss some jpeg headers but according to python-pillow/Pillow#4649 (comment).
The Error log is here
Pls say if i post to much here i could also stop this and wait ;)

from mirage.

@vSLG

Thanks for your guide and patches. I just tried following on a pretty clean Windows 10 build, but I seem to be having some trouble.

It was necessary to also run the following commands

pip uninstall typing
pip install setuptools

PYTHONHOME=/mingw64 ./release/mirage exits nearly immediately with code 127.

Following the steps to create a packagable release, release\io\thp\pyotherside\pyothersideplugin1.dll had to be renamed to pyothersideplugin.dll, even though mv -v /mingw64/share/qt5/qml/io/thp/pyothersideplugin{1,}.dll was not necessary (I ran it, and it didn't find the file).

After all that, it looks like some modules are still missing: https://gist.github.com/maciozo/ec4ac052f0474d5916866c964302d799

I'm not sure if it's something I've missed, but I've tried building Mirage twice now.

from mirage.

@maciozo Thanks for the feedback. These instructions are a bit old, and I'm to write new instructions.

PYTHONHOME=/mingw64 ./release/mirage exits nearly immediately with code 127.

Does ./release/mirage actually exist? Do you have any output log about this?

Following the steps to create a packagable release, release\io\thp\pyotherside\pyothersideplugin1.dll had to be renamed to pyothersideplugin.dll, even though mv -v /mingw64/share/qt5/qml/io/thp/pyothersideplugin{1,}.dll was not necessary (I ran it, and it didn't find the file).

It seems I have typed the wrong path for pyothersideplugin1.dll under /mingw64/share. Going to fix that in the next instructions. Thanks for pointing out.

After all that, it looks like some modules are still missing:
https://gist.github.com/maciozo/ec4ac052f0474d5916866c964302d799

This error is due to native library missing on the system. I guess installing mingw-w64-x86_64-libffi with pacman fixes this issue. You can try this and post if you find any further issue.

Also, I recommend you testing Mirage on a MSYS terminal, and not on cmd/ps like in your gist, and if all goes well, you can try on cmd/ps.

from mirage.

So i try this again ;) and now i going quite far without errors, thanks for your guide ;)

i copied the release to my desktop an d run the .exe ; i renamed also the release\io\thp\pyotherside\pyothersideplugin1.dll

The mirage window ist opening with a loading screen and in the cmd ist the following output:
Hope you can help me? ;)

`~`` 12:03:28 | "PyOtherSide error: Traceback (most recent call last):\n\n  File \"C:\\Users\\Led\\Desktop\\mirage\\lib\\python3.8\\site-packages\\Crypto\\Util\\_raw_api.py\", line 80, in <module>\n    ffi = FFI()\n\n  File \"C:\\Users\\Led\\Desktop\\mirage\\lib\\python3.8\\site-packages\\cffi\\api.py\", line 48, in __init__\n    import _cffi_backend as backend\n\nImportError: DLL load failed while importing _cffi_backend: Das angegebene Modul wurde nicht gefunden.\n\n\nDuring handling of the above exception, another exception occurred:\n\n\nTraceback (most recent call last):\n\n  File \"qrc:/src/backend/qml_bridge.py\", line 29, in <module>\n    from .pyotherside_events import CoroutineDone, LoopException\n\n  File \"qrc:/src/backend/pyotherside_events.py\", line 10, in <module>\n    from .utils import serialize_value_for_qml\n\n  File \"qrc:/src/backend/utils.py\", line 31, in <module>\n    from nio.crypto import AsyncDataT as File\n\n  File \"C:\\Users\\Led\\Desktop\\mirage\\lib\\python3.8\\site-packages\\nio\\__init__.py\", line 2, in <module>\n    from .client import *\n\n  File \"C:\\Users\\Led\\Desktop\\mirage\\lib\\python3.8\\site-packages\\nio\\client\\__init__.py\", line 3, in <module>\n    from .base_client import Client, ClientConfig\n\n  File \"C:\\Users\\Led\\Desktop\\mirage\\lib\\python3.8\\site-packages\\nio\\client\\base_client.py\", line 35, in <module>\n    from ..crypto import ENCRYPTION_ENABLED\n\n  File \"C:\\Users\\Led\\Desktop\\mirage\\lib\\python3.8\\site-packages\\nio\\crypto\\__init__.py\", line 16, in <module>\n    from .attachments import encrypt_attachment, decrypt_attachment\n\n  File \"C:\\Users\\Led\\Desktop\\mirage\\lib\\python3.8\\site-packages\\nio\\crypto\\attachments.py\", line 25, in <module>\n    from Crypto.Cipher import AES\n\n  File \"C:\\Users\\Led\\Desktop\\mirage\\lib\\python3.8\\site-packages\\Crypto\\Cipher\\__init__.py\", line 27, in <module>\n    from Crypto.Cipher._mode_ecb import _create_ecb_cipher\n\n  File \"C:\\Users\\Led\\Desktop\\mirage\\lib\\python3.8\\site-packages\\Crypto\\Cipher\\_mode_ecb.py\", line 29, in <module>\n    from Crypto.Util._raw_api import (load_pycryptodome_raw_lib,\n\n  File \"C:\\Users\\Led\\Desktop\\mirage\\lib\\python3.8\\site-packages\\Crypto\\Util\\_raw_api.py\", line 159, in <module>\n    import ctypes\n\n  File \"C:\\Users\\Led\\Desktop\\mirage\\lib\\python3.8\\ctypes\\__init__.py\", line 7, in <module>\n    from _ctypes import Union, Structure, Array\n\nImportError: DLL load failed while importing _ctypes: Das angegebene Modul wurde nicht gefunden.\n"
! 12:03:28 | Unhandled PyOtherSide error: Cannot import module: backend.qml_bridge (Traceback (most recent call last):

  File "C:\Users\Led\Desktop\mirage\lib\python3.8\site-packages\Crypto\Util\_raw_api.py", line 80, in <module>
    ffi = FFI()

  File "C:\Users\Led\Desktop\mirage\lib\python3.8\site-packages\cffi\api.py", line 48, in __init__
    import _cffi_backend as `backend`

ImportError: DLL load failed while importing _cffi_backend: Das angegebene Modul wurde nicht gefunden.


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "qrc:/src/backend/qml_bridge.py", line 29, in <module>
    from .pyotherside_events import CoroutineDone, LoopException

  File "qrc:/src/backend/pyotherside_events.py", line 10, in <module>
    from .utils import serialize_value_for_qml

  File "qrc:/src/backend/utils.py", line 31, in <module>
    from nio.crypto import AsyncDataT as File

  File "C:\Users\Led\Desktop\mirage\lib\python3.8\site-packages\nio\__init__.py", line 2, in <module>
    from .client import *

  File "C:\Users\Led\Desktop\mirage\lib\python3.8\site-packages\nio\client\__init__.py", line 3, in <module>
    from .base_client import Client, ClientConfig

  File "C:\Users\Led\Desktop\mirage\lib\python3.8\site-packages\nio\client\base_client.py", line 35, in <module>
    from ..crypto import ENCRYPTION_ENABLED

  File "C:\Users\Led\Desktop\mirage\lib\python3.8\site-packages\nio\crypto\__init__.py", line 16, in <module>
    from .attachments import encrypt_attachment, decrypt_attachment

  File "C:\Users\Led\Desktop\mirage\lib\python3.8\site-packages\nio\crypto\attachments.py", line 25, in <module>
    from Crypto.Cipher import AES

  File "C:\Users\Led\Desktop\mirage\lib\python3.8\site-packages\Crypto\Cipher\__init__.py", line 27, in <module>
    from Crypto.Cipher._mode_ecb import _create_ecb_cipher

  File "C:\Users\Led\Desktop\mirage\lib\python3.8\site-packages\Crypto\Cipher\_mode_ecb.py", line 29, in <module>
    from Crypto.Util._raw_api import (load_pycryptodome_raw_lib,

  File "C:\Users\Led\Desktop\mirage\lib\python3.8\site-packages\Crypto\Util\_raw_api.py", line 159, in <module>
    import ctypes

  File "C:\Users\Led\Desktop\mirage\lib\python3.8\ctypes\__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array

ImportError: DLL load failed while importing _ctypes: Das angegebene Modul wurde nicht gefunden.
)
~ 12:03:29 | "PyOtherSide error: Traceback (most recent call last):\n\n  File \"<string>\", line 1, in <module>\n\nNameError: name 'BRIDGE' is not defined\n"
! 12:03:29 | Unhandled PyOtherSide error: Function not found: 'BRIDGE.call_backend_coro' (Traceback (most recent call last):

  File "<string>", line 1, in <module>

NameError: name 'BRIDGE' is not defined
)` 

from mirage.

I'm having some issues of my own, make olm-python3 is returned with "Nothing to be done for olm-python3", and pip install . gives me this error

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

carreau@tmiz MINGW64 ~/olm/python
$ Processing c:/msys64/home/carreau/olm/python
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: C:/msys64/mingw64/bin/python.exe -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\msys64\\home\\carreau\\olm\\python/setup.py'"'"'; __file__='"'"'C:\\msys64\\home\\carreau\\olm\\python/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base C:/Users/carreau/AppData/Local/Temp/pip-pip-egg-info-80qgzx5m
       cwd: C:\msys64\home\carreau\olm\python/
  Complete output (23 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "C:\msys64\home\carreau\olm\python/setup.py", line 14, in <module>
      setup(
    File "C:/msys64/mingw64/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "C:/msys64/mingw64/lib/python3.9/distutils/core.py", line 108, in setup
      _setup_distribution = dist = klass(attrs)
    File "C:/msys64/mingw64/lib/python3.9/site-packages/setuptools/dist.py", line 459, in __init__
      _Distribution.__init__(
    File "C:/msys64/mingw64/lib/python3.9/distutils/dist.py", line 292, in __init__
      self.finalize_options()
    File "C:/msys64/mingw64/lib/python3.9/site-packages/setuptools/dist.py", line 837, in finalize_options
      ep(self)
    File "C:/msys64/mingw64/lib/python3.9/site-packages/setuptools/dist.py", line 858, in _finalize_setup_keywords
      ep.load()(self, ep.name, value)
    File "C:/msys64/mingw64/lib/python3.9/site-packages/cffi/setuptools_ext.py", line 219, in cffi_modules
      add_cffi_module(dist, cffi_module)
    File "C:/msys64/mingw64/lib/python3.9/site-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module
      execfile(build_file_name, mod_vars)

from mirage.

Any news on this? Going to give it another try later today. Since MSVC now uses LLVM/Clang since a while, I am kinda intrigued to try to just build the whole thing without MSYS/MinGW and see how far it goes. But would appreciate feedback from you if someone got it working nicely - and in a way that can be automated for releases. :)

from mirage.

No news, but if you try building that would be awesome!
Just a heads up, you might want to use Moment rather than Mirage

from mirage.