Changes between Initial Version and Version 1 of PyOtherSideForAndroidNotesArchive

Timestamp:

Nov 15, 2014, 8:37:09 PM (9 years ago)

Author:

Martin Kolman

Comment:

--

PyOtherSideForAndroidNotesArchive

@@ +1 @@
+= !PyOtherSide for Android development note archive =
+
+A litle out of date development notes from the !PyOtherSide for Android porting project. !PyOtherSide now works on Android and you can check the main [wiki:PyOtherSideForAndroid PyOtherSide for Android article] for more details.
+
+== !PyOtherSide ==
+
+=== Setting up ===
+This are the version of software versions I have used for my experiments, it is possible other versions might also work, but this is what I have worked with.
+
+'''Get !PyOtherSide 1.3 source code'''
+
+{{{wget https://thp.io/2011/pyotherside/pyotherside-1.3.0.tar.gz}}}
+
+'''Get & install the Qt 5.3 Android SDK'''
+
+Basically just follow [http://qt-project.org/doc/qt-5/androidgs.html the guide] on the Qt wiki to get the Qt 5 Android SDK & it's dependencies installed.
+
+'''Configure an Android device to be accessible by ADB'''
+I've tested the resulting software on my HP TouchPad running Cyanogenmod Android 4.0, accessible with ADB. So this 'guide/howto' expect that you also have an Android device accessible over ADB. The Android emulator might also work, but I haven't tried that.
+
+=== Compiling for Android ===
+
+==== Preparing the !PyOtherSide !QtCreator project ====
+'''If you don't care and just want the project tarball, skip to the end of this section. :)'''
+
+
+Unpack the !PyOtherSide tarball and enter the resulting directory:
+{{{
+tar -xvf pyotherside-1.3.0.tar.gz
+cd pyotherside-1.3.0
+}}}
+
+Next you need a Python 3 compiled for Android, so you can build !PyOtherSide against it. I have found such Python 3 distribution on the [https://code.google.com/p/android-python27/ Python 2.7 for Android project] website and used it drop in library bundle for compiling !PyOtherSide.
+
+'''Note about the precompiled Python 3.2 libs'''
+
+They originally come from these files:
+
+[https://code.google.com/p/android-python27/source/browse/apk_python32/res/raw/python_32.zip https://code.google.com/p/android-python27/source/browse/apk_python32/res/raw/python_32.zip]
+
+[https://code.google.com/p/android-python27/source/browse/apk_python32/res/raw/python_extras_32.zip https://code.google.com/p/android-python27/source/browse/apk_python32/res/raw/python_extras_32.zip]
+
+The binaries have been modified to drop .so versioning which is not supported on Android and breaks compilation:
+
+{{{rpl -R -e libpython3.2m.so.1.0 "libpython3.2.so\x00\x00\x00\x00\x00" .}}}
+
+'''End of the note about the precompiled Python 3.2 libs'''
+
+So download the lib bundle tarball to the Pyotherside folder and unpack it here:
+{{{
+wget http://www.modrana.org/platforms/android/pyotherside/guide/python3_android_libs_bundle.tar.gz
+tar xvf python3_android_libs_bundle.tar.gz
+}}}
+
+When you are done, there should be a ''libs'' folder in your ''pyotherside-1.3.0 folder''.
+
+In the next step we need to tell the project to use the Android compiled libraries instead of the system ones, so edit the ''src/src.pro'' file and replace the line reading:
+{{{
+INCLUDEPATH += .
+}}}
+With this:
+{{{
+INCLUDEPATH += $$PWD/../libs/python32/build_dependencies/ .
+LIBS += -L$$PWD/../libs/python32 -lpython3.2 -ldl
+}}}
+
+Then in ''tests/tests.pro'' replace:
+{{{
+INCLUDEPATH += . ../src
+}}}
+
+With:
+
+{{{
+INCLUDEPATH += $$PWD/../libs/python32/build_dependencies/ . ../src
+LIBS += -L$$PWD/../libs/python32 -lpython3.2
+}}}
+
+This tells the project to use our locally available Python 3 compiled for Android instead of the system-wide Python 3 when building !PyOtherSide.
+
+We also need to prevent fetching some other local Python stuff. Therefore in ''python.pri'' (directly in the ''pyotherside-1.3.0'' folder) comment out these lines:
+
+{{{
+isEmpty(PYTHON_CONFIG) {
+    PYTHON_CONFIG = python3-config
+}
+}}}
+
+And these lines:
+{{{
+QMAKE_LIBS += $$system($$PYTHON_CONFIG --ldflags)
+QMAKE_CXXFLAGS += $$system($$PYTHON_CONFIG --includes)
+}}}
+
+The resulting python.pri file should look like this:
+
+{{{
+#isEmpty(PYTHON_CONFIG) {
+#    PYTHON_CONFIG = python3-config
+#}
+
+message(PYTHON_CONFIG = $$PYTHON_CONFIG)
+
+#QMAKE_LIBS += $$system($$PYTHON_CONFIG --ldflags)
+#QMAKE_CXXFLAGS += $$system($$PYTHON_CONFIG --includes)
+}}}
+
+And for the last step, we need to edit the ''src/global_libpython_loader.cpp'' file: 
+
+ * changing ''#ifdef !__linux!__'' into ''#ifndef !__linux!__'' (so that the stuff we need will be used on Android)
+
+ * remove a GNU macro an linh.h file (both are broken on Android and break the build):
+
+{{{
+#define _GNU_SOURCE
+#include <link.h>
+}}}
+
+ * add a few headers file needed for compilation on Android (found by studying compile errors):
+
+{{{
+#include <dlfcn.h>
+#include <link.h>
+}}}
+
+The resulting file looks like this:
+
+{{{
+/**
+ * PyOtherSide: Asynchronous Python 3 Bindings for Qt 5
+ * Copyright (c) 2013, Thomas Perl <m@thp.io>
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
+ * REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
+ * FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
+ * INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+ * LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
+ * OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+ * PERFORMANCE OF THIS SOFTWARE.
+ **/
+
+#include "global_libpython_loader.h"
+
+namespace GlobalLibPythonLoader {
+
+#ifndef __linux__
+
+#include <stdio.h>
+#include <string.h>
+
+#include <dlfcn.h>
+#include <link.h>
+
+static int load_python_globally_callback(struct dl_phdr_info *info, size_t size, void *data)
+{
+    int major, minor;
+    const char *basename = strrchr(info->dlpi_name, '/');
+    int *success = (int *)data;
+
+    if (basename != NULL) {
+        if (sscanf(basename, "/libpython%d.%d.so", &major, &minor) != 2) {
+            if (sscanf(basename, "/libpython%d.%dm.so", &major, &minor) != 2) {
+                return 0;
+            }
+        }
+
+        void *pylib = dlopen(info->dlpi_name, RTLD_GLOBAL | RTLD_NOW);
+        if (pylib != NULL) {
+            *success = 1;
+        } else {
+            fprintf(stderr, "Could not load python library '%s': %s\n",
+                    info->dlpi_name, dlerror());
+        }
+    }
+
+    return 0;
+}
+
+bool loadPythonGlobally()
+{
+    int success = 0;
+    dl_iterate_phdr(load_python_globally_callback, &success);
+    return success;
+}
+
+
+#else /* __linux__ */
+
+bool loadPythonGlobally()
+{
+    /* On non-Linux systems, no need to load globally */
+    return true;
+}
+
+#endif /* __linux__ */
+
+}; /* namespace GlobalLibPythonLoader */
+
+}}}
+
+This last change makes !PyOtherSide compile with the crippled Bionic C library used on Android.
+
+'''The resulting project should look like contents of this tarball:'''
+
+[http://www.modrana.org/platforms/android/pyotherside/guide/pyotherside-1.3.0_android_project.tar.gz http://www.modrana.org/platforms/android/pyotherside/guide/pyotherside-1.3.0_android_project.tar.gz]
+
+
+==== Building the !PyOtherSide project in !QtCreator ====
+
+Now the !PyOtherSide project should be ready to be opened in !QtCreator (make sure you are running the one corresponding to the Qt 5.3 Android SDK you have downloaded and installed previously). So start the right !QtCreator and open the ''pyotherside-1.3.0/pyotherside.pro'' file.
+
+You should be able to build the !PyOtherSide project by pressing the "rebuild" button in !QtCreator.
+
+=== Basic deployment ===
+==== Theory ====
+For a Python program using !PyOtherSide to work it needs these things present on the Android device:
+* the application files
+* Qt 5
+* a Python 3 bundle
+* the !PyOtherSide library
+We don't need to care about Qt - its deployment is handled automatically by the Qt SDK Android support (it is either bundled in the APK, automatically copied to the device or provided by the Ministro library manager) and we don't need to care about it. The SDK also handles the deployment of the application files.
+
+But we need to get Python 3 & !PyOtherSide on the android device our self. For now we will just copy the files to a fixed location on the device with ''adb'' and hardcode a path pointing to them in the example application. This should be enough for getting a proof of concept of (hopefully) working !PyOtherSide on Android.
+
+In the future it should be possible deploy these files from a standalone APK without any need for manual intervention or rooting - either by using the new asset mechanism or the old method (used by the !PySide for Android port) of bundling zip files that are unpacked on first start by modified Java boilerplate.
+
+==== Deploying Python 3 & !PyOtherSide to the Android device ====
+
+Download the pre-prepared tarball with Python 3 and !PyOtherSide:
+
+{{{
+wget http://www.modrana.org/platforms/android/pyotherside/guide/python3_pyotherside_deployment_bundle.tar.gz
+}}}
+
+Then unpack it:
+
+{{{
+tar xvf python3_pyotherside_deployment_bundle.tar.gz
+}}}
+
+And push the complete ''poside'' directory directly to the ''/data'' directory on your Android device:
+
+{{{
+adb push poside /data/poside
+}}}
+
+This is where the hello world example will expect it. Also, we can't deploy to the ''/data/sdcard'' directory as it isi usually mounted noexec which breaks the library loading we need to do.
+
+==== Deploying & running a hello world application ====
+
+Next download the hello world project tarball and unpack it:
+
+{{{
+wget http://www.modrana.org/platforms/android/pyotherside/guide/pyotherside_hello_world_project.tar.gz
+}}}
+
+It is just a normal !QtQuick Controls Android project template modified to load the needed libraries from ''/data/poside'' and with all the needed environmental variables set.
+
+Open the project file in the !QtCreator provided by the Qt for Android SDK
+(the main project file is called ''A52.pro''), make sure the android kit with correct architecture for your device is selected, press the ''Run'' button and make sure your device with the deployed libs is selected as the deployment target. 
+
+!QtCreator should now do its thing and the application should successfully start on the device, presenting you a simple ''Hello world'' button in the middle of the screen.
+
+Now look on the main.qml file in !QtCreator and note that while it has the {{{import io.thp.pyotherside 1.0}}} line, the {{{Python {} }}} line that actually instantiates the Python element provided by !PyOtherSide is commented out.
+
+Next uncomment {{{Python {} }}} line and try to deploy the application again. This time the deployment fails and the applications crashes as soon as it is started.
+
+==== Current status quo ====
+
+So this how & how far I got - !PyOtherSide can be imported in a !QtQuick application running on Android, but crashes once the Python element is instantiated. 
+
+This guide should enable anyone get as far as I got and hopefully get far past the current roadblock! :)
+
+Any help with finding out how to fix the crash would be much appreciated! :)
+
+'''UPDATE: Works now, see the note on the top for more info. The likely cause of the crash was most probably the precompiled Python 3.2.'''