Changes between Version 25 and Version 26 of PyOtherSideForAndroid


Ignore:
Timestamp:
Nov 15, 2014, 8:37:30 PM (7 years ago)
Author:
Martin Kolman
Comment:

Archive the development notes to a separate wiki page.

Legend:

Unmodified
Added
Removed
Modified
  • PyOtherSideForAndroid

    v25 v26  
    1818
    1919
    20 == !PyOtherSide ==
    21 
    22 === Setting up ===
    23 This are the version of software versions I have used for my experiments, it is possible other versions might also work, but this is what I have worked with.
    24 
    25 '''Get !PyOtherSide 1.3 source code'''
    26 
    27 {{{wget https://thp.io/2011/pyotherside/pyotherside-1.3.0.tar.gz}}}
    28 
    29 '''Get & install the Qt 5.3 Android SDK'''
    30 
    31 Basically just follow [http://qt-project.org/doc/qt-5/androidgs.html the guide] on the Qt wiki to get the Qt 5 Android SDK & it's dependencies installed.
    32 
    33 '''Configure an Android device to be accessible by ADB'''
    34 I've tested the resulting software on my HP TouchPad running Cyanogenmod Android 4.0, accessible with ADB. So this 'guide/howto' expect that you also have an Android device accessible over ADB. The Android emulator might also work, but I haven't tried that.
    35 
    36 === Compiling for Android ===
    37 
    38 ==== Preparing the !PyOtherSide !QtCreator project ====
    39 '''If you don't care and just want the project tarball, skip to the end of this section. :)'''
    40 
    41 
    42 Unpack the !PyOtherSide tarball and enter the resulting directory:
    43 {{{
    44 tar -xvf pyotherside-1.3.0.tar.gz
    45 cd pyotherside-1.3.0
    46 }}}
    47 
    48 Next you need a Python 3 compiled for Android, so you can build !PyOtherSide against it. I have found such Python 3 distribution on the [https://code.google.com/p/android-python27/ Python 2.7 for Android project] website and used it drop in library bundle for compiling !PyOtherSide.
    49 
    50 '''Note about the precompiled Python 3.2 libs'''
    51 
    52 They originally come from these files:
    53 
    54 [https://code.google.com/p/android-python27/source/browse/apk_python32/res/raw/python_32.zip https://code.google.com/p/android-python27/source/browse/apk_python32/res/raw/python_32.zip]
    55 
    56 [https://code.google.com/p/android-python27/source/browse/apk_python32/res/raw/python_extras_32.zip https://code.google.com/p/android-python27/source/browse/apk_python32/res/raw/python_extras_32.zip]
    57 
    58 The binaries have been modified to drop .so versioning which is not supported on Android and breaks compilation:
    59 
    60 {{{rpl -R -e libpython3.2m.so.1.0 "libpython3.2.so\x00\x00\x00\x00\x00" .}}}
    61 
    62 '''End of the note about the precompiled Python 3.2 libs'''
    63 
    64 So download the lib bundle tarball to the Pyotherside folder and unpack it here:
    65 {{{
    66 wget http://www.modrana.org/platforms/android/pyotherside/guide/python3_android_libs_bundle.tar.gz
    67 tar xvf python3_android_libs_bundle.tar.gz
    68 }}}
    69 
    70 When you are done, there should be a ''libs'' folder in your ''pyotherside-1.3.0 folder''.
    71 
    72 In the next step we need to tell the project to use the Android compiled libraries instead of the system ones, so edit the ''src/src.pro'' file and replace the line reading:
    73 {{{
    74 INCLUDEPATH += .
    75 }}}
    76 With this:
    77 {{{
    78 INCLUDEPATH += $$PWD/../libs/python32/build_dependencies/ .
    79 LIBS += -L$$PWD/../libs/python32 -lpython3.2 -ldl
    80 }}}
    81 
    82 Then in ''tests/tests.pro'' replace:
    83 {{{
    84 INCLUDEPATH += . ../src
    85 }}}
    86 
    87 With:
    88 
    89 {{{
    90 INCLUDEPATH += $$PWD/../libs/python32/build_dependencies/ . ../src
    91 LIBS += -L$$PWD/../libs/python32 -lpython3.2
    92 }}}
    93 
    94 This tells the project to use our locally available Python 3 compiled for Android instead of the system-wide Python 3 when building !PyOtherSide.
    95 
    96 We also need to prevent fetching some other local Python stuff. Therefore in ''python.pri'' (directly in the ''pyotherside-1.3.0'' folder) comment out these lines:
    97 
    98 {{{
    99 isEmpty(PYTHON_CONFIG) {
    100     PYTHON_CONFIG = python3-config
    101 }
    102 }}}
    103 
    104 And these lines:
    105 {{{
    106 QMAKE_LIBS += $$system($$PYTHON_CONFIG --ldflags)
    107 QMAKE_CXXFLAGS += $$system($$PYTHON_CONFIG --includes)
    108 }}}
    109 
    110 The resulting python.pri file should look like this:
    111 
    112 {{{
    113 #isEmpty(PYTHON_CONFIG) {
    114 #    PYTHON_CONFIG = python3-config
    115 #}
    116 
    117 message(PYTHON_CONFIG = $$PYTHON_CONFIG)
    118 
    119 #QMAKE_LIBS += $$system($$PYTHON_CONFIG --ldflags)
    120 #QMAKE_CXXFLAGS += $$system($$PYTHON_CONFIG --includes)
    121 }}}
    122 
    123 And for the last step, we need to edit the ''src/global_libpython_loader.cpp'' file:
    124 
    125  * changing ''#ifdef !__linux!__'' into ''#ifndef !__linux!__'' (so that the stuff we need will be used on Android)
    126 
    127  * remove a GNU macro an linh.h file (both are broken on Android and break the build):
    128 
    129 {{{
    130 #define _GNU_SOURCE
    131 #include <link.h>
    132 }}}
    133 
    134  * add a few headers file needed for compilation on Android (found by studying compile errors):
    135 
    136 {{{
    137 #include <dlfcn.h>
    138 #include <link.h>
    139 }}}
    140 
    141 The resulting file looks like this:
    142 
    143 {{{
    144 /**
    145  * PyOtherSide: Asynchronous Python 3 Bindings for Qt 5
    146  * Copyright (c) 2013, Thomas Perl <m@thp.io>
    147  *
    148  * Permission to use, copy, modify, and/or distribute this software for any
    149  * purpose with or without fee is hereby granted, provided that the above
    150  * copyright notice and this permission notice appear in all copies.
    151  *
    152  * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
    153  * REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
    154  * FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
    155  * INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
    156  * LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
    157  * OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
    158  * PERFORMANCE OF THIS SOFTWARE.
    159  **/
    160 
    161 #include "global_libpython_loader.h"
    162 
    163 namespace GlobalLibPythonLoader {
    164 
    165 #ifndef __linux__
    166 
    167 #include <stdio.h>
    168 #include <string.h>
    169 
    170 #include <dlfcn.h>
    171 #include <link.h>
    172 
    173 static int load_python_globally_callback(struct dl_phdr_info *info, size_t size, void *data)
    174 {
    175     int major, minor;
    176     const char *basename = strrchr(info->dlpi_name, '/');
    177     int *success = (int *)data;
    178 
    179     if (basename != NULL) {
    180         if (sscanf(basename, "/libpython%d.%d.so", &major, &minor) != 2) {
    181             if (sscanf(basename, "/libpython%d.%dm.so", &major, &minor) != 2) {
    182                 return 0;
    183             }
    184         }
    185 
    186         void *pylib = dlopen(info->dlpi_name, RTLD_GLOBAL | RTLD_NOW);
    187         if (pylib != NULL) {
    188             *success = 1;
    189         } else {
    190             fprintf(stderr, "Could not load python library '%s': %s\n",
    191                     info->dlpi_name, dlerror());
    192         }
    193     }
    194 
    195     return 0;
    196 }
    197 
    198 bool loadPythonGlobally()
    199 {
    200     int success = 0;
    201     dl_iterate_phdr(load_python_globally_callback, &success);
    202     return success;
    203 }
    204 
    205 
    206 #else /* __linux__ */
    207 
    208 bool loadPythonGlobally()
    209 {
    210     /* On non-Linux systems, no need to load globally */
    211     return true;
    212 }
    213 
    214 #endif /* __linux__ */
    215 
    216 }; /* namespace GlobalLibPythonLoader */
    217 
    218 }}}
    219 
    220 This last change makes !PyOtherSide compile with the crippled Bionic C library used on Android.
    221 
    222 '''The resulting project should look like contents of this tarball:'''
    223 
    224 [http://www.modrana.org/platforms/android/pyotherside/guide/pyotherside-1.3.0_android_project.tar.gz http://www.modrana.org/platforms/android/pyotherside/guide/pyotherside-1.3.0_android_project.tar.gz]
    225 
    226 
    227 ==== Building the !PyOtherSide project in !QtCreator ====
    228 
    229 Now the !PyOtherSide project should be ready to be opened in !QtCreator (make sure you are running the one corresponding to the Qt 5.3 Android SDK you have downloaded and installed previously). So start the right !QtCreator and open the ''pyotherside-1.3.0/pyotherside.pro'' file.
    230 
    231 You should be able to build the !PyOtherSide project by pressing the "rebuild" button in !QtCreator.
    232 
    233 === Basic deployment ===
    234 ==== Theory ====
    235 For a Python program using !PyOtherSide to work it needs these things present on the Android device:
    236 * the application files
    237 * Qt 5
    238 * a Python 3 bundle
    239 * the !PyOtherSide library
    240 We don't need to care about Qt - its deployment is handled automatically by the Qt SDK Android support (it is either bundled in the APK, automatically copied to the device or provided by the Ministro library manager) and we don't need to care about it. The SDK also handles the deployment of the application files.
    241 
    242 But we need to get Python 3 & !PyOtherSide on the android device our self. For now we will just copy the files to a fixed location on the device with ''adb'' and hardcode a path pointing to them in the example application. This should be enough for getting a proof of concept of (hopefully) working !PyOtherSide on Android.
    243 
    244 In the future it should be possible deploy these files from a standalone APK without any need for manual intervention or rooting - either by using the new asset mechanism or the old method (used by the !PySide for Android port) of bundling zip files that are unpacked on first start by modified Java boilerplate.
    245 
    246 ==== Deploying Python 3 & !PyOtherSide to the Android device ====
    247 
    248 Download the pre-prepared tarball with Python 3 and !PyOtherSide:
    249 
    250 {{{
    251 wget http://www.modrana.org/platforms/android/pyotherside/guide/python3_pyotherside_deployment_bundle.tar.gz
    252 }}}
    253 
    254 Then unpack it:
    255 
    256 {{{
    257 tar xvf python3_pyotherside_deployment_bundle.tar.gz
    258 }}}
    259 
    260 And push the complete ''poside'' directory directly to the ''/data'' directory on your Android device:
    261 
    262 {{{
    263 adb push poside /data/poside
    264 }}}
    265 
    266 This is where the hello world example will expect it. Also, we can't deploy to the ''/data/sdcard'' directory as it isi usually mounted noexec which breaks the library loading we need to do.
    267 
    268 ==== Deploying & running a hello world application ====
    269 
    270 Next download the hello world project tarball and unpack it:
    271 
    272 {{{
    273 wget http://www.modrana.org/platforms/android/pyotherside/guide/pyotherside_hello_world_project.tar.gz
    274 }}}
    275 
    276 It is just a normal !QtQuick Controls Android project template modified to load the needed libraries from ''/data/poside'' and with all the needed environmental variables set.
    277 
    278 Open the project file in the !QtCreator provided by the Qt for Android SDK
    279 (the main project file is called ''A52.pro''), make sure the android kit with correct architecture for your device is selected, press the ''Run'' button and make sure your device with the deployed libs is selected as the deployment target.
    280 
    281 !QtCreator should now do its thing and the application should successfully start on the device, presenting you a simple ''Hello world'' button in the middle of the screen.
    282 
    283 Now look on the main.qml file in !QtCreator and note that while it has the {{{import io.thp.pyotherside 1.0}}} line, the {{{Python {} }}} line that actually instantiates the Python element provided by !PyOtherSide is commented out.
    284 
    285 Next uncomment {{{Python {} }}} line and try to deploy the application again. This time the deployment fails and the applications crashes as soon as it is started.
    286 
    287 ==== Current status quo ====
    288 
    289 So this how & how far I got - !PyOtherSide can be imported in a !QtQuick application running on Android, but crashes once the Python element is instantiated.
    290 
    291 This guide should enable anyone get as far as I got and hopefully get far past the current roadblock! :)
    292 
    293 Any help with finding out how to fix the crash would be much appreciated! :)