NDK BUILD SYSTEM IMPLEMENTATION Copyright (C) 2023-2024 Free Software Foundation, Inc. See the end of the file for license conditions. Emacs implements ndk-build itself, because the version that comes with the Android NDK is not easy to use from another Makefile, and keeps accumulating incompatible changes. The Emacs implementation of ndk-build consists of one m4 file: m4/ndk-build.m4 four Makefiles in build-aux, run during configure: build-aux/ndk-build-helper-1.mk build-aux/ndk-build-helper-2.mk build-aux/ndk-build-helper-3.mk build-aux/ndk-build-helper.mk one awk script in build-awx, run during configure: build-aux/ndk-module-extract.awk seven Makefiles in cross/ndk-build, cross/ndk-build/ndk-build-shared-library.mk cross/ndk-build/ndk-build-static-library.mk cross/ndk-build/ndk-build-executable.mk cross/ndk-build/ndk-clear-vars.mk cross/ndk-build/ndk-prebuilt-shared-library.mk cross/ndk-build/ndk-prebuilt-static-library.mk cross/ndk-build/ndk-resolve.mk and finally, two more Makefiles in cross/ndk-build, generated by configure: cross/ndk-build/Makefile (generated from cross/ndk-build/Makefile.in) cross/ndk-build/ndk-build.mk (generated from cross/ndk-build/ndk-build.mk.in) m4/ndk-build.m4 is a collection of macros which are used by the configure script to set up the ndk-build system, look for modules, add the appropriate options to LIBS and CFLAGS, and generate the Makefiles necessary to build the rest of Emacs. Immediately after determining the list of directories in which to look for ``Android.mk'' files, the version and type of Android system being built for, configure calls: ndk_INIT([$android_abi], [$ANDROID_SDK], [cross/ndk-build]) This expands to a sequence of shell script that enumerates all of the Android.mk files specified in "$with_ndk_path", sets up some shell functions used by the rest of the ndk-build code run by the configure script, and teaches the ndk-build system that the Makefiles to be generated are found in the directory "cross/ndk-build/Makefile". When configure is cross-compiling for Android, the macro EMACS_CHECK_MODULES will expand to the macro ndk_CHECK_MODULES, instead of pkg-config.m4's PKG_CHECK_MODULES. Thus, the following code: EMACS_CHECK_MODULES([PNG], [libpng >= 1.0.0]) will actually expand to: ndk_CHECK_MODULES([PNG], [libpng >= 1.0.0], [HAVE_PNG=yes], [HAVE_PNG=no]) which in turn expands to a sequence shell script that first invokes: make -f build-aux/ndk-build-helper.mk for each ``Android.mk'' file found by ndk_INIT, with the following variables given to Make: EMACS_SRCDIR=. # the source directory (in which configure is running) BUILD_AUXDIR=$ndk_AUX_DIR # the build-aux directory EMACS_ABI=$ndk_ABI # this is the $android_abi given to ndk_INIT ANDROID_MAKEFILE="/opt/android/libpng/Android.mk" ANDROID_MODULE_DIRECTORY="/opt/android/libpng" NDK_BUILD_DIR="$ndk_DIR" # this is the directory given as to ndk_INIT build-aux/ndk-build-helper.mk will then evaluate the contents $(ANDROID_MAKEFILE), the ``Android.mk'' file, for the first time. The purpose of this evaluation is to establish a list of packages (or modules) provided by the ``Android.mk'' file, and the corresponding Makefile targets and compiler and linker flags required to build and link to those targets. Before doing so, build-aux/ndk-build-helper.mk will define several variables and functions required by all ``Android.mk'' files. The most important of these are: my-dir # the directory containing the Android.mk file. BUILD_SHARED_LIBRARY # build-aux/ndk-build-helper-1.mk BUILD_STATIC_LIBRARY # build-aux/ndk-build-helper-2.mk BUILD_EXECUTABLE # build-aux/ndk-build-helper-3.mk CLEAR_VARS # build-aux/ndk-build-helper-4.mk Then, ``Android.mk'' will include $(CLEAN_VARS), possibly other ``Android.mk'' files, (to clear variables previously set), set several variables describing each module to the ndk-build system, and include one of $(BUILD_SHARED_LIBRARY), $(BUILD_STATIC_LIBRARY) and $(BUILD_EXECUTABLE). Each one of those three scripts will then read from the variables set by ``Android.mk'', resolve dependencies, and print out some text describing the module to Emacs. For example, the shared library module "libpng" results in the following text being printed: Building shared libpng /opt/android/libpng/png.c /opt/android/libpng/pngerror.c /opt/android/libpng/pngget.c /opt/android/libpng/pngmem.c /opt/android/libpng/pngpread.c /opt/android/libpng/pngread.c /opt/android/libpng/pngrio.c /opt/android/libpng/pngrtran.c /opt/android/libpng/pngrutil.c /opt/android/libpng/pngset.c /opt/android/libpng/pngtrans.c /opt/android/libpng/pngwio.c /opt/android/libpng/pngwrite.c /opt/android/libpng/pngwtran.c /opt/android/libpng/pngwutil.c -I/opt/android/libpng -L/opt/emacs/cross/ndk-build -l:libpng_emacs.so libpng_emacs.so End The output is arranged as follows: - The first line consists of the word ``Building'', followed by either ``shared'', ``static'', or ``executable'', depending on what type of module being built. - The second line consists of the name of the module currently being built. - The third line consists of all of the source code files comprising the module. - The fourth line consists of the text that has to be added to CFLAGS in order to find the includes associated with the module. - The fifth line consists of the text that has to be added to LIBS in order to link with this module and all of its dependencies. - The sixth line consists of the Make targets (more on this later) that will build the final shared object or library archive of this module, along with all of its dependencies. - The seventh line is either empty, or the name of a dependency on the C++ standard library. This is used to determine whether or not Emacs will include the C++ standard library in the application package. The output from Make is given to an awk script, build-aux/ndk-module-extract.awk. This is responsible for parsing the that output and filtering out modules other than what is being built: awk -f build-aux/ndk-module-extract.awk MODULE=libpng eventually generating this section of shell script: module_name=libpng module_kind=shared module_src="/opt/android/libpng/png.c /opt/android/libpng/pngerror.c /opt/android/libpng/pngget.c /opt/android/libpng/pngmem.c /opt/android/libpng/pngpread.c /opt/android/libpng/pngread.c /opt/android/libpng/pngrio.c /opt/android/libpng/pngrtran.c /opt/android/libpng/pngrutil.c /opt/android/libpng/pngset.c /opt/android/libpng/pngtrans.c /opt/android/libpng/pngwio.c /opt/android/libpng/pngwrite.c /opt/android/libpng/pngwtran.c /opt/android/libpng/pngwutil.c" module_includes="-I/opt/android/libpng" module_cflags="" module_ldflags=" -L/opt/emacs/cross/ndk-build -l:libpng_emacs.so" module_target="libpng_emacs.so" module_cxx_deps="" module_imports="" which is then evaluated by `configure'. Once the variable `module_name' is set, configure appends the remaining $(module_includes), $(module_cflags) and $(module_ldflags) to the module's CFLAGS and LIBS variables, and appends the list of Makefile targets specified to the variable NDK_BUILD_MODULES. In some cases, an ``Android.mk'' file may chose to import a module defined in ``--with-ndk-path'', but not defined inside its own ``Android.mk'' file. build-aux/ndk-build-helper.mk defines the `import-module' function to add the modules being imported to a variable, which is then printed out after ``ndk-build-helper.mk'' completes. For example, libxml2 imports the ``libicucc'' module, which results in the following text being printed: Building shared libxml2 /home/oldosfan/libxml2/SAX.c /home/oldosfan/libxml2/entities.c /home/oldosfan/libxml2/encoding.c /home/oldosfan/libxml2/error.c /home/oldosfan/libxml2/parserInternals.c /home/oldosfan/libxml2/parser.c /home/oldosfan/libxml2/tree.c /home/oldosfan/libxml2/hash.c /home/oldosfan/libxml2/list.c /home/oldosfan/libxml2/xmlIO.c /home/oldosfan/libxml2/xmlmemory.c /home/oldosfan/libxml2/uri.c /home/oldosfan/libxml2/valid.c /home/oldosfan/libxml2/xlink.c /home/oldosfan/libxml2/debugXML.c /home/oldosfan/libxml2/xpath.c /home/oldosfan/libxml2/xpointer.c /home/oldosfan/libxml2/xinclude.c /home/oldosfan/libxml2/DOCBparser.c /home/oldosfan/libxml2/catalog.c /home/oldosfan/libxml2/globals.c /home/oldosfan/libxml2/threads.c /home/oldosfan/libxml2/c14n.c /home/oldosfan/libxml2/xmlstring.c /home/oldosfan/libxml2/buf.c /home/oldosfan/libxml2/xmlregexp.c /home/oldosfan/libxml2/xmlschemas.c /home/oldosfan/libxml2/xmlschemastypes.c /home/oldosfan/libxml2/xmlunicode.c /home/oldosfan/libxml2/xmlreader.c /home/oldosfan/libxml2/relaxng.c /home/oldosfan/libxml2/dict.c /home/oldosfan/libxml2/SAX2.c /home/oldosfan/libxml2/xmlwriter.c /home/oldosfan/libxml2/legacy.c /home/oldosfan/libxml2/chvalid.c /home/oldosfan/libxml2/pattern.c /home/oldosfan/libxml2/xmlsave.c /home/oldosfan/libxml2/xmlmodule.c /home/oldosfan/libxml2/schematron.c /home/oldosfan/libxml2/SAX.c /home/oldosfan/libxml2/entities.c /home/oldosfan/libxml2/encoding.c /home/oldosfan/libxml2/error.c /home/oldosfan/libxml2/parserInternals.c /home/oldosfan/libxml2/parser.c /home/oldosfan/libxml2/tree.c /home/oldosfan/libxml2/hash.c /home/oldosfan/libxml2/list.c /home/oldosfan/libxml2/xmlIO.c /home/oldosfan/libxml2/xmlmemory.c /home/oldosfan/libxml2/uri.c /home/oldosfan/libxml2/valid.c /home/oldosfan/libxml2/xlink.c /home/oldosfan/libxml2/debugXML.c /home/oldosfan/libxml2/xpath.c /home/oldosfan/libxml2/xpointer.c /home/oldosfan/libxml2/xinclude.c /home/oldosfan/libxml2/DOCBparser.c /home/oldosfan/libxml2/catalog.c /home/oldosfan/libxml2/globals.c /home/oldosfan/libxml2/threads.c /home/oldosfan/libxml2/c14n.c /home/oldosfan/libxml2/xmlstring.c /home/oldosfan/libxml2/buf.c /home/oldosfan/libxml2/xmlregexp.c /home/oldosfan/libxml2/xmlschemas.c /home/oldosfan/libxml2/xmlschemastypes.c /home/oldosfan/libxml2/xmlunicode.c /home/oldosfan/libxml2/xmlreader.c /home/oldosfan/libxml2/relaxng.c /home/oldosfan/libxml2/dict.c /home/oldosfan/libxml2/SAX2.c /home/oldosfan/libxml2/xmlwriter.c /home/oldosfan/libxml2/legacy.c /home/oldosfan/libxml2/chvalid.c /home/oldosfan/libxml2/pattern.c /home/oldosfan/libxml2/xmlsave.c /home/oldosfan/libxml2/xmlmodule.c /home/oldosfan/libxml2/schematron.c -L/home/oldosfan/emacs-dev/emacs-android/cross/ndk-build -l:libxml2_emacs.so -l:libicuuc_emacs.so libxml2_emacs.so libicuuc_emacs.so End Start Imports libicuuc End Imports Upon encountering the ``Start Imports'' section, build-aux/ndk-module-extract.awk collects all imports until it encounters the line ``End Imports'', at which point it prints: module_imports="libicuuc" Then, if the list of imports is not empty, ndk_CHECK_MODULES additionally calls itself for each import before appending the module's own ``Android.mk'', ensuring that the module's imported dependencies are included by $ndk_DIR/Makefile before itself. Finally, immediately before generating src/Makefile.android, configure expands: ndk_CONFIG_FILES to generate $ndk_DIR/Makefile and $ndk_DIR/ndk-build.mk. Now, the $ndk_DIR directory is set up to build all modules upon which depends, and $ndk_DIR/ndk-build.mk includes a list of files required to link Emacs, along with the rules to chdir into $ndk_DIR in order to build them. $ndk_DIR/ndk-build.mk is included by cross/src/Makefile (Makefile.android) and java/Makefile. It defines three different variables: NDK_BUILD_MODULES the file names of all modules to be built. NDK_BUILD_STATIC absolute names of all library archives to be built. NDK_BUILD_SHARED absolute names of all shared libraries to be built. and then proceeds to define rules to build each of the modules in $(NDK_BUILD_MODULES). cross/src/Makefile arranges to have all dependencies of Emacs not already built built before linking ``libemacs.so'' with them. java/Makefile additionally arranges to have all shared object dependencies built before the application package is built, which is normally redundant because they should have already been built before linking ``libemacs.so''. Building the modules is performed through $ndk_DIR/Makefile, which contains the actual implementation of the ``ndk-build'' build system. First, it defines certain variables constant within the ``ndk-build'' build system, such as the files included by ``Android.mk'' to build shared or static libraries, and CLEAR_VARS. The most important of these are: CLEAR_VARS cross/ndk-build/ndk-clear-vars.mk BUILD_EXECUTABLE cross/ndk-build/ndk-build-executable.mk BUILD_SHARED_LIBRARY cross/ndk-build/ndk-build-shared-library.mk BUILD_STATIC_LIBRARY cross/ndk-build/ndk-build-static-library.mk PREBUILT_SHARED_LIBRARY cross/ndk-build/ndk-prebuilt-shared-library.mk PREBUILT_STATIC_LIBRARY cross/ndk-build/ndk-prebuilt-static-library.mk Then, it loads each Emacs dependency's ``Android.mk'' file. For each module defined there, ``Android.mk'' includes $(CLEAR_VARS) to unset all variables specific to each module, and then includes $(BUILD_SHARED_LIBRARY) or $(BUILD_STATIC_LIBRARY) for each shared or static library module. This results in cross/ndk-build/ndk-build-shared-library.mk or cross/ndk-build/ndk-build-static-library being included, just like the Makefiles in build-aux were inside the configure script. Each one of those two scripts then defines rules to build all of the object files associated with the module, and then link or archive them. The name under which the module is linked is the same as the Make target found on the sixth line of output from build-aux/ndk-build-helper.mk. In doing so, they both include the file ndk-resolve.mk. ndk-resolve.mk is expected to recursively add all of the exported CFLAGS and includes of any dependencies to the compiler and linker command lines for the module being built. When building a shared library module, ndk-resolve.mk is also expected to define the variables NDK_LOCAL_A_NAMES_$(LOCAL_MODULE) and NDK_WHOLE_A_NAMES_$(LOCAL_MODULE), containing all static library dependencies' archive files. They are to be linked in to the resulting shared object file. This is done by including cross/ndk-build/ndk-resolve.mk each time a shared or static library module is going to be built. How is this done? First, ndk-resolve.mk saves the LOCAL_PATH, LOCAL_STATIC_LIBRARIES, LOCAL_SHARED_LIBRARIES, LOCAL_EXPORT_CFLAGS and LOCAL_EXPORT_C_INCLUDES from the module. Next, ndk-resolve loops through the dependencies the module has specified, appending its CFLAGS and includes to the command line for the current module. Then, that process is repeated for each such dependency which has not already been resolved, until all dependencies have been resolved. libpng is a very simple module, providing only a single shared object module. This module is named libpng_emacs.so and is eventually built and packaged into the library directory of the Emacs application package. Now, let us look at a more complex module, libwebp: When built with libwebp, Emacs depends on a single library, libwebpdemux. This library is named ``libwebpdemux'' on Unix systems, and that is the name by which it is found with pkg-config. However, the library's module is only named ``webpdemux'' on Android. When ndk_CHECK_MODULES begins to look for a module, it first tries to see if its name is found in the variable `ndk_package_map', which was set inside ndk_INIT. In this case, it finds the following word: libwebpdemux:webpdemux and immediately replaces ``libwebpdemux'' with ``webpdemux''. Then, it locates the ``Android.mk'' file containing a static library module named webpdemux and gives the output from build-aux/ndk-build-helper.mk to the awk script, resulting in: module_name=webpdemux module_kind=static module_src="/opt/android/webp/src/demux/anim_decode.c /opt/android/webp/src/demux/demux.c" module_includes="-I/opt/android/webp/src" module_cflags="" module_ldflags=" cross/ndk-build/libwebpdemux.a cross/ndk-build/libwebp.a cross/ndk-build/libwebpdecoder_static.a " module_target="libwebpdemux.a libwebp.a libwebpdecoder_static.a" The attentive reader will notice that in addition to the ``libwebpdemux.a'' archive associated with the ``webpdemux'' library, Emacs has been made to link with two additional libraries. This is because the ``webpdemux'' module specifies a dependency on the ``webp'' module (defined in the same Android.mk). build-aux/ndk-build-helper.mk resolved that dependency, noticing that it in turn specified another dependency on ``webpdecoder_static'', which in turn was added to the linker command line and list of targets to build. As a result, all three dependencies will be built and linked to Emacs, instead of just the single ``webpdemux'' dependency that was specified. This file is part of GNU Emacs. GNU Emacs is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. GNU Emacs is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with GNU Emacs. If not, see .