aboutsummaryrefslogtreecommitdiff
path: root/lang/python/python3-package.mk
Commit message (Collapse)AuthorAge
* python-setuptools-rust: Set cargo profile from environment variableJeffery To2023-10-11
| | | | | | | | | | | This adds a patch (submitted upstream in https://github.com/PyO3/setuptools-rust/pull/364), to read the profile to pass to cargo from an environment variable. This also updates the Python include files to set the environment variable based on values from rust-values.mk. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* rust: Consolidate cargo environment variablesJeffery To2023-10-11
| | | | | | | | | | | | | | | | | | | | | | | This consolidates all environment variables for cargo into: * CARGO_HOST_CONFIG_VARS / CARGO_PKG_CONFIG_VARS These contain all cargo-specific environment variables, i.e. without "common" variables like CC. * CARGO_HOST_VARS / CARGO_PKG_VARS (renamed from CARGO_VARS) These contain all environment variables to be passed to cargo. This also: * Set the CARGO_BUILD_TARGET environment variable instead of using the --target command-line option * Update Python include files to use CARGO_HOST_CONFIG_VARS / CARGO_PKG_CONFIG_VARS Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* rust: Install to $(STAGING_DIR)/hostJeffery To2023-10-11
| | | | | | | | | | This allows rustc/cargo/etc to be called without having to set PATH, as $(STAGING_DIR)/host/bin is already in PATH. This also fixes CARGO_HOME not being set during Host/Configure and Host/Compile. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python: Add environment variables to build Rust extensionsJeffery To2023-09-27
| | | | Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Restore platform triplet to pathsJeffery To2023-08-25
| | | | | | | | | | | | | | | | | | This removes 014-remove-platform-so-suffix.patch and 016-adjust-config-paths.patch, restoring the platform triplet to paths for: * C extensions (*.cpython-311-*.so) * Build config data directory (/usr/lib/python3.11/config-3.11-*/) * sysconfig data file (/usr/lib/python3.11/_sysconfigdata_*.py) Setting `_PYTHON_SYSCONFIGDATA_NAME` during package builds ensures that sysconfig data for target Python is loaded, in particular so that C extensions built will have the correct extension / platform triplet. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python-packages: Remove __PYVENV_LAUNCHER__ environment variableJeffery To2023-05-25
| | | | | | | | | | | | | | | | | | | | | Setting __PYVENV_LAUNCHER__ for Python package builds was added in a91a992abb1fe9b7c4e4e5d11532dbf18c5b213d, but neither the commit message nor the pull request[1] explain its purpose in detail. My guess is this was done to set the shebang for installed Python scripts. We now have a Makefile recipe to set the shebang, so it would be unnecessary to set this variable for this purpose. It appears that Python 3.11 has changed the handling of this (internal) environment variable, and setting it appears to be causing build errors for all Python packages. This removes setting __PYVENV_LAUNCHER__ for Python package builds. [1]: https://github.com/openwrt/packages/pull/525 Fixes: https://github.com/openwrt/packages/issues/21162 Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python: Add proper support for pyproject.toml-based buildsJeffery To2023-04-24
| | | | | | | | | | | | | This removes the changes made in 61f202c0170785addbbc449e4de61cc5886f0833 and adds actual support for pyproject.toml-based (PEP 517) builds of Python packages. Packages can force the use of the old build process by setting PYTHON3_PKG_FORCE_DISTUTILS_SETUP:=1; this should only be a temporary workaround until the package can be updated/fixed to use the new build process. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python: Add pyproject.toml-based builds for host Python packagesJeffery To2023-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using pip to install host packages with pyproject.toml-based (PEP 517) builds is problematic: * If build isolation is used, pip will create an isolated build environment, install any build dependencies for the requested package, then build the requested package. It does not appear currently possible to have pip install the build dependencies with hash-checking mode enabled[1]. * If build isolation is not used, any build dependencies must be installed in the build environment before invoking pip to build the requested package[2]. This would require creating a package dependency resolution system to install build dependencies, and any dependencies of dependencies, in the correct order. * It is very difficult to patch the packages installed by pip. This adds a new include file (python3-host-build.mk) with recipes to install host Python packages with pyproject.toml-based builds. This is backwards-compatible with packages that require running setup.py. Besides addressing the above issues (the OpenWrt build system already resolves dependencies between packages, checks all source downloads against known hashes, and supports patching packages), host packages also: * Capture package licensing and maintainer information * Enable uscan checking for package updates/CVEs * Are a known concept for OpenWrt packagers/developers The existing functionality of using host pip to install packages will remain for now, but should be considered deprecated and expected to be removed in the future. This also updates Py3Build/CheckHostPipVersionMatch for the case where the host-pip-requirements directory does not exist or is empty. [1]: https://pip.pypa.io/en/stable/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020 [2]: https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-no-build-isolation Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python: Unset Python environment variablesJeffery To2023-03-30
| | | | | | | | | | | This will prevent the user's environment variables from affecting host Python, removing the need to manually override these variables. It is also not necessary to set PYTHONPATH (when not working on target Python packages) because the given directories are already included in Python's search path by default. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python-build: add support for pyproject.toml filesAlexandru Ardelean2023-01-22
| | | | | | | | | | | | | | | | | | A new PEP 517 (https://www.python.org/dev/peps/pep-0517/) has defined that Python packages can be shipped without any `setup.py` file, and that a `pyproject.toml` file is sufficient. A `setup.py` shim layer is suggested as a method for running the build. For these cases, we will add a support in the OpenWrt build-system to provide the default `setup.py` shim layer in case this file does not exist, but there is a `pyproject.toml` file. We also seem to need to tweak the shim layer with the PKG_VERSION, otherwise the detected version is 0.0.0. We will need to see if this will be fixed later in setuptools{-scm}. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
* python3: Add Py3Build/InstallBuildDepends recipeJeffery To2022-03-17
| | | | | | | | | | | This adds a recipe, Py3Build/InstallBuildDepends, that installs the requirements listed in HOST_PYTHON3_PACKAGE_BUILD_DEPENDS. This allows other (non-Python) packages to install host Python packages by calling this recipe, without having to know the internals of python3-package.mk. This also updates apparmor to call this recipe. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3-package.mk: fix syntax error in FindStdlibDependsMichal Vasilek2021-09-30
| | | | | | | | When running FindStdlib and running DependsCheckHostPipVersionMatch at the same time, both commands were joined together resulting in a syntax error. Signed-off-by: Michal Vasilek <michal.vasilek@nic.cz>
* python3-package.mk: add check for dependencies in host-pip-requirementsAlexandru Ardelean2021-08-30
| | | | | | | | | | | | | | | | | | | | It often happens that we update a package to a new version (e.g. cffi) to a newer version, but we forget to update the version for cffi in the `lang/python/host-pip-requirements/cffi.txt` file. This check adds a minimal check, so that when a build occurs for a Python package, if there is a mention/listing of this package in `lang/python/host-pip-requirements/` it will check that the versions match. This way, when we update a package, we get a build failure and update the host version as well. This will omit packages (like Cython) that are not packaged for OpenWrt, but are host-side dependencies only. But until we find some mechanism to check for those, we will probably only notice to update them when another build occurs (at the very least). Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
* python3: Use hash-checking mode when installing host pip packagesJeffery To2020-08-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | In hash-checking mode[1], pip will verify downloaded package archives (source tarballs in our case) against known SHA256 hashes before installing the packages. As a consequence, this requires the use of requirements files[2] and pinning packages to known versions. The syntax for package Makefiles has changed slightly; HOST_PYTHON3_PACKAGE_BUILD_DEPENDS no longer accepts requirement specifiers like "foo>=1.0", only requirements file names (which are the same as package names in the most common case). This also updates affected packages, in particular: * python-zipp: "setuptools_scm[toml]" has been split into "setuptools-scm toml" to reuse the requirements file for setuptools-scm (the extra depends installed by "setuptools_scm[toml]" is toml). * python-pycparser: This previously used ply 3.10, whereas the requirements file will now install 3.11. [1]: https://pip.pypa.io/en/stable/reference/pip_install/#hash-checking-mode [2]: https://pip.pypa.io/en/stable/user_guide/#requirements-files Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Use default _PYTHON_HOST_PLATFORMJeffery To2020-07-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This lets the Python build process set _PYTHON_HOST_PLATFORM instead of forcing an explicit value. Also: * Save the target _PYTHON_HOST_PLATFORM value during Build/InstallDev for use when building target Python packages (in python3-package.mk). * Use the (mostly) default PYTHON_FOR_BUILD value, instead patch configure to remove the platform triplet from the sysconfigdata file name. * Remove the "CROSS_COMPILE=yes" make variable (there is no indication that this variable is necessary). * Force host pip to build packages from source instead of downloading binary wheels. Previously, host pip can download universal (platform-independent) wheels but not platform-specific wheels, because of the custom _PYTHON_HOST_PLATFORM value. (Packages that do not have universal wheels would be compiled from source.) With a correct _PYTHON_HOST_PLATFORM, host pip can install platform-specific wheels as well. However, the pre-built shared object (.so) files in these wheels will have the host's platform triplet in their file names. When target Python packages are built (using the target's _PYTHON_HOST_PLATFORM), Python will not use these shared object files. By forcing host pip to build packages from source, the built shared object files will not have the platform triplet in their file names. (Host Python has been patched to remove the platform triplet from file names.) This allows these packages to be used when building target Python packages. (The net effect of this complete change is that platform-dependent packages will continue to be compiled from source, while platform-independent packages will now also be compiled from source.) Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Add script to help find standard library dependenciesJeffery To2020-05-14
| | | | | | | | | | | | | | | This adds a script that searches a Python package's source code to find imports for separately-packaged standard library modules. The script can be run by calling make with the configure target and "PY3=stdlib V=s" arguments, e.g. make package/python3-lxml/configure PY3=stdlib V=s This also updates the readme on how to call this script, as well as more information on Python package dependencies in general. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Use PYTHON3_PKG_BUILD to control default Python package buildJeffery To2020-04-19
| | | | | | | | | | | | | | This replaces the use of BUILD_VARIANT with PYTHON3_PKG_BUILD to opt in/out of the default Python package build recipe (Py3Build/Compile). PYTHON3_PKG_BUILD defaults to true (1), i.e. if a package includes python3-package.mk, then by default it will set the package's Build/Compile to Py3Build/Compile. If PYTHON3_PKG_BUILD is set to 0 before python3-package.mk is included, then Build/Compile will not be modified. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Remove MIPS16 changes from python3-package.mkJeffery To2020-04-19
| | | | | | | There are no bug reports or other evidence to suggest Python is not compatible with MIPS16 compilation. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Minor edits for python3-package.mkJeffery To2020-04-19
| | | | | | | | * Remove PYTHON3_BIN_DIR, it isn't used anywhere in the repo * Rephrase *-src package description * Reduce Py3Package/$(1)/install indentation Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Add canned recipe to invoke filespec processingJeffery To2020-04-19
| | | | | | | | | | | | This extracts filespec export and processing into Py3Package/ProcessFilespec. This also allows the filespec variable to be explicitly set to an empty value, to bypass filespec processing. (The default filespec is also available as Py3Package/filespec/Default to be explicitly assigned to the filespec variable.) Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Move functionality into python3-package.mkJeffery To2020-04-19
| | | | | | | | This moves functionality from python-package-install.sh into python3-package.mk, so that they can be reused separate from filespec processing. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Reorder recipes in python3-package.mkJeffery To2020-04-19
| | | | | | | | | Group Python3/* recipes together, group Py3Package and Py3Build together. This also adds headings and whitespace to separate major sections. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Rename canned recipes in python3-package.mkJeffery To2020-04-19
| | | | | | | This renames "internal" recipes to use the Python3/ prefix and clarifies the names (RunTarget to Run, Mod to ModSetup, Shebang to FixShebang). Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Rename canned recipes in python3-host.mkJeffery To2020-04-19
| | | | | | | | This changes the recipe name prefix from Build/Compile/HostPy3 to HostPython3, and clarifies some of the names (RunHost to Run, Mod to ModSetup). Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: Remove HostPython3 in python3-host.mkJeffery To2020-04-19
| | | | | | | | | | | | | | | HostPython3 only adds a few environment variables before running host Python. It has only two users, Build/Compile/HostPy3RunHost and Build/Compile/HostPy3RunTarget. HostPython3 also accesses $(PYTHON3PATH), even though python3-host.mk does not include python3-package.mk, where the variable is defined. This removes HostPython3 and has its two users run host Python directly. This also combines the environment variables of HostPython3 and the two users into HOST_PYTHON3_VARS and PYTHON3_VARS. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python3: hide -src package until main pkg selectedEneas U de Queiroz2020-04-10
| | | | | | | | | | | | This adds a 'Package/<pkg>-src/config' definition with a 'depends on <pkg>' line, which will hide <pkg>-src unless <pkg> is selected. This makes the long list of python packages a bit shorter, and also indents the src package: <M> python3-base................................ Python 3.8 interpreter < > python3-base-src................. Python 3.8 interpreter (sources) Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
* python3-package.mk: fix typo PYTHON3_PKG_SETUP_GLOABL_ARGS -> ↵Alexandru Ardelean2020-04-07
| | | | | | | | | | | | PYTHON3_PKG_SETUP_GLOBAL_ARGS This fixes a typo with the default PYTHON3_PKG_SETUP_GLOBAL_ARGS. Since in make context non-defined variables are empty anyway, this doesn't produce any issues. The fix is more semantic in nature. Fixes https://github.com/openwrt/packages/issues/11790 Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
* python,python3: add Py[3]Shebang functions & move outside of scriptAlexandru Ardelean2019-09-20
| | | | | | | | | | | | | | | | Some packages just install some Python binaries, that may need their shebang fixed. This change adds some utilities to help with that and try to centralize the sed rules a bit. It also removes the logic from the `python-package-install.sh` into the `python-package[3].mk` files. This does 2 things: 1. It minimizes the need for the shell script to know the Python version 2/3 2. Makes the logic re-usable in packages; especially if the install rules differ a bit Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
* django: move django* packages under django submenuEneas U de Queiroz2019-08-12
| | | | | | | | | | | This changes the python[3]-django dependencies in packages to be non-selecting, and adds an MDEPENDS line so that the *-src packages get placed inside the django menu as well. Added MENU:= to the src-package definitions in python[3]-package.mk, so it does not import that setting from the binary package. Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
* Merge pull request #9100 from jefferyto/isolate-host-pythonRosen Penev2019-06-03
|\ | | | | python,python3: Better isolate host Python
| * python,python3: Update host pip[3] install functionsJeffery To2019-05-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add --cache-dir option to set the pip cache to a directory in $(DL_DIR), instead of pip's default (build user's ~/.cache/pip), fixes #9066 * Add --disable-pip-version-check option, since the version check only prints a message saying a new version is available * Combine host_python_pip_install and host_python_pip_install_host into Build/Compile/HostPy[3]PipInstall * Remove --root and --prefix options, since this function is only used to install packages to host Python's default site-packages directory (setting these may serve to confuse pip) * Pass all of $(HOST_PYTHON[3]_PACKAGE_BUILD_DEPENDS) to the function, since pip can handle multiple arguments/packages Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* | python,python3: Clear more fields for src packagesJeffery To2019-05-28
|/ | | | | | | This clears the CONFLICTS, PROVIDES, EXTRA_DEPENDS, and USERID fields for -src packages. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python,python3: Add PYTHON[3]_PKG_SETUP_GLOBAL_ARGSJeffery To2019-04-05
| | | | | | | | | | | | Some packages (PyYAML, https://github.com/openwrt/packages/pull/8482#discussion_r270692276) recognize "global" options to setup.py; these must appear before the "install" command on the command line. This adds PYTHON[3]_PKG_SETUP_GLOBAL_ARGS, which let packages set these global options. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python,python3: Fix overridden usr/bin symlinksJeffery To2019-04-03
| | | | | | | | | | | | | Currently, all files in usr/bin (presumably all Python scripts) are run through sed to replace the shebang; sed will overwrite the file whether or not a match is found. This causes symlinks to be overridden and made into copies of their targets. python[3]-base and python[3]-dev are affected by this. This adds the --follow-symlinks flag to sed, in addition to using $(SED), so that symlinks are not overridden. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python,python3: allow users to override python args & varsAlexandru Ardelean2019-02-23
| | | | | | | | If users want to define these before including python[3]-package.mk, these vars will be overridden during the include. So, override these vars if they haven't been defined. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
* python,python3: Add PYTHON_PKG_SETUP_DIRJeffery To2019-02-23
| | | | | | | | | This adds a variable (PYTHON_PKG_SETUP_DIR / PYTHON3_PKG_SETUP_DIR) that allows a Python package Makefile to control the directory where setup.py is called (as part of PyBuild/Compile/Default / Py3Build/Compile/Default). Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python,python3: Fix calling default PyPackage/installJeffery To2019-02-18
| | | | | | | | | This fixes Package/*/install to call PyPackage/*/install correctly. Previously, if a package used the default PyPackage/*/install, then it would not called. (A custom-defined PyPackage/*/install would be called with no issue.) Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* python,python3: move .exe removal in `python-package-install.sh` scriptAlexandru Ardelean2019-02-04
| | | | | | | It's a common operation for both Python & Python3, so move it to the script `python-package-install.sh` script. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
* python,python3: fix recursive deps caused by dangling DEPENDSAlexandru Ardelean2018-01-17
| | | | | | | For python `src` packages we should clear out the DEPENDS to prevent recursive deps from happening. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
* Merge pull request #5457 from jefferyto/python-pkg-setup-args-varsHannu Nyman2018-01-17
|\ | | | | python,python3: add vars to customize setup arguments / variables
| * python,python3: add vars to customize setup arguments / variablesJeffery To2018-01-16
| | | | | | | | | | | | | | | | | | | | | | | | | | This adds: * PYTHON_PKG_SETUP_ARGS * PYTHON_PKG_SETUP_VARS * PYTHON3_PKG_SETUP_ARGS * PYTHON3_PKG_SETUP_VARS to customize Python package setup arguments / environment variables. Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* | python,python3: disable dependencies between python src packagesAlexandru Ardelean2018-01-16
|/ | | | | | | Related to: https://github.com/openwrt/packages/issues/5424 Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
* python,python3: merge package install scriptsAlexandru Ardelean2018-01-10
| | | | | | | | | | | | | The only difference just a parameter for Python3 [ -b to compile bytecodes in legacy mode ]. No need to keep 2 almost identical files now that they're exported. I'm a bit scared of that param, since it may get removed at some point. But let's see until then. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
* python,python3: simplify path to install shell-scriptAlexandru Ardelean2018-01-10
| | | | | | | Now that all files are exported, it makes sense to just reference the script directly. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
* python,python3: export mk files outside of python package dirsAlexandru Ardelean2018-01-10
Since `lang/python` is it's own folder of Python packages (for both Python 2 & 3), and these build rules are needed in a lot of packages [especially Python packages], putting them here makes sense architecturally, to be shared. This also helps get rid of the `include_mk` construct which relies on OpenWrt core to provide, and seems like a broken design idea that has persisted for a while. Reason is: it requires that Python 2/3 be built to provide these mk files for other Python packages, which seems like a bad idea. Long-term, there could be an issue where some other feeds would require these mk files [e.g. telephony] for some Python packages. We'll see how we handle this a bit later. For now we limit this to this feed. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Powered by cgit, Themed by Ted Yin.