Show HN: Morgan – PyPI Mirror for Restricted/Offline Environments Mirroring PyPI packages for environments/networks that do not have access to the Internet is hard. It's actually hard even in environments that do have access to the Internet. Most solutions out there either: 1. Depend on pip to download and cache package distributions. This means those downloads will probably only work in a similar environment (same Python interpreter, same libc), because of the nature of binary package distributions and the fact that packages have optional dependencies for different environments. 2. Depend on other PyPI packages, meaning installing the mirror in a restricted environment in itself is too difficult. 3. Cannot resolve dependencies of dependencies, meaning mirroring PyPI partially is extremely difficult, and PyPI is huge. Morgan works differently. It creates a mirror based on a configuration file that defines target environments (using Python's standard Environment Markers specification from PEP 345) and a list of package requirement strings (e.g. "requests>=2.24.0"). It downloads all files relevant to the target environments from PyPI (both source and binary distributions), and recursively resolves and downloads their dependencies, again based on the target environments. It then extracts a single-file server to the mirror directory that works with Python 3.7+, has no outside dependencies, and implements the standard Simple API. This directory can be copied to the restricted network, through whatever security policies are in place, and deployed easily with a simple `python server.py` command. I should note that Morgan can find dependencies from various metadata sources inside package distributions, including standard METADATA/PKG-INFO/pyproject.toml files, and non-standard files such as setuptools' requires.txt. There's more information in the Git repository. If this is interesting to you, I'll be happy to receive your feedback. Thanks! https://ift.tt/i7lKs3Q September 23, 2022 at 05:39AM
Comments
Post a Comment