This is a hotfix release that fixes --venv
mode shebangs being too long for some Linux
environments.
- Guard against too long
--venv
mode shebangs. (#1254) PR #1254
This release primarily hardens Pex venvs fixing several bugs.
- Fix Pex isolation. (#1250) PR #1250
- Support pre-compiling a venv. (#1246) PR #1246
- Support venv relocation. (#1247) PR #1247
- Fix --runtime-pex-root leak in pex bootstrap. (#1244) PR #1244
- Support venvs that can outlive their base python. (#1245) PR #1245
- Harden Pex interpreter identification. (#1248) PR #1248
- The pex venv script handles entrypoints like PEX. (#1242) PR #1242
- Ensure PEX files aren't symlinked in venv. (#1240) PR #1240
- Fix venv pex script for use with multiprocessing. (#1238) PR #1238
This release fixes another bug in --venv mode when PEX_PATH is exported in the environment.
- Fix --venv mode to respect PEX_PATH. (#1227) PR #1227
This release fixes bugs in --unzip and --venv mode PEX file execution and upgrades to the last release of Pip to support Python 2.7.
- Fix PyPy3 --venv mode. (#1221) PR #1221
- Make PexInfo.pex_hash calculation more robust. (#1219) PR #1219
- Upgrade to Pip 20.3.4 patched. (#1205) PR #1205
This is another hotfix release to fix incorrect resolve post-processing failing otherwise correct resolves.
- Pex resolver fails to evaluate markers when post-processing resolves to identify which dists satisfy direct requirements. (#1196) PR #1196 <https://github.com/pantsbuild/pex/pull/1196>_
This is another hotfix release to fix a regression in Pex --sources-directory
handling of
relative paths.
- Support relative paths in Chroot.symlink. (#1194) PR #1194 <https://github.com/pantsbuild/pex/pull/1194>_
This is a hotfix release that fixes requirement parsing when there is a local file in the CWD with the same name as the project name of a remote requirement to be resolved.
- Requirement parsing handles local non-dist files. (#1190) PR #1190
This release brings support for a --venv
execution mode to complement --unzip
and standard
unadorned PEX zip file execution modes. The --venv
execution mode will first install the PEX
file into a virtual environment under ${PEX_ROOT}/venvs
and then re-execute itself from there.
This mode of execution allows you to ship your PEXed application as a single zipfile that
automatically installs itself in a venv and runs from there to eliminate all PEX startup overhead
on subsequent runs and work like a "normal" application.
There is also support for a new resolution mode when building PEX files that allows you to use the
results of a previous resolve by specifying it as a -pex-repository
to resolve from. If you have
many applications sharing a requirements.txt / constraints.txt, this can drastically speed up
resolves.
- Improve PEX repository error for local projects. (#1184) PR #1184
- Use symlinks to add dists in the Pex CLI. (#1185) PR #1185
- Suppress
pip debug
warning. (#1183) PR #1183 - Support resolving from a PEX file repository. (#1182) PR #1182
- PEXEnvironment for a DistributionTarget. (#1178) PR #1178
- Fix plumbing of 2020-resolver to Pip. (#1180) PR #1180
- Platform can report supported_tags. (#1177) PR #1177
- Record original requirements in PEX-INFO. (#1171) PR #1171
- Tighten requirements parsing. (#1170) PR #1170
- Type BuildAndInstallRequest. (#1169) PR #1169
- Type AtomicDirectory. (#1168) PR #1168
- Type SpawnedJob. (#1167) PR #1167
- Refresh and type OrderedSet. (#1166) PR #1166
- PEXEnvironment recursive runtime resolve. (#1165) PR #1165
- Add support for -r / --constraints URL to the CLI. (#1163) PR #1163
- Surface Pip dependency conflict information. (#1162) Issue #9420 PR #1162
- Add support for parsing extras and specifiers. (#1161) PR #1161
- Support project_name_and_version metadata. (#1160) PR #1160
- docs: fix simple typo, orignal -> original (#1156) PR #1156
- Support a --venv mode similar to --unzip mode. (#1153) PR #1153
- Remove redundant dep edge label info. (#1152) PR #1152
- Remove our reliance on packaging's LegacyVersion. (#1151) PR #1151
- Implement PEX_INTERPRETER special mode support. (#1149) PR #1149
- Fix PexInfo.copy. (#1148) PR #1148
This release upgrades Pip to 20.3.3 + a patch to fix Pex resolves using
the pip-legacy-resolver
and --constraints
. The Pex package is
also fixed to install for Python 3.9.1+.
- Upgrade to a patched Pip 20.3.3. (#1143) Issue #9283 PR #1143
- Fix python requirement to include full 3.9 series. (#1142) PR #1142
This release upgrades Pex to the latest Pip which includes support for the new 2020-resolver (see: https://pip.pypa.io/en/stable/user_guide/#resolver-changes-2020) as well as support for macOS BigSur. Although this release defaults to the legacy resolver behavior, the next release will deprecate the legacy resolver and support for the legacy resolver will later be removed to allow continuing Pip upgrades going forward. To switch to the new resolver, use: --resolver-version pip-2020-resolver.
- Upgrade Pex to Pip 20.3.1. (#1133) PR #1133
This release fixes a deadlock that could be experienced when building PEX files in highly concurrent environments in addition to fixing pex --help-variables output.
A new suite of PEX tools is now available in Pex itself and any PEXes built with the new --include-tools option. Use PEX_TOOLS=1 pex --help to find out more about the available tools and their usage.
Finally, the long deprecated exposure of the Pex APIs through _pex has been removed. To use the Pex APIs you must include pex as a dependency in your PEX file.
- Add a dependency graph tool. (#1132) PR #1132
- Add a venv tool. (#1128) PR #1128
- Remove long deprecated support for _pex module. (#1135) PR #1135
- Add an interpreter tool. (#1131) PR #1131
- Escape venvs unless PEX_INHERIT_PATH is requested. (#1130) PR #1130
- Improve PythonInterpreter venv support. (#1129) PR #1129
- Add support for PEX runtime tools & an info tool. (#1127) PR #1127
- Exclusive atomic_directory always unlocks. (#1126) PR #1126
- Fix PythonInterpreter binary normalization. (#1125) PR #1125
- Add a requires_dists function. (#1122) PR #1122
- Add an is_exe helper. (#1123) PR #1123
- Fix req parsing for local archives & projects. (#1121) PR #1121
- Improve PEXEnvironment constructor ergonomics. (#1120) PR #1120
- Fix safe_open for single element relative paths. (#1118) PR #1118
- Add URLFetcher IT. (#1116) PR #1116
- Implement full featured requirment parsing. (#1114) PR #1114
- Fix --help-variables docs. (#1113) PR #1113
- Switch from optparse to argparse. (#1083) PR #1083
- Fix
iter_compatible_interpreters
withpath
. (#1110) PR #1110 - Fix
Requires-Python
environment marker mapping. (#1105) PR #1105 - Fix spurious
InstalledDistribution
env markers. (#1104) PR #1104 - Deprecate
-R
/--resources-directory
. (#1103) PR #1103 - Fix ResourceWarning for unclosed
/dev/null
. (#1102) PR #1102 - Fix runtime vendoring bytecode compilation races. (#1099) PR #1099
This release improves interpreter discovery to prefer more recent patch versions, e.g. preferring Python 3.6.10 over 3.6.8.
We recently regained access to the docsite, and https://pex.readthedocs.io/en/latest/ is now up-to-date.
- Prefer more recent patch versions in interpreter discovery. (#1088) PR #1088
- Fix
--pex-python
when it's the same as the current interpreter. (#1087) PR #1087 - Fix dir_hash vs. bytecode compilation races. (#1080) PR #1080
- Fix readthedocs doc generation. (#1081) PR #1081
This release adds the --python-path
option, which allows controlling the
interpreter search paths when building a PEX.
The release also removes --use-first-matching-interpreter
, which was a misfeature. If you want to use
fewer interpreters when building a PEX, use more precise values for --interpreter-constraint
and/or
--python-path
, or use --python
or --platform
.
- Add
--python-path
to change interpreter search paths when building a PEX. (#1077) PR #1077 - Remove
--use-first-matching-interpreter
misfeature. (#1076) PR #1076 - Encapsulate
--inherit-path
handling. (#1072) PR #1072
This release brings official support for Python 3.9 and adds a new --tmpdir
option to explicitly
control the TMPDIR used by Pex and its subprocesses. The latter is useful when building PEXes in
space-constrained environments in the face of large distributions.
The release also fixes --cert
and --client-cert
so that they work with PEP-518 builds in
addition to fixing bytecode compilation races in highly parallel environments.
- Add a
--tmpdir
option to the Pex CLI. (#1068) PR #1068 - Honor
sys.executable
unless macOS Framework. (#1065) PR #1065 - Add Python 3.9 support. (#1064) PR #1064
- Fix handling of
--cert
and--client-cert
. (#1063) PR #1063 - Add atomic_directory exclusive mode. (#1062) PR #1062
- Fix
--cert
for PEP-518 builds. (#1060) PR #1060
This release fixes a bug in --resolve-local-platforms
handling that made it unusable in 2.1.16
(#1043) as well as fixing a long standing file handle leak (#1050) and a bug when running under
macOS framework builds of Python (#1009).
- Fix --unzip performance regression. (#1056) PR #1056
- Fix resource leak in Pex self-isolation. (#1052) PR #1052
- Fix use of iter_compatible_interpreters. (#1048) PR #1048
- Do not rely on sys.executable being accurate. (#1049) PR #1049
- slightly demystify the relationship between platforms and interpreters in the library API and CLI (#1047) PR #1047
- Path filter for PythonInterpreter.iter_candidates. (#1046) PR #1046
- Add type hints to util.py and tracer.py (#1044) PR #1044
- Add type hints to variables.py and platforms.py (#1042) PR #1042
- Add type hints to the remaining tests (#1040) PR #1040
- Add type hints to most tests (#1036) PR #1036
- Use MyPy via type comments (#1032) PR #1032
This release fixes a bug in sys.path scrubbing / hermeticity (#1025)
and a bug in the -D / --sources-directory
and
-R / --resources-directory
options whereby PEP-420 implicit
(namespace) packages were not respected (#1021).
- Improve UnsatisfiableInterpreterConstraintsError. (#1028) PR #1028
- Scrub direct sys.path manipulations by .pth files. (#1026) PR #1026
- PEX zips now contain directory entries. (#1022) PR #1022
- Fix UnsatisfiableInterpreterConstraintsError. (#1024) PR #1024
A patch release to fix an issue with the --use-first-matching-interpreter
flag.
- Fix --use-first-matching-interpreter at runtime. (#1014) PR #1014
This release adds the --use-first-matching-interpreter
flag, which
can speed up performance when building a Pex at the expense of being
compatible with fewer interpreters at runtime.
The focus of this release is better support of the --platform
CLI
arg. Platforms are now better documented and can optionally be resolved
to local interpreters when possible via --resolve-local-platforms
to
better support creation of multiplatform PEXes.
- Add support for resolving --platform locally. (#1000) PR #1000
- Improve --platform help. (#1002) PR #1002
- Improve and fix --platform help. (#1001) PR #1001
- Ensure pip download dir is uncontended. (#998) PR #998
A patch release to deploy the PEX_EXTRA_SYS_PATH feature.
- A PEX_EXTRA_SYS_PATH runtime variable. (#989) PR #989
- Fix typos (#986) PR #986
- Update link to avoid a redirect (#982) PR #982
A patch release to fix a symlink issue in remote execution environments.
- use relative paths within wheel cache (#979) PR #979
- Fix Tox not finding Python 3.8 on OSX. (#976) PR #976
This release focuses on the resolver API and resolution performance. Pex 2 resolving using Pip is now at least at performance parity with Pex 1 in all studied cases and most often is 5% to 10% faster.
As part of the resolution performance work, Pip networking configuration is now exposed via Pex CLI
options and the NetworkConfiguration
API type / new resolver.resolve
API parameter.
With network configuration now wired up, the PEX_HTTP_RETRIES
and PEX_HTTP_TIMEOUT
env var
support in Pex 1 that was never wired into Pex 2 is now dropped in favor of passing --retries
and --timeout
via the CLI (See: Issue #94)
- Expose Pip network configuration. (#974) PR #974
- Restore handling for bad wheel filenames to
.can_add()
(#973) PR #973 - Fix wheel filename parsing in PEXEnvironment.can_add (#965) PR #965
- Split Pex resolve API. (#970) PR #970
- Add a
--local
mode for packaging the Pex PEX. (#971) PR #971 - Constrain the virtualenv version used by tox. (#968) PR #968
- Improve Pex packaging. (#961) PR #961
- Make the interpreter cache deterministic. (#960) PR #960
- Fix deprecation warning for
rU
mode (#956) PR #956 - Fix runtime resolve error message generation. (#955) PR #955
- Kill dead code. (#954) PR #954
This release introduces the ability to copy requirements from an existing PEX into a new one.
This can greatly speed up repeatedly creating a PEX when no requirements have changed. A build tool (such as Pants) can create a "requirements PEX" that contains just a static set of requirements, and build a final PEX on top of that, without having to re-run pip to resolve requirements.
- Support for copying requirements from an existing pex. (#948) PR #948
This release brings enhanced performance when using the Pex CLI or API to resolve requirements and improved performance for many PEXed applications when specifying the --unzip option. PEXes built with --unzip will first unzip themselves into the Pex cache if not unzipped there already and then re-execute themselves from there. This can improve startup latency. Pex itself now uses this mode in our [PEX release](https://github.com/pantsbuild/pex/releases/download/v2.1.8/pex).
- Better support unzip mode PEXes. (#941) PR #941
- Support an unzip toggle for PEXes. (#939) PR #939
- Ensure the interpreter path is a file (#938) PR #938
- Cache pip.pex. (#937) PR #937
This release brings more robust control of the Pex cache (PEX_ROOT).
The --cache-dir setting is deprecated in favor of build time control of the cache location with --pex-root and new support for control of the cache's runtime location with --runtime-pex-root is added. As in the past, the PEX_ROOT environment variable can still be used to control the cache's runtime location.
Unlike in the past, the [Pex PEX](https://github.com/pantsbuild/pex/releases/download/v2.1.7/pex) we release can now also be controlled via the PEX_ROOT environment variable. Consult the CLI help for --no-strip-pex-env to find out more.
- Sanitize PEX_ROOT handling. (#929) PR #929
- Fix PEX_* env stripping and allow turning off. (#932) PR #932
- Remove second urllib import from compatibility (#931) PR #931
- Adding --runtime-pex-root option. (#780) PR #780
- Improve interpreter not found error messages. (#928) PR #928
- Add detail in interpreter selection error message. (#927) PR #927
- Respect Requires-Python in PEXEnvironment. (#923) PR #923
- Pin our tox version in CI for stability. (#924) PR #924
- Don't delete the root __init__.py when devendoring. (#915) PR #915
- Remove unused Interpreter.clear_cache. (#911) PR #911
- Silence pip warnings about Python 2.7. (#908) PR #908
- Kill Pip.spawn_install_wheel overwrite arg. (#907) PR #907
- Show pex-root from env as default in help output (#901) PR #901
This release fixes the hermeticity of pip resolver executions when the resolver is called via the Pex API in an environment with PYTHONPATH set.
- readme: adding a TOC (#900) PR #900
- Fix Pex resolver API PYTHONPATH hermeticity. (#895) PR #895
- Fixup resolve debug rendering. (#894) PR #894
- Convert bdist_pex tests to explicit cmdclass. (#897) PR #897
This release fixes a performance regression in which pip would re-tokenize --find-links pages unnecessarily. The parsed pages are now cached in a pip patch that has also been submitted upstream.
- Revendor pip (#890) PR #890
- Add a clear_cache() method to PythonInterpreter. (#885) PR #885
- Error eagerly if an interpreter binary doesn't exist. (#886) PR #886
This release fixes a bug in which interpreter discovery failed when running from a zipped pex.
- Use pkg_resources when isolating a pex code chroot. (#881) PR #881
This release significantly improves performance and correctness of interpreter discovery, particularly when pyenv is involved. It also provides a workaround for EPERM issues when hard linking across devices, by falling back to copying. Resolve error checking also now accounts for environment markers.
- Revert "Fix the resolve check in the presence of platform constraints. (#877)" (#879) PR #879
- [resolver] Fix issue with wheel when using --index-url option (#865) PR #865
- Fix the resolve check in the presence of platform constraints. (#877) PR #877
- Check expected pex invocation failure reason in tests. (#874) PR #874
- Improve hermeticity of vendoring. (#873) PR #873
- Temporarily skip a couple of tests, to get CI green. (#876) PR #876
- Respect env markers when checking resolves. (#861) PR #861
- Ensure Pex PEX contraints match pex wheel / sdist. (#863) PR #863
- Delete unused pex/package.py. (#862) PR #862
- Introduce an interpreter cache. (#856) PR #856
- Re-enable pyenv interpreter tests under pypy. (#859) PR #859
- Harden PythonInterpreter against pyenv shims. (#860) PR #860
- Parallelize interpreter discovery. (#842) PR #842
- Explain hard link EPERM copy fallback. (#855) PR #855
- Handle EPERM when Linking (#852) PR #852
- Pin transitive dependencies of vendored code. (#854) PR #854
- Kill empty setup.py. (#849) PR #849
- Fix tox -epackage to create pex supporting 3.8. (#843) PR #843
- Fix Pex to handle empty ns package metadata. (#841) PR #841
This release restores and improves support for building and running multiplatform pexes. Foreign linux* platform builds now include manylinux2014 compatible wheels by default and foreign CPython pexes now resolve abi3 wheels correctly. In addition, error messages at both buildtime and runtime related to resolution of dependencies are more informative.
Pex 2.1.0 should be considered the first Pex 2-series release that fully replaces and improves upon Pex 1-series functionality.
- Fix pex resolving for foreign platforms. (#835) PR #835
- Use pypa/packaging. (#831) PR #831
- Upgrade vendored setuptools to 42.0.2. (#832) PR #832 PR #1830
- De-vendor pex just once per version. (#833) PR #833
- Support VCS urls for vendoring. (#834) PR #834
- Support python 3.8 in CI. (#829) PR #829
- Fix pex resolution to respect --ignore-errors. (#828) PR #828
- Kill pkg_resources finders monkey-patching. (#827) PR #827
- Use flit to distribute pex. (#826) PR #826
- Cleanup extras_require. (#825) PR #825
This release fixes a regression in handling explicitly requested --index or --find-links http (insecure) repos. In addition, performance of the pex 2.x resolver is brought in line with the 1.x resolver in all cases and improved in most cases.
- Unify PEX buildtime and runtime wheel caches. #821 PR #821
- Parallelize resolve. (#819) PR #819
- Use the resolve cache to skip installs. (#815) PR #815
- Implicitly trust explicitly requested repos. (#813) PR #813
This is a hotfix release that fixes a bug exposed when Pex was asked to use an interpreter with a non-canonical path as well as fixes for 'current' platform handling in the resolver API.
- Fix current platform handling. (#801) PR #801
- Add a test of pypi index rendering. (#799) PR #799
- Fix iter_compatible_interpreters path biasing. (#798) PR #798
This is a htofix release that fixes a bug when specifying a custom index (-i/--index/--index-url) via the CLI.
- Fix #794 issue by add missing return statement in __str__ (#795) PR #795
Pex 2.0.0 is cut on the advent of a large, mostly internal change for typical use cases: it now uses vendored pip to perform resolves and wheel builds. This fixes a large number of compatibility and correctness bugs as well as gaining feature support from pip including handling manylinux2010 and manylinux2014 as well as VCS requirements and support for PEP-517 & PEP-518 builds.
API changes to be wary of:
- The egg distribution format is no longer supported.
- The deprecated
--interpreter-cache-dir
CLI option was removed. - The
--cache-ttl
CLI option andcache_ttl
resolver API argument were removed. - The resolver API replaced
fetchers
with a list ofindexes
and a list offind_links
repos. - The resolver API removed (http)
context
which is now automatically handled. - The resolver API removed
precedence
which is now pip default precedence: wheels when available and not ruled out via the--no-wheel
CLI option oruse_wheel=False
API argument. - The
--platform
CLI option andplatform
resolver API argument now must be full platform strings that include platform, implementation, version and abi; e.g.:--platform=macosx-10.13-x86_64-cp-36-m
. - The
--manylinux
CLI option anduse_manylinux
resolver API argument were removed. Instead, to resolve manylinux wheels for a foreign platform, specify the manylinux platform to target with an explicit--platform
CLI flag orplatform
resolver API argument; e.g.:--platform=manylinux2010-x86_64-cp-36-m
.
In addition, Pex 2.0.0 now builds reproduceable pexes by default; ie:
- Python modules embedded in the pex are not pre-compiled (pass --compile if you want this).
- The timestamps for Pex file zip entries default to midnight on January 1, 1980 (pass --use-system-time to change this).
This finishes off the effort tracked by Issue #716
Changes in this release:
- Pex defaults to reproduceable builds. (#791) PR #791
- Use pip for resolving and building distributions. (#788) PR #788
- Bias selecting the current interpreter. (#783) PR #783
This release adds the --intransitive option to support pre-resolved requirements lists and allows for python binaries built under Gentoo naming conventions.
- Add an --intransitive option. (#775) PR #775
- PythonInterpreter: support python binary names with single letter suffixes (#769) PR #769
This release brings a consistency fix to requirement resolution and an isolation fix that scrubs all non-stdlib PYTHONPATH entries by default, only pre-pending or appending them to the sys.path if the corresponding --inherit-path=(prefer|fallback) is used.
- Avoid reordering of equivalent packages from multiple fetchers (#762) PR #762
- Include PYTHONPATH in --inherit-path logic. (#765) PR #765
This is a hotfix release for the bug detailed in #756 that was introduced by #752 in python 3.7 interpreters.
- Guard against modules with a __file__ of None. (#757) Issue #756 PR #757
- Fix sys.path scrubbing of pex extras modules. (#752) PR #752
- Fix pkg resource early import (#750) PR #750
- Fixup pex re-exec during bootstrap. (#741) PR #741
- Fix resolution of setup.py project extras. (#739) PR #739
- Tighten up namespace declaration logic. (#732) PR #732
- Fixup import sorting. (#731) PR #731
We now support reproducible builds when creating a pex via pex -o foo.pex, meaning that if you were to run the command again with the same inputs, the two generated pexes would be byte-for-byte identical. To enable reproducible builds when building a pex, use the flags --no-use-system-time --no-compile, which will use a deterministic timestamp and not include .pyc files in the Pex.
In Pex 1.7.0, we will default to reproducible builds.
- add delayed pkg_resources import fix from #713, with an integration test (#730) PR #730
- Fix reproducible builds sdist test by properly requiring building the wheel (#727) PR #727
- Fix reproducible build test improperly using the -c flag and add a new test for -c flag (#725) PR #725
- Fix PexInfo requirements using a non-deterministic data structure (#723) PR #723
- Add new --no-use-system-time flag to use a deterministic timestamp in built PEX (#722) PR #722
- Add timeout when using requests. (#726) PR #726
- Refactor reproducible build tests to assert that the original pex command succeeded (#724) PR #724
- Introduce new --no-compile flag to not include .pyc in built pex due to its non-determinism (#718) PR #718
- Document how Pex developers can run specific tests and run Pex from source (#720) PR #720
- Remove unused bdist_pex.py helper function (#719) PR #719
- Add failing acceptance tests for reproducible Pex builds (#717) PR #717
- Make a copy of globals() before updating it. (#715) PR #715
- Make sure PexInfo is isolated from os.environ. (#711) PR #711
- Fix import sorting. (#712) PR #712
- When iterating over Zipfiles, always use the Unix file separator to fix a Windows issue (#638) PR #638
- Fix pex file looses the executable permissions of binary files (#703) PR #703
This is the first release including only a single PEX pex, which supports execution under all interpreters pex supports.
- Fix pex bootstrap interpreter selection. (#701) PR #701
- Switch releases to a single multi-pex. (#698) PR #698
This release fixes long-broken resolution of abi3 wheels.
- Use all compatible versions when calculating tags. (#692) PR #692
This release un-breaks lambdex.
- Restore
pex.pex_bootstrapper.is_compressed
API. (#685) PR #685 - Add the version of pex used to build a pex to build_properties. (#687) PR #687
- Honor interpreter constraints even when PEX_PYTHON and PEX_PYTHON_PATH not set (#668) PR #668
This release changes the behavior of the --interpreter-constraint
option.
Previously, interpreter constraints were ANDed, which made it impossible to
express constraints like '>=2.7,<3' OR '>=3.6,<4'; ie: either python 2.7 or
else any python 3 release at or above 3.6. Now interpreter constraints are
ORed, which is likely a breaking change if you have scripts that pass multiple
interpreter constraints. To transition, use the native ,
AND operator in
your constraint expression, as used in the example above.
- Provide control over pex warning behavior. (#680) PR #680
- OR interpreter constraints when multiple given (#678) Issue #655 PR #678
- Pin isort version in CI (#679) PR #679
- Honor PEX_IGNORE_RCFILES in to_python_interpreter() (#673) PR #673
- Make run_pex_command more robust. (#670) PR #670
- Support de-vendoring for installs. (#666) PR #666
- Add User-Agent header when resolving via urllib (#663) PR #663
- Fix interpreter finding (#662) PR #662
- Add recipe to use PEX with requests module and proxies. (#659) PR #659
- Allow pex to be invoked using runpy (python -m pex). (#637) PR #637
- Make tox -evendor idempotent. (#651) PR #651
- Fix invalid regex and escape sequences causing DeprecationWarning (#646) PR #646
- Follow PEP 425 suggestions on distribution preference. (#640) PR #640
- Setup interpreter extras in InstallerBase. (#635) PR #635
- Ensure bootstrap demotion is complete. (#634) PR #634
- Fixup PEXEnvironment extras resolution. (#617) PR #617
- Repair unhandled AttributeError during pex bootstrapping. (#599) PR #599
This release brings an exit code fix for pexes run via entrypoint as well as a fix for finding scripts when building pexes from wheels with dashes in their distribution name.
- Update PyPI default URL to pypi.org (#610) PR #610
- Pex exits with correct code when using entrypoint (#605) PR #605
- Fix *_custom_setuptools_useable ITs. (#606) PR #606
- Update pyenv if neccesary (#586) PR #586
- Fix script search in wheels. (#600) PR #600
- Small Docstring Fix (#595) PR #595
This release brings a fix to handle top-level requirements with environment markers, fully completing environment marker support.
- Filter top-level requirements against env markers. (#592) PR #592
This release fixes pexes such that they fully support environment markers, the canonical use case being a python 2/3 pex that needs to conditionally load one or more python 2 backport libs when running under a python 2 interpreter only.
- Revert "Revert "Support environment markers during pex activation. (#582)"" PR #582
This is a hotfix release for 1.4.8 that fixes a regression in interpreter setup that could lead to resolved distributions failing to build or install.
- Cleanup PexInfo and PythonInterpreter. (#581) PR #581
- Fix resolve regressions introduced by the 1.4.8. (#580) PR #580
- Narrow the env marker test. (#578) PR #578
- Documentation for #569 (#574) PR #574
This release adds support for -c and -m pexfile runtime options that emulate the behavior of the same arguments to python as well a fix for handling the non-standard platform reported by setuptools for Apple system interpreters in addition to several other bug fixes.
- Fix PEXBuilder.clone. (#575) PR #575
- Fix PEXEnvironment platform determination. (#568) PR #568
- Apply more pinning to jupyter in IT. (#573) PR #573
- Minimize interpreter bootstrapping in tests. (#571) PR #571
- Introduce 3.7 to CI and release. (#567) PR #567
- Add OSX shards. (#565) PR #565
- Add support for -m and -c in interpreter mode. (#563) PR #563
- Ignore concurrent-rename failures. (#558) PR #558
- Fixup test_jupyter_appnope_env_markers. (#562) PR #562
This is a hotfix release for a regression in setuptools compatibility introduced by #542.
- Fixup PEX.demote_bootstrap: fully unimport. (#554) PR #554
This release opens up setuptools support for more modern versions that support breaking changes in setup used in the wild.
- Fix for super() usage on "old style class" ZipFile (#546) PR #546
- Cleanup bootstrap dependencies before handoff. (#542) PR #542
- Support -c for plat spec dists in multiplat pexes. (#545) PR #545
- Support - when running as an interpreter. (#543) PR #543
- Expand the range of supported setuptools. (#541) PR #541
- Preserve perms of files copied to pex chroots. (#540) PR #540
- Add more badges to README. (#535) PR #535
- Fixup CHANGES PR links for 1.4.5.
This release adds support for validating pex entrypoints at build time in addition to several bugfixes.
- Fix PEX environment setup. (#531) #531
- Fix installers to be insensitive to extras iteration order. (#532) #532
- Validate entry point at build time (#521) #521
- Fix pex extraction perms. (#528) #528
- Simplify .travis.yml. (#524) #524
- Fix PythonInterpreter caching and ergonomics. (#518) #518
- Add missing git dep. (#519) #519
- Introduce a controlled env for pex testing. (#517) #517
- Bump wheel version to latest. (#515) #515
- Invoke test runner at a more granular level for pypy shard. (#513) #513
This release adds support for including sources and resources directly in a produced pex - without the need to use pants.
- Add resource / source bundling to pex cli (#507) #507
Another bugfix release for the 1.4.x series.
- Repair environmental marker platform setting. (#500) #500
- Broaden abi selection for non-specified abi types. (#503) #503
This release repairs a tag matching regression for .egg dists that inadvertently went out in 1.4.1.
- Improve tag generation for EggPackage. (#493) #493
A bugfix release for 1.4.x.
- Repair abi prefixing for PyPy. (#483) #483
- Repair .egg resolution for platform specific eggs. (#486) #486
- Eliminate the python3.3 shard. (#488) #488
This release includes full Manylinux support, improvements to wheel resolution (including first class platform/abi tag targeting) and a handful of other improvements and bugfixes. Enjoy!
Special thanks to Dan Blanchard (@dan-blanchard) for seeding the initial PR for Manylinux support and wheel resolution improvements.
- Complete manylinux support in pex. (#480) #480
- Add manylinux wheel support and fix a few bugs along the way (#316) #316
- Skip failing tests on pypy shard. (#478) #478
- Bump travis image to Trusty. (#476) #476
- Mock PATH for problematic interpreter selection test in CI (#474) #474
- Skip two failing integration tests. (#472) #472
- Better error handling for missing setuptools. (#471) #471
- Add tracebacks to IntegResults. (#469) #469
- Fix failing tests in master (#466) #466
- Repair isort-check failure in master. (#465) #465
- Repair style issues in master. (#464) #464
- Fixup PATH handling in travis.yml. (#462) #462
- Add blacklist handling for skipping requirements in pex resolver #457 #457
This is a bugfix release for a regression that inadvertently went out in 1.3.0.
- inherit_path allows 'prefer', 'fallback', 'false' (#444) #444
- Change PEX re-exec variable from ENV to os.environ (#441) #441
- Bugfix for entry point targeting + integration test (#435) #435
- Add interpreter constraints option and use constraints to search for compatible interpreters at exec time (#427) #427
- Fix handling of pre-release option. (#424) #424
- Patch sys module using pex_path from PEX-INFO metadata (#421) #421
- Create --pex-path argument for pex cli and load pex path into pex-info metadata (#417) #417
- Allow passing a preamble file to the CLI (#400) #400
- Add first-class support for multi-interpreter and multi-platform pex construction. (#394) #394
- Sort keys in PEX-INFO file so the output is deterministic. (#384) #384
- Pass platform for SourceTranslator (#386) #386
- Fix for Ambiguous Resolvable bug in transitive dependency resolution (#367) #367
This release follows-up on 1.2.0 fixing bugs in the pre-release resolving code paths.
- Resolving pre-release when explicitly requested (#372) #374
- Pass allow_prerelease to other iterators (Static, Caching) (#373) #373
- Fix bug in cached dependency resolution with exact resolvable. (#365) #365
- Treat .pth injected paths as extras. (#370) #370
- Follow redirects on HTTP requests (#361) #361
- Fix corner case in cached dependency resolution (#362) #362
- Fix CacheControl import. (#357) #357
This release is a quick fix for a bootstrapping bug that inadvertently went out in 1.2.0 (Issue #354).
- Ensure packaging dependency is self-contained. (#355) #355 Fixes #354
This release changes pex requirement resolution behavior. Only stable requirements are resolved by default now. The previous behavior that included pre-releases can be retained by passing --pre on the pex command line or passing allow_prereleases=True via the API.
- Upgrade dependencies to modern version ranges. (#352) #352
- Add support for controlling prerelease resolution. (#350) #350 Fixes #28
- Add dummy flush method for clean interpreter exit with python3.6 (#343) #343
- Implement --constraints in pex (#335) #335
- Make sure namespace packages (e.g. virtualenvwrapper) don't break pex (#338) #338
- Expose a PEX instance's path. (#332) #332
- Check for scripts directory in get_script_from_egg (#328) #328
- Make PEX_PATH unify pex sources, as well as requirements. (#329) #329
- Adjust FileFinder import to work with Python 3.6. (#318) #318
- Kill zipmanifest monkeypatching. (#322) #322
- Bump setuptools range to latest. (#323) #323
- Fix #309 by deduplicating output of the distribution finder. (#310) #310
- Update wheel dependency to >0.26.0. (#304) #304
- Repair Executor error handling for other classes of IOError/OSError. (#292) #292
- Fix bdist_pex --pex-args. (#285) #285
- Inherit user site with --inherit-path. (#284) #284
- Repair passing of stdio kwargs to PEX.run(). (#288) #288
- Fix bdist_pex interpreter cache directory. (#286) #286
- Normalize and edify subprocess execution. (#255) #255
- Don't ignore exit codes when using setuptools entry points. (#280) #280 Fixes #137
- Update cache dir when bdist_pex.run is called directly. #278 Fixes #274
- Improve failure modes for os.rename() as used in distribution caching. #271 Fixes #265
- Bugfix: Open setup.py in binary mode. #264 Fixes #263
- Bugfix: Repair a regression in --disable-cache. #261 Fixes #260
- Add README and supported python versions to PyPI description. #258
- Use open with utf-8 support. #231
- Add --pex-root option. #206
This release is a quick fix for a regression that inadvertently went out in 1.1.5 (Issue #243).
- Fix the
bdist_pex
setuptools
command to work for python2. #246 Fixes #243 - Upgrade pex dependencies on
setuptools
andwheel
. #244 Fixes #238
- Fix
PEXBuilder.clone
and thusbdist_pex --pex-args
for--python
and--python-shebang
. #234 Fixes #233 - Fix old
pkg_resources
egg version normalization. #227 Fixes #226 - Fix the
inherit_path
handling. #224 - Fix handling of bad distribution script names when used as the pex entrypoint. #221 Fixes #220
This release is a quick fix for a regression that inadvertently went out in 1.1.3 (Issue #216).
- Add a test for the regression in
FixedEggMetadata._zipinfo_name
and revert the breaking commit. Fixes #216
This release includes an initial body of work towards Windows support, ABI tag support for CPython 2.x and a fix for version number normalization.
- Add python 2.x abi tag support. #214 Fixes #213
- Add .idea to .gitignore. #205
- Don't normalize version numbers as names. #204
- More fixes for windows. #202
- Fixes to get pex to work on windows. #198
- Bump setuptools & wheel version pinning. #194
- Unescape html in PageParser.href_match_to_url. #191
- Memoize calls to Crawler.crawl() for performance win in find-links based resolution. #187
- Fix infinite recursion when
PEX_PYTHON
points at a symlink. #182 - Add
/etc/pexrc
to the list of pexrc locations to check. #183 - Improve error messaging for platform constrained Untranslateable errors. #179
- Add support for
.pexrc
files for influencing the pex environment. See the notes here. #128. - Bug fix: PEX_PROFILE_FILENAME and PEX_PROFILE_SORT were not respected. #154.
- Adds the
bdist_pex
command to setuptools. #99. - Bug fix: We did not normalize package names in
ResolvableSet
, so it was possible to depend onsphinx
andSphinx-1.4a0.tar.gz
and get two versions build and included into the pex. #147. - Adds a pex-identifying User-Agent. #101.
- Bug fix: Accommodate OSX
Python
python binaries. Previously the OSX python distributions shipped with OSX, XCode and available via https://www.python.org/downloads/ could fail to be detected using thePythonInterpreter
class. Fixes #144. - Bug fix: PEX_SCRIPT failed when the script was from a not-zip-safe egg. Original PR #139.
- Bug fix:
sys.exit
called without arguments would cause None to be printed on stderr since pex 1.0.1. #143.
- Bug fix: PEX-INFO values were overridden by environment
Variables
with default values that were not explicitly set in the environment. Fixes #135. - Bug fix: Since 69649c1 we have been unpatching
the side-effects of
sys.modules
afterPEX.execute
. This takes all modules imported during the PEX lifecycle and sets all their attributes toNone
. Unfortunately,sys.excepthook
,atexit
and__del__
may still try to operate using these tainted modules, causing exceptions on interpreter teardown. This reverts just thesys
unpatching so that the abovementioned teardown hooks behave more predictably. Fixes #141.
- Allow PEXBuilder to optionally copy files into the PEX environment instead of hard-linking them.
- Allow PEXBuilder to optionally skip precompilation of .py files into .pyc files.
- Bug fix: PEXBuilder did not respect the target interpreter when compiling source to bytecode. Fixes #127.
- Bug fix: Fix complex resolutions when using a cache. Fixes: #120.
The 1.0.0 release of pex introduces a few breaking changes: pex -r
now takes requirements.txt files
instead of requirement specs, pex -s
has now been removed since source specs are accepted as arguments,
and pex -p
has been removed in favor of its alias pex -o
.
The pex command line interface now adheres to semver insofar as backwards incompatible CLI changes will invoke a major version change. Any backwards incompatible changes to the PEX environment variable semantics will also result in a major version change. The pex API adheres to semver insofar as backwards incompatible API changes will invoke minor version changes.
For users of the PEX API, it is recommended to add minor version ranges, e.g. pex>=1.0,<1.1
.
For users of the PEX CLI, major version ranges such as pex>=1,<2
should be sufficient.
- BREAKING CHANGE: Removes the
-s
option in favor of specifying directories directly as arguments to the pex command line. - BREAKING CHANGE:
pex -r
now takes requirements.txt filenames and not requirement specs. Requirement specs are now passed as arguments to the pex tool. Use--
to escape command line arguments passed to interpreters spawned by pex. Implements #5. - Adds a number of flag aliases to be more compatible with pip command lines:
--no-index
,-f
,--find-links
,--index-url
,--no-use-wheel
. Removes-p
in favor of-o
exclusively. - Adds
--python-shebang
option to the pex tool in order to set the#!
shebang to an exact path. #53. - Adds support for
PEX_PYTHON
environment variable which will cause the pex file to reinvoke itself using the interpreter specified, e.g.PEX_PYTHON=python3.4
orPEX_PYTHON=/exact/path/to/interpreter
. #27. - Adds support for
PEX_PATH
environment variable which allows merging of PEX environments at runtime. This can be used to inject plugins or entry_points or modules from one PEX into another without explicitly building them together. #30. - Consolidates documentation of
PEX_
environment variables and adds the--help-variables
option to the pex client. Partially addresses #13. - Adds helper method to dump a package subdirectory onto disk from within a zipped PEX file. This can be useful for applications that know they're running within a PEX and would prefer some static assets dumped to disk instead of running as an unzipped PEX file. #12.
- Now supports extras for static URLs and installable directories. #65.
- Adds
-m
and--entry-point
alias to the existing-e
option for entry points in the pex tool to evoke the similarity topython -m
. - Adds console script support via
-c/--script/--console-script
andPEX_SCRIPT
. This allows you to reference the named entry point instead of the exactmodule:name
pair. Also supports scripts defined in thescripts
section of setup.py. #59. - Adds more debugging information when encountering unresolvable requirements. #79.
- Bug fix:
PEX_COVERAGE
andPEX_PROFILE
did not function correctly when SystemExit was raised. Fixes #81. - Bug fix: Fixes caching in the PEX tool since we don't cache the source distributions of installable directories. #24.
This is the last release before the 1.0.0 development branch is started.
- Change the setuptools range to >=2.2,<16 by handling EntryPoint changes as well as
being flexible on whether
pkg_resources
is a package or a module. Fixes #55 and #34. - Adds option groups to the pex tool to make the help output slightly more readable.
- Bug fix: Make
pip install pex
work better by removingextras_requires
on theconsole_script
entry point. Fixes #48 - New feature: Adds an interpreter cache to the
pex
tool. If the user does not explicitly disable the wheel feature and attempts to build a pex with wheels but does not have the wheel package installed, pex will download it in order to make the feature work. Implements #47 in order to fix #48
- Bug fix: Honor installed sys.excepthook in pex teardown. RB #1733
- Bug fix:
UrllibContext
usedreplace
as a keyword argument forbytes.decode
but this only works on Python 3. Pull Request #46
- Bug fix: Fixup string formatting in pex/bin/pex.py to support Python 2.6 Pull Request #40
- Performance improvement: Speed up the best-case scenario of dependency resolution. RB #1685
- Bug fix: Change from
uuid4().get_hex()
touuid4().hex
to maintain Python3 compatibility of pex.common. Pull Request #39 - Bug fix: Actually cache the results of translation. Previously bdist translations would be created in a temporary directory even if a cache location was specified. RB #1666
- Bug fix: Support all potential abi tag permutations when determining platform compatibility. Pull Request #33
- Performance improvement: Don't always write packages to disk if they've already been cached. This can significantly speed up launching PEX files with a large number of non-zip-safe dependencies. RB #1642
- Bug fix: Allow pex 0.8.x to parse pex files produced by earlier versions of pex and twitter.common.python.
- Pin pex to setuptools prior to 9.x until we have a chance to make changes related to PEP440 and the change of pkg_resources.py to a package.
- Bug fix: Fix issue where it'd be possible to
os.path.getmtime
on a remoteLink
object Issue #29
- API change: Decouple translation from package iteration. This removes
the Obtainer construct entirely, which likely means if you're using PEX as
a library, you will need to change your code if you were doing anything
nontrivial. This adds a couple new options to
resolve
but simplifies the story around how to cache packages. RB #785 - Refactor http handling in pex to allow for alternate http implementations. Adds support for requests, improving both performance and security. For more information, read the commit notes at 91c7f32. RB #778
- Improvements to API documentation throughout.
- Renamed
Tracer
toTraceLogger
to prevent nondeterministic isort ordering. - Refactor tox.ini to increase the number of environment combinations and improve coverage.
- Adds HTTP retry support for the RequestsContext. RB #1303
- Make pex --version correct. Issue #19
- Bug fix: Fix over-aggressive sys.modules scrubbing for namespace packages. Under certain circumstances, namespace packages in site-packages could conflict with packages within a PEX, causing them to fail importing. RB #1378
- Bug fix: Replace uses of
os.unsetenv(...)
withdel os.environ[...]
Pull Request #11 - Bug fix: Scrub sys.path and sys.modules based upon both supplied path and realpath of files and directories. Newer versions of virtualenv on Linux symlink site-packages which caused those packages to not be removed from sys.path correctly. Issue #21
- Bug fix: The pex -s option was not correctly pulling in transitive dependencies. Issue #22
- Bug fix: Adds
content
method to HTTP contexts that does HTML content decoding, fixing an encoding issue only experienced when using Python 3. Issue #10
- Rename
twitter.common.python
topex
and split out from the twitter/commons repo.
- Change the interpretation of
-i
(and of PyPIFetcher's pypi_base) to match pip's-i
. This is useful for compatibility with devpi.
Ensures that .egg/.whl distributions on disk have their mtime updated even though we no longer overwrite them. This gives them a new time lease against their ttl.
Without this change, once a distribution aged past the ttl it would never be used again, and builds would re-create the same distributions in tmpdirs over and over again.
Fixes an issue where SourceTranslator would overwrite .egg/.whl distributions already on disk. Instead it should always check to see if a copy already exists and reuse if there.
This ordinarily should not be a problem but the zipimporter caches metadata by filename instead of stat/sha, so if the underlying contents changed a runtime error would be thrown due to seemingly corrupt zip file offsets. RB #684
- Adds
-i/--index
option to the pex tool.
- Adds
twitter.common.python.pex_bootstrap
bootstrap_pex_env
function in order to initialize a PEX environment from within a python interpreter. (Patch contributed by @kwlzn) - Adds stdin=,stdout=,stderr= keyword parameters to the
PEX.run
function. (Patch from @benjy)
- The crawler now defaults to not follow links for security reasons.
(Before the default behavior was to implicitly
--follow-links
for all requirements.) RB #293
- Improves scrubbing of site-packages from PEX environments. RB #289
- Silences exceptions reported during interpreter teardown (the exceptions resulting from incorrect atexit handler behavior) introduced by 0.4.3 RB #253 RB #249
- Adds
__hash__
toLink
so that Packages are hashed correctly intwitter.common.python.resolver
resolve
- Adds
twitter.common.python.finders
which are additional finders for setuptools including: - find eggs within a .zip - find wheels within a directory - find wheels within a .zip RB #86 - Adds a new Package abstraction by refactoring Link into Link and Package. RB #92
- Adds support for PEP425 tagging necessary for wheel support. RB #87
- Improves python environment isolation by correctly scrubbing namespace
packages injected into module
__path__
attributes by nspkg pth files. RB #116 - Adds
twitter.common.python.resolver
resolve
method that handles transitive dependency resolution better. This means that if the requirementfutures==2.1.2
and an unqualifiedfutures>=2
is pulled in transitively, our resolver will correctly resolve futures 2.1.2 instead of reporting a VersionConflict if any version newer than 2.1.2 is available. RB #129 - Factors all
twitter.common.python
test helpers intotwitter.common.python.testing
RB #91 - Bug fix: Fix
OrderedSet
atexit exceptions RB #147 - Bug fix: Fix cross-device symlinking (patch from @benjy)
- Bug fix: Raise a
RuntimeError
if we fail to writepkg_resources
into a .pex RB #115
- Upgrade to
setuptools>=1
twitter.common.python
is no longer a namespace package
- Kill the egg distiller. We now delegate .egg generation to bdist_egg. RB #55
- Short-circuit resolving a distribution if a local exact match is found. RB #47
- Correctly patch the global
pkg_resources
WorkingSet
for the lifetime of the Python interpreter. RB #52 - Fixes a performance regression in setuptools
build_zipmanifest
Setuptools Issue #154 RB #53
- Plumb through the
--zip-safe
,--always-write-cache
,--ignore-errors
and--inherit-path
flags to the pex tool. - Delete the unused
PythonDirWrapper
code. - Split
PEXEnvironment
resolution intotwitter.common.python.environment
and deconflateWorkingSet
/Environment
state. - Removes the monkeypatched zipimporter in favor of keeping all eggs
unzipped within PEX files. Refactors the PEX dependency cache in
util.py
- Adds interpreter detection for Jython and PyPy.
- Dependency translation errors should be made uniform. (Patch from @johnsirois)
- Adds
PEX_PROFILE_ENTRIES
to limit the number of entries reported whenPEX_PROFILE
is enabled. (Patch from @rgs_) - Bug fix: Several fixes to error handling in
twitter.common.python.http
(From Marc Abramowitz) - Bug fix: PEX should not always assume that
$PATH
was available. (Patch from @jamesbroadhead) - Bug fix: Filename should be part of the .pex cache key or else multiple identical versions will incorrectly resolve (Patch from @tc)
- Bug fix: Executed entry points shouldn't be forced to run in an
environment with
__future__
imports enabled. (Patch from @lawson_patrick) - Bug fix: Detect versionless egg links and fail fast. (Patch from @johnsirois.)
- Bug fix: Handle setuptools>=2.1 correctly in the zipimport monkeypatch (Patch from @johnsirois.)
- Bug fix: Fix handling of Fetchers with
file://
urls.
- Adds the pex tool as a standalone tool.
- Bug fix: Bootstrapped
twitter.common.python
should declaretwitter.common
as a namespace package.
- Make
twitter.common.python
fully standalone by consolidating external dependencies withintwitter.common.python.common
.
- Initial published version of
twitter.common.python
.