Default to Python 3.7 #10319

adamjstewart · 2019-01-11T22:21:48Z

Apologies for making a PR on such a controversial topic without first opening an issue for discussion, but I wanted to make it clear what I'm proposing. This PR should not be merged without majority approval from several people.

Currently, spack install python defaults to Python 2. With this PR, Spack will install Python 3 by default. This PR is motivated by the fact that Python 2 will be retired at the end of the year.

Currently, the following packages only support Python 2:

elemental (unmaintained, use hydrogen instead: Default to Python 3.7 #10319 (comment))
gdl (Python 3 support is a WIP, see Python 3 compatibility gnudatalanguage/gdl#91 and Initial attempt at python 3 support gnudatalanguage/gdl#468)
gplates (pyGPlates now supports Python 3, unsure about GPlates: https://www.gplates.org/news.html#gplates210)
graphviz (no current work on Python 3 porting: https://gitlab.com/graphviz/graphviz/issues/1383)
grib-api (replaced by ecCodes: https://confluence.ecmwf.int//display/GRIB/Home)
julia
manta (no current timeline on Python 3 support: Python2 is EOL soon - please switch to Python3 Illumina/manta#180)
mozjs
node-js
nwchem (docs still use Python 2 syntax: http://www.nwchem-sw.org/index.php/Python)
partitionfinder(no new releases since 2016: https://github.com/brettc/partitionfinder/releases)
plplot (possible Python 3 support: https://packages.debian.org/sid/python3-plplot)
precice (some work on Python 3 support, but CMake still searches for Python 2.7: Port python bindings to python3 precice/precice#68)
py-cdat-lite (no longer supported: https://github.com/cedadev/cdat_lite)
py-cogent (no new releases since 2016: https://pypi.org/project/cogent/#history)
py-crispresso (no longer supported: https://github.com/lucapinello/CRISPResso)
py-doxypy (no new releases since 2009: https://pypi.org/project/doxypy/#history)
py-espressopp (@junghans: Move to Python3 espressopp/espressopp#227)
py-merlin (no new releases since 2012: https://pypi.org/project/merlin/#files)
py-methylcode (no new updates since 2014: https://github.com/brentp/methylcode)
py-opentuner (no current support, but work in progress: Does the opentuner support python 3 jansel/opentuner#103)
py-picrust (replaced by PICRUSt2: https://github.com/picrust/picrust2)
py-pil (replaced by py-pillow)
py-pygtk (replaced by py-pygobject: https://pypi.org/project/PyGObject/)
py-pyrad (no new updates since 2016: https://github.com/dereneaton/pyrad)
py-pysqlite (builtin library in Python 3: https://docs.python.org/3/library/sqlite3.html)
py-pythia (no progress on converting to Python 3 so far: Convert to Python 3 geodynamics/pythia#4)
py-weave (scipy.weave was removed, new users are encourage to use cython instead: https://pypi.org/project/weave/)
rust (won't update to Python 3 until LLVM does: Python 2.7 build dependency rust-lang/rust#59838)
sensei
shiny-server (no Python 3 support: https://github.com/rstudio/shiny-server/wiki/Building-Shiny-Server-from-Source)
sst-core
tppred
visit

And the following packages only support Python 3:

catalyst*
charliecloud
codar-cheetah
flit
intel-gpu-tools
interproscan*
iso-codes
libpeas
meson
ont-albacore
paraview*
phist
portcullis
py-ase* (https://wiki.fysik.dtu.dk/ase/releasenotes.html)
py-astropy
py-atropos
py-cairocffi
py-cutadapt*
py-dnaio
py-docutils-stubs
py-fiona (Support for Python 2.x Toblerity/Fiona#716)
py-fsspec
py-gpaw* (https://wiki.fysik.dtu.dk/gpaw/releasenotes.html)
py-h5glance
py-illumina-utils
py-ipykernel*
py-kmodes
py-libensemble*
py-macs2*
py-matplotlib*
py-more-itertools*
py-mypy
py-numba
py-numpy* (https://docs.scipy.org/doc/numpy/release.html)
py-pandas* (https://pandas.pydata.org/pandas-docs/stable/whatsnew/v0.25.0.html)
py-pipits* (https://github.com/hsgweon/pipits#a1-install)
py-pyani
py-pyproj (End of Support for Python 2.x pyproj4/pyproj#331)
py-pytest* (https://docs.pytest.org/en/latest/py27-py34-deprecation.html)
py-rasterio (End of Support for Python 2.x rasterio/rasterio#1613)
py-rpy2*
py-scikit-learn*
py-scipy* (https://docs.scipy.org/doc/scipy/reference/release.1.3.0.html)
py-sphinx*
py-sphinxcontrib-applehelp
py-sphinxcontrib-devhelp
py-sphinxcontrib-jsmath
py-sphinxcontrib-htmlhelp
py-sphinxcontrib-serializinghtml
py-sphinxcontrib-qthelp
py-tables*
py-typed-ast
py-unittest2py3k
saga-gis
snakemake
sparta
spot*
verrou*
wget

* The latest version of these packages require Python 3, although older versions can be built with Python 2.

Feel free to update this list as it changes. Many of the packages that only support Python 2 may have already been updated upstream but the Spack package simply contains an old version. It would be good to push the developers of the rest of the packages to support Python 3.

Currently, there are more packages that require Python 2 than there are that require Python 3. But this will continue to change as the countdown clock reaches 2020. In particular, the reason I opened this PR is the meson package, which is becoming the default build system for GTK and many GNOME packages. Meson requires Python 3, and trying to build anything with meson anywhere in the spec causes Spack to crash. This could obviously be resolved with better concretization, but I haven't seen any progress made on that.

Some possibly biased Python 3 users who would be interested in this topic: @svenevs @citibeth @tgamblin

tgamblin · 2019-01-13T01:46:12Z

I like this in principle, but I suspect that it will surprise a lot of users. I'm also not sure how robust the python 3 support is yet. Can we hold off until we can ensure that a decent subset of the python 3 packages build fine as binaries? I'm fine with having an opinion about Python as long as 3 works as well as 2, but right now I worry that we'll be enforcing this and providing a less robust default stack.

citibeth · 2019-01-13T03:26:22Z

I have a lot of information (not opinions) on this issue but no chance yet to say much. Puthon3 is now well developed and everything works with it. However the spack packages are often designed in a way that breaks things. For example, there are many backports of puthon3 functionality that exist as python2 only packages. These are listed as dependencies on packages, and the concretizer is not smart enough to remove them as dependencies when building with python3. I have designed a fix for this that makes it work with both; but that fix has languished in “unofficial” land basically forever. Short version: if we want to make it work by default we can and will. One day we will have to do this. The only question is timing

…

On Sat, Jan 12, 2019 at 20:46 Todd Gamblin ***@***.***> wrote: I like this in principle, but I suspect that it will surprise a lot of users. I'm also not sure how robust the python 3 support is yet. Can we hold off until we can ensure that a decent subset of the python 3 packages build fine as binaries? I'm fine with having an opinion about Python as long as 3 works as well as 2, but right now I worry that we'll be enforcing this *and* providing a less robust default stack. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10319 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB1cdzA-76C1vA7sZjE6y6D7hQE7qgZGks5vCo_lgaJpZM4Z8RZe> .

citibeth · 2019-01-15T02:34:11Z

@adamjstewart Have you reviewed / incorporated #7926 in this PR? Currently, that PR is required to build Python3-based software stacks. It could be merged in a way that allows both Python2 and Python3-based stacks. If we're going to have Python3 be the default, we might wish to reverse the "sense" of #7926 so that extra variants are required for Python2.

#7926 would not be needed with a better concretizer. But we don't currently have one.

citibeth · 2019-01-15T02:50:46Z

My thoughts on this:

Apart from Spack, the Python3 ecosystem is rather mature. Most things that most people want to do "just work." All the "core" Python libraries work fine with Python3. I've been running a Python3 stack with Spack for 3 years, and it works great.

Setting the default version of Python to 3.x in your packages.yaml file is not very hard. Therefore, this PR doesn't do much.

HOWEVER... the reality is that, as it stands, Spack blows up if you just do that. Many packages work with Python2 or Python3; however, when built with Python2, they require Python2-only support libraries that are backports of standard Python3 features. The Spack packages aren't smart enough to eliminate these dependencies when building with Python3, thereby making the Spack recipe (but not the upstream Python library) not work with Python3. This issue must be ironed out before we can talk seriously about changing Python version defaults. Either we run with something like what we have in #7926; or we fix the concretizer so we can do stuff like depends_on('py-backport', when='^python@2').

Summary: before we can switch the default version to Python3, we need to make sure that Python3-based stacks build smoothly without modifying / patching / merging in from other branches.

As for the Python2-only packages identified above, someone needs to do the research and categorize them as:

A backport that is not needed in Python3.
A Python2-only package that has been made obsolete by some other Python3-only package (and identify what that Python3 package is).
A package that was Python2-only at some point; but now also works with Python3.
A true Python2-only package, with no functional replacement for Python3.
Non-Python packages that happen to use Python2 scripts to build or something; and nobody has bothered to convert them to Python3.

At this point, there aren't many packages in category (4). My guess is they are mostly Python interfaces to specialized domain-specific software with only a small number of users. For example, PISM only supported Python2 bindings until Dec 2018. Apparently, PETSc still requires Python2 to build (although it's not a Python package per se, so it falls in category 5).
pism/pism#359

ISSM is another example. They mostly use MATLAB (not Python) in-house anyway. ISSM is also still running on SVN (not git). And they hand-build 50+ packages as prerequisites, rather than using Spack. So I wouldn't expect Python3 bindings any time soon. OTOH, ISSM is specialty software with a small user base.
https://issm.jpl.nasa.gov/download/unix/

adamjstewart · 2019-01-15T02:56:18Z

The tut build tests are failing. After looking into this, it looks like the version of waf shipped with tut does not support Python 3.7. This bug was fixed upstream in waf but needs to be updated in tut:
https://gitlab.com/ita1024/waf/commit/facdc0b173d933073832c768ec1917c553cb369c

citibeth · 2019-01-15T03:05:21Z

Python 3.7 is still rather new. If you want to determine Python3 compatibility, I would test against Python 3.5 or maybe 3.6.

citibeth · 2019-03-06T13:58:28Z

Note the list of packages that declare they will require Python3, starting in the next year or so:
https://python3statement.org/

This includes all the basics: numpy, matplotlib, basemap, scipy, ipython, jupyter, pandas. I think that sometime in 2019 is the time for Spack to commit to Python3 by default.

baberlevi · 2019-05-23T20:33:03Z

I'm trying to understand what's holding back this PR (besides the tut build test). The way I'm looking at it, the packages that support python2-only aren't negatively affected by this change - they already have a dependency on python2 explicitly and concretize correctly even with the PR in place. The python3-only packages are likewise fine with this PR. The packages that are some combination of depends_on(python@2:2.9,3:) are the ones currently 'broken' in the out-of-the-box user experience, and would be 'fixed' again with this PR.

citibeth · 2019-05-24T00:07:02Z

Lack of a coherent agreed-upon plan is holding us back for now. See #11468.

adamjstewart · 2019-07-15T03:18:00Z

The tut build tests are failing. After looking into this, it looks like the version of waf shipped with tut does not support Python 3.7. This bug was fixed upstream in waf but needs to be updated in tut:
https://gitlab.com/ita1024/waf/commit/facdc0b173d933073832c768ec1917c553cb369c

Filed an issue: mrzechonek/tut-framework#18

adamjstewart · 2019-07-22T04:30:08Z

@citibeth does pism support Python 3 yet?

adamjstewart · 2019-07-22T04:53:56Z

Went through all the packages in Spack that claim to only support Python 2. A few now have Python 3 support, but the vast majority are either a work in progress or are no longer supported. A few like julia are important enough that we might want to wait for Python 3 support.

citibeth · 2019-07-22T15:03:42Z

pism/pism#359

…

On Mon, Jul 22, 2019 at 12:30 AM Adam J. Stewart ***@***.***> wrote: @citibeth <https://github.com/citibeth> does pism support Python 3 yet? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10319?email_source=notifications&email_token=AAOVY55OWI3QCTX5ZSGP2KLQAUZVNA5CNFSM4GPRCZPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2OXMYI#issuecomment-513635937>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOVY5ZAU3ZYEDSUJJWRMDTQAUZVNANCNFSM4GPRCZPA> .

adamjstewart · 2019-07-26T04:21:55Z

#12145 may put the brakes on this PR for awhile. The latest version of Python 3 can't be built with the Intel compilers. Hopefully they will come out with a patch soon.

adamjstewart · 2019-07-28T03:39:50Z

Note that we've reached the point where the latest releases of numpy, scipy, matplotlib, pandas, and scikit-learn no longer support Python 2. This will continue to grow rapidly: https://python3statement.org/

svenevs · 2019-07-28T03:42:01Z

- no longer support Python 3
+ no longer support Python 2

😉

sethrj · 2019-07-28T16:59:29Z

Also note that py-pil has been essentially superseded by Pillow, which does have Python 3 support and in fact will soon by dropping support for 2.7.

goxberry · 2019-08-12T21:27:26Z

@adamjstewart Elemental is unmaintained. There have been a couple maintenance patches here and there, but the author has explicitly stated he's not working on it anymore, and to the best of my knowledge, no one has stepped up to maintain it.

As far as I know, Hydrogen (as in, the Spack package) forks Elemental, but last I checked, Hydrogen strips out everything except BLAS level-1 primitives, and then adds GPU support. (I could be wrong about the specifics, but conversations with Hydrogen maintainers left me with this overall impression.) I think dropping support for Elemental during a migration to Python 3 would be an entirely reasonable position by the Spack maintainers, but I don't know how many people would be affected by such a change, and how much it would affect them.

adamjstewart · 2019-08-12T21:42:59Z

I think dropping support for Elemental during a migration to Python 3 would be an entirely reasonable position by the Spack maintainers, but I don't know how many people would be affected by such a change, and how much it would affect them.

Just to clarify, a switch to Python 3 by default won't drop support for anything. Users can still specify spack install elemental ^python@:2 until the concretizer is fixed. Once the concretizer is fixed, users won't even notice that the new default is Python 3.

adamjstewart · 2019-08-12T21:47:00Z

Can we hold off until we can ensure that a decent subset of the python 3 packages build fine as binaries?

@tgamblin What do you consider to be a decent subset? Most core Python libraries (numpy, scipy, matplotlib, pandas, sklearn) only support Python 3 at this point. I use all of these in my research and haven't run into any problems so far. I'm willing to test a larger subset if that's what it takes to get this PR merged.

adamjstewart · 2019-10-20T02:34:22Z

Python 3.8.0 was recently released. I've been playing around with it, and it's horribly buggy so far (see #13256). Based on this, and the fact that Anaconda and Homebrew still don't offer Python 3.8 support, I've decided to switch the default to Python 3.7 instead of Python 3. I propose we move in lockstep with Anaconda, as they are kind of the authority on Python distributions these days.

citibeth · 2019-10-20T02:35:36Z

Sounds good to me

…

On Sat, Oct 19, 2019 at 22:34 Adam J. Stewart ***@***.***> wrote: Python 3.8.0 was recently released. I've been playing around with it, and it's horribly buggy so far (see #13256 <#13256>). Based on this, and the fact that Anaconda <https://www.anaconda.com/distribution/#download-section> and Homebrew <https://github.com/Homebrew/homebrew-core/blob/master/Formula/python.rb> still don't offer Python 3.8 support, I've decided to switch the default to Python 3.7 instead of Python 3. I propose we move in lockstep with Anaconda, as they are kind of the authority on Python distributions these days. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10319?email_source=notifications&email_token=AAOVY53ILXXF5GBID2DIFN3QPO7TJA5CNFSM4GPRCZPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBYA45A#issuecomment-544214644>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOVY53T3DCVFBVF5URJ7UDQPO7TJANCNFSM4GPRCZPA> .

adamjstewart · 2019-10-21T00:38:56Z

We're finally seeing progress on fixing the Python build for the Intel compilers! python/cpython#16717

If this patch works, we can remove the conflict and this PR should be ready to merge.

adamjstewart · 2019-10-21T00:49:58Z

Can we hold off until we can ensure that a decent subset of the python 3 packages build fine as binaries?

@tgamblin can you define a decent subset? I've been using the following packages with Python 3.7.4 without any problems:

$ spack find
==> 200 installed packages
-- darwin-catalina-ivybridge / [email protected] ---------------
[email protected]              [email protected]    [email protected]                 [email protected]        [email protected]                      [email protected]
[email protected]         [email protected]       [email protected]                    [email protected]                 [email protected]                        [email protected]
[email protected]          [email protected]        [email protected]                  [email protected]          [email protected]                         [email protected]
[email protected]        [email protected]_12        [email protected]                     [email protected]  [email protected]                     [email protected]
[email protected]               [email protected]        [email protected]            [email protected]       [email protected]                  [email protected]
[email protected]          [email protected]         [email protected]                  [email protected]            [email protected]                 [email protected]
[email protected]            [email protected]      [email protected]          [email protected]            [email protected]                 [email protected]
[email protected]           [email protected]      [email protected]                [email protected]         [email protected]                   [email protected]
[email protected]            [email protected]     [email protected]                 [email protected]               [email protected]                    [email protected]
[email protected]         [email protected]    [email protected]                 [email protected]         [email protected]                  [email protected]
[email protected]         [email protected]        [email protected]            [email protected]         [email protected]                [email protected]
[email protected]           [email protected]  [email protected]              [email protected]             [email protected]                          [email protected]
[email protected]       [email protected]         [email protected]           [email protected]               [email protected]                         [email protected]
[email protected]  [email protected]        [email protected]                 [email protected]     [email protected]                         [email protected]
[email protected]          [email protected]      [email protected]               [email protected]               [email protected]                      [email protected]
[email protected]            [email protected]       [email protected]                   [email protected]    [email protected]                  [email protected]
[email protected]             [email protected]        [email protected]                   [email protected]            [email protected]                          [email protected]
[email protected]        [email protected]        [email protected]              [email protected]            [email protected]                    [email protected]
[email protected]              [email protected]       [email protected]               [email protected]             [email protected]                 [email protected]
[email protected]             [email protected]        [email protected]              [email protected]             [email protected]                           [email protected]
[email protected]            [email protected]    [email protected]             [email protected]           [email protected]                [email protected]
[email protected]         [email protected]            [email protected]               [email protected]            [email protected]                         [email protected]
[email protected]            [email protected]            [email protected]             [email protected]             [email protected]               [email protected]
[email protected]              [email protected]      [email protected]       [email protected]                 [email protected]        [email protected]
[email protected]       [email protected]       [email protected]             [email protected]            [email protected]          [email protected]
[email protected]      [email protected]            [email protected]               [email protected]             [email protected]         [email protected]
[email protected]        [email protected]           [email protected]                [email protected]                 [email protected]           [email protected]
[email protected]             [email protected]         [email protected]  [email protected]        [email protected]           [email protected]
[email protected]            [email protected]          [email protected]                 [email protected]           [email protected]
[email protected]       [email protected]          [email protected]            [email protected]           [email protected]
[email protected]       [email protected]       [email protected]              [email protected]           [email protected]
[email protected]               [email protected]       [email protected]               [email protected]           [email protected]

In addition, the following packages ONLY support Python 3: https://python3statement.org/

tgamblin · 2019-10-21T05:32:21Z

@adamjstewart: that looks like a decent subset to me. I agree with picking 3.7 instead of 3.8.

The only other question, which I had for @scheibelp and I believe he fixed in #13243, is whether things will still continue to work as well as can be expected if you prefer python 2.7 in packages.yaml. I'm pretty sure that will continue working fine.

If you agree, I'm totally fine with this and think it's the right way to go.

adamjstewart · 2019-10-21T15:55:57Z

@tgamblin I've been preferring 3.7 in my packages.yaml for a couple years now, so I imagine preferring 2.7 should work the same. The only problem is that the concretizer still isn't capable of determining the latest version of numpy/scipy/matplotlib/etc that supports Python 2.7.

Let me test python/cpython#16717 first. If that patch works, I'll add it to this PR and we can finally merge this!

tgamblin · 2019-10-21T16:53:43Z

The only problem is that the concretizer still isn't capable of determining the latest version of numpy/scipy/matplotlib/etc that supports Python 2.7.

Exactly - I believe this is what #13243 was meant to fix. @scheibelp? I am hoping to have an "experimental" new concretizer in sometime this month so that people can start trying it when they have these issues.

scheibelp · 2019-10-21T18:44:31Z

The only problem is that the concretizer still isn't capable of determining the latest version of numpy/scipy/matplotlib/etc that supports Python 2.7.

Exactly - I believe this is what #13243 was meant to fix. scheibelp?

I don't think so: If you want to support something like

spack spec py-numpy ^python@:2

and you want Spack to choose the latest version of py-numpy that can build against Python 2.x, then #13243 will not help the concretizer find the appropriate constraints. The result would most likely be a failure, since those packages will need to encode constraints like

depends_on(python@3:, when=@x:)

I didn't consider this a major problem since a user could explicitly specify the version of py-numpy that they want to build along with Python 2.x like

spack spec py-numpy@x ^python@:2

As for what #13243 fixes: if a package only needs some dependencies for Python 2.x versions, #13243 fixes a bug where such constraints were not respected. I think that was a more serious bug because it led Spack to add dependencies that weren't needed (i.e. the concretization completed without errors but produced an incorrect DAG).

So my question is: do we consider it a blocker that users who want py-numpy with python 2.x will have to specify a particular version? I think if so, we could recommend a configuration that sets preferences on some py-* packages that allows them to build with Python 2.x

citibeth · 2019-10-21T19:37:19Z

So my question is: do we consider it a blocker that users who want py-numpy with python 2.x will have to specify a particular version? I think if so, we could recommend a configuration that sets preferences on some py-* packages that allows them to build with Python 2.x

Maybe it's time to use the concretizer workaround I proposed earlier with the `+python2` variant?

…

scheibelp · 2019-10-21T20:01:06Z

So my question is: do we consider it a blocker that users who want py-numpy with python 2.x will have to specify a particular version? I think if so, we could recommend a configuration that sets preferences on some py-* packages that allows them to build with Python 2.x

Maybe it's time to use the concretizer workaround I proposed earlier with the +python2 variant?

I think you're referring to #11468 and #7926? Some of the changes I see in #7926 e.g.

PyMatplotlib():
...
    depends_on('py-subprocess32', type=('build', 'run'), when='~python3')
    depends_on('py-functools32', type=('build', 'run'), when='~python3')

are resolved by #13243 e.g.

PyMatplotlib():
...
    depends_on('py-subprocess32', type=('build', 'run'), when='^python@:2')
    depends_on('py-functools32', type=('build', 'run'), when='^python@:2')

To support spack spec py-numpy ^python@:2 more-smoothly (and again I think it's a matter of debate whether it is considered a blocker at all) I think a specialized config file could be added to Spack's etc/ directory to select all the appropriate preferred python versions for Python 2.x (and the user could activate it with spack -C etc/py2-config/ ...).

adamjstewart · 2019-10-21T20:31:27Z

So my question is: do we consider it a blocker that users who want py-numpy with python 2.x will have to specify a particular version? I think if so, we could recommend a configuration that sets preferences on some py-* packages that allows them to build with Python 2.x

So you're asking if the fact that this fails:

$ spack install py-numpy ^python@:2

should be a blocker to merging this PR? What about the fact that without this PR, the following fails:

$ spack install py-numpy

I consider that a much more serious issue.

scheibelp · 2019-10-21T20:47:24Z

What about the fact that without this PR, the following fails:
$ spack install py-numpy
I consider that a much more serious issue.

I agree with you.

You made a statement earlier:

The only problem is that the concretizer still isn't capable of determining the latest version of numpy/scipy/matplotlib/etc that supports Python 2.7.

I assume you were implying that the following would fail to concretize with this PR:

spack spec py-numpy ^python@:2

which I think is true but I also don't think it's a blocker. I was asking if other people thought it was. In particular I wanted to ask because in #10319 (comment) it was asked whether #13243 would resolve this problem; it does not, so the issue you brought up would remain an issue with the PR (but again I don't think it's a blocker).

adamjstewart · 2019-10-21T20:50:13Z

Gotcha. I personally don't consider it a blocker, since you already have to specify the correct version of numpy that supports Python 2 with or without this PR. I think we can wait on the new concretizer to resolve that issue.

tgamblin · 2019-10-21T21:02:42Z

I agree completely with:

What about the fact that without this PR, the following fails:
$ spack install py-numpy
I consider that a much more serious issue.

and:

I personally don't consider it a blocker, since you already have to specify the correct version of numpy that supports Python 2 with or without this PR.

@adamjstewart: Let's go ahead and merge this.

adamjstewart · 2019-10-22T05:40:15Z

The patch worked! I was able to successfully installs Python 3.7.4 with Intel 18.0.3.222 on Cray CNL5. I think this PR is finally ready to merge as soon as Travis passes.

adamjstewart · 2019-10-22T15:25:21Z

Thanks for everyone's help in getting this PR merged!

If you still rely on packages that require Python 2, please encourage the developers to update to Python 3. If the package should support Python 3 but has build problems, let me know and I'll take a look. If you want to restore the behavior before this PR, make the following change to your packages.yaml:

packages:
  python:
    version: [':2']

R.I.P. Python 2, long live Python 3!

* Default to Python 3 * Fix build with Intel compilers

adamjstewart added the python3 label Jan 11, 2019

citibeth changed the title ~~Default to Python 3~~ Default to Python3 Jan 15, 2019

adamjstewart mentioned this pull request Feb 1, 2019

Add Python 3.7.2 #10491

Closed

adamjstewart force-pushed the packages/python3 branch from 72fa411 to 9770b31 Compare May 7, 2019 18:43

citibeth mentioned this pull request May 15, 2019

Python3 Migration Strategy #11468

Closed

This was referenced May 23, 2019

package requires python version 3:, but spec asked for 2.7.16 #11531

Closed

default python preference for packages.yaml #11539

Closed

scheibelp mentioned this pull request Jun 18, 2019

Automatically use Python 3 if available #11783

Closed

adamjstewart force-pushed the packages/python3 branch from 347b435 to 3d2054c Compare July 22, 2019 03:27

adamjstewart mentioned this pull request Jul 26, 2019

Add conflicts statement for Python-3.6.7+ and Intel compiler #12145

Merged

adamjstewart mentioned this pull request Oct 11, 2019

specify preinstalled numpy in ~/.spack/packages.yaml #13190

Closed

adamjstewart mentioned this pull request Oct 20, 2019

Tut cannot be built with Python 3.7+ #13341

Merged

adamjstewart force-pushed the packages/python3 branch from 3d2054c to efbe0f4 Compare October 20, 2019 02:31

adamjstewart changed the title ~~Default to Python3~~ Default to Python 3.7 Oct 20, 2019

adamjstewart force-pushed the packages/python3 branch from efbe0f4 to b4148fb Compare October 21, 2019 02:11

Default to Python 3

faf75c9

adamjstewart force-pushed the packages/python3 branch from b4148fb to faf75c9 Compare October 21, 2019 21:14

Fix build with Intel compilers

b9cf69a

adamjstewart merged commit 526ee11 into spack:develop Oct 22, 2019

adamjstewart deleted the packages/python3 branch October 22, 2019 15:22

healther mentioned this pull request Oct 24, 2019

Should we fetch python sources exclusively from pypi? #13427

Closed

jrmadsen pushed a commit to jrmadsen/spack that referenced this pull request Oct 30, 2019

Default to Python 3.7 (spack#10319)

6ec52b4

* Default to Python 3 * Fix build with Intel compilers

Default to Python 3.7 #10319

Default to Python 3.7 #10319

Conversation

adamjstewart commented Jan 11, 2019 • edited Loading

tgamblin commented Jan 13, 2019

citibeth commented Jan 13, 2019 via email

citibeth commented Jan 15, 2019

citibeth commented Jan 15, 2019 • edited Loading

adamjstewart commented Jan 15, 2019

citibeth commented Jan 15, 2019 via email

citibeth commented Mar 6, 2019

baberlevi commented May 23, 2019 • edited Loading

citibeth commented May 24, 2019

adamjstewart commented Jul 15, 2019

adamjstewart commented Jul 22, 2019

adamjstewart commented Jul 22, 2019 • edited Loading

citibeth commented Jul 22, 2019 via email

adamjstewart commented Jul 26, 2019

adamjstewart commented Jul 28, 2019 • edited Loading

svenevs commented Jul 28, 2019

sethrj commented Jul 28, 2019

goxberry commented Aug 12, 2019

adamjstewart commented Aug 12, 2019

adamjstewart commented Aug 12, 2019

adamjstewart commented Oct 20, 2019

citibeth commented Oct 20, 2019 via email

adamjstewart commented Oct 21, 2019

adamjstewart commented Oct 21, 2019

tgamblin commented Oct 21, 2019

adamjstewart commented Oct 21, 2019

tgamblin commented Oct 21, 2019

scheibelp commented Oct 21, 2019 • edited Loading

citibeth commented Oct 21, 2019 via email

scheibelp commented Oct 21, 2019

adamjstewart commented Oct 21, 2019

scheibelp commented Oct 21, 2019 • edited Loading

adamjstewart commented Oct 21, 2019

tgamblin commented Oct 21, 2019

adamjstewart commented Oct 22, 2019

adamjstewart commented Oct 22, 2019

adamjstewart commented Jan 11, 2019 •

edited

Loading

citibeth commented Jan 15, 2019 •

edited

Loading

baberlevi commented May 23, 2019 •

edited

Loading

adamjstewart commented Jul 22, 2019 •

edited

Loading

adamjstewart commented Jul 28, 2019 •

edited

Loading

scheibelp commented Oct 21, 2019 •

edited

Loading

scheibelp commented Oct 21, 2019 •

edited

Loading