Installation#

PyNX supports python version 3.8 and above.

You should either follow the quick install instructions, or the more complete installation ones, which will both create a conda environment with PyNX and the required libraries.

Installation has been tested on linux (e.g. ubuntu >=20.04) systems as well as macOS computers. See below the notes regarding installation on windows computers.

Generic Instructions#

PyNX is focused on using Graphical Processing Units (GPU) for faster calculations, so you will need:

  • a GPU (which can be an integrated GPU)

  • an OpenCL installation (drivers and libraries)

  • and/or CUDA (which gives better performance), which requires CUDA drivers and development tools (nvcc, nvrtc library)

PyNX should still work on a CPU only, but without any optimisation, and will therefore be very slow, especially for 3D CDI and Ptychography algorithms. Also, not all features are available in the CPU version, so the CPU version is mostly intended for basic testing and educational purposes.

The requirements for PyNX are listed in the source distribution as a conda definition file conda-environment.yaml (link).

Note regarding conda(-forge) and mamba#

PyNX installation should be done using conda rather than regular python virtual environments (though these still work for linux & macOS, but are less convenient).

We recommend using mamba, which is a faster, drop-in replacement for the conda command-line tool, to install the pynx environment. The package resolution is much faster. Alternatively, recent versions of conda (>=2023.10) use the mamba library solver so should benefit from the same speedup. (the bottom line is that you should be able to use conda install or mamba install indifferently in the following).

Most packages used by pynx are available through conda-forge. To activate it, we recommend using the Mamba-forge conda distribution. Alternatively, you can also using a vanilla conda installation, and activate the conda-forge channel by default (see the conda-forge link) so that all dependencies are available.

Quick installation using conda#

This allows to install PyNX quickly, assuming that you already have:

  • installed conda with conda-forge using the mamba-forge distribution

  • all the GPU development tools (opencl, nvcc and cuda libraries) available on your system.

The following command will only install base packages, without options for MPI or jupyter, in a conda environement named pynx-py312-env, using python 3.12.

conda create -n pynx-py312-env python=3.12 "numpy<2" scipy pooch matplotlib scikit-image \
      scikit-learn ipython h5py hdf5plugin h5glance silx-base fabio psutil mako numexpr \
      cython ocl-icd-system pytest "pytools<=2024.1.3"
conda activate pynx-py312-env
pip install pyopencl pycuda pyvkfft nabu tomoscan nxtomo --no-cache-dir
pip install https://gitlab.esrf.fr/favre/PyNX/-/archive/devel/PyNX-devel.tar.gz

Notes:

  • Remove pycuda if you do not use an nvidia card.

  • We are installing pycuda, pyopencl and pyvkfft using pip as it is easier to get the right version than with conda. The --no-cache-dir option is to make sure that no other pre-compiled version is installed (for example compiled against a different cuda version).

  • if you rather install pycuda and pyvkfft using conda, you can specify the desired cuda version by adding e.g. cuda-version=12.3. This may be necessary to match your installation (the cuda toolkit version must support your GPU card and be <= to your installed cuda driver version).

Once this has been installed, you can test pynx.

Complete installation (with MPI and jupyter tools)#

The following script should work on any POSIX (Linux, macOS) system, and will also install the MPI optional libraries, jupyter tools, and libraries for tomography. This also includes tools to generate the pynx documentation.

The simplest installation uses the conda-environment.yaml file available from this link , with the following command:

mamba env create --prefix test-env --file conda-environment.yaml

Alternatively, you can do this manually with:

conda create -n pynx-py312-env python=3.12 "numpy<2" scipy pooch matplotlib scikit-image \
      scikit-learn ipython h5py hdf5plugin silx-base fabio psutil mako numexpr \
      cython jupyterlab jupyterlab-h5web h5glance ocl-icd-system ipympl pyqt pyopengl \
      ipyvolume ipyfilechooser sphinx nbsphinx nbsphinx-link sphinx-argparse \
      mpi4py pydata-sphinx-theme pytest pythreejs voila siphash24 tifffile
conda activate pynx-py312-env
pip install pyopencl pycuda pyvkfft nabu~=2024.2 tomoscan nxtomo --no-cache-dir
pip install https://github.com/lebedov/scikit-cuda/archive/refs/heads/master.zip --no-cache-dir
pip install https://gitlab.esrf.fr/favre/PyNX/-/archive/devel/PyNX-devel.tar.gz --no-cache-dir

Notes:

  • Remove pycuda if you do not use an nvidia card.

  • if you want to use cuda, make sure the proper tools (nvcc and the nvrtc library) are available in the path before installing pycuda and pyvkfft.

  • On macOS, remove pycuda and replace ocl-icd-system by ocl_icd_wrapper_apple

  • to use jupyter-hub (e.g. at ESRF), also include jupyterhub~=4.1.5 batchspawner~=1.3.0

Once this has been installed, you can test pynx

Windows installation on a PC with an nVidia card#

Windows is not officially supported, as it can be complex to get the correct environment tools to make PyNX work. You are therefore encouraged to use PyNX on a linux workstation or cluster node, or on a macOS machine.

Nevertheless the following was found to work on a PC with Windows 10 and an nVidia GTX 1080:

With all this PyNX runs correctly with CUDA in a notebook or ipython console.

Notes:

  • on-the-fly compilation of GPU kernels is significantly slower on windows than on linux and macOS, so when running a script or notebook, the first time the kernels are compiled (which happens transparently), the process may seem to hang. Once this has been done the optimisation will run at the same speed as on other platforms, depending on the GPU.

Development version#

If you want to test recent features, you can install the (public) development version. This branch is automatically tested using continuous integration, so is usually safe to use even if there can be some changes in the API and command-line options. You can also check the status of the CI pipelines.

It can be installed directly (make sure you do this in an environement where all the requirements are already available) using:

pip install https://gitlab.esrf.fr/favre/PyNX/-/archive/devel/PyNX-devel.tar.bz2

Other branches can be installed in a similar way.

Testing the installation#

Once installed, you can use:

pynx-info

which will tell you the version of pynx installed, and if you have support for cuda and/or opencl.

You can also test your support for cuda & opencl using:

pyvkfft-info

You can then test pynx from the console by using:

pynx-test

Note that this takes a long time (30-40’), as it goes through a large selection of options, notably for the CDI and Ptycho command-line scripts.

You can also run more specific tests using command-line keywords (combinations are possible):

  • pynx-test processing_unit : only run basic OpenCL and CUDA tests

  • pynx-test cdi : only run CDI tests

  • pynx-test cdi_runner : only run CDI runner tests

  • pynx-test ptycho : only run ptychography tests

  • pynx-test ptycho_runner : only run ptychography runner tests

  • pynx-test --cuda : only run CUDA tests

  • pynx-test --opencl : only run opencl tests

Use pynx-test --help for help on this script.