spack

参考：

官方文档 https://spack.readthedocs.io/en/latest/

https://docs.zjusct.io/operation/software/spack/

juejin.cn/post/7038836266158260238

Spack 入门指南

https://packages.spack.io/

https://docs.easybuild.io/

https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs

使用¶

$ spack bootstrap

查看支持spack安装的所有软件

$ spack list

模糊查看，如所有包含 blas 的软件

$ spack list *blas*
batchedblas  blast-legacy  cblas     flexiblas   hipblas-common     liblas           netlib-xblas  psblas     samblaster
blaspp       blast-plus    clblast   graphblast  hipblaslt          ncbi-magicblast  nvpl-blas     py-liblas  xtensor-blas
blast        blast2go      cublasmp  hipblas     libblastrampoline  ncbi-rmblastn    openblas      rocblas
==> 26 packages

查看软件的详细信息，如软件说明、支持的版本、编译方式等

$ spack info samtools
Package:   samtools

Description:
    SAM Tools provide various utilities for manipulating alignments in the
    SAM format, including sorting, merging, indexing and generating
    alignments in a per-position format

Homepage: https://www.htslib.org

Preferred version:  
    1.21      https://github.com/samtools/samtools/releases/download/1.21/samtools-1.21.tar.bz2

Safe versions:  
    1.21      https://github.com/samtools/samtools/releases/download/1.21/samtools-1.21.tar.bz2
    1.19.2    https://github.com/samtools/samtools/releases/download/1.19.2/samtools-1.19.2.tar.bz2
    1.19      https://github.com/samtools/samtools/releases/download/1.19/samtools-1.19.tar.bz2
    1.18      https://github.com/samtools/samtools/releases/download/1.18/samtools-1.18.tar.bz2
    1.17      https://github.com/samtools/samtools/releases/download/1.17/samtools-1.17.tar.bz2
    1.16.1    https://github.com/samtools/samtools/releases/download/1.16.1/samtools-1.16.1.tar.bz2
    1.15.1    https://github.com/samtools/samtools/releases/download/1.15.1/samtools-1.15.1.tar.bz2
    1.15      https://github.com/samtools/samtools/releases/download/1.15/samtools-1.15.tar.bz2
    1.14      https://github.com/samtools/samtools/releases/download/1.14/samtools-1.14.tar.bz2
    1.13      https://github.com/samtools/samtools/releases/download/1.13/samtools-1.13.tar.bz2
    1.12      https://github.com/samtools/samtools/releases/download/1.12/samtools-1.12.tar.bz2
    1.11      https://github.com/samtools/samtools/releases/download/1.11/samtools-1.11.tar.bz2
    1.10      https://github.com/samtools/samtools/releases/download/1.10/samtools-1.10.tar.bz2
    1.9       https://github.com/samtools/samtools/releases/download/1.9/samtools-1.9.tar.bz2
    1.8       https://github.com/samtools/samtools/releases/download/1.8/samtools-1.8.tar.bz2
    1.7       https://github.com/samtools/samtools/releases/download/1.7/samtools-1.7.tar.bz2
    1.6       https://github.com/samtools/samtools/releases/download/1.6/samtools-1.6.tar.bz2
    1.5       https://github.com/samtools/samtools/releases/download/1.5/samtools-1.5.tar.bz2
    1.4       https://github.com/samtools/samtools/releases/download/1.4/samtools-1.4.tar.bz2
    1.3.1     https://github.com/samtools/samtools/releases/download/1.3.1/samtools-1.3.1.tar.bz2
    1.2       https://github.com/samtools/samtools/releases/download/1.2/samtools-1.2.tar.bz2
    0.1.8     https://github.com/samtools/samtools/archive/0.1.8.tar.gz

Deprecated versions:  
    None

Variants:
    build_system [generic]        generic
        Build systems supported by the package

Build Dependencies:
    c  cxx  gmake  htslib  ncurses  zlib-api

Link Dependencies:
    htslib  ncurses  zlib-api

Run Dependencies:
    perl  python

Licenses: 
    MIT

查看软件的 spec 及其依赖的 spec

$ spack  spec samtools
 -   samtools@1.21 build_system=generic platform=linux os=openeuler22 target=aarch64 %c,cxx=gcc@10.3.1
 -       ^compiler-wrapper@1.0 build_system=generic platform=linux os=openeuler22 target=aarch64 
[e]      ^gcc@10.3.1~binutils+bootstrap~graphite~nvptx~piclibs~profiled~strip build_system=autotools build_type=RelWithDebInfo languages:='c,c++,fortran' patches:=0d13622,2c18531,b5e049d,cc6112d platform=linux os=openeuler22 target=aarch64 
 -       ^gcc-runtime@10.3.1 build_system=generic platform=linux os=openeuler22 target=aarch64 
[e]      ^glibc@2.34 build_system=autotools platform=linux os=openeuler22 target=aarch64 
[e]      ^gmake@4.3~guile build_system=generic patches:=599f134 platform=linux os=openeuler22 target=aarch64 
 -       ^htslib@1.21+gcs+libcurl+libdeflate+pic~plugins+s3 build_system=autotools platform=linux os=openeuler22 target=aarch64 %c,cxx=gcc@10.3.1
 -           ^bzip2@1.0.8~debug~pic+shared build_system=generic platform=linux os=openeuler22 target=aarch64 %c=gcc@10.3.1
 -               ^diffutils@3.12 build_system=autotools platform=linux os=openeuler22 target=aarch64 %c=gcc@10.3.1
 -                   ^libiconv@1.18 build_system=autotools libs:=shared,static platform=linux os=openeuler22 target=aarch64 %c=gcc@10.3.1
 -           ^curl@8.15.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs:=shared,static tls:=openssl platform=linux os=openeuler22 target=aarch64 %c,cxx=gcc@10.3.1
 -               ^nghttp2@1.65.0 build_system=autotools platform=linux os=openeuler22 target=aarch64 %c,cxx=gcc@10.3.1
 -           ^gnuconfig@2024-07-27 build_system=generic platform=linux os=openeuler22 target=aarch64 
 -           ^libdeflate@1.18~ipo build_system=cmake build_type=Release generator=make platform=linux os=openeuler22 target=aarch64 %c=gcc@10.3.1
[e]              ^cmake@3.22.0~doc+ncurses+ownlibs~qtgui build_system=generic build_type=Release patches:=dbc3892,fdea723 platform=linux os=openeuler22 target=aarch64 
 -               ^gzip@1.13 build_system=autotools platform=linux os=openeuler22 target=aarch64 %c=gcc@10.3.1
 -           ^openssl@3.4.1~docs+shared build_system=generic certs=mozilla platform=linux os=openeuler22 target=aarch64 %c,cxx=gcc@10.3.1
[+]              ^ca-certificates-mozilla@2025-08-12 build_system=generic platform=linux os=openeuler22 target=aarch64 
 -           ^xz@5.6.3~pic build_system=autotools libs:=shared,static platform=linux os=openeuler22 target=aarch64 %c=gcc@10.3.1
 -       ^ncurses@6.5-20250705~symlinks+termlib abi=none build_system=autotools patches:=7a351bc platform=linux os=openeuler22 target=aarch64 %c,cxx=gcc@10.3.1
 -           ^pkgconf@2.5.1 build_system=autotools platform=linux os=openeuler22 target=aarch64 %c=gcc@10.3.1
 -       ^perl@5.42.0+cpanm+opcode+open+shared+threads build_system=generic platform=linux os=openeuler22 target=aarch64 %c=gcc@10.3.1
 -           ^berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches:=26090f4,b231fcc platform=linux os=openeuler22 target=aarch64 %c,cxx=gcc@10.3.1
 -           ^gdbm@1.25 build_system=autotools platform=linux os=openeuler22 target=aarch64 %c=gcc@10.3.1
 -               ^readline@8.3 build_system=autotools patches:=21f0a03 platform=linux os=openeuler22 target=aarch64 %c=gcc@10.3.1
[e]      ^python@3.9.9+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl+tix+tkinter+uuid+zlib build_system=generic patches:=0d98e93,ebdca64,f2fd060 platform=linux os=openeuler22 target=aarch64 
 -       ^zlib-ng@2.2.4+compat+new_strategies+opt+pic+shared build_system=autotools platform=linux os=openeuler22 target=aarch64 %c,cxx=gcc@10.3.1

安装软件

# @ 后：软件版本，% 后：编译器（可自己装），arch=：强制架构（同集群节点可复用）
$ spack install gcc@12.2.0 %gcc@9.5.0 arch=linux-rocky8-zen3

卸载软件

$ spack  uninstall  hisat2
    -- linux-rocky8-icelake / no compilers --------------------------
    n62sowe hisat2@2.2.1

==> 1 packages will be uninstalled. Do you want to proceed? [y/N] y
==> Successfully uninstalled hisat2@2.2.1 build_system=generic platform=linux os=rocky8 target=icelake/n62sowe

查看已安装的软件

# 查看某个软件是否安装成功
$ spack find hisat2
-- linux-rocky8-icelake / no compilers --------------------------
hisat2@2.2.1
==> 1 installed package

# 查看所有安装成功的软件
$ spack find 
-- linux-openeuler22-aarch64 / no compilers ---------------------
bamtools@2.5.2                      cmake@3.22.0  glibc@2.34  py-htseq@2.0.9  py-wheel@0.45.1  python-venv@1.0
ca-certificates-mozilla@2025-08-12  gcc@10.3.1    gmake@4.3   py-pip@25.1.1   python@3.9.9

-- linux-rocky8-icelake / %c,cxx=gcc@10.3.1 ---------------------
berkeley-db@18.1.40  ncurses@6.5-20250705  zlib-ng@2.2.4

-- linux-rocky8-icelake / %c=gcc@10.3.1 -------------------------
bzip2@1.0.8  diffutils@3.12  gdbm@1.25  libiconv@1.18  perl@5.42.0  pkgconf@2.5.1  readline@8.3

-- linux-rocky8-icelake / no compilers --------------------------
compiler-wrapper@1.0  gcc@10.3.1  gcc-runtime@10.3.1  glibc@2.28  gmake@4.3  hisat2@2.2.1  python@3.9.9
==> 28 installed packages

# 查看直接安装的软件
$ spack find -x
-- linux-openeuler22-aarch64 / no compilers ---------------------
bamtools@2.5.2  py-htseq@2.0.9

-- linux-rocky8-icelake / no compilers --------------------------
hisat2@2.2.1
==> 3 installed packages

查看安装路径

$ spack location -i hisat2
/share/home/software/package/spack/opt/spack/hisat2/2.2.1

查看 package 依赖的 DAG

$ spack graph bamtools
o bamtools@2.5.2/5h3sywe
|\
| o cmake@3.22.0/yr3ua3e
| 
o gmake@4.3/xwax6aq

查看所有配置文件

$ spack config get

添加镜像

$ spack config edit mirrors

mirrors:
  cluster-public: file:///share/home/software/package/spack/pkg

查看镜像

$ spack mirror list
cluster-public [sb] file:///share/home/software/package/spack/pkg
spack-public   [s ] https://mirror.spack.io

配置文件¶

系统管理员和系统级 spack，配置文件建议存放在 $SPACK_ROOT/etc/spack 。这样既可以使得配置对所有用户生效，又不影响其他的 spack 安装。注意不要覆盖和修改改文件夹下 default 中的配置文件。

对于文件的修改，有两种模式，一种是直接修改对应的配置文件，另一种是通过 spack config edit 的方式来修改，config 的参数有以下加粗的几个，对应不同方向的配置内容。不过命令行这种方式，修改的文件实际上是 ~/.spack/ 中的配置文件。通过 spack config get 可以查看最终完整的配置。

配置文件中也可以出现一些 spack 提供的变量，主要的有 $spack 作为 spack 安装路径，$ user 作为当前用户等

自定义¶

配置文件¶

~/.spack/config.yaml 主要配置安装目录，同时去掉了目录中的编译器和哈希后缀。

~/.spack/config.yaml

config:
  install_tree:
    root: $spack/opt/spack
    projections:
      all: "{name}/{version}"
  module_roots:
    tcl:   $spack/modules   # Tcl

~/.spack/modules.yaml 主要配置 modulefile 文件，同时去掉了 modulefile 文件中的编译器和哈希后缀。

~/.spack/modules.yaml

modules:
  default:
    arch_folder: false
    tcl:          
      hash_length: 0                    # 可选：去掉哈希
      naming_scheme: '{name}/{version}'

自定义 repo¶

$ spack repo create my-repo myrepo
$ spack repo add my-repo/spack_repo/myrepo

$ tree my-repo
my-repo
└── spack_repo
    └── myrepo
        ├── packages
        └── repo.yaml

# 查看repo
$ spack repo list

Autotools¶

spack create -N myrepo -t autotools -n clustalw /home/username/package/spack/pkg/clustalw_2.1.tar.gz

生成软件安装模板代码 my-repo/spack_repo/myrep/packages/clustalw/package.py。

其中 -N 指定 repo，-t 指定编译模板，这里使用的 autotools

根据软件安装的实际步骤修改 package.py 代码，clustalw 是经典的三步安装法 configure->make->make install。需要用 sha256sum 计算安装包的哈希值填入代码中。

spack external find gmake cmake 使用系统的 make gcc 和 cmake。

~/.spack/packages.yaml

packages:
  gmake:
    externals:
    - spec: gmake@4.3
      prefix: /usr
    buildable: false
  cmake:
    externals:
    - spec: cmake@3.22.0
      prefix: /usr
    buildable: false
  gcc:
    externals:
    - spec: gcc@10.3.1 languages:='c,c++,fortran'
      prefix: /usr
      extra_attributes:
        compilers:
          c: /usr/bin/gcc
          cxx: /usr/bin/g++
          fortran: /usr/bin/gfortran
    buildable: false

使用 AutotoolsPackage 包，更多选项见 autotoolspackage

package.py

from spack_repo.builtin.build_systems.generic import Package

from spack.package import *

class Clustalw(AutotoolsPackage):
    """ClustalW: multiple sequence alignment program for DNA/protein."""

    homepage = "http://www.clustal.org/clustal2/"
    url = "file:///home/username/spack/pkg/clustalw_2.1.tar.gz"

    version("2.1", sha256='e052059b87abfd8c9e695c280bfba86a65899138c82abccd5b00478a80f49486')

使用 Package 包

package.py

from spack_repo.builtin.build_systems.generic import Package

from spack.package import *

class Clustalw(Package):
    """ClustalW: multiple sequence alignment program for DNA/protein."""

    homepage = "http://www.clustal.org/clustal2/"
    url = "file:///home/username/spack/pkg/clustalw_2.1.tar.gz"

    version("2.1", sha256='e052059b87abfd8c9e695c280bfba86a65899138c82abccd5b00478a80f49486')

    depends_on('gmake', type='build')

    def install(self, spec, prefix):
        configure('--prefix={0}'.format(prefix))   # ./configure --prefix=$prefix
        make()
        make("install")

make¶

hifiasm 有 Makefile 文件，可以直接 make 安装。

package.py

from spack_repo.builtin.build_systems.generic import Package

from spack.package import *

class Hifiasm(Package):
    homepage = "https://github.com/hifiasm/hifiasm"
    url      = "file:///home/username/spack/pkg/hifiasm_0.25.0.tar.gz"

    version("0.25.0", sha256="51633138865207a9d41630da9377d46e4921ad4fc5facaa1740ceccae8611f1f")

    depends_on('gmake', type='build')

    def install(self, spec, prefix):
        make()
        # 把可执行文件拷到 prefix.bin
        mkdirp(prefix.bin)
        install('hifiasm', prefix.bin)

cmake¶

spack create -b -N myrepo -t cmake -n bamtools /home/username/spack/pkg/bamtools-2.5.2.tar.gz

生成软件安装模板代码 my-repo/spack_repo/myrep/packages/bamtools/package.py。

其中 -N 指定 repo，-t 指定编译模板，这里使用的 cmake

根据软件安装的实际步骤修改 package.py 代码。需要用 sha256sum 计算安装包的哈希值填入代码中。

from spack_repo.builtin.build_systems.cmake import CMakePackage

from spack.package import *


class Bamtools(CMakePackage):
    """FIXME: Put a proper description of your package here."""

    homepage = "https://www.example.com"

    license("UNKNOWN", checked_by="github_user1")

    version("2.5.2", sha256="4d8b84bd07b673d0ed41031348f10ca98dd6fa6a4460f9b9668d6f1d4084dfc8", url = "file:///home/username/spack/pkg/bamtools-2.5.2.tar.gz")

    # FIXME: Add dependencies if required.
    # depends_on("foo")

    def cmake_args(self):
        # FIXME: Add arguments other than
        # FIXME: CMAKE_INSTALL_PREFIX and CMAKE_BUILD_TYPE
        # FIXME: If not needed delete this function
        args = []
        return args

预编译软件¶

部分软件已经有预编译好的版本，可直接拷贝至安装目录，如 ncbi-blast、hisat2 等。

以blast为例，

spack create -b -N myrepo -t generic -n blast /home/username/spack/pkg/ncbi-blast-2.16.0+-x64-linux.tar.gz

更改后的 package.py 如下。

from spack_repo.builtin.build_systems.generic import Package

from spack.package import *


class Blast(Package):
    """FIXME: Put a proper description of your package here."""

    homepage = "https://blast.ncbi.nlm.nih.gov/"

    version("2.16.0", sha256 = "b0b13098c901d23b324ad1700e7471bb7408a7f7f517d74d5faad711be76e8f4", url ="file:///home/username/spack/pkg/ncbi-blast-2.16.0+-x64-linux.tar.gz")

    # FIXME: Add dependencies if required.
    # depends_on("foo")

    def install(self, spec, prefix):
        #make()
        #make("install")
        install_tree("bin", prefix.bin)

以 hisat2 为例，挑选部分编译好的二进制文件和脚本目录拷贝至安装目录。

spack create -b -N myrepo -t generic -n hisat2 /home/username/spack/pkg/hisat2-2.2.1-Linux_x86_64.zip

from spack_repo.builtin.build_systems.generic import Package

from spack.package import *

class Hisat2(Package):
    """FIXME: Put a proper description of your package here."""

    homepage = "https://daehwankimlab.github.io/hisat2/"

    # FIXME: Add proper versions here.
    version("2.2.1", sha256 = "ae53af930729787a126944f7db34d4065c06f589c4fb05f4cfa9a348cacd5cb4", url = "file:///home/username/spack/pkg/hisat2-2.2.1-Linux_x86_64.zip")

    # FIXME: Add dependencies if required.
    # depends_on("foo")

    def install(self, spec, prefix):
        # FIXME: Unknown build system
        #make()
        #make("install")
        install_tree("example", prefix.example)
        install_tree("scripts", prefix.scripts)

        mkdirp(prefix.bin)
        install("hisat2", prefix.bin)
        install("hisat2-align-s", prefix.bin)
        install("hisat2-align-l", prefix.bin)
        install("hisat2-build", prefix.bin)
        install("hisat2-build-s", prefix.bin)
        install("hisat2-build-l", prefix.bin)
        install("hisat2-inspect", prefix.bin)
        install("hisat2-inspect-s", prefix.bin)
        install("hisat2-inspect-l", prefix.bin)
        install("hisat2-repeat", prefix.bin)
        install("*.py", prefix.bin)

python¶

以 htseq 为例

bin/spack create -b -N myrepo -t python -n htseq

from spack_repo.builtin.build_systems.python import PythonPackage

from spack.package import *


class PyHtseq(PythonPackage):
    """FIXME: Put a proper description of your package here."""

    homepage = "https://pypi.org/project/HTSeq/"

    pypi = "HTSeq/htseq-2.0.9.tar.gz"
    license("UNKNOWN", checked_by="github_user1")

    # FIXME: Add proper versions here.
    version("2.0.9", sha256 = "3bbec23f033d35f40ab33a40c2b5c43f75e382c424b804c099dea635b52c2b12")
    depends_on("python@3.9", type=("build", "run"))

    # FIXME: Add additional dependencies if required.
    # depends_on("py-foo", type=("build", "run"))

    def config_settings(self, spec, prefix):
        settings = {}
        return settings

    def install(self, spec, prefix):
        pip = which('pip', required=True)
        # ① 只装 binary，不回落源码；② 禁用 deps 避免重复
        pip('install',
            '--no-deps',                    # 依赖由 Spack 负责
            '--only-binary', ':all:',       # 强制 wheel
            '--prefix', prefix,
            'numpy==1.22.2','pysam')        # 下载好的 wheel


        pip('install',
            '--no-deps',                    # 依赖由 Spack 负责
            '--only-binary', ':all:',       # 强制 wheel
            '--prefix', prefix,
            self.stage.archive_file)        # 下载好的 wheel

    def setup_run_environment(self, env):
        # 把依赖也加入同一 PYTHONPATH
        for dep in ('py-numpy', 'py-pandas', 'py-pysam', 'py-matplotlib'):
            env.prepend_path('PYTHONPATH',
                             join_path(self.spec[dep].prefix,
                                       'lib/python{0}/site-packages'.format(
                                           self.spec['python'].version.up_to(2))))

本文阅读量次
本站总访问量次