Learning Wire-Cell

# Install Wire-Cell Toolkit

There are several ways to install Wire-Cell Toolkit on your local computer (see, e.g. here (opens new window)). To avoid choice overloading and get you started as quickly as possible, we describe an officially recommended way of installation: using the cvmfs (opens new window) network file system and the singularity (opens new window) container. This way allows you to both develop Wire-Cell by itself and integrate it with other software frameworks such as LArSoft.

# Install CVMFS

CVMFS (opens new window) (CernVM File System) is a utility that mounts remote directories over HTTP. CVMFS provides a simple way to distribute software binaries. Follow Brett's instruction (opens new window) for details of installation. For the lazy ubuntu'ers:

sudo apt-get install lsb-release
wget https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest_all.deb
sudo dpkg -i cvmfs-release-latest_all.deb
sudo apt-get update
sudo apt-get install cvmfs cvmfs-config-default
sudo cvmfs_config setup
1
2
3
4
5
6

Create a configuration file at /etc/cvmfs/default.local as follows:

# required, add more if needed.
CVMFS_REPOSITORIES=larsoft.opensciencegrid.org,uboone.opensciencegrid.org,dune.opensciencegrid.org,icarus.opensciencegrid.org
# requires, replace with actual proxy, or just "DIRECT" if none
CVMFS_HTTP_PROXY="DIRECT"
# BNL Physics department users may use
# CVMFS_HTTP_PROXY="http://batch3.phy.bnl.gov:3128;DIRECT"

CVMFS_QUOTA_LIMIT=25000
CVMFS_CACHE_BASE=/mnt/ssd/cvmfs
1
2
3
4
5
6
7
8
9

Now test with

cvmfs_config probe
1

, and you should see three Probing ... OK messages.

(Optional) If you also want access to the data in dune.osgstorage.org, for example, the photon library in the protoDUNE-SP simulation as will be introduced shortly, it is required to append dune.osgstorage.org to CVMFS_REPOSITORIES in /etc/cvmfs/default.local. Check that the other repositories work:

cvmfs_config chksetup
cvmfs_config probe
1
2

The second command may fail on dune.osgstorage.org, we fix that next.

cp /cvmfs/config-osg.opensciencegrid.org/etc/cvmfs/domain.d/osgstorage.org.conf /etc/cvmfs/domain.d/
cvmfs_config chksetup
cvmfs_config probe
1
2
3

Now, dune.osgstorage.org should be "OK" and the directory should be accessible: /cvmfs/dune.osgstorage.org/

# Install Singularity

Singularity (opens new window) is a lightweight container that provides an operating-system level virtualization. It is popular in the world of scientific high-performance computing (HPC). For Wire-Cell development, we use singularity to provide a virtual scientific linux (opens new window) environment. Follow here (opens new window) for detailed installation instructions. For the lazy ubuntu'ers, first, install the go language:

sudo apt-get update && sudo apt-get install -y \
  build-essential libssl-dev uuid-dev \
  libgpgme11-dev squashfs-tools libseccomp-dev pkg-config
sudo wget https://dl.google.com/go/go1.11.5.linux-amd64.tar.gz # or other versions
sudo tar -C /usr/local -xf go1.11.5.linux-amd64.tar.gz
echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
    echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
    source ~/.bashrc
go get -u github.com/golang/dep/cmd/dep
1
2
3
4
5
6
7
8
9

Then, install singularity from source

go get -d github.com/sylabs/singularity
cd $GOPATH/src/github.com/sylabs/singularity
git checkout -b v3.0.3  v3.0.3 # or other tags
./mconfig && \
  make -C ./builddir && \
  sudo make -C ./builddir install
1
2
3
4
5
6

# Troubleshooting

Depending on the filesystem, singularity might not be able to mount certain drives and will abort with an error message similar to:

error: can't mount image /proc/self/fd/3: failed to mount squashfs filesystem: invalid argument
1

This has been patched from v3.5.3 onwards, but to force it to work on v3.0.3, one can modify the source code by opening:

$EDITOR ${GOPATH}/src/github.com/sylabs/singularity/internal/pkg/util/fs/mount/mount.go
1

and replacing the codeblock:

options = fmt.Sprintf("loop,offset=%d,sizelimit=%d,errors=remount-ro", offset, sizelimit)
1

with

options = fmt.Sprintf("loop,offset=%d,sizelimit=%d", offset, sizelimit)
if fstype == "ext3" {
        options += ",errors=remount-ro"
}
1
2
3
4

and rebuilding with the mconfig command as before. This is because remount-ro is an invalid argument to provide while mounting squashfs drives, as squashfs doesn't recognize it. More context is given here (opens new window)

# Use wcdo

wcdo is a command line tool that provides convenient methods to do stuff with Wire-Cell in a Singularity container. Follow Brett's instruction (opens new window) to get started, or see the next section for example workflows.

← Typical Workflows →