Skip to content

Changes

Page history
Updated Build (markdown) authored by TomTheBear's avatar TomTheBear
Show whitespace changes
Inline Side-by-side
Build.md
View page @ 319e1748
# How to build likwid
<wiki:toc max_depth="1" />
# Introduction
Likwid is build using GNU make and has no external dependencies apart from
the Linux kernel and the standard C library.
It should build on any Linux distribution with a recent gcc compiler
and 2.6 or newer kernel without any changes.
Likwid is build using GNU make and Perl. Besides the Linux kernel and the standard C library, all required dependencies are shipped with the archive ([Lua](http://www.lua.org/) and [hwloc](http://www.open-mpi.org/projects/hwloc/)). It should build on any Linux distribution with a recent GCC or CLANG compiler and 2.6 or newer Linux kernel without any changes.
There is one generic top level Makefile and one .mk configuration file for each
compiler (at the moment gcc and icc). Please note that I can only test LIKWID
with gcc. icc is only tested for basic functionality.
compiler (at the moment GCC, CLANG and ICC). Please note that we test LIKWID only with GCC. CLANG and ICC is only tested for basic functionality.
There is one exception: If you want to use LIKWID on a Intel Xeon Phi card you have
to choose the MIC as Compiler in config.mk, which is based on Intel icc compiler.
to choose the MIC as Compiler in `config.mk`, which is based on Intel ICC compiler.
# Directory structure
All source files are in the *src* directory. All header files are located in
*src/includes* . Each application main source files are in
*src/applications*.
*src/includes* . Lua application source files are in *src/applications*. All external tools, namely HWLOC and Lua, are located in *ext/*. The *bench/* folder contains all files of the benchmarking suite of LIKWID.
All build products are generated in the directory *./TAG*, where *TAG* is the
compiler configuration, default *./GCC*.
All build products are generated in the directory *./TAG*, where *TAG* is the compiler configuration, default *./GCC*.
# Configuration
Usually the only thing you have to configure is the *PREFIX* install path.
Usually the only thing you have to configure is the `PREFIX` install path in the build config file `config.mk` in the top directory.
## Changing color of likwid-pin output
## Changing color of [[likwid-pin|likwid-pin]] output
Depending on the background of your terminal window you can choose a color
for likwid-pin output.
for [[likwid-pin|likwid-pin]] output.
## Usage of accessDaemon
## Usage of access daemon [[likwid-accessD|likwid-accessD]]
Usually you can use LIKWID in the standard setup with direct access to the MSR
files. If you install LIKWID on a shared system as a HPC compute cluster you
may consider to use the accessDaemon. This is a proxy application which was
implemented with security in mind and performs address checks for allowed access.
Using the accessDaemon the measurements involve more overhead, especially if you use
likwid-perfctr in timeline mode or with the marker API.
Usually on your own system, you can use LIKWID with direct access to the MSR files. If you install LIKWID on a shared system as a HPC compute cluster you may consider to use the access daemon [[likwid-accessD|likwid-accessD]]. This is a proxy application which was implemented with security in mind and performs address checks for allowed access. Using the access daemon, the measurements involve more overhead, especially if you use [[likwid-perfctr|likwid-perfctr]] in timeline mode or with the marker API.
To enable using the accessDaemon:
To enable using the access daemon, configure in `config.mk`:
1. Set BUILDDAEMON to true
1. Configure the path to the accessDaemon binary (usually keep default)
1. Set the default ACCESSMODE
1. Set `BUILDDAEMON` to `true`
2. Configure the path to the access daemon binary at `ACCESSDAEMON`
3. Set the `ACCESSMODE` to `accessdaemon`
ACCESSMODE can be direct, accessdaemon and sysdaemon (not yet officially supported).
You can overwrite the default setting on the command line.
`ACCESSMODE` can be `direct`, `accessdaemon` and `sysdaemon` (not yet officially supported).
You can overwrite the default setting on the command line with the `-M` commandline option.
## Enable SandyBridge Uncore
If you want to access Uncore performance counters that are located in the PCI memory range, like they are implemented in Intel SandyBridge EP and IvyBridge EP, you have to use the access daemon or have root privileges because access to the PCI space is only permitted for highly privileged users.
This is only relevant if you also want to use LIKWID no SandyBridge. This
setting does not influence the usage of the same binary on other
microarchitectures. If you want to measure the Uncore counters on SandyBridge
you have to use the accesDaemon at the moment. If you need direct access you
can disable Uncore support for SandyBridge.
## Usage of frequency daemon [[likwid-setFreq|likwid-setFreq]]
The application [[likwid-setFrequencies|likwid-setFrequencies]] uses another daemon to modify the frequency of CPUs. The daemon is build and later installed if `BUILDFREQ` is set to `true` in `config.mk`.
## Build likwid marker API as shared library
## Build LIKWID as shared library
Per default the LIKWID library is build as a shared library. You need the library
is necessary if you want to use the Marker API. You can also use the LIKWID modules
directly. This is still not officially supported at the moment. In some settings it
is necessary to build LIKWID as a shared library. To do so set SHARED_LIBRARY to true.
Per default the LIKWID library is build as a shared library. You need the library if you want to use the Marker API. You can also use the LIKWID modules like `perfmon` directly. This is not officially supported at the moment. In some settings it is necessary to build LIKWID as a shared library. To do so set `SHARED_LIBRARY` to `true` in `config.mk`.
## Instrument likwid-bench for usage with likwid-perfctr
## Instrument [[likwid-bench|likwid-bench]] for usage with [[likwid-perfctr|likwid-perfctr]]
likwid-bench is instrumented for use with likwid-perfctr. This allows you to
measure various metrics of your likwid-bench kernels. Enable instrumentation by
setting INSTRUMENT_BENCH to true.
[[likwid-bench|likwid-bench]] is instrumented for use with [[likwid-perfctr|likwid-perfctr]]. This allows you to measure various metrics of your [[likwid-bench|likwid-bench]] kernels. Enable instrumentation by setting `INSTRUMENT_BENCH` to `true` in `config.mk`.
## Enabling Fortran interface for marker API
## Enabling Fortran interface for Marker API
If you want to use the Marker API in Fortran programs LIKWID offers a native
Fortran90 interface. To enable it uncomment he FORTRAN_INTERFACE line.
More information using LIKWID with other programming languages is
in OtherLanguages.
If you want to use the Marker API in Fortran programs, LIKWID offers a native Fortran90 interface. To enable it, set `FORTRAN_INTERFACE` to `true` in `config.mk`. More information using LIKWID with Fortran look at [[Marker API with Fortran90|TutorialMarkerF90]].
# Dependencies
Although we tried to minimize the external dependencies of LIKWID, some advanced tools or only specific tool options require external packages.
- [[likwid-perfscope|likwid-perfscope]] uses the Perl script [feedGnuplot](https://github.com/dkogan/feedgnuplot) to forward the real-time data to [gnuplot](http://www.gnuplot.info/). [feedGnuplot](https://github.com/dkogan/feedgnuplot) is included into LIKWID, but [gnuplot](http://www.gnuplot.info/) itself is not.
- [[likwid-agent|likwid-agent]] provided multiple backends to output the periodically measured data. The syslog backend requires the shell tool `logger` to be installed. The [RRD](https://oss.oetiker.ch/rrdtool/) backend requires `rrdtool` and the GMetric backend the `gmetric` tool, part of the [Ganglia Monitoring System](http://ganglia.sourceforge.net/).
- In order to create the HTML documentation of LIKWID, the tool [Doxygen](www.doxygen.org) is required.
# Build targets
You have to edit config.mk to configure your build and install path.
You have to edit `config.mk` to configure your build and install path.
The following make targets are available:
- **make** - Build all applications apart from likwid-bench
- **make** - Build everything
- **make likwid-bench** - Build likwid-bench
- **make clean** Remove the object file directory *./GCC*, keep the executables
- **make distclean** Remove all generated files
The build system has a working dependency tracking, therefore make clean is only
- **make likwid-accessD** - Build likwid-accessD
- **make likwid-setFreq** - Build likwid-setFreq
- **make docs** - Create HTML documentation using doxygen
- **make clean** - Remove the object file directory *./GCC*, keep the executables
- **make distclean** - Remove all generated files
- **make local** - Adjust paths in Lua scripts to work from the build directory. Requires the daemons to be already installed. Mainly used for testing.
The build system has a working dependency tracking, therefore **make clean** is only
needed if you change the Makefile configuration.
# Installing
NOTE: The pinning functionality and the !accessDaemon only works if configured in config.mk and
NOTE: The pinning functionality and the daemons only work if configured in `config.mk` and
installed with **make install**. If you do not use the pinning functionality the tools
can be used without installation.
......@@ -108,71 +91,29 @@ can be used without installation.
# Setting up access for hardware performance monitoring
Hardware performance monitoring on x86 is configured using model-specific
registers (MSR). MSR registers are special registers not part of the
instruction set architecture. To read and write to these registers the x86 ISA
provides special instructions. These instructions can only be executed in
protected mode or in other work only kernel code can execute these
instructions. Fortunately any Linux kernel 2.6 or newer provides access to these
registers via a set of device files. This allows to implement all of the
functionality in user space. Still it does not allow to use those more advanced
features of hardware performance monitoring which require to setup
interrupt service routines.
Per default only root has read/write access to these msr device files. In order
to use the LIKWID tools, which need access to these files (likwid-perfctr and
likwid-powermeter) as standard user you need to setup access rights to these
files.
likwid-perfctr, likwid-powermeter and likwid-features require the Linux msr
kernel module. This module is part of most standard distro kernels. You have to
be root to do the initial setup.
1. Check if the msr module is loaded with *lsmod | grep msr* . There should be an output.
2. It the module is not loaded load it with *modprobe msr* . For automatic loading at startup
consult your distros documentation how to do so.
3. Adopt access rights on the msr device files for normal user. To allow everybody access you can
use *chmod o+rw /dev/cpu/*/msr* . This is only recommended on single user desktop systems.
As a general access to the msr registers is not desired on security sensitive
systems you can either implement a more sophisticated access rights settings
with e.g. setgid. A common solution used on many other device files, e.g. for
audio, is to introduce a group and make a chown on the msr device files to that
group. Now if you execute likwid-perfctr with setgid on that group the
executing user can use the tool but cannot directly write or read the msr
device files.
Some distributions backported the capabilities check for the msr device to older kernels. If there are problems with accessing the msr device for older kernels with file system permissions set to read&write, please check your kernel code (*arch/x86/kernel/msr.c*) for the backport and set the MSR capabilities in case.
A secure solution is to use the [accessDaemon](MSRDaemon), which encapsulates
the access to the msr device files and performs a address check for allowed
registers. For more information how to setup and use this solution have a look
at the [WIKI page](MSRDaemon).
Hardware performance monitoring on x86 is enabled using model-specific registers (MSR). MSR registers are special registers not part of the instruction set architecture. To read and write to these registers the x86 ISA provides special instructions. These instructions can only be executed in protected mode or in other words only kernel code can execute these instructions. Fortunately, any Linux kernel 2.6 or newer provides access to these registers via a set of device files. This allows to implement all of the functionality in user space. Still it does not allow to use those more advanced features of hardware performance monitoring which require to setup interrupt service routines or kernel located memory.
If you have problems please let me know on the
[likwid user mailing list](http://groups.google.com/group/likwid-users)
Per default only root has read/write access to these MSR device files. In order to use the LIKWID tools, which need access to these files ([[likwid-perfctr|likwid-perfctr]], [[likwid-powermeter|likwid-powermeter]] and [[likwid-agent|likwid-agent]]) as standard user, you need to setup access rights to these files.
[[likwid-perfctr|likwid-perfctr]], [[likwid-powermeter|likwid-powermeter]] and [[likwid-agent|likwid-agent]] require the Linux `msr` kernel module. This module is part of most standard distro kernels. You have to be root to do the initial setup.
## Setting up access to SandyBridge Uncore
- Check if the `msr` module is loaded with `lsmod | grep msr`. There should be an output.
- It the module is not loaded, load it with `modprobe msr`. For automatic loading at startup consult your distros documentation how to do so.
- Adopt access rights on the MSR device files for normal user. To grant access to anyone, you can use `chmod o+rw /dev/cpu/*/msr`. This is only recommended on single user desktop systems.
Starting with SandyBridge Intel started to use PCI config address memory to
configure and use the Uncore hardware performance monitoring. In Linux there is
no way to gain access to this mapped address ranges as normal user. Therefore
likwid-perfctr can only be used with the SandyBridge Uncore if using the
accessDaemon setup with setuid root. Note that you can overwrite this on the
command line if you , e.g., want to make very accurate measurements as root in
DIRECT mode.
A demo for a root exploit involving the msr device files was published. As a consequence the security settings for access to the msr device files are tightened in recent kernels.
As in general access to MSRs is not desired on security sensitive systems, you can either implement a more sophisticated access rights settings with e.g. setgid. A common solution used on many other device files, e.g. for audio, is to introduce a group and make a `chown` on the MSR device files to that group. Now if you execute [[likwid-perfctr|likwid-perfctr]] with setgid on that group, the executing user can use the tool but cannot directly write or read the MSR device files.
## Issues on recent kernels
Some distributions backported the capabilities check for the MSR device to older kernels. If there are problems with accessing the MSR devices for older kernels with file system permissions set to read&write, please check your kernel code (`arch/x86/kernel/msr.c`) for the backport and set the MSR capabilities in case.
Just setting the file access rights or using suid root on the access daemon is not sufficient anymore. You have to register your binary now to get access.
A secure solution is to use the access daemon [[likwid-accessD|likwid-accessD]], which encapsulates the access to the MSR device files and performs a address check for allowed registers.
This is done by calling
Some newer kernels implement the so-called capabilities, a fine-grained permission system that can allow access to the MSR files for common users. On the downside, it may be not enough anymore to set the suid-root flag for the access daemon, the executable must be registerd at the `libcap`.
`sudo setcap cap_sys_rawio+ep EXECUTABLE`
```
sudo setcap cap_sys_rawio+ep EXECUTABLE
```
on the executables. This is only possible on local file systems. A feasible way is to use the likwid-accessD for all accesses and just enable the capabilities for this one binary. This will enable the usage for all LIKWID tools and also for all instrumented binaries.
This is only possible on local file systems. A feasible way is to use the [[likwid-accessD|likwid-accessD]] for all accesses and just enable the capabilities for this one binary. This will enable the usage for all LIKWID tools and also for all instrumented binaries. If the [[likwid-perfctr|likwid-perfctr]] utility should only be used in wrapper mode, it is suitable to set the capabilities for [[likwid-perfctr|likwid-perfctr]] only. Please remember to set the file permission of the MSR device file to read/write for all users, even if capabilites are configured correctly.
\ No newline at end of file