Summary of Select Activities

Dmitri Smirnov

Updated

<h1>Fun Facts and Statistics</h1>

<div class="my-bg-gry" style="font-size: 70%; position: absolute; top: 10%; right: 5%;">Based on Git repo as of August 2019</div>

- Overall **5M lines of code** in the most popular languages<br>
  <pre><code data-trim style="font-size: 0.7em;" class="shell">
  $tokei --sort=code --type="FORTRAN Legacy,C,C Header,C++,C++ Header,Python,Perl" star-cvs/
  </code></pre>

<pre class="hcenter" style="font-size: 0.7em; margin-left: 2em; border: 0;">
--------------------------------------------<b>---------</b>
 Language            Files        Lines     <b>    Code </b>
--------------------------------------------<b>---------</b>
 FORTRAN Legacy       3673      2400728     <b> 1944928 </b>
 C Header             3657      2165256     <b> 1869860 </b>
 C++                  3630      1357126     <b>  895478 </b>
 C                   10405       475461     <b>  329001 </b>
 Perl                  340       120573     <b>   81639 </b>
 C++ Header            974        76729     <b>   40200 </b>
 Python                176        38364     <b>   29307 </b>
--------------------------------------------<b>---------</b>
 Total               22855      6634237     <b> 5190413 </b>
--------------------------------------------<b>---------</b>
</pre>

- About **3.7M lines of code** when exclude multiple versions of MC generators and Geant3<br>
  <pre><code data-trim style="font-size: 0.7em;" class="shell">
  $tokei --exclude "StRoot/StarGenerator" --exclude "*geant3*" ...
  </code></pre>

<h1>Fun Facts and Statistics</h1>

<div class="my-bg-gry" style="font-size: 70%; position: absolute; top: 10%; right: 5%;">Based on Git repo as of August 2019</div>

<div style="width: 50%; float: left; height: 85%;">

<pre class="hcenter" style="font-size: 0.5em; max-height: 100%; overflow-x: hidden; border: 1;">
<b>             commits     files/c    lines++    lines--</b>
    fisyak      4844         8.2    5711940    1614370
   didenko      4595         3.1     475511     128456
<span class="my-bg-red">     perev      3842         3.2    2733529    1433259</span>
   jeromel      3424         2.6    2098468      60738
      fine      2880         2.6     617199     106234
     jwebb      1457         4.2    5481898      41585
<span class="my-bg-red">    genevb      1210         2.3     283911      37494</span>
     kathy       988         2.3      63332      33938
   ullrich       945         2.9     190422      32948
<span class="my-bg-red">  smirnovd       812         2.5      42806      26669</span>
  balewski       745         3.4     164257      31887
     tonko       710         2.2      66361      16473
<span class="my-bg-red">  potekhin       578         6.6    1607303       6615</span>
       jcs       558         1.9     251117      16502
       tai       550         7.0     303363      59626
    caines       512         2.3      72928      18276
    nevski       493         2.3     222354     120066
    lasiuk       488         2.1      65275      18604
       jml       477         3.0     104915      16943
    pibero       472         2.7      52648      37253
   mmiller       459         7.5     341599      36296
      akio       448         4.6     347134      18082
   pruneau       446         4.9      74694      58822
      tull       440         4.9     162712      26052
  calderon       431         3.0      56566      14644
  sakrejda       373         1.6      31555       7118
    suaide       339         3.1     109206      27500
      ward       317         2.4      37787       6898
   rfatemi       299         1.4     477356       6937
      posk       291         3.6      59492      27624
      laue       286         3.6      39720      10031
  kocolosk       270         3.1     149353      20530
    porter       255         8.1      71702      17809
<span class="my-bg-red">    dmitry       244         1.9      10679       1053</span>
  andrewar       243         1.3      17096       2498
   avossen       230         2.5      43680       5147
    geurts       221         1.8      28456       6281
      oldi       219         2.5      32010      11972
      zhux       218         1.2       7358       1296
     yepes       204         5.1      76010      16497
   prindle       198         6.0     124399      35711
     dongx       180         1.8      25669       4280
       bum       180         3.1      11013       6635
      hack       179         6.3      73366       8660
<span class="my-bg-red">    wenaus       177         2.5      13133       2404</span>
...
<span class="my-bg-red">  panitkin         1         2.0        151          0</span>
...
</pre>

</div>
<div style="width: 50%; float: right;">

- Legend

**`commits`** : Total number of commits<br>
  **`files/c`** : Files changed per commit<br>
  **`lines++`** : Lines added<br>
  **`lines--`** : Lines removed<br>
  <span class="my-bg-red">name</span> : NPPS group members

- First commit

`Author: starlib  <>  1993-02-09`

- 44,300 total commits by 250 authors

- 4.6 commits per day

- No activity during 35% of the time

<div class="hcenter">
<span class="my-bg-gry" style="font-size: 70%;">Statistics by <a href="http://gitstats.sourceforge.net/">GitStats</a></span>
</div>

</div>

<h1>Git Repository</h1>

<div class="my-bg-gry" style="font-size: 70%; position: absolute; top: 8%; right: 5%;"><a href="github.com/star-bnl/star-cvs">github.com/star-bnl/star-cvs</a></div>

- The import script is based on `cvs2git` converting selected CVS subdirectories to a Git repository

- Several CVS subdirectories excluded resulting in the size of selected files of 1.7GB

- We run <a href="https://rtyley.github.io/bfg-repo-cleaner/">BFG repo cleaner</a> to correct imported history

- E.g. CVS history contains a commit of a `.git` folder 😀 (Must be removed)

- Size of bare `.git` with full history is about 500MB and the full checkout is 900MB
    <pre style="margin-top: 0;"><code data-trim style="font-size: 0.7em;" class="shell">
    $git clone ...
    $git checkout master
    $du --exclude .git -h -d1 star-cvs/ | sort -h
    ...
    878M    star-cvs/
    </code></pre>

- Compare to ROOT repository sizes of 700MB and 1.7GB respectively

- The code of external packages (e.g. MC generators) still can be removed from the DVCS making the STAR software more compact and attractive

<h1>Building in a Docker Container</h1>

- Building STAR software inside a docker container can be used for quick test builds followed by test jobs. A possible general scenario:

<ol>
    <li> A docker image is created with all STAR software dependencies, i.e. ROOT, CERNLIB, and other libraries.
         <strong>Based on Ubuntu 16 the image size is 2GB</strong>

<li> A container with STAR code is built on top of that image and tests are executed.<br>
         <strong>Image size increases to 2.3GB</strong>

<li> On success the newly created image with STAR libraries is tagged "latest" in the docker repository

<li> Subsequent builds can use the "latest" image as cache resulting in faster incremental build
  </ol>

- For example, see <a href="https://blog.callr.tech/building-docker-images-with-gitlab-ci-best-practices/">Best practices for building docker images with GitLab CI</a>

- Dockerfiles are available in https://github.com/star-bnl/star-sw/docker

- A couple measurements with incremental builds:

- When `touch` all files in `StRoot/StEvent` incremental build takes ~6m
  - When `touch` all files in `StRoot/Sti*` incremental build takes ~1m 30s

Alternative Implementations: $ASA^T$

Alternative Implementations: Weighted Average

Comparison of Options: Weighted Average


    set1  = "-O2 -m32 -msse -mno-avx -ftree-vectorize"
    set2  = "-O2 -m32 -msse -mno-avx -fno-tree-vectorize"
    set3  = "-O2 -m32 -msse -mno-avx -ftree-vectorize -D EIGEN_DONT_VECTORIZE"
    set4  = "-O2 -m32 -msse -mno-avx -fno-tree-vectorize -D EIGEN_DONT_VECTORIZE"
    set5  = "-O3 -m32 -msse -mno-avx -ftree-vectorize"
    set6  = "-O3 -m32 -msse -mno-avx -fno-tree-vectorize"
    set7  = "-O3 -m32 -msse -mno-avx -ftree-vectorize -D EIGEN_DONT_VECTORIZE"
    set8  = "-O3 -m32 -msse -mno-avx -fno-tree-vectorize -D EIGEN_DONT_VECTORIZE"
    set9  = "-O2 -m64 -msse -mno-avx -ftree-vectorize"
    set10 = "-O2 -m64 -msse -mno-avx -fno-tree-vectorize"
    set11 = "-O2 -m64 -msse -mno-avx -ftree-vectorize -D EIGEN_DONT_VECTORIZE"
    set12 = "-O2 -m64 -msse -mno-avx -fno-tree-vectorize -D EIGEN_DONT_VECTORIZE"
    set13 = "-O3 -m64 -msse -mno-avx -ftree-vectorize"
    set14 = "-O3 -m64 -msse -mno-avx -fno-tree-vectorize"
    set15 = "-O3 -m64 -msse -mno-avx -ftree-vectorize -D EIGEN_DONT_VECTORIZE"
    set16 = "-O3 -m64 -msse -mno-avx -fno-tree-vectorize -D EIGEN_DONT_VECTORIZE"

<h1>STAR <code>picoDST</code> Data Format</h1>

- In STAR `muDST` (aka `$\mu$DST`, micro DST) is the primary output format used
by all physics analyses

- The format is based on objects of same type packed in ROOT's `TCloneArray`s
  attached to a common `TTree` as `TBranches`

- Cross references between entities within event are implemented with raw
  indices

- With increasing luminosity the need for a more compact format for user
analyses become apparent. The `picoDST` was proposed

- In the initial version some observables were packed/unpacked in 16-bit
   integer with preset ranges and scale factors

- There was lots of confusion on how `Double32_t` and `Float16_t` work so, [a
   study with various packing
   options](https://plexoos.github.io/my-tests/root-branch-compress/) was
   carried out to verify the performance and confirm the usage benefits