Release Procedure
=================

These notes enumerate the steps required every time we release a new
version of Arbor.

Pre-release
-----------

Update tags/versions and test
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#. Check if some files are up to date
    
   - README.md, ATTRIBUTIONS.md, CONTRIBUTING.md, CHANGELOG.md. For autogenerated CHANGELOG, see below.
   - Verify ``MANIFEST.in`` (required for PyPI sdist).
   - Double-check that all examples/tutorials/etc are covered by CI.
   - Check Python/pip/PyPi metadata and scripts, e.g., ``pyproject.toml``.

#. Create new temp-branch ending in ``-rc``. E.g. ``v0.9.0-rc``
#. Bump the ``VERSION`` file:

   - See also :ref:`dev-version`
   - Append ``-rc``. (Make sure there's no ``-dev``)

#. Create a **draft PR**. Tag and push with ``-rc``. E.g. ``v0.9.0-rc``

   - on cmdline: ``git tag -a TAGNAME``
   - ``git push upstream refs/tags/TAGNAME``
   - Githubs autogenerated release notes can only be generated on a tag. Use
     this ``-rc`` tag to do so, and add the commit to the draft PR. Ensure
     Github takes the previous release as base.

#. Run and pass all tests.

   - They should run automatically based on the draft PR. Pushing the tag kicks of ``cibuildwheel``.
   - In case you want to test ``ciwheel.yml`` manually on your local repo ``origin``, push to a branch called ``ciwheel``, e.g. ``git push origin HEAD:ciwheel``.
   
     - ``ciwheel.yml`` pushes automatically to `test.pypi.org
       <https://test.pypi.org/project/arbor/>`_. This only passes if run as a
       branch of the main ``arbor-sim`` repo (as that's where the PyPI secret
       lives). On your own repo, the upload will fail (the rest should pass). If
       you want to test uploading, then force push to the _upstream_ ``ciwheel``
       branch, e.g. ``git push upstream HEAD:ciwheel --force``.
   
   - If you want to test the PyPI submission (consider asking other OS-users):

     .. code-block:: bash

        python -m venv env && source env/bin/activate
        pip install numpy
        pip install -i https://test.pypi.org/simple/ arbor #should select the latest build, otherwise e.g. arbor==0.8rc0
        python -c 'import arbor; print(arbor.__config__)'

Release
-------

#. Make sure no errors were encountered in the pre-release phase, working wheels were produced, and were installable from `Test.PyPI.org <https://test.pypi.org/project/arbor/>`_ and that no problems were reported.
   
#. Change ``VERSION``. Make sure does not end with ``-rc`` or ``-dev``. Push to the **draft PR** opened earlier for the rc.

#. When tests pass, merge the PR.

#. Tag

   - commit and either reuse the draft PR for the RC, or open a new one.
   - once merged, make and push tag.

      - on cmdline: ``git tag -a TAGNAME``
      - ``git push upstream TAGNAME``

#. Upload to pypi & verify

   Get the wheels from test PyPI or the GitHub Action that produced the release artifacts. Make sure you have pip installed ``twine`` and have your ``~/.pypirc`` file setup with the right token (see `docs <https://packaging.python.org/en/latest/specifications/pypirc/>`_).

   .. code-block:: bash

      twine upload -r arborpypi dist/*

      python -m venv env && source env/bin/activate
      pip install arbor
      python -c 'import arbor; print(arbor.__config__)'

#. Create GitHub Release: https://github.com/arbor-sim/arbor/releases

   - The Github action that produced the release artifacts should have prepared a draft Release.
   - Update body text with ``CHANGELOG.md``
   - (Probably: delete the GitHub draft release for the ``-rc``)
   - If no Github Release was automatically drafted:
   - Go to `GH tags`_ and click “…” and “Create release”
   - Categorize/edit Github's autogenerated release notes (alternatively, go through merged PRs to come up with a changelog).
   - Manually build full tarball:

   ``scripts/create_tarball ~/loc/of/arbor tagname outputfile``

   - eg ``scripts/create_tarball /full/path/to/arbor v0.5.1 ~/arbor-v0.5.1-full.tar.gz``

Post Release
------------

#. Start a new release on Zenodo. This allocated a DOI, but you don't have to finish it right away. Add new Zenodo DOI badge/link to ``CITATION.bib`` and ``docs/index.rst``.

#. Update Zenodo with authors and changelog created in the previous step and submit.

#. Make a new PR setting ``VERSION`` to the next with a trailing ``-dev``. E.g. if you just released ``3.14.15``, change ``VERSION`` to ``3.15.16-dev``. Make sure the number portion always consists of a triple. Shorter versions are uninstallable by Spack (``spack install arbor@0.8`` will install v0.8.1, due to anything shorter than a triple being interpreted as a version range). 

   - Include changes such as to ``CITATIONS``, ``doc/index.rst`` in postrel PR. Copy Zenodo BibTex export to ``CITATIONS``.

#. Update ``scripts/check-all-tags.sh`` to check the current tag.

#. Update spack package / Ebrains Lab / Opensourcebrain

   - Spack upstream: `PR here <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/arbor/package.py>`_
   - Ebrains Lab: `MR here <https://gitlab.ebrains.eu/technical-coordination/project-internal/devops/platform/ebrains-spack-builds/>`_
   - OSB: update `requirementsfile <https://github.com/OpenSourceBrain/OSBv2/blob/master/applications/jupyterlab/requirements.txt>`_ if needed.

     - Make sure that `Notebooks <https://www.v2.opensourcebrain.org/repositories/38>`_ work on the version that their image is built with.

#. Announce on our website
#. [AUTOMATED] Add tagged version of docs on ReadTheDocs

#. FZJ admin

   - https://juser.fz-juelich.de/submit

GUI
===

Release automation is a bit more advanced for Arbor GUI: the act of pushing a new tag, auto-drafts a release with the relevant artifacts.
The post-release steps mentioned above are largely the same. A list of the places where an update must be entered:

#. https://doi.org/10.5281/zenodo.7415129

.. _GH tags: https://github.com/arbor-sim/arbor/tags
.. _AUTOMATED: https://github.com/arbor-sim/arbor/blob/master/.github/workflows/ebrains.yml