Releases

Release cycle

0. release every 3 months (at time T)

1. T-11 weeks: all add your favorite Issues to the next-rel column

2. T-10 weeks: Scrum Master prep dev meet (internal)

   • Update/trim next-release column in Kanban

   • Prepare agenda, include possible additions not covered by Kanban/Issues

   • Add milestone tags (nextver, nextver+1, etc.)

   • Add highlighted new features to RELEASE_NOTES.md

3. T-8 weeks: Release Manager dev meet (external/public)

   • Use Kanban as starter

   • Move issues around based on input

   • Add milestone tags, for this release or future releases

   • Update highlighted new features to RELEASE_NOTES.md

4. T-1: all reserve week for wrapping up PRs and review.

5. T±0: Release Manager release!

   • Have a look at Python release schedule, and time Arbor release optimally with new Python minor version. It is nice to generate wheels for the new minor as soon as minor is released.

6. T+1 weeks: Scrum Master retrospective

   • set date for next release

Procedure

These notes enumerate the steps required every time we release a new version of Arbor.

Pre-release

Update tags/versions and test

1. Check if some files are up to date

   • README.md, ATTRIBUTIONS.md, CONTRIBUTING.md, CHANGELOG.md. For autogenerated CHANGELOG, see below.

   • Verify MANIFEST.in (required for PyPI sdist)

   • Double check that all examples/tutorials/etc are covered by CI

   • Check Python/pip/PyPi metadata and scripts, e.g. setup.py, pyproject.toml

2. Create new temp-branch ending in -rc. E.g. v0.9.0-rc

3. Bump the VERSION file:

   • For as long as scikit-build-core does not support loading fields from external files, also bump in pyproject.toml

   • See also Version and build information

   • Append -rc. (Make sure there’s no -dev)

4. Create a draft PR. Tag and push with -rc. E.g. v0.9.0-rc

   • on cmdline: git tag -a TAGNAME

   • git push upstream refs/tags/TAGNAME

   • Githubs autogenerated release notes can only be generated on a tag. Use this -rc tag to do so, and add the commit to the draft PR. Ensure Github takes the previous release as base.

5. Run and pass all tests.

   • They should run automatically based on the draft PR. Pushing the tag kicks of cibuildwheel.

   • In case you want to test ciwheel.yml manually on your local repo origin, push to a branch called ciwheel, e.g. git push origin HEAD:ciwheel.

     • ciwheel.yml pushes automatically to Test.PyPI.org. This only passes if ran as branch of the main arbor-sim repo (as that’s where the PyPI secret lives). On your own repo, the upload will fail (the rest should pass). If you want to test uploading, then force push to the _upstream_ ciwheel branch, e.g. git push upstream HEAD:ciwheel --force.

   • If you want to test the PyPI submission (consider asking other OS-users):

     python -m venv env && source env/bin/activate
     pip install numpy
     pip install -i https://test.pypi.org/simple/ arbor #should select the latest build, otherwise e.g. arbor==0.8rc0
     python -c 'import arbor; print(arbor.__config__)'
     

Release

1. Make sure no errors were encountered in the pre-release phase, working wheels were produced, and were installable from Test.PyPI.org and that no problems were reported.

2. Change VERSION. Make sure does not end with -rc or -dev. Push to the draft PR opened earlier for the rc.

3. When tests pass, merge the PR.

4. Tag

   • commit and either reuse the draft PR for the RC, or open a new one.

   • once merged, make and push tag.

     • on cmdline: git tag -a TAGNAME

     • git push upstream TAGNAME

5. Upload to pypi & verify

   Get the wheels from test PyPI or the Github Action that produced the release artifacts. Make sure you have pip installed twine and have your ~/.pypirc file setup with the right token (see docs).

   twine upload -r arborpypi dist/*
   
   python -m venv env && source env/bin/activate
   pip install arbor
   python -c 'import arbor; print(arbor.__config__)'
   

6. Create Github Release: https://github.com/arbor-sim/arbor/releases

   • The Github action that produced the release artifacts should have prepared a draft Release.

   • Update body text with CHANGELOG.md

   • (Probably: delete the Github draft release for the -rc)

   • If no Github Release was automatically drafted:

   • Go to GH tags and click “…” and “Create release”

   • Categorize/edit Github’s autogenerated release notes (alternatively go through merged PRs to come up with a changelog).

   • Manually build full tarball:

   scripts/create_tarball ~/loc/of/arbor tagname outputfile

   • eg scripts/create_tarball /full/path/to/arbor v0.5.1 ~/arbor-v0.5.1-full.tar.gz

Post Release

1. Start a new release on Zenodo, this allocated a DOI, but you don’t have to finish it right away. Add new Zenodo DOI badge/link to CITATION.bib and docs/index.rst.

2. Update Zenodo with authors and changelog created in previous step and submit.

3. Make a new PR setting VERSION to the next with a trailing -dev. E.g. if you just released 3.14.15, change VERSION to 3.15.16-dev. Make sure the number portion always consists of a triple. Shorter versions are uninstallable by Spack (spack install arbor@0.8 will install v0.8.1, due to anything shorter than a triple being interpreted as a version range).

   • Update spack/package.py. The checksum of the targz is the sha256sum.

   • Include changes such as to CITATIONS, doc/index.rst in postrel PR. Copy Zenodo BibTex export to CITATIONS.

4. Update scripts/check-all-tags.sh to check the current tag.

5. Update spack package / Ebrains Lab / Opensourcebrain

   • Spack upstream: PR here

   • Ebrains Lab: MR here

   • OSB: update requirementsfile if needed.

     • Make sure that Notebooks work on the version that their image is built with.

6. Announce on our website

7. Announce on HBP newsletter newsletter@humanbrainproject.eu, communications@ebrains.eu, HBP Twitter/socials evan.hancock@ebrains.eu

8. [AUTOMATED] Add tagged version of docs on ReadTheDocs

9. HBP internal admin

   • Plus: https://plus.humanbrainproject.eu/components/2691/

   • TC Wiki: https://wiki.ebrains.eu/bin/view/Collabs/technical-coordination/EBRAINS%20components/Arbor/

   • KG: https://search.kg.ebrains.eu/instances/5cf4e24b-b0eb-4d05-96e5-a7751134a061

     • Update howto: https://wiki.ebrains.eu/bin/view/Collabs/swc-guide#HHowtoupdateexistingSoftwareinstances

     • tldr: shoot your ticket here: curation-support@humanbrainproject.eu

     • Supported file formats

       • ContentTypes: https://humanbrainproject.github.io/openMINDS/v3/core/v4/data/contentType.html

       • details: https://github.com/HumanBrainProject/openMINDS_core/tree/v3/instances/data/contentTypes

10. FZJ admin

    • https://juser.fz-juelich.de/submit

GUI

Release automation is a bit more advanced for Arbor GUI: the act of pushing a new tag, auto-drafts a release with the relevant artifacts. The post release steps mentioned above are largely the same. A list of the places where an update must be entered:

1. https://doi.org/10.5281/zenodo.7415129

2. https://plus.humanbrainproject.eu/components/3178