Polygene™
Introduction
Tutorials
Javadoc
Samples
Core
Libraries
Extensions
Tools
Glossary 

Releasing Polygene™

Warning

You need a unix-like environment to actually perform the release process. This tutorial is known to work on Linux and Mac.

This tutorial is intended for Apache Polygene™ commiters who want to release a new version of Apache Polygene™ (Java Edition) at The Apache Software Foundation. It describes the tools and processes of a typical release.

It is intended to be a recommendation of best practices for the Apache Polygene™ project. The instructions provided here are consistent with, but not a replacement for the ASF Release Guidelines.

TL;DR

  • Build and publish/stage release candidate bits
  • VOTE thread
  • NO: clean things up
  • YES: promote the release candidate bits as final and update all properties

Preparing a release

Select a Release Manager

A Polygene committer (normally one of the development team) should post an email to the development list proposing that a release be made and nominating a release manager. Typically, the proposer volunteers as the release manager and it passes by lazy consensus.

Clone/Checkout all repositories

Clone/checkout all needed repositories, next to each other:

mkdir polygene-repos
cd polygene-repos
git clone https://git-wip-us.apache.org/repos/asf/polygene-java.git polygene-java
git clone https://git-wip-us.apache.org/repos/asf/polygene-website.git polygene-website
cd polygene-website
git checkout asf-site
cd ..
svn checkout --depth empty https://dist.apache.org/repos/dist/ polygene-dist
cd polygene-dist
svn update --set-depth immediates dev
svn update --set-depth immediates release
svn update --set-depth infinity dev/polygene
svn update --set-depth infinity release/polygene
cd ..

You should then get the following directory tree:

.
└── polygene-repos
    ├── polygene-java           # Apache Polygene™ (Java Edition) source
    ├── polygene-website        # https://polygene.apache.org/ website
    └── polygene-dist
        ├── dev
        |   └── polygene        # Releases candidate distributions
        └── release
            └── polygene        # Releases distributions

Tip

From now on, all command line snippets start from the polygene-repos directory.

Build Apache Polygene™ (Java Edition)

Ensure you can test, build Apache Polygene™ (Java Edition), including the documentation minisite generation using Asciidoc and artifact signing.

Here is what should pass before going further:

cd polygene-java
git checkout develop
./gradlew -Dversion="<RELEASE-VERSION>" checkDistributions

See the Build System tutorial for details.

Important

Also ensure that the checkDists CI build is passing.

Install Jekyll

Moreover, you will need to have a valid Jekyll installation as the Apache Polygene™ website is generated using it.

Setup git flow

git-flow is a git extension that add git commands to easily use the git flow branching model the Apache Polygene™ project follows. See the installation instructions.

Setup git signing

Ensure you can sign git tags.

git config --global user.signingkey <YOUR-PGP-ID>

See the Git Tools - Signing Your Work section of the Git book.

Setup Apache Nexus credentials

See the Apache Publishing Maven Artifacts guide and the Apache Polygene™ (Java Edition) Build System tutorial.

Setup nodejs and NPM registry credentials

First, install NodeJS.

If you don’t have a npmjs.org user yet, create one now:

npm adduser

If your npmjs.org user is not registered as a maintainer for the generator-polygene package, ask for the privilege on dev@polygene.apache.org.

Update the KEYS files if needed.

The reference KEYS file can be found at the polygene-java repository’s root, that is polygene-java/KEYS. Ensure that it contains your public key.

Next, diff it against the ones present in the dev and release distribution areas:

diff polygene-java/KEYS polygene-dist/dev/polygene/KEYS
diff polygene-java/KEYS polygene-dist/release/polygene/KEYS

And update them if needed:

cp polygene-java/KEYS polygene-dist/dev/polygene/KEYS
cp polygene-java/KEYS polygene-dist/release/polygene/KEYS
cd polygene-dist/dev/polygene
svn add dev/polygene/KEYS release/polygene/KEYS
svn commit -m "polygene: updating Polygene KEYS"

Creating a Release Candidate

Resolve JIRA issues

Resolve all issues on that version! They can be resolved by:

  • fixing
  • marking them as INVALID or WONTFIX
  • changing their fix version to another unreleased version

See the POLYGENE project on JIRA.

Prepare Release-Notes

Apache Polygene™ release-notes are generated from JIRA issues.

Open the target Polygene™ version’s release-notes in JIRA and review them.

JIRA can produces release-notes as HTML or plain-text. Set it up to generate plain-text release-notes.

We will need these in several formats. Starting from the plain-text one we will generate the others.

First save the text-plain release-notes in a file named apache-polygene-java-<RELEASE-VERSION>-release-notes.txt. A good place for this file would be in the polygene-repos directory created earlier, alongside all repositories.

Convert to Asciidoc:

cat "apache-polygene-java-<RELEASE-VERSION>-release-notes.txt" | \
  sed -e "s/\[POLYGENE-\([0-9]\)*\]/https:\/\/issues.apache.org\/jira\/browse\/POLYGENE-\1[POLYGENE-\1]/" | \
  sed -e "s/    \* /- /" | sed -e "s/^\*\*/====/" \
  > "apache-polygene-java-<RELEASE-VERSION>-release-notes.adoc"

Convert to Markdown:

cat "apache-polygene-java-<RELEASE-VERSION>-release-notes.txt" | \
  sed -e "s/\[POLYGENE-\([0-9]*\)\]/[POLYGENE-\1](https:\/\/issues.apache.org\/jira\/browse\/POLYGENE-\1)/" | \
  sed -e "s/    \* /- /" | sed -e "s/^\*\*/####/" \
  > "apache-polygene-java-<RELEASE-VERSION>-release-notes.md"

You should then have the following files:

.
├── apache-polygene-java-<RELEASE-VERSION>-release-notes.txt
├── apache-polygene-java-<RELEASE-VERSION>-release-notes.adoc
└── apache-polygene-java-<RELEASE-VERSION>-release-notes.md

We will use them later.

Create a RC branch

We use <RELEASE-VERSION>-RC# where RELEASE-VERSION is the target release version and RC# for Release Candidate and an incremental number in case the release process has to be done several times.

cd polygene-java
git flow release start "<RELEASE-VERSION>-RC#"

This will eventually generates a <RELEASE-VERSION>-RC# tag that we will amend with a <RELEASE-VERSION> signed tag if the vote passes, see below.

Audit artifacts and distributions

Make a complete build, deploying maven artifacts locally:

cd polygene-java
./gradlew -Dversion="<RELEASE-VERSION>" assemble

Review maven artifacts in distributions/build/stage/maven-binaries.

Also review the release distributions in distributions/build/distributions where you should find the following files:

.
├── apache-polygene-java-<RELEASE-VERSION>-src.tgz
├── apache-polygene-java-<RELEASE-VERSION>-src.tgz.MD5
├── apache-polygene-java-<RELEASE-VERSION>-src.tgz.SHA-512
├── apache-polygene-java-<RELEASE-VERSION>-src.tgz.asc
├── apache-polygene-java-<RELEASE-VERSION>-src.zip
├── apache-polygene-java-<RELEASE-VERSION>-src.zip.MD5
├── apache-polygene-java-<RELEASE-VERSION>-src.zip.SHA-512
└── apache-polygene-java-<RELEASE-VERSION>-src.zip.asc

If any, make the required changes, commit them and iterate.

Close the RC branch

Once you are satisfied with the produced artifacts, close the release candidate branch:

cd polygene-java
git flow release finish "<RELEASE-VERSION>-RC#"
Checkout the RC tag

To build the release candidate bits, we need to checkout the release candidate tag, that will eventually be promoted as a signed release tag, because the Apache Polygene™ build system generates versionning information based on git metadata.

cd polygene-java
git checkout "<RELEASE-VERSION>-RC#"
Build RC artifacts and distributions
cd polygene-java
./gradlew -Dversion="<RELEASE-VERSION>" clean assemble
Stage RC maven artifacts

Stage artifacts to repository.apache.org :

cd polygene-java
./gradlew -Dversion="<RELEASE-VERSION>" uploadArchives

Close the staging Nexus repository by following the Closing the staged repository guide.

Stage NPM registry packages

Publish the new version of the generator-polygene NPM package tagged as beta:

cd polygene-java
./gradlew tools:generator-polygene:prepareTemplateGradleWrapper
cd tools/generator-polygene
npm publish --tag beta

If the release VOTE passes, we’ll remove the beta tag, making the new version available.

Then open https://www.npmjs.com/package/generator-polygene to check that the new version was published and is tagged as beta.

Upload RC distributions

Source distribution, checksums and signatures must be uploaded to dist.apache.org/repos/dist/dev/polygene. This build created these in the buid/distributions directory, named apache-polygene-java-<RELEASE-VERSION>-src*.*. As this release still is a simple candidate, we’ll rename them before upload to advertise this in their names.

# Source ZIP
cp "polygene-java/distributions/build/distributions/apache-polygene-java-<RELEASE-VERSION>-src.zip" \
   "polygene-dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip"
cp "polygene-java/distributions/build/distributions/apache-polygene-java-<RELEASE-VERSION>-src.zip.MD5" \
   "polygene-dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.MD5"
cp "polygene-java/distributions/build/distributions/apache-polygene-java-<RELEASE-VERSION>-src.zip.SHA-512" \
   "polygene-dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.SHA-512"
cp "polygene-java/distributions/build/distributions/apache-polygene-java-<RELEASE-VERSION>-src.zip.asc" \
   "polygene-dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.asc"
# Source TAR.GZ
cp "polygene-java/distributions/build/distributions/apache-polygene-java-<RELEASE-VERSION>-src.tgz" \
   "polygene-dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz"
cp "polygene-java/distributions/build/distributions/apache-polygene-java-<RELEASE-VERSION>-src.tgz.MD5" \
   "polygene-dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.MD5"
cp "polygene-java/distributions/build/distributions/apache-polygene-java-<RELEASE-VERSION>-src.tgz.SHA-512" \
   "polygene-dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.SHA-512"
cp "polygene-java/distributions/build/distributions/apache-polygene-java-<RELEASE-VERSION>-src.tgz.asc" \
   "polygene-dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.asc"

And then upload them:

cd polygene-dist/dev/polygene
svn add * --force
svn commit -m "polygene: upload <RELEASE-VERSION>-RC# to dist/dev/polygene"

Run the vote

Send a "VOTE" to the developer mailing list including links to release artifacts. A VOTE always contains two parts. Send an email to the developer mailing list with the subject line:

[VOTE] Release Polygene (Java Edition) version <RELEASE-VERSION>

Here is a sample template:

Dear community,

I am happy to start the VOTE thread for Apache Polygene (Java Edition) <RELEASE-VERSION>!

The changelog for this release can be found here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316820&version=12332997

Tag: https://git-wip-us.apache.org/repos/asf?p=polygene-java.git;a=tag;h=refs/tags/<RELEASE-VERSION>-RC#

The distributions to be voted on are located here: https://dist.apache.org/repos/dist/dev/polygene/

Convenience artifacts in a maven repository are staged here: https://repository.apache.org/content/groups/staging/org/apache/polygene/

Release distributions and convenience artifacts are signed with the following key: https://dist.apache.org/repos/dist/dev/polygene/KEYS

Some guidance on how to validate distributions and artifacts can be found here: https://polygene.apache.org/java/develop/releasing-apache.html#_verifying_a_release_candidate

Please vote on releasing this package as Apache Polygene (Java Edition) <RELEASE-VERSION>.

The vote is open for the next 72 hours and passes if a majority of at least three +1 PMC votes are cast and there must be more positive than negative votes.

[ ] +1 Release Apache Polygene (Java Edition) <RELEASE-VERSION>
[ ]  0 I don't have a strong opinion about this, but I assume it's ok
[ ] -1 Do not release Apache Polygene (Java Edition) <RELEASE-VERSION> because...

Cheers

After the vote is over, send a "RESULT" email to the list with the subject line:

[RESULT][VOTE] Release Polygene (Java Edition) version <RELEASE-VERSION>

Here is a sample template:

To: "Polygene Developers List" <dev@polygene.apache.org>
CC: "Polygene Project Management Committee List" <private@polygene.apache.org>
Subject: [RESULT][VOTE] Release Polygene (Java Edition) version <RELEASE-VERSION>

Hi,

The vote has passed|failed with the following result:

+1 (binding): <<list of names>>
+1 (non binding): <<list of names>>

I will promote|drop the distributions and artifacts.

Votes on whether a package is ready to be released use majority approval — i.e., at least three PMC members must vote affirmatively for release, and there must be more positive than negative votes.

Verifying a Release Candidate

In order to verify a Apache Polygene™ (Java Edition) release candidate you’ll need valid wget, md5sum, shasum with support for SHA-5.12, GnuPG and Java JDK >= 7 installations.

If the setup feels like a chore, keep reading, you probably already have what’s needed for some of the checks. A single check is better than none.

Download RC

First go downloading the release candidate distributions from dist.a.o/repos/dist/dev/polygene. Including *.MD5, *.SHA-512 and *.asc files. Put them in a directory:

cd polygene-check-rc
wget https://dist.apache.org/repos/dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz
wget https://dist.apache.org/repos/dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.MD5
wget https://dist.apache.org/repos/dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.SHA-512
wget https://dist.apache.org/repos/dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.asc
wget https://dist.apache.org/repos/dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip
wget https://dist.apache.org/repos/dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.MD5
wget https://dist.apache.org/repos/dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.SHA-512
wget https://dist.apache.org/repos/dist/dev/polygene/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.asc
Verify RC signatures

Verify PGP signatures:

cd polygene-check-rc
gpg apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.asc
gpg apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.asc

This must output gpg: Good signature from "Signer Name <signer@ema.il>" for each distribution.

Verify MD5 checksum:

cd polygene-check-rc
echo "$(cat apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.MD5) apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz" | md5sum -c -
echo "$(cat apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.MD5) apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip" | md5sum -c -

This must output some-filename: OK for each distribution.

Verify SHA-512 checksum:

cd polygene-check-rc
echo "$(cat apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.MD5)  apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz" | shasum -a 512 -c -
echo "$(cat apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.MD5)  apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip" | shasum -a 512 -c -

This also must output some-filename: OK for each distribution.

Verify source distribution

Unpack the source distributions, we will use the zip distribution here, both should be checked.

cd polygene-check-rc
unzip apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip
cd apache-polygene-java-<RELEASE-VERSION>-RC#-src

LICENSE & NOTICE

Review LICENSE and NOTICE file(s) ; see the Apache’s Licensing Howto.

Tools like Apache Rat™ and other projects from Apache Creadur™ can come in handy. You can also grep the whole source tree.

Build RC from sources

cd polygene-check-rc/apache-polygene-java-<RELEASE-VERSION>-RC#-src
./gradlew assemble check

VOTE passes

Seal the release

Create and sign the release git tag from the unsigned release candidate tag:

cd polygene-java
git tag -s "<RELEASE-VERSION>" "<RELEASE-VERSION>-RC#"

Push all git changes:

cd polygene-java
git checkout master
git push origin master
git checkout develop
git push origin master
git push origin --tags
Publish bits

Promote the staged Nexus repository so it gets synched to Maven Central by following the Promoting a repo guide.

Move the release distributions, checksums and signatures from polygene-dist/dev/polygene to polygene-dist/release/polygene:

cd polygene-dist
# Source ZIP TODO
export DIST_DEV_URL="https://dist.apache.org/repos/dist/dev/polygene"
export DIST_RELEASE_URL="https://dist.apache.org/repos/dist/release/polygene"
svn move "$DIST_DEV_URL/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip" \
     "$DIST_RELEASE_URL/apache-polygene-java-<RELEASE-VERSION>-src.zip" \
     -m "polygene: promoting release <RELEASE-VERSION> distributions"
svn move "$DIST_DEV_URL/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.MD5" \
     "$DIST_RELEASE_URL/apache-polygene-java-<RELEASE-VERSION>-src.zip.MD5" \
     -m "polygene: promoting release <RELEASE-VERSION> distributions"
svn move "$DIST_DEV_URL/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.SHA-512" \
     "$DIST_RELEASE_URL/apache-polygene-java-<RELEASE-VERSION>-src.zip.SHA-512" \
     -m "polygene: promoting release <RELEASE-VERSION> distributions"
svn move "$DIST_DEV_URL/apache-polygene-java-<RELEASE-VERSION>-RC#-src.zip.asc" \
     "$DIST_RELEASE_URL/apache-polygene-java-<RELEASE-VERSION>-src.zip.asc" \
     -m "polygene: promoting release <RELEASE-VERSION> distributions"
# Source TAR.GZ
svn move "$DIST_DEV_URL/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz" \
     "$DIST_RELEASE_URL/apache-polygene-java-<RELEASE-VERSION>-src.tgz" \
     -m "polygene: promoting release <RELEASE-VERSION> distributions"
svn move "$DIST_DEV_URL/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.MD5" \
     "$DIST_RELEASE_URL/apache-polygene-java-<RELEASE-VERSION>-src.tgz.MD5" \
     -m "polygene: promoting release <RELEASE-VERSION> distributions"
svn move "$DIST_DEV_URL/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.SHA-512" \
     "$DIST_RELEASE_URL/apache-polygene-java-<RELEASE-VERSION>-src.tgz.SHA-512" \
     -m "polygene: promoting release <RELEASE-VERSION> distributions"
svn move "$DIST_DEV_URL/apache-polygene-java-<RELEASE-VERSION>-RC#-src.tgz.asc" \
     "$DIST_RELEASE_URL/apache-polygene-java-<RELEASE-VERSION>-src.tgz.asc" \
     -m "polygene: promoting release <RELEASE-VERSION> distributions"

This produces one commit per file but keep svn history and runs fast, without re-uploading the distributions.

Wait 24 hours

For mirrors to pick up the new bits.

Prepare an announcement

Coordinate a press release with press@apache.org.

TODO

You can reuse the release-notes content from the txt/adoc/md files created earlier.

This annoucement will be used in a variety of media like emails, websites etc… Start with a text version and once satisfied create at least a Markdown version for the website, see below.

Publish generator-polygene to the NPM registry
npm dist-tag rm generator-polygene beta

Then open https://www.npmjs.com/package/generator-polygene to check that the package is not tagged as beta anymore.

Update the Polygene™ website

Generate the documentation and javadoc minisite:

cd polygene-java
./gradlew -Dversion="<RELEASE-VERSION>" copyAsfDocumentation

This will automatically put all files into the polygene-website website repository.

Create a new post on the Polygene™ website by creating a new Markdown file:

cd polygene-website
touch "src/_posts/YYYY-MM-DD-apache-polygene-java-<RELEASE-VERSION>.md"
open !$

You can reuse the Markdown formatted announcement content.

Add the new released version in polygene-website/site/content/java/versions.json below the latest entry:

{
    "develop": "develop",
    "latest": "latest",
    "<RELEASE-VERSION>", "<RELEASE-VERSION>",
    "2.0": "2.0",
    "<=1.4.x": "1.4"
}

Finally, edit polygene-website/site/src/_data/releases.yml with the new release data. Upmost is the latest:

- version: <RELEASE-VERSION>
  announcement: YYYY/MM/DD/apache-polygene-java-<RELEASE-VERSION>
  signer: John Doe
  pgpId: FB751943

You can run the Polygene™ website locally:

cd polygene-website
jekyll serve

Open http://127.0.0.1:4000/ to live-preview your changes.

Once you are satisfied with the changes, build the production website:

cd polygene-website
jekyll build

Link /latest to the latest release:

cd polygene-website/content/java
ln -s <RELEASE_VERSION> latest

And publish it:

cd polygene-website
git add .
git commit -m "update website"
git push
Register the release

Register the new release at reporter.apache.org

Announce

Finally, send an announcement to dev@ mailing list. Email announcements should have the subject line:

[ANNOUNCE] Released Polygene (Java Edition) version <RELEASE-VERSION>

The announcement email should contains the release-notes as text, remember they are in the apache-polygene-java-<RELEASE-VERSION>-release-notes.txt file you created earlier.

VOTE fails or is cancelled

Record failure

We keep the release candidate git history. It can be useful for reviewers to have access to it. Remember, we created a release candidate branch and tags, no signed release tag.

cd polygene-java
git checkout master
git push origin master
git checkout develop
git push origin master
git push origin --tags
Drop RC artifacts and distributions

Drop the Nexus staging repository by following the Dropping a repo guide.

Drop distributions, checksums and signatures from polygene-dist/dev/polygene

cd polygene-dist/dev/polygene/
svn rm "*<RELEASE-VERSION>-RC#*.*"
svn commit -m "polygene: dropping <RELEASE-VERSION>-RC# from dist/dev/polygene as the vote failed/was cancelled"

Drop NPM packages:

npm unpublish generator-polygene@<RELEASE-VERSION>
Start over

If a new RC is to be created, restart the process as described above.