Skip to content

CASSJAVA-144: Move ASF CI from ci-cassandra Jenkins to GH Action - #2104

Open
SiyaoIsHiding wants to merge 8 commits into
apache:trunkfrom
SiyaoIsHiding:trunk
Open

SiyaoIsHiding wants to merge 8 commits into
apache:trunkfrom
SiyaoIsHiding:trunk

Conversation

@SiyaoIsHiding

Copy link
Copy Markdown
Contributor

Example run:
https://github.com/SiyaoIsHiding/java-driver/actions/runs/40934985232
Largely reusing Node.js driver's GHA. Example user interface on a PR: https://redirect.github.com/apache/cassandra-nodejs-driver/pull/472

Disclose of Generative AI usage.

echo "CCM_UPDATE_PID_DEFAULT_TIMEOUT=120" >> "$GITHUB_ENV"
# "-" tells bin/cassandra not to use jemalloc, skipping its unbounded `find` over the
# library dirs, which can exceed the pid timeout on a cold cache under parallel tests.
echo "CASSANDRA_LIBJEMALLOC=-" >> "$GITHUB_ENV"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is prolly the only essential difference from the Node.js driver's GHA. AI found it out this is the reason why sometimes the first integration test likes to fail with cassandra pid not found.
It makes sense to me. I think I will port this change to Node.js driver's GHA, too.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The CI report scan splits filenames containing spaces and suppresses read errors, allowing jobs to pass despite TCK failures.

3 open issues
What changed in this PR

Migrates the Java driver’s ASF CI from Jenkins to GitHub Actions, with reusable test-environment setup and test-result reporting.

Changes:

  • Adds a Java 17/21 test matrix against Cassandra 3.11, 4.0, 4.1, and 5.0.
  • Adds CCM setup, report uploads, and result publication.
  • Removes Jenkins infrastructure and switches documentation builds to Java 17.
File Description
Jenkinsfile-asf Removes the ASF Jenkins pipeline.
ci/​run-tests.sh Removes the Jenkins test runner.
ci/​install-jdk.sh Removes the legacy JDK installer.
ci/​create-user.sh Removes the container user-setup script.
.github/​workflows/​publish-docs-gp-pages.yml Uses Java 17 for documentation builds.
.github/​workflows/​ci.yml Adds matrix testing and test-result reporting.
.github/​actions/​setup-ccm/​action.yml Prepares CCM, JDKs, certificates, Simulacron, and Cassandra.

Review effort: Balanced

Comment thread .github/workflows/ci.yml Outdated
Comment on lines +79 to +80
failed=$(grep -lE '<testsuite[^>]* (failures|errors)="[1-9]' \
$(find . -path '*/target/*-reports/TEST-*.xml') /dev/null || true)
Comment thread .github/workflows/ci.yml
Comment on lines +22 to +25
on:
push:
pull_request:
workflow_dispatch: # allow manual trigger from the Actions page

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, I guess...

The only thing scheduled builds are going to do for us are pick up variation in the patch version of a given Cassandra major/minor tree. Same is true for ccm, right, and ccm changes even less frequently. I guess I don't mind scheduled builds but it seems like the benefit here would be pretty small.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added cron.
As for other branches, I think we can get this PR merged first, and we will see whether it works well, then we can talk about migrating other branches from Jenkins to GHA, too.

Comment thread .github/workflows/ci.yml Resolved
using: composite
steps:
# ---- Python for ccm ----
- name: Set up Python 3.15.2

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another diff from the Node.js driver - installing python 3.9 started to fail on default github action runners because the default one is updated to Ubuntu 26.04, and python 3.9 is EOL.
so, the python version is updated to 3.15, and the below

python -m pip install --upgrade pip setuptools
pip install .

I will update the Node.js driver CI on this changes, too.

Comment thread .github/workflows/ci.yml

- name: Run unit and integration tests
run: |
mvn -B -V verify -T 1 \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old invocation was:

mvn -B -V verify -T 1 -Ptest-jdk-${TEST_JAVA_MAJOR_VERSION} -DtestJavaHome=$(jabba which ${TEST_JAVA_VERSION}) -Dccm.version=${PATCH_SERVER_VERSION} -Dccm.dse=false -Dmaven.test.failure.ignore=true -Dmaven.javadoc.skip=true

It seems we are missing -Ptest-jdk-${TEST_JAVA_MAJOR_VERSION} is that intentional?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, have we explored the option of using more threads? -T 1 is the slowest option (but also the safest).

id: z11
uses: actions/setup-java@v5
with:
distribution: zulu

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any specific reason to go with Zulu? The docs CI uses temurin, which seems a more common choice.

Comment thread .github/workflows/ci.yml
path: artifacts

- name: Publish test results
uses: EnricoMi/publish-unit-test-result-action@v2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I double-checked that this action is allow-listed by the ASF, and it is. That said, it would be safer to pin it to a specific commit SHA rather than a tag.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I wonder if this action will fail when executed from a forked repo, because of the required privileges: checks: write pull-requests: write. I haven't confirmed this though.

If that proves correct, it could be solved by switching off checks and comments and keeping only the summary:

- uses: EnricoMi/publish-unit-test-result-action@v2
  with:
    files: artifacts/**/TEST-*.xml
    comment_mode: off
    check_run: false

... in which case, we don't need checks: write pull-requests: write.

Comment thread .github/workflows/ci.yml
name: Unit and Integration Tests

on:
push:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: restrict to pushes to main branches only:

Suggested change
push:
push:
branches:
- trunk # default branch; adjust if it's named differently
- '[0-9]*.x' # release branches: 3.x, 4.x, 4.5.x, ...

Comment thread .github/workflows/ci.yml
# Saturday 23:00 PST / Sunday 00:00 PDT; cron is in UTC and runs on the default branch only
- cron: "0 7 * * 0"

permissions:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: add a concurrency section to limit the number of concurrent runs, since the matrix is quite large:

concurrency:
  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}

Comment thread .github/workflows/ci.yml
workflow_dispatch: # allow manual trigger from the Actions page
schedule:
# Saturday 23:00 PST / Sunday 00:00 PDT; cron is in UTC and runs on the default branch only
- cron: "0 7 * * 0"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that on Jenkins, a weekly job would run on every release branch. Here, we are losing that (this cron applies only to the main branch, afaik). To recover the ability to run CI weekly on all release branches, we'd need a separate workflow.

Comment thread .github/workflows/ci.yml
unit-integration-tests:
name: Unit and Integration Tests (JDK ${{ matrix.TEST_JAVA_VERSION }}, C* ${{ matrix.SERVER_VERSION }})
runs-on: ubuntu-latest
strategy:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: add a hard timeout:

timeout-minutes: 120

@@ -0,0 +1,139 @@
name: Set up CCM for integration tests

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing the ASF license header.

Comment on lines +131 to +133
- name: Print environment
shell: bash
run: printenv | sort

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can print sensitive data. Do we need it?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants