Skip to content

Commit d86aa08

Browse files
RitaChen609rjrudinGAdityaVarmaSameeraPriyathamTadikondajonmille
authored
MLE-31572 release 8.2.0 (#1977)
* MLE-27481 Bumped to 8.2-SNAPSHOT * MLE-27841 Generating javadocs on each publish This simplifies life for the docs team by allowing them to download the zip of docs. * MLE-27841 Bumped undertow and ml-gradle Now using ml-gradle 6.2.0 instead of SNAPSHOT * SECCMP-1797: Downgrade contents permission from write to read The copyright-check reusable workflow only needs contents: read to checkout code. contents: write was unnecessarily granting the workflow token write access to repository contents, which expands the blast radius of any PwnRequest-style attack. The called workflow (copyright-check.yml) already declares its own permissions block with contents: read. * SECCMP-1797: Harden permissions - add top-level read, fix contents: write Adds top-level permissions: contents: read to restrict the default GITHUB_TOKEN. Also fixes the previous contents: write on the copyright-validation job to contents: read. This follows the principle of least privilege recommended in GitHub's PwnRequest security guidance. * PDP-1182: Remove per-repo pr-workflow.yaml * MLE-28459 Added regression-12 arm stage (#1934) MLE-28459 Added regression-12 arm stage * MLE-28498 11.3.5 Test Fixes for Security Update (#1935) * MLE-28498 Permission fixes * MLE-28498 testResultIgnore() Test fix * MLE-27881 Add cts.param support to Java Client API * MLE-28334 Implement Fragment Option (#1937) * MLE-28334 Implement Fragment Option * MLE-28334 Copilot Suggested Fixes + AbstractFromSearchFragmentTest.java: Refactored FromSearchDocsWithFragmentTest.java and FromSearchWithFragmentTest.java + Changed JavaDoc for fromsearchDocs in PlanBuilderBase.java * MLE-28583 Test fixes for ML12.1 (#1940) Ignore tests that are no longer supported in ML12. Getting the user a different way since xdmp calls have been restricted. * MLE-30241: Apply XXE protections (#1941) * MLE-30241: Apply XXE protections Replace bare DocumentBuilderFactory.newInstance() in OpenCSVBatcher.write() with XmlFactories.getDocumentBuilderFactory() * MLE-30241 Copyright and Copilot suggestions * MLE-30240: Redact auth headers from OkHttp logs (#1942) * MLE-30240: Redact auth headers from OkHttp logs Redact Authorization header from OkHttp logs * MLE-30240 Copilot Suggestions * MLE-30240 Refactor Extract the OkHttp logging message handler and header-redaction logic from OkHttpServices into a dedicated RedactingHttpLogger class. Move the unit test to RedactingHttpLoggerTest in the same package. Revert CONTRIBUTING.md changes (HTTP logging docs belong in product documentation, not the contributor guide). * MLE-30237 fix serverStartStop to prevent OS command injection (#1946) * MLE-30237 fix string comparison operator in serverStartStop to prevent OS command injection * MLE-30237 add -- after ssh to terminate option parsing * MLE-30243 Ensure password is sanitized in any logging * MLE-30667 Added toString() override to ClientCookie (#1949) Added a toString() override that returns `name + "=" + value`. * MLE-29888 Add cts expressions as allowable cts params * MLE-30244 move Jenkins registry credentials to the publish stage and … (#1951) * MLE-30244 move Jenkins registry credentials to the publish stage and validate the MARKLOGIC_IMAGE_TAGS build parameter * MLE-30244 refine the changes as the copilot reviews * MLE-30244 update regex for image tags. * MLE-30244 tighten the regex for image tags * MLE-30239 Use secure trust store * MLE-31024 Refactor Replace/Ignore Transform Tests (#1955) * MLE-31024 Refactor Replace/Ignore Transform Tests * MLE-30953 MLE-30952 MLE-30968 Vulnerability fixes (#1952) * MLE-30953 * MLE-30952 * MLE-30953 Kotlin upgrade Upgraded Kotlin to resolve opentelemetry java vulnerability * MLE-30952 Update logback to 1.5.37 for newer version * MLE-30968 Bump JacksonVersion to 2.22.0 Bump jacksonVersion from 2.21.1 to 2.22.0 * MLE-30953 Bump okhttpVersion to 5.4.0 * MLE-30952 Deleting pom.xml (#1956) Deleting pom.xml so that Black Duck does not scan it * MLE-29643 Configure OkHttp Logging with dedicated logger (#1957) * MLE-29643 Configure OkHttp Logging with dedicated logger --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * MLE-30967 Update CODEOWNERS * MLE-31167 Bump jacksonVersion to 2.22.1 (#1958) * MLE-31109 Bump kotlin plugin and remove kotlin.system.exitProcess (#1961) * MLE-31109 Bump kotlin plugin and remove kotlin.system.exitProcess Bump kotlin plugin from 2.4.0 to 2.4.10 Removed imported instances of kotlin.system.exitProcess from com.squareup.okhttp3:okhttp:5.4.0 vulnerable dependency org.jetbrains.kotlin:kotlin-stdlib:2.2.21 * MLE-31642 infostudio-user removal test fixes (#1962) Removed the infostudio-user role since it has been removed from MLS. Added the update capability to the app-user role. These changes fixed these 3 failing tests: - testWriteMultipleXMLDocWithXMLMetadata - testMetadataXMLCRUD - testRollbackMetadata * MLE-31642 Update noSslContext Test (#1963) * MLE-31642 Update noSslContext Test MarkLogic 12.1 is now returning a 403 error code, like we expect, instead of closing the connection and throwing a MarkLogicIOException for ML 12.1. -Renamed noSslContext() -> noSslContextWithMarkLogic11OrLower() -Added RequiresML12Dot0.java to only run tests against 12.0 versions -Renamed noSslContextWithMarkLogic12() to noSslContextWithMarkLogic12Dot0() -noSslContextWithMarkLogic12Dot0() ExtendWith updated to new RequiresML12Dot0.class to only run against 12.0 versions -Added noSslContextWithMarkLogic12Dot1OrHigher() for new ML 12.1 behavior * MLE-31642 fromLexicons Test fix (#1964) * MLE-31642 fromLexicons Test fix propertiesFragmentShouldNotBeReturnedByFromLexicons() was failing against ML12.1 due to intended behavior change, validated in MLE-31757. The test has been updated to only run on ML version >12.0. A new test has been added to validate the new ML 12.1 fromLexicons behavior. * MLE-31877 revert xdmp-login changes (#1966) * MLE-31877 Removal of xdmp-login * MLE-31877 Privilege Update Adding back the unprotected-uri and unprotected- collections privilege removed as part of xdmp-login removal * MLE-31877 Privilege Update Adding "xdmp:xslt-invoke", "xdmp:xslt-eval", and "xdmp:value" to test-rest-writer role to fix tests. These were removed when java-test-xdmp-login.json was deleted. * MLE-31874 update LICENSE file (#1965) * MLE-31887 Refactor xdmp privileges in test-rest-writer (#1967) * MLE-31877 Removal of xdmp-login * MLE-31877 Privilege Update Adding back the unprotected-uri and unprotected- collections privilege removed as part of xdmp-login removal * MLE-31877 Privilege Update Adding "xdmp:xslt-invoke", "xdmp:xslt-eval", and "xdmp:value" to test-rest-writer role to fix tests. These were removed when java-test-xdmp-login.json was deleted. * MLE-31887 Refactor xdmp privileges in test-rest-writer Moved privileges `xdmp:invoke`, `xdmp:xslt-invoke`, `xdmp:xslt-eval`, and `xdmp:value` from test-rest-writer role to new role test-invoke. Created new user writer-no-default-invoke to use new role test-invoke. Updated AbstractOpicUpdateTest.java to use new user writer-no-default-invoke since it needs the `xdmp:invoke` privilege * MLE-29883 (GH #1938) Include document version in bulk reads * MLE-32054: [java-client-api][polaris]Improper Resource Shutdown or * MLE-31570: Generate SBOM and merge to develop * MLE-32022 Avoid null pointer exception when FailedRequest is null * MLE-32111 Add annotation support to StructuredQueryBuilder (#1648) Adds StructuredQueryBuilder.annotation(String...) so callers can attach one or more <annotation> elements to a structured query. The server turns these into cts:annotation elements; they are ignored during evaluation and are useful for documenting or marking parts of a query. Annotation content may be arbitrary XML (including namespaced elements) or plain text. Content is parsed and copied into the query via DOMWriter, with a plain-text fallback for non-XML content. Annotations are serialized after the query elements to satisfy the search schema's ordering requirement. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Update StructuredQueryBuilder related copyright * MLE-31569 update NOTICE.txt for 8.2.0 release * MLE-32227 Updating build.gradle to publish SBOM artifact (#1975) * MLE-32227 Updating build.gradle to publish SBOM artifact Updated the build.gradle to imbed the marklogic-client-api-<version>.spdx.json SBOM file under META-INF/sbom. Update the publications section to publish the marklogic-client-api-<version>.spdx.json SBOM file as a stand alone artifact. * MLE-32227 Copyright Updates * MLE-31993 update version to 8.2.0 (#1976) --------- Co-authored-by: Rob Rudin <rudin@progress.com> Co-authored-by: GAdityaVarma <agottumu@progress.com> Co-authored-by: SameeraPriyathamTadikonda <preethu4476@gmail.com> Co-authored-by: Jonathan Miller <jonmille@progress.com> Co-authored-by: Ryan Dew <ryan.dew@progress.com> Co-authored-by: Ryan Dew <146854757+rjdew-progress@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: godugu@progress.com <godugu@progress.com> Co-authored-by: ngodugu-marklogic <74939216+ngodugu-marklogic@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent 67a55b0 commit d86aa08

88 files changed

Lines changed: 7671 additions & 2289 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/copilot-instructions.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Project Guidelines
2+
3+
## Architecture
4+
- `marklogic-client-api`: core Java client library for interacting with the MarkLogic REST API.
5+
- `ml-development-tools`: Gradle plugin and generators for Data Services endpoint proxies/tests.
6+
- `marklogic-client-api-functionaltests`: functional/regression-style tests, split into fragile/fast/slow groups.
7+
- `test-app`: ml-gradle deployment project that provisions test infrastructure in MarkLogic.
8+
- `examples`: supporting code used by tests and usage examples.
9+
10+
## Build And Test
11+
- Prefer Gradle from the repo root.
12+
- Quick compile verification: `./gradlew clean build -x test`
13+
- Core module tests: `./gradlew marklogic-client-api:test`
14+
- Plugin tests (includes generated tests workflow): `./gradlew ml-development-tools:test`
15+
- Functional tests must run in this order to reduce flakiness:
16+
1. `./gradlew marklogic-client-api-functionaltests:runFragileTests`
17+
2. `./gradlew marklogic-client-api-functionaltests:runFastFunctionalTests`
18+
3. `./gradlew marklogic-client-api-functionaltests:runSlowFunctionalTests`
19+
20+
## Test Environment
21+
- Java 17+ is required for current releases.
22+
- Most tests require a running MarkLogic instance and deployed test resources.
23+
- Typical setup sequence:
24+
1. `docker compose up -d --build`
25+
2. `./gradlew -i mlWaitTillReady`
26+
3. `./gradlew -i mlDeploy`
27+
4. Run module tests
28+
- Override local MarkLogic connection settings via `gradle-local.properties` (`mlHost`, `mlPassword`).
29+
30+
## Quality Controls
31+
- Treat compile warnings as failures: project builds enforce `-Xlint:unchecked`, `-Xlint:deprecation`, and `-Werror`.
32+
- Keep dependency security constraints intact (e.g. forced/excluded dependencies for CVE mitigation in Gradle files).
33+
- When adding or changing first-party Java/Kotlin code, run security scanning steps used by this workspace workflow before finalizing changes.
34+
- Do not relax quality gates (tests/compilation) to make a change pass; fix the underlying issue.
35+
36+
## Code Generation And Automation
37+
- Data Services proxy generation is automated; use `generateEndpointProxies` instead of hand-writing proxy classes.
38+
- `ml-development-tools` test automation uses `generateTests` and `fixMjsModulesForMarkLogic12` before `test`.
39+
- Generated sources commonly include an "IMPORTANT: Do not edit" header. Regenerate from source declarations instead of editing generated output directly.
40+
- For changes affecting generation logic, validate both generator behavior and generated artifact compilation/tests.
41+
42+
## Conventions For Changes
43+
- Keep edits scoped to the target module; avoid cross-module churn unless required.
44+
- Prefer existing patterns in nearby code over introducing new abstractions.
45+
- For test-related fixes, document whether behavior changes impact unit tests, functional tests, or deployment setup.
46+
47+
## Docs To Link (Do Not Duplicate)
48+
- `README.md`: product overview, dependency usage, Java compatibility.
49+
- `CONTRIBUTING.md`: local build/test workflow and MarkLogic test setup.
50+
- `ml-development-tools/src/test/example-project/README.md`: plugin-focused usage/testing notes.

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ xml-with-props.xml
3636
ml-development-tools/src/test/ml-modules
3737
ml-development-tools/src/test/java/com/marklogic/client/test/dbfunction/generated
3838

39+
.codesight
3940
.vscode
4041
docker/
4142

‎CODEOWNERS‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22
# Each line is a file pattern followed by one or more owners.
33

44
# These owners will be the default owners for everything in the repo.
5-
* @billfarber @rjrudin @stevebio
5+
* @rjrudin @jonmille @ngodugu-marklogic @RitaChen609 @rjdew-progress

‎CONTRIBUTING.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
To develop the Java Client and ultimately submit a pull request, you'll first need to be able to build the client and
2-
run the tests locally.
1+
To develop the Java Client and ultimately submit a pull request, you'll first need to be able to build the client and
2+
run the tests locally.
33

44
Please see [this guide](.github/CONTRIBUTING.md) for information on creating and submitting a pull request.
55

@@ -11,7 +11,7 @@ To build the client locally, complete the following steps:
1111
4. Verify that you can build the client by running `./gradlew build -x test`
1212

1313
"Running the tests" in the context of developing and submitting a pull request refers to running the tests found
14-
in the `marklogic-client-api` module. The tests for this module depend on a
14+
in the `marklogic-client-api` module. The tests for this module depend on a
1515
[ml-gradle](https://github.com/marklogic-community/ml-gradle) application being deployed from the `test-app` module.
1616
This application contains a number of database and security resources that the tests depend on.
1717
The `./gradle.properties` file defines the connection properties for this application; these default
@@ -53,5 +53,5 @@ You can also undeploy the test application if you do not wish to keep it around
5353

5454
./gradlew mlUndeploy -i -Pconfirm=true
5555

56-
To test the `ml-development-tools` Gradle plugin, see the README.md file located at
56+
To test the `ml-development-tools` Gradle plugin, see the README.md file located at
5757
`ml-development-tools/src/test/example-project`.

‎Jenkinsfile‎

Lines changed: 54 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -135,8 +135,6 @@ pipeline {
135135
environment {
136136
JAVA_HOME_DIR = getJavaHomePath()
137137
GRADLE_DIR = ".gradle"
138-
DMC_USER = credentials('MLBUILD_USER')
139-
DMC_PASSWORD = credentials('MLBUILD_PASSWORD')
140138
PLATFORM = "linux/amd64"
141139
MARKLOGIC_INSTALL_CONVERTERS = "true"
142140
}
@@ -189,14 +187,35 @@ pipeline {
189187
}
190188
}
191189
steps {
192-
sh label: 'publish', script: '''#!/bin/bash
193-
export JAVA_HOME=$JAVA_HOME_DIR
194-
export GRADLE_USER_HOME=$WORKSPACE/$GRADLE_DIR
195-
export PATH=$GRADLE_USER_HOME:$JAVA_HOME/bin:$PATH
196-
cp ~/.gradle/gradle.properties $GRADLE_USER_HOME;
197-
cd java-client-api
198-
./gradlew publish
199-
'''
190+
withCredentials([
191+
string(credentialsId: 'MLBUILD_USER', variable: 'DMC_USER'),
192+
string(credentialsId: 'MLBUILD_PASSWORD', variable: 'DMC_PASSWORD')
193+
]) {
194+
sh label: 'publish', script: '''#!/bin/bash
195+
export JAVA_HOME=$JAVA_HOME_DIR
196+
export GRADLE_USER_HOME=$WORKSPACE/$GRADLE_DIR
197+
export PATH=$GRADLE_USER_HOME:$JAVA_HOME/bin:$PATH
198+
cp ~/.gradle/gradle.properties $GRADLE_USER_HOME;
199+
cd java-client-api
200+
./gradlew publish
201+
'''
202+
}
203+
}
204+
post {
205+
always {
206+
sh label: 'generate-javadoc', script: '''#!/bin/bash
207+
export JAVA_HOME=$JAVA_HOME_DIR
208+
export GRADLE_USER_HOME=$WORKSPACE/$GRADLE_DIR
209+
export PATH=$GRADLE_USER_HOME:$JAVA_HOME/bin:$PATH
210+
cd java-client-api
211+
./gradlew javadoc
212+
echo "Zipping javadocs for easy download..."
213+
cd marklogic-client-api/build/docs
214+
zip -r javadoc.zip javadoc/
215+
mv javadoc.zip $WORKSPACE/
216+
'''
217+
archiveArtifacts artifacts: 'javadoc.zip', fingerprint: true
218+
}
200219
}
201220
}
202221

@@ -210,7 +229,14 @@ pipeline {
210229

211230
steps {
212231
script {
213-
def imageTags = params.MARKLOGIC_IMAGE_TAGS.split(',')
232+
// Validate MARKLOGIC_IMAGE_TAGS to prevent argument injection via user-supplied image tag values;
233+
// each entry is validated individually after trimming to reject empty values and tags
234+
// containing spaces or other shell-injectable characters
235+
def imageTags = params.MARKLOGIC_IMAGE_TAGS.split(',').collect { it.trim() }
236+
def invalidTags = imageTags.findAll { it.isEmpty() || !(it ==~ /^[a-z0-9]+(?:[._-][a-z0-9]+)*(?::[A-Za-z0-9_][A-Za-z0-9_.-]{0,127})?$/) }
237+
if (!invalidTags.isEmpty()) {
238+
error("Invalid MARKLOGIC_IMAGE_TAGS entries: ${invalidTags}. Expected comma-delimited docker image refs like 'marklogic-server-ubi:latest-12'")
239+
}
214240
def imagePrefix = 'ml-docker-db-dev-tierpoint.bed-artifactory.bedford.progress.com/marklogic/'
215241

216242
imageTags.each { tag ->
@@ -313,7 +339,7 @@ pipeline {
313339
}
314340
}
315341

316-
stage('regressions-11 arm infrastructure') {
342+
stage('regressions-arm infrastructure') {
317343
when {
318344
beforeAgent true
319345
branch 'develop'
@@ -334,14 +360,22 @@ pipeline {
334360
submoduleCfg : [],
335361
userRemoteConfigs : scm.userRemoteConfigs])
336362

337-
runTests("ml-docker-db-dev-tierpoint.bed-artifactory.bedford.progress.com/marklogic/marklogic-server-ubi9-arm:latest-11")
338-
}
339-
post {
340-
always {
341-
archiveArtifacts artifacts: 'java-client-api/**/build/reports/**/*.html'
342-
junit '**/build/**/TEST*.xml'
343-
updateWorkspacePermissions()
344-
tearDownDocker()
363+
script {
364+
def imagePrefix = 'ml-docker-db-dev-tierpoint.bed-artifactory.bedford.progress.com/marklogic/marklogic-server-ubi9-arm:'
365+
366+
['latest-11', 'latest-12'].each { tag ->
367+
def fullImage = imagePrefix + tag
368+
stage("regressions-arm-${tag}") {
369+
try {
370+
runTests(fullImage)
371+
} finally {
372+
archiveArtifacts artifacts: 'java-client-api/**/build/reports/**/*.html'
373+
junit '**/build/**/TEST*.xml'
374+
updateWorkspacePermissions()
375+
tearDownDocker()
376+
}
377+
}
378+
}
345379
}
346380
}
347381
}

0 commit comments

Comments
 (0)