Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HPCC4J-594 Add support for skipping tests #710

Open
wants to merge 2 commits into
base: candidate-9.2.x
Choose a base branch
from
Open

HPCC4J-594 Add support for skipping tests #710

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6328081
HPCC4J-594 Add support for skipping tests
jpmcmu May 29, 2024
a3c2310
Code review changes
jpmcmu May 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions .github/workflows/baremetal-regression-suite.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,5 +176,23 @@ jobs:
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2

- name: Build with Maven
run: mvn -B --activate-profiles jenkins-on-demand -Dmaven.gpg.skip=true -Dmaven.javadoc.skip=true -Dmaven.test.failure.ignore=false -Dhpccconn=http://eclwatch.default:8010 -Dwssqlconn=http://sql2ecl.default:8510 -DHPCC30117=open install
- name: Exclude Tests
env:
EXCLUDES: ${{ vars.EXCLUDED_TESTS }}
run: |
echo $EXCLUDES >> excluded-tests.txt
cat excluded-tests.txt

- name: Build Project
run: mvn clean install -DskipTests=true

- name: Run Commons-HPCC Tests
run: mvn -pl commons-hpcc --activate-profiles jenkins-on-demand -Dmaven.gpg.skip=true -Dmaven.javadoc.skip=true -Dmaven.test.failure.ignore=false verify

- name: Run WSClient Tests
run: mvn -pl wsclient --activate-profiles jenkins-on-demand -Dmaven.gpg.skip=true -Dmaven.javadoc.skip=true -Dmaven.test.failure.ignore=false -Dhpccconn=http://eclwatch.default:8010 -Dwssqlconn=http://sql2ecl.default:8510 verify

- name: Run DFSClient Tests
run: mvn -pl dfsclient --activate-profiles jenkins-on-demand -Dmaven.gpg.skip=true -Dmaven.javadoc.skip=true -Dmaven.test.failure.ignore=false -Dhpccconn=http://eclwatch.default:8010 verify


20 changes: 18 additions & 2 deletions .github/workflows/k8s-regression-suite.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,5 +143,21 @@ jobs:
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2

- name: Build with Maven
run: mvn -B --activate-profiles jenkins-on-demand -Dmaven.gpg.skip=true -Dmaven.javadoc.skip=true -Dmaven.test.failure.ignore=false -Dhpccconn=https://eclwatch.default:8010 -Dwssqlconn=https://sql2ecl.default:8510 -DHPCC30117=open install
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's make sure all references to HPCC30117 are removed in code

- name: Exclude Tests
env:
EXCLUDES: ${{ vars.EXCLUDED_TESTS }}
run: |
echo $EXCLUDES >> excluded-tests.txt
cat excluded-tests.txt

- name: Build Project
run: mvn clean install -DskipTests=true

- name: Run Commons-HPCC Tests
run: mvn -pl commons-hpcc --activate-profiles jenkins-on-demand -Dmaven.gpg.skip=true -Dmaven.javadoc.skip=true -Dmaven.test.failure.ignore=false verify

- name: Run WSClient Tests
run: mvn -pl wsclient --activate-profiles jenkins-on-demand -Dmaven.gpg.skip=true -Dmaven.javadoc.skip=true -Dmaven.test.failure.ignore=false -Dhpccconn=https://eclwatch.default:8010 -Dwssqlconn=https://sql2ecl.default:8510 verify

- name: Run DFSClient Tests
run: mvn -pl dfsclient --activate-profiles jenkins-on-demand -Dmaven.gpg.skip=true -Dmaven.javadoc.skip=true -Dmaven.test.failure.ignore=false -Dhpccconn=https://eclwatch.default:8010 verify
3 changes: 3 additions & 0 deletions commons-hpcc/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven.surefire.version}</version>
<configuration>
<excludesFile>../excluded-tests.txt</excludesFile>
</configuration>
</plugin>
</plugins>
</build>
Expand Down
1 change: 1 addition & 0 deletions dfsclient/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
<value>org.hpccsystems.ws.client.TestResultNotifier</value>
</property>
</properties>
<excludesFile>../excluded-tests.txt</excludesFile>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems ok to have a single excluded-tests file at the hpcc4j level, but I could also see an argument for dedicated excluded-tests files per sub project.
what are the main drawbacks/advantages of the current implementation?

</configuration>
</plugin>
<plugin>
Expand Down
3 changes: 3 additions & 0 deletions excluded-tests.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# See: https://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html#multiple-formats-in-one
# Example:
# **/*RampsDevRegressionTest.java
6 changes: 5 additions & 1 deletion pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
<maven.deploy.version>2.8.2</maven.deploy.version>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we have a sample entry in the above excluded-tests.txt file?
Perhaps create a new test simply for the purpose of always excluding it via this file?
Also, does the github action mechanism append excluded tests entries, or does it overwrite the file?

<maven.install.version>2.5.2</maven.install.version>
<maven.jar.version>3.0.2</maven.jar.version>
<maven.surefire.version>2.22.1</maven.surefire.version>
<maven.surefire.version>3.2.5</maven.surefire.version>
<antlr.version>4.10.1</antlr.version>
<javax.mail.version>1.4</javax.mail.version>
<jsch.version>0.1.54</jsch.version>
Expand Down Expand Up @@ -213,6 +213,7 @@
<configuration>
<argLine>${argLine}</argLine>
<groups>${groups}</groups>
<excludesFile>excluded-tests.txt</excludesFile>
</configuration>
</plugin>
<plugin>
Expand Down Expand Up @@ -433,6 +434,7 @@
<configuration>
<argLine>${argLine}</argLine>
<groups>${groups}</groups>
<excludesFile>excluded-tests.txt</excludesFile>
</configuration>
</plugin>
</plugins>
Expand Down Expand Up @@ -485,6 +487,7 @@
<configuration>
<argLine>${argLine}</argLine>
<groups>${groups}</groups>
<excludesFile>excluded-tests.txt</excludesFile>
</configuration>
</plugin>
</plugins>
Expand Down Expand Up @@ -560,6 +563,7 @@
<configuration>
<argLine>${argLine}</argLine>
<groups>${groups}</groups>
<excludesFile>excluded-tests.txt</excludesFile>
</configuration>
</plugin>
</plugins>
Expand Down
1 change: 1 addition & 0 deletions wsclient/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@
<value>org.hpccsystems.ws.client.TestResultNotifier</value>
</property>
</properties>
<excludesFile>../excluded-tests.txt</excludesFile>
</configuration>
</plugin>
</plugins>
Expand Down
10 changes: 0 additions & 10 deletions wsclient/src/test/java/org/hpccsystems/ws/client/WSFileIOClientTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ public class WSFileIOClientTest extends BaseRemoteTest
private final static String testfilename = System.getProperty("lztestfile", "myfilename.txt");
private final static String targetLZ = System.getProperty("lzname", "localhost");
private final static String targetLZPath = System.getProperty("lzpath", "/var/lib/HPCCSystems/mydropzone");
private final static String HPCC_30117 = System.getProperty("HPCC30117", "fixed");

static
{
Expand All @@ -57,18 +56,12 @@ public class WSFileIOClientTest extends BaseRemoteTest

if (System.getProperty("lzpath") == null)
System.out.println("lzpath not provided - defaulting to /var/lib/HPCCSystems/mydropzone");

if (System.getProperty("HPCC30117") == null)
System.out.println("HPCC30117 status not provided - defaulting to fixed");
else
System.out.println("HPCC30117 status: '" + HPCC_30117 + "'");
}

@Test
public void copyFile() throws Exception
{
Assume.assumeFalse("Test not valid on containerized HPCC environment", client.isTargetHPCCContainerized());
assumeTrue("Ignoring test 'copyFile' because HPCC-30117 is not fixed", HPCC_30117.equalsIgnoreCase("fixed"));
String lzfile=System.currentTimeMillis() + "_csvtest.csv";
String hpccfilename="temp::" + lzfile;
client.createHPCCFile(lzfile, targetLZ, true);
Expand Down Expand Up @@ -145,15 +138,13 @@ public void copyFile() throws Exception
public void AcreateHPCCFile() throws Exception, ArrayOfEspExceptionWrapper
{
Assume.assumeFalse("Test not valid on containerized HPCC environment", client.isTargetHPCCContainerized());
assumeTrue("Ignoring test 'copyFile' because HPCC-30117 is not fixed", HPCC_30117.equalsIgnoreCase("fixed"));
System.out.println("Creating file: '" + testfilename + "' on LandingZone: '" + targetLZ + "' on HPCC: '" + super.connString +"'");
Assert.assertTrue(client.createHPCCFile(testfilename, targetLZ, true));
}

@Test
public void BwriteHPCCFile() throws Exception, ArrayOfEspExceptionWrapper
{
assumeTrue("Ignoring test 'copyFile' because HPCC-30117 is not fixed", HPCC_30117.equalsIgnoreCase("fixed"));
System.out.println("Writing data to file: '" + testfilename + "' on LandingZone: '" + targetLZ + "' on HPCC: '" + super.connString +"'");
byte[] data = "HELLO MY DARLING, HELLO MY DEAR!1234567890ABCDEFGHIJKLMNOPQRSTUVXYZ".getBytes();
Assert.assertTrue(client.writeHPCCFileData(data, testfilename, targetLZ, true, 0, 20));
Expand All @@ -163,7 +154,6 @@ public void BwriteHPCCFile() throws Exception, ArrayOfEspExceptionWrapper
public void CreadHPCCFile() throws Exception, ArrayOfEspExceptionWrapper
{
Assume.assumeFalse("Test not valid on containerized HPCC environment", client.isTargetHPCCContainerized());
assumeTrue("Ignoring test 'copyFile' because HPCC-30117 is not fixed", HPCC_30117.equalsIgnoreCase("fixed"));

System.out.println("reading data from file: '" + testfilename + "' on LandingZone: '" + targetLZ + "' on HPCC: '" + super.connString +"'");
byte[] data = "HELLO MY DARLING, HELLO MY DEAR!1234567890ABCDEFGHIJKLMNOPQRSTUVXYZ".getBytes();
Expand Down
48 changes: 0 additions & 48 deletions wsclient/src/test/java/org/hpccsystems/ws/client/WsAttributesClientIntegrationTest_620.java
View file
Open in desktop

This file was deleted.

31 changes: 0 additions & 31 deletions wsclient/src/test/java/org/hpccsystems/ws/client/WsWorkunitsClientIntegration_54_Test.java
View file
Open in desktop

This file was deleted.

30 changes: 0 additions & 30 deletions wsclient/src/test/java/org/hpccsystems/ws/client/WsWorkunitsClientIntegration_620_Test.java
View file
Open in desktop

This file was deleted.

2 changes: 2 additions & 0 deletions wsclient/src/test/java/org/hpccsystems/ws/client/platform/RampsDevRegressionTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,11 @@

import org.hpccsystems.ws.client.wrappers.ArrayOfEspExceptionWrapper;
import org.hpccsystems.ws.client.wrappers.wsdfu.DFUDataColumnWrapper;
import org.junit.Ignore;
import org.junit.Test;
import org.junit.experimental.categories.Category;

@Ignore
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@drealeed I'd like to integrate these tests into the standard wsclient tests, is there anything RAMPS specific being tested? If so, could you provide ECL to recreate the structures being testsed here?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this was just a way to get a really comprehensive test of all possible record structures; I picked ramps dev because it had a lot of files for the regression test and I had access to it. And at the time the ecl errors occurred reading files from ramps dev so I pointed the test at the previously failing files to make sure they worked now.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately the files tested in testFileMetadata() no longer exist. That test can be removed. We have similar tests in tardis for different hpcc ecl structures now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless those ecl structures are proprietary, I'd like to absorb as many tests as possible. Does it make sense to donate those tests given the fact we can preemptively create the target files via ecl scripts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@drealeed do you recall how test::wsclient::noecl was created? it seems like an edge case we should continue to test

@Category(org.hpccsystems.commons.annotations.IntegrationTests.class)
public class RampsDevRegressionTest extends EclParseRegressionTest
{
Expand Down
Loading