WIP: Maven -> Gradle

[vlsi]

This migrates the build system to Gradle.

Pros:

• afl-proxy can be built with the same tool
• api, implementation, test implementation and test runtime classpath is much easier to configure
• build scripts are much shorter

WDYT?

Just in case, sample GitHub Actions CI output can be seen here: https://github.com/vlsi/jqf/commit/38675b76e5fdf794b7a88b3e0f022753406b43d9/checks?check_suite_id=409014475 (see https://github.com/vlsi/jqf/actions)

Note: this PR does not yet build jqf-maven-plugin, however, it is good enough to build all the rest jars, and publication to Maven Central should work as well.

If you are not familiar with Gradle, you might https://github.com/dougborg/gdub useful.

Sample commands to try:

$ ./gradlew # prints basic info
$ ./gradlew tasks
$ ./gradlew projects

$ ./gradlew :jqf-fuzz:dependencies

$ ./gradlew build # build everything
$ ./gradlew :jqf-fuzz:build # build everything
$ ./gradlew :jqf-fuzz:jar # build jar for jqf-fuzz (the jar is placed to fuzz/build/libs/...)
$ ./gradlew :jqf-fuzz:test # run tests for jqf-fuzz
$ ./gradlew :afl-proxy:build # build afl-proxy (the binary is placed to afl-proxy/build/exe/main/debug)

[vlsi]

Just in case, sample GitHub Actions CI output can be seen here: https://github.com/vlsi/jqf/commit/38675b76e5fdf794b7a88b3e0f022753406b43d9/checks?check_suite_id=409014475 (see https://github.com/vlsi/jqf/actions)

[vlsi]

Ok, basic build and test works in Linux and macOS: https://github.com/vlsi/jqf/commit/527c70ea6ec01806b29afd8155ee1bc7cd0d30a0/checks?check_suite_id=409048657

It looks like there are compilation failures for afl-proxy in Windows. @rohanpadhye , am I right it never worked in Windows?

[rohanpadhye]

@rohanpadhye , am I right it never worked in Windows?
That's right. AFL on Windows has always been problematic (even in C/C++ land, I believe).

In general, I'm glad that JQF can be built with Gradle, but I don't see why this PR needs to be merged into master. I am currently happy with Maven and do not see the need to migrate to a different build system. As you pointed out in #77, it is possible to use JQF as-it-is from a Gradle project. My understanding is that Gradle projects can pull in artifacts from Maven central pretty easily.

If you wish to develop and maintain a Gradle plugin similar to the jqf-maven-plugin, your contribution is welcome. But I think JQF will stick to using Maven for its own build for now.

[vlsi]

I don't see why this PR needs to be merged into master.

Well. Gradle simplifies a lot of things, and the entry curve is very straightforward.

With Maven you have to build modules (==install to Maven local repository) in the right order, otherwise, it won't work.
With Gradle you just tell it what you need and the build system figures out the steps to build it.

For instance, git clone jqf; cd fuzz; mvn install does not work with Maven.

I've recently migrated Apache JMeter (Ant -> Gradle) and Apache Calcite (Maven -> Gradle).
Both projects have active community and there were lots of emails like "we are happy with the current build system".
However, after the change no-one complained, and there was lots of feedback that build became faster, easier to maintain, and so on.

Gradle simplifies things like "making a classpath".
For instance, I see you have a bunch of shell scripts.
With Gradle it would be much easier to build the classpath for the script.

For instance, here's what we have in Apache Calcite:
https://github.com/apache/calcite/blob/8fb5ca9898dc2c39a3b708a29f95a8904423c871/sqlline#L32-L37

sqlline script resolves the required dependencies and launches Sqlline.
Note: it builds the relevant project jars if they are needed.

Here's how the classpath is declared:
https://github.com/apache/calcite/blob/8fb5ca9898dc2c39a3b708a29f95a8904423c871/build.gradle.kts#L160-L199

Gradle makes sqlline script easy to follow, and the dependencies are declared only once.

With Maven you copy dependencies to target/dependency, however you need to clean that folder from time to time, otherwise, duplicate jars might appear there (e.g. asm-6.0.jar and asm-7.0.jar at the same time)
With Gradle you don't copy files (which is good), and you depend on the right versions.

Note: fuzz depends on multiple different asm versions:

--- maven-dependency-plugin:2.8:tree (default-cli) @ jqf-fuzz ---
edu.berkeley.cs.jqf:jqf-fuzz:jar:1.4-SNAPSHOT
+- junit:junit:jar:4.12:compile
|  \- org.hamcrest:hamcrest-core:jar:1.3:compile
+- com.pholser:junit-quickcheck-generators:jar:0.8:compile
+- com.pholser:junit-quickcheck-core:jar:0.8:compile
|  +- org.javaruntype:javaruntype:jar:1.3:compile
|  |  \- org.antlr:antlr-runtime:jar:3.1.2:compile
|  +- ognl:ognl:jar:3.1.12:compile
|  |  \- org.javassist:javassist:jar:3.20.0-GA:compile
|  +- ru.vyarus:generics-resolver:jar:2.0.1:compile
|  \- org.slf4j:slf4j-api:jar:1.7.25:compile
+- edu.berkeley.cs.jqf:jqf-instrument:jar:1.4-SNAPSHOT:compile
|  \- org.ow2.asm:asm:jar:7.2:compile  <-- HERE
+- org.jacoco:org.jacoco.report:jar:0.8.2:compile
|  \- org.jacoco:org.jacoco.core:jar:0.8.2:compile
|     +- org.ow2.asm:asm-commons:jar:6.2.1:compile <-- HERE
|     |  \- org.ow2.asm:asm-analysis:jar:6.2.1:compile
|     \- org.ow2.asm:asm-tree:jar:6.2.1:compile
+- org.mockito:mockito-core:jar:2.23.4:test
|  +- net.bytebuddy:byte-buddy:jar:1.9.3:test
|  +- net.bytebuddy:byte-buddy-agent:jar:1.9.3:test
|  \- org.objenesis:objenesis:jar:2.6:test
+- org.hamcrest:hamcrest-library:jar:1.3:test
\- info.picocli:picocli:jar:4.0.4:compile

it is possible to use JQF as-it-is

Even though it is possible, I would like to highlight that Gradle has composite builds
What it does it allows to build multiple independent projects as if it were a single build.

For instance, if I want to alter jqf a bit (e.g. like in #79 ), I need the following:

1. Build JQF, install it to maven local repository
2. Launch my own build to evaluate if the change worked

If JQF build was Gradle-based, then I could just launch my own build, and Gradle would pick jqf transparently. No maven local would be required.

[rohanpadhye]

Thanks for all the relevant points. I agree that Gradle probably has more features and customizability than Maven, as evidenced by the fact that so many projects are making the migration. However, projects like JMeter and Calcite have a much larger community than JQF, which also fosters a better development and support network. The issue with making a large change in JQF (this includes this PR as well as suggestions such as #80) is primarily that of human resources.

I am currently the only active maintainer of JQF*. The original implementation of JQF was motivated by the need for a research platform to experiment with new structure-aware fuzzing algorithms, and it has been successful at that (e.g. recent extensions using RL).

While I am happy to spend time fixing bugs in JQF and making it more usable for users who would like to extend JQF (e.g. #52) , who require more control over hardcoded configs or options (e.g. #75), or to add features that will help find bugs in real systems (e.g. #25, #26), I do not currently have the bandwidth to learn how Gradle works and ensure that I can continue maintaining and supporting JQF if its build system is changed from what I am familiar with.

If you would like to host and maintain a fork of JQF that uses Gradle to build and that will be in-sync with master, I would be more than happy to include a link in the README to your fork. However, unless there is a task that is currently impossible to do in the current version of JQF, I am not enthusiastic about merging this PR.

* If someone more familiar with Gradle takes over the project, build, and release management, that is a different scenario.

PS: 47d20a9 is a great change, and I'll be happy to merge this in isolation. Thanks!

[vlsi]

If you would like to host and maintain a fork of JQF that uses Gradle to build and that will be in-sync with master

I see no value in that

[stheid]

Thanks for publishing this PR