Quarkus extension support

[StephenOTT]

Support for Quarkus.

Thoughts on what it would take?

[aklish]

Interesting idea.

Elide includes a number of JAX-RS endpoints we could leverage with Quarkus:

• JSON-API
• GraphQL
• Swagger

What remains is configuration. Quarkus feels similar to spring boot in that configuration consists in creating beans for dependency injection. Elide has the following configuration beans:

https://github.com/yahoo/elide/blob/master/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java

Spring creates some of the beans on its own like the EntityManagerFactory. Those must be resolved another way with Quarkus. Also, Elide does its own dependency injection of hooks, models, and checks. It needs
an Injector to do that:

https://github.com/yahoo/elide/blob/master/elide-core/src/main/java/com/yahoo/elide/core/dictionary/Injector.java

A custom one of these will be needed that wraps the Quarkus CDI context.

Overall, I don't think it would be too hard to get something running on Quarkus that runs on the JVM. There maybe more work to support GraalVM native images.

[aklish]

I see the description states extension. Will need to think about what else might be required to make that work.

[aklish]

Ok. Spent more time unpacking what it means to actually build an extension.

In short, Elide does some class path scanning at service boot that finds its way into the EntityDictionary. The most important part of this is discovering Elide models (static or dynamic).

We would need a small refactoring to move class path scanning behind an interface that is constructed through the DI framework.

The elide Quarkus deploy module would be responsible for doing all the class path scanning and model discovery in build steps (using Quarkas/Jandex builtin scanner). The deploy module would have to register all of the Elide models (and related types) for reflection as well as provide the list of discovered classes to a Recorder which would be responsible for ultimately initializing the new class path scanning interface (just lookups instead of scans), creating an entity dictionary, creating one or more data stores, and creating an instance of Elide. All of this could be done at static init time.

The only part of Elide that would require runtime configuration is the async API which creates a thread pool.

I do think there will likely be some work making Elide work with Graal compilation (but not sure without actually doing it). I'm sure some unknown unknowns will come up as well.

My overall thought is this is probably worth doing. I'm not sure which framework (Quarkus or Micronaut) will win the day, but I don't see a well thought out plan for extensions with Micronaut.

I think Quarkus boot times look amazing and the ability to make edits in the IDE without having to recompile and redeploy is also really nice. This is definitely nicer than Spring Boot IMHO.

Are you interested because you want to try to make this happen, or are you inquiring if Elide will support this on the roadmap? My gut feeling is this would take 2-3 weeks of dedicated work.

[StephenOTT]

Thanks for doing the initial review and providing your thoughts. Really appreciate the effort!!

My interest stemmed from working with quarkus and finding "missing" features in the graphql generation, jpa, etc. Elide seemed to be a great next step!

I would be happy to contribute and support the effort to make it happen.

I have worked with micronaut as well: doing quarkus would likely not be a lost effort when/if doing micronaut as ~same concepts and stuff to deal with.

[aklish]

#2284 is all that is needed to unblock this I think. We'll probably do another release next week.

I think a minimum POC would be to:

• Use one of the quarkus template/examples to scaffold the deploy & runtime builds.
• Create a BuildStep in the deploy module that uses Jandex to scan for the annotations here
• Create a Recorder in the runtime module that creates CDI beans for a ClassScanner & EntityDictionary using the annotations found in the prior step.
• See if the dictionary is accessible in a running quarkus service

If that can all come together, I think the rest will be just fleshing out the other beans to get a complete service.

[StephenOTT]

Sounds great. I will watch for the release and begin some testing!

[aklish]

@StephenOTT

I started playing with this. It is not turning out to be as much work as I thought it might be (thus far).

Here is the branch I'm working on: QuarkusScaffolding

It is pretty barebones, but I think the remainder needed is:

1. Hook up configuration so it controls Elide bean construction.
2. A lot of tests
3. Add and test other Elide features (async, aggregations, dynamic models, swagger, etc). These are probably lower priority.

I'd like to get this to a place where CRUD for JSON-API and GraphQL are working reasonably well. We don't use Quarkus in production here at Yahoo. I would love to get your feedback using this if you have some cycles once it has reasonable test coverage.

Also, we'll need to add a lot more Elide test models that cover use cases like:

• Security Checks
• Lifecycle Hooks
• Relationships
• Complex Attributes
• Inheritance
• GraphQL & JSON-API

Feel free to add models and tests to this branch that reflect your use cases if you have time/interest.

[StephenOTT]

For the tests: what are you worried about not working compared to non-quarkus build (with the existing tests)?

I have not done your level of analysis: so just thinking from my initial review: if the tests are checking if proper bean construction is in place/active, then what else do you want to check?

[aklish]

You have to register classes reflection. I'm not sure what would happen if you didn't. Maybe this only breaks native mode applications.

I guess the bigger concern is running in native. Lots of things might not work.

Maybe the solution here is to change Elide's integration test framework to run with Quarkus and that will cover all of the use cases.

[aklish]

Update:

I was able to get a draft extension for Elide and also build an example native application (boots in 20-50ms on my mac). I've only tested very basic things including:

1. JSON-API post & get
2. GraphQL upsert & get
3. Swagger & Graphiql schemas

Every use case really needs tests in native mode because if there is any reflection in that code path (including in dependent libraries), the use case will not work.

You can check out the bare bones Elide extension on the branch QuarkusScaffolding

I've also created an example project here:
https://github.com/aklish/elide-quarkus-example

There is still a lot of work and a lot of bugs to be found

[aklish]

One comment. To get the example project to work, the entire elide project needs to be built and installed locally (including the new elide-quarkus module).

For native, you will also need to install the GraalVM, set the right environment variables, and install its native image (all documented elsewhere).

Java 11 is required for this branch.

If you try it out and have issues with native, let me know and I'll see if I can help.

[aklish]

bash-3.2$ ./target/elide-quarkus-example-1.0.0-SNAPSHOT-runner 
produceElide
Scanning classes
produceDictionary
Overriding data store
__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2021-09-14 10:20:45,989 INFO  [com.yah.eli.Elide] (main) Registering serde for type : class java.net.URL
2021-09-14 10:20:46,015 INFO  [com.yah.eli.Elide] (main) Registering serde for type : class java.util.Date
2021-09-14 10:20:46,015 INFO  [com.yah.eli.Elide] (main) Registering serde for type : class java.util.TimeZone
2021-09-14 10:20:46,016 INFO  [com.yah.eli.Elide] (main) Registering serde for type : class java.sql.Time
2021-09-14 10:20:46,016 INFO  [com.yah.eli.Elide] (main) Registering serde for type : class java.time.Instant
2021-09-14 10:20:46,016 INFO  [com.yah.eli.Elide] (main) Registering serde for type : class java.sql.Timestamp
2021-09-14 10:20:46,016 INFO  [com.yah.eli.Elide] (main) Registering serde for type : class java.time.OffsetDateTime
2021-09-14 10:20:46,016 INFO  [com.yah.eli.Elide] (main) Registering serde for type : class java.sql.Date
2021-09-14 10:20:46,019 INFO  [io.quarkus] (main) elide-quarkus-example 1.0.0-SNAPSHOT native (powered by Quarkus 2.2.2.Final) started in 0.050s. Listening on: http://0.0.0.0:8080
2021-09-14 10:20:46,019 INFO  [io.quarkus] (main) Profile prod activated. 
2021-09-14 10:20:46,019 INFO  [io.quarkus] (main) Installed features: [agroal, cdi, elide-extension, hibernate-orm, narayana-jta, resteasy, smallrye-context-propagation]

[aklish]

To build locally, you will need to run:
mvn clean install
mvn clean install -f elide-quarkus

[StephenOTT]

Thanks. Have not lost track of this. Should be back on this shortly!

[aklish]

No worries. I'm taking a break from Quarkus to get GraphQL subscriptions out.

What's really missing for Quarkus is a way to run native tests that work with CI/CD. Then that list of tests can be expanded to make sure there aren't any hidden paths that leverage reflection and blow up in native mode.

[smsajid]

Any update on this? Is there any plan to publish this in the extensions registry of Quarkus?

[peteroyle]

Hi @smsajid, I can't speak for publishing to the registry, just a contributor here. But to put some sort of update out there - I'm still hopeful of helping to move this extension forward. Sadly the work stream I have that involves using Quarkus and Elide together has taken a temporary back seat. I hope to return to that as soon as possible.

I agree that publishing to the registry might be useful, maybe the next logical area of development.