REST Client using Multipart

[KimJohn Quinn]

Hello everyone.

First, our experience moving over to Quarkus has been absolutely spectacular!  Everything, including the docs, have been amazing and my only regret is not making the jump earlier.

I have run into an issue with the REST client, RESTEasy, and multipart payloads.

If I invoke the resource directly, through a test case or even curl, it seems to work fine.  Invoking any "client" methods  

The issues I am having are:

1. If I move the multipart payload to a different module (jar) I get this error in the resource "ClassNotFoundException: io.platform.api.publish.FilePackage_generated_populator"
   
   
2. The RESTClient will not generate the multipart signature/payload when the multipart signature is present.  Other methods/payloads work fine.  Here I get " Failed to generate client for class interface io.platform.api.publish.PublishClient : Unsupported multipart form field type: java.util.List<java.io.File> in field" yet I use the exact same class, with the resource, and it seems to work directly.
   
   
3. I am confused about the dependencies, we have standardized on a reactive approach and not sure what the interaction is between resteasy vs. mutiny vs. the Quarkus rest client, etc.  For a while I had issues with conflicting reactive vs. non-reactive when building.

The relevant code is:

Client Definition:

@POST

@Path("/{client}/package")

@Consumes(MULTIPART_FORM_DATA)

@Produces(APPLICATION_JSON)

@Operation(hidden = true)

Uni<Version> publish(@PathParam("client") final String client, @MultipartForm FilePackage request);

Client MultiPart Payload:

public class FilePackage {

    @FormParam("project")

    public String project;

    @FormParam("artifact")

    public String artifact;

    @FormParam("folder")

    public String folder;

    @FormParam("index")

    @PartType(APPLICATION_JSON)

    public Map<String, Integer> index;

    @FormParam("file")

    @PartType(APPLICATION_OCTET_STREAM)

    public List<File> files;

    public List<FileObject> getPackageFiles() { ... }

}

Resource Method (note PackageUpload is exactly the same as the FilePackage above - if i only use the simple parameters it marshals from the client to this resource fine - when I uncomment the files or index it fails - the reason I have two payloads, the same, is because of issue #1 above):

@POST

@Path("/{client}/package")

@Consumes(MULTIPART_FORM_DATA)

@Produces(APPLICATION_JSON)

public Uni<Version> publishPackage(@MultipartForm final PackageUpload upload, @BeanParam final ClientRequest request) throws Exception { ... }

Client Test Cases:

@Inject

@RestClient

PublishClient publishClient;

This works using the client...

final JsonPackage pkg = testSupport.readJson(JsonPackage.class);

Version actual = publishClient.publish("my-client", pkg).await().indefinitely();

assertNotNull(actual);

I dont even get here with the client...

final FilePackage pkg = new FilePackage();

pkg.project = "my-project";

pkg.artifact = "my-artifact";

pkg.folder = "my-folder";

pkg.index = new HashMap<>();

pkg.files = new ArrayList<File>();

Version actual = publishClient.publish("my-client", pkg).await().indefinitely();

assertNotNull(actual);

This "resource" test looks to work (direct / no client)...

given().pathParam("client", "my-client")

        // Folder

        .formParam("project", "my-project")

        .formParam("artifact", "my-artifact")

        .formParam("folder", "my-folder")

        // Index

        .multiPart("index", index, APPLICATION_JSON)

        // Files

        .multiPart("file", file)

        .multiPart("file", file)

        .multiPart("file", file)

        .when()

        .post("/s3/publish/{client}/package")

        .then()

        .log().everything(true)

        .statusCode(OK.getStatusCode());

Maven Dependencies:

<dependency>

    <groupId>io.quarkus</groupId>

    <artifactId>quarkus-rest-client-reactive-jackson</artifactId>

 </dependency>

 <dependency>

     <groupId>io.quarkus</groupId>

     <artifactId>quarkus-resteasy-reactive-jackson</artifactId>

 </dependency>

Any help or suggestions would be appreciated...

[Michał Szynkiewicz]

Sending a collection of files as you try to do is not supported.

If you know in advance what files you want to send, you can have them in separate fields and this should work.

Take a look at [1] for an example of client definition and payload definitions.

Sending a collection of files may make sense in some cases. Can you create an issue in Github issues to discuss supporting it?

HTH,

Michał

[1] https://github.com/quarkusio/quarkus/blob/main/integration-tests/resteasy-reactive-rest-client-multipart/src/main/java/io/quarkus/it/rest/client/multipart/MultipartClient.java#L20

--
You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/c2c5b855-b7d0-41d4-8293-a2ea7cacbc16n%40googlegroups.com.

[Michał Szynkiewicz]

Re the other issues: 

Could you create a reproducer for the ClassNotFoundException issue, and create a Github issue with it attached?

quarkus-rest-client-reactive works with Mutiny without any additional extensions. I saw a few times people thought they needed quarkus-rest-client-mutiny to add support for it and ended up in a mix.

REST extensions work best if you don't mix reactive and classic versions. We've recently added an error message to warn users if they do that in their application.

[KimJohn Quinn]

Thanks Michal.

Sure, I can create an issue but let me double-check first, from my test-cases, which finish up with an S3 check against the buckets, it "looked" like I was sending multiple files.  

More of my concern isn't necessarily a single or multiple file (i can work around multiple files) - it's the fact I can seem to create a REST client that consumes a multipart body and sends it to the service.

I have a similar payload for a SingleFileUpload (see below).  I get the same issues trying to create the REST client with the Multipart and if I move the payload class to another module it gives me the same "cannot find populator" error - issues #1 and #2 in the original post (I will double-check just to make sure shortly).

Thanks.

Single File Upload (request):

public class FileUpload {

    @FormParam("file")

    @PartType(APPLICATION_OCTET_STREAM)

    public File data;

    @FormParam("filename")

    @PartType(TEXT_PLAIN)

    public String fileName;

    @FormParam("path")

    @PartType(TEXT_PLAIN)

    public String path;

    @FormParam("mimetype")

    @PartType(TEXT_PLAIN)

    public String mimeType;

}

--

KimJohn Quinn
k...@logicdrop.com

Logicdrop
O 888.229.2817
M 248.882.0728
www.logicdrop.com

[KimJohn Quinn]

Thanks Michal.

Sure, I can create an issue but let me double-check first, from my test-cases, which finish up with an S3 check against the buckets, it "looked" like I was sending multiple files.  

More of my concern isn't necessarily a single or multiple file (i can work around multiple files) - it's the fact I can seem to create a REST client that consumes a multipart body and sends it to the service.

I have a similar payload for a SingleFileUpload (see below).  I get the same issues trying to create the REST client with the Multipart and if I move the payload class to another module it gives me the same "cannot find populator" error - issues #1 and #2 in the original post (I will double-check just to make sure shortly).

Thanks.

Single File Upload (request):

public class FileUpload {

    @FormParam("file")

    @PartType(APPLICATION_OCTET_STREAM)

    public File data;

    @FormParam("filename")

    @PartType(TEXT_PLAIN)

    public String fileName;

    @FormParam("path")

    @PartType(TEXT_PLAIN)

    public String path;

    @FormParam("mimetype")

    @PartType(TEXT_PLAIN)

    public String mimeType;

}

[KimJohn Quinn]

Michal.

You were right!  I could have sworn everything was not working but maybe I just needed to step back for a bit.  It does fails, specifically, with the REST Client, Multipart, and multiple files/other parts.  The simple happy path seems to work.

I've built a series of scenarios...

• Issue 1 - Having the "payload" defined in the API gives the "missing populator class" exception.
• Issue 2 - Validates the multipart client (with multiple files) is the culprit
• Issue 3 - Adding an additional part type (JSON) to the single file throws the "Unrecognized part" error in the client.

...and...

• Working 1 - Shows (i think) that multipart/multi files seems to work when going directly to the resource with the payload.
• Working 2 - Shows the single file scenario working.

I think I can work around all of these issues but I am curious why #1 an issue?

Thanks.

[Michał Szynkiewicz]

With "If I move the multipart payload to a different module (jar) I get this error in the resource "ClassNotFoundException: io.platform.api.publish.FilePackage_generated_populator"" - this is the server part, I don't know that much about it.

Could you make sure the jar you move the payload class to has a Jandex index? You can find instructions on adding it here: https://stackoverflow.com/questions/55513502/how-to-create-a-jandex-index-in-quarkus-for-classes-in-a-external-module

[KimJohn Quinn]

Hello Michal.

Regarding Jandex, all the projects are invoking the Jandex plugin and I see the IDX in the classes (I forgot to include Jandex in my reproducer).  I also tried adding an empty "beans.xml".  Finally, excluding "multipart" payloads any other classes uses in other services are working which seems to demonstrate the Jandex stuff is being picked up (?).  I have other REST payloads and client payloads that are recognized.

[INFO] --- jandex-maven-plugin:1.1.0:jandex (make-index) @ fusion-platform-api ---

[INFO] Saving Jandex index: /home/.../api/target/classes/META-INF/jandex.idx

I created this GitHub ticket - https://github.com/quarkusio/quarkus/issues/19037.

Thanks.

[Michał Szynkiewicz]

Thanks, I'll try to take a look at the issue tomorrow. Could you also create issues (of type enhancement) for the two other things?

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/1aa8dfd8-fb5f-4243-a479-0ea167c76005n%40googlegroups.com.

[KimJohn Quinn]

I created these two enhancements in GitHub per your suggestion (I believe they are both specific to usage within the REST client as I have tests going directly to the resource and it appears to work):

• Support uploading multiple files - https://github.com/quarkusio/quarkus/issues/19041
  
• Support multiple parts in payload - https://github.com/quarkusio/quarkus/issues/19042

Thanks.