Load Testing for thrift API

[Kumar Pratik]

Hello,

I was looking into iago and thought of giving it a try for my java thrift api load test. Recently read a thread in this group which says that "iago strictly can't be used with  apache-thrift generator?"

Does this means that I will not be able to use iago library? 

Also I am little confused from where I have to start? Any pointers would be very helpful.

Thanks,

Pratik

[James Waldrop]

Iago uses Finagle, which is strictly async (as that ends up being a preferable way to model RPC in general, and certainly if you want to do performant RPC). The Apache Thrift generator only produces synchronous interfaces. Without Finagle-aware interfaces, Iago will not work properly. You will end up occupying the request dispatch thread sending requests in serial, which will highly constrain the degree to which your load test can scale (and will end up defeating most of the value in using an approach like Iago in the first place).

For more info on Finagle, you can look here: http://twitter.github.com/finagle

Note this does *not* mean your service has to use Finagle. You simply have to produce Finagle-ized Thrift bindings for Iago to consume. This is relatively straightforward.

--
 
---
You received this message because you are subscribed to the Google Groups "Iago Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iago-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Kumar Pratik]

Hello James,

Thanks for your input. I have decided to give Iago a try and see where it will land me.

I have downloaded the project from https://github.com/twitter/iago/zipball/master 

Unzipped the project

Have done mvn package and mvn install

Created a tmp directory

copied the jar file iago-0.6.8.jar to tmp

Now when I am running the command java -jar iago-0.6.8.jar -f ../config/test-thrift.scala , I am getting following error:

Error: Could not find or load main class com.twitter.parrot.launcher.LauncherMain

Could you please let me know what I am missing?

Thanks in advance.

Pratik

[James Waldrop]

Run from target. 

[Tom Howland]

By "run from target" James means you have to run from a place where java can find the jars your iago is depending upon.

Iago ships with a simple thrift example:

https://github.com/twitter/iago/tree/master/examples/echo

[Kumar Pratik]

I was referring to the thrift example you have mentioned.

I executed cd examples/echo

Now when I run mvn package I am getting following error:

Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.0.1:compile (scala-compile-first) on project iago-echo: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1) -> [Help 1]

Any help.

Thanks

[Tom Howland]

Hi Kumar

Sounds like your maven doesn't know how to invoke the scala compiler. If I were you, I'd verify that your maven-scala plugin is working.

[Kumar Pratik]

Thank you. That error is gone after i modified pom.xml for echo example but now I am getting this error:

[ERROR] Failed to execute goal com.twitter:maven-finagle-thrift-plugin:0.0.7:compile (thrift-sources) on project iago-echo: Execution thrift-sources of goal com.twitter:maven-finagle-thrift-plugin:0.0.7:compile failed: Plugin com.twitter:maven-finagle-thrift-plugin:0.0.7 or one of its dependencies could not be resolved: Failed to collect dependencies for com.twitter:maven-finagle-thrift-plugin:jar:0.0.7 (): Failed to read artifact descriptor for com.twitter:util-core:jar:4.0.1: Could not transfer artifact com.twitter:util-core:pom:4.0.1 from/to artifactory (http://artifactory.local.twitter.com/repo): artifactory.local.twitter.com: Unknown host artifactory.local.twitter.com -> [Help 1]

Please help

[Tom Howland]

Hi Kumar

Are you developing Iago on Windows? Iago won't build on Windows although a fix is in the pipeline.

What is happening is maven is looking for the maven-finagle-thrift-plugin and isn't finding one for Windows. There is a hardcoded search list in one of the parent poms that makes it look first in Maven Central and then, failing that, the Twitter Artifactory.

[Kumar Pratik]

Hey I am working on Ubuntu. 

Also I notice in the error it says "maven-finagle-thrift-plugin:0.0.7" where as in POM.xml version for maven-finagle-thrift-plugin is defined as 0.0.9

Thanks,

[Kumar Pratik]

Any solution for this?

[Trisha Quan]

Hi Kumar,

Can you attach the pom.xml you're using in iago-echo and the command you're using, so we can repro?

[Kumar Pratik]

Hi,

 Command which I have tried is

mvn package and also mvn clean install

I am trying to build examples/echo inside the twitter-iago-cb7ac23

I am not sure how can i attach file here, Here is the content of POM

<?xml version="1.0"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

  <modelVersion>4.0.0</modelVersion>

  <groupId>com.twitter</groupId>

  <artifactId>iago-echo</artifactId>

  <packaging>jar</packaging>

  <version>1.0</version>

  <parent>

    <groupId>com.twitter</groupId>

    <artifactId>service-292</artifactId>

    <version>0.0.12</version>

  </parent>

  <properties>

    <git.dir>${project.basedir}/../../.git</git.dir>

    <mainClass>com.twitter.parrot.launcher.LauncherMain</mainClass>

  </properties>

  <repositories>

    <repository>

      <id>Twitter public Maven repo</id>

      <url>http://maven.twttr.com</url>

    </repository>

  </repositories>

  <pluginRepositories>

    <pluginRepository>

      <id>Twitter public Maven repo</id>

      <url>http://maven.twttr.com</url>

    </pluginRepository>

  </pluginRepositories>

  <dependencies>

    <dependency>

      <groupId>com.twitter</groupId>

      <artifactId>iago</artifactId>

      <version>0.6.8</version>

    </dependency>

  </dependencies>

  <build>

  <pluginManagement>

    <plugins>

      <plugin>

        <groupId>com.twitter</groupId>

        <artifactId>maven-build-properties-plugin</artifactId>

        <configuration>

          <buildPropertiesPackage>com.twitter.iago-echo</buildPropertiesPackage>

        </configuration>

      </plugin>

      <plugin>

        <groupId>com.twitter</groupId>

        <artifactId>maven-finagle-thrift-plugin</artifactId>

<version>0.0.8</version>

        <configuration>

          <thriftGenerators>

            <thriftGenerator>finagle</thriftGenerator>

          </thriftGenerators>

        </configuration>

      </plugin>

      <plugin>

                <groupId>net.alchim31.maven</groupId>

               <artifactId>scala-maven-plugin</artifactId>

      <version>3.0.1</version>

                <executions>

                    <execution>

                        <goals>

                            <goal>compile</goal>

                            <goal>testCompile</goal>

                        </goals>

                    </execution>

                </executions>

            </plugin>

    </plugins>

    </pluginManagement>

  </build>

</project>

Thanks

[Kumar Pratik]

I have finally did a new clone and built the project successfully. 

Closing the thread

[James Waldrop]

Thanks for the update.