exec chain based on Session attribute

[Andrew Krischer]

I understand that exec can take (Session => Validation[Session]) or a ChainBuilder (like http("foo").get("bar")... ). But what if I want to execute a chain stored in the session? Here's an example

- Session contains ("key", exec(http("foo").get("bar")))

I would like to say,

exec("$key") 

or 

exec(session("key").as[ChainBuilder]) 

but I see no way to access the session in an exec block in any meaningful way. To give some context, in the project I'm working on, some requests are dependent on other requests having been called in the same session already. I would ideally like to do some logic to get a sequence of ChainBuilders to execute the dependencies (this logic I already have) and then exec them, and then finally execute the first request.

...

foreach("${dependencies}", "dependency") {

 exec("$key")

}

.exec(actualRequest)

Can anyone shed some light? Thanks.

[Stéphane LANDELLE]

Not possible.

Use a doSwitch for this.

There was a very same question of SOF: http://stackoverflow.com/questions/29024763/create-a-weighted-feeder-in-gatling/29408078#29408078

Stéphane Landelle

Lead developer

slan...@gatling.io

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Krischer]

Thanks for the quick response! I've been using Gatling for the past few months now, learning as I go and it's great to have such an active community. Now onto the point...

Hmm, suppose the ChainBuilder is wrapped in a case class R, and I have R instances r1, r2, and r3. Looking at the docs it look like I should be able to do

...

foreach("{dependencies}", "dependency") {

doSwitch("${dependency}") (

  r1 -> exec(someChain1)

  r2 -> exec(someChain2)

  r3 -> exec(someChain3)

)

}

.exec(actualRequest)

Am I understanding you correctly?

[Stéphane LANDELLE]

foreach("{dependencies}", "dependency") {

doSwitch("${dependency}") (

  r1 -> exec(someChain1)

  r2 -> exec(someChain2)

  r3 -> exec(someChain3)

)

}

.exec(actualRequest)

 

Am I understanding you correctly?

Yes.

If someChainXXX are chains, you don't need the execs:

doSwitch("${dependency}") (

  r1 -> someChain1

  r2 -> someChain2

  r3 -> someChain3

)

[Andrew Krischer]

Ah, very cool. Thanks! I just tried a proof-of-concept test and it does seem to be behaving as I want :)

The only followup I have is, is there a way to reference the key in the case statements? All I actually want is

doSwitch("${dependency}") (

  "${dependency}" -> "${dependency}"

)

which obviously does not work as is.

Maybe understanding more of what's happening under the covers can achieve this?

[Andrew Krischer]

doSwitch("${dependency}") (

  "${dependency}" -> "${dependency}"

)

 

edit: seems like it's an alias for Map[Expression[Any], Any]?

[Stéphane LANDELLE]

Once again: no.

Stéphane Landelle

Lead developer

slan...@gatling.io

--

[Andrew Krischer]

Hm, that's extremely unfortunate. I've now run into a problem where the builder goes into an infinite loop:

I have my dependency map

foreach("{dependencies}", "dependency") {

doSwitch("${dependency}") (

  dependency1 -> dependency1

  dependency2 -> dependency2

  dependency3 -> dependency3

)

}

and the general structure for executing a request is exec(dependencies).exec(request). So I have

dependency1 = exec(dependencies).exec(request), and remember a dependency is simply a different request

So when Gatling's building, it gets stuck in an infinite loop:

dependency1 --> exec(dependencies)

dependencies --> doSwitch statement

doSwitch statement --> dependency1

dependency1 --> exec(dependencies)

A dependency will never be a dependency of itself, but it looks like in the build stage it gets stuck in an infinite loop, probably because technically any case could be matched. That's why I'm slightly irked that I have to use doSwitch as an exec and I have to enumerate each case.

The loop could easily be broken if I had access to the session within an exec block or if I could return a value dynamically in the doSwitch block.

[Andrew Krischer]

I've been racking my brain trying to come up with a solution. Is there any way at all to do a sort of do that takes an Expression[T]?

[Stéphane LANDELLE]

If what you mean is: can I build Gatling DSL components and chain them at runtime while the virtual users are running, that's still a big NO.

Gatling DSL components are built and chained together when the Simulation instance is created.

Then, you've been very theoretical so far and described a technical solution (case classes, expressions) you have in mind.

What's your need exactly? You might just be heading the wrong direction.

Stéphane Landelle

Lead developer

slan...@gatling.io

[Andrew Krischer]

Then, you've been very theoretical so far and described a technical solution (case classes, expressions) you have in mind.

What's your need exactly? You might just be heading the wrong direction.

True, that can very much be the case, haha. So here's some context:

I'm building a testing framework to load test an api server. All endpoints are RESTful, and my strategy is to build a bunch of Gatling endpoints that much up to the server's endpoints. Then I can create scenarios using Gatling that mirror real-world use cases. I decided to break each endpoint into 3 parts: Dependencies on other endpoints, OAuth, and the actual request. Most requests (but not all) require OAuth. These are my main concerns:

- Easily modify the guts of the request while minimizing code copy-n-paste. This includes specifying the action (get, patch, update etc) and the types of params (body, url, form, etc)

- Easily specify Dependencies a request relies on. A Dependency is another endpoint that in one way or another sets session variable that this request uses.

- Easily execute OAuth

- For any endpoint, be able to omit Dependency or OAuth execution.

Here's my current structure:

abstract class Request extends Requestable represents a Request which contains the meat of the request (url, name, params, etc...). To create Gatling endpoints, I create objects which extend Request. The use case is that I can then simply call exec(Endpoint) to use an endpoint.

Requestable is a trait which has

var request: ChainBuilder

def executable: ChainBuilder

var dependentParams: Set[String]

var paramsSet: Set[String]

The idea behind Requestable is it represents a request to execute. There's an implicit conversion from Requestable --> ChainBuilder via executable.

What is really tripping me up is how I'm resolving dependencies. Currently, the abstract Request class has a case class Dependencies(request: Request) extends Requestable. The input is used to getDependencies(request, session) so that it can execute each. getDependencies simply returns a Seq[Requestable] representing a sequence of endpoints that should be executed before the actual request is executed. Here's the Dependencies case class:

case class Dependencies(req: Request) extends Requestable {

  request = 
    // Anytime an endpoint is added, it should also be placed here. This allows it to be executed as a Dependency.
    // Currently, the builder runs into an infinite loop since each endpoint invariably points to itself in dependencies...
    foreach(session => DependencyManager.getDependencies(req, session), "dependency") {

      doSwitch("${dependency}") (

        CreateUser -> CreateUser,
        GetUserInformation -> GetUserInformation,
        UpdateUserInformation -> UpdateUserInformation
      )
    }
}

Request has these methods which utilize the case classes. And remember, request is a var in Requestable, which is the ChainBuilder that represents the actual request.

protected final def dependencies: Dependencies = Dependencies(this)
protected final def oauth = OAuth(this)

To give an example of how I then create an endpoint:

object UpdateUserInformation extends Request {
  jsonBody = (session: Session) =>
    s"""
       |"user":${session("user").as[String] + "a"},
       |"email":"newEmail${session("user").as[String]}@something.com",
       |"pass":"fooPass"
     """.stripMargin

  paramsSet = Set()
  dependentParams = Set("user", "email", "pass", "userId")
  addDependency(this)
  requestName = "Update User Information"
  url = "foo"

  request = exec(
    checkRequest(
      http(requestName)
        .patch(session => baseUrl + url + session("userId").as[String])
        .header("Authorization", "${accessToken}")
    )
  ).pause(Environments.shortPause)

  protected override def checkRequest(request: HttpRequestBuilder): HttpRequestBuilder = {
    request.check(
      status.is(200),
      jsonPath("$.user").saveAs("user"),
      jsonPath("$.email").optional.saveAs("email"),
      jsonPath("$.pass").optional.saveAs("pass")
    )
  }
}

in this case, executable is not overridden, and by default is exec(dependencies).exec(oauth).exec(request)

Ok, so that's a lengthy explanation but I'd rather give more context rather than less. I do appreciate your help. My background is mostly object-oriented--Java and C#--so I tend to think very much in that mindset. There may be a simple solution I'm missing. If so, I'd love to know :)

[Andrew Krischer]

I'm wondering if anyone has suggestions or a direction I should go in?

[Stéphane LANDELLE]

And it seems to me that you're moving a lot of complexity into your DependenciesManager.

I failed to see why you need this manager in the first place. It makes the relation between a request and its dependencies non trivial.

Why not simply have:

abstract class Request(req: Request, dependencies: Request*) {

  def toChain() = 

   exec(dependenciesToChain)

   .exec(req)

}

It's not clear to me if a Dependency is supposed to be executed every time a Request is, or at least once.

If the latter, it's the a matter of setting a flag into the session, and check if it exists.

Stéphane Landelle

Lead developer

slan...@gatling.io

--

[Andrew Krischer]

 

And it seems to me that you're moving a lot of complexity into your DependenciesManager.

I failed to see why you need this manager in the first place. It makes the relation between a request and its dependencies non trivial.

The idea here is to separate logic for finding and executing dependencies. I believe an endpoint should not contain logic for finding its dependencies, but rather execute what's given.

Why not simply have:

abstract class Request(req: Request, dependencies: Request*) {

  def toChain() = 

   exec(dependenciesToChain)

   .exec(req)

}

Hmm, this definitely seems like a good idea :)

It is in fact the latter--to only call the dependency if it hasn't been called yet. for exec(dependencies.toChain), how does that work exactly with passing in multiple ChainBuilders? E.g. could I call exec(Seq[ChainBuilder])?

In that case I could call exec(getDependencies(this)) to execute all needed dependencies.

In terms of mutability, the only mutable parts I see are the implementation parts of the different Request objects. In fact, I had to make some parts mutable that were immutable because Gatling saves Sequences as scala.collection.Seq in the session attributes. I'm going to modify the project based on your suggestions, thank you. But I'm curious what areas are particularly concerning to you.

Best,

Andrew

[Stéphane LANDELLE]

The idea here is to separate logic for finding and executing dependencies. I believe an endpoint should not contain logic for finding its dependencies, but rather execute what's given.

Basically, you prefer ServiceLocator over Dependency Injection.

Can't say I'd go this way.

Hmm, this definitely seems like a good idea :)

It is in fact the latter--to only call the dependency if it hasn't been called yet. for exec(dependencies.toChain), how does that work exactly with passing in multiple ChainBuilders? E.g. could I call exec(Seq[ChainBuilder])?

Absolutely: https://github.com/gatling/gatling/blob/master/gatling-core/src/main/scala/io/gatling/core/structure/Execs.scala#L28-L30

 

In terms of mutability, the only mutable parts I see are the implementation parts of the different Request objects.

Yep. Ugly, and dangerous: depend on some proper initialization lifecycle, at some point, you might need regular instances and not objects, etc.

Non local mutability is usually a code smell.

Here, the only reason you use mutability is because of your DependenciesManager design.

 

In fact, I had to make some parts mutable that were immutable because Gatling saves Sequences as scala.collection.Seq in the session attributes.

Sorry, I don't get how this is related.

[Andrew Krischer]

Yeah, I suppose I am preferring the Service Locator pattern. In any case, I feel like I've found a solution that I'm both happy and slightly sad about (since I still need to explicitly list each endpoint's dependencies):

private val checkAndInitDependency = (dependency: Requestable) => {
  val params: Seq[String] = dependencies.map((d: Dependency) => d.requestable).find((r: Requestable) => dependency == r) match {
    case r: Option[Requestable] => r.get.paramsSet
    case _ => Seq()
  }
  foreach(params, "param") {
    doIf(session => !session.contains(session("param").as[String])) {
      dependency
    }
  }
} : ChainBuilder

It's used like so:

dependencies = DependencyManager.checkAndInitDependencies(CreateUser, GetAllUsers)

It's still mutable, so I'm sure you're still shaking your head haha. I am too. But at the moment it's working, isn't messy, and can be refactored if needed in the future.

I appreciate your help in all of this. Since dependencies cannot be circular, objects work, but I agree it's smelly that it's now dependent on how it's initialized.

Thanks again!