Procedure call graph

[Steve Upton]

Hi All,

Fairly new to Netlogo. We have a good sized model and would like to create a procedure call graph, i.e., for each procedure, identify what procedures it calls, and so forth. I've found a post on reflection and can get the procedures (as well as globals, turtles, breeds), so about to explore further, but wondered if this wasn't already available.

thank you

steve upton

[Forrest Stonedahl]

You might take a look at the profiler extension (or source code thereof), if you haven't already...

https://github.com/NetLogo/Profiler-Extension

Cheers,

Dr. Forrest Stonedahl
Computer Science
Augustana College

--
You received this message because you are subscribed to the Google Groups "netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netlogo-deve...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netlogo-devel/4cbab5a6-b52a-48d5-87fa-e272397c4e7bo%40googlegroups.com.

[Steve Upton]

Thank you for the pointer. Hadn't looked at that yet. The profiler extension is a start, but it only gives me the calls to the procedures, not what other procedures called that procedure, etc., in a graph. From your pointer, I found the Reflection-Extension, which looked like it produce what I want, but if I'm reading it correctly, I'd have to put calls to "reflection:procedure" and "reflection:callers" in each procedure. I was hoping for something more like a static analysis of the netlogo code.

Seth Tissue had posted code back in 2012 that got me started, and I had been looking at the procedures map that is returned. I can get a dump but don't see how I can just get the children's names (i.e., the callers). With the dump,  I can at least parse that to get what I'm looking for.

thanx

steve

On Tuesday, June 9, 2020 at 8:45:40 PM UTC-4, Forrest wrote:

You might take a look at the profiler extension (or source code thereof), if you haven't already...

https://github.com/NetLogo/Profiler-Extension

Cheers,

Dr. Forrest Stonedahl
Computer Science
Augustana College

On Tue, Jun 9, 2020, 11:39 AM Steve Upton <upto...@gmail.com> wrote:

Hi All,

Fairly new to Netlogo. We have a good sized model and would like to create a procedure call graph, i.e., for each procedure, identify what procedures it calls, and so forth. I've found a post on reflection and can get the procedures (as well as globals, turtles, breeds), so about to explore further, but wondered if this wasn't already available.

thank you

steve upton

--
You received this message because you are subscribed to the Google Groups "netlogo-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to netlog...@googlegroups.com.

[Seth Tisue]

I think you will need to walk the ASTs representing the procedure bodies and look for _call and _callreport nodes.

Seth

[Steve Upton]

Maybe a teensy bit more help :-)

I found this post "analysis/visualization of internal representations?" but still not grokking how to get the FrontEnd.

object ExtractProcedures2 {

  def extract(filename : String) : Unit = {

    val workspace = HeadlessWorkspace.newInstance

    workspace.open("models/Fire.nlogo")

val frontend : FrontEndInterface = workspace.compiler.frontEnd

/* val procNames = procs.keySet

println("printing the procNames")

procNames foreach println

println("*******printing the calls")

procs.foreach{case(k,p) => {

println(s"working procedure: $k")

p.code foreach println

}}  */ 

    workspace.dispose()

  }

}

but the FrontEndInterface isn't getting me the AST (i'm sure I'm missing something simple).

steve

[Seth Tisue]

It's now been more than five years since I worked on NetLogo; there's a limit to how much I still remember.

In particular, I don't remember whether it's possible to get back to the ASTs after the entire compiler has run, or whether they have already been discarded and it's too late.

I found some prior discussions in this area in the NetLogo room on Gitter:

* https://gitter.im/NetLogo/NetLogo?at=5be1a45dda57ff676c9d4305  — "you want `FrontEndResults` and not `CompilerResults`. "FrontEndResults contains ProcedureDefinition objects, these have the ASTs inside them. see AstNode.scala"

* https://gitter.im/NetLogo/NetLogo?at=5bba30ccc7bf7c3662cda279

But anyway, although my first thought was to walk the ASTs, on second thought it might be easier and just as effective for your use case to walk the fully compiled model (the `Array[Command]` inside each procedure) and look for the `_call` and `_callreport` nodes there. Any `_call`s will be in the `Array[Command]` directly, but to find the `_callreport`s you'll need recursively walk the `args` tree under each `Command`.

Seth

[Steve Upton]

Shoot, has it been that long??

Your answers, and references, are enough to get me going. I'll parse the Commands since I don't see where to get the FrontEndResults - maybe they have been discarded. Or maybe someone else will pipe in.

thanx again

steve

[Steve Upton]

In case anyone is interested, I came up with this simple code to generate the call graph:

import org.nlogo.headless.HeadlessWorkspace

import org.nlogo.nvm._

case class GraphEdge(from : String, to: String, isolate : Boolean) {

override def toString = s"$from -> $to"

def toCSV = s"$from,$to,$isolate"

}

object GraphEdge {

def apply(from: String, to: String) : GraphEdge = {

val isolate = to match {

case "" => true

case _ => false

}

apply(from, to, isolate)

}

}

 object GenerateCallGraph {

  def extractCallGraph(procedures : Procedure.ProceduresMap) : Set[GraphEdge] = {

val procNames = procedures.keySet

println("printing the procNames")

procNames foreach println

println("*******printing the calls")

val edges = procedures.map{case(parent,child) => {

println(s"working procedure: $parent")

val (gcall,gargs) = child.code.partition(_.toString.contains("_call"))

val topEdges = gcall.map(c => GraphEdge(from=parent,to=c.displayName))

// greports : Array[Array[org.nlogo.nvm.Reporter]] 

val greports = gargs.map(_.args).filter(a => a.exists(_.toString.contains("_callreport")))

val argEdges = greports.map(r => {

val edge = GraphEdge(from=parent, to=r(0).displayName)

println(s"making edge in proc: $parent => $edge")

edge

})

argEdges match {

case e if e.isEmpty => topEdges.toList

case _ => topEdges.toList ::: argEdges.toList

}

}}  

val edgeSet = edges.toList.flatten.toSet

edgeSet foreach println

edgeSet

  }

def run(modelFileName : String, callGraphOutFileName : String) : Unit = {

val workspace = HeadlessWorkspace.newInstance

workspace.open(modelFileName)

val procs = workspace.procedures

val callGraph = extractCallGraph(procs)

val output = callGraph.map(_.toCSV).toList

Utils.writeFile(callGraphOutFileName, output)

workspace.dispose()

[Seth Tisue]

This looks it will produce correct output in many cases, but I'm not convinced it's correct in general.

Does it work on a case like this, and on even more deeply nested cases?

to-report foo [x] report x end

to-report bar [x] report x end

to baz __ignore foo bar 0 end

I'm looking at your logic and it isn't clear to me that all possible nestings (including recursive nestings) of _call and _callreport are handled properly, hence the test case. I don't see anywhere in your code that recursively processes entire argument trees, and not just the first level of the tree.

Also a nitpick that won't usually affect correctness: you shouldn't need to resort to looking for "_call" and "_callreport' as strings — _call and _callreport are classes, you should be able to detect instances of them via pattern matching (or `isInstanceOf`).

Seth

[Steve Upton]

Seth,

Thank you for following up! Learning something new about Netlogo and Scala everyday!  

You are correct - I am only looking at the first level of the tree. I had assumed I only needed to go one level to capture all the procedures that called a set of procedures within its code block, and that all procedures would be included in the workspace.procedures map (and interested in user-defined procedures mainly). For your test case (I made a slight mod to make an edge from a procedure that didn't make any calls to a "Nothing" node), I get:

FOO -> Nothing

BAR -> Nothing

BAZ -> FOO

so clearly missed the call to Bar. 

BTW, what is __ignore? I don't see it in the Netlogo dictionary.

Also, thanks for the nitpick - always good to clean up code.

steve

[Seth Tisue]

On Thu, Jun 25, 2020 at 7:46 AM Steve Upton <upto...@gmail.com> wrote:
> BTW, what is __ignore? I don't see it in the Netlogo dictionary.

see https://github.com/NetLogo/NetLogo/wiki/Unofficial-features

Seth

[Steve Upton]

Thanks again, Seth!!

steve