global and local variables

[Ben Ward]

Hi,

I have a pretty big function - call it the outer function, that defines three inner functions and then defines a load of variables, then calls the inner functions previously defined. The three inner functions are supposed to have no arguments between the () and are supposed to change those variables defined by the outer function. In R I think this would be done by the <<- assignment. 

ConceptualExample (It is R):

OuterFunc(){
            Infunc1(){
                     #Dostuff to A
                     }
            Infunc2(){

                     #Dostuff to A
                     }
            Infunc3(){
                     #Dostuff to A
                     }
            # Define A
            # if statements deciding which of the three to call to modify A
}            

The reason these three inner functions are called to operate on the outer functions variables is because they are called often (dry principle). If I want these inner functions to be able to modify the variables that are one level above them in scope. How should I best  go about this? Should I define the variables as global? Or would metaprogramming be a better way of doing this in julia?

Best,

Ben.

[John Myles White]

Honestly, I would encourage you not to do things this way. Scope exists for a reason. The "<<-" only exists to violate a good rule.

Are you not able to pass the variables to the functions?

 -- John

[Tomas Lycken]

I haven't ever worked in R, so I don't know what the motivations between this (anti-)pattern might be, but in Julia there are quite clean ways to achieve the same thing, since the variable will be passed "by reference". Thus, the inner functions could be defined to take one parameter, change stuff on that variable, and the changes would affect the variable outside of the inner function scope as well.

Say you have the following type defined:

type A

    x::Real

    y::Real
end

and then you define a function like this

function dosomething!(a::A)

    a.x += 1

end

Note the exclamation mark at the end of the function name; it shows that the function will mutate its argument (it is purely conventional, so you could mutate a anyway, but the convention is used widely in Julia). Now, you could do something like this:

b = A(2,3) # b.x = 2; b.y = 3

dosomething!(b) # after this call, b.x == 3

So there's really no need to access the variables in the outer scope - just pass them to your inner functions and change the input variables as you need. If you need a function that makes changes to its input arguments internally, but doesn't change them in the calling scope, you can use copy() to create a local variable with the same content, and change that instead.

// Tomas

[Stefan Karpinski]

I'm not entirely sure if I follow your main use case, but some things with inner functions will just work:

julia> function outer(n)

         inner1(k) = n += 1k

         inner2(k) = n += 2k

         inner1, inner2

       end

# methods for generic function outer

outer(n) at none:2

julia> f,g = outer(10)

(inner1,inner2)

julia> f(1)

11

julia> f(0)

11

julia> g(0)

11

julia> g(1)

13

julia> f(0)

13

julia> g(2)

17

julia> f(0)

17

[Stefan Karpinski]

Another example:

julia> function outer(k)

         n = k

         get() = n

         set(k) = n = k

         get, set

       end

# methods for generic function outer

outer(k) at none:2

julia> g,s = outer(0)

julia> g,s = outer(0)

(get,set)

julia> g()

0

julia> s(1)

1

julia> g()

1

julia> s(5)

5

julia> g()

5

[Ben Ward]

I've tried to pass values to the function but it's not quite doing what I want, I'll provide another abstract example that is exhibiting the behaviour :

j = 1

function example(j)

j += 1

end

What I want if for the outer j to be modified and increased by 1, currently this does not happen, the functions prints 2 to the screen but if I call j again it is still one.

I have gone about this another way which appears to mimic these R functions I'm trying to rewrite in Julia, rather than have a function and a load of inner functions to do the task, I have the outer function, and have replaced the inner functions with quote blocks:

 function outer()

         increment = quote

         j += 1

         end

         j = 1

         eval(increment)

       end

This is a small example so obviously I would normally just to j+=1 in the function outer with normal code. But I think this should work, I define a function, which at the start I define the quote blocks instead of functions, then eval these blocks later one once I have my variables defined, then the quote blocks when evaluated modify the variables, that way I don't have to have functions in functions confusing scope. 

[John Myles White]

The example you're using is quite hard to discuss because you've chosen an example in which variables are passed by value and not by reference, so that it's very difficult to achieve the effects you're looking for. If j were a matrix, for example, it would be easy to do:

j = [1]

function example!(j)

  j[1] += 1

end

example!(j)

example!(j)

Since your example uses integers, there's no reason to be trying to edit anything outside of scope. You can just return the value:

j = 1

function example(j)

  j + 1

end

j = example(j)

j = example(j)

 -- John

[Ben Ward]

So if arrays are passed to a function they are passed by reference but a variable such as a = 7 is passed by value?

In my real code (which is pretty long - a re-write of an R function for Julia). The function retrieves several of these values like a=7 it's supposed to change, and it also receives arrays - elements of which it is also supposed to change. I suppose an alternative is just writing out the code in full - but it's not as neat or compact.

Best,

Ben.

On Thursday, June 20, 2013 7:00:45 PM UTC+1, Ben Ward wrote:

[John Myles White]

Yes, all types of variables -- except for a small subset including simple primitives like numbers and strings -- are passed by reference.

You can mutate all of variables passed by reference and return the value of all of the variables that are passed by value. See the example below for what such a function might look like.

function foo!(a::Integer, b::Float64, c::Vector)

  c[1] = c[1] * pi

  return a + 1, b + exp(2)

end

 

 -- John

[Ben Ward]

I've managed to use code blocks to solve the task I wanted to do so I'm asking now out of interest and education.

So If you return a and b, as in the example, do you need to use = to assign the retuned values back to the original and and b? I seem to remember somewhere reading multiple values can be returned but they come out as a tuple. But then I suppose you could put a, b = in front of the function call and that would work.

---Ben

[John Myles White]

Yeah, just as you guessed, to use the hypothetical foo!() you would assign to the values you want to update,

a, b = foo!(a, b, c)

where we make the assumption that c is being updated appropriately during the course of running foo!().

 -- John

[Ben Ward]

Ah, just found out my use of code blocks does not work, I'm going to give in an put the full function here for anyone interested or who can see what I'm doing wrong. What it is is an implementation in Julia of a function from an R package. There are three code blocks at the top defined called AddInternal, AddTerminal, and GoDown. The idea is that these manipulate the variables "edge", "currentNode", "index", "j", "k", "l", "tipLabel", "tip", and "nodeLabel". All of those variables are then defined and set up by the function, and the function then enters a loop that is supposed to call those blocks, dependant on some if statements and choose which block to use to manip the variables for that iteration. This works when I just paste this in my interpreter as code, but when I wrap it up into a function called CladoBuild and call CladoBuild - it does not work, it seems the quote blocks are not accessing the variables.

function CladoBuild(tp)

tp = "$tp;"

AddInternal = quote

edge[j, 1] = currentNode

node += 1

edge[j, 2] = currentNode = node

index[node] = j

j += 1

end

AddTerminal = quote

edge[j, 1] = currentNode

edge[j, 2] = tip

index[tip] = j

tipLabel[tip] = tpc[k]

k += 1

tip += 1

j += 1

end

GoDown = quote

l = index[currentNode]

nodeLabel[currentNode - nbTip] = tpc[k]

k += 1

currentNode = edge[l, 1]

end

tsp = split(tp, "")

tp = replace(tp, ")", ")NA")

tp = replace(tp, "\s", "")

tpc = split(tp, r"[\\(\\),;]")

tpc = tpc[!bool([i == "" for i in tpc])]

skeleton = tsp[bool([i == "(" || i == ")" || i == "," || i == ";" for i in tsp])]

nsk = length(skeleton) # Length of the nexus skeleton.

nbNode = length(skeleton[bool([i == ")" for i in skeleton])]) # Number of nodes.

nbTip = length(skeleton[bool([i == "," for i in skeleton])]) + 1 # Number of tips.

nbEdge = nbNode + nbTip # Number of edges for tree is number of tips and nodes.

nodeLabel = Array(ASCIIString, nbNode)

tipLabel = Array(ASCIIString, nbTip)

edge = Array(Int, nbEdge,2)

currentNode = node = nbTip + 1

edge[nbEdge, 1] = 0

edge[nbEdge, 2] = node

index = Array(Int, nbEdge+1)

index[node] = nbEdge

j = k = tip = 1

for i in 2:nsk

if skeleton[i] == "("

eval(AddInternal)

end

if skeleton[i] == "," && skeleton[i-1] != ")"

eval(AddTerminal)

end

if skeleton[i] == ")"

if skeleton[i - 1] == ","

eval(AddTerminal)

eval(GoDown)

end

if skeleton[i - 1] == ")"

eval(GoDown)

end

end

end

edge = edge[1:nbEdge-1, 1:2]

nodeLabel = [replace(i, r"^NA", "") for i in nodeLabel]

phyloobject = Clado(edge, tipLabel, nbNode, nodeLabel)

return phyloobject

end

On Thursday, June 20, 2013 7:00:45 PM UTC+1, Ben Ward wrote:

[Stefan Karpinski]

On Thu, Jun 20, 2013 at 8:40 PM, Ben Ward <axolotl...@gmail.com> wrote:

I've tried to pass values to the function but it's not quite doing what I want, I'll provide another abstract example that is exhibiting the behaviour :

j = 1

function example(j)

j += 1

end

What I want if for the outer j to be modified and increased by 1, currently this does not happen, the functions prints 2 to the screen but if I call j again it is still one.

On Thu, Jun 20, 2013 at 10:03 PM, John Myles White <johnmyl...@gmail.com> wrote:

The example you're using is quite hard to discuss because you've chosen an example in which variables are passed by value and not by reference, so that it's very difficult to achieve the effects you're looking for.

I feel like I've been having this conversation a lot [1], so this seems to be a confusing issue for a lot of people. Leah wrote a good blog post about how values and variables work that's resulted from one of these conversations [2]. This is *not* actually a case of j being passed by value – Julia only has reference semantics. This is a scope issue: you can change the value that the j inside of the example function is bound to, but that j is local to the function because function arguments are local. The j outside of example is a different variable that is shadowed by the function argument. Thus, Ben's will work if you don't make j an argument of example:

julia> begin

         j = 1

         example() = (j += 1)

       end

# methods for generic function example

example() at none:3

julia> ex

example     exp         exp2        expanduser  expm1       export      exprnd

exit        exp10       expand      expm        exponent    expr        

julia> example()

2

julia> example()

3

Note that this has to be done in begin block, not at top-level, since otherwise j will be deemed local to example, rather than global, since it is assigned to.

[1] https://groups.google.com/forum/#!topic/julia-users/qJa2EyUXJfo[2] http://blog.leahhanson.us/well-that-was-embarrassing.html

[Stefan Karpinski]

On Fri, Jun 21, 2013 at 10:33 AM, Ben Ward <axolotl...@gmail.com> wrote:

So if arrays are passed to a function they are passed by reference but a variable such as a = 7 is passed by value?

On Fri, Jun 21, 2013 at 11:28 AM, John Myles White <johnmyl...@gmail.com> wrote:

Yes, all types of variables -- except for a small subset including simple primitives like numbers and strings -- are passed by reference.

This is not the case – everything in Julia behaves as if it is passed by reference. Integers are immutable, however, so the compiler is free to *implement* it as pass by value, because there's no way to tell the difference between reference and value with immutable things.

[John Myles White]

I stand corrected. Thanks for fixing my mistake, Stefan.

Is the begin block trick as efficient as passing in all of the variables needed?

 -- John

[Stefan Karpinski]

On Thu, Jun 20, 2013 at 8:40 PM, Ben Ward <axolotl...@gmail.com> wrote:

 

I have gone about this another way which appears to mimic these R functions I'm trying to rewrite in Julia, rather than have a function and a load of inner functions to do the task, I have the outer function, and have replaced the inner functions with quote blocks:

 function outer()

         increment = quote

         j += 1

         end

         j = 1

         eval(increment)

       end

This is a small example so obviously I would normally just to j+=1 in the function outer with normal code. But I think this should work, I define a function, which at the start I define the quote blocks instead of functions, then eval these blocks later one once I have my variables defined, then the quote blocks when evaluated modify the variables, that way I don't have to have functions in functions confusing scope.

I know that this is just an exploration, but I feel like I should point out that one should not use eval like this, and it will not work as intended. In Julia, eval always evaluates expressions in the top-level global scope of the current module – because one should only ever use it there. In this case the eval will attempt to increment a global variable called j, rather than the local j inside of outer. Calling eval inside of a function or macro is almost always the wrong thing to do. Pretty much the main legitimate use case for eval is stuff like this:

for fn in _numeric_conversion_func_names

    @eval $fn(z::Complex) = complex($fn(real(z)),$fn(imag(z)))

end

[Ben Ward]

Ok so to get my head around this - something like the following has the effect I desire:

function AddInternal(edge, currentNode, node, index, j)

edge[j, 1] = currentNode

node += 1

edge[j, 2] = currentNode = node

index[node] = j

j += 1

return currentNode, node, j

end

currentNode, node, j = AddInternal(edge, currentNode, node, index, j)

To edit the values, but also I can do this by not providing arguments to the function:

function AddInternal()

edge[j, 1] = currentNode

node += 1

edge[j, 2] = currentNode = node

index[node] = j

j += 1

end

AddInternal()

Would have the same effect?

--Ben.

On Thursday, June 20, 2013 7:00:45 PM UTC+1, Ben Ward wrote:

[Edward Garson]

Hi Ben

I'm pretty sure you should strongly prefer the first version, i.e. the
referentially transparent[1] one, which is just a fancy way of saying
"give functions all the information they need to produce a return
value". To be sure, this implies that you should minimize what you
reference outside of a given function (ideally nothing).

This style is beneficial for several reasons. For one, it ensures that
your functions are idempotent, which means they produce the same
output for a given input, every time. This makes testing much easier,
and as the code grows, it makes it much easier to reason about the
behaviour of your program, and furthermore makes it much easier to
find defects. On the other hand, if you modify state outside of your
functions, and there are many variables and functions and branches, it
becomes much harder to figure out where things went wrong.

But perhaps worse still is that you affect function composition: if
the results of some functions are inputs to others, you can mix and
match them more easily to produce more terse, readable code. Taking
this further, you should prefer small functions with few arguments as
these are more easily composed and re-used. I'm also pretty sure that
referentially transparent functions are more efficient in terms of
memory access patterns, but I'm not yet sufficiently au fait with
Julia's internals to know this for sure.

I trust that the friendly Julia community will let me know where I may
have put my foot in my mouth ^_^

Edward

[1] - http://en.wikipedia.org/wiki/Referential_transparency_(computer_science)