How to parallelize text/template long running functions?

[Michael Brown]

I am designing a system that will heavily use text/template processing and I've run into one issue that is going to be a show stopper for me if I can't figure out a way around it.

Execute() on a template will run all of the functions in the template serially. 

For example, when you run the code below, you can see it output "slept" once every second until it completes after 3 seconds. In this example, the sleep is simulating an RPC call to another process that may take some considerable time (few tenths of a second), but there will be a large number of these calls that could all theoretically run in parallel (ie. there are no data dependencies between them). I'd really like to know a way that I could have the templating engine run all of the functions at once and collect the output, ie. in the example below, the entire program should run in 1 second.

package main

import (

    "text/template"

    "os"

    "time"

)

var funcMap = template.FuncMap {

    "sleep": func() string { time.Sleep(1 * time.Second); return "slept" },

}

func main() {

    tmpl, _ := template.New("test").Funcs(funcMap).Parse("{{sleep}} {{sleep}} {{sleep}}")

    tmpl.Execute(os.Stdout, nil)

}

[Egon]

The best idea I can think of (without digging and modifying text/template) is to use special tokens and replace them afterwards...

Of course this approach has limitations in what it can do.

// note: code untested and incomplete

type Token string
type Queries struct {

    pending sync.WaitGroup

    mu sync.Mutex

    responses map[Token]string

}

func (q *Queries) createToken() Token {
    return unique token
}

func (q *Queries) Do(fn func() string) Token {

    token := q.createToken()
    q.pending.Add(1)
    go func(){

        defer q.pending.Done()

        result := fn()

        q.mu.Lock()

        q.responses[token] = result

        q.mu.Unlock()

    }()

    return token
}

func (q *Queries) Wait(){ q.pending.Wait() }

func (q *Queries) Patch(data []byte) []byte {

    // replace tokens with responess

}

func main() {

   q := NewQueries()

    var funcMap = template.FuncMap {

        "sleep": func() Token { return q.Do(func() string { time.Sleep(1 * time.Second); return "slept" }) },

    }

    tmpl, _ := template.New("test").Funcs(funcMap).Parse("{{sleep}} {{sleep}} {{sleep}}")

    var buf bytes.Buffer

    tmpl.Execute(&buf, nil)

    q.Wait()

    os.Stdout.Write(q.Patch(buf.Bytes))

}

The other approach would be to do multiple passes:

1. execute template
2. collect funcs that haven't been run yet

2.1. no funcs left --> output

3. execute these funcs, cache the func values
4. goto step 1 using the cache

[Tom Limoncelli]

There may be a solution to the problem you have, but let me recommend you take a different approach and pre-compute RPC calls before calling the template.  It will make your code easier to test and debug.  Personally, the functions my templates call tend to be self-contained or zero dependencies: formatting a string, selecting items in a list, etc. It works better for me... YMMV.

Changing your code this way may require duplicating a loop outside the template and inside the template. For example, if the RPC call is made in a template loop, you'll then need a related loop to precompute the answers prior to the template.  A change to the template will require a change to the code that precomputes values.  That's a maintenance burden.  However there may be situations where you can move the RPC-calling functions to a subtemplate and isolate the precompute code to be near there, to make maintenance easier.

Best,

Tom

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Email: t...@whatexit.org    Work: tlimo...@StackOverflow.com
Blog:  http://EverythingSysadmin.com

[Michael Brown]

This is basically the answer that everybody gives, however this is just not an option. (Which also makes this a very frustrating question to ask, as many people insist that you are just simply wrong for asking it without any understanding of our requirements.)

We have data sets that we cannot pull into memory, we have to do an RPC to get the data. The template is the only entity that knows exactly which data from the data set is needed. Pre-computing the data would completely defeat the purpose of using templates at all.

For reference, I'm running on an embedded system with a limited amount of RAM and need to pull values from a database on flash storage. Which values do I need? The template knows. I don't have enough RAM to pull in all of the values. Additionally, some of the values come from real-time data collected over i2c, which is very slow and expensive operation. I can't just go pull all of the values.

--

Michael

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Brown]

The best thing I can think of is to modify text/template to add futures support, and then do multi-pass rendering. The place to add this looks relatively simple, however the implementation looks complicated (and I just wrote my first program in go a couple weeks ago...)

The problem that I see with the solution below is that text/template execution tries to instantiate the values of each function serially and essentially waits for completion on each, serially. I've spent the last couple hours examining the text/template implementation.

--

Michael

[Egon Elbre]

Both of my described approaches run the funcs serially however it does not wait for the response and later patxhes the results.

Can you describe the whole thing you are building? Piecing the requirements and purpose together from comments is difficult.

I.e. how much memory, how big is request latency, who gets the output, where do the templates come from...

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/j0WwjQE11rw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Michael Brown]

This is almost certainly going to be a case where you have the correct solution and I don't have the go experience to properly understand it. I wasn't quite understanding how your code was "patching" the results and how the template package knows to wait for it. Let me describe my problem.

I have a REST interface running on an embedded system. The REST api is mandated by another entity and I have zero control over the output, I have to produce specific conformant output. The data that I need to build the interface is on the system in a variety of other processes, and I can get that data via a couple of different RPC mechanisms, however sometimes these RPC mechanisms can be slow. (I'm in progress building go bindings for them).

The current code which creates the REST interface is a huge morass of C code that generates the JSON output, so the exact structure of the JSON output is hardcoded in the code.

I have an opportunity here, and that is that it appears to me that, with a little work, I can completely templatize the rest output. That is, I can have zero page-specific code and render almost all of the output by providing a small number of generic data access functions. The issue I ran into is the serial nature of the substitution kills the performance in my prototype.

Yes, I can make a specific go function per page that gathers all the data I need and provides it to the template. But that would mean that I'd need to maintain both the templates and the data access functions. This is still an improvement on the old way of doing things, but I was hoping to jump straight to a fully templatized system, which according to my initial analysis, would be considerably less code and would not really be abusing the template system (most of the function calls are very straightforward and there is no heavy processing of the output).

--

Michael

To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

[Egon]

On Wednesday, 31 May 2017 20:34:56 UTC+3, Michael Brown wrote:

This is almost certainly going to be a case where you have the correct solution and I don't have the go experience to properly understand it. I wasn't quite understanding how your code was "patching" the results and how the template package knows to wait for it.

Let's say you have a template

{{ sleep }} xxx {{ sleep }}

Once you render with that approach you get something like this...

<<token1>> xxx <<token2>>

So instead of returning the actual value you replace them with a token. And spawn a go routine to fetch the answers.

You wait for the results or just some, doesn't matter.

Once you get a results, you replace it in the buffer...

token1 -> "alpha"

token2 -> "beta"

"alpha" xxx "beta"

Or whatever the result is.

However this approach is pretty limited when you want to range over the results.

Doing two passes over the template, might work better, but needs somewhat better logic to handle lookup; but that could work for handling multiple results...

{{ sleep }}

In the first pass you start the goroutines and each output goes into an array of channels

var rpcs = []func() interface{}

var funcMap = template.FuncMap { "sleep": func() { rpcs = append(rpcs, sleep)  },    }

// now you schedule goroutines in whatever way you need for the rpcs and write the results to a channel

var results = []chan interface{}

the second pass would do it as

current := 0

var funcMap = template.FuncMap { "sleep": func() interface{} {

    result := <-results[current]

    current++

    return result

// ...

Without knowing the actual template, it is hard to recommend something better simpler or easier.

[Michael Brown]

Ah! I think I have the kernel of a usable idea here. Thanks.

How does this sound:  template 1 outputs template 2. Input to template 1 kicks off the goroutines and places {{ get_results token_xyz }} as the output, but also has an output channel that we can wait on for all of the answers Then, template 2 is executed with a function that pulls the output from the channel. Sort of hard to explain, but seems on the face to be workable. I'll post results after I prototype it.

--

Michael

[Michael Brown]

And after I re-read your answer, I realize that I probably just restated very badly what you wrote, Egon. Thanks! I'll have some time this evening to hack on it.

[robfig]

We do this exact thing except using closure templates
https://blog.gopheracademy.com/advent-2014/soy-programmable-templates/

[Michael Brown]

Great! If I had checked this thread when you posted this I (probably) could have saved myself 3 hours of work.

I got it working with a two-pass scheme using text/template and some channels.

Here is what I came up with:

package main

import (

    "text/template"

    "bytes"

    "context"

    "fmt"

    "github.com/google/uuid"

    "time"

)

type resultpair struct {

    key   string

    value string

}

func getFuncMap(ctx context.Context, resultsQueue chan resultpair) (*int, template.FuncMap) {

    totalJobs := 0

    var funcMap = template.FuncMap{

        "sleep": func() string {

            totalJobs = totalJobs + 1

            token := uuid.New()

            go func() {

                time.Sleep(1 * time.Second)

                resultsQueue <- resultpair{token.String(), "REAL_VALUE!"}

            }()

            return "{{ getOutput \"" + token.String() + "\" }}"

        },

    }

    return &totalJobs, funcMap

}

func getFuncMapNested(ctx context.Context, output map[string]string) template.FuncMap {

    var funcMapNested = template.FuncMap{

        "getOutput": func(input string) string { return output[input] },

    }

    return funcMapNested

}

func main() {

    initial := "{{sleep}} {{sleep}} {{sleep}}"

    resultsQueue := make(chan resultpair)

    outputQueue := make(chan map[string]string)

    // totalJobs is decieving: only ever accessed by one thread at a time, so shouldn't need locking (I think)

    totalJobs, funcMap := getFuncMap(context.TODO(), resultsQueue)

    fmt.Printf("About to execute first template: %s\n", initial)

    fmt.Printf("TOTAL JOBS: %d\n", *totalJobs)

    tmpl, _ := template.New("test").Funcs(funcMap).Parse(initial)

    var buf bytes.Buffer

    tmpl.Execute(&buf, nil)

    fmt.Printf("Got translated template: %s\n", buf.String())

    fmt.Printf("TOTAL JOBS: %d\n", *totalJobs)

    go func(totalJobs *int) {

        var results map[string]string

        results = make(map[string]string)

        for i := 0; i < *totalJobs; i++ {

            res := <-resultsQueue

            results[res.key] = res.value

        }

        outputQueue <- results

        close(outputQueue)

    }(totalJobs)

    output := <-outputQueue

    close(resultsQueue)

    fmt.Printf("Output of the goroutine: %s\n", output)

    funcMapNested := getFuncMapNested(context.TODO(), output)

    tmpl2, _ := template.New("nested").Funcs(funcMapNested).Parse(buf.String())

    var buf2 bytes.Buffer

    tmpl2.Execute(&buf2, nil)

    fmt.Printf("results: %s\n", buf2.String())

}

OUTPUT:

$ time go run ./commands/try.go

About to execute first template: {{sleep}} {{sleep}} {{sleep}}

TOTAL JOBS: 0

Got translated template: {{ getOutput "bc7dcfa0-89d9-45e9-bd40-eb2db6f51db0" }} {{ getOutput "f2539f15-378b-408d-8c6e-d3822e985a6b" }} {{ getOutput "56c9d239-d08d-43e8-80de-dd97ef157b6a" }}

TOTAL JOBS: 3

Output of the goroutine: map[bc7dcfa0-89d9-45e9-bd40-eb2db6f51db0:REAL_VALUE! f2539f15-378b-408d-8c6e-d3822e985a6b:REAL_VALUE! 56c9d239-d08d-43e8-80de-dd97ef157b6a:REAL_VALUE!]

results: REAL_VALUE! REAL_VALUE! REAL_VALUE!

real    0m1.319s

user    0m0.278s

sys     0m0.098s

[roger peppe]

Here's a proof-of-concept of a somewhat general mechanism
for making template functions concurrent. It involves some
fun reflection code (first time I've used FuncOf, MakeFunc,
StructOf and MapOf for real).

https://play.golang.org/p/7qXx5pCh9N

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an

[roger peppe]

On 2 June 2017 at 10:35, roger peppe <rogp...@gmail.com> wrote:
> Here's a proof-of-concept of a somewhat general mechanism
> for making template functions concurrent. It involves some
> fun reflection code (first time I've used FuncOf, MakeFunc,
> StructOf and MapOf for real).
>
> https://play.golang.org/p/7qXx5pCh9N

Note that it won't work correctly if you pass any functions whose results
are used in template expressions to Execute.

[Michael Brown]

Your solution is certainly nice. It has the same limitation that mine does. I'll take a look, I like how you don't have to modify the funcmap in yours.

--

Michael

[Simon Ritchie]

I was puzzled by something you said earlier:

> The template is the only entity that knows exactly which data from the data set is needed.

You've just given a bit more information, but I still don't quite understand.

You have some C code that you don't control and it's producing some information.  You have some Go which is executing a template. You say that the template has access to some data that the rest of the Go application doesn't have.

A template is really just a function expressed in a rather opaque language.  The Go application invokes this function by calling the execute method, passing a destination and some data.  The function glues together the text in the template and the data, creates a piece of text and sends it to the destination.   While it is doing that, it can call other functions, so if it has a piece of data that the rest of the application doesn't have, it can call a function to hand it back.

Fundamentally, the template is part of your Go application, it's just expressed in a cranky language that was only ever designed to describe how to render some text.  So I don't understand how your template can be holding a piece of data that the rest of the application can't get at.

That cranky language is the reason why lots of people think what you are doing is a bad idea.  The conventional wisdom is that a template should be very simple.  You should do as much processing as possible in Go before you call the template, and pass the results to the template as data.  Your application will be much easier to maintain if you do that.

Put simply, if your template code can get hold of some data, I don't understand why your Go code can't get hold of the same data, given that the template code and the Go are part of the same application.

If this is an HTTP client/server system, the template produces an HTTP response, perhaps some HTML, and sends to a web browser to be rendered.  The web browser could be holding some data that the application doesn't have, but that's not the template, it's the client, and there are lots of ways to get data from a client to a server.

Can you explain the problem you have in a bit more detail?