HTTP Chunked streaming to WebSocket

[Tharshan Muthulingam]

I want to listen to multiple HTTP streams that are transfer encoded responses, then fetch the messages from them line by line and then push the messages to one channel. I want to then read from the channel and push through a websocket later. 

    func subscribe(ws chan<- string, group string) (scanner *bufio.Scanner, err error){
        res, _ := req(STREAM_URL, channelTemplate(group))
        reader := bufio.NewScanner(res.Body)
        return reader, reader.Err()
    }
   
    func main() {
        ws := make(chan string)
        request, _ := http.NewRequest("GET", URL, nil)
        request.Header.Add("Content-Type", "application/json")
        client := &http.Client{}
        resp, _ := client.Do(request)
        ac := ACResponse{}
        json.NewDecoder(resp.Body).Decode(&ac)
        resp.Body.Close()
        var scanners = make([]*bufio.Scanner, 0)
        for _, group := range ac.Groups {
            fmt.Println("Started worker for", group)
            //listen to all stream URLs
            scanner, err := subscribe(ws, group)
            if err != nil {
                panic(err)
            }
            // keep track of Scanner to read later
            scanners = append(scanners, scanner)
        }
        for {
            select {
            case msg := <-ws:
                fmt.Println("[events] ", msg)
            default:
                randScanner := rand.Intn(len(ac.Groups)-1)
                fmt.Println("Reading from", randScanner)
                reader := scanners[randScanner]
                reader.Scan()
                if err := reader.Err(); err != nil {
                    panic(err)
                }
                text := reader.Text()
                ws <- text
            }
        }
    }

The program is blocking at `reader.Scan()`. The output is `Reading from 1` and nothing else. I looked at wireshark, and the messages are coming through.

How can I design this problem better with Go?

[Dave Cheney]

> resp, _ := client.Do(request)

You need to handle the error here, and also check the response code so that you are not streaming the body of a non 200 request.

> json.NewDecoder(resp.Body).Decode(&ac)

You need to handle the error here, because otherwise ac will remain its zero value, executing zero iteration of your subscribe loop.