Manifold questions: Creating sinks/sources

[Atamert Ölçgen]

Hi,

AFAIK the only way to create (just) a source (or sink) is:

(def my-source (s/source-only (s/stream ...)))

This results in creating a stream and then wrapping it with a SourceProxy. We don't keep a reference to the stream and the SourceProxy doesn't allow taking.

But if I'm not missing something since SourceProxy keeps a reference of the original stream, there is a sink nobody is using there.

If I create a sink separately and connect my-source to it, now my-source would be connected to two sinks.

My questions are:

1. Is there another method for creating only sinks or sources?

2. Should the extra/unused source/sink I mentioned above cause concern?

PS: I'm not talking about a scenario where we're creating thousands streams and connecting them all like there's no tomorrow.

--

Kind Regards,
Atamert Ölçgen

◻◼◻

◻◻◼

◼◼◼

www.muhuk.com

[Zach Tellman]

Hi Atamert,

For future reference, posting these questions to https://groups.google.com/forum/#!forum/aleph-lib will ensure I'll see them sooner.  

The `source-only` method is just a way to make sure that the chance for confusion is minimized, it doesn't prevent the underlying object from being used normally.  As for your scenario, I'm not sure I completely understand.  In a standard stream, the flow of data is:

[ sink -> source ]

So if you `put!` something into the sink, you can `take!` it from the source.  If you use `source-only` and connect that to some other sink, then the flow of data will be

[ sink -> source ] -> sink

So the source is only connected to one sink, not two.  Maybe I'm misunderstanding you, though.

Zach

[Atamert Ölçgen]

On Thu, Aug 27, 2015 at 10:51 PM, Zach Tellman <ztel...@gmail.com> wrote:

Hi Atamert,

Hi Zach,

 

For future reference, posting these questions to https://groups.google.com/forum/#!forum/aleph-lib will ensure I'll see them sooner.  

I've subscribed. Thanks.

 

The `source-only` method is just a way to make sure that the chance for confusion is minimized, it doesn't prevent the underlying object from being used normally.  As for your scenario, I'm not sure I completely understand.  In a standard stream, the flow of data is:

[ sink -> source ]

So if you `put!` something into the sink, you can `take!` it from the source.  If you use `source-only` and connect that to some other sink, then the flow of data will be

[ sink -> source ] -> sink

So the source is only connected to one sink, not two.  Maybe I'm misunderstanding you, though.

I have initially written it right. I was looking through the docs and the source at the same time and I got confused. Then I replaced source with sink and vice versa. So if you are misunderstanding, you are understanding. :)

Let me try again. If we have:

(def my-sink (s/sink-only (s/stream ...))) 

(def my-source (s/source-only (s/stream ...)))

(s/connect my-source my-sink)   ;; BTW this got me confused, I thought flow was like my-source -> my-sink

After I have written the comment above, and playing with this code in the REPL, I have realized the flow IS from source to sink.

I think I am un-confused now, and it all makes sense. I have a producer side and a consumer side. Instead of repeating a function call, I want to connect these two components using a stream. I thought I'd need half a stream here and half a stream there. But now I realize I need two whole streams, connect them, put! on one side, take! on the other. I hope it makes sense. Sorry for the noise.

 

Zach

On Thursday, August 27, 2015 at 4:13:35 AM UTC-7, Atamert Ölçgen wrote:

Hi,

AFAIK the only way to create (just) a source (or sink) is:

(def my-source (s/source-only (s/stream ...)))

This results in creating a stream and then wrapping it with a SourceProxy. We don't keep a reference to the stream and the SourceProxy doesn't allow taking.

But if I'm not missing something since SourceProxy keeps a reference of the original stream, there is a sink nobody is using there.

If I create a sink separately and connect my-source to it, now my-source would be connected to two sinks.

My questions are:

1. Is there another method for creating only sinks or sources?

2. Should the extra/unused source/sink I mentioned above cause concern?

PS: I'm not talking about a scenario where we're creating thousands streams and connecting them all like there's no tomorrow.

--

Kind Regards,
Atamert Ölçgen

◻◼◻

◻◻◼

◼◼◼

www.muhuk.com

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jason Zwolak]

I don't think you need two streams at all. Unless there's more code in between the two streams that you're not showing.

Would the following work:

(def -private-stream- (s/stream ...))

(def my-sink (s/sink-only -private-stream-)) 

(def my-source (s/source-only -private-stream-))

Or if you really want `-private-stream-` to be out of scope, then a fn to create the source and sink could be used. Something like

(defn create-sink-and-source []

  (let [private-stream (s/stream ...)]

    {:sink (s/sink-only private-stream)

     :source (s/source-only private-stream}))

(def sink-and-source (create-sink-and-source))

(def my-sink (:sink sink-and-source))

(def my-source (:source sink-and-source))

Since a stream is both a sink and a source, you can mask it with `sink-only` and `source-only` to get the appearance of having just a sink _and_ just a source. I would like to point out that having two streams does nothing different than having one other than the overhead and complexity of moving data between two streams. Unless, as I said earlier, you're doing something in between the two streams.