how to read Stream synchronously?

[calathus]

I'm developing an object relational mapping tool like JPA in Dart.

And I need to assign an entity to the entity field of an entity class when the getter for the field is called.

But when I  look at the Stream API, like first, they return future, it seems only supporting asynchronous API. But in this context, if we don't assign a value synchronously, and just  return the getter, the main code will not get right value at the moment.

So I need to have synchronous stream API, are there such API in Dart?

Or would it be possible to change Future to synchronous evaluation.

if we write a wrapper to wait until future is completed, they will do the job, but sleep in dart:io seems not appropriate for this since it stops other asynchronous call.

Any suggestion, appreciated.

thanks,

-nc

[Lasse R.H. Nielsen]

Streams are inherently asynchronous. The synchronous analogue of a stream is an Iterable. The synchronous analogue of a Future is just an expression. 

There is no way to turn an asynchronous operation into a synchronous one. A synchronous operation expects its result to be ready immediately, and the asynchronous one doesn't have a value at that point. 

So, in general, if you use an asynchronous feature, your program will be asynchronous. You can embed synchronous operations into each asynchronous event, but the other direction isn't possible.

/L

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new

To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

--
Lasse R.H. Nielsen - l...@google.com  

'Faith without judgement merely degrades the spirit divine'
Google Denmark ApS - Frederiksborggade 20B, 1 sal - 1360 København K - Denmark - CVR nr. 28 86 69 84

[dangli...@gmail.com]

>> The synchronous analogue of a stream is an Iterable. The synchronous analogue of a Future is just an expression.

To be more correctly.
Future and Stream has no synchronous analogues. Both are asynchronous and return results as notifications for subscribers.
The subscribers of "Future" is a another "Future's" which are notified as listeners when subject completed successful.
And subscribers of "Stream" is "StreamSubscription's".
"Future" acts as producer of any expression.
"Stream" acts as producer of sets of values.
The Dart "isolate" executes in one thread and this "isolate" does not allow "sleeping" and "resuming" operations.
In this case it is not possible in Dart switch an asynchronous operation into synchronous.

[calathus]

Thank you for clarification.

The origin of this problem is in api for connecting database, they only support asynchronous call.

I don't know the original version of stream, but it looks like they would have been implemented with synchronous API.

Essentially synchronous mode is important for initialization. If we cannot guarantee something has certain properties, we cannot guarantee subsequent behavior,  or forced to use future very often where it should not have been required. consider caching, for the first time, it need to fetch from remote server, but the next time, we can just return the cached value.

If it was the main concern to avoid cluttering API by supporting both mode, it may be OK if we can convert a future to synchronous expression like toSynch(new Future());

This kind of function would not be implemented in Dart language, but at native level, it may use (hidden) thread, and keep watching the completion, that would be possible.in particular, io is ony supported in server side, the difficulty to conver it to javascript will not have to be considered.

I think a language(library) should not forbit legitimate style of implementation or force certain style of  program and making simple code bloat unncessarily.

it is important user can decide which behavior synch or asynch to use depending on the situation.

Also file stream is actually supporting both mode(readAsStringSynch()), it is inconsistent to drop synchronous mode from stream.

If there is some way to address this issue, please let me know.

Thanks,

-nc

--

For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new

To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

--
Cheers,
calathus

[Justin Fagnani]

On Fri, Nov 1, 2013 at 9:26 AM, calathus <cala...@gmail.com> wrote:

Thank you for clarification.

The origin of this problem is in api for connecting database, they only support asynchronous call.

I don't know the original version of stream, but it looks like they would have been implemented with synchronous API.

Essentially synchronous mode is important for initialization. If we cannot guarantee something has certain properties, we cannot guarantee subsequent behavior,  or forced to use future very often where it should not have been required. consider caching, for the first time, it need to fetch from remote server, but the next time, we can just return the cached value.

Then the cache should have an async API, since you can't tell whether or not the value will be immediately available. Dart programs, like JavaScript programs, are likely to be pervasively async, the best thing to do is handle the asynchronicity correctly, always return Future or Streams from functions that call Futures and Streams, always handle or chain async errors, etc.

 

If it was the main concern to avoid cluttering API by supporting both mode, it may be OK if we can convert a future to synchronous expression like toSynch(new Future());

This kind of function would not be implemented in Dart language, but at native level, it may use (hidden) thread, and keep watching the completion, that would be possible.in particular, io is ony supported in server side, the difficulty to conver it to javascript will not have to be considered.

It's pretty much impossible to implement in JavaScript without some sort of CPS transform. What is sounds like you really want is for the infamous dartbug.com/104 to be fixed, which I would like too.

[calathus]

As I mentioned, this problem arised when I try to implement db binding system. In many language, Java/C++ etc, it is quite common to have a simple class which is mapped to database table, s.t:

class A extends Persistence {

  int i;

  B _b;

  B get b => get(const Symbol("b")); 

}

let assume b is bound to another table.

Since we cannot fetch all data eagerly(otherwise it may need to fetch all data in database), we need to fetch property b lazy way(at certain point at least).

So as you suggest, if we use asynchronous API, we will need to change this to:

class A extends Persistence {

  int i;

  B _b;

  Future<B> get b => get(const Symbol("b")); 

}

the code accessing this object will become very messy. Also conceptually this kind of system is to hide underlying db, exposing something internal to getter level as Future is not good idea, also the second time, we  call this geter b, actually it is cached and no remote call is involved, it is not appropriate to use Future.

If Dart support synchronous Stream, we can avoid thin kind of problem.

For the 104, I don't know about it, but if it only run on server side, it will not have to be converted to javascript, so that feature can be supported only in DartVM.

So I thought it may not be so difficult. But if it is difficult, just providing synchronous version of stream is another way to fix this issue.

-nc

[Johannes Singler]

I don't fully understand what you want, but there is e.g.

new StreamController<T>.broadcast(sync: true)

which I use to set up some observer pattern.
Hope that helps,

Johannes

[calathus]

Thanks for the suggestion.

In my case, I'm using postgresql lib which returns a stream for passing  sql results.

But Stream API are asynchronous, for instance 'first' returns  a Future.

The goal is to implement synchronous getter(b) for mapped Dart class(A) with lazy evaluation, which means, when an object a is created it has an entity id of B, but the object b is not yet instantiated, but when getter b is called, internally it invokes a query to db and get the corresponding row data and create an instance of B from it.

But right now, getting row from Stream is asynchronous, that makes it impossible to implement this simple feature.

Ideally if we can convert the returned Stream to synchronous Stream, the required change will be minimum.

I checked the StreamController, it supports addStream, so can I add asynchronous stream to it, and get synchronous stream by StreamController.stream?

Also even we may use option sync: true, it still returns Stream.

So what is the meaning of sync?

for instance if I call first it will return Future, how can I get the result scynchronously.

if we have a following function, would it work (assuming f.then is immediately evaluated)?

Object toSync(Future<Object> f) {

  Object v = null;

  f.then((v0){v = v0;});

  return v;
}

But if there are some delay for evaluating f.then, this may not work..

Thanks,

--
Cheers,
calathus

[Greg Lowe]

The Dart socket API is asynchronous, and there are no features such as Thread.wait() in Dart (nor should there be), so a postgresql query must also be asynchronous.

At some point Dart will implement new language primitives that will make using future chains more pleasant.

For example bug #104.

In the meantime you'll need to use:

query('select * from blah').toList().then((result) {

  ...

});

[calathus]

Thanks for the comment.

I went though the current dart:io source codes, and it was clear they are only supporting asynchronous behaviour.

But I think this is essentially design bug.

If dart provide synchronous API for file related classes, the same reason applies for io as well.

This is an over simplification, it creates a lots of problem.

I think right now, the most user of dart seems interested in client/browser side application, that would have been the reason there were not much complain for such changes.

But if more people start using Dart, who had more experience for server side, they will attack the current design sooner or later. Actually such restriction will kill the potential of developing many of server side applications.

Even theoretically,  no system(except dart) would have been proposed which support only asynchronous behaviour without join/synchronous mechanism.

I would say there is no future if there is only futures.

It likes current US style economy, relying on borrowing from future, sometime we must do it right now.

Also in server side, except websocket, I wonder how much asynchronous approach will be useful in server side application. for database binding system, actually all the API should be synchronous. since the main motivation of binding is to hide underlying data base access, and all the mapped class API should be normal synchronous API.

This kind of system had been developed(actually JPA replaced infamous J2EE) in many language.

It is quite pity if Dart cannot allow to develop such system,

But this is not a language issue, it is just a mis-design of io API.

I think even if Dart provides await like feature, still it is better to have direct support of synchronous io API.

I also checked the current io source codes.

It looks like these asynchronous behaviour was introduced at dart patch class level.
So it seems possible just provide new dart patch class which support sync API.without providing new c level code to support sync API (but I'm not so sure about this)

Anyway, I'm feeling that if Dart team has no plan to support sync io, I may need to modify dart vm.

Of course that is the last resort, and I want to avoid such situation...

[Filipe Morgado]

Dart is doing fine.

Async is the correct and best way to handle any kind of IO.

It helps to keep your app responsive (if you parallelize operations or wait for other events) and will very likely reduce the size of your stack.

Every language I know is leaning towards async. Even Javascript is getting Futures.

Synchronous IO is "emulated" on top of asynchronous events. Your app is blocked, unable to process anything until an OS event notifies it to resume computation where it stopped.

It's not a coincidence that almost all event-based applications are way faster than their synchronous counter-parts.

On Monday, 4 November 2013 03:53:50 UTC, Greg Lowe wrote:

At some point Dart will implement new language primitives that will make using future chains more pleasant.

For example bug #104. 

 This is what we're looking for, not synchronicity.

I agree that async complicates APIs when caching is involved, but at some point, you will probably move the cache to its own Isolate if you want to go multi-core, which will, again, require async calls.

Well ... my 2 cts.

[Søren Gjesse]

The current execution model for Dart is based on isolates each having one thread of execution. In this model using synchronous operations for IO does not make much sense. An isolate waiting for IO will be blocked, and the application will have to be structures into using a number of isolates for handling IO. Of-loading the IO to isolates needs an async API, so async will still be used to wait for IO.

For file operations we have sync versions of most API's to support writing simple straight code scripts.

I am not sure why you cannot use futures for accessing a potentially cached value. The following sample will cache the content of the file test.dart, and get it three times - first time the cache is filled and the second and third time the cache is used.

import "dart:async";

import "dart:io";

var _cachedValue = null;

Future asyncGetValue() => new File('test.dart').readAsString();

Future getValue() {

  if (_cachedValue != null) {

    return new Future.value(_cachedValue);

  } else {

    return asyncGetValue().then((value) {

      _cachedValue = value;

      return value;

    });

  }

}

main() {

  getValue()

    .then((_) {

      print(_);

      return getValue();

    })

    .then((_) {

      print(_);

      return getValue();

    })

    .then((_) => print(_));

}

--

[calathus]

Thanks for reply.

I know it is still possible to use cache with future.

But the main problem is related to side effect base problem, and cluttering of codes.

for instance if we have access to several attributes of entity class A, like a.b, a.c and have expression involved them like  if (a.b.i == a.c.j) count++,  we will need nested future or future Future.wait

int count =0;

Future.wait([a.b, a.c])
      .then((List responses) { if((responses[0] as B).i == (responses[1] as C).j) count++;} )
      .catchError((e) => handleError(e));

this is huge increase of code side.and very awkward to use case operator, 
these are very artificial constrants intorduced by using  future.

If we need to write code in this way, it will be a big pain.

Also if we need to share count variable from other codes, actually above code won't work.

We may need to replace count to Future<int> count.

and all other code which need to access count must be invoked by wait

Future<int> count4 = future.wait([count1, count2, count3])
      .then((List responses) { 
             int i1 = responses[0] as int; 

             int i2 = responses[1] as int;

             int i3 = responses[2] as int;

             completer.completed(i1+i2+i4);

             return compreter.future;

             } ).then(lisner1).then(listen2)..

 .catchError((e) => handleError(e));connt.then((_){..})

I wonder who can keep writing this kind of codes, and if situation got more complicated, 
it will be impossible to write manageable code in this style.
Sharing state among futures will be very difficult.

Also if count4 is update, how to do? at least it won't as simple as count++.

also listener may be a function which may be invoked form outside and it may need to access count4.

then such fuction must return as as future which will be triggered by completing count value.

In this way, future constraint cascade to every where, which I see in postgresql cases.

In particular, caching base implementation, the first access may be slow, 
but the rest will be on memory, so very fast, no need to use Future.

--
Cheers,
calathus