MicroServices vs. FBP

[Paul Tarvydas]

Another question (not a statement) - is the new buzzword "microservices" mostly the same as FBP?

If they differ, how?

pt

[Raoul Duke]

Microservices isn't that new a buzzword. And it is sorta up to the
individual to decide what they are. Probably Amazon made them famous:
https://gigaom.com/2011/10/12/419-the-biggest-thing-amazon-got-right-the-platform/

Anyway, they are these days mostly RESTful which seems different
enough from FBP. But other folks have wired them up in other ways, of
course: https://duckduckgo.com/?q=Your+Server+as+a+Function&t=canonical

[Paul Tarvydas]

Thank you for taking time to reply!

Here is a quote from a 2017 book (Microservices: Flexible Software Architecture).  The *only* thing I see that's different is the use of REST.  I read the original REST PhD thesis and found "nothing new" in it.

I remain wondering how close we are....

Microservice: Preliminary Definition

The new aspect is that microservices use modules that run as distinct processes. This approach is based on the philosophy of UNIX, which can be reduced to three aspects:

• One program should fulfill only one task, but it should perform this task really well.

• Programs should be able to work together.

• A universal interface should be used. In UNIX this is provided by text streams.

The term microservice is not firmly defined. Chapter 3, “What Are Microservices,” provides a more detailed definition. However, the following criteria can serve as a first approximation:

• Microservices are a modularization concept. Their purpose is to divide large software systems into smaller parts. Thus they influence the organization and development of software systems.

• Microservices can be deployed independently of each other. Changes to one microservice can be taken into production independently of changes to other microservices.

• Microservices can be implemented in different technologies. There is no restriction on the programming language or the platform for each microservice.

• Microservices possess their own data storage: a private database or a completely separate schema in a shared database.

• Microservices can bring their own support services along, for example a search engine or a specific database. Of course, there is a common platform for all microservices—for example virtual machines.

• Microservices are self-contained processes or virtual machines, e.g., to bring the supporting services along.

• Microservices have to communicate via the network. To do so microservices use protocols that support loose coupling, such as REST or messaging.

[ern0]

Microservices and FBP have similarities.

Microservice manifesto says "thu shall use http for interfacing", but
it only means that you should use a simple, common interface. We used
RabbitMQ for transport and Haxe for packaging messages, and it was
still microservices.

Microservice manifesto says no word about reusability, as far as I
remember. It's about an application, which, says, should be split to
smallest independent parts.

As I was writing, suddenly I got enlightened, I think, this is the
most important difference:
- FBP: build app from small bricks,
- MS: explode app to small bricks.

> --
> You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



--
ern0
dataflow evangelist

[Akashicflow{dot}com]

Just wanted to add that I am currently developing an app in that mixes both.

I use unikernels.org with FBP arch, also helps greatly with security. Was able to increase speed by 825%, full consistency at 100% and reduce storage by 80% with databases.

I call it the JPMMaS = John Paul Morison Model as a service.

Let me know if anyone is interested in applying this or testing it out.

Cheers!

A.

P.S."John thank you so much for all your time and effort."

> To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.
>

--
ern0
dataflow evangelist

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Cheers!!!

A.

ph:202.643.3569

[Paul Morrison]

I for one would be very interested in knowing more about JPMMaS...  and just to mention that Morrison has two 'r's...!

And, Ernő, I think I both build and explode - in my book, I talk about going top-down (explode?) and bottom-up (build)...  plus also centre-out, and back-end frontwards - also useful directions!

BTW Does anyone have a succinct definition of REST - maybe there is less there than meets the eye!  ;-)

Cheers,

Paul M.

--
ern0
dataflow evangelist

--

Cheers!!!

A.

ph:202.643.3569

[stewart mackenzie]

I took a stab at it https://github.com/fractalide/fractalide

It's time to hang this project up now.

[Raoul Duke]

https://looselyconnected.wordpress.com/2011/03/09/the-richardson-maturity-model-of-rest-and-roy-fielding/

[Brad Christensen]

On Wednesday, March 8, 2017 at 2:23:42 PM UTC-7, stewart mackenzie wrote:

I took a stab at it https://github.com/fractalide/fractalide

It's time to hang this project up now.

Dang.

I was looking forward to stable. FWIW, though it took some time to understand how Fractalide was put together, I think its a great composition. Are you 'hanging it up' due to lack of up take, or have you found other difficulties in achieving your goals?

Well, at least I've started to learn Rust and found Nix/OS.

Cheers, Brad

[ern0]

> And, Ernő, I think I both build and explode - in my book, I talk about going
> top-down (explode?) and bottom-up (build)... plus also centre-out, and
> back-end frontwards - also useful directions!

There is a development department of a company, called Emarsys, in
Budapest, they're developing spamming robo..., err... online marketing
tool. They are organizing the "Craft Meetup", they record all
presentations and publishing on YouTube, they're great (sadly, in
Hungarian language, which I don't suppose for human consumption).

They have a legacy PHP monster app, and several microservices. Any
time they decide to rewrite or refactor a bigger part of the PHP
thing, instead of using PHP, they create a microservice (90% NodeJS,
10% else, like Ruby), so the PHP monolith is melting, and one day,
hopefully sooner than North Pole Ice Rock, will melt away finally.

In this case the direction is unquestionable: top-down. Slice (careful explode).

A year ago, I've written a pet project application, a sniffer, which
collects certain packets then stores statistical info in a database.
I've choosen Microservices architecture, just to play with it. I split
the problem to the following parts:
- Capture: it was a shell script calling NetCat, etc., which passed
all collected packets and passed metainfo with a http get call via
curl (it's like wget) to...
- Bridge: it's a small python app, which collects metainfo from one or
more Capture module, filters out duplicates, and tries pass the
filtered information to the Backend, via http get.

On a host, many capture and one Bridge application should run, the
other microservices, see below, are running on a central node, only
one instance per service.

- Backend: a PHP web service, which collects info from nodes, then
calls out some smaller services to gather all the information, and
finally stores it to a MySQL db.
- VIP check: it's a tiny http service written in Python, tells about a
packet whether it's on the VIP list or not, the PHP calls it. Okay, it
would be much more simpler to write it as a PHP function, but I wanted
to create as many microservices as many I could.
- Metainfo provider, same as VIP check, tells some info about a
packet, just to increase the number of services.

For the process management, I was using Supervisord, it's a bit
similar to Init (the traditional startup organizer in Unix systems),
it can start-stop apps, rotate logs.

It's a kind of "build" style of creating applications from
microservices. The main goal of splitting the task was that the
Capture should be run on more nodes, which are located at different
places.

So, you're right, it does not matter whether top-down or down-top,
both are a way of creating (refactoring) a certain application. But
here's the other side of the coin:

Years ago, I have written a dataflow framework (with help of a friend
of mine, C++, automation purpose, PoC/prototype stability), where,
when we created it, was only a blurry goal: create a system which will
be able to serve home/automation purposes. I've created the server, a
simple compiler and basic components (traffic, path selection,
network), my friend's wrote several protocol and utility components,
but it was only a kit, to build any apps.

So, we have several modules, and we can build apps using them, without
programming. If you want to create another app, you can build from the
same set of components, maybe there'll be a need for a component does
not exists yet, then you should also write the missing component. This
is a different approach compared to microservices, which says that
write your app as small programs, then glue them together, and if you
want to create another app, you may write small programs again, and if
you had a general-purpose service before, probably you can re-use it.

(Am I clear? I think, not really, and also I should find an English teacher.)
--
ern0
dataflow evangelist

[Ged Byrne]

Hanging up the project?

Can I ask what you mean by that?

Do you mean "hanging up" like a picture in a gallery, for everyone to see?  I hope so,

Or do you mean "hang up" like a phone conversation, bringing it to an end?  I hope not.

Regards, 

Ged

On 8 March 2017 at 21:23, stewart mackenzie <seto...@gmail.com> wrote:

I took a stab at it https://github.com/fractalide/fractalide

It's time to hang this project up now.

[Ged Byrne]

Hi Paul,

I'd like to take a different direction on this question.  I see FBP has having an important place within MicroServices because it gives us the mechanism for choreography that is not event driven.  The benefit is that while events are transient, IPs are persistent with a managed lifetime.

For my definition of Microservices I'm going to use Martin Fowler's seminal blog post: https://martinfowler.com/articles/microservices.html 

First of all I'd like to look at Microservices as the successor to SOA.  My personal view is that Microservices is SOA done right.  Fowler uses the ESB as the real distinguishing factor between SOA and Microservices:

"it's significantly different to the style we're describing here, usually due to a focus on ESBs used to integrate monolithic applications."  (Sidebar: Microservices and SOA)

What is an ESB?

"An enterprise service bus (ESB) implements a communication system between mutually interacting software applications."

https://en.wikipedia.org/wiki/Enterprise_service_bus 

In a nutshell the ESB is the plumbing that holds everything together.  These are smart pipes, doing lots of clever things.

Microservices reject that approach.  They advocate "Smart Endpoints and Dumb Pipes":

"In a monolith, the components are executing in-process and communication between them is via either method invocation or function call. The biggest issue in changing a monolith into microservices lies in changing the communication pattern. "

Within SOA there was already a move to change the communication pattern using the Messaging Enterprise Integration Patterns, with components communicating though message queues, usually with the ESB as transport.: http://www.enterpriseintegrationpatterns.com/patterns/messaging/MessagingComponentsIntro.html

I describe my mapping between FBP and EIP here: https://groups.google.com/d/msg/flow-based-programming/S1MZzFXYQFQ/EeSmP-_u5dEJ

One key pattern SOA is orchestration.  Behaviours are separated out into task and utility services and then orchestrated together.  In SOA it is considered best practice to orchestrate these activities on the ESB:

"Application or service orchestration is the process of integrating two or more applications and/or services together to automate a process, or synchronize data in real-time. Often, point-to-point integration may be used as the path of least resistance. However, point-to-point integration always leads to a complex tangle of application dependencies (often referred to as "spaghetti code") that is very hard to manage, monitor and maintain"

https://www.mulesoft.com/resources/esb/what-application-orchestration

For Microservices choreography based on an event stream has become the favoured approach:

"Services collaboration through events leads to a much more scalable architecture, with simpler and better tailored models, that will reduce the complexity of your digital assets, increase scalability, and improve your ability to collect data for analytics purposes."

https://www.thoughtworks.com/de/insights/blog/scaling-microservices-event-stream

Personally, I like working with Events but the problem is that the overall flow gets lost. It is difficult to trace the end-to-end flow in an event driven architecture.  There is also the error handling flow.  Handling errors on an event stream can be tricky, as the above article explains:

"Handling endpoint failures, which has to be carefully handcrafted in a point-to-point situation, is now pushed to the messaging system. Therefore, the messaging system itself and the choreography of services around this component need to be built with these failures in mind.

"If a service if down for a short period of time, event stores will naturally re-post missed events as soon as the subscriber service comes back up. However, if a service is down for a longer period of time, then the service must reload the history of past events and check which ones have been missed, as shown by the code snippet below." (ibid)

With FBP we have a framework for composing behaviours from components without losing the overall flow.  For this specific scenario FBP has a better solution:

With FBP the use of bounded buffers between components avoids the need to re-post when a service comes back up.  When a downstream component goes down, upstream components are made aware through back pressure.

FBP also provides a mechanism for tracking IPs, so there is no need to reload event histories to check if any were missed.  What is probably one of the most overlook FBP features takes care of that:

"If you inject an IP in at one end of the network, you know it is going to get processed unless or until some process explicitly destroys it! Conversely, if you build a table IP, and forget to drop it when you are finished, many people might argue that it would be nice to have the system dispose of it for you. On the other hand... I could argue that such an error (if it is an error) may be a sign of something more fundamentally wrong with the design or the code. And anyway, all recent FBP implementations detect this error, and list the IPs not disposed of, so it is easy to figure out what you did wrong. 

At the moment I mix FBP with the event stream, with FBP components doing choreography against the event stream.

What I'd like to create (if only time would allow!) is the idea of creating an IP service similar to Amazon's Kinesis (https://aws.amazon.com/kinesis/streams/) that would support the creation of flow networks and the management of IPs that are processed through those networks.  This service would itself be a microservice; a smart endpoint managing and tracking the IPs.  

The API would be very similar to Kinesis.  They key difference is the management of the IPs.

The Kenisis API has getRecord putRecord: 

  * http://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetRecords.html

  * http://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html

The FBP API would have receiveIP and  sendIP.  The key difference is that the IP is managed.  Only one component can have receipt of an IP at any one time.  Once a component receives an IP it takes ownership.  It must either send it on or explicitly destroy it.  It cannot be silently dropped or missed.  

So we would also have a createIP and deleteIP,  Kenesis has no analogue because events are transient so the records appear when they are sent and disappear once they have been received.

In conclusion, I don't see it as FBP vs Microservices.  I see FBP as an alternative to Event Streams as an alternative choreography strategy.  

An alternative, not a replacement.  Both have there place.  Event Streams for transient events and IPs for persistent entities.

To draw the distinction I'd like to refer to Domain Driven Design.  IPs are used for managing Entities while Events are a type Value Object: http://domainlanguage.com/wp-content/uploads/2016/05/DDD_Reference_2015-03.pdf

• "Entities: Many objects represent a thread of continuity and identity, going through a lifecycle, though their attributes may change."
  
• "Value Objects: Some objects describe or compute some characteristic of a thing.   Many objects have no conceptual identity. "

When to use which?

"So whether an object is an Entity or a Value Object really depends on the context of how you are using it within your application. Generally speaking objects like location, dates, numbers or money will nearly always be Value Objects, and objects like people, products, files or sales will nearly always be entities."

http://culttt.com/2014/04/30/difference-entities-value-objects/ 

Regards, 

Ged

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-programming+unsubscri...@googlegroups.com.

[stewart mackenzie]

Hmm more like hanging a reusable coat up for a while.

I've been hired by the ethereum classic community to build a new virtual machine for them. This is inline with what I want to do with fractalide in that I was going to implement a ethereum virtual machine on my own time... just this time I'm being paid for it. (Yay no more footing the bills myself!)

The idea is to head towards an environment whereby fractalide is the application development environment with close ties to crypto contracts. In other words it'll be an advanced ethereum client.

The rust side of Fractalide is pretty much stable, you can build proper stable software in it now. I suppose I should bump Fractalide to stable. Oh yes, denis is working on making the virtual machine hot swappable. So this means you can reload the graph of FBP components without needing to stop the cluster of machines. This should hopefully come soon. This shouldn't be a stopper for a stable release I suppose...

Though why I'd not bumped to stable: I'd like to implement a whole new FBP in purescript sitting inside the Fractalide repo , allowing users to compose reusable and reproducible purescript components together. IPs flow from the server side rust FBP to client side purescript FBP.

I don't have time to implement this, once I get back onto it I'll continue with the purescript.

Indeed I've made space for multiple other FBP language implementations in Fractalide. Each implementation will make use of the knowledge and patterns discovered in rust FBP, i.e. same concept of reproducible reusable components communicating with capnproto contracts. You guys can start rolling out other FBP implementations in python, C, whatever language you want. Each implementation will give the community access to that language's libraries. So you can imagine a micro service deployment with multiple different FBP implementations shunting IPs over to machines where they are best processed (numpy, other specialist libs etc).

Having multiple implementations most likely means I need to swap out capnproto for something called colfer. Capnproto is super fast on platforms which support pointers, but it's pretty sucky on platforms like javascript. Colfer would be a more appropriate binary serialization format for a polyglot environment. Nevertheless If I do that port I'd port the whole eco system which would be crippling but needed.

Also the uptake has been disheartening. People are too familiar with their local environments and pretty much don't see the benefit of a reproducible congruent configuration management system which leverages FBP reusability. Manually reproducing large graphs of FBP components is a drag... in some senses each component is like an application with it's own lifecycle. That's too much overhead if you don't have a reproducible environment. The uptake story will change when the platform is used as a cryptocontract client. Nothing like a bit of deflationary currency to gather the people.

So if you want to get your hands dirty in a stable environment and start growing out the system with rust components, documentation and even new implementations of FBP please be my guest, I'll also gladly made contributors maintainers of the project. 

Hit me up with a Pull Request is this interests you!

kr/sjm 

[Ged Byrne]

Getting paid for stuff is nice.

FBP running on a blockchain, that's something I've been thinking about.

I wish I had time to get involved.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Flow Based Programming" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flow-based-progra...@googlegroups.com.

[Raoul Duke]

hi Stewart, you sure hit on a nice collection of keywords that excite
me; I wish you all the best of fortunes pushing this vision ahead!
Just know there are some people who do appreciate things like NixOS,
even if we aren't early adopters in fact, we are in spirit? :)

[stewart mackenzie]

This vision requires a large body of people, so far even getting a few contributors has like pulling teeth and this is the FBP mailing list!

We'll see what happens, I need a break from Fractalide, it was consuming me, though I'm quite happy with the results so far. 

Still a few warts left (nixcrates needs more work, purescript frontend, and colfer contract replacement)

Ged, technically it won't be running *on* the blockchain. It'll still be normal EVM bytecode but that executing bytecode will send signals which'll be converted into IPs and propagated through the FBP network.

Which in turn might switch switches or display a number on a website. Whatever.

Gosh even the NixOS community was openly hostile to Fractalide. They already have reproducible applications and language level libraries, i'd have imagined they'd have appreciated reproducible reusable functions.

Again no contributors from there.

I'll open this tool shed again some time in the future.

Anyway, FBP is very good for implementing the concept of microservices properly, by far it's the best approach I've seen. But! Your FBP implementation must subscribe to a congruent configuration management system. (https://www.usenix.org/legacy/event/lisa02/tech/full_papers/traugott/traugott_html/).

kr/sjm

[Brad Christensen]

Thanks for the update Stewart!

Hmm more like hanging a reusable coat up for a while.

Fwew.
 

The rust side of Fractalide is pretty much stable, you can build proper stable software in it now. I suppose I should bump Fractalide to stable. Oh yes, denis is working on making the virtual machine hot swappable. So this means you can reload the graph of FBP components without needing to stop the cluster of machines. This should hopefully come soon. This shouldn't be a stopper for a stable release I suppose...

I noticed the VM work Denis was doing, as well as the addition of Purescript related stuff. With this, I put a pause on investigating/asking questions with regard to how the VM works. But since it's come up, after understanding your existing documentation I was excited to learn more about the VM and its workings. Are there any other info on this apart from the implementation itself?
 

Though why I'd not bumped to stable: I'd like to implement a whole new FBP in purescript sitting inside the Fractalide repo , allowing users to compose reusable and reproducible purescript components together. IPs flow from the server side rust FBP to client side purescript FBP.

I don't have time to implement this, once I get back onto it I'll continue with the purescript.

This sounds like it's also the piece needed for fractalide's dataflow UI?

Indeed I've made space for multiple other FBP language implementations in Fractalide. Each implementation will make use of the knowledge and patterns discovered in rust FBP, i.e. same concept of reproducible reusable components communicating with capnproto contracts. You guys can start rolling out other FBP implementations in python, C, whatever language you want. Each implementation will give the community access to that language's libraries. So you can imagine a micro service deployment with multiple different FBP implementations shunting IPs over to machines where they are best processed (numpy, other specialist libs etc).

Good to know the structure is in place. The README also mentioned waiting for nixpkgs/nixos to support some features of systemd 232, are these ready to be taken advantage of? Similarly, integration of tokio-rs was mentioned on the road to stable. Was this for fractal related functionality, or to the fractalide core itself? Lastly, I had a play with nixcrates and did notice the number of derivations named after rust errors. Do I assume correctly that it is generally going to be a manual and possibly labor intense process in order to correct these builds?
 

Having multiple implementations most likely means I need to swap out capnproto for something called colfer. Capnproto is super fast on platforms which support pointers, but it's pretty sucky on platforms like javascript. Colfer would be a more appropriate binary serialization format for a polyglot environment. Nevertheless If I do that port I'd port the whole eco system which would be crippling but needed.

Is Colfer much slower than Capnproto? Crippling in the sense that all edges would need to be rewritten?
 

Also the uptake has been disheartening. People are too familiar with their local environments and pretty much don't see the benefit of a reproducible congruent configuration management system which leverages FBP reusability. Manually reproducing large graphs of FBP components is a drag... in some senses each component is like an application with it's own lifecycle. That's too much overhead if you don't have a reproducible environment. The uptake story will change when the platform is used as a cryptocontract client. Nothing like a bit of deflationary currency to gather the people.

FWIW, my initial reaction to Fractalide was similarly avoiding (prior to your adding documentation) due to my lack of understanding Nix. It looked really interesting, but I thought I didn't have the time to get into such an alternative. Nevertheless I did eventually end up investigating Nix/OS and am glad I did. A full deep dive. It did take some work, but only a few weeks to get up to speed (and the nixos community was helpful in this regard). Like you, I also find it odd those in the Nix community who are already invested in Nix/OS gave you such push back. Why some of them are dismissive of a new environment that makes such proper and practical use of their tools is puzzling. Perhaps some part of it was lost in translation, or it is the other paradigm move to FBP that put them off, who knows. Like you said, most are reluctant to stray from what's familiar.

Fractalide's use with crypto-contracts sounds interesting as well.
 

So if you want to get your hands dirty in a stable environment and start growing out the system with rust components, documentation and even new implementations of FBP please be my guest, I'll also gladly made contributors maintainers of the project.

Is there a preferred area for Fractalide related discussion and questions?

Thank you, Brad

[Paul Morrison]

Hi Ged, this is a great summary!  Your last para fits my image of the world of data exactly!  You also say, "Microservices ... advocate 'Smart Endpoints and Dumb Pipes'".  I think that is my approach also - I seem to want any complexity to be in the components, with the "pipes" being very simple (IMO) [FIFO] bounded buffers.  Whenever people try to put more complexity into the pipe mechanism, I find myself pushing back!  See also Chap. 15 of my book, 2nd ed., around Fig. 15.2...

Ernő, the last para of your last post seems to me to describe exactly how we went about building our applications.  This discussion seems to me to involve the whole issue of how one goes about developing reusable components.  In my book, I stress that it takes a fair bit of experience.  This is not meant as a put-down - what I mean is that components start to emerge when you find yourself doing the same job 3 times (Wayne Stevens' rule)!  Obviously this takes a certain amount of elapsed time.  It's even better when the components converge from multiple applications.   In my book, Chap. 4, I say, 

     "I seem to remember from my anthropology classes that the Australian aboriginals have a wonderful tool which is a combination of all the things
     they find most useful in their everyday life, and yet is highly portable. It is a combination spear-thrower, shield, dish and fire-starter.
     These kinds of tools are not invented overnight – they take time and experience, requiring progressive refinement by many creative people.
     The best tools will always evolve in this way."

And, Ernő, I think your English is fine!

Paul M.

[stewart mackenzie]

On Friday, March 10, 2017 at 5:13:13 AM UTC+8, Brad Christensen wrote:

 Are there any other info on this apart from the implementation itself?

Yeah the implementation itself, no point in documenting further, no users. 

 

This sounds like it's also the piece needed for fractalide's dataflow UI?

 

Well there's the the fractalide/fractal_ui_js library depends on a crate on crates.io that doesn't build with nixcrates. Denis wants to go the direction of using fractal_ui_js but I see a richer place by making fractalide a ployglot environment with multiple FBP implementations.

Just each implementation takes so much energy. 

Good to know the structure is in place. The README also mentioned waiting for nixpkgs/nixos to support some features of systemd 232, are these ready to be taken advantage of?

 

Yes the nix community have bumped to support systemd232 this will give a service a dynamically allocated uid which resolves permissions to writing to databases. I have not tested this yet as the fractalide/fractal_todo_* applications have stopped working since we ported to nixcrates.

 

Similarly, integration of tokio-rs was mentioned on the road to stable. Was this for fractal related functionality, or to the fractalide core itself?

 

Yes actually it's important for core functionality i.e. messaging from service to service. But one could use nanomsg... which unfortunately also doesn't work with nixcrates. Cargo plays really badly with nix, it does far too much. I don't regret setting cargo aside despite everything breaking with nixcrates, that software really does far too much. For more details on that you can read my rants here https://github.com/rust-lang/rust-roadmap/issues/12

 

Lastly, I had a play with nixcrates and did notice the number of derivations named after rust errors. Do I assume correctly that it is generally going to be a manual and possibly labor intense process in order to correct these builds?

It shouldn't be too intense, a small rust program could be written to read a Cargo.toml file and output key values onto stdout. Then this binary would be called in the buildPhase here: https://github.com/fractalide/nix-crates-index/blob/master/default.nix#L63 supplying the correct information where needed. The lack of reading a Cargo.toml file is the major issue.

I'm not sure what it'll take to fix those builds. I put enough money and time into nixcrates to make the whole fractalide experience as hermetically sealed as possible. But again no contributors, so it's a waste of my time and money. A grand cathedral is useless without people, a market is dead without traders.

Is Colfer much slower than Capnproto? Crippling in the sense that all edges would need to be rewritten?

In a pointer friendly environment capnproto is faster, they sacrifice safety for speed, in the sense that index tables are used and this is complex and finicky. Colfer is the better performer if you're looking at a polyglot environment.... which ideally fractalide should be.

Crippling in the sense that it would be a roughish job to strip out capnproto out of every component and fractalide/edges/** and replace every schema with a colfer schema then test the whole system to work.

I'd like to have one place, one schema language. So every component implemented in every language would talk one serialization format.

 

FWIW, my initial reaction to Fractalide was similarly avoiding (prior to your adding documentation) due to my lack of understanding Nix. It looked really interesting, but I thought I didn't have the time to get into such an alternative. Nevertheless I did eventually end up investigating Nix/OS and am glad I did. A full deep dive. It did take some work, but only a few weeks to get up to speed (and the nixos community was helpful in this regard). Like you, I also find it odd those in the Nix community who are already invested in Nix/OS gave you such push back. Why some of them are dismissive of a new environment that makes such proper and practical use of their tools is puzzling. Perhaps some part of it was lost in translation, or it is the other paradigm move to FBP that put them off, who knows. Like you said, most are reluctant to stray from what's familiar.

Yeah, if the FBP AND the NixOS AND the Rust community don't bite and take this forward it's dead in the water, a waste of my time. I initially thought that having an intersection of 3 communities would be superb. I couldn't be so wrong. 

   

Fractalide's use with crypto-contracts sounds interesting as well.

Well the hope is to make a decent language in blue waters. There is an existing community that forked from the original Ethereum guys, hence the master fork branded as the Ethereum Classic group are a  bunch of investors, miners and traders... without developers their currency is shot. So they treat devs much better than devs treating devs. Offering things to programmers is a waste of time, most CEOs are techno-peasant sheep and force programmers to follow a herd mentality, making them replaceable. Bah I could talk like a bitter man on the subject, but in the Ethereum Classic community you have all sorts of people. It'll be more interesting to explore this area. Maybe Fractalide can become their Ethereum Client? 

Let's see!

I shan't be round here nor the NixOS mailing list much! 

Thanks everyone for your help in making Fractalide! 

[Kenneth Kan]

Yeah, if the FBP AND the NixOS AND the Rust community don't bite and take this forward it's dead in the water, a waste of my time. I initially thought that having an intersection of 3 communities would be superb. I couldn't be so wrong.

There may be another community that would resonate with what Fractalide is solving. It sounds like each of these three communities already have something that they're attached to. What other communities have you explored so far?

[stewart mackenzie]

Yes Fractalide is different enough that each community finds it alien enough not to adopt. Nevermind!

I will put a 'fractal' in this account github.com/ethereumproject after I finish github.com/ethereumproject/sputnikvm (or before, if someone wants to get started). I hope it'll grow into a fully fledged ethereum client like http://ethcore.io.

I'm designing sputnikvm to be a rust library which can easily plug into a fractalide rust node. From there we'll plug (a completed} github.com/ethereumproject/emeralds-rs wallet in too.

Thus laying a good foundation for real world commercial applications to be built using fractalide.

Let's see!

[Kenneth Kan]

Yea, I'd love to see how that goes! I'm not too familiar with either Rust or Ethereum to be of help. Do keep us posted though!