Reviving Sandstorm

[Lyre Calliope]

I've been lurking since the Indiegogo campaign a few years ago, and I check in every once in awhile to see Sandstorm's status. I've been sad to see that it's largely stalled. A few months ago I was working on a tech security research project and discovered the object-capability model which has led me back here. 

I'd like to start conversation on bringing life back to the Sandstorm project.

I do a lot of work with small organizations and I can't tell you how often I've heard people independantly envision a platform like Sandstorm. Collaborative and security needs go beyond the scope of hosting project like Cloudron and Yunohost. As far as I’ve been able to find, there's nothing else on the open Internet like Sandstorm, and I think this is something the world needs.

While forking is always an option for continued development, I think a key part of what gives Sandstorm such potential is its' community. Looking at (relatively) recent conversations across the web I get the impression that there are plenty of true believers waiting to see if Sandstorm returns to being a platform they can confidently use in the long-term.

I understand that there are a number of challenges here such as funding and the availability of Kenton and others who have deep understanding of the underlying platform. I don't think these challenges are insurmountable.

I want to devote some of my time and resources to this. I have a background in community organizing within open source tech and would like to get some feedback on how I could help this project and community get back on its feet.

Cheers!

Lyre Calliope

[Adam Bliss]

On Fri, Nov 29, 2019, 05:12 Lyre Calliope <ly...@securingchange.org> wrote:

I've been lurking since the Indiegogo campaign a few years ago, and I check in every once in awhile to see Sandstorm's status. I've been sad to see that it's largely stalled. A few months ago I was working on a tech security research project and discovered the object-capability model which has led me back here. 

I'd like to start conversation on bringing life back to the Sandstorm project.

I feel the exact same way. I should have some time to contribute some code over the next year, and would love to see the community project revived.

I'm also fascinated by the approach of Helm ( https://www.thehelm.com ).I'd like to see if Sandstorm could be made to work well on an embedded ARM device with limited RAM. I wish my mother could buy a little box, like an answering machine, and instantly join the "redecentralize" movement of self-hosters and escape digital serfhood.

 --Adam

[Nolan Darilek]

I really miss working on Sandstorm apps. Here are some things I'd like to build but haven't because, quite frankly, I don't want to deal with the pain of having to host and secure them on more traditional infrastructure:

 * A web-based podcatcher. I want to start a podcast on my desktop, resume listening on my phone, and maybe pick up later on my Xbox. Maybe the podcast files are exported via WebDAV so I can access them locally. My feeds can be imported and exported, and I never have to worry about the service changing or shutting down.

 * A public transit schedule display that, given static GTFS data and an optional GTFS real-time feed, shows me arrival and departure information sliced and diced according to the needs of people who actually rely on public transit. So many of these systems are probably built by engineers who take Lyfts or single trains to their desks and start designing. I want an instant dashboard of all real-time bus departures within half a mile of where I am now. Don't make me click through individual stops or plot a route. Show me the information I need and let me decide what to do with it.

 * An app that lets me scan barcodes with my phone via Web RTC, looks up the code to identify the object, and stores that information server-side. So often I get a brand of something I like, but because I can't see the container, I don't remember what it was. I'd love to scan it, save it, and not worry about losing that data when my phone is lost, my PC wiped, etc.

All of these require outbound network access and some sort of keepalive support. I know outbound networking has always been available if I'm willing to learn CapnProto, learn whether a binding exists for my language, potentially write from scratch any networking code my app needs...I think we may have gotten outbound proxy support at some point late in the game, but I had trouble getting this to work IIRC.

I hate to put this out there since I know it conflicts with one of Sandstorm's primary design goals, but I think the increased security made it somewhat unattractive for me to develop for past a certain point. And while I understood the concern of apps phoning home and spying on users, I didn't want to phone home and spy on myself, or *anyone* for that matter. I wanted an SPK I could upload to my server and start receiving podcasts, or building a better bus schedule, or... The alternative isn't that I've written either and hosted it on my VPS. The alternative to the above is that I just haven't written either at *all*, because I don't want to build a one-off app for my bus schedule and open source it for others to try hosting, nor have I built my podcatcher because I'd rather focus on my core need and not authentication/security. I think Sandstorm was, and potentially still is, a good argument for the case that 70% security was a massive leap over 0%, but starting at 95%/99% made it a hard enough sell such that I'd consider Sandstorm for a handful of use cases but rule it out for many more. And saying that makes me a bit sad.

I'd love to see it revived, though. I wonder what some meaningful steps forward might be?

--
You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/CAEAeJWxWQbR9D5%2Bg56MPrkuZOByGvDHANdozcD3kFeOYc6SbDA%40mail.gmail.com.

[Ian Denhardt]

Quoting Adam Bliss (2019-11-29 08:54:28)
> On Fri, Nov 29, 2019, 05:12 Lyre Calliope <[1]ly...@securingchange.org>

> wrote:
>
> I've been lurking since the Indiegogo campaign a few years ago, and I
> check in every once in awhile to see Sandstorm's status. I've been sad
> to see that it's largely stalled. A few months ago I was working on a
> tech security research project and discovered the object-capability
> model which has led me back here.�
> I'd like to start conversation on bringing life back to the Sandstorm
> project.

I would definitely jump back in if others started doing stuff. I've
expressed to people that I think the project could come back to life if
some folks showed up and started to work on it.

Historically, I've had a hard time motivating myself to hack on
sandstorm itself, just because it's written in two languages I don't
really enjoy (I don't think it's productive to get into the why; folks
have their preferences). I had tried to contribute by building stuff
around sandstorm (dev tools, apps...) but we need folks hacking on core.
If others jump in, I might help. I will definitely get back to doing
stuff around sandstorm if it seems like it might pick up.

> I'd like to see if Sandstorm could be made to work well on an embedded
> ARM device with limited RAM.

There are a few problems to be solved to make this work; there's an open
issue about this, for status:

https://github.com/sandstorm-io/sandstorm/issues/2083

-Ian

[Ian Denhardt]

Quoting Nolan Darilek (2019-11-29 12:00:04)

I think the difficulty of writing apps that actually talking to the
network has been a big problem for a lot of folks; it's been a
showstopper for a lot of things that I've wanted to write too. I
actually *did* talk to the sandstorm APIs to get network access for a
couple things, but it wasn't the best experience. The powerbox APIs went
in like a week before Sandstorm Inc, shut down, so it's unsurprising
that the experience is rough.

I've definitely had the experience of not writing something because I
didn't want to figure out hosting for it.

I think much of this experience can be improved with stuff I can write
without touching sandstorm core. If you can give me precise requirements
for functionality, and think that given some support you might hack on
some of this, I can spitball some designs for stuff to make your use
case easier, and build some things if they seem like good ideas. We can
go from there.

I would love something like your transit app right now; I recently moved
to an area where there are like four different ways for me to get home
via public transit, and figuring out which one makes sense to use based
on estimated arrival times is a PITA.

Let me know,

-Ian

[Jacob Weisz]

I feel like there's a particular handful of lapses in Sandstorm core which need to be addressed for a lot of apps to make sense. One of them is literally sitting in an unmerged PR, which is cron support. I believe the initial hesitation to add it was the difficulty of policing it's abuse on Oasis, but that is no longer a long-term concern.

I think Sandstorm also badly needs a clear way to see and control the Powerbox tools it has. The various hacks for things like accessing outside web servers have no real visibility in the UI. So even if you believe that you are safe on Sandstorm, the unfortunate reality is Tiny Tiny RSS can reach out to any web server it wants right now. :/

As much as work on tools and such can help, I think someone who has time to dedicate to core development is absolutely a must-have. And we need more app maintenance, especially on core apps. Wekan is doing well. I would like to get EtherCalc up to date in the near future. Kenton would have to update or transfer Etherpad. Davros is very abandoned right now.

And assuming we can get all of this, if there's any development on Sandstorm itself, I presume Kenton would need to delegate an additional gatekeeper of some sort to the project/organization, as if anything, I imagine his schedule is only getting busier.

--
Jacob Weisz
in...@jacobweisz.com

> --
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to sandstorm-de...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/sandstorm-dev/157504983882.1734.13494628012944512589%40localhost.localdomain.
>

[Nolan Darilek]

My use case is probably overly broad. I want to find arbitrary libraries
in arbitrary languages that may communicate with the network, use them
in Sandstorm, and get many of the security/packaging benefits without
rewriting the networking layer of someone else's library.


Put more specifically, if I find a GTFS-RT library that pulls feeds from
the network in Python or Elixir, I want to use it in my app without
having to isolate and rewrite the networking code to use powerbox, and
then maintain that isolated fork in addition to the app I'm also
writing. I already write low-level software for my dayjob, want my side
projects to be more fun and higher-level, and if I'm having to rewrite
someone else's library to be Sandstorm-compatible, then I might as well
just use it directly and host my app behind HTTP basic auth outside of
Sandstorm. Put another way, I want Sandstorm to make my job easier, and
I don't see maintaining powerbox code for network access as any easier
than the problems I used Sandstorm to avoid.


I hate to pick on networking so much, but networking was the wall most
of my app ideas ran into when they became impractical. I wonder what
might be done to retain many of the benefits while making it easier for
app developers. Delay outbound connections until the user consents?
Track a list of DNS lookups so anything suspicious might be easier to
spot? I understand the threat model as it relates to Facebook/Google,
but for the open source developer wanting to make apps available for
easy hosting, it really felt like the model fought me more than it
should have. And I really tried to work within the constraints, but when
Sandstorm became a platform where many of my ideas couldn't happen, I
started searching for alternatives.

[Ian Denhardt]

Quoting Nolan Darilek (2019-12-01 18:02:12)

> I hate to pick on networking so much, but networking was the wall most
> of my app ideas ran into when they became impractical. I wonder what
> might be done to retain many of the benefits while making it easier for
> app developers. Delay outbound connections until the user consents?
> Track a list of DNS lookups so anything suspicious might be easier to
> spot? I understand the threat model as it relates to Facebook/Google,
> but for the open source developer wanting to make apps available for
> easy hosting, it really felt like the model fought me more than it
> should have. And I really tried to work within the constraints, but when
> Sandstorm became a platform where many of my ideas couldn't happen, I
> started searching for alternatives.

I actually really empathize with this; the set of things I want to build
that need to run on a server but don't need outbound networking access
is pretty small. Given these constraints, the initial focus on
productivity apps made a lot of sense, but it's also something that just
wasn't that exciting to me.

Spitballing: What if somewhere in your sandstorm-manifest.capnp, you
could specify url prefixes that your app might try to access? Something
like:


(
wantURLs = [
(
name = "Example Service",
prefix = "api.example.com/v4/"
)
]
)

Then, the first time the app makes a request to something that matches
one of these prefixes, sandstorm-http-bridge would make a powerbox
request which, on fulfillment, would give it a capability to make
requests under that prefix.

The bridge would proxy the connection transparently, so you wouldn't
have to do anything special at run-time.

Thoughts?

-Ian

[Jacob Weisz]

That’s kind of what I'd expect, since many apps will only look to speak with a short list of domains.

 

Sent from my Windows 10 device

--

You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/157524351520.908.14814592529273726164%40localhost.localdomain.

 

[David Chizmadia]

That could probably be generalized to a whitelist block in the sandstorm-manifest.capnp that would identify the protocol and the whitelisted URIs. Part of the SPK installation process could be a dialog between the installation admin and the Powerbox to select the protocol handler (if there is more than one) and authorize the URIs for the handler. That pattern would be familiar to anyone who has installed apps on their smartphone - if at a (potentially) finer level of granularity than usual.

Slightly off this subtopic, but relevant to the Subject line, I was recently browsing through the LibreOfiice website and discovered that they have a version of the suite intended as an open source version of Google Docs that is in search of a platform that would handle cybersecurity and other enterprise-grade considerations. My first thought was that it would be a fantastic "Killer App" for Sandstorm.

Dave Chizmadia

[Ian Denhardt]

Quoting David Chizmadia (2019-12-01 22:46:14)

> That could probably be generalized to a whitelist block in the
> sandstorm-manifest.capnp that would identify the protocol and the
> whitelisted URIs. Part of the SPK installation process could be a
> dialog between the installation admin and the Powerbox to select the
> protocol handler (if there is more than one) and authorize the URIs for
> the handler. That pattern would be familiar to anyone who has installed
> apps on their smartphone - if at a (potentially) finer level of
> granularity than usual.

I agree generally; this technique could be used for other protocols. I
think doing it at install/first start time is probably the wrong default
though, as it is easier for a user to understand requests if they come
in the context of using the app. So a first-access policy as a default
makes more sense to me.

Of course, especially with server apps it may be the case that the app
will first need the capability sometime when the user is not there to
grant it, so sometimes requesting access up front is unavoidable.

> Slightly off this subtopic, but relevant to the Subject line, I was
> recently browsing through the LibreOfiice website and discovered that
> they have a version of the suite intended as an open source version of
> Google Docs that is in search of a platform that would handle
> cybersecurity and other enterprise-grade considerations. My first
> thought was that it would be a fantastic "Killer App" for Sandstorm.

Interesting. Link? I didn't find it quickly poking around on the website
myself. It does seem like the perfect thing for someone to try to port.

-Ian

[David Chizmadia]

On Monday, December 2, 2019 at 2:37:22 AM UTC-5, Ian Denhardt wrote:

Quoting David Chizmadia (2019-12-01 22:46:14)

>    That could probably be generalized to a whitelist block in the
>    sandstorm-manifest.capnp that would identify the protocol and the
>    whitelisted URIs. Part of the SPK installation process could be a
>    dialog between the installation admin and the Powerbox to select the
>    protocol handler (if there is more than one) and authorize the URIs for
>    the handler. That pattern would be familiar to anyone who has installed
>    apps on their smartphone - if at a (potentially) finer level of
>    granularity than usual.

I agree generally; this technique could be used for other protocols. I
think doing it at install/first start time is probably the wrong default
though, as it is easier for a user to understand requests if they come
in the context of using the app. So a first-access policy as a default
makes more sense to me.

I can see your point if the decision to confirm the whitelist is per-grain. I do think that the choice between per-grain and site-wide confirmation of the whitelist should be made explicitly during installation. 

Of course, especially with server apps it may be the case that the app
will first need the capability sometime when the user is not there to
grant it, so sometimes requesting access up front is unavoidable.

>    Slightly off this subtopic, but relevant to the Subject line, I was
>    recently browsing through the LibreOfiice website and discovered that
>    they have a version of the suite intended as an open source version of
>    Google Docs that is in search of a platform that would handle
>    cybersecurity and other enterprise-grade considerations. My first
>    thought was that it would be a fantastic "Killer App" for Sandstorm.

Interesting. Link? I didn't find it quickly poking around on the website
myself. It does seem like the perfect thing for someone to try to port.

I thought that it would make more sense to discuss LibreOffice (Online) in a separate thread, so I started

https://groups.google.com/forum/#!topic/sandstorm-dev/jXJixf8_Otk

-Ian

Dave Chizmadia 

[Jacob Weisz]

There's cases for on-install permissions (for all of the app's grains) and on-use permissions (each grain when it needs access), and I think it'd be ideal to support both. For example, if you have say, a "Gmail Importer" app, it'd make sense for installing the app to come with network permission to gmail.com by default, but something like draw.io, which the version Sandstorm has calls out to export.draw.io or something like that when generating a PDF, should ask for permission to export.draw.io when you try to use that feature. Presumably one should define both "must-haves" and "on-uses" within the package definition, in part so they can be reviewed during app review. Something like TinyTinyRSS, if updated for the modern Powerbox way of doing things, would need to be able to arbitrarily request new domains when a feed is added by the user.

--

  Jacob Weisz

  goo...@jacobweisz.com

--

You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/479ef656-d283-4d47-a4f0-411687f915eb%40googlegroups.com.

[xet7]

friday 29. november 2019 20.27.28 UTC+2 Jacob Weisz wrote:

in the near future. Kenton would have to update or transfer Etherpad. Davros is very abandoned right now.

Well, Davros is broken because File ID and permissions are empty on that code:
https://github.com/owncloud/client/issues/6775#issuecomment-422423147

Davros code is here:
https://github.com/mnutt/davros/ 

 

My current todo list is quite full already with Wekan. If someone would like to contribute to some Sandstorm app,
I can fork it to https://github.com/sandstormports , move issues, and provide git access.

I don't know if some in previous comments are just waiting some "start signal" before starting to contribute,

I would wish that they just start asking any questions and contributing, like I have done for 3 years already.

BR,

xet7

[Nolan Darilek]

This wouldn't support 2/3 of my use cases, unfortunately. Transit feed URLs vary per agency, and podcast URLs are arbitrary.

If the app can whitelist URLs via wildcard, that should be fine. I'd hope users would understand the need to whitelist all URLs for a podcatcher, or for a transit schedule needing to fetch feeds and data from arbitrary places. But, while I could lock certain apps to specific URLs, I couldn't for these two.

[Ian Denhardt]

In addition to the install time vs. use time split, I also want to point
out that in many cases what you want is not a *specific* resource, but
an instance of some *type* of resource.

For something like a feed reader/podcatcher, the natural "powerbox" way
of doing things would be something like:

* User clicks "add feed"
* App says "hey sandstorm, I need access to a URL; can you get one from
the user for me?"
* Powerbox UI prompts the user for a URL
* User enters feed URL into powerbox UI
* Sandstorm gives the app a capability to fetch that URL.

For apps where this flow makes sense, I'm wary of making asking for
*specific* resources feel like the natural way to do things. I think
it's important that app developers don't have to tear their low-level
networking code apart, but I don't want to end up in the situation
we often are with smartphone apps where you install an app and it says
"I need access to <stupidly long list of permissions with no
explanation>, take it or leave it."

I think in most cases where an app can take an *arbitrary* URL, you
should see a workflow like the above.

I think you could still put together a proxying implementation for the
actual use of the URL, where sandstorm-http-bridge sits between the app
and sandstorm, and when the app makes an HTTP request, looks up the URL
in its list of capabilities it's been granted, and uses the appropriate
one.

Retrofitting the UI flow on to existing apps might be a bit of work. But
I'm pessimistic about our hopes for ever being able to port whole apps
without modification and have features that assume broad ambient
authority "just work." I'm personally more inclined to tune for the use
case where you're writing a new app, but want to be able to use existing
*libraries*.

-Ian

Quoting Jacob Weisz (2019-12-02 10:50:57)

> an email to [1]sandstorm-de...@googlegroups.com.

>
> To view this discussion on the web visit

> [2]https://groups.google.com/d/msgid/sandstorm-dev/479ef656-d283-4d47-a
> 4f0-411687f915eb%40googlegroups.com.

>
> --
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to [3]sandstorm-de...@googlegroups.com.

> To view this discussion on the web visit

> [4]https://groups.google.com/d/msgid/sandstorm-dev/1a570583-2017-4f47-8
> f77-0b94089c5340%40www.fastmail.com.
>
> Verweise
>
> 1. mailto:sandstorm-de...@googlegroups.com
> 2. https://groups.google.com/d/msgid/sandstorm-dev/479ef656-d283-4d47-a4f0-411687f915eb%40googlegroups.com?utm_medium=email&utm_source=footer
> 3. mailto:sandstorm-de...@googlegroups.com
> 4. https://groups.google.com/d/msgid/sandstorm-dev/1a570583-2017-4f47-8f77-0b94089c5340%40www.fastmail.com?utm_medium=email&utm_source=footer

[Lyre Calliope]

Here are some observations I’m making based on this thread so far. Let me know how this resonates or if I’m missing the mark. 

This thread has largely been an exploration of project ideas and challenges (personal projects, "killer apps", experiments, etc) as well as discussion on the limitations of the platform.

The core platform has a number of important features that are incomplete or missing, many of which are already documented in github and the roadmap. Some of these features seem to be tied to UX work that has yet to be done around the Powerbox model.

There’s tension between porting existing apps vs building sandstorm native apps from scratch. This tension is in part a strategic question because it informs decision making around what’s worth spending time and resources on building at the core, personal, and ecosystem levels.

Many development decisions were made around plans for Oasis. Now that these strategic assumptions no longer apply, there’s a bunch of brush that can be cleared out in order to create a new path forward.

There’s plenty of latent opportunity for people to get involved, but these opportunities are not very visible or accessible for those who aren’t willing to take a deep dive into the world of Sandstorm documentation.

It was also mentioned that bringing in new core contributors would necessitate Kenton onboarding new gatekeepers to the main codebase. This has me wondering: beyond contribution to the main codebase, what are other gates that might be keeping more people from getting involved and getting things done?

I’ve seen a bunch of retrospective-type writing from Kenton accross the web and especially on hackernews. Has anyone collected this writing into a more cohesive project retrospective form that we can learn from? If not, is this something anyone would be interested in helping me take this on?

-Lyre

To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/157532182884.887.16013839353107910732%40localhost.localdomain.

[xet7]

At Hacker news, Kenton's submissions and comments links are here:

https://news.ycombinator.com/user?id=kentonv

There is also some Sandstorm blogposts:

https://sandstorm.io/news/

AFAIK Sandstorm still is the only platform where grains are running only when they are used.

Other platforms have similar containers running all the time, taking much more RAM and CPU time.

So Sandstorm is ahead of Docker, Snap, LXC, LXD, etc.

And now Kenton is working on most advanced leading tech,

CloudFlare Workers that deploy JS/WASM globally in seconds,

backed by keyvalue store, to store data.

Fastly also has tech in production that is similar to CloudFlare Workers.

Javascript can be used or Workers, or some other programming

language like Rust/Go/WebAssembly compiled to binary WASM format.

For existing applications, it's easier to port to Sandstorm.

For Workers, usually it requires creating new applications and

trying to keep the amounf of code small.

BR,

xet7

[Jacob Weisz]

I'm less worried about a retrospective or epitaph then trying to get the project back moving again. ;) Like, I keep getting this feeling Sandstorm was just before the market was ready to accept it. Sandstorm was there when you had to explain to someone why they'd want to avoid putting all their data into Google services. The focus on getting enterprise customers at a time enterprise customers didn't see the benefits was probably a big part of the problem.

App development will be a manageable problem if we can get core development happening again. So the question to me, is how do we get core development moving? Ian remains the absolute leading community member who understands the backend of Sandstorm and how the Powerbox works, but I don't know if he has time/interest to commit to working with a stack he doesn't like. The other person here who probably has the requisite skill to contribute to core is Lauri, who's been working on Wekan for years and knows the frontend/shell side of things. I don't know if Lauri has time/interest in it, but I know he does features and fixes to Wekan for money, if he has time, perhaps he could do shell work on Sandstorm for the same?

Fundamentally, I think for people to invest any amount of time in Sandstorm, we need to find money for them. Back when I did the bounty board I was ready to put up some money for new features and new apps, and that's still true (though I need a new link for the board probably). In fact, my $24 a month will no longer be going to Oasis in the new year, so I have some budget available to work with.

To roll back around to Lauri again: Look at what he did with Wekan. Wekan was a dead project. The person who wrote Wekan originally is not an active participant. As an outside contributor he stepped in, overhauled the project, and there's community contributors who have collected three-figure bounties and a commercial support strategy in place.

--

  Jacob Weisz

  goo...@jacobweisz.com

[Ian Denhardt]

Quoting Jacob Weisz (2019-12-03 09:55:55)

> I'm less worried about a retrospective or epitaph then trying to get
> the project back moving again. ;)

+1

I have a suspicion that, had Sandstorm managed to onboard a couple
external contributors before going out of business, the project would
have kept moving; I think much of the problem is that suddenly going
from 10 people working full time to one person doing security patches
and not much else really knocked the wind out of it. But as a community
we'd gotten used to being able to count on the full-time staff to do
core stuff, and back then my to actually use the native APIs were much
more useful to the project than I think contributing to core would have
been. That situation changed abruptly without the community really being
able to get used to the idea that core needed contributions.

> App development will be a manageable problem if we can get core
> development happening again.

+1. The first pass of the powerbox API landed literally *weeks* before
the going-out-of-buisness blog post, So I think the experience of using
it can get a lot better.

> Ian remains the absolute leading community member who understands the
> backend of Sandstorm and how the Powerbox works, but I don't know if
> he has time/interest to commit to working with a stack he doesn't
> like.
>

> [...]

>
> Fundamentally, I think for people to invest any amount of time in
> Sandstorm, we need to find money for them.

FWIW, getting paid would probably be enough to sweeten the deal; I've
worked with less savory tech stacks on less interesting projects in the
past. And I've learned enough JS in the context of contract work by now
that the prospect of digging into the shell seems like less of a big
deal, and even though I'm not big on C++, I do miss doing systems
programming.

---

Besides finding new outside contributors, I worry about Kenton's ability
to keep up with PRs if we get some momentum going. There are several
open PRs that are fairly small that have just been sitting there. But
perhaps if that doesn't happen folks can coalesce around a fork.

-Ian

[Jacob Weisz]

From what I've gathered of Kenton's whereabouts right now, he is super super busy and will be for a while. But as Sandstorm is now designated as a community project going forward, I imagine he would be open to delegating some rights if a plan was in place, given good practices were to continue to be upheld. (I recall Sandstorm-the-company entailing every change being in a PR and another contributor reviewing any non-trivial PR, etc.)

Getting an easier-to-use Powerbox for networking, and maybe getting the Cron PR tested/merged/etc. would probably enable a lot of new app possibilities pretty quickly, and hopefully get people interested in new app packages and updates.

I'd like to imagine we could come up with some sort of developer-supporting model using GitHub Sponsors or Patreon or Bountysource or such. Sandstorm-the-company obviously didn't take donations, but presumably Sandstorm-the-community-project could. And if we get moving again, we can "market" again. If there's new stuff happening in Sandstorm, we can get it back in front of the HN crowd, and hopefully pick up even more interest from there. Right now, it's hard to recommend Sandstorm because of where it sits on maintenance and future outlook.

--
Jacob Weisz
goo...@jacobweisz.com

[Ian Denhardt]

Quoting Jacob Weisz (2019-12-03 14:26:00)

> Getting an easier-to-use Powerbox for networking.

I am keen on working on this, but I want input from others, as I think
my own intuition for what would make it nicer to work with will only
take me so far.

> Sandstorm-the-company obviously didn't take donations, but presumably
> Sandstorm-the-community-project could.

Long term, if we can get things organized enough, it might make sense to
consider joining the Software Freedom Conservancy or some other umbrella
organization, to make donations and logistical stuff easier. But I think
we need to breathe some life into the project before it makes sense to
think about that. Maybe something to keep in the back of our minds.

-Ian

[xet7]

Yes I can do paid work for Sandstorm.

For Wekan, there still is some bounties at BountySource that I have not done yet. Some time ago BountySource had trouble with PayPal and were unable to accept new bounties. Sometimes logging in to BountySource does not work. That's why I don't recommend BountySource.

Currently I require prepayment before starting to implement a feature, because one of my customers did not pay for feature I developed for him. Another reason is, that I do receive new bills for rent, electricity, etc all the time and they have extra fees when paying of bill is late, they can not wait.

For prepayments, I did setup my own PayPal payment page to https://wekan.team/commercial-support/ . It has mostly worked OK, but in one case PayPal cancelled payment. PayPal also has quite high fees and is slow to move money.

Another option TransferWise https://transferwise.com/ , it's very fast and with minimal fees, I think TransferWise is so excellent that they will continue having business much longer than PayPal.

If someone needs invoice, it's better for me to send invoice that has IBAN bank account number.

I have signed up to GitHub Sponsors, but I think it's too complicated, I'm not sure yet am I brave enough to use it. With it, I would need to connect GitHub Sponsors to my Stripe account. Stripe is also more complicated than TransferWise, Stripe has higher fees than TransferWise, etc. Stripe has multiple payment options, like VISA/MasterCard/AliPay etc etc, but I have not yet been brave enough to use it, or try to configure all of it.

I have not used Patreon, I don't know how well it works.

Well, I should probably add more details about payment options to my website.

Anyway, Meteor is what I have done for these 3 years. I have upgraded to new Meteor versions many times, made some changes to other Meteor-based software like RocketChat and WireFlow https://github.com/vanila-io/wireflow/pull/31  , etc.

(I have not yet figured out this CloudFlare Workers stuff, it's at very yearly beginning.)

BR,

xet7

[Kenton Varda]

Hi all,

Sorry for not responding earlier. The list has been so quiet that I had kind of stopped paying attention to it.

It's great to see that people are interested in contributing to Sandstorm! I would be happy to help by doing code reviews, advising on how to design features, and of course continuing to push releases. Maybe if I find some time I'll even contribute code.

Yeah, there are some PRs that have been sitting for a while without action. Sorry about that. It's been difficult to be motivated to review little PRs when it seemed like the project as a whole wasn't going anywhere, but if people are serious about contributing I can be more serious about reviewing.

Note that I will probably only have time on weekends -- and probably only a few hours each weekend. I have a pretty busy day job and a 6-month-old baby to take care of. Also this month I am coordinating a move from Palo Alto to Austin, and after moving I'll be spending time over the next year or two coordinating construction of a new house in Austin. So... lots on my plate. But I'll make time for reviews, and if I can't, I'll arrange for others to get merge rights.

One big worry I have is that the Sandstorm codebase is not in an excellent state of cleanliness. Over the course of development, we learned a lot of things about how we ought to have been organizing things, but we rarely had time to go back and update the old code to the new standards. So, you'll find a lot of inconsistency, where some of the older code is written in a much uglier way, with half-finished attempts to clean things up.

Also worrying is that it's all built on Meteor, which itself is a somewhat stalled project. Yes, Meteor is still officially supported, but the company that built it originally actually sold it off to a holding company in order to focus on Apollo GraphQL instead. These days Meteor has fallen behind the state of the art and it doesn't look likely to catch up. Worse, Sandstorm's HTML is mostly based on Blaze templates, which is what Meteor was pushing before they gave in to React. I liked the high-level design of Blaze, but the lower-level APIs turn out pretty ugly, IMO mostly due to a lack of object-orientation. You end up with lots of callback functions that are called in various poorly-defined contexts; the same callback might be called with a variety of different input states, and it's extremely hard to tell from reading the callback what state it should expect. But Sandstorm is all built on this and it's kind of a mess.

But trying to rewrite it from scratch is clearly not remotely practical, especially given the very limited development resources here. So we're going to have to live with it. But we should really nail down exactly the standards we want for new code, hopefully with the possibility of incrementally improving the old code to match...

-Kenton

--
You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/f8fd952d-bc32-40b9-b109-5ea52e398148%40www.fastmail.com.

[xet7]

I don't know is Meteor really stalled. Pull request for Node.js 12 is in progress:

https://github.com/meteor/meteor/pull/10527

At Meteor forums there are info about new updated Meteor packages etc.

It could just be that newbie talk that about every programming language

and web framework is dead. I did write blogpost about it:

https://dev.to/xet7/illusion-of-differences-between-operating-systems-programming-languages-and-web-frameworks-4j8f

For example, Perl 5 is not dead, it's getting regular releases, like mentioned here:

https://twit.tv/shows/floss-weekly/episodes/544

Wekan is also made with Meteor and Blaze. According to OpenHub, Wekan

has took estimated 13 years of effort:

https://www.openhub.net/p/wekan

Most likely it has also taken a lot of time to get Sandstorm to usable state,

with a lot of bugfixes, features, etc.

What I have not yet found anywhere, is how to really rewrite some large software,

in any sensible amount of time. Usually that would require big team of programmers,

and many years of effort.

Really, Sandstorm grains are still more advanced than Docker/Snap/LXC/LXD etc.

BR,

xet7

[Ian Denhardt]

I do wonder how much of this is the usual tendency in the JS community
to assume something is dead if it hasn't made breaking changes in 2
weeks... :P

But I think we're all on the same page that re-writing sandstorm is not
on the table. We could talk about migrating subsystems to other
foundations piecemeal, if we feel the need, but I'm more keen on trying
to make progress and seeing what actually needs doing.

Migrating bits of the code to typescript might make future refactoring
easier. If we can get node-capnp to spit out typescript interface files,
that would probably make a dent in the amount of untyped code we're
working with. Food for thought.

In the meantime, I'm going to poke at the scheduling pr; I've got the
merge conflicts fixed on a branch (that was trivial), and am fussing
with some build issues. Will keep y'all posted.

-Ian

Quoting xet7 (2019-12-05 14:34:47)

> I don't know is Meteor really stalled. Pull request for Node.js 12 is
> in progress:

> [1]https://github.com/meteor/meteor/pull/10527

> At Meteor forums there are info about new updated Meteor packages etc.
> It could just be that newbie talk that about every programming language
> and web framework is dead. I did write blogpost about it:

> [2]https://dev.to/xet7/illusion-of-differences-between-operating-system
> s-programming-languages-and-web-frameworks-4j8f

> For example, Perl 5 is not dead, it's getting regular releases, like
> mentioned here:

> [3]https://twit.tv/shows/floss-weekly/episodes/544

> Wekan is also made with Meteor and Blaze. According to OpenHub, Wekan
> has took estimated 13 years of effort:

> [4]https://www.openhub.net/p/wekan

> Most likely it has also taken a lot of time to get Sandstorm to usable
> state,
> with a lot of bugfixes, features, etc.
> What I have not yet found anywhere, is how to really rewrite some large
> software,
> in any sensible amount of time. Usually that would require big team of
> programmers,
> and many years of effort.
> Really, Sandstorm grains are still more advanced than
> Docker/Snap/LXC/LXD etc.
> BR,
> xet7
>

> --
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to [5]sandstorm-de...@googlegroups.com.

> To view this discussion on the web visit

> [6]https://groups.google.com/d/msgid/sandstorm-dev/7086f434-4a09-400f-a
> 5e0-b1f922588970%40googlegroups.com.
>
> Verweise
>
> 1. https://github.com/meteor/meteor/pull/10527
> 2. https://dev.to/xet7/illusion-of-differences-between-operating-systems-programming-languages-and-web-frameworks-4j8f
> 3. https://twit.tv/shows/floss-weekly/episodes/544
> 4. https://www.openhub.net/p/wekan
> 5. mailto:sandstorm-de...@googlegroups.com
> 6. https://groups.google.com/d/msgid/sandstorm-dev/7086f434-4a09-400f-a5e0-b1f922588970%40googlegroups.com?utm_medium=email&utm_source=footer

[Dan Krol]

I'm excited to see this resurgence in interest. I was hoping this would happen, since Sandstorm has some unique things to offer among the self-hosting platforms.

I would be interested in jumping into some Sandstorm development, and I'll have availability over the next few months. However, so far I've only done app development. I'm wondering if anybody has onboarding suggestions for core development? Maybe some easy starting tasks to tackle?

To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/157557660177.11687.15819991046486418538%40localhost.localdomain.

[Jacob Weisz]

Shell enhancement is probably the "easiest" to get into because it doesn't require understanding as much of the plumbing that makes Sandstorm work. There's a lot of UI functionality I feel Sandstorm is missing that may not be that sophisticated to implement. There's also good instructions in Docs for hacking on the Sandstorm shell, last I checked. I went through some of the Meteor tutorial at some point to get the gist of how the Meteor framework does things.

From seeing what you did with the Kiwix package, you seem to have a knack for making things pretty user-friendly too.

--

  Jacob Weisz

  in...@jacobweisz.com

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/CAAWRcS9KiCO8mvJ1phhEcbL58nKyzZTn%3DOaQKfu8bVD6q3Kutw%40mail.gmail.com.

[Lyre Calliope]

It sounds like we have a good combo of knowledge and expertise here to get the engine running again. Now we need some fuel.

Quoting Jacob:

I'd like to imagine we could come up with some sort of developer-supporting model using GitHub Sponsors or Patreon or Bountysource or such. Sandstorm-the-company obviously didn't take donations, but presumably Sandstorm-the-community-project could.

From Ian:

Long term, if we can get things organized enough, it might make sense to
consider joining the Software Freedom Conservancy or some other umbrella
organization, to make donations and logistical stuff easier. But I think
we need to breathe some life into the project before it makes sense to
think about that.

Fiscal Sponsorship is a tool within the non-profit space that I think is highly underutilized. A shorter-term fiscal host could be worth talking about now as a logistical detail within a larger fundraising effort.

A tool for collecting and managing funds that might be a good fit for Sandstorm is Open Collective. It’s a bit like Patreon for projects rather than for individuals, and it’s being largely built around the needs of open source projects. From what I understand it handles the logistics of community crowdfunding (for both one-time and reoccurring payments), open budgeting, transparent spending/dispersing of funds, AND fiscal hosts can join the platform to offer their legal infrastructure to projects. 

One fiscal host within Open Collective with a US 501c6 tax designation is the Open Source Collective which looks like it might be a great match given other projects tech projects they’re fiscally hosting.

I’d love to get y’all’s feedback on whether Open Collective might be a good fit for helping us get the money flowing. Here’s the non-markety documentation around what the platform offers: https://docs.opencollective.com/help/product/product

Before setting specific fundraising goals, we should have some discussion around strategy and budget. Yes we can raise money to pay for some short-term dev work, but I think we should be more ambitious and set the stage for greater fundraising later on. For example, I think we should discuss budgeting for a writer or two to focus on updating documentation and writing grants. 

Also, a UX designer with some coding chops who really -gets- ocaps. (Let’s throw some real money at the powerbox!)

For my own part, I’d like to be able to do some of the non-code work that needs to be done. Community organizing, participation design and new contributor onboarding, some product management-type activities, documentation, etc, etc. (FYI, I should probably mention that II don’t code, but I am highly code literate.)

Maybe it’s time to break funding/resourcing out into its own thread. 

Moving on to the codebase discussion:

From Kenton:

One big worry I have is that the Sandstorm codebase is not in an excellent state of cleanliness.

Also worrying is that it's all built on Meteor, which itself is a somewhat stalled project.

Meteor did have a slowdown for a bit, but the new team seems to be picking up steam and figuring out new priorities to devote their resources too. 

I think updating to the latest version of Meteor and then migrating from Blaze to React will set the codebase up for more flexible shell enhancement that more people can get involved with.

Here are some links that may help with thinking through incremental migration to React:

• From a Meteor forum thread: "In a nutshell you start at the leaf nodes and then pass the blaze data down into it. Then just keep working upward until you hit the router and at that point you can use react-layout to render the component."
  
• Video mentioned in that thread: https://www.youtube.com/watch?v=BF58ZJ1ZQxY
• Here’s a github issue that shows recent signs of life: “Turning react-meteor-data to a React hook”
  

Btw, does anyone have experience with Stenciljs? Before even beginning to migrate to React, the shell could be broken down into framework-agnostic, standards-based web components.

Naturally not a current priority, but perhaps exploring the state of the art in developing PWAs and native mobile applications might inform what front-end code standards should look like in the short-term.

One thing that’s really stalled with Meteor is their Cordova integration which is like two versions behind. Switching to React however gives us the ability to play with React Native, or even the recently released Capacitor which is backwards compatible with Cordova. 

Come to think of it, migrating to React makes the whole Ionic stack something worth looking at. From October: Announcing Ionic React. And just this week: Announcing Ionic React Hooks. 

-Lyre

[xet7]

Huh, React or Web Components? That's so yesterday's tech.

https://dev.to/richharris/why-i-don-t-use-web-components-2cia

https://svelte.dev/blog/virtual-dom-is-pure-overhead

In that Meteor Node 12 PR there have been discussions of updating Cordova.

I also only know about Blaze. I don't know about React and Web Components.

If something like React or React Native would be needed, I don't know why

Sandstorm UI layer itself should be converted to it. If that is for some mobile app,

it's better to just more (user or admin) REST API like this:

https://github.com/wekan/wekan/blob/master/models/cards.js#L1885

And using it from any mobile web app framework, and not tying browser+mobile

so tightly together.

BR,

xet7

[Jacob Weisz]

I think radical restructure discussions are a bit premature at this juncture. Presumably at some future date, parts of Sandstorm's stack will be unsupported/broken and alternatives will have to be considered, but for now we should just get Sandstorm itself moving again.

Regarding funding, any large scale assignment of the project to some organization or another would have to be made by Kenton. Given his availability right now, let's keep it simple for now. I know I'd dedicated a couple hundred bucks to the cron PR, and I'm still up for handing some coffee money over to folks for app ports and updates. (Bounty board redux: https://ocdhost.sandcats.io:6080/shared/Wm1x3RotbGIqJxi2zi-Ls3ZpCnFV9-3VnWPpoeFdtxR contains my funding offers sans everyone else's which may not still be upheld given the passage of time.) Hopefully some others can join that effort, and we can maybe make some incremental progress which justifies looking at more significant steps.

I am similar to Lyre in that I can help with documentation/community, and I can dabble in the code a bit, but I don't sit on the level of the developers who put this thing together. ;)

One particular new interest I have is that I've got a PinePhone preorder in, so I'm going to be super interested in finding client apps and sync integrations that'll let me use a true Linux phone with Sandstorm.

--

  Jacob Weisz

  in...@jacobweisz.com

--

You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/41ac94a2-01ab-4c08-a81a-c8c03b1538b1%40googlegroups.com.

[xet7]

Yes, sorry that I above was not clear enough.

I did mean, that rewrite is a dirty word. Nothing should be rewritten, ever. It's just same work and bugfixes again.

Original author of Wekan tried to rewrite Wekan, got tired in rewrite, so I had to continue maintain Wekan.
https://github.com/wekan/wekan/wiki/FAQ#what-was-wekan-fork--wefork

I have multiple failed rewrite attempts, it just is not possible.
https://github.com/wekan/demo/wiki/Roadmap

Also see:
https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/

BR,
xet7

[Lyre Calliope]

Def agree that restructuring is premature and rewriting is a pretty bad idea. (At least without something like 6 figure investment.)

On Dec 6, 2019, 18:42 -0500, Jacob Weisz <in...@jacobweisz.com>, wrote:

...for now we should just get Sandstorm itself moving again. 

This isn’t your first time saying this in this thread. Pragmatism is definitely important for making things happen, and I’d like to find some clarity on what ‘just getting sandstorm moving again’ entails. 

Questions on my mind:

• How do we recognize if we’ve been successful?
• Can this be articulated as a milestone of some kind? 
  
• What scope of activity is appropriate to accomplish this, and what should be out of scope?
• Once we have consensus around all this, how can we discuss things that are out of the scope of this milestone in productive ways?

If we can find answers to the first two questions especially, I think we’ll be in a better position to evaluate what specific actions, commitments, and resources are needed to successfully revive this project.

-Lyre

[Adam Bliss]

Here's an idea for a tractable milestone: is anyone else interested in moving the Sandstorm development community to be entirely self-hosted? I personally would love to see it move off of github and googlegroups. Whatever new features might be needed for such a migration would be a good set to focus on.

--Adam  

--

You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/8bd9c738-c93c-48af-83de-736417947698%40Spark.

[Jacob Weisz]

Personally, I prefer GitHub for the collaborative benefits. It's harder to get first time contributions or drive-by fixes in an isolated Git site. (And honestly, GitLab.com doesn't sound inherently "better" right now as a corporate host.) I definitely wouldn't mind being away from Google Groups though. The general operation of Git repos on Sandstorm is workable already, mind you, the email side of Sandstorm is probably one of the weakest, platform-wise.

Roadmap-wise, I'd want to chat with Ian a bit about what he thinks is realistic for platform features, with Kenton about what we can do to take some of the non-code burden off his plate, and then I think we can assemble kinda a six-month outlook of where we'd like the project to be. I'd like to see a good idea what features are coming to Sandstorm, what apps can get updated and added to Sandstorm, etc.

--

  Jacob Weisz

  in...@jacobweisz.com

[xet7]

Moving off github or googlegroups just moves things around, breaks all existing links, breaks all github forks of sandstorm, and makes everyone create new user accounts. It's too early and premature for that.

First Sandstorm and it's all apps should be made up-to-date with automated builds and pull requests merged.

BR,
xet7

[Dan Krol]

On the topic of automated builds - how do people feel about automated tests with Selenium, to make sure apps aren't broken with Sandstorm upgrades? The isolation of the apps in Sandstorm makes this so much easier than it would otherwise be.

Perhaps for a later milestone.

--
You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/bd4a346c-bf5d-4515-99ce-5426c1df7cfb%40googlegroups.com.

[xet7]

Sure at later milestone some automated tests would be nice, if someone would contribute tests. For example Wekan does only have lint/prettier checks, and not any tests.

There was some tests at some Wekan groups code that was sent to me, but that code is not yet in Wekan, I have not yet figured out how that code and tests would work.

BR,
xey7

[Jacob Weisz]

Sandstorm I think has tests. Testing Sandstorm apps for breakage in Sandstorm releases isn't too valuable though, as I believe the very first SPK written for Sandstorm still works on current releases.

 

The biggest breakage will probably happen when network access hacks get removed/closed.

 

Sent from my Windows 10 device

 

From: xet7
Sent: Saturday, December 7, 2019 10:42 AM
To: Sandstorm Development
Subject: Re: [sandstorm-dev] Reviving Sandstorm

 

Sure at later milestone some automated tests would be nice, if someone would contribute tests. For example Wekan does only have lint/prettier checks, and not any tests.

--

You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/946d0182-74ab-4c57-b3c2-8a14196251e8%40googlegroups.com.

 

[Ian Denhardt]

If you're talking about running Sandstorm infrastructure on a Sandstorm
box, I think that's much further off than a first milestone; there are
some non-trivial design questions about how "public-facing" apps would
even work. I'd love for folks to start thinking about that, but I don't
think this line of development is at the point where it makes sense for
us to make a major push. It's still very much in the "needs thought"
phase.

While I'd love to make Sandstorm suitable for self-hosting public facing
apps, I think there's lower-hanging fruit, and especially with the
project just getting back on its feet, I'd like to see us hit some
smaller targets.

Below is a brain dump of some things I think are worth paying attention
to near-term. Some of these already have issues open for them, probably
it makes sense to take detailed discussion of each them on to the issue
tracker.

---

In terms of core feature development, I think most immediate thing
we should be working on is making app development easier. I'd like to
hear folks talk about what apps they're interested in building, and
where they've gotten stuck; that will help identify things we need to
work on to un-block those developers.

The cron API is something that I think concretely makes sense to work on
getting merged, as it's something that's partially done and many folks
have requested.

Nolan has talked about how getting outbound network access is
prohibitive for a number of his project ideas, and so I'd like to think
about how to make that easier.

At the same time, it has always hypothetically been the plan for
Sandstorm to block outbound network access from the client as well. I
think if we can expect this to ever actually happen we need to start
exploring it *soon*; if this requires breaking changes to apps, it's
going to get harder and harder as time goes on and more apps get built
with this implicitly available. I worry if we keep waiting we'll never
actually be able to achieve the isolated-by-default goal, because at
some point the breakage will have become completely untenable. As a
first step, I'd like to implement this and make it available to folks to
test; it would not be pushed to mainline. The goal of this is so that we
can get a sense of how much and what it's going to break. From there it
will be easier to figure out how to proceed.

On the internals side, one of my personal desires is improve the build
system, to make it more reproducible and easier for a new developer to
successfully actually build the thing and start hacking. It seems like
every time I get up the impetus to contribute to core, I get hung up on
some problem with the build system, and though I usually work out the
individual issues, it wastes time, and I can't help but think: if I'm
still consistently having this much trouble, both between a fairly
in-depth knowledge of how a Linux system is put together and how build
systems work, and having done this a bunch of times before, how much of
a barrier does this present to potential contributors? I'm exploring
nix[1] as a possible way to solve this problem "once and for all."

[1]: https://nixos.org/

Quoting Adam Bliss (2019-12-07 09:48:31)

> Here's an idea for a tractable milestone: is anyone else interested in
> moving the Sandstorm development community to be entirely self-hosted?
> I personally would love to see it move off of github and googlegroups.
> Whatever new features might be needed for such a migration would be a
> good set to focus on.
> --Adam� �
>

> On Fri, Dec 6, 2019, 20:00 Lyre Calliope <[1]ly...@securingchange.org>

> wrote:
>
> Def agree that restructuring is premature and rewriting is a pretty bad
> idea. (At least without something like 6 figure investment.)

> On Dec 6, 2019, 18:42 -0500, Jacob Weisz <[2]in...@jacobweisz.com>,

> wrote:
>
> ...for now we should just get Sandstorm itself moving again.�
>
> This isn�t your first time saying this in this thread. Pragmatism is
> definitely important for making things happen, and I�d like to find
> some clarity on what �just getting sandstorm moving again� entails.�
> Questions on my mind:

> * How do we� recognize if we�ve been successful?
> * Can this be articulated as a milestone of some kind?�
> * What scope of activity is appropriate to accomplish this, and what

> should be out of scope?

> * Once we have consensus around all this, how can we discuss things

> that are out of the scope of this milestone in productive ways?
>
> If we can find answers to the first two questions especially, I think
> we�ll be in a better position to evaluate what specific actions,
> commitments, and resources are needed to successfully revive this
> project.
> -Lyre
>
> --
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
> To unsubscribe from this group and stop receiving emails from it,

> send an email to [3]sandstorm-de...@googlegroups.com.

> To view this discussion on the web visit

> [4]https://groups.google.com/d/msgid/sandstorm-dev/8bd9c738-c93c-48a
> f-83de-736417947698%40Spark.

>
> --
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to [5]sandstorm-de...@googlegroups.com.

> To view this discussion on the web visit

> [6]https://groups.google.com/d/msgid/sandstorm-dev/CAEAeJWyc3ApMN99VQ7L
> -8hWr2xkDe6-bOWZ9K%3DwDF_mXkoPMNQ%40mail.gmail.com.
>
> Verweise
>
> 1. mailto:ly...@securingchange.org
> 2. mailto:in...@jacobweisz.com
> 3. mailto:sandstorm-de...@googlegroups.com
> 4. https://groups.google.com/d/msgid/sandstorm-dev/8bd9c738-c93c-48af-83de-736417947698%40Spark?utm_medium=email&utm_source=footer
> 5. mailto:sandstorm-de...@googlegroups.com
> 6. https://groups.google.com/d/msgid/sandstorm-dev/CAEAeJWyc3ApMN99VQ7L-8hWr2xkDe6-bOWZ9K%3DwDF_mXkoPMNQ%40mail.gmail.com?utm_medium=email&utm_source=footer

[Kevin Reid]

On Sat, Dec 7, 2019 at 12:21 PM Ian Denhardt <i...@zenhack.net> wrote:

In terms of core feature development, I think most immediate thing we should be working on is making app development easier. I'd like to hear folks talk about what apps they're interested in building, and where they've gotten stuck; that will help identify things we need to work on to un-block those developers.

A general-purpose debugging feature relevant to porting existing apps is broken: https://github.com/sandstorm-io/vagrant-spk/issues/213

"vagrant-spk enter-grain: Value too large for defined data type"

[Kenton Varda]

My thoughts:

- Definitely don't embark on any "rewrites" of Sandstorm core right now, they would be a huge time sink while making no actual progress on product goals. Yes, it means the tech debt continues to pile up, but you can't pay down your debt when you're broke.

- I think we should stick with GitHub for now. It keeps the barrier low for new contributors. I could be more easily convinced to replace Google Groups with a self-hosted solution, provided we can get good e-mail support working, which brings us to the next point...

- The biggest thing I'd personally love to see on Sandstorm is email support good enough to use as your primary email. There are some decent self-hostable mail UIs out there, but we need to do some serious plumbing work in Sandstorm itself to support e-mail ingress and egress in a good way. Can we even make it work without relying on external services like Sendgrid? How do we fight spam?

- I agree this seems like a good time to tighten up the sandbox (including client-side), provided we can simultaneously polish the tools people will need to request permissions properly. I think we should avoid breaking existing apps by coming up with some grandfathering scheme.

- I don't think we can realistically update all of the apps on the app market, there are just too many. Maybe we need to select a subset that we really want to focus on.

- Incidentally, I have been planning to rewrite the sandcats.io DNS and certificate issuance service on Cloudflare Workers sometime soon, with certificates from Let's Encrypt. We have used about 40% of the certificates we paid for from GlobalSign, so we're not in imminent danger of running out. But, I'd be a lot more comfortable relying on Let's Encrypt. Also, Cloudflare DNS would be *much* faster than our current single-homed DNS. And I can probably get away with spending some day-job hours on this, since it will be valuable for me to get some "dogfooding" experience.

-Kenton

On Sat, Dec 7, 2019 at 12:21 PM Ian Denhardt <i...@zenhack.net> wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/157575007503.910.11262999652743486559%40localhost.localdomain.

[Kenton Varda]

Oh, also:

Sandstorm does in fact have a Selenium-based test suite for testing Sandstorm itself, but it's not really intended for testing apps. On the topic of this test suite, it's pretty incomplete right now. Writing tests may not be the most exciting way to contribute, but it more tests means we can move faster with less fear that we might break people. It may also be something that's more accessible for less-experienced programmers.

-Kenton

[Ian Denhardt]

Quoting Kenton Varda (2019-12-07 17:38:00)

> My thoughts:
> - Definitely don't embark on any "rewrites" of Sandstorm core right
> now, they would be a huge time sink while making no actual progress on
> product goals. Yes, it means the tech debt continues to pile up, but
> you can't pay down your debt when you're broke.
> - I think we should stick with GitHub for now. It keeps the barrier low
> for new contributors. I could be more easily convinced to replace
> Google Groups with a self-hosted solution, provided we can get good
> e-mail� support working, which brings us to the next point...

+1

> Can we even make it work without relying on external services like
> Sendgrid?

The hardest part of this is that small email systems tend to have
deliverability problems. Even if you do everything else right, to be
able to reliably deliver email your server needs an actively good
reputation, and someone who's running a server just for themselves and
maybe a few others just doesn't send enough mail for this to happen.

I ran my own email server for about a decade, and recently decided to
shut it down and point my domain at Fastmail. My house's social mailing
list is using Mailgun now.

So while we could potentially make it really easy for people to set up
their own mail servers, I worry about this aspect of the experience, and
I don't know how to solve it without using an external service for
delivery.

But I definitely like the idea of getting to the point where you can at
least use Sandstorm for everything else.

> I think we should avoid breaking existing apps by coming up with some
> grandfathering scheme.

+1

-Ian

[r...@arts-betel.org]

Hi,

I seriously looked into Sandstorm a year ago for a new company I was starting, installed it, admired it, but decided against it. 

Why? I wasn't able to get a high level overview of the overall system. I couldn't figure out how authentication actually worked from the docs, how the apps were separated, and where all the user documents were stored. Also I could not find a guide on 'how to adapt an application to work in Sandstorm'. And why it did not natively support Docker was a big question mark.

Now you may say: there are docs, and this mailing list, and Docker support is actually there, but the crux of the matter is: it's either all rather opaque, hard to find, or incomprehensible. Even for someone like me who has been in the IT  industry for over 30 years, as a CTO and a CEO, who is deeply technical, and currently a college professor teaching Software Engineering. Not to blow my own horn, but to illustrate how the documentation is lacking. And I really tried.

Most open source projects that succeed, have a few things in common, like a benevolent dictator, corporate backing, timing, but good documentation seems to be a common denominator. And my personal preference would be a 'how do I convert an application to perfectly work in sandstorm' document.

Just my € 0,02

Ron Arts

Op vrijdag 29 november 2019 11:12:51 UTC+1 schreef Lyre Calliope:

I've been lurking since the Indiegogo campaign a few years ago, and I check in every once in awhile to see Sandstorm's status. I've been sad to see that it's largely stalled. A few months ago I was working on a tech security research project and discovered the object-capability model which has led me back here. 

I'd like to start conversation on bringing life back to the Sandstorm project.

I do a lot of work with small organizations and I can't tell you how often I've heard people independantly envision a platform like Sandstorm. Collaborative and security needs go beyond the scope of hosting project like Cloudron and Yunohost. As far as I’ve been able to find, there's nothing else on the open Internet like Sandstorm, and I think this is something the world needs.

While forking is always an option for continued development, I think a key part of what gives Sandstorm such potential is its' community. Looking at (relatively) recent conversations across the web I get the impression that there are plenty of true believers waiting to see if Sandstorm returns to being a platform they can confidently use in the long-term.

I understand that there are a number of challenges here such as funding and the availability of Kenton and others who have deep understanding of the underlying platform. I don't think these challenges are insurmountable.

I want to devote some of my time and resources to this. I have a background in community organizing within open source tech and would like to get some feedback on how I could help this project and community get back on its feet.

Cheers!

Lyre Calliope

[Jacob Weisz]

Fundamentally a key issue is that Sandstorm is intended to be a "new way of doing things", so moving existing apps into it tends to entail a bit of custom work. But making it easier to allow outside network access to be defined as Ian would like to do will help a lot in smoothing this over.

--

  Jacob Weisz

  goo...@jacobweisz.com

--

You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/40dcf046-384d-4585-9dd7-b0c3ee5fe1b5%40googlegroups.com.

[Jake W]

On Saturday, December 7, 2019 at 4:38:39 PM UTC-6, Kenton Varda wrote:

- I don't think we can realistically update all of the apps on the app market, there are just too many. Maybe we need to select a subset that we really want to focus on.

I think the focus should be on the core apps, and then apps that will significantly benefit from new features/improvements being worked on. TTRSS is basically my number one app, and I rely on an outside service poking it because of the lack of a cron API, so suffice to say, I feel it's one badly deserving an update once that feature is possible.

[Ian Denhardt]

Thanks for weighing in. I agree the docs could use a ton of improvement.
Some of my own thoughts in that regard:

Just *organizing* the existing docs would be a huge improvement; a lot
of the stuff people get tripped up in *is* actually there, but it seems
to be empirically the case that nobody can find anything. There are some
things that are in really weird places. Just one example: the docs for
building from source are buried in the administering section as install
option #4 of 7. That should probably go in a section for folks
interested in hacking on sandstorm itself, with maybe a cross reference
from the install docs.

There are also definitely things that are out of date (e.g. the FAQ
entry on let's encrypt hasn't been updated since before they added
wildcard support). Or just actually not clearly documented anywhere.

This could be a good low barrier of entry opportunity for folks in the
community to contribute. The whole docs site is just in the `docs/`
folder in the sandstorm repo.

I like the idea of having a high-level architecture document for the
system; maybe I'll sit down and do a first-pass brain dump soon.

As Jacob points out, the issues around porting apps extend beyond
documentation; fundamentally you'll never be able to have a push-button
porting process to convert apps that are written assuming they have
access to everything so that they'll work in a setting where they are
heavily sandboxed. But there's lots of room for improvement, both in
terms of documenting what's there now, and building better tools & APIs.

-Ian

Quoting r...@arts-betel.org (2019-12-03 06:20:50)

> --
> You received this message because you are subscribed to the Google
> Groups "Sandstorm Development" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to [1]sandstorm-de...@googlegroups.com.

> To view this discussion on the web visit

> [2]https://groups.google.com/d/msgid/sandstorm-dev/40dcf046-384d-4585-9
> dd7-b0c3ee5fe1b5%40googlegroups.com.
>
> Verweise
>
> 1. mailto:sandstorm-de...@googlegroups.com
> 2. https://groups.google.com/d/msgid/sandstorm-dev/40dcf046-384d-4585-9dd7-b0c3ee5fe1b5%40googlegroups.com?utm_medium=email&utm_source=footer

[Tim McCormack]

On Fri, 6 Dec 2019 16:56:02 -0500, Lyre Calliope wrote:
> It sounds like we have a good combo of knowledge and expertise here
> to get the engine running again. Now we need some fuel.

I'd be happy to put $500 or more towards Sandstorm continuing to exist
and adapt to changing circumstances. I don't have specific, burning
needs, but I would love to sponsor work on improving the Powerbox,
making apps easier to author and maintain, etc. Whatever is needed.

So yeah, I think setting up a simple governance structure would be
good—people who can set priorities and then connect funders to
developers. (I don't have specific experience with Open Collective, but
I guess that looks reasonable?)

I know the most important thing is code and expertise, but that
requires time, so instead I am offering the next best thing. :-)

- Tim McCormack

[Jake W]

Right now another person has added to the Cron API bounty, so that puts it at $400. I went ahead and committed to another $100 split between two other improvements we've discussed. (Disabling client side network access and improving the developer flow for the Powerbox.) I might raise them later but I don't know how fast we're going to be moving, so I don't want to overcommit myself. ;)

My impression is right now while we're figuring out what we can accomplish, the "best" way is to directly offer and send to people who are supporting Sandstorm, either by PR or by month (Patreon-style) or whatever. And if we're seeing enough code contribution and monetary offer on a regular basis to justify the more organized effort, we should assemble a more formal structure, primarily just to make it easy for people to "donate to Sandstorm". The 5% fee of Open Collective seems likely to be worth it to avoid needing a non-profit corporate entity to manage payment processing.

[Jake W]

Help me buy up all of Ian's time!

Ian's GitHub Sponsors is up! https://github.com/sponsors/zenhack

The highlight point here: Ian signed up before the cut-off for the GitHub matching fund, so the more money we send Ian to make Sandstorm better, the more Microsoft is sending Ian to make Sandstorm better (up to the max threshold, of course). There is of course a beautiful irony in Microsoft funding an open source productivity suite...