Regular Expressions?

[phil jones]

Quick question.

Are there regular expressions in NewSpeak? https://bracha.org/newspeak-modules.pdf suggests that there is a package, but searching in the system for Regex and RegularExpression don't seem to give me anything. Nor does anything in the String class look like its doing them.

cheers

Phil

[Gilad Bracha]

I've never used it, but try

https://github.com/newspeaklanguage/newspeak/blob/master/RegexMatcher.ns

--
You received this message because you are subscribed to the Google Groups "Newspeak Programming Language" group.
To unsubscribe from this group and stop receiving emails from it, send an email to newspeaklangua...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/newspeaklanguage/847590dc-fba9-4eb7-a603-17cb325ec48en%40googlegroups.com.

--

Cheers, Gilad

[phil jones]

Hmmm ...

so where is that? I see that file in the source-code I tried to build from, but I don't see a RegexMatcher in the online version of NewSpeak that I'm using.

If it isn't there, can I import it somehow?

cheers

Phil

[phil jones]

OK. So I tried copying that RegexMatcher code into a new top level class. And it's compiled.

And I'm now trying to test it from the Workspace.

I've got as far as trying this :

[ | re ::= RegexMatcher new RxParser new.  | re parse: 'abc'] value

Which I believe should make me a new parser and create a parse-tree from 'abc'.

The problem is that RegexMatcher itself only has a primary factory method that takes a platform.

class RegexMatcher usingPlatform: platform = (

And in the workspace I get a "MessageNotUnderstood: RegexMatcher class new" error.

So how do I get an appropriate platform object in the workspace to pass to the RegexMatcher?

[Gilad Bracha]

Yes. From the home screen, go to Newspeak Source. The menu at right has an option 'Compile Files'. You can choose files via an OS dialog. Choose the Regex file.It will load it into the system.

To view this discussion on the web visit https://groups.google.com/d/msgid/newspeaklanguage/16812141-a574-4a78-a75a-4e36fb1983cbn%40googlegroups.com.

--

Cheers, Gilad

[Gilad Bracha]

Ok, so you got the RegexMatcher class in the hard way - no need to do that.

Now you create an instance via

RegexMatcher usingPlatform: platform

(platform is made available in the workspace scope).

Then go from there. Unclear if it  works - it may need to be adjusted as the APIs on the web differ from the ones we had when the regex work was done.

To view this discussion on the web visit https://groups.google.com/d/msgid/newspeaklanguage/0f0d43c3-6020-4187-8c65-09a8e3bced3fn%40googlegroups.com.

--

Cheers, Gilad

[phil jones]

OK. So "platform" is a variable already given in the workspace?

Is there a way to list what variables are currently set there?

So I tried this :

[ | re ::= RegexMatcher  usingPlatform: platform RxParser new.  | re parse: 'abc'] value

And I got this error :

MessageNotUnderstood: PlatformWithHopscotch RxParser

So I then tried this :

[ | re ::= (RegexMatcher  usingPlatform: platform) RxParser new.  | re parse: 'abc'] value

And I get this.

MessageNotUnderstood: PlatformWithHopscotch squeak

Any ideas what's happening here? I understand I need to make an instance of RegexMatcher in order to ask it for the RxParser factory. But I'm guessing I'm not successfully doing that ...

why not?

cheers

Phil

[Gilad Bracha]

On Wed, Apr 28, 2021 at 9:55 AM phil jones <inte...@gmail.com> wrote:

OK. So "platform" is a variable already given in the workspace?

platform is a message supported by the workspace. 

 

Is there a way to list what variables are currently set there?

Yes: type platform and evaluate it (make sure you select 'platform' or type it on a new line, so you don't end up evaluating other stuff in the workspace). Then click on the link, which iwll take you to an object presenter (aka object inspector) on the platform object. 

So I tried this :

[ | re ::= RegexMatcher  usingPlatform: platform RxParser new.  | re parse: 'abc'] value

And I got this error :

MessageNotUnderstood: PlatformWithHopscotch RxParser

Of course. Syntax is very similar to Smalltalk  - this would not work in Smalltalk either.

So I then tried this :

[ | re ::= (RegexMatcher  usingPlatform: platform) RxParser new.  | re parse: 'abc'] value

And I get this.

MessageNotUnderstood: PlatformWithHopscotch squeak

Any ideas what's happening here?

Yes. Somewhere inside the RegexMatcher class, there is a dependency on  Squeak specific stuff. We isolated that dependency by making Squeak features available through a gateway object, accessed via the message #squeak, which the Squeak implementation of platform supported. The WASM version obviously does not.

This is a worrying thing though. It means the RegexMatcher won't work out of the box. It needs some work to port it.

An alternative might be to use whatever regex libs JS has (I have no idea what those are).

Also, as an aside. The MessageNotUnderstood notice is a link to the debugger. That might help you too.

To view this discussion on the web visit https://groups.google.com/d/msgid/newspeaklanguage/d4758424-2085-4023-b5c4-52b192776f53n%40googlegroups.com.

--

Cheers, Gilad

[phil jones]

OK. I see that.

So a lot of this squeak stuff it's getting seems like the generic collections etc.

    private Association = platform squeak Association.
    private Map = platform squeak Dictionary.
    private List = platform squeak OrderedCollection.
    private Set = platform squeak Set.
    private WriteStream = platform squeak WriteStream.
    private ReadStream = platform squeak ReadStream.
    private Transcript = platform squeak Transcript.
    private MessageNotUnderstood = platform squeak MessageNotUnderstood.

Presumably the web / hopscotch version of NewSpeak has equivalents for all those that they could be replaced with?

I'm guessing the "platform" therefore is a wrapper for a lot of lower level infrastructure managing resources.

And maybe this is a stupid question, but looking around I see a fair few classes that seem to do something similar.

You pass the platform to the factory method / constructor and then there seems to be a lot of boilerplate like this  :

    (* imports *)
    private Map = p collections Map.
    private Presenter = p hopscotch Presenter.
    private Subject = p hopscotch Subject.

etc.

Why do you not just store the Platform object itself in a slot, and only ask it for these other objects when you need them? 

The other question is that in all these factories, the Platform gets explicitly asked with "squeak" or "hopscotch" etc. messages to get these UI elements like Presenter which means the code couples tightly to the UI. Why isn't Platform an abstraction that hides all that? So you can just say something like "p getPresenter"  and have the platform itself know whether it's really  running on Squeak and should give you a Squeak Presenter, or on a web wrapper for Hopscotch that should give you a Hopscotch Presenter, etc?

cheers

Phil

[phil jones]

So I thought I'd try changing the RegexMatcher class like this

  private Association = (Collections usingPlatform: platform) Association.

(Ie. to try to pull the Association class out of the Collections top level class rather than pull it out of the platform argument. I see that Collections does have an Association class defined in it.)

But I'm getting an error : MessageNotUnderstood: RegexMatcher Collections

So presumably you can't see the Collections top-level class from inside the RegexMatcher top-level class? Is this correct? Even top-level classes are invisible to each other?

In which case how do you ever get access to them? Presumably this is why you pass "Platform" objects around? Because they are your only access to the rest of the system?

So I tried changing the line like this :

    private Association = platform collections Association.

Ie. assuming I could pull Association out of whatever collections the platform has access to.

But I get this error :

MessageNotUnderstood: CollectionsForPrimordialSoup Association

Suggesting that the platform collections method returns CollectionsForPrimordialSoup rather than Collections. And that that doesn't have an Association class. (It doesn't.)

So what am I meant to do here? If I just want an Association in my RegexMatcher world, do I have to get it from the platform object I've been passed? Or is there another way to get it?

And if I do have to get it from the platform, what do I do if the particular bag of Collections I'm passed, doesn't have it? Is CollectionsForPrimordialSoup really meant to have everything that Collections has? Or is it that the platform object "collections" message should really return the Collections top-level object?

cheers

Phil

[Gilad Bracha]

On Wed, Apr 28, 2021 at 11:36 AM phil jones <inte...@gmail.com> wrote:

OK. I see that.

So a lot of this squeak stuff it's getting seems like the generic collections etc.

    private Association = platform squeak Association.
    private Map = platform squeak Dictionary.
    private List = platform squeak OrderedCollection.
    private Set = platform squeak Set.
    private WriteStream = platform squeak WriteStream.
    private ReadStream = platform squeak ReadStream.
    private Transcript = platform squeak Transcript.
    private MessageNotUnderstood = platform squeak MessageNotUnderstood. 

Presumably the web / hopscotch version of NewSpeak has equivalents for all those that they could be replaced with?

More or less, yes.  But this clearly old code, and there are doubtless API differences. The use of squeak indicates a reliance on those API differences.

I'm guessing the "platform" therefore is a wrapper for a lot of lower level infrastructure managing resources.

That's one way to look at it . Basically, it is the platform API.

And maybe this is a stupid question, but looking around I see a fair few classes that seem to do something similar.

You pass the platform to the factory method / constructor and then there seems to be a lot of boilerplate like this  :

    (* imports *)
    private Map = p collections Map.
    private Presenter = p hopscotch Presenter.
    private Subject = p hopscotch Subject.

etc.

Why do you not just store the Platform object itself in a slot, and only ask it for these other objects when you need them? 

You could do that, but that would be bad coding style.  The code above acts (as the comment says) as a series of imports of classes that the module depends on.  If you store the individual modules as above then you (and the system) know, looking at the factory, exactly what the module's dependencies are. If you store the entire platform, you don't know what you're using; it's like a Java program (or a Smalltalk program, for that matter).

Newspeak is largely about modularity; this is a big part of that story.

Other languages have learned this the hard way. In Java for example, they no longer recommend using wildcard imports, for similar reasons.  

The other question is that in all these factories, the Platform gets explicitly asked with "squeak" or "hopscotch" etc. messages to get these UI elements like Presenter which means the code couples tightly to the UI.

TL; DR  Not really.

First, the call to squeak is very unusual; it's only intended to be used to access platform-specific code - something that other platforms don't support.  So normally you would not be asking Squeak for it's Set class. It's done here, presumably because the code would not work with Newpeak's Set class (it was ported from Squeak after all) and so it demands that class specifically. 

The use #hopscotch, on the other hand, is standard. It is the object representing the UI library, and so you get it, and then get specific classes you need from it.

You cannot reliably flatten the namespace, as different subsystems might have methods with the same name.

 

Why isn't Platform an abstraction that hides all that?

It is. As long as the APIs are the same. But it does not lock you in - that's why you have the option of accessing platform-specific stuff, by going through an accessor method. On the WASM system, that accessor is #js, and it gives you access to Javascript.

 

So you can just say something like "p getPresenter"  and have the platform itself know whether it's really  running on Squeak and should give you a Squeak Presenter, or on a web wrapper for Hopscotch that should give you a Hopscotch Presenter, etc?

Squeak doesn't even have a Presenter.  The UI's APIs are totally different.  

To view this discussion on the web visit https://groups.google.com/d/msgid/newspeaklanguage/8e073686-b4e1-4c08-b964-9b295ed8066fn%40googlegroups.com.

--

Cheers, Gilad

[Gilad Bracha]

On Wed, Apr 28, 2021 at 12:18 PM phil jones <inte...@gmail.com> wrote:

So I thought I'd try changing the RegexMatcher class like this

  private Association = (Collections usingPlatform: platform) Association.

(Ie. to try to pull the Association class out of the Collections top level class rather than pull it out of the platform argument. I see that Collections does have an Association class defined in it.)

But I'm getting an error : MessageNotUnderstood: RegexMatcher Collections

So presumably you can't see the Collections top-level class from inside the RegexMatcher top-level class? Is this correct? Even top-level classes are invisible to each other?

Exactly. Newspeak has fundamental design ideas, that make it very different from Smalltalk.

a. There is no global namespace. 

b.  Everything is a message send. 

I think you really should watch Vassili's talk. It's an hour, but it focuses on how Newspeak differs from Smalltalk.

In which case how do you ever get access to them? Presumably this is why you pass "Platform" objects around? Because they are your only access to the rest of the system?

Roughly speaking, yes. "Only" isn't strictly accurate, but close.

A top level class is a module. It has no access to anything except a few basic classes inherited form Object, and whatever is passed into its factory (it need not be platform). The arguments to the factory are capabilities, in the sense of the object-capability security model (ocap). 

Now to actually build things, you use the IDE as a namespace to collect the various pieces you need. 

 

So I tried changing the line like this :

    private Association = platform collections Association.

Ie. assuming I could pull Association out of whatever collections the platform has access to.

But I get this error :

MessageNotUnderstood: CollectionsForPrimordialSoup Association

Suggesting that the platform collections method returns CollectionsForPrimordialSoup rather than Collections.

Yes

 

And that that doesn't have an Association class. (It doesn't.)

So what am I meant to do here? If I just want an Association in my RegexMatcher world, do I have to get it from the platform object I've been passed? Or is there another way to get it?

There is, though it isn't clear that is what you want to do. RegexMatcher is clearly a port of a Smalltalk program, an dthe port was designed to work on Newspeak on Squeak, in the version that existed years ago. A proper port to the current version may well choose not to use Association.

Now, if it made sense to use Association, you could decide to add a new parameter to the RegexMatches factory, and pass in Association. You could get it form the old Collections module.

And if I do have to get it from the platform, what do I do if the particular bag of Collections I'm passed, doesn't have it?

As I said, if you want other classes, you can get them in the IDE and pass them as arguments.

 

Is CollectionsForPrimordialSoup really meant to have everything that Collections has?

That's an open question. It was pruned down mercilessly by Ryan. He probably went too far; I've already added some stuff.

The system is very much in a pre-alpha state right now, and APIs will evolve. 

Or is it that the platform object "collections" message should really return the Collections top-level object?

No. 

I hope you aren't getting frustrated.  The WASM system isn't ready for prime time, and is very different from Smalltalk. It's designed to be modular, secure and live all at the same time. We're going to have to build it out over time.

The good news is you are learning rapidly. What doesn't kill you makes you stronger :-)

I'm guessing this is your preferred cognitive style - experiment a lot, make many mistakes and learn from them. Some people might prefer to study the system in a more Aristotelian manner, because much of what you've discovered is explained in the various resources we sent you. 

To view this discussion on the web visit https://groups.google.com/d/msgid/newspeaklanguage/0811560f-d72f-4b16-b70c-518b02942a7fn%40googlegroups.com.

--

Cheers, Gilad

[phil jones]

Frustrated? Boy. You guys are crazy extremists, aren't you? :-)

I should have taken your name more seriously! (And that point about you starting Newspeak by deleting Smalltalk's arrays.)

Yeah. I do understand what you're saying and the logic behind it. But I gotta say, it does feel like if I'm going to pick up a new highish level language to do something with, I shouldn't be having to start out by trying to figure out how to inject basic collections like lists and dictionaries into my code by threading them as arguments through my constructors.

Shouldn't these at least be part of the "few basic classes inherited from Object" that we can just rely on? Like a stdlib or prelude? I mean, the irony here is that, yes, I know this isn't Smalltalk. But AFAICT you are in the Smalltalk tradition, which is about empowering programmers and giving them a nice big library of toys to get going with. And AFAICT you DO have things like Associations and Lists to use off-the-shelf in your nice big library.

It's just that you've got a security model that's so hardcore that you make us jump through hoops to actually get access to them. In about 3 mouse clicks in my first session, I can drill into, read and even change the source code of an Association class. But to actually get access to a scope containing one in order to use it, I have to spend hours getting my head around a strange and complex security model and do reasoning about  - I know you don't like the term but ... - "dependency injection". What's the "right way" to get access to this namespace in that part of the code? Etc.

That seems inconsistent.

But look, I'm not easily defeated. There are lots of things that look really good about Newspeak. I'm going to persevere. And yeah, obviously I like learning by trying to DO something and see if I can make it work. But I would say that if you care about having users and people adopting this language then having such a hard security model between people starting out, and being able to do anything serious, seems a big barrier to entry that you might want to think about.

Actually ... on further consideration ... I have more ... ;-)

AFAICT this isn't about still being in alpha. It seems to me that this security model will always get in the way of programmers.

If I want to use any UI component in my own project I have to know how to get to it. Can I get it through the "platform" object? Well, who is responsible for making sure it's in the platform object in the first place? Suppose one third party developer creates a new UI widget in a new module. And I as another third party developer want to use it? I have to either wait for the core maintainers to put that third party widget into the default platform object? Or I have to add it as an explicit extra argument to my factory / constructor? Which pushes the responsibility onto the users of my code to add it every time they call my factory?

Am I understanding this correctly? There's no way that one third party developer can include another third party developer's module as a dependency without either getting the Newspeak maintainers to include it in the platform object? Or forcing the end user to explicitly think about it?

And if I write software that depends on 20 or 30 third party libraries, I'll have to write a factory with 30 specific arguments? Which the user will have to explicitly pass to me when they use my code? And if I update my code to use a 31st library. I'll have to update my factory with a 31st argument, and break the code of everyone who currently uses it? 

I mean ....  ¯\_(ツ)_/¯

It can't really be like that ... can it?

Phil

[Gilad Bracha]

On Wed, Apr 28, 2021 at 3:34 PM phil jones <inte...@gmail.com> wrote:

Frustrated? Boy. You guys are crazy extremists, aren't you? :-)

I should have taken your name more seriously!

Not sure what my name tells you, and even less sure I want to know :-).

 

Yeah. I do understand what you're saying and the logic behind it. But I gotta say, it does feel like if I'm going to pick up a new highish level language to do something with, I shouldn't be having to start out by trying to figure out how to inject basic collections like lists and dictionaries into my code by threading them as arguments through my constructors.

A few comments:

Your point is well taken, in that there is a real tension between solid modularity and security and ease of use, and bridging them is a challenge.

Tooling can help. A lot. The Squeak tools would highlight messages that are out of scope. We have always wanted to extend that so that they also offer to write a boilerplate import for you.  So you'd type List in your code, and it would be highlighted in red because it's not the lexical scope, and you'd hover and get a tool tip asking if you want to import it, and presto, it would add the import, and ask you to confirm. That's just one example. Another is supporting a workflow that starts with a workspace and allows you to add code there and then transform it into a new class.

Another point is that you started out trying to port a Regex library to a new platform. Not exactly 'Hello World" is it? Normally, one would probably start with evaluating basic stuff in a workspace where you'll find that all modules loaded in the IDE are accessible. Clicking the "?" icon for help is also a good idea.

Shouldn't these at least be part of the "few basic classes inherited from Object" that we can just rely on? Like a stdlib or prelude? I mean, the irony here is that, yes, I know this isn't Smalltalk. But AFAICT you are in the Smalltalk tradition, which is about empowering programmers and giving them a nice big library of toys to get going with. And AFAICT you DO have things like Associations and Lists to use off-the-shelf in your nice big library.

Our library is not big, and it is in transition. A large library is a double edged sword by the way. But yes, it is largely about empowering programmers. 

It's just that you've got a security model

It's actually a modularity model. Security is one of its implications. Things like software engineering, deployment and distribution are others.

 

that's so hardcore that you make us jump through hoops to actually get access to them. In about 3 mouse clicks in my first session, I can drill into, read and even change the source code of an Association class. But to actually get access to a scope containing one in order to use it, I have to spend hours getting my head around a strange and complex security model

It certainly isn't complex; and strange, while pejorative, is the eye of the beholder. It is actually a perfect fit with objects.

 

and do reasoning about  - I know you don't like the term but ... - "dependency injection". What's the "right way" to get access to this namespace in that part of the code? Etc.

I'll quote Alan Perlis: A language that doesn't affect the way you think about programming, is not worth knowing. I really think that taking a bit of time (like an hour to listen to Vassili's talk) would be worth your time. It would have saved you several hours of headbanging. It's still a good idea.

But look, I'm not easily defeated. There are lots of things that look really good about Newspeak. I'm going to persevere. And yeah, obviously I like learning by trying to DO something and see if I can make it work. But I would say that if you care about having users and people adopting this language then having such a hard security model between people starting out, and being able to do anything serious, seems a big barrier to entry that you might want to think about.

You may be surprised, but I've thought about it before ... 

Actually ... on further consideration ... I have more ... ;-)

AFAICT this isn't about still being in alpha. It seems to me that this security model will always get in the way of programmers.

As I said, there is a tension, and some of it will never go away. Minimizing that tension is a matter of design, polish and evolution of the tools.

 

It's also true that it does more than get in the way of programmers. It gets many things out of their way (like how do I deploy my code; how do I solve those awful bugs around static state; how do build a program that is't a hazard), but it is true that all those things involve delayed gratification.

If I want to use any UI component in my own project I have to know how to get to it. Can I get it through the "platform" object? Well, who is responsible for making sure it's in the platform object in the first place? Suppose one third party developer creates a new UI widget in a new module. And I as another third party developer want to use it? I have to either wait for the core maintainers to put that third party widget into the default platform object? Or I have to add it as an explicit extra argument to my factory / constructor? Which pushes the responsibility onto the users of my code to add it every time they call my factory?

Am I understanding this correctly?

No. You'd have to use a technique similar to how one builds applications, but for libraries.

There are two related sets of issues here.

First, Somehow, all the dependencies of an artifact have to be made available by getting them to the "build site".

This is commonly addressed (solved is too strong a word) by package managers. That much is true in Smalltalk as well. In Smalltalk you put these in your image, which provides a global namespace; in most other systems you put them in the file system, which again, is a global namespace.  In Newspeak, you put them in the IDE, which as a global namespace as well. I am hoping to avoid the plague of package managers; the IDE should be able to do this based on some metadata conventions (the details of which the margin of this email is too narrow to contain; also they don't yet exist, but I have enough of an idea that I am  not worried).

Next, there is the question of how you put together your program given the global namespace. This is the domain of build scripts in traditional software. In Newspeak, you can write these scripts in Newspeak itself.  These would take a namespace object as a parameter (what we often call a manifest). You'd typically pass in the IDE top level namespace (try evaluating ide namespacing manifest in a workspace). Of course, you can always override the behavior of a given manifest by wrapping it.

So you write class with a #packageLibraryUsing: method that takes a manifest and instantiates your library as you wish. The manifest needs have all the code you need. Importantly, the manifest is still under 'end user control' and should contain only top level classes (we can also enforce that) so no state or access to the outside world is provided. Thus, the #packageLibraryUsing: methods are like build scripts, and they can call other #packageLibraryUsing: methods, just like build scripts or makefiles refer to others. The difference being that none of this is hardwired to a specific global namespace.

As I said, this is very similar to how one builds apps.  There are no examples of this pattern, because we have no third party libraries yet.

 

There's no way that one third party developer can include another third party developer's module as a dependency without either getting the Newspeak maintainers to include it in the platform object? Or forcing the end user to explicitly think about it?

See above.

And if I write software that depends on 20 or 30 third party libraries, I'll have to write a factory with 30 specific arguments?

No, you can aggregate them into an object (a manifest or submanifest, or something else). Also note that in many cases, you will write factories that take actula module instances rather than classes as arguments.

 

Which the user will have to explicitly pass to me when they use my code? And if I update my code to use a 31st library. I'll have to update my factory with a 31st argument, and break the code of everyone who currently uses it? 

I mean ....  ¯\_(ツ)_/¯

It can't really be like that ... can it?

No it isn't. But thanks for asking.

--

Cheers, Gilad

[phil jones]

Hi Gilad,

Thanks for having the patience to respond to my ranting ...   :-)

> Not sure what my name tells you, and even less sure I want to know :-).

Sorry. That wasn't about your personal name. I was talking about the whole Orwellian nomenclature of the Newspeak project and the totalitarian vibe it brings

> but it is true that all those things involve delayed gratification.

Meh! ... that's what people people tried to tell me about Java. "Ah but in the long run all this is for your own good"  :-/

> . I really think that taking a bit of time (like an hour to listen to Vassili's talk) would be worth your time. It would have saved you several hours of headbanging.

Yes. You are right. That was useful. I'd actually watched the first 25 minutes or so a few days ago, but paused it just before it got to the bit that's relevant to us now. Which obviously would have given me a better understanding. Though this discussion here has been very helpful, and I think he's still rather happy in talking about the "magic" of the platform object just having the things you need.

Maybe if I hadn't immediately bumped into this issue of trying to do something where "platform collections" gave me CollectionsForPrimordialSoup, a package of collections that was NOT the full package of collections that I would expect, while I could blatantly see, right next to it, in the source browser, the Collections package that WAS full of all the things I would expect, then I might have remained more trusting in the magic a bit longer, and not been so concerned about how to go out and get things from other packages myself.

Even so ... I'm still to be convinced that this kind of magic can reliably work. As the number of potential libraries and packages and applications increases, how will core maintainers keep up with ensuring that all the things you might want to use, are handily available via this platform object?

But anyway, if I understand you now, there is an alternative.

A manifest (or more specifically, that thing you get from ide namespacing manifest ) is the "global namespace" I was looking for? (I was interpreting you saying there was no global namespace as meaning that there wasn't something like this and that we shouldn't have one.) But actually this manifest has references to all the top level classes installed in the system. So if I needed in my own top-level class, to get at, say,  Collections, I could ask the manifest for it? 

But you don't want to call this a "global namespace" because it's just an object, and something that could potentially be replaced by an alternative proxy or stub or sandbox etc? I know you say you don't like "dependency injection". But as far as I can tell, these are all approaches to the same problem. How do you simultaneously make your code sensitive, to or parameterized by, a context in which it's going to run, while also making it self-contained and not hard-wiring in dependencies on that specific context. In Newspeak, the solution is that that context comes in wrapped in the form of a manifest?

But I didn't really get what #packageLibraryUsing:  is meant to do. (I was just trying to search for an example in the IDE, but couldn't find one). Is that just the way to inject this manifest into a package? Or is it some kind of callback or hook you write in your package which is used to register it with the IDE in some way, to make it visible from within other packages?

Are you saying that when I create a module, I actively register it with or put it into a manifest? Much as I'd bundle up a library to put on npm or maven etc?

> There are no examples of this pattern, because we have no third party libraries yet.

Well, if I write some code I guess it would be one. I have an interest in grokking this.

re: 30 parameters.

> No, you can aggregate them into an object (a manifest or submanifest, or something else)

Sure. But the crucial question is not so much how you write them but how explicit they have to be. My thinking was that normally (Smalltalk, Java, Python etc.), when class X uses (depends on) class Y which uses (depends on) class Z1, then the author of class Y is free to change the implementation to use class Z2 instead of Z1 without class X having to know or be involved. But the need to pass access to the Z2 namespace explicitly from X into Y creates a 2-way dependency where Y depends on X, and if you wanted to change Y, you'd also have to change X. It wouldn't have mattered whether it was a method selector with 30 arguments or a single argument that was a dictionary of 30 objects, the fact of that dependency and X having to know what those 30 namespaces were, would be the same.

However, as I understand it now, X just passes to Y a general manifest assembled elsewhere, up in the IDE, so it doesn't have to know anything specific about what namespaces Y is using.

Is that a fair assessment of the situation?

In which case, though, what's the real difference between, say, the "platform" object and a manifest? Don't they both do the same thing : provide a bundle of access to other resources available within the system? Or is platform actually a subclass of, or instance of manifest? The webIde object that's passed to, say, WorkspaceManager seems to be another example of the same thing. Why isn't webIde  just another thing you pull out of platform, the way hopscotch is?

cheers

Phil

[Gilad Bracha]

Detailed answers later in the day. You're still very hazy on some stuff, I can see.

Key points:

The Newspeak *language* has no global namespace.

The Newspeak IDE provides a global namespace, which is of course a real object that you can pass around.  That object is a manifest.

Manifests and platforms are totally different. Manifests are a development thing. Platforms are a deployment thing.

The platform is something that gives you the only connections to the world outside Newspeak. 

I understand you need something more concrete/detailed to help you with your withdrawal symptoms from the Smalltalk global namespace. Alas, I have other things I must attend to in life, so that will wait until later today.

--

You received this message because you are subscribed to the Google Groups "Newspeak Programming Language" group.
To unsubscribe from this group and stop receiving emails from it, send an email to newspeaklangua...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/newspeaklanguage/b1d43056-3ada-4f33-bd4e-b694fa0b4fd6n%40googlegroups.com.

--

Cheers, Gilad

[phil jones]

No problem  ... take your time. I'm sure you have many better things to do than just answer my questions.

(Meanwhile I have to get back to writing boring Java ...  for me, this is all a utopian displacement activity from that ;-)

But one quick extra thought in response to re-reading this :

> We have always wanted to extend that so that they also offer to write a boilerplate import for you.  So you'd type List in your code, and it would be highlighted in red because it's not the lexical scope, and you'd hover and get a tool tip asking if you want to import it, and presto, it would add the import, and ask you to confirm.

Although I'm reiterating my main point, perhaps this states it more clearly. The issue I see is NOT that if I want to add a new dependency (say List to my class Y), that it's hard for me to add an extra parameter to my Y factory. Sure, tooling can write that boilerplate for me, just as Android Studio does. But doing that by hand is not a big issue.

The BIG issue is that if to "import" List into my class, I have to do it through an extra argument to the factory, I therefore have to change the signature of the factory. And then I'm going to break all the existing code that's already using that factory.  Isn't that the case? And isn't that a massive issue?

Again, maybe I'm misunderstanding, but I'm looking further at various documents, and videos and code examples, and still haven't seen anything inconsistent with that interpretation. In general, when I add a new dependency to a class, I'm expected to pass it as an argument to the factory. This is very different from Java and Python etc. where the import mechanism for namespaces into classes is orthogonal to the mechanism for passing arguments to a constructor for objects, and so adding new imports doesn't affect the signature of the constructor. In Java and Python adding a new dependency to a library is merely the concern of the authors of that library, and doesn't concern the users of that library. In Newspeak it requires an update of the factory signature and therefore leaks out to affect the users.

I've been reading https://bracha.org/newspeak-modules.pdf and I see WHY you're doing things this way and the virtues you want from that. BUT that brittleness seems a very high price to pay.

Or ... maybe what I should understand from that paper (section 2.3) is that you don't intend for people people to publish "libraries" at all. Only full "Applications" which are basically immutable black boxes which users only access through their "main" function. Is that the model you're thinking of?

Phil

[Gilad Bracha]

Ok, the full answer will still wait until evening, but I have a few minutes.

On Thu, Apr 29, 2021 at 9:52 AM phil jones <inte...@gmail.com> wrote:

The BIG issue is that if to "import" List into my class, I have to do it through an extra argument to the factory, I therefore have to change the signature of the factory. And then I'm going to break all the existing code that's already using that factory.  Isn't that the case? And isn't that a massive issue?

I understood your point the first time. It would be a massive issue, if it were true. Put your mind at rest. There are various patterns to get around this.

 In Java and Python adding a new dependency to a library is merely the concern of the authors of that library, and doesn't concern the users of that library. 

I have some understanding of how Java works. 

I've been reading https://bracha.org/newspeak-modules.pdf and I see WHY you're doing things this way and the virtues you want from that. BUT that brittleness seems a very high price to pay.

Or ... maybe what I should understand from that paper (section 2.3) is that you don't intend for people people to publish "libraries" at all. Only full "Applications" which are basically immutable black boxes which users only access through their "main" function. Is that the model you're thinking of?

No.  But the pattern would be very similar. Imagine a convention whereby every library intended for distribution is sent out as a class with has a factory (or in general, class method) method #packageLibraryUsingManifest: . Each such class has a #build: method, that, given a platform object, produces a working instance of the module we actually want to distribute. 

Now developer A (Alice) intends to distribute a module MyMod1.

It depends on some other code she developed, say, MyMod2, which in turn depends on a 3rd party library from developer B (Bob).

The module Alice distributes is below.

class MyModules packageLibraryUsingManifest: manifest = ( 

 (* packageManager: ... metadata describing the expected dependencies *)

  |

  MyMod1 = manifest MyMod1.

  MyMod 2= manifest MyMod2.

  My3rdPartyDep = manifest My3rdPartyDep packageUsingManifest: manifest.

   |

) (

   public build: platform = (

      |

      my3rdPartDependency = My3rdPartyDep buildFromPlatform: platform.

      myMod2 = MyMod2 usingPlatform: platform and: my3rdPartDependency.

      myMod1 = MyMod1 usingPlatform: platform mod2: myMod2.

      |

     ^myMod1

   )

)

The #build: method encapsulates the knowledge of how to build Alice's code, using an internal library she wrote (MYMod2) and Bob's library.  

Note that Alice is using the same convention as Bob, and builds Bob's code with no knowledge of its internal dependencies.

Developer C (Carol) uses these same conventions to build Alice's code. She can do so regardless of whether she is building an app

(where she'd call #packageLibraryUsing from the app's factory, and #build: from #main:args:) or another library ((where she'd call #packageLibraryUsing from the library factory).

If Alice decides to replace Bob's code with code from developer D (David), she changes MyModules, but Carol's code does not change. Likewise, if Bob or David change their dependencies, neither Alice nor Carol change their code.

It isn't necessary for everyone to follow the exact same convention - what's critical is that a given module maintains its convention so its build API is stable. Of course, a common convention is good, especially for tools. 

Alice could just distribute an instance of MyModules, but this hardwires the versions of all the dependencies.  Assuming she doesn't do that, it is true that Carol needs to download all the pieces and their sub-pieces from Bob and Alice etc. She loads them into the IDE (or the IDE does so by reading the metadata) and the IDE's namespace is used to produce the manifest object passed in when anyone builds an app.

Somehow, I'm sure you'll have more questions. Note that platform and manifest need to be distinct. Platforms are for runtime capabilities and are security critical. Manifests are for code construction. 

Anyway, I think this answers the question. You are right that we haven't used this pattern, and I have not ever written it up. I've discussed some variations with people over the years, with people who come up with scenarios of the "how will this ever work" variety.

I have yet to encounter a scenario that Newspeak modularity doesn't handle. I'm not aware of any other language that has that property.

Thanks for forcing me to write it up. I have an ambition to write a live Newspeak book (in Newspeak system of course) and you've just made me start writing pieces of it in email :-).

--

Cheers, Gilad

[Gilad Bracha]

Oh, there were typos in the code, as I switched convention mid-way; fixed below:

class MyModules packageLibraryUsingManifest: manifest = ( 

 (* packageManager: ... metadata describing the expected dependencies *)

  |

  MyMod1 = manifest MyMod1.

  MyMod 2= manifest MyMod2.

  My3rdPartyDep = manifest My3rdPartyDep packageLibraryUsingManifest: manifest.

   |

) (

   public build: platform = (

      |

      my3rdPartDependency = My3rdPartyDep build: platform.

      myMod2 = MyMod2 usingPlatform: platform and: my3rdPartDependency.

      myMod1 = MyMod1 usingPlatform: platform mod2: myMod2.

      |

     ^myMod1

   )

)

--

Cheers, Gilad

[phil jones]

OK. Fair enough.

I won't remember that pattern or be able to reinvent it off the top of my head, but I can see how it does its thing.

Obviously I look forward to the full write up or book-in-progress, but consider my mind at rest that there's a solution.

One thing that would be interesting would be to see the full worked out example for a very toy MyMod1 class and the appropriate MyModules for it with examples of the code that calls the build and the code which uses the library package. I presume even though you aren't doing any of this in the codebase today, this pattern could start being used today. There's nothing extra it needs in the language?

Also, I wonder, as this looks quite boilerplate-y, whether it would be possible to code-generate MyModules for MyMod1 based on just the list of dependencies. So that you could have a "Librify This" button in the IDE that could turn any top-level class into a library.

cheers

Phil

[Gilad Bracha]

On Thu, Apr 29, 2021 at 5:51 PM phil jones <inte...@gmail.com> wrote:

OK. Fair enough.

I won't remember that pattern or be able to reinvent it off the top of my head,

No problem - that's what I'm here for.

 

but I can see how it does its thing.

Obviously I look forward to the full write up or book-in-progress, but consider my mind at rest that there's a solution.

One thing that would be interesting would be to see the full worked out example for a very toy MyMod1 class and the appropriate MyModules for it with examples of the code that calls the build and the code which uses the library package. I presume even though you aren't doing any of this in the codebase today, this pattern could start being used today. There's nothing extra it needs in the language?

No, this could be run today.  I'll see if I can work it in somewhere in an example or tutorial. 

Also, I wonder, as this looks quite boilerplate-y, whether it would be possible to code-generate MyModules for MyMod1 based on just the list of dependencies. So that you could have a "Librify This" button in the IDE that could turn any top-level class into a library.

Yes. The handling of apps is very similar, and automating that is also a longstanding goal.  

--

You received this message because you are subscribed to the Google Groups "Newspeak Programming Language" group.
To unsubscribe from this group and stop receiving emails from it, send an email to newspeaklangua...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/newspeaklanguage/8b577758-da6c-487a-87ef-4055636dcee6n%40googlegroups.com.

--

Cheers, Gilad