programming as an active language skill

87 views
Skip to first unread message

Gergely Buday

unread,
Mar 3, 2020, 12:16:01 PM3/3/20
to ppig-d...@googlegroups.com
Hi there,

classic, human language teaching emphasizes the difference between
passive and active language use: capable of understanding words and
grammatical structures in reading and hearing, and actively using them
in writing and in speech.

Are there studies that investigate programming as a form of active language use?

- Gergely

Brett Becker

unread,
Mar 3, 2020, 12:58:34 PM3/3/20
to ppig-d...@googlegroups.com
Hi Gergely,

There was a thread on this list about a year ago (available on the archive) related to natural languages and programming that led to me to presenting a paper on that topic at PPIG 2019. I've attached that paper here. If you search the archive for "Programming languages have keywords in English" You should find it (started April 16, 2019).

In terms of your question, the closest paper I know of is: Josh Tenenberg and Yifat Ben-David Kolikant "Computer programs, dialogicality, and intentionality" ICER 2014. That paper that won the "John Henry Award" that year: https://dl.acm.org/doi/10.1145/2632320.2632351

I'm quite interested in where human language meets programming languages. Let me know if I can provide more information.

Kind regards,

Brett



--
You received this message because you are subscribed to the Google Groups "PPIG Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ppig-discuss...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/ppig-discuss/CA%2B3iOzn%2B2oreMfg6_V%3DyVG9hx2ctPfJb-yg6OGLD4trVEKHd_Q%40mail.gmail.com.


--
Dr Brett A. Becker, Assistant Professor, School of Computer Science, University College Dublin

Steering Committee Chair, ACM Global Computing Education Conference (CompEd): Chengdu, China, May 17-19, 2019 @ACM_CompEd

International Liaison, SIGCSE Technical Symposium 2020, Portland, Oregon, USA, March 11-14, 2020
PPIG_Second_Language_Final.pdf

Gergely Buday

unread,
Mar 3, 2020, 3:14:33 PM3/3/20
to Brett Becker, ppig-d...@googlegroups.com
Hi Brett,

thanks indeed for this paper and I have found that thread, yes.

I had this idea of active language use for long and today in my
programming class I had to realise that my students are not capable of
making experiments on their own, they expect me to dictate the
solution -- which would not be a very smart pedagogy. That's why I
came here.

- Gergely
> To view this discussion on the web, visit https://groups.google.com/d/msgid/ppig-discuss/CAFbHUKmq84BhtfX4dOEMMTg%3DXGEZMFf_E_tyf1ssBvTDb-ND2Q%40mail.gmail.com.

Thomas Green

unread,
Mar 3, 2020, 4:05:28 PM3/3/20
to Brett Becker, ppig-d...@googlegroups.com
Hi Brett

You say "I'm quite interested in where human language meets programming languages”; I hope the following is pertinent .

There is a language for interactive fiction games called Inform 7 http://inform7.com/  much of which is a subset of standard English. Here’s a randomly-chosen example (http://inform7.com/book/RB_9_1.html): 

The House is a room. The mouse is an animal in the House.
The player carries some green eggs and a ham.
A food is a kind of thing that is edible. Food has some text called flavor. The flavor of food is usually "Tolerable."

And so on. And it’s not just the syntax. Inform 7 makes reasonable context-based inferences in some situations, the sort that an English speaker/hearer might well make.  I got quite excited about this and talked about it at PPIG - well, I waved my hands about - and Judith Good and (I think) Katherine Howlett got interested, because they were working on  faciitating children telling stories and creating adventure games. They did a proper study, getting children to work in Inform 7, and came to the conclusion that there was an ‘uncanny valley’ effect: Inform 7 is close enough to English that users make mistakes because they expect it to behave more like English than it does. Presented at a PPIG workshop a while back.

So we can speculate that where human language meets programming languages, there will frequently be a difficult area of confusion. 

I hadn’t foreseen that, and was disappointed, because I thought that Inform 7 was rather promising; but with hindsight it seems quite predictable, at least for Inform 7 - other language designs might fnd a solution.

Please note, I’m not saying" don’t investigate what happens where human language meets programming languages", nor am I saying “Inform 7 doesn’t work”. I think it’s a very interesting area, and I still think Inform 7 is very exciting. Just that  the uncanny valley effect seems to exact a cost. The benefits of English-like progamming may well outweigh the cost in many situations. 

Do please keep the PPIG community up to date with anything you do in this area.

Thomas


To view this discussion on the web, visit https://groups.google.com/d/msgid/ppig-discuss/CAFbHUKmq84BhtfX4dOEMMTg%3DXGEZMFf_E_tyf1ssBvTDb-ND2Q%40mail.gmail.com.
<PPIG_Second_Language_Final.pdf>

73 Huntington Rd, York YO31 8RL
01904-673675
mob. 0794 700 2581



Linda McIver

unread,
Mar 3, 2020, 4:16:26 PM3/3/20
to Thomas Green, Brett Becker, PPIG Discuss
That's really interesting, Thomas, because I have seen the same effect in Python. "Close enough" to English seems to lead to ambiguity - which is consistent with English itself, really! I've also noticed that syntactic sugar like "for item in list" is actually bad for the kids' mental model of the machine, because they think it's magic, and are then puzzled why other magic doesn't work. They have no idea of the underlying mechanisms, and it breaks their conceptual machine in all kinds of unexpected ways. To the extent that I started teaching while loops over lists first - which is a terribly unPython thing to do - because they got a better sense of how lists and indexing and the conceptual machine actually work, and then we could actually use the term syntactic sugar to explain for loops and they lost the sense of magic happening.

Note that this is not research, merely classroom observations. :)

On a side note, if anyone has any stunning insights on evaluating teacher professional development I would love to hear it. I run a data science education charity that trains teachers to use real datasets and authentic problems to teach STEM concepts, and we really need to be able to measure impact - hard enough when teaching kids, an extra layer of challenge when teaching teachers! Because ultimately the impact we care about (and which attracts funding) is on the kids...


To view this discussion on the web, visit https://groups.google.com/d/msgid/ppig-discuss/B056ADA9-6274-4CB9-A5C5-B2DD2C88D503%40gmail.com.


--

Exploring Life, Parenting and Social Justice: http://lindamciver.wordpress.com/     
The Australian Data Science Education Institute: http://adsei.org               

Dr Linda McIver
Executive Director, Australian Data Science Education Institute
--
Buy Fair Trade - Change the world one coffee at a time

Paul.Mulholland

unread,
Mar 4, 2020, 7:17:46 AM3/4/20
to Linda McIver, Thomas Green, Brett Becker, PPIG Discuss, Paul.Warren
One of my PhD students, Paul Warren, did a series of studies on a language called OWL [1]. OWL is a logic language in which formal symbols have been replaced with common English words. For example, the intersection of classes A and B is written as

A and B

rather using the term intersection or the upside down U symbol used in logic.

He found that when people were solving complex problems, the user friendly "and" symbol often led to errors, as participants slipped into reasoning about "and" in the ambiguous everyday English sense, where sometimes "and" means union (e.g. this event is for young and old alike") and sometimes intersection (e.g. "this event is only for people who are under 25 and in full time education"). When he replaced the term "and" with the less friendly term "intersection" the problem largely disappeared as participants did not stray from the intended meaning. 

A similar problem was found with OWL statements such as:

John has only sons

which means all the children that John has are sons but does not imply that he actually has any (he may have no children and therefore no sons). However, when completing cognitively demanding tasks, participants often slipped into assuming the additional implication that John must have sons.

The participants all understood the specific meaning of terms like "and" and "only" in OWL, but under high cognitive demand slipped into everyday forms of reasoning that introduced additional ambiguities or implications.

So I would say the lesson from his work is that standard English words can be useful but take care to avoid words that lead to additional ambiguities or implications. It isn't necessarily enough to ensure the users know the specific meaning of the term implied, as they can revert to everyday forms of reasoning when solving more complex problems.

Regards
Paul



Brett Becker

unread,
Mar 4, 2020, 7:55:58 AM3/4/20
to Paul.Mulholland, Linda McIver, Thomas Green, Brett Becker, PPIG Discuss, Paul.Warren
Hi all, 

I'll reply to Thomas, Linda and Paul in that order. First, this is *absolutely fascinating stuff*. I really want to do more work in this area. #

Thomas, that valley effect is really neat - hiding in plain sight! Do you have references for any of the Good/Howlett studies? I took a stab on Scholar but came up empty. 

Linda, also super interesting. I'd really like to do some research in this area. As for your teacher PD, the only thing I can think of is possibly Scott Portnoff (who is a teacher). He came to mind because he's done a lot of work on using foreign language instruction techniques to teach programming. He's written in ACM inroads on that (fairly recently). I mention him in the year-old thread I mentioned above in this email. 

Paul and Paul, that is really neat. I never (consciously) looked at "and" being used in English sometimes as a union and sometimes as an intersection. But I have noticed the effects (confusion) caused by it. I'm going to read that thesis! I might get back to you! Thanks! 

Brett

To view this discussion on the web, visit https://groups.google.com/d/msgid/ppig-discuss/5AC22102-CD0B-448A-9074-69532F7B9A22%40open.ac.uk.


--
Dr Brett A. Becker, Assistant Professor, School of Computer Science, University College Dublin @brettabecker
- Chair, SIRCSEire, the Ireland ACM SIGCSE Chapter @SIGCSEire
- International Liaison, SIGCSE Technical Symposium 2020 (& 2021) Portland, Oregon, USA, March 11-14, 2020
- Associate Editor, ACM Transactions on Computing Education (TOCE)

    John Pane

    unread,
    Mar 4, 2020, 10:10:58 AM3/4/20
    to PPIG Discuss
    Here is one of my favorite examples from when I studied issues with boolean keywords in everyday usage 20 years ago. 
    And, below, some excerpts from: Pane, J. F., Ratanamahatana, C. A., & Myers, B. A. (2001). Studying the Language and Structure in Non-Programmers’ Solutions to Programming Problems. International Journal of Human-Computer Studies, 54(2), 237-264.

    6.4. COMPLEX CONDITIONALS AND NOT

    …

    When the participants did use the not operator they gave it low precedence, which is contrary to the precedence that it has in most programming languages. A subsequent study found the use of not to be inconsistent: sometimes it was used with high precedence and other times with low precedence; and using parentheses was not effective to clarify precedence (Pane & Myers, 2000). Operator precedence errors were among the high- frequency bugs observed by Spohrer and Soloway (1986) in novice programs in a traditional programming language. However, in a recent study of a natural language style programming language, Bruckman and Edwards (1999) found that operator precedence errors were very infrequent. Further study is warranted to determine how languages should deal with issues of precedence.

    ...

    6.8. AND, OR AND BUT

    The raters found that often the word and was used as a sequencing word rather than as a Boolean operator. Also, in study two the raters examined the Boolean uses of and, and found that 75% were used in situations where the or operator would be required to achieve the desired effect in today's programming languages, as well as the query languages used for most database search engines. For example, a subject said, "if you score 90 and above'', but the score cannot simultaneously be 90 and greater than 90. Because the natural uses of and have such diverse meanings, and most of them are inconsistent with the Boolean operator, designers of future language should consider substituting a different name or symbol for this operator.

    Or and but appeared too rarely in these studies to draw firm conclusions without further research. When or was used, a Boolean interpretation would result in correct results. The infrequent use of or may be because disjunctive expressions are cognitively more difficult than conjunctive ones (Bourne, 1966). 


    John


    Thomas Green

    unread,
    Mar 4, 2020, 10:37:32 AM3/4/20
    to Brett Becker, Paul.Mulholland, Linda McIver, Brett Becker, PPIG Discuss, Paul.Warren
    @Brett - Hi Brett, here's the proper reference. I got some details wrong including Kate Howland's name. Kate and Judith, if you're reading this, I grovel apologetically. I think you'll find the paper very interesting. Quote:
    We describe three empirical studies investigating the use of natural language for computation in which we found that although natural language provides support for understanding computational concepts, it introduces additional difficulties when used for coding. 

    Reference and URL:
    Good, Judith and Howland, Kate (2017) Programming language, natural language? Supporting
    the diverse computational activities of novice programmers. Journal of Visual Languages and
    Programming, 39. pp. 78-92. ISSN 1045-926X
    This version is available from Sussex Research Online: http://sro.sussex.ac.uk/id/eprint/65897/

    Brett, you might be interested in the studies by Lance Miller in the 1970's, especially 'Programming in natural English', LA Miller and CA Becker, IBM Memo RC 5137, 1974. (John Pane might have a copy but I don't.)

    @John Pane: what a superb example. I can't imagine how the writer and their copy editor failed to see the absurdity. 

    A wild guess here ...... Natural language coding will only work well when EITHER the code is pretty simple OR (there is some ability to do semantic inference to resolve ambiguities AND there is a back-channel). Inform 7 has made a small start on the semantic inference but it clearly needs more. I'd imagine those conditions are necessary but far from sufficient.

    Thomas
    --

    73 Huntington Road, York YO31 8RL

    01904 673675

    07947 002581



    Thomas Green

    unread,
    Mar 4, 2020, 2:20:41 PM3/4/20
    to Judith Good, Brett Becker, Paul.Mulholland, Linda McIver, Brett Becker, PPIG Discuss, Paul.Warren
    And one mustn't forget Applescript, the read-only programming language. 

    Easy to read, nearly impossible to write.

    Thomas

    73 Huntington Road
    YO31 8RL
    07947 002 581
    01904 673675

    On Wed, 4 Mar 2020, 6:19 p.m. Judith Good, <J.G...@sussex.ac.uk> wrote:

    Thank you, Thomas, was planning to join in and provide a reference, but you beat me to it!

     

    Brett, as Thomas said, our conclusion was that natural langauge is great for trying to understand a program (it’s easier for people to read it back to themselves, and also to communicate their intentions to others, so it makes debugging more straightforward as a result).

     

    However, it’s not great at all for trying to actually write programs (for lots of reasons described in our paper). The article Thomas refers to is the most comprehensive one, but feel free to get in touch if you have any questions!

     

    Judith

     

    P.S. Thomas, no grovelling necessary. ☺

    Judith Good

    unread,
    Mar 4, 2020, 4:36:30 PM3/4/20
    to Thomas Green, Brett Becker, Paul.Mulholland, Linda McIver, Brett Becker, PPIG Discuss, Paul.Warren

    Thank you, Thomas, was planning to join in and provide a reference, but you beat me to it!

     

    Brett, as Thomas said, our conclusion was that natural langauge is great for trying to understand a program (it’s easier for people to read it back to themselves, and also to communicate their intentions to others, so it makes debugging more straightforward as a result).

     

    However, it’s not great at all for trying to actually write programs (for lots of reasons described in our paper). The article Thomas refers to is the most comprehensive one, but feel free to get in touch if you have any questions!

     

    Judith

     

    P.S. Thomas, no grovelling necessary. ☺

     

     

     

    From: <ppig-d...@googlegroups.com> on behalf of Thomas Green <thos...@gmail.com>
    Date: Wednesday, 4 March 2020 at 15:38
    To: Brett Becker <brett....@ucd.ie>
    Cc: "Paul.Mulholland" <paul.mu...@open.ac.uk>, Linda McIver <linda....@gmail.com>, Brett Becker <bretta...@gmail.com>, PPIG Discuss <ppig-d...@googlegroups.com>, "Paul.Warren" <paul....@open.ac.uk>
    Subject: Re: [ppig-discuss] programming as an active language skill

     

    @Brett - Hi Brett, here's the proper reference. I got some details wrong including Kate Howland's name. Kate and Judith, if you're reading this, I grovel apologetically. I think you'll find the paper very interesting. Quote:

    John Pane

    unread,
    Mar 4, 2020, 9:50:34 PM3/4/20
    to PPIG Discuss
    I do not find in my electronic files the Miller article Thomas cites. There was also another 1974 article that I do not find in my e-files. However I do have a Miller, 1981 and can provide a copy upon request.

    John

                Miller, L. A. (1974). Programming by Non-Programmers. International Journal of Man-Machine Studies, 6(2), 237-260. 
                Miller, L. A. (1981). Natural Language Programming: Styles, Strategies, and Contrasts. IBM Systems Journal, 20(2), 184-215. 
     

    Brett Becker

    unread,
    Mar 5, 2020, 12:29:50 AM3/5/20
    to John Pane, PPIG Discuss
    Thanks all for the links and insight. I enjoy a good bit of academic archaeology. 

    John, if you could somehow get that Miller 1981 article it would be much appreciated. I had a good look around and can't find a version I can access.

    Much appreciated. 

    Brett

    --
    You received this message because you are subscribed to the Google Groups "PPIG Discuss" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to ppig-discuss...@googlegroups.com.
    To view this discussion on the web, visit https://groups.google.com/d/msgid/ppig-discuss/C1677E5E-30F4-4DBC-A15A-08FDFFC33C47%40pane.net.

    Paul Warren

    unread,
    Mar 6, 2020, 9:38:19 AM3/6/20
    to PPIG Discuss
    Quite apart from programming languages, 'and' and 'or' have been shown to be used ambiguously in medical terminology, e.g. see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2232237/  To be fair, this was 20 years ago, before the terminological standard was formalised using Description Logics.  Paul 

    Raoul Duke

    unread,
    Mar 6, 2020, 12:22:23 PM3/6/20
    to Paul Warren, PPIG Discuss
    good thing every human involved in decision making has the Decision Logics wetware preloaded. :)
    Reply all
    Reply to author
    Forward
    0 new messages