Melodic intervals in semitones?

[Claire Arthur]

OK everyone: what's the easiest way to get melodic intervals in semitones while keeping direction?  **semits doesn't help much. Convert to MIDI representation, context n=2 and subtract?  I'm surprised that I've never needed to do this before now. Would be nice if this was an option for the mint command!

Claire

[Huron, David]

I presume by "direction" that negative and positive numerical values would suffice?

The "xdelta" tool calculates numerical differences (between successive numerical tokens - i.e., x-axis), and the "ydelta" tool calculates numerical differences between current tokens - i.e., y-axis).

For example:

semits input | xdelta

will give you semitone differences with up and down evident in positive and negative values.

Both the xdelta and ydelta commands allow regular expressions to identify tokens to skip or ignore (like barlines or rests) or restart the calculation (like phrase markers).  For example, the following option will skip **semits barlines and rests when calculating difference values:

semits b | xdelta -s [r=]

Type "xdelta -h" for other options.

-David

---

From: stars...@googlegroups.com <stars...@googlegroups.com> on behalf of Claire Arthur <claire....@gmail.com>
Sent: Thursday, November 12, 2020 1:02 PM
To: starstarhug <stars...@googlegroups.com>
Subject: [**HUG] Melodic intervals in semitones?

 

OK everyone: what's the easiest way to get melodic intervals in semitones while keeping direction?  **semits doesn't help much. Convert to MIDI representation, context n=2 and subtract?  I'm surprised that I've never needed to do this before now. Would be nice if this was an option for the mint command!

Claire

--
--
This is a message is from the **HUG newgroup.
To post to this group, send email to stars...@googlegroups.com
To unsubscribe from this group, send email to
starstarhug...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/starstarhug?hl=en
---
You received this message because you are subscribed to the Google Groups "starstarhug" group.
To unsubscribe from this group and stop receiving emails from it, send an email to starstarhug...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/starstarhug/e21b8598-e494-4cce-8f75-3c7cafedc914n%40googlegroups.com.

[Claire Arthur]

Thanks David,

Yes that would be the simplest. The problem is that my xdelta function (as well as my yank function) are not working properly on my machine. I tested it with Nat as well he is getting the same output, which makes me suspect there is a bug. I suppose I should work with Craig (or you) to try to fix them.

Cheers,

Claire

[Craig Sapp]

Hi Claire,

I created a tool called "semitones" which should do what you want, and it will be useful since extracting semitone intervals is a generally useful thing that warrants a dedicated tool.  It is part of the humlib repository so that it can also be used in Verovio Humdrum Viewer.

Example use:

humcat h://chorales/chor001.krn | semitones

**kern **tti **kern **tti **kern **tti **kern **tti

*k[f#] *k[f#] *k[f#] *k[f#] *k[f#] *k[f#] *k[f#] *k[f#]

*M3/4 *M3/4 *M3/4 *M3/4 *M3/4 *M3/4 *M3/4 *M3/4

4GG 12 4B 0 4d 0 4g 0

=1 =1 =1 =1 =1 =1 =1 =1

4G -3 4B 1 4d 2 2g 7

4E 2 8cL -1 4e -2 . .

. . 8BJ -2 . . . .

4F# 1 4A -2 4d 0 4dd -3

=2 =2 =2 =2 =2 =2 =2 =2

4G -5 4G -1 2d -3 4.b -2

4D 2 4F# 1 . . . .

. . . . . . 8a -2

4E -4 4G 5 4B 5 4g 0

=3 =3 =3 =3 =3 =3 =3 =3

4C -1 8cL -1 8eL -2 4.g 2

. . 8BJ 1 8d 2 . .

8BBL -2 4c 2 8e 2 . .

8AAJ -2 . . 8f#J 1 8a 2

4GG 7 4d 0 4g -1 4b -2

=4 =4 =4 =4 =4 =4 =4 =4

2D; -7 2d; 0 2f#; 1 2a; 2

4GG -1 4d -5 4g -5 4b 3

=5 =5 =5 =5 =5 =5 =5 =5

The semitone intervals are placed in spines called *tti (meaning twelve-tone intervals).  The interval is placed at the note attack of the first note in the interval, with a positive value meaning the following note is above the current pitch, and negative being below, and 0 meaning a repeated note.  If a rest follows a note, an "r" is placed at the last note

before the rest (this is useful for avoiding calculating intervals across rests).    The first note in a chord is the only note considered, and secondary tied notes are ignored.    Data can contain spine splits, although there will be corner cases such as when an interval is intervened by a spine split, the interval is calculated to the left subspine after the split.  If a merge splits the interval, then both notes (or more) before the merger will share the ending note after the merger.

To extract just the data spines:

humcat h://chorales/chor001.krn | semitones | extract -i **tti

or 

humcat h://chorales/chor001.krn | semitones | extractx -i tti

**tti   **tti   **tti   **tti

*clefF4 *clefGv2 *clefG2 *clefG2

*k[f#]  *k[f#]  *k[f#]  *k[f#]

*M3/4   *M3/4   *M3/4   *M3/4

12      0       0       0

=1      =1      =1      =1

-3      1       2       7

2       -1      -2      .

.       -2      .       .

1       -2      0       -3

=2      =2      =2      =2

-5      -1      -3      -2

2       1       .       .

.       .       .       -2

-4      5       5       0

=3      =3      =3      =3

-1      -1      -2      2

.       1       2       .

-2      2       2       .

-2      .       1       2

7       0       -1      -2

=4      =4      =4      =4

-7      0       1       2

-1      -5      -5      3

=5      =5      =5      =5

The humdrum extras tool "serialize" can be used to create a monophonic sequence out of such data.  Here is a pipeline to create a histogram of the melodic semitones found in the chorale:

humcat h://chorales/chor001.krn | semitones | extractx -i tti  | serialize | ridx -H | sortcount

52 -2

45 2

31 -1

26 0

25 1

7 -4

6 5

5 7

5 3

4 4

4 -3

4 -7

4 -5

1 12

The most common interval is -2 and the least common is +12.

The -m option can be given to list the MIDI note number of the note in the parallel **kern spine (only the first note in a chord will be considered):

humcat h://chorales/chor001.krn | semitones -m

**kern   **mnn    **kern   **mnn    **kern   **mnn    **kern   **mnn    

*clefF4  *clefF4  *clefGv2 *clefGv2 *clefG2  *clefG2  *clefG2  *clefG2

*k[f#]   *k[f#]   *k[f#]   *k[f#]   *k[f#]   *k[f#]   *k[f#]   *k[f#]   

*M3/4    *M3/4    *M3/4    *M3/4    *M3/4    *M3/4    *M3/4    *M3/4    

4GG      43       4B       59       4d       62       4g       67       

=1       =1       =1       =1       =1       =1       =1       =1

4G       55       4B       59       4d       62       2g       67       

4E       52       8cL      60       4e       64       .        .

.        .        8BJ      59       .        .        .        .

4F#      54       4A       57       4d       62       4dd      74

=2       =2       =2       =2       =2       =2       =2       =2       

4G       55       4G       55       2d       62       4.b      71

4D       50       4F#      54       .        .        .        .        

.        .        .        .        .        .        8a       69

4E       52       4G       55       4B       59       4g       67       

=3       =3       =3       =3       =3       =3       =3       =3

4C       48       8cL      60       8eL      64       4.g      67       

.        .        8BJ      59       8d       62       .        .

8BBL     47       4c       60       8e       64       .        .        

8AAJ     45       .        .        8f#J     66       8a       69

4GG      43       4d       62       4g       67       4b       71       

=4       =4       =4       =4       =4       =4       =4       =4

2D;      50       2d;      62       2f#;     66       2a;      69       

4GG      43       4d       62       4g       67       4b       71

=5       =5       =5       =5       =5       =5       =5       =5       

With **mnn meaning "MIDI note number".  **tti and **mnn are impromptu spine types and can be adjusted to a more formal exiting spine type if needed.  The main thing is that both have an "r", with "r" in **tti meaning that there is a rest present after the note so no interval can be calculated, and in **mnn, the "r" means that the associated **kern token is a rest.

To download humlib and install:

git clone https://github.com/craigsapp/humlib

cd humlib

make

make install

If you already have humlib downloaded, then go to that directory and type:

git pull

make

make install

Check to see if semitones is available:

which semitones

and the computer should reply 

    /usr/local/bin/semitones

Source code for the semitones tool:

https://github.com/craigsapp/humlib/blob/master/src/tool-semitones.cpp

=================

I implemented this in humlib so that it can be used in VHV to display the semitone intervals above the notes in the graphical score:

Direct link to the data used in the above example:

http://verovio.humdrum.org/?file=chorales/chor001.krn

Go to the filter toolbar and type "semitones -c" then press enter to activate the filter (or click on the filter icon which is highlighted in yellow in the above image, which indicates that the filter is currently active.  You can alternatively add this line anywhere in the input data to show the semiton intervals in the notation to display the semitone intervals above the notes:

!!!filter: semitones -c

The -c option is used to make the spine type be **cdata rather than **tti.  **cdata means "chord-like data" which will cause the data in that spine to be printed as text above the staff (otherwise **tti is not known to VHV and will be ignored).

Here is the MIDI note number data, using the filter "semitones -mc":

-=+Craig

To view this discussion on the web visit https://groups.google.com/d/msgid/starstarhug/c2a2ce6d-59ac-4df6-94da-9f1b22d56531n%40googlegroups.com.

[Huron, David]

Claire,

Can you send me a test file and output showing the problem with xdelta?

-David

---

From: stars...@googlegroups.com <stars...@googlegroups.com> on behalf of Craig Sapp <crai...@gmail.com>
Sent: Friday, November 13, 2020 5:42 AM
To: stars...@googlegroups.com <stars...@googlegroups.com>
Subject: Re: [**HUG] Melodic intervals in semitones?

 

To view this discussion on the web visit https://groups.google.com/d/msgid/starstarhug/CAPcjuFdCkVZVYk2A1Bb%3DRhzSe%3DUZA7XO14pquz3G5qXG4uPC%2Bw%40mail.gmail.com.

[Craig Sapp]

Hello **HUGers,

I added more options and documentation for the semitones tool/filter:

      https://doc.verovio.humdrum.org/filter/semitones

Example for creating a histogram of intervals:

     https://doc.verovio.humdrum.org/filter/semitones/#interval-histograms-for-bach-chorales

Creating a plot of the extracted data:

Most of the new options are related to highlighting notes by refined contours (steps, leaps, and directions):

     https://doc.verovio.humdrum.org/filter/semitones/#vhv-examples

Try this live example (use the repertory navigation buttons at the top left corner of the page to view analyses of other chorales).

-=+Craig

[Craig Sapp]

Hi Claire,

Given this input data:

**kern

*M3/4

=1

4c

4d

4e

=2

4b

4e

4d

=3

2.c

=

*-

The command:

yank -o ^= -r 2,3 test.krn | kern -x |  semits | xdelta -s [^0-9-]

runs as expected for me using MacOS 10.15.4, resulting in:

**Xsemits

*M3/4

=2

[11]

-7

-2

=3

-2

*-

If this does not do the same for you:

(1) report the result of the command:

      echo $shell

(2) The result of these commands:

      head -n1 $(which yank)

       head -n1 $(which yank)

(3) and the result of the outputs of each of these commands:

      echo $AWK_VER

      which awk

      which gawk

at least one of those commands should return the location of the awk interpreter, such as

     /usr/bin/awk

If so, then run a command similar to:

     /usr/bin/awk --version

(4) run md5 (on macos), or md5sum (on linux) to get the checksums of the following files:

     md5 $(which yank)*

     md5 $(which xdelta)*

-=+Craig

[Craig]

Hi Claire,

I suspect that I have found the problem, which may have been encountered before:  Most Humdrum Toolkit commands use the getopts interface to process command-line options in shell scripts (such as "yank"):

     https://sookocheff.com/post/bash/parsing-bash-script-arguments-with-shopts

and this interface probably does not allow any options after the first argument is found.

Example input:

**kern

*M3/4

=1

4c

4d

4e

=2

4b

4e

4d

=3

2.c

=

*-

The command:

 yank -o ^= -r 2,3 test.krn

Produces the desired output:

**kern

*M3/4

=2

4b

4e

4d

=3

2.c

*-

but the command:

yank -o ^= test.krn -r 2,3

Produces the error:

yank: ERROR: No range specified.

USAGE: yank -h                       (Help Screen)

       yank -l -r range [-c] [file ...]

       yank -m regexp -r range [-c] [file ...]

       yank -n regexp -r range [-c] [file ...]

       yank -o regexp [-e regexp] -r range [-c] [file ...]

       yank -s section_label -r range [-c] [file ...]

The problem is that the option -r 2,3 comes after the filename test.krn, and the program does not understand when options are placed after the filename.

My C++ programs do allow options after the arguments, as well as PERL's getopt equivalent.  So for programs created with C++ or PERL, there is no such limitation.

-=+Craig

[Huron, David]

> My C++ programs do allow options after the arguments, as well as PERL's getopt equivalent. So for programs created with C++ or PERL, there is no such limitation.

Sorry, Craig. I absolutely love what you do with your humdrum extensions. But on the matter of command-line parsing, chaos is being creating. The command:

yank -o ^= test.krn -r 2,3

simply does not conform to the POSIX portability standard. All options should appear before any input file(s) unless the file is intended as an argument/operand for an option. By writing programs that parse this, you are actually leading users to form a wrong impression about command-line syntax -- which they will then apply to other commands. What you are thinking as a feature is actually promoting a seriously undesirable user interaction habit.

I highly recommend keeping to the POSIX portability standard.

-David

---

From: stars...@googlegroups.com <stars...@googlegroups.com> on behalf of Craig <crai...@gmail.com>
Sent: Monday, November 23, 2020 10:54 PM
To: starstarhug <stars...@googlegroups.com>

Subject: Re: [**HUG] Melodic intervals in semitones?

--
--
This is a message is from the **HUG newgroup.
To post to this group, send email to stars...@googlegroups.com
To unsubscribe from this group, send email to
starstarhug...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/starstarhug?hl=en
---
You received this message because you are subscribed to the Google Groups "starstarhug" group.
To unsubscribe from this group and stop receiving emails from it, send an email to starstarhug...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/starstarhug/d8b9b50b-5cc8-439a-935a-ad232761a64an%40googlegroups.com.

[Craig Sapp]

Hi David,

Absolutely not.  What getopts is doing is anachronistic: most other unix tools do not exhibit such behavior.  So instead, the Humdrum Toolkit option handling should be updated to conform to modern conventions, such as using a PERL command-line interface to the AWK scripts rather than a bash script.  User experience is worse because of this limitation since there is no proper feedback about what the problem is, just a cryptic warning/error unrelated to the problem.

Most other standard unix tools available in MacOS and Linux allow mixing of options and argument parameters. As a demonstration, use this file as input:

aaa

bbb

ccc

ddd

to the grep command:

     grep -n ccc test.txt

which produces the expected output:

     3:ccc

And then try the command:

     grep ccc test.txt -n

which also produces

      3:ccc

rather than a cryptic error (getopts set the filename to be "test.txt -n" which causes strange errors in the awk scripts since the filename now has a space).

By your logic, grep should be fixed as well to disallow -n to follow the filename(s).

If you want scripts to be portable, then you definitely should put options before arguments in case grep is fully POSIX compliant on some other unix platforms, but that would be a rare configuration for modern computers.  This includes the rowfind script in chapter 34 of the Humdrum User Guide:

     https://www.humdrum.org/guide/ch34/#locating-and-identifying-tone-rows

where I had to move the options in front of the filename for the patt command, and which lead me to discover the likely source of Claire's problem.

There are far greater portability problems that happen due to differences between GNU and BSD versions of unix tools such as awk and cut, which have independent code bases and are beyond the scope of POSIX.   Even grep has different options on MacOS versus linux.  For example, I like to use grep -P on Linux computers to use PERL regular expressions (making it easier to insert a tab character into the search string with \t in particular).  On MacOS, the grep -P option is not available.  To use it, I have to install the GNU versions of unix tools:

      brew install grep

and then run the GNU version by prefixing a "g":

     ggrep -P "\t\d" file.txt

So if you want to write portable scripts, you also have to know all of the differences between BSD and GNU command options.  This is not practical, and by extension is it not practical to write portable bash scripts.

More info about POSIX command-line option parsing from the GNU programming manual:

GNU Program Argument Syntax Conventions

https://www.gnu.org/software/libc/manual/html_node/Argument-Syntax.html

On this page of the GNU programming manual:

https://www.gnu.org/software/libc/manual/html_node/Standard-Environment.html

it says:

_POSIX_OPTION_ORDER
If this environment variable is defined, it suppresses the usual reordering of command line arguments by getopt and argp_parse

This sentence implies that getopt is non-POSIX by default, but it seems to be in actuality that getopt is POSIX compliant by default (both in MacOS and Red Hat Linux, which I just checked).   I tried setting this variable and running grep with an option after an argument:

_POSIX_OPTION_ORDER=1

grep ccc test.txt -n

But the -n option was parsed as before, resulting in 

    3:ccc

rather than an error.  So I cannot force grep to be POSIX complain as the GNU documentation is describing.  And the BSD version also is not POSIX compliant nor checks the state of the _POSIX_OPTION_ORDER.

Other discussions related to this issue:

https://unix.stackexchange.com/questions/379944/does-getopts-read-in-command-line-arguments-in-some-order

-=+Craig

[Craig Sapp]

Hello **HUGers,

I created a documentation page for URL parameters that can be set when loading VHV:

https://doc.verovio.humdrum.org/topics/url-parameters

Of particular note is a new parameter that I added, allowing Humdrum data to be directly loaded from the URL:

https://doc.verovio.humdrum.org/topics/url-parameters/#ttext

For example, the URL:

https://verovio.humdrum.org/?t=KiprZXJuCjFjCj0KKi0 

contains the URL parameter t=KiprZXJuCjFjCj0KKi0

which is the MIME-encoded text for:

**kern

1c

=

*-

So clicking on the above URL will result in that Humdrum data loaded into the text editor:

The documentation page contains an interactive text box where you can type or copy-and-paste Humdrum data to be encoded into a URL.  The character limit is about 2000 characters, which allows for small files -- typically a musical example.  I was able to encode an entire Bach chorale into a URL that opened into Chrome, but the maximum length of the URL will depend on the browser.

-=+Craig