Yet another Fortran string library

[Stefano Zaghi]

Dear All,

as it is well known Fortran strings handling by language builtins is somehow limited. Many (many) Fortran libraries are available on the net for overcoming such limits, e.g.

+ Arjen Markus strings-related modules in FLIBS, http://flibs.sourceforge.net/
+ "tomedunn" module https://github.com/tomedunn/fortran-string-utility-module
+ many snippets into great projects, i.e. Jacob Williams https://github.com/jacobwilliams/json-fortran

I have studied these projects and many others and now I roll out my own library that I like to share with the community.

My library, StringiFor, is hosted here:

https://github.com/szaghi/StringiFor

It is young, but I hope already useful in production.

In the following I report the main details, the interested Fortraners can read the documentation into the StringiFor homepage.

StringiFor, Strings Fortran Manipulator, yet another strings Fortran module

A KISS pure Fortran library providing astrings (class) manipulator for modern (2003+) Fortran projects.

+ StringiFor is a pure Fortran (KISS) library providing a strings manipulator for modern Fortran projects;
+ StringiFor is Fortran 2003+ standard compliant;
+ StringiFor is OOP designed;
+ StringiFor is TDD designed;
+ StringiFor is a Free, Open Source Project.

StringiFor exposes only one class (OO-designed), the string type, that should be used as a more powerful string variable with respect a standard Fortran character variable. The main features of this class are:

+ seamless interchangeability with standard character variables, e.g. concatenation, IO, etc...;
+ handy builtin methods, e.g. split, search, basename, join, etc...;
+ low memory consumption: only one deferred length allocatable character member is stored, allowing for efficient memory allocation in array of strings, the elements of which can have different lengths;
+ safe: almost all methods are elemental or pure;
+ robust: the library is Test Driven Developed TDD, a comprehensive tests suite is provided.

StringiFor is an open source project, it is distributed under a multi-licensing system:

+ for FOSS projects:
+ GPL v3;
+ for closed source/commercial projects:
+ BSD 2-Clause;
+ BSD 3-Clause;
+ MIT.

Anyone is interest to use, to develop or to contribute to StringiFor is welcome, feel free to select the license that best matches your soul.

A taste of StringiFor

----
use stringifor
type(string) :: astring
type(string) :: strings(3)

astring = '0123456789'
print "(A)", astring%reverse()//'' ! print "9876543210"

astring = '<test> <first> hello </first> <first> not the first </first> </test>'
print "(A)", astring%search(tag_start='<first>', tag_end='</first>')//'' ! print "<first> hello </first>"

strings(1) = 'one'
strings(2) = 'two'
strings(3) = 'three'
print "(A)", astring%join(strings)//'' ! print "onetwothree"
print "(A)", astring%join(strings, sep='-')//'' ! print "one-two-three"
----

Hope to help someone,

My best regards.

[baf]

On 5/11/2016 7:24 AM, Stefano Zaghi wrote:
> Dear All,
>
> as it is well known Fortran strings handling by language builtins is somehow limited. Many (many) Fortran libraries are available on the net for overcoming such limits, e.g.
>

>

> astring = '0123456789'
> print "(A)", astring%reverse()//'' ! print "9876543210"
>
> astring = '<test> <first> hello </first> <first> not the first </first> </test>'
> print "(A)", astring%search(tag_start='<first>', tag_end='</first>')//'' ! print "<first> hello </first>"
>
> strings(1) = 'one'
> strings(2) = 'two'
> strings(3) = 'three'
> print "(A)", astring%join(strings)//'' ! print "onetwothree"
> print "(A)", astring%join(strings, sep='-')//'' ! print "one-two-three"
> ----
>

Interesting work, thanks for sharing. Why the need for the annoying
concatenation of a null at the end of the print lists?

[Stefano Zaghi]

Il giorno mercoledì 11 maggio 2016 17:25:32 UTC+2, baf ha scritto:
> Interesting work, thanks for sharing. Why the need for the annoying
> concatenation of a null at the end of the print lists?

Hi baf,

thank you for your feedback.

Indeed, you have not to concatenate a string (null or not) into IO, but if you avoid the concatenation:

+ you must use "(DT)" format specifier if supported by the compiler (GNU gfortran does not support defined IO);
+ or you have to use the method "chars()" to convert on the fly the string to standard character in order to use the standard IO (as I do in the examples).

The concatenation trick is just to convert on the fly the string to a standard character variable.

My best regards.

[FortranFan]

On Wednesday, May 11, 2016 at 10:24:47 AM UTC-4, Stefano Zaghi wrote:

> Dear All,
>
> as it is well known Fortran strings handling by language builtins is somehow limited. Many (many) Fortran libraries are available on the net for overcoming such limits, e.g.
>
> + Arjen Markus strings-related modules in FLIBS, http://flibs.sourceforge.net/
> + "tomedunn" module https://github.com/tomedunn/fortran-string-utility-module
> + many snippets into great projects, i.e. Jacob Williams https://github.com/jacobwilliams/json-fortran
>
> I have studied these projects and many others and now I roll out my own library that I like to share with the community.
>
> My library, StringiFor, is hosted here:
>
> https://github.com/szaghi/StringiFor
>
> It is young, but I hope already useful in production.
>

> ..

@Stefano Zaghi,

Kudos for great work, a wonderful contribution to the Fortran world!

I suppose it is too late: instead of a generic name 'string' for your type, it might have helped to have something slightly different, say even a "_t" suffix, as in string_t. So users can declare: type(string_t) :: string.

I take it you are familiar with ISO_VARYING_STRING and recent variation of it by Ian Henry: https://groups.google.com/forum/#!searchin/comp.lang.fortran/iso_varying_string%7Csort:date/comp.lang.fortran/53eUZC4GK_Q/kIHyN8fMAAAJ

Perhaps at your GitHub site, you can include your commentary on views such as this one: http://www.fortran.bcs.org/2015/suggestion_string_handling.pdf

and how your latest work addresses the points raised in this document.

Salutations,

[entropy]

Very nice job, Stefano! This library could replace also a large number of F90 routines that I often use taken from the awesome John Burkardt's collection [1].

You and Jacob Williams (with the release of the new version of json-fortran) are tempting me very much...

Congratulations,

Emanuele


[1] https://people.sc.fsu.edu/~jburkardt/f_src/f_src.html

[Stefano Zaghi]

Il giorno mercoledì 11 maggio 2016 17:47:42 UTC+2, FortranFan ha scritto:
> Kudos for great work, a wonderful contribution to the Fortran world!

You are too kind, mine is a very trivial contribution...

> I suppose it is too late: instead of a generic name 'string' for your type, it might have helped to have something slightly different, say even a "_t" suffix, as in string_t. So users can declare: type(string_t) :: string.

Indeed, I have thought to it for a while, but the suffix hurts my "bad aesthetic" preferences :-)

> I take it you are familiar with ISO_VARYING_STRING and recent variation of it by Ian Henry: https://groups.google.com/forum/#!searchin/comp.lang.fortran/iso_varying_string%7Csort:date/comp.lang.fortran/53eUZC4GK_Q/kIHyN8fMAAAJ

Yes, I know ISO_VARYING_STRING, I forget to mention it at the beginning, my bad.

> Perhaps at your GitHub site, you can include your commentary on views such as this one: http://www.fortran.bcs.org/2015/suggestion_string_handling.pdf
>
> and how your latest work addresses the points raised in this document.

I do not know this document, very interesting, thank you very much. It is probable that my library does not address any of those points, rather it adds new issues :-) I will study it.

My best regards.

[FortranFan]

On Wednesday, May 11, 2016 at 11:47:42 AM UTC-4, FortranFan wrote:

> ..

>
> I take it you are familiar with ISO_VARYING_STRING and recent variation of it by Ian Henry: https://groups.google.com/forum/#!searchin/comp.lang.fortran/iso_varying_string%7Csort:date/comp.lang.fortran/53eUZC4GK_Q/kIHyN8fMAAAJ
>

> ..

I meant "Ian Harvey" - my deepest apologies for my error.

[Stefano Zaghi]

Dear FortranFan,

I have open a specific issue for comparing StringiFor to other implementations, you can find it here

https://github.com/szaghi/StringiFor/issues/7

I have started to analyze the Clive's study on the standard character type, my comments are on the above link. I will complete the comparison soon.

My best regards.

[vladim...@gmail.com]

Looks interesting, thanks. Which versions of gfortran are supported, considering it uses deferred length structure components? Also there were issues with strings stored in class(*) in gfortran, but that probably does not affect you.

[Stefano Zaghi]

Dear vladimir,

You are right, GNU gfortran has bugged support for deferred length allocatable chars. As a matter of facts, I had have to add some workarounds to overcome such bugs, in particular when I must manipulate slices of the "string%raw" data. In the library there are many preprocessing macro to catch those workarounds thare unnecessary with other compilers.

To develop the library I use essentially gcc 5.3 and Intel Fortran 16.0.3, but I test with also gcc 6.1 and soon I will try IBM XL Fortran.

Note that when the library is compiled with gcc the defined IO is disabled because the compiler still not support defined IO.

My best regards.

[Clive Page]

Stefano

I have only just seen this thread, and not yet had time to examine your
new string-handling library in detail, but it looks to me as if you have
done about as well in overcoming the limitations of string-handling in
Fortran as is possible within the limitations of the current language
and the lack of implementation of derived type (DT) I/O.

The paper of mine that you mention was designed to provoke a bit of
discussion at a meeting - which it did. But I didn't get more than
lukewarm support for my contention that the next Fortran Standard ought
to have better string-handling facilities. So a library like yours is
probably the best that Fortran users are going to get for some years.


--
Clive Page

[Stefano Zaghi]

Il giorno giovedì 12 maggio 2016 00:20:31 UTC+2, Clive Page ha scritto:
> Stefano
>
> I have only just seen this thread, and not yet had time to examine your
> new string-handling library in detail, but it looks to me as if you have
> done about as well in overcoming the limitations of string-handling in
> Fortran as is possible within the limitations of the current language
> and the lack of implementation of derived type (DT) I/O.
>
> The paper of mine that you mention was designed to provoke a bit of
> discussion at a meeting - which it did. But I didn't get more than
> lukewarm support for my contention that the next Fortran Standard ought
> to have better string-handling facilities. So a library like yours is
> probably the best that Fortran users are going to get for some years.
>
>
> --
> Clive Page

Dear Clive,

thank you very much for your help. Your talk has been very useful: it clearly summarizes the main issues of Fortran strings handling. I am using it to improve/fix StringiFor. Can I have your permission to upload your PDF to the github repository (citing your copyright)? It is helpful for me having all documents in one place...

Thank you again,

My best regards.

[Wolfgang Kilian]

On 11.05.2016 17:53, Stefano Zaghi wrote:
> Il giorno mercoledì 11 maggio 2016 17:47:42 UTC+2, FortranFan ha scritto:
>> Kudos for great work, a wonderful contribution to the Fortran world!
>
> You are too kind, mine is a very trivial contribution...
>
>
>> I suppose it is too late: instead of a generic name 'string' for your type, it might have helped to have something slightly different, say even a "_t" suffix, as in string_t. So users can declare: type(string_t) :: string.
>
> Indeed, I have thought to it for a while, but the suffix hurts my "bad aesthetic" preferences :-)

I'd like to support FortranFan's suggestion here. Coincidentally, I
always write

use iso_varying_string, string_t => varying_string

I follow (my own) style guide that type names get the _t suffix -- and
reserve names without suffix for object instances.

>
>> I take it you are familiar with ISO_VARYING_STRING and recent variation of it by Ian Henry: https://groups.google.com/forum/#!searchin/comp.lang.fortran/iso_varying_string%7Csort:date/comp.lang.fortran/53eUZC4GK_Q/kIHyN8fMAAAJ
>
> Yes, I know ISO_VARYING_STRING, I forget to mention it at the beginning, my bad.
>
>> Perhaps at your GitHub site, you can include your commentary on views such as this one: http://www.fortran.bcs.org/2015/suggestion_string_handling.pdf
>>
>> and how your latest work addresses the points raised in this document.
>
> I do not know this document, very interesting, thank you very much. It is probable that my library does not address any of those points, rather it adds new issues :-) I will study it.

I appreciate any effort to introduce a convenient string facility in
Fortran. Since ISO_VARYING_STRING has been part of the standard for
quite some time, my strong preference is for the committee to merge that
specification with the intrinsic character string functionality of
Fortran. ISO_VARYING_STRING actually handles almost everything in a
statisfactory way (for me) -- except that conversion wrappers are needed
when interoperation with intrinsic CHARACTER type is required, and (less
important) the lack of Fortran substring syntax.

This appears to be what is said in that paper. From Clive's comment, I
understand that progress is unlikely to happen soon, unfortunately.

And, no, allocatable character is not a solution at all. It is not
possible to handle an array of such objects, naturally varying length
among elements, without extra glue code. In practice, this issue is
much worse than the conversion functions currently required for
varying_string.

-- Wolfgang

--
E-mail: firstnameini...@domain.de
Domain: yahoo

[Clive Page]

On 12/05/2016 09:42, Wolfgang Kilian wrote:
> This appears to be what is said in that paper. From Clive's comment, I
> understand that progress is unlikely to happen soon, unfortunately.

Well, the meeting at which I spoke was that of tbe BCS Fortran Special
Interest Group so only represented a tiny fraction of the world's
Fortran users. I understand that there are currently no plans for
improved character-handling in the current drafts for the next standard.

Part of the problem is that Fortran users, and even more so the
Standards Committee members, are concentrating on extending Fortran for
big number crunching applications, hence all the work on coarrays etc.
String-handling is pretty unimportant in this context, so it will be
hard to drum up enthusiasm unless the changes to the language are minor
with no adverse consequences on existing code.

> And, no, allocatable character is not a solution at all. It is not
> possible to handle an array of such objects, naturally varying length
> among elements, without extra glue code. In practice, this issue is
> much worse than the conversion functions currently required for
> varying_string.

I agree, the extra glue needed is rather significant.

I think my own preference would be for the existing ISO_VARYING_STRING
to be integrated fully into the language, perhaps with a less verbose
name for the type, e.g. VSTRING rather than VARYING_STRING. This
integration would need to allow data items of type VSTRING to be usable
in READ and WRITE statements, with A format permitted. Over time this
might make CHARACTER type redundant, but there's no reason to remove it.
The SQL language has had both CHAR and VARCHAR for many years without
apparent problems. That would seem to be a relatively small change to
Fortran with no serious compatibility consequences, but I have no
expertise whatever in this area.


--
Clive Page

[Clive Page]

On 12/05/2016 08:56, Stefano Zaghi wrote:
> thank you very much for your help. Your talk has been very useful: it clearly summarizes the main issues of Fortran strings handling. I am using it to improve/fix StringiFor. Can I have your permission to upload your PDF to the github repository (citing your copyright)? It is helpful for me having all documents in one place...

Yes, of course, I have no copyright claims on it, but having my name
attached would be appreciated. I can let you have it in another format
than PDF if that would be better. Contact me by email if you want (the
address use...@page2.eu does work).

Regards


--
Clive Page

[robin....@gmail.com]

On Thursday, May 12, 2016 at 12:24:47 AM UTC+10, Stefano Zaghi wrote:
> Dear All,
>
> as it is well known Fortran strings handling by language builtins is somehow limited. Many (many) Fortran libraries are available on the net for overcoming such limits, e.g.
>
> + Arjen Markus strings-related modules in FLIBS, http://flibs.sourceforge.net/
> + "tomedunn" module https://github.com/tomedunn/fortran-string-utility-module
> + many snippets into great projects, i.e. Jacob Williams https://github.com/jacobwilliams/json-fortran
>
> I have studied these projects and many others and now I roll out my own library that I like to share with the community.
>
> My library, StringiFor, is hosted here:
>
> https://github.com/szaghi/StringiFor
>
> It is young, but I hope already useful in production.
>
> In the following I report the main details, the interested Fortraners can read the documentation into the StringiFor homepage.
>
> StringiFor, Strings Fortran Manipulator, yet another strings Fortran module
>
> A KISS pure Fortran library providing astrings (class) manipulator for modern (2003+) Fortran projects.
>
> + StringiFor is a pure Fortran (KISS) library providing a strings manipulator for modern Fortran projects;
> + StringiFor is Fortran 2003+ standard compliant;
> + StringiFor is OOP designed;
> + StringiFor is TDD designed;
> + StringiFor is a Free, Open Source Project.
>
> StringiFor exposes only one class (OO-designed), the string type, that should be used as a more powerful string variable with respect a standard Fortran character variable. The main features of this class are:
>
> + seamless interchangeability with standard character variables, e.g. concatenation, IO, etc...;
> + handy builtin methods, e.g. split, search, basename, join, etc...;
> + low memory consumption: only one deferred length allocatable character member is stored, allowing for efficient memory allocation in array of strings, the elements of which can have different lengths;

This is all very interesting, but this has been available in PL/I
for 50 years. In particular, elements of arrays can have different lengths.

[Stefano Zaghi]

Il giorno giovedì 12 maggio 2016 11:32:01 UTC+2, Clive Page ha scritto:
> Yes, of course, I have no copyright claims on it, but having my name

> attached would be appreciated...


Dear Clive, you are very kind. The PDF format fits well, later soon I will update the github repo and post here the page.

Thank you very much.

My best regards.

[robin....@gmail.com]

Look, if you want real string handling, use PL/I
where the facilities have been there for 50 years,

[Stefano Zaghi]

Il giorno giovedì 12 maggio 2016 11:58:38 UTC+2, robin....@gmail.com ha scritto:
> This is all very interesting, but this has been available in PL/I
> for 50 years. In particular, elements of arrays can have different lengths.

Dear Robin,

you are right, but I am a poor Fortran man not a rich PL/I one. I have (and I want, I like...) to work with Fortran. Anyhow, if you can post the most useful string characteristics of PL/I language we can try to improve our libraries. My efforts are largely inspired by Python, but learning about other languages is always welcome.

My best regards.

[Wolfgang Kilian]

On 12.05.2016 11:28, Clive Page wrote:
> On 12/05/2016 09:42, Wolfgang Kilian wrote:
>> This appears to be what is said in that paper. From Clive's comment, I
>> understand that progress is unlikely to happen soon, unfortunately.
>
> Well, the meeting at which I spoke was that of tbe BCS Fortran Special
> Interest Group so only represented a tiny fraction of the world's
> Fortran users. I understand that there are currently no plans for
> improved character-handling in the current drafts for the next standard.
>
> Part of the problem is that Fortran users, and even more so the
> Standards Committee members, are concentrating on extending Fortran for
> big number crunching applications, hence all the work on coarrays etc.
> String-handling is pretty unimportant in this context, so it will be
> hard to drum up enthusiasm unless the changes to the language are minor
> with no adverse consequences on existing code.

Yes, understood.

My programs spend 90% or more of their time in number crunching, this is
a good reason to use Fortran. Coarrays may help improving this part
even more. But more than 90% of the program code concerns data
management, interface to external programs, user interface, and so on.
Those 90% make heavy use of string handling.

Unless the standards committee wants to encourage programmers migrating
that 90% to Python etc., like many have already done, there is a reason
to think about user convenience also for non-numeric objects. The OO
facilities of modern Fortran have gone a long way, but there are two
areas where Fortran is definitely lacking - generic programming for
type-safe data containers, and string handling.

>> And, no, allocatable character is not a solution at all. It is not
>> possible to handle an array of such objects, naturally varying length
>> among elements, without extra glue code. In practice, this issue is
>> much worse than the conversion functions currently required for
>> varying_string.
>
> I agree, the extra glue needed is rather significant.
>
> I think my own preference would be for the existing ISO_VARYING_STRING
> to be integrated fully into the language, perhaps with a less verbose
> name for the type, e.g. VSTRING rather than VARYING_STRING. This
> integration would need to allow data items of type VSTRING to be usable
> in READ and WRITE statements, with A format permitted. Over time this
> might make CHARACTER type redundant, but there's no reason to remove it.
> The SQL language has had both CHAR and VARCHAR for many years without
> apparent problems. That would seem to be a relatively small change to
> Fortran with no serious compatibility consequences, but I have no
> expertise whatever in this area.

Fully agree.

[FX]

> This is all very interesting, but this has been available in PL/I for
> 50 years.

Out of curiosity, since you talk so much about PL/I and I've never
encountered it myself, I have a question. What actively maintained
compilers exist for modern architectures (Windows, Mac OS X, x86 linux,
iOS and arm/Android)?

--
FX

[Gary Scott]

ack. Absolutely hate that inconsistency. I'd prefer something like:

character(*), <varying|fixed> :: myString

character(256), varying :: mystring !Varying but max length of 256

[Gary Scott]

Please, no comments about emperor and clothes.

[Stansfield Temmelmeier]

On 2016-05-12, Wolfgang Kilian <kil...@invalid.com> wrote:
> Unless the standards committee wants to encourage programmers migrating
> that 90% to Python etc., like many have already done, there is a reason
> to think about user convenience also for non-numeric objects. The OO
> facilities of modern Fortran have gone a long way, but there are two
> areas where Fortran is definitely lacking - generic programming for
> type-safe data containers, and string handling.

Also needed are true pointers and unsigned integers in various sizes.
For me it's more important than string enhancement but I don't argue with
your view.

Stan

[Stansfield Temmelmeier]

Windows and x86 Linux is no longer modern architecture.

Anyway PL/I is very good general purpose language with more features than
most other HLL including native multitasking as language feature (not
library). Main compiler is for IBM mainframe but I think Windows version
also exists.

Nobody can argue PL/I is not better general purpose language than Fortran.
PL/I covers business and scientific programming and gp programming very
completely. But Fortran is still simpler and more special purpose language.
Fortran can add some amount of features and become more general purpose
language.

This group is not home for language comparison but it's true people can look
to commercially successful (PL/I) and commercially not so successful (Ada)
good languages for ideas to implement needed feature instead of reinventing
some wheel every few years. Sometimes it was already done right the first
time. (Usually whatever IBM did was right first time.)

Stan

[Stansfield Temmelmeier]

On 2016-05-12, Stansfield Temmelmeier <s...@optika.de> wrote:
> On 2016-05-12, FX <cou...@alussinan.org> wrote:
>> Out of curiosity, since you talk so much about PL/I and I've never
>> encountered it myself, I have a question. What actively maintained
>> compilers exist for modern architectures (Windows, Mac OS X, x86 linux,
>> iOS and arm/Android)?

Just because some language or system is not available for home entertainment
device is not automatic disqualification for serious programming. If you
have this nearsighted view you will miss most serious and good hardware and
software. Nothing on your list qualifies as good AFAICT.

IBM invented FORTRAN COBOL PL/I and much more than that. Not Apple or
Microsoft or even FSF.

Stan

[FortranFan]

On Thursday, May 12, 2016 at 7:19:20 AM UTC-4, Wolfgang Kilian wrote:

> ..

>
> My programs spend 90% or more of their time in number crunching, this is
> a good reason to use Fortran. Coarrays may help improving this part
> even more. But more than 90% of the program code concerns data
> management, interface to external programs, user interface, and so on.
> Those 90% make heavy use of string handling.
>
> Unless the standards committee wants to encourage programmers migrating
> that 90% to Python etc., like many have already done, there is a reason
> to think about user convenience also for non-numeric objects. The OO
> facilities of modern Fortran have gone a long way, but there are two
> areas where Fortran is definitely lacking - generic programming for
> type-safe data containers, and string handling.
>

> ..


Agree completely.

So on the point about "user convenience also for non-numeric objects" and "generic programming for type-safe data containers, and string handling", what can be done if the current standards committee is focused too much on things like coarrays and other esoteric developments around "atomic" computing?

* can some or all of you - Wolfgang Kilian, Ian Harvey, Clive Page, Stefano Zaghi, etc. - join the committee and bring more balance?

* and/or collaborate separately to develop near-complete proposals that the standard committee is then tasked with minimal effort to review, refine, endorse, and accept them?

In my simple-minded thinking, similar to a concept of a Turing-complete language, is the idea of functionally-convenient language for numeric computing (toward scientific/engineering/technical/economics applications). My view is that Fortran has constantly fallen short in this respect since 1980s; the two points mentioned by Wolfgang Kilian on generic programming and string handling being glaring examples.

There has to be a pressing urgency to fill such gaps and immediately (meaning in 20 years! since that is how long it will take for a coder like me to be able to find a compiler with reliable, standard and improved string handling feature if the committee started work today) make the language convenient for users, otherwise there will be no point to the rest of the language development. Looking at books on C++ and public codes written in them (Trilinos, Cantera, etc.) for scientific/technical applications, I get the impression there are a few features and some programming paradigms and coding styles around containers, strings, generics, etc. that numeric packages make use of, such world is otherwise slow or hesitant to employ the latest fad or some bleeding edge comp.sci. concept in its applications. Under such circumstances, Fortran should be able to more readily adopt and bring-in the features so the programmers can write similarly convenient code.

If 90% of an application code is handled via non-Fortran approaches due to user convenience and Fortran standard bearers think they can simply focus on the balance 10% to make it more compute-efficient (whatever), it will likely prove very detrimental to the actual practitioners of Fortran. With suitable approaches, many other languages - especially C++ - will achieve (or already have achieved) similar computing efficiency. Most organizations then will follow the CERN, NASA example (and many in industry) and either abandon Fortran altogether or otherwise slate it for death in legacy libraries which all get ported over time to other languages.

Why doesn't the committee seem to understand this? Or at least if they do, I'm now convinced the follow-up is grossly inadequate, it's too little and way too slow.

[Stefano Zaghi]

Dear all,

concerning the general Fortran improvement, I agree with all you. My idea is that Fortran is great for number crunching (but I prefer to say for "Formula translator"): scientific applications can be more easily developed with Fortran than other languages (IMHO), independently if you have to do a lot of computation or not. Python is very close to Fortran for not so big calculation (and wins on Fortran on many "minor" aspects). Obviously, when the computations are huge, Fortran is the best. C and C++ are widely used in my research field (CFD), but I do not like them. FortranFan seems to be pessimistic about the future of Fortran, on the contrary I think that Fortran will have a shining future such as its past. In particular, I am thinking to CAF: I am an (ab)user of MPI/OpenMP due to my work, so I know how could be frustrating such kind of codes. CAF is really a revolution: obtaining efficient parallel codes has not been ever so simple, elegant, funny... Fortran will rocks!

Nevertheless, as you pointed, I am conscious that Fortran has some lacks (that thrust me to develop my poor libraries): string handling is one of them, but also unsigned integer and actual pointers are valuable features. In my top list, the most important missing feature is availability of abstract container(s). One of us, Chris MacMackin, is developing a great library to address such a lack.

This is just my 2 cents about the general consideration of Fortran.

Regarding the string manipulators...

Il giorno giovedì 12 maggio 2016 15:42:27 UTC+2, FortranFan ha scritto:
> Agree completely...

> * can some or all of you - Wolfgang Kilian, Ian Harvey, Clive Page, Stefano Zaghi, etc. - join the committee and bring more balance?

Thank you for including in this list, but I am not up to them, I can only do an "umpa lumpa" engineering work :-) maybe I can bring the coffee... one cup at time, I have not yet parallelized myself :-)

> * and/or collaborate separately to develop near-complete proposals that the standard committee is then tasked with minimal effort to review, refine, endorse, and accept them?

I can only share my poor FOSS libraries.

My best regards.

[Wolfgang Kilian]

On 12.05.2016 15:42, FortranFan wrote:
> On Thursday, May 12, 2016 at 7:19:20 AM UTC-4, Wolfgang Kilian wrote:
>
>> ..
>>
>> My programs spend 90% or more of their time in number crunching, this is
>> a good reason to use Fortran. Coarrays may help improving this part
>> even more. But more than 90% of the program code concerns data
>> management, interface to external programs, user interface, and so on.
>> Those 90% make heavy use of string handling.
>>
>> Unless the standards committee wants to encourage programmers migrating
>> that 90% to Python etc., like many have already done, there is a reason
>> to think about user convenience also for non-numeric objects. The OO
>> facilities of modern Fortran have gone a long way, but there are two
>> areas where Fortran is definitely lacking - generic programming for
>> type-safe data containers, and string handling.
>>
>> ..
>
>
> Agree completely.
>
> So on the point about "user convenience also for non-numeric objects" and "generic programming for type-safe data containers, and string handling", what can be done if the current standards committee is focused too much on things like coarrays and other esoteric developments around "atomic" computing?

Coarrays etc. - better: facilities that make optimal use of current HPC
and PC hardware, integrated in the language itself - are essential for
Fortran as a prime language for numerical computations. That's no more
esoteric than Fortran as such in the IT world.

My point is that there is an advantage in keeping it in line with
current CS mainstream also as a general-purpose language. Actually, the
standards committee has done an excellent job in that respect, avoiding
quite a few pitfalls on the road.

> * can some or all of you - Wolfgang Kilian, Ian Harvey, Clive Page, Stefano Zaghi, etc. - join the committee and bring more balance?

AFAIK, members have been representatives of major players (vendors) in
the Fortran world, which makes perfect sense. Some of the current and
former members read this group and post here frequently, so I don't
think that ideas disappear unnoticed. I don't know if any of the others
have been involved more closely with the committee's agenda.

> * and/or collaborate separately to develop near-complete proposals that the standard committee is then tasked with minimal effort to review, refine, endorse, and accept them?

If there is actual need for proposals, it might be useful if some
committee member could comment ... I guess much of the time is spent
fixing flaws and improving previous standards and documents, so even
minimal effort is nontrivial if entirely new ideas are involved. It
would be interesting to know if there are topics that will receive
increased attention after completion of the current (2017?) standard.

> In my simple-minded thinking, similar to a concept of a Turing-complete language, is the idea of functionally-convenient language for numeric computing (toward scientific/engineering/technical/economics applications). My view is that Fortran has constantly fallen short in this respect since 1980s; the two points mentioned by Wolfgang Kilian on generic programming and string handling being glaring examples.

I wouldn't continue using (modern) Fortran if it was not for its
well-designed layout and structure at all scales, and widespread
availability. It's just a few issues that reappear frequently, where
good solutions exist, but are realized only in other languages.

> There has to be a pressing urgency to fill such gaps and immediately (meaning in 20 years! since that is how long it will take for a coder like me to be able to find a compiler with reliable, standard and improved string handling feature if the committee started work today) make the language convenient for users, otherwise there will be no point to the rest of the language development. Looking at books on C++ and public codes written in them (Trilinos, Cantera, etc.) for scientific/technical applications, I get the impression there are a few features and some programming paradigms and coding styles around containers, strings, generics, etc. that numeric packages make use of, such world is otherwise slow or hesitant to employ the latest fad or some bleeding edge comp.sci. concept in its applications. Under such circumstances, Fortran should be able to more readily adopt and bring-in the features so the programmers can write similarly convenient code.

If a mixed-language approach (such as doing only a small part of the
project in Fortran, and using Python - or Java, or Scala, or Swift,
whatever ... for the rest) is superior, than I'd recommend it as a
natural solution. It's just that I still find the overall design of
Fortran more convincing and, most importantly, error-proof than the main
alternatives Python and C++, that I'm pursuing a monolithic approach
which exposes some shortcomings of the Fortran standard.

> If 90% of an application code is handled via non-Fortran approaches due to user convenience and Fortran standard bearers think they can simply focus on the balance 10% to make it more compute-efficient (whatever), it will likely prove very detrimental to the actual practitioners of Fortran. With suitable approaches, many other languages - especially C++ - will achieve (or already have achieved) similar computing efficiency. Most organizations then will follow the CERN, NASA example (and many in industry) and either abandon Fortran altogether or otherwise slate it for death in legacy libraries which all get ported over time to other languages.
>
> Why doesn't the committee seem to understand this? Or at least if they do, I'm now convinced the follow-up is grossly inadequate, it's too little and way too slow.

I assume that they are perfectly aware of this view, but there are
constraints, and different opinions. Even if they would initiate a
revolution of the standard (for whatever reason), it would take a time
before this is reflected in compilers.

I think that if a particular concept can be proven to be worthwhile by
implementing it as an extension in a real compiler and successfully
applying this to real problems, it could convince people more than
theory. Problem is, that I won't have time (and the C programming
skills) to actually do anything in that direction.

[Stefano Zaghi]

Dear all,

I have made some progress comparing StringiFor to ISO_VARYING_STRING and allocatable deferred length character, on the basis Clive's document.

Here my comments : https://github.com/szaghi/StringiFor/issues/7

I am now integrating the defined IO of Ian Harvey (what a great piece of code!). Unfortunately, this is disabled for GNU gfortran users. (I will cite Ian and all others into the project sources/doc very soon, please be patient).

My best regards.

[FortranFan]

On Thursday, May 12, 2016 at 11:31:24 AM UTC-4, Wolfgang Kilian wrote:

> On 12.05.2016 15:42, FortranFan wrote:

> ..
> >
> .. what can be done if the current standards committee is focused too much on things like coarrays and other esoteric developments around "atomic" computing?

>
> Coarrays etc. - better: facilities that make optimal use of current HPC
> and PC hardware, integrated in the language itself - are essential for
> Fortran as a prime language for numerical computations. That's no more
> esoteric than Fortran as such in the IT world.
>

My point is not that coarrays are unimportant or that it itself is esoteric, but enough observations have been made on this forum and elsewhere and visible in the changes to draft for the next Fortran standard that suggest a significant attention of the committee has been to "atomic" operations with coarrays and as to whether it will all pay off is questionable, in the minds of several who are far closer to the field than I am. Then if all that attention is coming in the ways of other aspects such as filling gaps in generics and string handling, for me it is a perfectly valid concern.

> ..

>
> I assume that they are perfectly aware of this view, but there are
> constraints, and different opinions. Even if they would initiate a
> revolution of the standard (for whatever reason), it would take a time
> before this is reflected in compilers.
>

> ..

Looking online, I have found discussions calling for improved string handling in Fortran with essentially the same points as by Clive Page in PDF mentioned upthread, some by Clive Page himself if I recall correctly, and some which are dated late 1900s when Fortran 2000 was in works (which later seemed to have become Fortran 2003). Status quo in 2016, what's up with that?

There was a new thread on generics on this forum a year or so ago and the only feedback from an active committee member was 'use cases' will be needed - seriously!?

[FortranFan]

On Thursday, May 12, 2016 at 10:12:51 AM UTC-4, Stefano Zaghi wrote:


> ..

>
> Nevertheless, as you pointed, I am conscious that Fortran has some lacks (that thrust me to develop my poor libraries): string handling is one of them, but also unsigned integer and actual pointers are valuable features. In my top list, the most important missing feature is availability of abstract container(s). One of us, Chris MacMackin, is developing a great library to address such a lack.
>

> ..


@Stefano Zaghi,

The work that you and others are doing with Fortran in the FOSS area is tremendous, to be much appreciated.

In some other role somewhere, I might be able to make direct use of it and even try to contribute in my own limited way. But that is not the case right now and unfortunately (and for several cannot-be-named reasons) the FOSS libraries are out of reach for me; they are of academic interest only.

Hence what will be nice for me will be if Fortran offered built-in capabilities for features such as those for enhanced string handling either in the main standard or in the 'other' part, such as the ISO_VARYING_STRING thingy, possibly as 'standard' libraries, say 'standard' derived types with type-bound procedures.

My view is in year 2016, why should you or anyone else have to design, define, implement, document, test, and roll-out a derived type for string handling?

If the language cannot offer an intrinsic type, then at least it can define an ISO type of interface for a derived type which can then be offered by implementations. Meaning, instead of StringiFor, there can be an ISO_XXX (similar to ISO_C_BINDING) standard intrinsic module which includes a standard derived type for a string class (analogous to C_PTR, similar to your string derived type) with standardized interfaces for the type-bound procedures, defined operations, etc. The implementations - GCC/gfortran, Intel, NAg, Cray, etc. - can then offer the type as and when they get around to making the effort, as they do presently with C_PTR and C_FUNPTR.

Mind you, if the language had not included such a C_PTR type, probably several programmers could easily have hand-rolled one, similar to what you did for string, each in their own ways and all probably better than the standard version of C_PTR. But it would be impossible for some end-users such as me to use any of them; however a 'standard' derived type such as C_PTR with even only private components (just as you have in your string type), now that makes a world of difference for me! So having facilities, even derived types, which are part of the language offering, even and especially with cast-in-stone interfaces, are a big help.

I'm not claiming the above analogy with ISO_C_BINDING is totally apropos, but hopefully you get the idea. Note also, I don't mean to convey the language has to offer standard derived types for everything e.g., ODE/PDE solvers, etc. do seem far more suited for true experts. But a string class? Yes, it feels so basic that, come on, Fortran has to have something 'standard' for it. If not an intrinsic type, then a 'standard' derived type.

Thanks much for all your coding and sharing,

[Stefano Zaghi]

Dear FortranFan,

I agree with you.

> In some other role somewhere, I might be able to make direct use of it and even try to contribute in my own limited way. But that is not the case right now and unfortunately (and for several cannot-be-named reasons) the FOSS libraries are out of reach for me; they are of academic interest only.

Sad to read this: I know that in some area FOSS projects are feasible, this makes me always sad.

> Hence what will be nice for me will be if Fortran offered built-in capabilities for features such as those for enhanced string handling either in the main standard or in the 'other' part, such as the ISO_VARYING_STRING thingy, possibly as 'standard' libraries, say 'standard' derived types with type-bound procedures.

I totally agree.

> My view is in year 2016, why should you or anyone else have to design, define, implement, document, test, and roll-out a derived type for string handling?...

You are right. I adopted TDD so even a so simple library like StringiFor consumes many of my free hours to develop the tests suite: a standard library implemented by serious programmers would be very welcome.

>I'm not claiming the above analogy with ISO_C_BINDING is totally apropos, but hopefully you get the idea. Note also, I don't mean to convey the language has to offer standard derived types for everything e.g., ODE/PDE solvers, etc. do seem far more suited for true experts. But a string class? Yes, it feels so basic that, come on, Fortran has to have something 'standard' for it. If not an intrinsic type, then a 'standard' derived type.

I totally agree.

>Thanks much for all your coding and sharing,

Thank you too!

My best regards.

[FortranFan]

On Thursday, May 12, 2016 at 11:31:24 AM UTC-4, Wolfgang Kilian wrote:

> .. Actually, the

> standards committee has done an excellent job in that respect, avoiding
> quite a few pitfalls on the road.

> ..


Just as 'justice delayed is justice denied', features delayed are effectively features denied.

As someone who provided the link in the other thread,

http://cerncourier.com/cws/article/cern/30873

".. Also in the 1980s, discussions on the next version of the Fortran standard – known as 8X at the time – were blocked in the committees involved. The US committee X3J3, with around 50 members, manufacturers, large laboratories and other users, expected the ISO WG5 committee to adopt its recommendations. Instead, a stalemate ensued over the content and a compromise Fortran 90 was not reached until 1991.

Michael Metcalf, formerly of CERN and co-author of Fortran 90/95 Explained, was a member and significant contributor to X3J3 and must have been appalled at the delays. CERN therefore adopted C++ in the 1990s as the recommended programming language for the LHC experiments. Nonetheless, the accelerator and engineering communities at CERN continue to use Fortran. The LHC magnet design program ROXIE, accelerator design tools like SixTrack and MAD-X (the MAD 8 successor) or the PS operational tool for phase space tomography are recent developments in modern Fortran. Even the LHC experiments still continue to use some existing Fortran applications for simulated-event generation. .."

[Ian Harvey]

On 2016-05-12 7:28 PM, Clive Page wrote:
> On 12/05/2016 09:42, Wolfgang Kilian wrote:
>> This appears to be what is said in that paper. From Clive's comment, I
>> understand that progress is unlikely to happen soon, unfortunately.
>
> Well, the meeting at which I spoke was that of tbe BCS Fortran Special
> Interest Group so only represented a tiny fraction of the world's
> Fortran users. I understand that there are currently no plans for
> improved character-handling in the current drafts for the next standard.
>
> Part of the problem is that Fortran users, and even more so the
> Standards Committee members, are concentrating on extending Fortran for
> big number crunching applications, hence all the work on coarrays etc.
> String-handling is pretty unimportant in this context, so it will be
> hard to drum up enthusiasm unless the changes to the language are minor
> with no adverse consequences on existing code.
>
>> And, no, allocatable character is not a solution at all. It is not
>> possible to handle an array of such objects, naturally varying length
>> among elements, without extra glue code. In practice, this issue is
>> much worse than the conversion functions currently required for
>> varying_string.
>
> I agree, the extra glue needed is rather significant.

I disagree about the significance. If the wrapped component is public,
the "glue code" is typically just a component reference, at most.

Beyond that the usual trade-off exists, where more effort can be put
into library code to reduce the amount of effort required in client
code. Given string manipulation is a common activity I would expect
that tradeoff to sit towards the library code end of the spectrum, but
that library code only needs to be written (or otherwise acquired) once.

I think there are options for language development that could reduce the
need for the component reference, that have more general application
than character handling. From the point of view of the language, it is
far more productive to pursue features with widespread application, than
isolated "bolt-ons".

> I think my own preference would be for the existing ISO_VARYING_STRING
> to be integrated fully into the language, perhaps with a less verbose
> name for the type, e.g. VSTRING rather than VARYING_STRING. This
> integration would need to allow data items of type VSTRING to be usable
> in READ and WRITE statements, with A format permitted. Over time this
> might make CHARACTER type redundant, but there's no reason to remove it.
> The SQL language has had both CHAR and VARCHAR for many years without
> apparent problems. That would seem to be a relatively small change to
> Fortran with no serious compatibility consequences, but I have no
> expertise whatever in this area.

With the addition of defined input/output, you can use VARYING_STRING
from ISO_VARYING_STRING directly from input/output statements. I posted
code that extended ISO_VARYING_STRING in this way recently, as a
demonstration of this. No need to make difficult and expensive changes
to the language and compilers.

So the only change being proposed above is really to permit derived
types to be input/output under data edit descriptors other than DT.
That is a far more general issue than character handling.

(If you are worried about verbosity, then there are many other areas in
Fortran that I would look at first, before I started changing intrinsic
type names.)

In my experience, another practical consideration frequently arises in
this area too, which is that the use cases for an array of varying
length string data *only* are not that common *in the longer term*.
Many times I have started out with such a data structure in my code (so,
in the shorter term development of a code they are common), because that
naturally models the nature of the data at that stage of program
development, but on quite a few occasions I have then had to change the
type, because I need to store more than a string in each element of the
array. If you want to have some chance of minimising changes in the
interface of the type as it evolves, you are not going to immediately
jump in to some sort of new intrinsic type from the get go - wrapping an
allocatable component works quite well.

~~

It is a fundamental concept in the language that array elements only
differ from element to element in an array in *value*. Keep that in
mind when proposing changes like this - this not trivial language change
territory.

[Ian Harvey]

On 2016-05-13 1:31 AM, Wolfgang Kilian wrote:
...

> I think that if a particular concept can be proven to be worthwhile by
> implementing it as an extension in a real compiler and successfully
> applying this to real problems, it could convince people more than
> theory. Problem is, that I won't have time (and the C programming
> skills) to actually do anything in that direction.

Excuses, excuses. If you don't know C well enough, then write your
compiler, by yourself, as a project in your spare time, with no
experience in writing compiler related stuff, with compilers that are
yet to mature to recent standards, in Fortran!





(I think that complete specifications and working prototypical
implementation very useful for arguing the case around language evolution.)

[Wolfgang Kilian]

On 12.05.2016 22:55, Ian Harvey wrote:
> On 2016-05-12 7:28 PM, Clive Page wrote:
>> On 12/05/2016 09:42, Wolfgang Kilian wrote:
>>> This appears to be what is said in that paper. From Clive's comment, I
>>> understand that progress is unlikely to happen soon, unfortunately.
>>
>> Well, the meeting at which I spoke was that of tbe BCS Fortran Special
>> Interest Group so only represented a tiny fraction of the world's
>> Fortran users. I understand that there are currently no plans for
>> improved character-handling in the current drafts for the next standard.
>>
>> Part of the problem is that Fortran users, and even more so the
>> Standards Committee members, are concentrating on extending Fortran for
>> big number crunching applications, hence all the work on coarrays etc.
>> String-handling is pretty unimportant in this context, so it will be
>> hard to drum up enthusiasm unless the changes to the language are minor
>> with no adverse consequences on existing code.
>>
>>> And, no, allocatable character is not a solution at all. It is not
>>> possible to handle an array of such objects, naturally varying length
>>> among elements, without extra glue code. In practice, this issue is
>>> much worse than the conversion functions currently required for
>>> varying_string.
>>
>> I agree, the extra glue needed is rather significant.
>
> I disagree about the significance. If the wrapped component is public,
> the "glue code" is typically just a component reference, at most.

The glue code is in the inevitable declaration for the wrapper type, and
making this declaration available to client code. The component
reference is no worse than an explicit conversion function call, agreed.
In effect, packing allocatable character objects is as costly as using
iso_varying_string from the beginning, and it becomes equivalent if the
internal representation of iso_varying_string happens to be allocatable
character. Now, iso_varying_string was available before allocatable
character was introduced, so ....

Fortran doesn't provide a convenient container facility if the container
contents vary in any property other than value. Yes, the problem is
easily solved by introducing a wrapper type, a new one for each
situation. But the solution feels like reinventing the wheel each time,
cluttering code with structure without content. A good proposal would
solve this core problem in a way that cooperates nicely with existing
array syntax and semantics.

[Dan Nagle]

Hi,

On 2016-05-12 15:31:22 +0000, Wolfgang Kilian said:

>
>
>> * can some or all of you - Wolfgang Kilian, Ian Harvey, Clive Page,
>> Stefano Zaghi, etc. - join the committee and bring more balance?
>
> AFAIK, members have been representatives of major players (vendors) in
> the Fortran world, which makes perfect sense. Some of the current and
> former members read this group and post here frequently, so I don't
> think that ideas disappear unnoticed. I don't know if any of the
> others have been involved more closely with the committee's agenda.

There is no need to be affiliated of a major player to work on the committees.

WG5 is the international policy-setting committee. Significant new features
require approval by WG5. WG5 meets once a year. This year is in Boulder
during June 6 - 10. Details are on www.j3-fortran.org

J3 (officially PL22.3) is the technical implementation committee. J3 takes
direction from WG5 and produces the standard. J3 meets three times a year,
once with WG5 and twice more. J3 usually meets in Las Vegas (inexpensive,
easy travel (for many), and it's a place equipped to handle visitors).

I was self-funding, representing myself, for about 12 years, now I
represent NCAR on J3.

If you have any desire to join J3, contact me.

The obligation is that you _must_ participate, or the standards rules require
that your membership is ended. There are some travel costs, and it
takes some time.
And you must pay (IMNSHO absurd) INCITS membership fees (around 2K$ per year).

Personal initiative, some time and money, and a desire to contribute
are the most important prerequisites.

New members are more than welcome.

--
Cheers!

Dan Nagle

[Gary Scott]

On 5/11/2016 5:20 PM, Clive Page wrote:
> On 11/05/2016 17:39, Stefano Zaghi wrote:
>> Dear FortranFan,
>>
>> I have open a specific issue for comparing StringiFor to other
>> implementations, you can find it here
>>
>> https://github.com/szaghi/StringiFor/issues/7
>>

1) substring reference syntax must be maintained in any new string type
2) must be integrated with fixed length type in a natural way
3) must be an attribute specification of character type
4) must have optional specification of maximum length
5) must inter-operate naturally with fixed length strings, as they exist

character(len=*), varying :: mystring !Compiler limit with a
"reasonable" max
character(len=256), varying :: mystring !Variable with within this max

[Stefano Zaghi]

Dear Gary, thank for your feadback. I am not sure your list is a "desiderata API" or only lacks of StringiFor, I try to elaborate it

>1) substring reference syntax must be maintained in any new string type

StringiFor cannot be used directly with slicing-substring-indexes, it being a derived type, but it has a method that mimic this, e.g. "astring%slice(12, 45)" is an elemental function that returns the substring 12:45 in standard character type. I know it isvverbose, it is different from standard substring... but I feel that it is the mamixum we can obtain with a user derived type.

>2) must be integrated with fixed length type in a natural way

StringiFor sring instances do this almost well (IMHO): concatenation, buitins procedurs, IO can be seamless mixed with standard character type.

>3) must be an attribute specification of character type

Sorry, but I cannot help on this, I am a poor Fortraner not a commitee member defining new standard.

>4) must have optional specification of maximum length

Very nice suggestion. It could be feasible at the cost of more memory storage for string having this limit. I could try to implemdnt it.

>5) must inter-operate naturally with fixed length strings, as they exist

>character(len=*), varying :: mystring !Compiler limit with a "reasonable" max
>character(len=256), varying :: mystring !Variable with within this max

I cannot provides such a syntax (obviously), but I can try to implement the concept.

Thank you for your help,

My best regards.

[Clive Page]

On 14/05/2016 20:44, Gary Scott wrote:
> 1) substring reference syntax must be maintained in any new string type
> 2) must be integrated with fixed length type in a natural way
> 3) must be an attribute specification of character type
> 4) must have optional specification of maximum length
> 5) must inter-operate naturally with fixed length strings, as they exist
>
> character(len=*), varying :: mystring !Compiler limit with a
> "reasonable" max
> character(len=256), varying :: mystring !Variable with within this max

I think you may be right - the simple integration of the current
ISO_VARIABLE_STRING into the language will not be good enough in the
long term, as it will not solve problem (1).

So I would support your syntax change allowing the "varying" component.
People will object that this is a special feature only applying to
strings, but in Fortran 77 the LENGTH component was also special, but
now accepted as part of the language.

Only one query: why do you want an optional maximum length
specification? This brings with it the risk of inadvertent string
truncation, which is one of the features of fixed-length strings that I
think is undesirable.



--
Clive Page

[dpb]

On 05/15/2016 7:51 AM, Clive Page wrote:
...

> Only one query: why do you want an optional maximum length
> specification? This brings with it the risk of inadvertent string
> truncation, which is one of the features of fixed-length strings that I
> think is undesirable.

One thing I can think of in an application with an external interface
where a faulty piece of instrumentation or line noise or whatever can
cause the terminating message character(s) to be missed...yet the device
keeps sending. (Don't ask me where/why I have the thought to suggest
this possibility...it wasn't pretty! :) but I didn't think of it a
priori happening the way it did.)

--

[Richard Maine]

dpb <no...@non.net> wrote:

> On 05/15/2016 7:51 AM, Clive Page wrote:
> ...
> > Only one query: why do you want an optional maximum length
> > specification? This brings with it the risk of inadvertent string
> > truncation, which is one of the features of fixed-length strings that I
> > think is undesirable.
>
> One thing I can think of in an application with an external interface
> where a faulty piece of instrumentation or line noise or whatever can
> cause the terminating message character(s) to be missed...yet the device

> keeps sending....

I could be misunderstanding the reasons (particularly since I don't find
myself agreeing that it is an important requirement), but I tend to
think of this sort of string as a compromise for facilitating a
particular implementation approach rather than actually wanting the
length limit. With a modest length limit, you can implement this as a
fixed amount of storage, with the system just keeping track of how much
of it you are using. Without the length limit, you are pretty much
forced to dynamic allocation.

--
Richard Maine
email: last name at domain . net
dimnain: summer-triangle

[robin....@gmail.com]

Fixed-length strings have their uses, where the length doesn't change.
In PL/I, the run-tiome checks that truncation doesn't occur,
and if truncation does occur, issues a warning.

Also in PL/I, varying-length strings are permitted, with obvious advantages in
string processing.

I've already pointed out in this thread that the lengths of array elements
can be different in PL/I.

[Gary Scott]

There are situations where I want the compiler/RT to warn me of attempt
to exceed the limit. I would like to add some template parsing
procedures as well, ala REXX, but I expect to be dead before any of
these things happen.

[FortranFan]

On Sunday, May 15, 2016 at 10:03:29 AM UTC-4, Richard Maine wrote:

> ..
>
> I could be misunderstanding the reasons (particularly since I don't find

> myself agreeing that it is an important requirement), ..


What prevents the Fortran standards committee from making the character substring and array section designator as a generic capability which can be overloaded by the programmer, similar to other unary/binary operators and assignment? Sure the terminology is different in the language as described in the current standard, but is there any room to work with it and extend the capabilities?.

Such an option will deliver far more useful benefits, I think at the moment, to users needing 'jagged arrays' of objects in general, the existing facility of derived type to create a 'container' for a character variable to mimic a string-like feature being just one specific instance under discussion in this thread.

If the language can extend the generic operator/assignment capabilities further into the designator domain and allow flexibility as shown below, it can prove very useful:

-- begin pseudo code --
..
type, public :: foo_t
private
..
end type foo_t

type, public :: bar_t ! container for foo_t
private
type(foo_t), allocatable :: m_foo(:) ! type/class of foo_t
..
contains
private
procedure, pass(this) :: foo_array_section_handler
generic, public :: designator(:) => foo_array_section_handler ! NOTE: new keyword DESIGNATOR
..
end type bar_t
..

contains

elemental subroutine foo_array_section_handler(lhs, istart, iend, rhs)

class(foo_t), intent(inout) :: lhs
integer, intent(in) :: istart
integer, intent(in) :: iend
type(foo_t), intent(in) :: rhs

..

end subroutine foo_array_section_handler

end module

..
type(bar_t) :: bar(N)
..
!.. bar(x:y) in code will now function per the coder's definition of
! foo_array_section_handler bound to type bar_t

-- end pseudo code --

If one keeps the above pseudo code snippet in mind, the point is foo_t can be anything, even an intrinsic type such as the Fortran CHARACTER variable. The idea will be that the new DESIGNATOR keyword (or some other token) can be bound to user procedures that would be required to follow a particular set of rules and constraints, similar to generic operators in the current language, starting perhaps with the requirement the procedure needs to be ELEMENTAL (and even perhaps with derived types which have a SINGLE, PRIVATE component with a LENGTH attribute?).

I don't think the exact details are as important right now, rather it will help if those who steer the technical aspects of the language can take an unbiased look at the basic concept and contemplate what is achievable and for what all use cases.

The immediate use case is the ISO_VARYING_STRING or any other string library offered by wonderful contributors such as Stefano Zaghi where the developer(s) might want to implement:

-- begin pseudo code --
type, public :: string_t
private
character(kind=XX,len=:), allocatable :: m_s
..
contains
private
procedure, pass(this) :: substring_handler1
procedure, pass(this) :: substring_handler2
..
generic, public :: designator(:) => substring_handler1, substring_handler2, ..
..
end type
..

contains

elemental subroutine substring_handler1(lhs, istart, iend, rhs)

class(string_t), intent(inout) :: lhs
integer, intent(in) :: istart
integer, intent(in) :: iend
character(kind=XX,len=*), intent(in) :: rhs

..

end subroutine substring_handler1

elemental subroutine substring_handler2(lhs, istart, iend, rhs)

character(kind=XX,len=:), allocatable, intent(out) :: lhs
integer, intent(in) :: istart
integer, intent(in) :: iend
type(string_t), intent(in) :: rhs

..

end subroutine substring_handler2

end module m

..
type(string_t) :: s1
type(string_t) :: s2(4)
character(len=:), allocatable :: s3(:)
..
s1(1:3) = 'xyz' !.. Coder defined substring handler kicks to assign 'xyz' to string_t
..
s3 = s2(3:4)(1:2) !.. Combination of normal Fortran array section in combination with user-defined
! substring handler to create a rank-1 Fortran character array of shape 2 with
! each component a Fortran character variable of length 2 with values as
! returned for coder defined handler

-- end pseudo code --

If the committee can make something like above fly, then it takes away the only limitation of ISO_VARYING_STRING which is the lack of support for substring notation that Fortran coders have gotten so used to with CHARACTER variables. And this can help avoid a bigger problem of adding to or modifying the system of intrinsic types in the language, even if just for CHARACTER type as suggested in this thread and elsewhere, which is understandably a very big deal.

In addition, the above suggestion can allow coders to develop CONTAINERs of all kinds of objects - for lists, stacks, trees, queues, maps, etc. - and users can consume such container classes with array section syntax; and perhaps over time, such objects can gain the same kind of optimization benefits that ordinary arrays provide in Fortran.

[Clive Page]

I don't see what's wrong with dynamic allocation - works well enough
with arrays.

And for the case of faulty instrumentation - well the option of a fixed
length string will surely still be available, as Fortran Standards
rarely delete a feature.


--
Clive Page

[Ian Harvey]

On 2016-05-16 5:28 AM, FortranFan wrote:
> On Sunday, May 15, 2016 at 10:03:29 AM UTC-4, Richard Maine wrote:
>
>> ..
>>
>> I could be misunderstanding the reasons (particularly since I don't
>> find myself agreeing that it is an important requirement), ..
>
>
> What prevents the Fortran standards committee from making the
> character substring and array section designator as a generic
> capability which can be overloaded by the programmer, similar to
> other unary/binary operators and assignment? Sure the terminology is
> different in the language as described in the current standard, but
> is there any room to work with it and extend the capabilities?.

I think work in this sort of arena, which has much broader applicability
than simplifying use of character objects, would ultimately be more
productive. More work to specify and implement, but far greater benefit.

Rather than try and squeeze specific solutions into the base language,
add instead facilities that enable programmers to extend the language by
writing and using libraries.

[Ian Harvey]

On 2016-05-13 7:41 PM, Wolfgang Kilian wrote:
...

> Fortran doesn't provide a convenient container facility if the container
> contents vary in any property other than value. Yes, the problem is
> easily solved by introducing a wrapper type, a new one for each
> situation. But the solution feels like reinventing the wheel each time,
> cluttering code with structure without content. A good proposal would
> solve this core problem in a way that cooperates nicely with existing
> array syntax and semantics.

I agree.

I find that much of the re-invention is associated with the need to
duplicate aspects of the interface of the thing being wrapped in the
wrapper type. The ability to link the interface of a wrapper type to a
particular component would help. (Somewhat similar, but not identical,
to how an extension type satisfies the interface of a parent type.)

[Gary Scott]

For practical usage, it is necessary to be able to go round trip to disk
IO and to have a mechanism (non-portable assumed) to read a variable
length string back in. I'm hoping there is a length value specified
somewhere in the construct (structure) that it will be possible to read
from the file in order to know how to input the string properly. This
isn't merely an internal facility. You have to make it practical for
round trip IO.

[Stansfield Temmelmeier]

On 2016-05-15, Clive Page <use...@page2.eu> wrote:
> On 15/05/2016 15:03, Richard Maine wrote:
>> dpb <no...@non.net> wrote:
>>
>>> On 05/15/2016 7:51 AM, Clive Page wrote:
>>> ...
>>>> Only one query: why do you want an optional maximum length
>>>> specification? This brings with it the risk of inadvertent string
>>>> truncation, which is one of the features of fixed-length strings that I
>>>> think is undesirable.
>>>
>>> One thing I can think of in an application with an external interface
>>> where a faulty piece of instrumentation or line noise or whatever can
>>> cause the terminating message character(s) to be missed...yet the device
>>> keeps sending....
>>
>> I could be misunderstanding the reasons (particularly since I don't find
>> myself agreeing that it is an important requirement), but I tend to
>> think of this sort of string as a compromise for facilitating a
>> particular implementation approach rather than actually wanting the
>> length limit. With a modest length limit, you can implement this as a
>> fixed amount of storage, with the system just keeping track of how much
>> of it you are using. Without the length limit, you are pretty much
>> forced to dynamic allocation.
>
> I don't see what's wrong with dynamic allocation - works well enough
> with arrays.

For strings it's needlessly expensive. Most languages I know with good or ok
string support use internally string of 0 - max length. This solves many
problems. Performance is very good. Overlay is impossible. Code is efficient
to move strings of known length. You must not search for C null char.

Stan

[dpb]

On 05/15/2016 3:41 PM, Clive Page wrote:
> On 15/05/2016 15:03, Richard Maine wrote:
>> dpb <no...@non.net> wrote:

...

> And for the case of faulty instrumentation - well the option of a fixed
> length string will surely still be available, as Fortran Standards
> rarely delete a feature.

Surely; I was "just thinkin'" and reminded of the situation.
Although where the issue arose wasn't actually in the Fortran (this was
all written long before ALLOCATABLE) but in the C vendor library for the
instrument. But, if the same idea had been used with a variable length
string, could see it causing the same or similar symptom.

--

[Stefano Zaghi]

Dear All,

I have pushed some improvements on StringiFor v0.8.4 master branch at https://github.com/szaghi/StringiFor

The most important documentation updates are the following:

+ a more detailed comparison with other approaches
https://github.com/szaghi/StringiFor#comparison-to-other-approaches
https://github.com/szaghi/StringiFor#comparison-results
https://github.com/szaghi/StringiFor#stringifor-peculiarities
I hope to have correctly cited the works of Clive and Ian
+ an updated list of the 'string' class methods
https://github.com/szaghi/StringiFor#methods-api
+ an example of how implement a (naive) csv parser in 9 statements:
https://github.com/szaghi/StringiFor#a-naive-csv-parser

Unfortunately, I presently support only GNU gfortran 6.x trunk. Intel Fortran v16.0.3 works better (support also defined IO). I would like to recall that the licensing system is very permissive (GPL, BSD, MIT...), essentially do whatever you want with it.

My best regards.

[entropy]

Renewed congratulations. Please note that the "compiler support" section of the relevant github page
https://github.com/szaghi/StringiFor
states "GNU v5.2.0+". Is it a typo?

Cheers,

Emanuele

[Stefano Zaghi]

Yes, there are many typos...

Thank you Emanuele, tomorrow I will fix this one.

My best regards.

[Stefano Zaghi]

To whom it may concern,

I have re-introduced support for GNU gfortran 5.3.0 version.

My best regards.

[Thomas Koenig]

Stansfield Temmelmeier <s...@optika.de> schrieb:

> Also needed are true pointers

What features do you think current Fortran pointers lack?
For example, I rather like the lack of pointer arithmetic (and
the lack of need to use it :-)

> and unsigned integers in various sizes.

Unsigned types are full of pitfalls. They could be useful as a
bucket of bits of a predefined size (which could also be handled
another way). Otherwise, I think they offer no features which
are really necessary, and a few traps and pitfalls.

See, for example

http://www.soundsoftware.ac.uk/c-pitfall-unsigned

or

https://stackoverflow.com/questions/30395205/why-are-unsigned-integers-error-prone

[vladim...@gmail.com]

> Unsigned types are full of pitfalls. They could be useful as a
> bucket of bits of a predefined size (which could also be handled
> another way). Otherwise, I think they offer no features which
> are really necessary, and a few traps and pitfalls.

Their important feature is the permitted overflow and underflow. Many algorithms now silently pretend that it is allowed with Fortran integers and do it anyway. And then one day they are run on a compiler which checks it and it fails.

[Wolfgang Kilian]

I remember that I wrote a MD5 implementation in Fortran which suffered
from this problem.

Fortran does support bit-manipulation functions (IAND etc.) which
suggests that integers are to be considered as bit buckets in certain
context. Insisting on sign and not permitting overflow is somehow
contradictory to this interpretation.

[herrman...@gmail.com]

On Monday, June 6, 2016 at 3:33:58 AM UTC-7, vladim...@gmail.com wrote:
> > Unsigned types are full of pitfalls.

(snip)

> Their important feature is the permitted overflow and underflow.
> Many algorithms now silently pretend that it is allowed with Fortran integers
> and do it anyway. And then one day they are run on a compiler which
> checks it and it fails.

I suppose, but they might also assume twos complement overflow.

Both C and Fortran allow for sign magnitude and ones complement.

Fortran (but not C) allows for any integer base greater than one.

Since programs do assume silent overflow, it is usual for Fortran implementations
to allow it if possible. Users will have to adapt for non-twos-complement machines.

[FortranFan]

On Wednesday, May 11, 2016 at 10:24:47 AM UTC-4, Stefano Zaghi wrote:

> .. now I roll out my own library that I like to share with the community.
>
> My library, StringiFor, is hosted here:
>
> https://github.com/szaghi/StringiFor
>
> ..


Hello Stefano,

AT the StringiFor site on GitHub, you mention:

"GNU partial support

GNU gfortran does not support user-defined-type-IO, thus some class features are disabled if GNU is used."

You have seen this recent thread at comp.lang.fortran:
https://groups.google.com/forum/#!topic/comp.lang.fortran/dNkR9JC4us8

Have you seen the latest post from Jerry Delisle On Thursday, September 29, 2016 at 9:33:45 PM UTC-4:
"On 23rd of September, the internal units portion of the UD-DTIO was committed to gfortran trunk, aka Version 7.

The gcc git mirror is here: https://gcc.gnu.org/wiki/GitMirror for those who may want to git it.

Otherwise got here: https://gcc.gnu.org/ and look on the roght side for SVN Read accessn and above that instructions to build it under https://gcc.gnu.org/install/

Testing and bug reports much appreciated.

Jerry "

Perhaps you can test StringiFor with the latest gfortran update and feedback any issues to the GCC folks? And plan to update the GitHub site for StringiFor following successful testing?

Regards,

[Stefano Zaghi]

Dear FortranFan,

you are reading my brain (a poor trip :-).

I have compiled the GNU gcc 7.0.0 trunk last weak. I have tried it with few, simple tests. This week I was out of office, bht the next one I have planned to test it with StringiFor.

As soon as possible I'll report my feedback.

As always thank for your suggestions, they are very appreciated.

My best regards.

[nobat]

Dear Stefano

I tried to compile your code (nice job! thank you for sharing) with Intel Fortran compiler on Windows (Inter Visual Fortran 2017, 2016.3 and 2016.4). They all generate internal compiler error. I reported back on Intel Visual Fortran Forums (https://software.intel.com/en-us/comment/1880574#comment-1880574). I am waiting for some response.

I was wondering how Intel Compiler didn’t fail on Linux platform supposing that there should be common compiler internals for both platforms (don’t know anything about compiler internals – just wondering)

Thank you again
Sam

[Stefano Zaghi]

Il giorno lunedì 3 ottobre 2016 14:43:02 UTC+2, nobat ha scritto:
> Dear Stefano
>
> I tried to compile your code (nice job! thank you for sharing) with Intel Fortran compiler on Windows (Inter Visual Fortran 2017, 2016.3 and 2016.4). They all generate internal compiler error. I reported back on Intel Visual Fortran Forums (https://software.intel.com/en-us/comment/1880574#comment-1880574). I am waiting for some response.
>
> I was wondering how Intel Compiler didn’t fail on Linux platform supposing that there should be common compiler internals for both platforms (don’t know anything about compiler internals – just wondering)
>
> Thank you again
> Sam

Dear Sam,

you are too much kind, thank you very much.

I am a *Nix man, I have not access to MS Windows, thus your test is full of significance.

I offer a few comments due to my "OCD" soul.

I see your report on Intel forum, thank you very much again. I have not the time to check the example you posted there, but I like to advice you, StringiFor depends on:

+ PENF (https://github.com/szaghi/PENF) and
+ BeFoR64 (https://github.com/szaghi/BeFoR64)

thus you have to download them manually if you have not cloned (recursively) the git repository. The "internal compiler error" seems to exclude a simple "module not found" issue, but I learned here to not guess.

If you want to be sure to download all dependencies, please use git:

$ git clone --recursive https://github.com/szaghi/StringiFor
$ cd StringiFor
$ git submodule update --init --recursive

These commands will download all sources you need. Moreover, I see you are using Visual Studio that (I assume) does not use the provided makefile, rather it uses its built-in "intelligence". In this case, be sure that Visual Studio "views" also PENF and BeFor64.

These are just my "improbable" concerns due to the fact that is more probable that my code is bugged rather than Intel Compiler, but never say never.

My best regards.

[nobat]

Thank you for your input Stefano
Yes, I have included your other projects (PENF and BeFore64). I ported the entire projects FiNer/StringiFor/BeFoR64/PENF projects and their test cases to Intel Visual Fortran (Actually, I was aiming for your FiNer project, that brought me back to your StringiFor project and of course PENF and BeFoR64).

On Windows :
PENF and its test cases are working properly (with proper pre-processing)
BeFoR64 and its test case are working properly (though the results are not exactly the same as you coded in your test cases)
StringiFor  Fails on internal compiler error
FiNer  could not test it

Once the problem of compiler internal error is resolved I can contribute to your projects with Windows solution files .

Thank you again Stefano (great job)

[Stefano Zaghi]

Il giorno lunedì 3 ottobre 2016 17:39:33 UTC+2, nobat ha scritto:
> Thank you for your input Stefano
> Yes, I have included your other projects (PENF and BeFore64). I ported the entire projects FiNer/StringiFor/BeFoR64/PENF projects and their test cases to Intel Visual Fortran (Actually, I was aiming for your FiNer project, that brought me back to your StringiFor project and of course PENF and BeFoR64).
>
> On Windows :
> PENF and its test cases are working properly (with proper pre-processing)
> BeFoR64 and its test case are working properly (though the results are not exactly the same as you coded in your test cases)
> StringiFor  Fails on internal compiler error
> FiNer  could not test it
>
> Once the problem of compiler internal error is resolved I can contribute to your projects with Windows solution files .
>
> Thank you again Stefano (great job)

Dear Sam,

you are great! Thank you very much for your feedback! You are right, I have "broken" FiNeR backward compatibility when I added StringiFor, sorry for the issue.

Cheers.

[Steve Lionel]

On 10/3/2016 8:42 AM, nobat wrote:
> I tried to compile your code (nice job! thank you for sharing) with Intel Fortran compiler on Windows (Inter Visual Fortran 2017, 2016.3 and 2016.4). They all generate internal compiler error. I reported back on Intel Visual Fortran Forums (https://software.intel.com/en-us/comment/1880574#comment-1880574). I am waiting for some response.
>
> I was wondering how Intel Compiler didn’t fail on Linux platform supposing that there should be common compiler internals for both platforms (don’t know anything about compiler internals – just wondering)

The compiler error occurs only if you enable bounds checking, which
isn't the default. You were using a Debug configuration in Visual Studio
which turns that on. If you disable bounds checking, the example you
supplied does build. I have sent this on to the developers.

--
Steve Lionel
Developer Products Division
Intel Corporation
Merrimack, NH

For email address, replace "invalid" with "com"

User communities for Intel Software Development Products
http://software.intel.com/en-us/forums/
Intel Software Development Products Support
http://software.intel.com/sites/support/
My Fortran blog
http://www.intel.com/software/drfortran

Refer to http://software.intel.com/en-us/articles/optimization-notice
for more information regarding performance and optimization choices in
Intel software products.

[Stefano Zaghi]

Il giorno lunedì 3 ottobre 2016 18:38:08 UTC+2, Steve Lionel ha scritto:
> The compiler error occurs only if you enable bounds checking, which
> isn't the default. You were using a Debug configuration in Visual Studio
> which turns that on. If you disable bounds checking, the example you
> supplied does build. I have sent this on to the developers.
>
> --
> Steve Lionel
> Developer Products Division
> Intel Corporation
> Merrimack, NH
>
> For email address, replace "invalid" with "com"
>
> User communities for Intel Software Development Products
> http://software.intel.com/en-us/forums/
> Intel Software Development Products Support
> http://software.intel.com/sites/support/
> My Fortran blog
> http://www.intel.com/software/drfortran
>
> Refer to http://software.intel.com/en-us/articles/optimization-notice
> for more information regarding performance and optimization choices in
> Intel software products.

Dear Steve and Sam,

I am very sorry, I was aware about the check-bounds issue, I completely forget to mention this to Sam in my previous concerns list, my bad.

I did not pushed the check-bounds issue to Intel Forum because I argued that providing StringiFor as a "minimal working example" raising the issue was not accepted by Intel developers, the sources being too long.

Steve, can I argue that you accept also "not so minimal examples" raising some issue? I could have some others...

Thank very much to both of you.

My best regards.

[Steve Lionel]

On 10/3/2016 12:59 PM, Stefano Zaghi wrote:
> Steve, can I argue that you accept also "not so minimal examples" raising some issue? I could have some others...

Yes, of course. Any issues you have with the Intel compiler you are
welcome to report in our user forum (link below) or using Intel Premier
Support (if you have a support license.) Please don't rely on me
spotting things in c.l.f. and I would not suggest sending me mail either
as I might not be around to read it.

[nobat]

Thank you

[nobat]

No problem Stefano ... Later, I will check your codes with suggested compiler options...
Thank you

[Stefano Zaghi]

Dear Steve,

>Yes, of course. Any issues you have with the Intel compiler you are
welcome to report in our user forum (link below) or using Intel Premier
Support (if you have a support license.)

Wonderful, in the past the length of my "examples" prevent me to post on Intel forum, I promise I'll report all from mow, altough I have the license for FOSS developing not a premiere one.

> Please don't rely on me
spotting things in c.l.f. and I would not suggest sending me mail either
as I might not be around to read it.

Sure, absolutely, when I said "you accept..." I meant "Intel" not "Steve" :-) Anyhow, I like to tell you that your "presence" here is a wonderful, great resource for me (and I bet I am not alone).

My best regards.

P.S. Sam if you are using FiNeR, feel free to ask any bug-fixes/new-features you want, I'll be happy to try to support you.

[Steve Lionel]

On 10/3/2016 3:46 PM, Stefano Zaghi wrote:
> Wonderful, in the past the length of my "examples" prevent me to post on Intel forum, I promise I'll report all from mow, altough I have the license for FOSS developing not a premiere one.

Gzipped tar files are fine. Do what you can to reduce the test case,
we'll take it from there.

--
Steve Lionel
Developer Products Division
Intel Corporation
Merrimack, NH

For email address, replace "invalid" with "com"

User communities for Intel Software Development Products

https://software.intel.com/en-us/forums/

Intel Software Development Products Support

https://software.intel.com/sites/support/

My Fortran blog
http://www.intel.com/software/drfortran

Refer to https://software.intel.com/en-us/articles/optimization-notice