Script to remove comments from a script file

[Harald Oehlmann]

Dear TCL'ers,

is there a script that will remove all comments from a script file ?

when I used eTCL, there was an option to strip all comments from TCL
source files. eTCL left the remaining commands and the line numbers
intact, as newlines were preserved.

The aim is to pack the script files in a DLL and the comments are anyway
never be shown up any more.

I suppose, it is a non-resolvable problem in general to get all
comments. But to get some should be possible.

So:

# Comment 1

proc name {args} {
# Comment 2
}
apply {fun arg} {
# Comment 3
} test

array set a {
# nocomment
}

set d 44 ; # Comment 4

switch $t {
#nocomment { # Comment 5
}
}

Quite complex...

Thank you for any idea,
Harald

[Alexandru]

This is how I do it, but it's more primitive that what you have in mind.
Warning: The core overwrite original file (but creates a backup in advance).

file copy -force $tclfile $tclfile-orig
set fid [open $tclfile r]
fconfigure $fid -encoding utf-8
set content [read $fid]
close $fid

# Remove comments
set fid [open $tclfile w]
fconfigure $fid -encoding utf-8
set content [regsub -all {[ \t]*\n[ \t]*} $content "\n"]
set content [regsub -all {[ ]*\\[ ]*\n[ ]*} $content " "]
foreach line [split $content \n] {
set line [string trim $line]
if {[string index $line 0]=="#" || $line==""} {
continue
}
puts $fid $line
}
close $fid

[Harald Oehlmann]

Thank you, Alexandru.
That is indeed quite basic.

Thanks,
Harald

[fr]

Am Freitag, 29. Mai 2020 12:37:14 UTC+2 schrieb Harald Oehlmann:

I have following script in use, comments starting with #. are not removed, keeps original line numbers, destination directory has to exist.
Roland Frank

proc undebug {sourcedir destdir} {
set exclude_pattern {(pp|undebug|x|test)\.tcl$}
if {![file isdirectory $sourcedir]} {
error "$sourcedir is not a directory"
}
if {![file isdirectory $destdir]} {
error "$destdir is not a directory"
}
set sdir [file normalize $sourcedir]
#puts sdir=$sdir
set ddir [file normalize $destdir]
#puts ddir=$ddir
if {[string equal $sdir $ddir]} {
error "overwrite-protection"
}
set pattern [file join $sdir *.tcl]
set start [string length $sdir]
# exclude trailing /
incr start
foreach ff [glob -nocomplain $pattern] {
#puts \t$exclude_pattern
if {[regexp $exclude_pattern $ff]} {
#puts "skip $ff"
continue
}
#puts ff=$ff
set destfile [file join $ddir [string range $ff $start end]]
#puts "->$destfile"
unde $ff [file join $ddir $destfile]
}
}
proc unde {sfn dfn} {
if {[file exists $dfn]} {
file stat $dfn ds
file stat $sfn ss
if {$ds(mtime) > $ss(mtime)} {
#puts "$sfn unmodified: skip"
return
}
}
if {[string equal $sfn $dfn]} {
puts "unde $sfn $dfn >skipped"
return
}
puts "unde: using $sfn"
set fh [open $sfn r]
set cc [read $fh]
close $fh
if {[catch {set fh [open $dfn w]} err]} {
puts "error: $err"
puts "skip file $dfn"
} else {
set tmp0 [regsub -all -line {^\s*borg\s+log.*$} $cc {}]
#keep line numbering when removing comments
set tmp [regsub -all -line {^\s*#[^\.].*$} $tmp0 {}]
# remove comment lines
set tmp0 [regsub -all -line {^\s*#[^\.].*$} $tmp {}]
# remove puts without file
# but do not remove puts [time { ...}]
#
if {1} {
#set tmp [regsub -all -line {^\s*puts(\s+-nonewline){0,0}\s+[^\$\[].*$} $tmp0 {}]
set tmp [regsub -all -line {^\s*puts(\s+-nonewline){0,0}\s+[^\$\[].*$} $tmp0 {}]
puts $fh [regsub {\n+$} $tmp0 {}]
} else {
set tmp [regsub -all -line {^\s*puts(\s+-nonewline){0,0}\s+[^\$\[].*$} $tmp0 {}]
puts $fh [regsub {\n+$} $tmp {}]
}

# get vimlines
foreach line [split $cc \n] {
if {[regexp {^# vi\:} $line]} {
puts $fh $line
}
}
close $fh
}
}

[Harald Oehlmann]

Hi Roland,

thank you for the complete solution, that is appreciated.

Don't you scare to hit a line of type:

array set a {
# nocomment
}
set d 44 ; # Comment 4

switch $t {
#nocomment { # Comment 5
}

I am sure, I have those lines....

But I suppose, it is just better to write:

array set a {
"#" nocomment
}
set d 44 ; # Comment 4

switch $t {
"#nocomment" { # Comment 5
}

Thank you,
Harald

[heinrichmartin]

On Friday, May 29, 2020 at 12:37:14 PM UTC+2, Harald Oehlmann wrote:
> is there a script that will remove all comments from a script file ?
>

> ...

>
> The aim is to pack the script files in a DLL and the comments are anyway
> never be shown up any more.

Have you verified the approximate amount of saved storage (i.e. your ratio between code and comments)? And is it worth it? File compression should be worth more.

With this in mind, you could focus on block comments, i.e. {(?:\n|^)( *)#[^\n]*\n(?:\1#[^\n]*\n)+}. This should be a good trade-off to protect the #nocomments. Otherwise, you could package bytecode instead.

[Harald Oehlmann]

Thank you, Martin !

Well, only removing the first line comments would be great.
I have often 300 lines of changelog in my files so this is considerable
much data.

And compiling would be super-great. I have seen that you have
implemented it in your wrapper. That is a dream to implement that! The
complexity scares me so far. The byte compiler is a don't touch region,
but this may change.

Thank you,
Harald

[heinrichmartin]

On Sunday, May 31, 2020 at 9:49:11 AM UTC+2, Harald Oehlmann wrote:
> Am 31.05.2020 um 01:15 schrieb heinrichmartin:
> > On Friday, May 29, 2020 at 12:37:14 PM UTC+2, Harald Oehlmann wrote:
> >> is there a script that will remove all comments from a script file ?
> >>
> >> ...
> >>
> >> The aim is to pack the script files in a DLL and the comments are anyway
> >> never be shown up any more.
> >
> > Have you verified the approximate amount of saved storage (i.e. your ratio between code and comments)? And is it worth it? File compression should be worth more.
> >
> > With this in mind, you could focus on block comments, i.e. {(?:\n|^)( *)#[^\n]*\n(?:\1#[^\n]*\n)+}. This should be a good trade-off to protect the #nocomments. Otherwise, you could package bytecode instead.
> >
>
> Thank you, Martin !
>
> Well, only removing the first line comments would be great.
> I have often 300 lines of changelog in my files so this is considerable
> much data.

The regexp should match any block comment, i.e. successive commented lines (with optional but identical space indent), not just file header comments. Good to hear that it helps :-)

> And compiling would be super-great. I have seen that you have
> implemented it in your wrapper. That is a dream to implement that!

You got my name right, but I do not know what you refer to.

[Harald Oehlmann]

Am 01.06.2020 um 23:55 schrieb heinrichmartin:
>> And compiling would be super-great. I have seen that you have
>> implemented it in your wrapper. That is a dream to implement that!
>
> You got my name right, but I do not know what you refer to.
>

Sorry, it is about tclexecomp which features a -compile option.
I thought it was your baby.

Sorry,
Harald

[heinrichmartin]

No need to be. I just didn't want to take someone else's credits (nor can I help with it).