Pike 8.0 release 15 (8.0.1738) now available

[wil...@welliver.org]

Hello, Friends of Pike!

The Pike development team is pleased to announce the 15th release of
Pike 8.0: build 8.0.1738. I’m in the process of submitting updates for
homebrew and pkgsrc, so the new version should appear there in the near
future.
As always, bug reports, comments and suggestions are welcome!

Bill

----

Source:

https://pike.lysator.liu.se/pub/pike/all/8.0.1738/Pike-v8.0.1738.tar.gz

Binary builds:
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1738/

RPMs for RHEL/CentOS:
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1738/el6
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1738/el7
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1738/el8

Windows Installer:

https://pike.lysator.liu.se/pub/pike/all/8.0.1738/Pike-v8.0.1738-Windows7-x86.msi

Changes since Pike 8.0.1116 (release 14)

New Features
------------

o Calendar
Updated to use tzdata2021e.

o Parser.XML.Tree
Added set_text() to update the contents of a text node.

o Sql.Sql
Add query_db() to API.

o SSL.SSLFile
Add {query,set}_timeout(). This adds an API to set an optional timeout
to be
used for blocking operations. When the timeout expires it is handled
as if
the peer sent a fatal internal error alert.

o Standards.PEM
Added helper methods, including: get_certificates() ,
get_private_keys(),
get_private_key() and decrypt_fragment().

o Runtime and EFUNS
- describe_backtrace() now reports cpu time --with-profiling.
- decode_value can be called with codec -1, which disables decoding of
functions, objects, programs and types.

Bug Fixes
---------

o crypt
Fixed a bug which caused crypt() with a random salt to sometimes
return
"*0" instead of an encrypted password.

o Concurrent
- The future generated by ->timeout() would not necessarily resolve
with
the fullfilled or rejected values if the original Promise/Future was
resolved or rejected before hitting the timeout.
- If the original promise resolves or rejects, we now attempt to
unschedule the timeout call_out from the backend we scheduled it in,
not the one (now) configured in the Future.

o Critbit
- Fix a number of memory and behavior errors.

o Inotify
add_watch() now returns -1 for filsystem-related failures
that may be due to the filsystem being in flux.

o Image.BMP
Improved size check better detects malformed BMP images.

o Image.GIF
- Improve detection of malformed LZW bit sizes.
- Improve detection of malformed LZW dictionary extensions.

o Image.ILBM
- Guard alloca() allocations with check_c_stack().
- Improved length accounting in unpackByteRun1().

o Image.JPEG
- Do not over-allocate memory for decoded image.
- Limit decoded size of images to 2 GB.

o Image.PCX
Improved size checks better detect malformed PCX images.

o Image.PNG
Improved size check better detects malformed PNG images.

o Image.PSD
Improved size check better detects malformed PSD images.

o Image.TGA
- Improve detection of malformed TGA images.
- Correctly update pointer when horizontal or vertical flipping is
requested by the TGA image.

o Image.TIFF
Improved size checks in image decoding

o Image.TIM
Improved size checks better detect malformed TIM images.

o Image.WBF
Improved size check better detects malformed WBF images.

o Image.X
Guard alloca() allocations with check_c_stack().

o Image.XCF
Fixed to properly initialize the layer mask structure.

o Image.XWD
Improved size check better detects malformed XWD images.

o Protocols.LDAP.client
LDAP connections secured with TLS/SSL now timeout if the server does
not
respond within 10 seconds.

o Sql.mysql
Improved character set handling

o Sql.pgsql
Numerous bug fixes.

Building & Tools
----------------

o Improved check for stack direction to also report correct results when
compiling with Clang 11.

o _TIME_BITS is now set to 64, to get a 64-bit time_t on 32-bit systems
running glibc 2.34 or newer.

o Improvements to building and installer code signing on macOS.

o Fixes to better support building with future versions of GNU make.

o Use updated FFI API in Java module, to better support recent libffi.

o Fix testsuite generation for BSD m4 users; POSIX systems must continue
to use GNU m4 (gm4).

[David Estopa]

Hello, Friends of Pike!

On Pike 8.0 1116,   have anybody experienced similar malloc error trying
to include several .h files into a program?

  malloc(): invalid next size (unsorted)
  corrupted size vs. prev_size

My program includes several .h to define several languages constants. If
I include 18 works fine, but cannot include one more.

Is there a max size limit to include files into a pike program?

Thanks in advance.

David Estopà

[wil...@welliver.org]

Hi David-

I'm not aware of a limitation... is it possible you have a circular
reference somewhere? Do you suspect an overall size limitation, or do
you think it's because of the number of includes? Removing the content
from one or more of the files might give an indication of one vs the
other.

Is this a problem you're discovering right after switching to 8.0.1116,
or have you been using that version for a while?

Bill

[H William Welliver]

Just a quick update: I wrote a (basically empty) program that included 20 empty files and had no problems on 8.0.1116. So there must be something more to it. If you can find a minimal test case, we might have better luck.

Bill

[David Estopa]

Hi, William.

Thaks for your response. I appreciated it.

I'm triing to reproduce the problem with a more basic program, and some
constants autogenerated, but we obtain another diferent error or
limitation on the tests.

It seems that there is a limit when you define more than 64K constants
with .h files:

Up to 64K constants a pike will compile correctly  (32 files .h with
2001 constants each one) .

After include the 33rth file, seems that exceeds the current capacity:
'too many identifiers'.

datos_33.h:1549:Too many identifier_references.
datos_33.h:1550:Too many identifiers.
Violación de segmento (`core' generado)

I've done the test in two systems:
  Rocky   with    4GB RAM  Pike v8.0 release 1116
  Mint19 with  15GB RAM  Pike v8.0 release 498

If I compare this situation whith the other one, that results in the
malloc error (corrupted size vs. prev_size) is not really comprensible,
because the program that fails
does not exceed these limits, but fails when I add a new .h.


** malloc error    (corrupted size vs. prev_size)
number of constants in .h files:  1545
size of .h files .........................:   680KB


** max constants error  (Too many identifiers.)
number of constants in .h files....: Fine up to 64.000 (up to 33.h)
total size for the 40 .h files..........:   298MB

As you can see I'm getting the malloc error with only 1500 constants
defined (<64K)  and no more thant 1MB of data payload.

Below you have the content of 3 files to autogenerate the .h payload and
test these situations:
First run with includes commented to generate a payload.
Then run with the include.h enabled to see the result.

$ cat include_0.h
constant SALTO="\n\n";


$ cat includes.h
#ifndef INCLUDES_H
#define INCLUDES_H
#include "./datos_1.h"
#include "./datos_2.h"
#include "./datos_3.h"
#include "./datos_4.h"
#include "./datos_5.h"
#include "./datos_6.h"
#include "./datos_7.h"
#include "./datos_8.h"
#include "./datos_9.h"
#include "./datos_10.h"
#include "./datos_11.h"
#include "./datos_12.h"
#include "./datos_13.h"
#include "./datos_14.h"
#include "./datos_15.h"
#include "./datos_16.h"
#include "./datos_17.h"
#include "./datos_18.h"
#include "./datos_19.h"
#include "./datos_20.h"
#include "./datos_21.h"
#include "./datos_22.h"
#include "./datos_23.h"
#include "./datos_24.h"
#include "./datos_25.h"
#include "./datos_26.h"
#include "./datos_27.h"
#include "./datos_28.h"
#include "./datos_29.h"
#include "./datos_30.h"
#include "./datos_31.h"
#include "./datos_32.h"
#include "./datos_33.h"
#include "./datos_34.h"
#include "./datos_35.h"
#include "./datos_36.h"
#include "./datos_37.h"
#include "./datos_38.h"
#include "./datos_39.h"
#include "./datos_40.h"
#endif


$ cat includes.pike
#!/usr/bin/pike
#include "./include_0.h"

//Once autogenerated the payload, uncomment next line to include the of
h files:  datos 1 a 40
//  #include "./includes.h"

int main()
{
    string
PAYLOAD="ABBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
PAYLOAD=PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+
PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+
PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD+PAYLOAD;

    for (int i=1;i<=40; i++)
    {
        string f="./datos_"+i+".h";
        if (Stdio.is_file(f))
            continue;
        else
            Stdio.stdout.write("#include \"./datos_"+i+".h\"\n");

string AUX=""+
"#ifdef DATOS_"+i+"_H\n"+
"#warning Ya se ha hecho include previamente del fichero de constantes
tsa_mail_ES.h\n"+
"\n"+
"#else\n"+
"#define DATOS_"+i+"_H\n"+
"\n";
        Stdio.append_file(f, AUX);

        for (int j=0; j<2000; j++)
        {
            Stdio.append_file(f, "constant
CT_"+i+"_"+j+"=\""+PAYLOAD+"\"\+SALTO;\n");
        }

        Stdio.append_file(f, "\n#endif\n");
    }

   Stdio.stdout.write("ALL DONE!\n");

   thread_create(nothing);
   return -1;
}

void nothing()
{
while(1)
{
sleep(1);
}
}


On the test callbacks are enabled only to reproduce another running
condition.

If anyone has more ideas to try, comments will be appreciated.

Best Regards

David Estopà




El 28/4/22 a las 2:56, H William Welliver escribió: