Pike 8.0 release 14
15 views
Skip to first unread message
H William Welliver
Jan 27, 2021, 1:51:21 PM1/27/21
to Pike mailinglist
Hello, Friends of Pike!
The Pike development team is pleased to announce the 14th release of Pike 8.0: build 8.0.1116. I’m in the process of submitting updates for homebrew and pkgsrc, so the new version should appear there in the near future.
As always, bug reports, comments and suggestions are welcome!
Bill
----
Source:
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1116/Pike-v8.0.1116.tar.gz
Binary builds:
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1116/
RPMs for RHEL/CentOS:
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1116/el6
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1116/el7
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1116/el8
Windows Installer:
https://pike.lysator.liu.se/pub/pike/all/8.0.1116/Pike-v8.0.1116-win32-x86.msi
Changes since Pike 8.0.702 (release 13)
New Features
------------
o Core
- Added "->?" as the safe indexing variant of "->".
- Added int(7,8,16bit).
o ADT.History
Support encode/decode_value().
o Pike.FakeObject
This is used as a placeholder used in place of other objects in backtraces.
o Pike.DestructImmediate
An empty class that can be inherited to get the PROGRAM_DESTRUCT_IMMEDIATE
flag set.
o Image.Dims.exif_get and Image.exif_decode
New convenience functions to get the functionality of
Image.Dims.exif_get_JPEG and Image.JPEG._decode (rotation of the image
according to the EXIF orientation information) without needing to check
if the image is JPEG first.
o Concurrent
- Add delay() to postpone a future.
- Rewrite timeout() to reduce the memory footprint of a future in the
common case.
- Added query_{success,failure}_callbacks() and try_get() to
Concurrent.Future.
o Process.Process
- On NT it is now possible to specify the preferred console handler
via the "conpty" option. The default is to take it from the first
pty of "stdin", "stdout", and "stderr". If none of them is a pty
the console handler is inherited from the current process.
o SSL
The SSL module now supports session ticket both client and server side.
This functionality has been backported from 8.1/master.
o Standards.X509
Standards.X509 now allows the user to select acceptable signature
algorithms to be used in the verification of certificates.
o Stdio.Buffer
Added truncate() and the ability to search for substrings.
o Stdio.File
Support openat() with a single argument().
o Stdio.PROP_TTY
Stdio.File()->pipe() can now be used to create tty-capable
pipes (aka pseudo-ttys, aka ptys).
Bug fixes
---------
o ADT.CritBit.FloatTree
Fix FloatTree when using 128bit floats.
o ADT.CritBit.IntTree
Fixed the signature of ADT.CritBit.IntTree()->_values().
values(ADT.CritBit.IntTree()) claimed to return an array(int),
whereas IntTrees can hold any type as value.
Hide mutex keys and crypto contexts from backtraces.
Having unexpected references to mutex keys may cause hangs and
complaints about attempting to lock mutexes recursively.
o combine_path()
Fixed corner case where combine_path() on strings starting with
"./../" used to gain one directory level. Eg:
Old behavior:
> combine_path(".", "../foo");
(1) Result: "../foo"
> combine_path("./..", "foo");
(2) Result: "foo"
> combine_path("./../foo");
(3) Result: "foo"
New (fixed) behavior:
> combine_path(".", "../foo");
(1) Result: "../foo"
> combine_path("./..", "foo");
(2) Result: "../foo"
> combine_path("./../foo");
(3) Result: "../foo"
o describe_backtrace()
- Do not use value identifiers that have not been defined in
the visible backtrace.
- Fix clipped/canclip propagation.
o mktime()
Fix zone handling when localtime is in daylight savings time.
o strftime()
Remove non-standard conversions %k and %l.
o types()
Fixed reference leak.
o crypt()
Do not throw errorss on validation failure. Primarily ocurrs on
RHEL 8, return a failure code instead, consistent with historical
use.
o Backend.PollBackend
Signal read events on POLLHUP.
o Calendar
- Fixed typo in the Catalan word for the month December.
- Updated tzdata to 2019c.
o Compiler
- Adhere to the calling convention more in the machine code
generator on amd64, namely by keeping the stack pointer aligned
before calling into C code. GCC 8 started to emit instructions
relying on the correct, alignment, causing General Protection
Faults.
- Fixed some corner cases where file names where missing from
backtraces.
- Check the proper flag to determine if a local variable may be used
before optimizing away certain assignments if the assignment is
immediately followed by a return of the variable in question.
o Concurrent.Promise
- Fixed race condition where success- and/or failure- callbacks
could get lost.
- Convert errors thrown by the executor passed to Concurrent.Promise()
on create into promise rejection.
- Fixes for some recursive mutex lock errors.
o Crypto.ECC
Now works against Nettle 3.5.
o Filesystem.Tar
Fix extraction of S_ISUID and S_ISGID bits.
POSIX mandates that after a chown() (by unprivileged users), the
S_ISUID and S_ISGID bits are cleared, Linux 2.2.13 removed the
special case for root. The order of chmod() and chown() have been
reordered trying to fix an issue with applying utime() under
Windows. With this change, utime() is applied first, then chown()
and then chmod(), restoring the historical order for chown() and
chmod().
o Gettext
Prevent argument of textdomain from being empty.
o Image.Dims
- Fixed the inconsistency that Image.Dims.get_JPEG (and thus
Image.Dims.get) would flip the dimensions on JPEGs with EXIF
orientation information, even though Image.JPEG.decode does not
actually perform the rotations.
o Inotify
Improved handling of invalid event callbacks. [bug 8042]
o Odbc
The module's configure script now respects ABI suffixes when searching
for the ODBC library dir.
o Parser.HTML
Fixed a Pike stack overflow occuring with certain inputs to the HTML
parser.
o Pike.count_memory
Prevents a loop with ADT.List objects.
o Process.run
- Don't error if stdin modifier is supplied as the empty string.
[LysLysKOM 23099651]
o Protoocls.HTTP
- Protocols.HTTP.do_async_proxied_method() did not pass on request
headers unless credentials were given to the proxy or the protocol
to be used was HTTPS.
- Add Host header to proxied requests.
o Protocols.HTTP.Server
Fixed bug in Request()->update_mime_var(). [bug 10033]
o Protocols.HTTP.Session
- Fix race-condition when multiple threads call give_me_connection()
with the same url at the same time.
- Don't attempt to reuse SessionQuery objects.
give_me_connection() now always returns a new SessionQuery object
(albeit sometimes with a reused connection).
This should alleviate issues with stuff being left around from
previous queries.
o Protocols.DNS
Improved error handling when calling async callbacks.
Survive Stdio.UDP()->bind() failing with EACCES.
o Protocols.HTTP.Query
On Solaris 11 Stdio.File()->connect() often fails with EADDRINUSE.
If this happens, retry the connection.
o Protocols.Websocket
Tentative fix for error 'indexing the NULL value with "set_nonblocking'.
o Runtime
- Certain operations on arrays of only objects could have triggered a type
confusion under some circumstances, causing integers to be freed as
objects, and thus a NULL derefence (SIGSEGV).
- Fix some gratuitous rounding to double with long-double-precision.
- Improved Fix some gratuitous rounding to double with
long-double-precision.
o Search.Filter.HTML
Fix title always being set to "". [bug 10020]
o Sql.pgsql
- Ensure that the db connection is closed when the portal is closed.
- Small speed boost.
- Fix the only known deadlock-cause logged in the last year; happened
(rarely) on multiple parallel queries on a single connection.
- Fix the only known race condition logged in the last year; happened
on some queries that store >32KB data per query.
- Cast integers to floats, when floats are desired.
- Simplify error handling.
- Toggle cache_autoprepared_statements default to off;
turning it on triggers a bug in PostgreSQL sometimes
that causes spikes in CPU usage of the database.
- Fix support for text-multiqueries (separated by semicolons).
- resync() is synchronous now, and prevents race conditions
if followed back-to-back with new queries.
- disregard pending statements on connection destruction.
o SSL
- When verifying the hostname against the certificate, only accept *
(wildcard) for one level, i.e. *.example.com matches foo.example.com,
but not www.foo.example.com. Previously, all levels were matched.
- No longer accept MD2, MD5, and SHA1 as signature algorithms when
verification is requested. MD2 and MD5 are outright dangerous at
this point, and SHA1 should no longer be very relevant as web
browsers already force CAs for several years to no longer issue any
SHA1 certs anymore.
Acceptable signature algorithms are however configurable via
SSL.Context now, in case support of older/insecure algorithms is
vehemently required by an application.
o SSL.File
- Do not close automatically on write error.
The internal ssl_write_callback() would call shutdown()
on write error, which would cause the object to enter
one of the closed states, without the user actually
having closed the file.
This in turn caused errors like "Not open" from
functions like read() and set_nonblocking().
- Fix a situation where setting callbacks before the TLS
handshake had completed would cause the connection to hang
if an alert was generated (for instance when the peers are
unable to agree on a cipher suite.)
- Permit data received to be read() after the connection is
closed by the peer. Previously, read would throw a "Not open"
error rendering the data irretrievable.
o Standards.JSON
- encode() now allows other threads to run every now and then.
- Fix floating point literals with long-double-precision
o Standards.EXIF
Improved detection of EXIF section in get_properties().
o Standards.URI
- Updated list of unsafe characters to RFC 3986. [bug 8004]
- String representations of the URI that are generated by the object
are now canonical again. This also fixes an issue where Standards.URI
instances compared identical under the == comparison, but would hash
differently.
o Standards.X509
Use old syntax for compatibility.
o sybase
The module's configure script now respects ABI suffixes when searching
for the sybase library dir.
o System
- System.openlog() now keeps a reference to the ident passed, since
system level syslog() may access the string pointed to by the ident
passed to openlog. The reference will be released after a subsequent
call to openlog() or closelog().
o Thread.MutexKey
When simulated, mark for immediate destruction.
o Thread.Queue
Attempt to work around glibc bugs.
Glibc 2.24 and earlier have a broken implementation of condition
variables (cf https://sourceware.org/bugzilla/show_bug.cgi?id=13165).
Attempt to work around the issue by having the signalling thread
release the associated mutex before signalling.
o Thread.ResourceCount
Fixed mutex handling.
o Tools.Standalone.httpserver
The builtin webserver tool shipped with Pike (pike -x httpserver) was
previously susceptible to a directory traversal attack via URL encoding.
Building & Tools
----------------
o Add --with-exclude-site-prefixes configure option.
This will cause any directory specified in this option to be
excluded from the search path for binaries/includes/libraries. This
option will override any paths found by other means, allowing "system"
directories like /usr/local to be exluded for making binary distributions.
o Drop support for Postgres 7.1.
o Fix compilation with Nettle 3.1.1.
o Support recent versions of libfreetype.
o Drop use of the obsolete (~20 years) Solaris ioctl /proc API.
The ioctl-based /proc API was obsoleted in Solaris 2.6 (July 1997),
but to be supported "until the next major release". In Solaris 11.4
(August 2018) the support has now finally been removed.
Most of the removed code was from 1995, and was thus older than Pike...
o Improved pkg-config search path detection on Solaris.
Solaris 11.4 provides a single pkg-config binary, which by default
searches in /usr/lib/pkgconfig/ and /usr/share/pkgconfig/.
Unfortunately, /usr/lib/pkgconfig/ contains information for
the 32-bit packages (64-bit d:o is in /usr/lib/64/pkgconfig/).
This issue caused eg 32-bit headers for libffi to be used, causing
a SIGSEGV in ffi_closure_unix64_inner() due to the differing sizes
of ffi_closure in 32-bit and 64-bit abi.
o Check for actual availability of YP functions, not just headers.
o Attempt to use paxctl to alter the PaX flags for the pike binary to
allow for generation of machine code on NetBSD.
o Silence GL deprecation warning and fix a missing definition error on
Darwin for newer versions of MacOS.
o Fix check for my_bool and compilation errors related to MySQL 8.
The Pike development team is pleased to announce the 14th release of Pike 8.0: build 8.0.1116. I’m in the process of submitting updates for homebrew and pkgsrc, so the new version should appear there in the near future.
As always, bug reports, comments and suggestions are welcome!
Bill
----
Source:
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1116/Pike-v8.0.1116.tar.gz
Binary builds:
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1116/
RPMs for RHEL/CentOS:
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1116/el6
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1116/el7
https://pike.lysator.liu.se/download/pub/pike/all/8.0.1116/el8
Windows Installer:
https://pike.lysator.liu.se/pub/pike/all/8.0.1116/Pike-v8.0.1116-win32-x86.msi
Changes since Pike 8.0.702 (release 13)
New Features
------------
o Core
- Added "->?" as the safe indexing variant of "->".
- Added int(7,8,16bit).
o ADT.History
Support encode/decode_value().
o Pike.FakeObject
This is used as a placeholder used in place of other objects in backtraces.
o Pike.DestructImmediate
An empty class that can be inherited to get the PROGRAM_DESTRUCT_IMMEDIATE
flag set.
o Image.Dims.exif_get and Image.exif_decode
New convenience functions to get the functionality of
Image.Dims.exif_get_JPEG and Image.JPEG._decode (rotation of the image
according to the EXIF orientation information) without needing to check
if the image is JPEG first.
o Concurrent
- Add delay() to postpone a future.
- Rewrite timeout() to reduce the memory footprint of a future in the
common case.
- Added query_{success,failure}_callbacks() and try_get() to
Concurrent.Future.
o Process.Process
- On NT it is now possible to specify the preferred console handler
via the "conpty" option. The default is to take it from the first
pty of "stdin", "stdout", and "stderr". If none of them is a pty
the console handler is inherited from the current process.
o SSL
The SSL module now supports session ticket both client and server side.
This functionality has been backported from 8.1/master.
o Standards.X509
Standards.X509 now allows the user to select acceptable signature
algorithms to be used in the verification of certificates.
o Stdio.Buffer
Added truncate() and the ability to search for substrings.
o Stdio.File
Support openat() with a single argument().
o Stdio.PROP_TTY
Stdio.File()->pipe() can now be used to create tty-capable
pipes (aka pseudo-ttys, aka ptys).
Bug fixes
---------
o ADT.CritBit.FloatTree
Fix FloatTree when using 128bit floats.
o ADT.CritBit.IntTree
Fixed the signature of ADT.CritBit.IntTree()->_values().
values(ADT.CritBit.IntTree()) claimed to return an array(int),
whereas IntTrees can hold any type as value.
Hide mutex keys and crypto contexts from backtraces.
Having unexpected references to mutex keys may cause hangs and
complaints about attempting to lock mutexes recursively.
o combine_path()
Fixed corner case where combine_path() on strings starting with
"./../" used to gain one directory level. Eg:
Old behavior:
> combine_path(".", "../foo");
(1) Result: "../foo"
> combine_path("./..", "foo");
(2) Result: "foo"
> combine_path("./../foo");
(3) Result: "foo"
New (fixed) behavior:
> combine_path(".", "../foo");
(1) Result: "../foo"
> combine_path("./..", "foo");
(2) Result: "../foo"
> combine_path("./../foo");
(3) Result: "../foo"
o describe_backtrace()
- Do not use value identifiers that have not been defined in
the visible backtrace.
- Fix clipped/canclip propagation.
o mktime()
Fix zone handling when localtime is in daylight savings time.
o strftime()
Remove non-standard conversions %k and %l.
o types()
Fixed reference leak.
o crypt()
Do not throw errorss on validation failure. Primarily ocurrs on
RHEL 8, return a failure code instead, consistent with historical
use.
o Backend.PollBackend
Signal read events on POLLHUP.
o Calendar
- Fixed typo in the Catalan word for the month December.
- Updated tzdata to 2019c.
o Compiler
- Adhere to the calling convention more in the machine code
generator on amd64, namely by keeping the stack pointer aligned
before calling into C code. GCC 8 started to emit instructions
relying on the correct, alignment, causing General Protection
Faults.
- Fixed some corner cases where file names where missing from
backtraces.
- Check the proper flag to determine if a local variable may be used
before optimizing away certain assignments if the assignment is
immediately followed by a return of the variable in question.
o Concurrent.Promise
- Fixed race condition where success- and/or failure- callbacks
could get lost.
- Convert errors thrown by the executor passed to Concurrent.Promise()
on create into promise rejection.
- Fixes for some recursive mutex lock errors.
o Crypto.ECC
Now works against Nettle 3.5.
o Filesystem.Tar
Fix extraction of S_ISUID and S_ISGID bits.
POSIX mandates that after a chown() (by unprivileged users), the
S_ISUID and S_ISGID bits are cleared, Linux 2.2.13 removed the
special case for root. The order of chmod() and chown() have been
reordered trying to fix an issue with applying utime() under
Windows. With this change, utime() is applied first, then chown()
and then chmod(), restoring the historical order for chown() and
chmod().
o Gettext
Prevent argument of textdomain from being empty.
o Image.Dims
- Fixed the inconsistency that Image.Dims.get_JPEG (and thus
Image.Dims.get) would flip the dimensions on JPEGs with EXIF
orientation information, even though Image.JPEG.decode does not
actually perform the rotations.
o Inotify
Improved handling of invalid event callbacks. [bug 8042]
o Odbc
The module's configure script now respects ABI suffixes when searching
for the ODBC library dir.
o Parser.HTML
Fixed a Pike stack overflow occuring with certain inputs to the HTML
parser.
o Pike.count_memory
Prevents a loop with ADT.List objects.
o Process.run
- Don't error if stdin modifier is supplied as the empty string.
[LysLysKOM 23099651]
o Protoocls.HTTP
- Protocols.HTTP.do_async_proxied_method() did not pass on request
headers unless credentials were given to the proxy or the protocol
to be used was HTTPS.
- Add Host header to proxied requests.
o Protocols.HTTP.Server
Fixed bug in Request()->update_mime_var(). [bug 10033]
o Protocols.HTTP.Session
- Fix race-condition when multiple threads call give_me_connection()
with the same url at the same time.
- Don't attempt to reuse SessionQuery objects.
give_me_connection() now always returns a new SessionQuery object
(albeit sometimes with a reused connection).
This should alleviate issues with stuff being left around from
previous queries.
o Protocols.DNS
Improved error handling when calling async callbacks.
Survive Stdio.UDP()->bind() failing with EACCES.
o Protocols.HTTP.Query
On Solaris 11 Stdio.File()->connect() often fails with EADDRINUSE.
If this happens, retry the connection.
o Protocols.Websocket
Tentative fix for error 'indexing the NULL value with "set_nonblocking'.
o Runtime
- Certain operations on arrays of only objects could have triggered a type
confusion under some circumstances, causing integers to be freed as
objects, and thus a NULL derefence (SIGSEGV).
- Fix some gratuitous rounding to double with long-double-precision.
- Improved Fix some gratuitous rounding to double with
long-double-precision.
o Search.Filter.HTML
Fix title always being set to "". [bug 10020]
o Sql.pgsql
- Ensure that the db connection is closed when the portal is closed.
- Small speed boost.
- Fix the only known deadlock-cause logged in the last year; happened
(rarely) on multiple parallel queries on a single connection.
- Fix the only known race condition logged in the last year; happened
on some queries that store >32KB data per query.
- Cast integers to floats, when floats are desired.
- Simplify error handling.
- Toggle cache_autoprepared_statements default to off;
turning it on triggers a bug in PostgreSQL sometimes
that causes spikes in CPU usage of the database.
- Fix support for text-multiqueries (separated by semicolons).
- resync() is synchronous now, and prevents race conditions
if followed back-to-back with new queries.
- disregard pending statements on connection destruction.
o SSL
- When verifying the hostname against the certificate, only accept *
(wildcard) for one level, i.e. *.example.com matches foo.example.com,
but not www.foo.example.com. Previously, all levels were matched.
- No longer accept MD2, MD5, and SHA1 as signature algorithms when
verification is requested. MD2 and MD5 are outright dangerous at
this point, and SHA1 should no longer be very relevant as web
browsers already force CAs for several years to no longer issue any
SHA1 certs anymore.
Acceptable signature algorithms are however configurable via
SSL.Context now, in case support of older/insecure algorithms is
vehemently required by an application.
o SSL.File
- Do not close automatically on write error.
The internal ssl_write_callback() would call shutdown()
on write error, which would cause the object to enter
one of the closed states, without the user actually
having closed the file.
This in turn caused errors like "Not open" from
functions like read() and set_nonblocking().
- Fix a situation where setting callbacks before the TLS
handshake had completed would cause the connection to hang
if an alert was generated (for instance when the peers are
unable to agree on a cipher suite.)
- Permit data received to be read() after the connection is
closed by the peer. Previously, read would throw a "Not open"
error rendering the data irretrievable.
o Standards.JSON
- encode() now allows other threads to run every now and then.
- Fix floating point literals with long-double-precision
o Standards.EXIF
Improved detection of EXIF section in get_properties().
o Standards.URI
- Updated list of unsafe characters to RFC 3986. [bug 8004]
- String representations of the URI that are generated by the object
are now canonical again. This also fixes an issue where Standards.URI
instances compared identical under the == comparison, but would hash
differently.
o Standards.X509
Use old syntax for compatibility.
o sybase
The module's configure script now respects ABI suffixes when searching
for the sybase library dir.
o System
- System.openlog() now keeps a reference to the ident passed, since
system level syslog() may access the string pointed to by the ident
passed to openlog. The reference will be released after a subsequent
call to openlog() or closelog().
o Thread.MutexKey
When simulated, mark for immediate destruction.
o Thread.Queue
Attempt to work around glibc bugs.
Glibc 2.24 and earlier have a broken implementation of condition
variables (cf https://sourceware.org/bugzilla/show_bug.cgi?id=13165).
Attempt to work around the issue by having the signalling thread
release the associated mutex before signalling.
o Thread.ResourceCount
Fixed mutex handling.
o Tools.Standalone.httpserver
The builtin webserver tool shipped with Pike (pike -x httpserver) was
previously susceptible to a directory traversal attack via URL encoding.
Building & Tools
----------------
o Add --with-exclude-site-prefixes configure option.
This will cause any directory specified in this option to be
excluded from the search path for binaries/includes/libraries. This
option will override any paths found by other means, allowing "system"
directories like /usr/local to be exluded for making binary distributions.
o Drop support for Postgres 7.1.
o Fix compilation with Nettle 3.1.1.
o Support recent versions of libfreetype.
o Drop use of the obsolete (~20 years) Solaris ioctl /proc API.
The ioctl-based /proc API was obsoleted in Solaris 2.6 (July 1997),
but to be supported "until the next major release". In Solaris 11.4
(August 2018) the support has now finally been removed.
Most of the removed code was from 1995, and was thus older than Pike...
o Improved pkg-config search path detection on Solaris.
Solaris 11.4 provides a single pkg-config binary, which by default
searches in /usr/lib/pkgconfig/ and /usr/share/pkgconfig/.
Unfortunately, /usr/lib/pkgconfig/ contains information for
the 32-bit packages (64-bit d:o is in /usr/lib/64/pkgconfig/).
This issue caused eg 32-bit headers for libffi to be used, causing
a SIGSEGV in ffi_closure_unix64_inner() due to the differing sizes
of ffi_closure in 32-bit and 64-bit abi.
o Check for actual availability of YP functions, not just headers.
o Attempt to use paxctl to alter the PaX flags for the pike binary to
allow for generation of machine code on NetBSD.
o Silence GL deprecation warning and fix a missing definition error on
Darwin for newer versions of MacOS.
o Fix check for my_bool and compilation errors related to MySQL 8.
Reply all
Reply to author
Forward
0 new messages