wxWidgets forum not working properly

[utelle]

Since a week using the wxWidgets forum ( https://forums.wxwidgets.org) is almost impossible. Most of the time login is not working for me. After clicking the login button in the login dialog always returns to the start page without performing the login. A session id is appended to the url (a new one after each click). Even if the login succeeds on rare occasions, it then often happens that I can't post a message, and logout doesn't work in such cases either.

IMHO this is a server-side issue, because clearing the browser cache and deleting cookies doesn't help.

I'm ready to assist in solving the problem, but without access to the server and its log files this is not possible.

Kind regards,

Ulrich

[Vadim Zeitlin]

On Sat, 18 Sep 2021 03:42:04 -0700 (PDT) utelle wrote:

u> Since a week using the wxWidgets forum ( https://forums.wxwidgets.org) is
u> almost impossible. Most of the time login is not working for me. After
u> clicking the login button in the login dialog always returns to the start
u> page without performing the login. A session id is appended to the url (a
u> new one after each click). Even if the login succeeds on rare occasions, it
u> then often happens that I can't post a message, and logout doesn't work in
u> such cases either.
u>
u> IMHO this is a server-side issue, because clearing the browser cache and
u> deleting cookies doesn't help.
u>
u> I'm ready to assist in solving the problem, but without access to the
u> server and its log files this is not possible.

Hi,

Unfortunately while I do have access to this server (it also hosts the
docs), I don't have access to nginx log files on it as wxdocs account is
not a member of group adm and I don't have any other account credentials
there (Bryan, please correct me if I'm forgetting something again), so I
can't do much neither and I can only hope Bryan could have look at this
and/or give me and/or you the privileges required to fix this.

Regards,
VZ

[Igor Korot]

Ulrich, Vadim,

This is not the first outcry about the forum.

It looks like Bryan is very busy at work and don't have time to look at this.

Thank you.

 Regards,
VZ

[utelle]

Hi Vadim,

similar problems were reported on the forum for quite a while since May 2020 (see https://forums.wxwidgets.org/viewtopic.php?f=17&t=47133). Up to a week ago I wasn't affected by those problems, but I think that was pure luck. The problems occur intermittently and often they vanish after a couple of hours or so. However, one never knows - for example, today, I was out of luck all the time.

Since I tried to access the forum from several different devices using different browser, I'm pretty sure that the problem is on the server side, although some users reported that deleting cookies and/or clearing the browser cache solved the problem for them, but I think that was simply coincidence. Maybe the phpBB software empties the server-side cache regularly and thereafter it works for a certain period of time. Wild guessing only.

Looking at the server log files could possibly help to detect phases of high activity. One hint I found was that heavy bot activities caused similar problems.

Maybe there is a flaw in the phpBB software itself. For example, in normal operation the session id is not visible in the url, but when the login doesn't work a session id is appended to the url. As far as I know the wxWidgets forum uses an older phpBB version. Maybe upgrading to a newer version would help.

Regards,

Ulrich

[wsu]

How does that forum relate (or not relate?) to https://groups.google.com ?

[utelle]

> How does that forum relate (or not relate?) to https://groups.google.com ?

The wxWidgets developers provide the forum https://forums.wxwidgets.org for users of the wxWidgets library. However, the wxWidgets core developers post on the forum only infrequently, if at all. So, the wx developers are not necessarily aware of technical problems with the forum. Therefore it seemed to be the right thing to report the problem here on the wx-dev list.

Regards,

Ulrich

[Vadim Zeitlin]

On Sat, 18 Sep 2021 09:28:44 -0700 (PDT) wsu wrote:

w> How does that forum relate (or not relate?) to https://groups.google.com ?

It's completely independent and the main difference is that this mailing
list is hosted by Google on their infrastructure, while the forum is hosted
by Bryan on his own host.

Regards,
VZ

[Bryan Petty]

On Sat, Sep 18, 2021 at 6:49 AM Vadim Zeitlin <va...@wxwidgets.org> wrote:
> Unfortunately while I do have access to this server (it also hosts the
> docs), I don't have access to nginx log files on it as wxdocs account is
> not a member of group adm and I don't have any other account credentials
> there (Bryan, please correct me if I'm forgetting something again), so I
> can't do much neither and I can only hope Bryan could have look at this
> and/or give me and/or you the privileges required to fix this.

All I can see in the error logs was mentions about insufficient SSL
session cache, so I've attempted to bump that up, and it's not
complaining for now at least anymore:

ngx_slab_alloc() failed: no memory in SSL session shared cache "le_nginx_SSL"

I bumped it up by 400%. I have no idea if this would contribute to
forum login sessions, but maybe it does.

--
Regards,
Bryan Petty

[Bryan Petty]

On Sat, Sep 18, 2021 at 7:03 AM Igor Korot <ikor...@gmail.com> wrote:
> This is not the first outcry about the forum.
> It looks like Bryan is very busy at work and don't have time to look at this.

Yeah, I don't have nearly as much time or motivation to maintain these
servers as I used to, and it really is a significant amount of work.

There's tons of regular maintenance backlog to catch up with on this
server, and I can imagine this error might just resolve itself
entirely if I could find the time:

1. The server is running completely out of date (and EOL) Ubuntu with
EOL PHP as well.
2. The next major version update available for the forums requires
newer PHP than the server is running.
3. Upgrading the server not only requires upgrading phpBB on the
forums, but also upgrading MediaWiki for the wiki at the same time,
which is in the same situation as the forums.

All of this is one large move that's likely to involve significant
down time, and likely break a bunch of things that will require
additional follow-up, and that's why none of it has happened yet, even
though it's been necessary for a couple years now.

Whenever I've done these upgrades in the past, I also almost always
end up spending days fixing back up the spam blocking tools we're
using, and we end up with either a new flood of spam again (on both
the forums and wiki), or blocking legitimate users from logging in,
registering new accounts, editing wiki pages, or all of the above.

Of course, none of this is nearly as bad as the situation we're in
with Trac, which I'd say is nearly dead in the water. The Trac project
itself is barely "maintained" these days, and regardless, we've been
stuck on a 1.0.x release that's over 4 years old now, since 1.1+ (and
they're out with 1.4 now) breaks a handful of our extensions,
including the core ones that have been very popular for many Trac
sites that we use for user registration and authentication. Honestly,
wx should just enable GitHub issues, and start migrating things over,
so Trac can just be retired entirely.

--
Regards,
Bryan Petty

[Vadim Zeitlin]

On Sat, 18 Sep 2021 12:39:40 -0600 Bryan Petty wrote:

BP> On Sat, Sep 18, 2021 at 7:03 AM Igor Korot <ikor...@gmail.com> wrote:
BP> > This is not the first outcry about the forum.
BP> > It looks like Bryan is very busy at work and don't have time to look at this.
BP>
BP> Yeah, I don't have nearly as much time or motivation to maintain these
BP> servers as I used to, and it really is a significant amount of work.

I understand this perfectly and I know (probably better than anybody else
considering the number of times I've bothered you during all these years)
how much time you've already spent on it -- and, of course, am very
thankful for it! The question is whether somebody else can help with it or
if we really need to migrate to some hosted solution. Of course, I'd rather
not start changing this now and concentrate on releasing 3.2 instead, but
OTOH it's probably never going to be a good time to do it anyhow, so maybe
we should do it now...

BP> There's tons of regular maintenance backlog to catch up with on this
BP> server, and I can imagine this error might just resolve itself
BP> entirely if I could find the time:
BP>
BP> 1. The server is running completely out of date (and EOL) Ubuntu with
BP> EOL PHP as well.
BP> 2. The next major version update available for the forums requires
BP> newer PHP than the server is running.
BP> 3. Upgrading the server not only requires upgrading phpBB on the
BP> forums, but also upgrading MediaWiki for the wiki at the same time,
BP> which is in the same situation as the forums.
BP>
BP> All of this is one large move that's likely to involve significant
BP> down time, and likely break a bunch of things that will require
BP> additional follow-up, and that's why none of it has happened yet, even
BP> though it's been necessary for a couple years now.

I wonder if we could create a new server (it's a VPS, isn't it?) and move
things there piece-wise?

BP> Of course, none of this is nearly as bad as the situation we're in
BP> with Trac, which I'd say is nearly dead in the water. The Trac project
BP> itself is barely "maintained" these days, and regardless, we've been
BP> stuck on a 1.0.x release that's over 4 years old now, since 1.1+ (and
BP> they're out with 1.4 now) breaks a handful of our extensions,
BP> including the core ones that have been very popular for many Trac
BP> sites that we use for user registration and authentication. Honestly,
BP> wx should just enable GitHub issues, and start migrating things over,
BP> so Trac can just be retired entirely.

I didn't want to have all our eggs in GitHub basket, but by now we have
not only Git there, but also 99% of the CI jobs, which are even more
critical, so I guess we could migrate from Trac to it too. Especially
because GitHub Issues gained some functionality since then, e.g. they now
allow setting milestone for the tickets which is quite useful. I wonder if
anybody here has any experience with Trac-to-GitHub migration? There seems
to be quite a number of tools for doing it.

Using the same logic, we might even migrate the forum to GitHub
discussions, but having barely used the former and never the latter I don't
know if it's really a good idea.

Finally, I wonder if we could host our documentation on GitHub Pages too?
VZ

[Bryan Petty]

On Sat, Sep 18, 2021 at 3:55 PM Vadim Zeitlin <va...@wxwidgets.org> wrote:
> Finally, I wonder if we could host our documentation on GitHub Pages too?

This search won't work on Pages:
https://docs.wxwidgets.org/trunk/search.php?query=wxlistctrl

Doxygen does have an alternative Javascript-powered search, but it's
not as good, and all previous versions of docs would need to be
regenerated with that enabled (but you'd probably just do it for the
latest stable and dev).

This isn't terribly bad either though since the main site search is
pretty good at using Google's index of docs to find things, but it is
restricted to trunk docs, since we intentionally disable indexing for
everything else. This could potentially be linked to from the doc
pages (again, it'd require regenerating docs though):
http://www.wxwidgets.org/search/?q=wxlistctrl

The way we use symlinks to alias latest doc versions appears to still
be supported on GitHub Pages, i.e.:
- https://docs.wxwidgets.org/stable/
- https://docs.wxwidgets.org/3.0/
- https://docs.wxwidgets.org/3.0.5/
^ all of those point to the same single copy of 3.0.5 docs, and this
should still work on GitHub Pages exactly the same way

This is a pretty obscure feature of docs.wxwidgets.org that not many
know about, but it actually has some extra intelligence when you hit a
404, which was built so that any external links to versions of docs
that ultimately get removed would still provide an easy way to browse
to the same page in other (newer) versions of the manual. This would
not be possible on Pages though:
- https://docs.wxwidgets.org/2.8/overview_events.html

I don't think that's a big deal either though. You can at least still
configure a custom 404 page, but it can't do anything dynamic like
this one does. It might only provide some basic help, and just point
users either to the site search or just https://docs.wxwidgets.org/

Finally, it's probably worth mentioning that anything published to
GitHub Pages still has to be committed to a git repo, which has a
repository size soft limit somewhere around 5GB. Our current
collection of docs comes out to 3GB right now. This might just mean it
requires culling old versions earlier. It likely also means that any
commits need to be squashed into a single commit so old versions being
removed are actually legitimately removed from the git history.

In short though, I believe this can work.

--
Regards,
Bryan Petty

[Bryan Petty]

On Sat, Sep 18, 2021 at 5:27 PM Bryan Petty <br...@ibaku.net> wrote:
> Finally, it's probably worth mentioning that anything published to
> GitHub Pages still has to be committed to a git repo, which has a
> repository size soft limit somewhere around 5GB. Our current
> collection of docs comes out to 3GB right now.

Oh, actually, GitHub Pages has additional limits that are more strict:
https://docs.github.com/en/pages/getting-started-with-github-pages/about-github-pages#usage-limits

This limits it to 1GB, so we'd definitely have to cull down a bunch of versions.

--
Regards,
Bryan Petty

[Bryan Petty]

On Sat, Sep 18, 2021 at 3:55 PM Vadim Zeitlin <va...@wxwidgets.org> wrote:

> Using the same logic, we might even migrate the forum to GitHub
> discussions, but having barely used the former and never the latter I don't
> know if it's really a good idea.

It was at least suggested at one point in the past that GitHub
discussions be enabled (even if forums were left running) back in
January.

I really have no clue how the wxForums community overall would handle
a forced migration to GitHub Discussions, with the assumption that the
goal is ultimately shutting down the phpBB forums. I think whatever
happens there, I'm sure it would be necessary to at least have GitHub
Discussions enabled and in use for a long period of time just to see
how well it works out in tandem.

There's certainly a big benefit in collaboration and utility if
everything is running on the same platform there on GitHub, with
pulls, issues, the code, discussions, and potentially the wiki too
(with wxWiki content moved there maybe?). It would solve our current
problem where if you want to be involved in any of those tasks, you
need separate user accounts registered with Trac, phpBB, MediaWiki,
and GitHub.

--
Regards,
Bryan Petty

[utelle]

Thanks for looking into this issue, Bryan.

At the moment, the forum seems to function properly again.

Maybe it would be interesting to look at the activity log, too. I had the impression that there always was a rather high number of guest users active (~100 or more), before the forum login stopped working. If there is a high number of bot users, this maybe could explain why there is sometimes insufficient SSL session cache.

Regards,

Ulrich

[utelle]

VZ schrieb am Samstag, 18. September 2021 um 23:55:04 UTC+2:

BP> All of this is one large move that's likely to involve significant
BP> down time, and likely break a bunch of things that will require
BP> additional follow-up, and that's why none of it has happened yet, even
BP> though it's been necessary for a couple years now.

I wonder if we could create a new server (it's a VPS, isn't it?) and move
things there piece-wise?

Setting up a separate new server and moving things one after the other seems to be a better approach than trying to upgrade the existing server.

 

BP> [...] Honestly,

BP> wx should just enable GitHub issues, and start migrating things over,
BP> so Trac can just be retired entirely.

I didn't want to have all our eggs in GitHub basket, but by now we have
not only Git there, but also 99% of the CI jobs, which are even more
critical, so I guess we could migrate from Trac to it too.

IMHO it would be a good idea to enable GitHub issues and migrate Trac to GitHub. It would make it a lot easier to report issues.

Using the same logic, we might even migrate the forum to GitHub
discussions, but having barely used the former and never the latter I don't
know if it's really a good idea.

This would only make sense if the existing forum posts could be somehow migrated to GitHub discussions. However, I doubt that there is a feasible way to do that taking into acount that the forum has >22k registered users (of which most do not have a GitHub account) and almost 200k postings.

Of course, GitHub discussions could be enabled in parallel, as Bryan proposed. That has its pros and cons: on one hand it would make it easier to report actual wx issues by converting a discussion into an issue; on the other hand it would be yet another place for communicating about wx, as there are already the mailing lists, the forum, and Trac resp GitHub issues.

 

Finally, I wonder if we could host our documentation on GitHub Pages too?

In principle, this should be possible and could have its advantages. However, I don't know the workflow how the documentation is updated and how much effort it would require to adjust the workflow to use GitHub pages. Additionally, Bryan already mentioned that GitHub pages have certain restrictions and limits.

Regards,

Ulrich

[Eric Jensen]

Hello Bryan,

Sunday, September 19, 2021, 1:53:09 AM, you wrote:

BP> On Sat, Sep 18, 2021 at 3:55 PM Vadim Zeitlin <va...@wxwidgets.org> wrote:
>> Using the same logic, we might even migrate the forum to GitHub
>> discussions, but having barely used the former and never the latter I don't
>> know if it's really a good idea.

BP> It was at least suggested at one point in the past that GitHub
BP> discussions be enabled (even if forums were left running) back in
BP> January.

BP> I really have no clue how the wxForums community overall would handle
BP> a forced migration to GitHub Discussions, with the assumption that the
BP> goal is ultimately shutting down the phpBB forums. I think whatever
BP> happens there, I'm sure it would be necessary to at least have GitHub
BP> Discussions enabled and in use for a long period of time just to see
BP> how well it works out in tandem.

I think you should also consider that even now the forum has a
different audience than e.g. the mailing list. We get a lot of newbie
questions, often not even directly wx related, but general programming
or C++ related questions. There are many questions about building wx,
setting up IDEs, compilers etc.

Additionally, there are many sub categories, which are a little more
distant from core wx development, like for database related questions,
or third party extensions.

And a Russian and Chinese sub-forum.

IOW, i think there are good reasons making an effort to keep the forum
alive.

Regards,
Eric



--

[Vadim Zeitlin]

On Sat, 18 Sep 2021 17:53:09 -0600 Bryan Petty wrote:

BP> On Sat, Sep 18, 2021 at 3:55 PM Vadim Zeitlin <va...@wxwidgets.org> wrote:
BP> > Using the same logic, we might even migrate the forum to GitHub
BP> > discussions, but having barely used the former and never the latter I don't
BP> > know if it's really a good idea.
BP>
BP> It was at least suggested at one point in the past that GitHub
BP> discussions be enabled (even if forums were left running) back in
BP> January.

Enabling them is not difficult at all (just a checkbox in the settings
AFAIK), so we can do it, of course, but if people don't want to migrate to
them from the forum, the Discussions might not get much activity and just
result in some unwanted community fragmentation.

BP> I really have no clue how the wxForums community overall would handle
BP> a forced migration to GitHub Discussions, with the assumption that the
BP> goal is ultimately shutting down the phpBB forums. I think whatever
BP> happens there, I'm sure it would be necessary to at least have GitHub
BP> Discussions enabled and in use for a long period of time just to see
BP> how well it works out in tandem.

Does the forum support polls? People love polls, so what about making one
there to gauge the interest in moving to GitHub Discussions?

BP> There's certainly a big benefit in collaboration and utility if
BP> everything is running on the same platform there on GitHub, with
BP> pulls, issues, the code, discussions, and potentially the wiki too
BP> (with wxWiki content moved there maybe?). It would solve our current
BP> problem where if you want to be involved in any of those tasks, you
BP> need separate user accounts registered with Trac, phpBB, MediaWiki,
BP> and GitHub.

Yes, there is absolutely a benefit in centralizing everything. I probably
don't need to mention that there are also other important, even if less
tangible, benefits to not doing it :-/

Right now I think we should consider migrating wxTrac to GitHub Issues
(hopefully one of the existing migration tools will be able to do it
decently well), but keeping the forum and the docs site running, if you can
still support it -- or if we can find somebody else who could. FWIW I
probably could run/maintain the docs site myself as it's relatively simple,
but I wouldn't want to deal with the forum as I don't know much about PHP
and don't even use the forum myself (and, besides, if I were to maintain
this, it wouldn't do any good to the project bus factor).

Regards,
VZ

[Bryan Petty]

On Sun, Sep 19, 2021 at 8:06 AM Vadim Zeitlin <va...@wxwidgets.org> wrote:
> Right now I think we should consider migrating wxTrac to GitHub Issues
> (hopefully one of the existing migration tools will be able to do it
> decently well), but keeping the forum and the docs site running, if you can
> still support it

Yes, I'm still going to try to find the time to catch up on some of
this. My intention was just to express the current situation, not
provoke an immediate mass migration of everything.

We should be fine with the docs, forums, and wiki for some time still,
and hopefully that SSL shared cache issue actually at least improved
the situation with sessions to buy more time to look at server
upgrades.

--
Regards,
Bryan Petty

[Igor Korot]

Bryan,
I just got this "circling login" with the "session id" URL.

Thank you.

> --
> You received this message because you are subscribed to the Google Groups "wx-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to wx-dev+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/wx-dev/CAKB8V%2BrB7MgBNu_ZNNKnznWndirUaLszkYeQ8Uccqct%3DbJy19w%40mail.gmail.com.

[Vadim Zeitlin]

On Sun, 19 Sep 2021 09:51:39 -0600 Bryan Petty wrote:

BP> On Sun, Sep 19, 2021 at 8:06 AM Vadim Zeitlin <va...@wxwidgets.org> wrote:
BP> > Right now I think we should consider migrating wxTrac to GitHub Issues
BP> > (hopefully one of the existing migration tools will be able to do it
BP> > decently well), but keeping the forum and the docs site running, if you can
BP> > still support it
BP>
BP> Yes, I'm still going to try to find the time to catch up on some of
BP> this.

Thanks a lot!

BP> We should be fine with the docs, forums, and wiki for some time still,
BP> and hopefully that SSL shared cache issue actually at least improved
BP> the situation with sessions to buy more time to look at server
BP> upgrades.

If you'd like to have some help with it, please let me know. And/or maybe
someone else here would be willing to help. And I'd like to reiterate the
proposal to migrate things to another server rather than doing it in place
as it would allow to do them incrementally (and also have a working backup
solution in case anything goes wrong).

BTW, I've also been thinking that we perhaps should create something like
wxWidgets/infrastructure repository on GitHub and leave at least some notes
about how things are set up currently in its wiki and maybe also add any
scripts or whatever we're using to it.

Anyhow, please let me know how can I help with anything here,
VZ

[Danny Scott]

Hi Bryan. If you can give me a backup copy of the forum/database, I can spin up a KVM VM to test upgrading the forum to see what issues I encounter so you can be better prepared when you do the upgrade. I have worked with SimpleMachines in the past.

I am currently fully retired so I have lots of time available. I have a 1 Gbps up/down unmetered connection so I can also host this for testing purposes and provide you with remote testing of the upgrade process if you wish.

--

Danny Scott

[PB]

Unfortunately, I can confirm that the issue still persists, I have just run into it.

To make sure, I cleared the cookies both via "Delete cookies" link on the forum main page as well as using Firefox's "Clear cookies and site data" feature, but it did not help (it never did before either).

Regards,

PB

[Stefano Mtangoo]

I also cannot login without checking remember me as we speak. All juggling around clearing cookies is helpeless. Checking remember me logs me in but any attempt to post mysteriously logs me out!