Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

good news, bad news and discussion

3 views
Skip to first unread message

shine at mozilla

unread,
Jun 6, 2020, 4:04:48 AM6/6/20
to community-kerala, sy...@mozilla-community.org, Kumaresan C S
Hello,

I have good news and bad news. I'll go in chronological order; so the
bad news comes first.
The server that ran mozillakerala.org went offline sometime 2 weeks
ago. I noticed it on Saturday, May 23, 2020. Apparently, the server
that was running had reached its end of life and was turned off ( we
were not given any intimation of this; at least not that I'm aware of
).

The good news :
With the help of Syam and Stefan Costen, we got the server back online
for backup and recovery on Thursday, May 27, 2020. However, I was only
notified about the server coming back online on the night of Friday,
May 28, 2020. I took a backup of the server ( 1.6 GB tar.gz
compressed; 4.1 GB uncompressed ). We also got a new server ordered to
replace the old one that could go down anytime.

Since we're at this juncture, I'd like to open a discussion about
other alternatives ( services offered by mozilla ).

Community Portal
---------------
The initial intention of getting this server provisioned was to build
a "portal" for the community. However, what we have on the server
right now is a Wordpress blog [1], card generator [2] and some event
websites [3]. Previously, we had Bugzilla and MediaWiki instances
running on the server. We took the MediaWiki instance down when
vandalism became unfendable. I don't really remember what happened to
Bugzilla. I think it was because of an outdated version that is
probably unsupported now. We also had a Piwik instance for web
analytics, but it has been dead for a while now.

Mozilla now has a community portal [4]. This portal aligns very well
with what our original idea was. I don't want to go into the details
of the things the portal is capable of. Syam and Kumaresan have set up
a group [5] on behalf of the community as well.

Discussion
----------
We had already previously decided and agreed that we should use
Discourse and the mailing list more for discussions rather than using
Telegram. The advantages of either systems over Telegram is the
ability to have separate discussions simultaneously. People can choose
to subscribe or unsubscribe from a discussion. On Telegram, there's
only a single thread and consequently, more than 2 topics
simultaneously can get confusing and difficult to follow very quickly
not to mention not having the ability to unsubscribe from a particular
discussion ( you could achieve similar functionality by muting
notifications for a particular period but not for specific topics ).

Blog
-----
The only system on the server that is probably kind of relevant right
now is the blog ( which is also dead in terms of posts / content ).
However, we should not just pull the plug on it when the server goes
down. I believe we should at least memorialize whatever content there
is on the internet.
Mozilla has a service offering for communities in partnership with
WPEngine which offers Wordpress instances to communities. Since our
blog is running Wordpress, we could potentially migrate it to the
WPEngine offering.

Email
-----
We have 3 shared email inboxes that are running on Outlook now.
Mozilla has a partnership with Google and offers GSuite to
communities. We could migrate over our emails to GSuite, create
mailing lists for the shared inboxes with relevant people as members.

Cards Generator, Event webpages and others
----------------------------------------
Like I mentioned earlier, I'm a strong believer that we should
memorialize whatever we've had running on our server. We could
leverage GitHub and GitHub pages to host the static event webpages.
Most of the webpages were already git-based and stored on GitHub
anyway. We just need to enable GitHub pages on the repositories to
serve the websites. Then we just need to change the DNS records ( more
on that below ). GitHub pages provides free TLS on the websites as
well. I had previously migrated the HackrGirls [6] and MakerParty
Kochi [7] websites to use GitHub pages ( because I had control on the
domains ).

The only service that would be left orphaned would be the card
generator since it requires PHP ( which has a traditional server
dependency ). Though the existing templates on the service is probably
irrelvant, I don't want to kill it. It still works. We could use the
replacement server to host the service ( another DNS update )

We can ignore Bugzilla and MediaWiki because we haven't used them
much. Piwik ( now Matomo ) too can be avoided since we won't be
hosting anything ourselves anymore ( assuming we decided to make all
the migrations suggested above ). Whatever service we migrate to
should provide the relevant analytics. We shouldn't really be looking
at analytics to do any kind of analysis anyway.

DNS
---
Now, all of the above is relevant right now because the existing
server is going away soon ( even if we are getting a replacement
machine ). That means all the existing DNS records that point to the
server has to be modified. We don't have control over the
mozillakerala.org domain, so making any changes to the domain would
require creating a bug on bugzilla and waiting on the relevant people
to make the changes. We could possibly ask for some kind of limited
access to modify the records only for the specific domain.

Telegram Group
---------------
Assuming we decided to migrate to the community portal, the question
arises - what happens to the Telegram group? Well, I've wanted to do
this for a while now, but I was just juggling the idea in my head. I'm
going to voice it now. After we had decided to migrate to use
Discourse as our discussion platform, the Telegram group had already
become irrelevant. At the time, mozilla was using Telegram actively,
so, it made sense to keep the group around. And because we didn't
really enforce the discourse discussion decision, it just lived on.
However, today, almost all the mozilla-related Telegram groups have
become mostly inactive and have moved on to Matrix [8] ( after moznet
was decommisioned [9] ).
Now, while I wouldn't want to force the existing Telegram group to be
migrated to Matrix, but we could easily set up a semi-official bridge
to it. Or should we just abandon it altogether ( in the sense - not to
connect it to Matrix )? We could still maintain it as an unofficial
synchronous discussion platform but then what would Matrix be?
Maintain Matrix as the official synchronous discussion platform while
maintaining Telegram as a low-barrier-to-entry unofficial synchronous
discussion platform? In my opinion, having too many platforms is a
technical debt and unnecessary segregation of the community. Others
might argue that being on more platforms would translate to more reach
and visibility. While I don't debate the argument, I just don't see
the value in it when it comes as a compromise of segregation of the
community.

There's no TL;DR for this email because I think it has to be read
fully to understand the context. What do you think about my proposal?
Should we do all of the migrations suggested above? Should we do some
and ignore the others? Do you have any other alternate suggestions?

~ shine

Links
[1] blog.mozillakerala.org
[2] cards.mozillakerala.org
[3] events.mozillakerala.org
[4] https://community.mozilla.org
[5] https://community.mozilla.org/groups/mozilla-kerala
[6] https://hackrgirls.womoz.in
[7] https://kochi.2014.makerparty.in
[8] https://chat.mozilla.org
[9] https://matrix.org/blog/2020/03/03/moznet-irc-is-dead-long-live-mozilla-matrix
[Mozilla Kerala] [Notification] Planned Maintenance - mozillakerala.org.eml

shine at mozilla

unread,
Jul 12, 2020, 2:22:12 PM7/12/20
to community-kerala, Akshay S Dinesh, Kumaresan C S, sy...@mozilla-community.org
Hello,

It's been over a month since I opened up this discussion. Since nobody
had any opinions let alone objections, I'm going to assume a favorable
outcome - meaning that we are going to set in motion all of the
proposed actions. The website and blog has been down for over a few
weeks now ( the old server was taken offline ). I'll try and work with
the community web services team and see how we can do a transition of
resources to mozilla.

Akshay,

I have not discounted your opinion. As always, thank you for
participating. I did take a look at your work. It was good. I strongly
think that we should move all of the communities' services under one
umbrella - and I believe that is the mission of the community web
services team. Unless you have a strong opinion that we should migrate
the blog over to Jekyll and host it on GitHub, I'm going to be in
favor of my opinion to move to mozilla's hosted WordPress solution
which would also be the easiest way to migrate too.

~ shine

On Sun, Jun 7, 2020 at 1:36 AM Akshay S Dinesh <asdof...@gmail.com> wrote:
>
> I reserve my comment on most things.
>
> But https://asd.learnlearn.in/migrating-wordpress-to-jekyll/ may be handy.
>
> Akshay
>> ---------- Forwarded message ----------
>> From: shine at mozilla <shine....@gmail.com>
>> To: community-kerala <communit...@lists.mozilla.org>
>> Cc: Akshay S Dinesh <asdof...@gmail.com>, "Gautam krishna.R" <r.gauta...@gmail.com>
>> Bcc:
>> Date: Sat, 25 Apr 2020 18:21:35 +0530
>> Subject: Re: [Mozilla Kerala] [Notification] Planned Maintenance - mozillakerala.org
>> Hello,
>>
>> This is to inform that the activity has been stalled due to an
>> outdated kernel version ( 2.6.32 ) that is used to boot the server
>> with. A ServerFault [1] question indicates that it is possible that
>> the server provider is using NetBoot to boot the machine ( because
>> there is no kernel in the filesystem ). This limits the option to
>> upgrade the kernel to the latest release. I've reached out to tad who
>> used to be the point of contact for the server to see if this issue
>> can be resolved in some way.
>>
>> However, the good news is, I was able to upgrade the operating system
>> ( with warnings about the outdated kernel ) to Debian Stretch. It's
>> not the greatest, but that's the closest I could get the system to.
>> The other good news is that Debian automatically upgraded mySQL to
>> mariaDB during the OS upgrade from Debian Jessie to Debian Stretch.
>> So, that's one follow-up task checked off of the list. I'll probably
>> pick some of the follow up tasks if I can until we get a response to
>> the kernel issue.
>>
>> All existing services are back online in their previous good form.
>> I'll resume this activity once we have an update about the kernel
>> upgrade from the server provider.
>>
>> Links
>> -----
>> [1] https://serverfault.com/questions/586171/cant-update-linux-kernel-on-vps
>>
>> ~ shine
>>
>> On Sat, Apr 25, 2020 at 3:03 PM shine at mozilla
>> <shine....@gmail.com> wrote:
>> >
>> > Hello,
>> >
>> > The planned activity will be starting soon. While I'd like to give
>> > commentary-style updates, I think that'd be spam to the rest of the
>> > people in the list.
>> >
>> > Hence, I'm not going to be sending minor updates to the
>> > community-kerala list; but I'll send it to people already on the cc
>> > list. If you'd like to get updates on this activity, please reply to
>> > this email with 'Subscribe' in the email body.
>> >
>> > Once you're on the list, and you think the commentary are too
>> > 'spammy', you can also send 'Unsubscribe' to unsubscribe from the
>> > updates as well.
>> >
>> > So, here goes. See you on the other side.
>> >
>> > ~ shine
>> >
>> > On Fri, Apr 24, 2020 at 5:36 PM shine at mozilla
>> > <shine....@gmail.com> wrote:
>> > >
>> > > I believe Livepatch is an Ubuntu service, but we run Debian.
>> > >
>> > > Livepatch is a snap app and I definitely don't want to install snaps
>> > > on to the server. Let's just say I have a very strong prejudice
>> > > towards snaps.
>> > >
>> > > ~ shine
>> > >
>> > > ~ shine
>> > >
>> > >
>> > > On Fri, Apr 24, 2020 at 4:41 PM Gautam krishna.R
>> > > <r.gauta...@gmail.com> wrote:
>> > > >
>> > > > Can we enable livepatch as well: https://ubuntu.com/livepatch so that it will do critical updates automatically?
>> > > >
>> > > > Thanks and Regards,
>> > > >
>> > > > Gautam Krishna R
>> > > >
>> > > > Software Engineer
>> > > >
>> > > > Red Hat India (APAC)
>> > > >
>> > > > Tower X, Magarpatta City, Pune
>> > > >
>> > > > gau...@redhat.com
>> > > > M: +91 9446335048 | GitHub: git.io/gkr
>> > > >
>> > > > @RedHat Red Hat Red Hat
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Fri, Apr 24, 2020 at 3:42 AM shine at mozilla <shine....@gmail.com> wrote:
>> > > >>
>> > > >> Realizing that is precisely what prompted me to revive the server back
>> > > >> up. We've always been rebellious. So, why not some more fun?
>> > > >>
>> > > >> So I guess, what I'm trying to say is - until mozilla decides to pull the
>> > > >> plug on the server itself, we should maintain as much of it as we can.
>> > > >>
>> > > >>
>> > > >> ~ shine
>> > > >>
>> > > >> On Thu, Apr 23, 2020 at 11:40 PM Akshay S Dinesh <asdof...@gmail.com> wrote:
>> > > >> >
>> > > >> > I don't know if it counts as objection but I doubt your effort is worth it. Especially the way mozilla wants to centralize groups on to its community portal.
>> > > >> >
>> > > >> > On Thu, Apr 23, 2020 at 9:51 PM shine at mozilla <shine....@gmail.com> wrote:
>> > > >> >>
>> > > >> >> Hello,
>> > > >> >>
>> > > >> >> This is to inform that there will be a planned maintenance of the
>> > > >> >> server that hosts mozillakerala.org over the weekend ( April 25-26,
>> > > >> >> 2020 ).
>> > > >> >>
>> > > >> >> The main intention is to attempt an operating system upgrade from
>> > > >> >> Debian 8 ( Jessie ) to Debian 10 ( Buster ). The server hasn't had any
>> > > >> >> updates for a very long time.
>> > > >> >>
>> > > >> >> TL; DR
>> > > >> >> -----
>> > > >> >> 1. Planned maintenance activity on mozillakerala.org server over the weekend
>> > > >> >> 2. Upgrade operating system to Debian Buster
>> > > >> >> 3. Database engine migration to mariaDB
>> > > >> >> 4. Downtime of 10 - 30 minutes
>> > > >> >> 5. Other follow-up activities if time permits
>> > > >> >> ===============
>> > > >> >>
>> > > >> >> Long version
>> > > >> >> ---------------
>> > > >> >>
>> > > >> >> I know ( and acknowledge ) that the server has not seen any love at
>> > > >> >> all, both in terms of updates or usage, in the past years. I wanted to
>> > > >> >> change that, at least in terms of updates, with this activity.
>> > > >> >>
>> > > >> >> The plan is to follow the normal upgrade path. Since the server is
>> > > >> >> running Debian, it would be stable enough to even skip the version in
>> > > >> >> between altogether and jump straight to the latest version, but I
>> > > >> >> don't want to take the risk.
>> > > >> >> The action plan is as follows :
>> > > >> >>
>> > > >> >> 1. Stop web server ( apache2, php5-fpm )
>> > > >> >> 2. Stop database server ( mysql )
>> > > >> >> 3. Take a database dump ( just to be safe )
>> > > >> >> 4. Update sources.list to point to stretch
>> > > >> >> 5. apt update; apt dist-upgrade
>> > > >> >> 5.1. if all goes well, attempt start of apache2 without php
>> > > >> >> 5.1.1. verify static landing page ( mozillakerala.org ) is active
>> > > >> >> 6. if that goes well, repeat steps 1, 4 and 5 with repo pointing to buster
>> > > >> >> 9. once we are on buster, then start php-fpm and mysql
>> > > >> >>
>> > > >> >> That would conclude the main activity. If all of the above steps go
>> > > >> >> well ( which generally doesn't happen that easy in the linux server
>> > > >> >> world ), then I have another bunch of activities to do in my mind :
>> > > >> >>
>> > > >> >> 1. migrate database engine to mariadb
>> > > >> >> 2. migrate web server to nginx
>> > > >> >> 3. clean up existing domains list
>> > > >> >> 3.1 fix unknown ServerName redirections
>> > > >> >> 4. set up Transport Layer Security ( TLS )
>> > > >> >> 5. upgrade Piwik to Matomo
>> > > >> >> 6. set up some basic server hardening
>> > > >> >>
>> > > >> >> Downtime time window
>> > > >> >> --------------------
>> > > >> >> If the operating system upgrade goes well, there should only be an
>> > > >> >> intermittent downtime of around 5 - 10 minutes for the main website (
>> > > >> >> mozillakerala.org ) considering there is good internet network speed
>> > > >> >> on the server. The blog, however, would be down for further if the
>> > > >> >> migration to the mariadb database engine is also taken up. While mySQL
>> > > >> >> and mariaDB are data-compatible, there can be unknown factors that
>> > > >> >> could crash which could extend the downtime of the database engine
>> > > >> >> which in turn could extend the downtime of the blog. The other
>> > > >> >> follow-up activities can be done with the server running because they
>> > > >> >> do not need to keep the server down for operation.
>> > > >> >>
>> > > >> >> If you have any objections about this activity, you have time until 9
>> > > >> >> AM on Saturday, April 25, 2020 to voice your opinions after which the
>> > > >> >> activity will assume to be unopposed and in motion.
>> > > >> >> If you have any other opinion about the action plan or the follow-up
>> > > >> >> activities; or if you'd like to add to the list of activities, please
>> > > >> >> reply to this email and we can discuss the feasibility of including it
>> > > >> >> in the action items.
>> > > >> >>
>> > > >> >> I'd also like to point out that the above plan ( especially the follow
>> > > >> >> up action items ) is subject to change considering the possibility of
>> > > >> >> breakages or breakdowns during the main activity. The main target of
>> > > >> >> the activity would be to complete the operating system upgrade to get
>> > > >> >> to the latest versions of packages and system patches. The follow up
>> > > >> >> action items would be taken up only if the main activity goes through
>> > > >> >> successfully and there is enough time to tackle the rest of the follow
>> > > >> >> up action items.
>> > > >> >>
>> > > >> >> ~ shine
>> > > >> >> _______________________________________________
>> > > >> >> community-kerala mailing list
>> > > >> >> communit...@lists.mozilla.org
>> > > >> >> https://lists.mozilla.org/listinfo/community-kerala
>> > > >> _______________________________________________
>> > > >> community-kerala mailing list
>> > > >> communit...@lists.mozilla.org
>> > > >> https://lists.mozilla.org/listinfo/community-kerala
>> _______________________________________________
>> community-kerala mailing list
>> communit...@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/community-kerala

Syam kumar

unread,
Oct 18, 2020, 2:05:19 AM10/18/20
to shine at mozilla, community-kerala, Akshay S Dinesh, Kumaresan C S
Hey shine,
Already raised the bugs for DNS change and blog creation.
Mail DNS change dkim creation is pending since DNS is not verified.
For media wiki we can use plesk panel instance.
@shine
Once DNS change is done you will get admin access to the wp site.
Regards,
Syam

On Fri, 16 Oct, 2020, 10:07 pm shine at mozilla, <shine....@gmail.com>
wrote:

> Update
>
> This has been long overdue. I have initiated communication with the MCWS
> team and have gotten some good progress from their end.
>
> They've set up a new WordPress instance to host the Mozilla Kerala blog
> and have created bugzilla bugs to change DNS records.
>
> Though I have gotten access to the new blog instance, I don't have the
> necessary privileges to perform a bulk migration yet. I have asked for it
> and am hoping for a positive update.
>
> More updates will follow as they come.
>
> ~ shine
>
> On Sun, Jul 12, 2020, 11:51 PM shine at mozilla <shine....@gmail.com>

shine at mozilla

unread,
Oct 18, 2020, 2:05:19 AM10/18/20
to Syam kumar, community-kerala, Akshay S Dinesh, Kumaresan C S
> For media wiki we can use plesk panel instance.

We had started using MozillaWiki at wiki.mozilla.org. We did migrate some
of our pages from the old MediaWiki instance to MozillaWiki as well.
I do not think we are actively using MediaWiki right now or will do in the
future. Hence, I am in favour of continuing to use MozillaWiki as a wiki
for the minor use-cases we would have for a wiki. I do not see the need to
maintain a separate wiki instance to host a few pages. Especially when a
more encompassing instance already exists.

> Once DNS change is done you will get admin access to the wp site.

Do we have to wait till the DNS change is complete? I do not think that the
2 tasks are dependent on each other that they can be done only
synchronously. I believe that we can do the migration in parallel so that
the blog is already migrated and ready by the time the DNS change is
complete.

~ shine

On Fri, Oct 16, 2020, 10:12 PM Syam kumar <sy...@mozilla-community.org>
wrote:

shine at mozilla

unread,
Oct 18, 2020, 2:05:19 AM10/18/20
to community-kerala, Akshay S Dinesh, Kumaresan C S, sy...@mozilla-community.org
0 new messages