My next commit will delete your whole site

[ringmaster]

I'm about to commit changes on gsnedder's dbupgrade branch to svn
trunk. This change will convert the active Habari database into the
UTF-8 character set.

DOES THIS APPLY TO ME?
If you are running a development (svn trunk) version of Habari, this
may apply to you.
If you are running from a zip file, this does not apply to you.

I've tested it a few times, even on a large dataset, and it seems to
work fine. Nonetheless...

BACKUP YOUR DATABASE BEFORE YOU APPLY THIS REVISION!

Contrary to my hyperbolic subject line, I don't expect this code to
delete anything, but if for some reason the code explodes on your
server and you lose everything, don't come crying to me that you
didn't make a backup. You have been warned.

Owen

[Michael C. Harris]

On Sat, Mar 29, 2008 at 07:59:14PM -0700, ringmaster wrote:
>
> I'm about to commit changes on gsnedder's dbupgrade branch to svn
> trunk. This change will convert the active Habari database into the
> UTF-8 character set.

Worked a charm for me, with a DB that was affected by #201. Everything
is good now. Let there be rejoicing.

--
Michael C. Harris, School of CS&IT, RMIT University
http://twofishcreative.com/michael/blog

[Ali B.]

/me rejoiced :)

Worked well for me as well. Tested on a both a utf8 database and a none utf8 one (Latin2).

--
Ali B / dmondark
http://www.awhitebox.com

[Christian Mohn]

I don't know if it deleted anything, but my site is now officially borked.
My MySQL processes take 100% CPU and my load is skyrocketing.
I do have a backup and I should just revert but I would like to know what my
MySQL server is doing? Sadly I have no clue.

Christian

[Christian Mohn (h0bbel)]

Ok, seems to have been able to sort it. It looks like my sessions
table had 55k records in it and after I emptied that out and restarted
MySQL the site seems to be back up. :)

Christian

[rick c]

Using an SQLite database created before version 0.4.1, this update
broke my site. I had to manually enter the indexes. In the process of
doing that I found that duplicate entries had been entered in the
log_types table, preventing the creation of the index on it. After
deleting the duplicates and entering the indexes, everything was good.

[Geoffrey Sneddon]

On 30 Mar 2008, at 03:59, ringmaster wrote:
>
> I'm about to commit changes on gsnedder's dbupgrade branch to svn
> trunk. This change will convert the active Habari database into the
> UTF-8 character set.

Is there any reason why this hasn't been commited into the branch to
get some testing before it goes into trunk, especially seeming it is a
high-risk change?

> I've tested it a few times, even on a large dataset, and it seems to
> work fine. Nonetheless...

Of note on the large dataset point is the fact that on a large dataset
it will take a while to execute, and the CPU usage will all be on the
MySQL side.

--
Geoffrey Sneddon
<http://gsnedders.com/>

[Christian Mohn]

A while? I let mine run for over an hour. But, 55k sessions are insane
anyway which may be indicative of other issues.

Christian

> -----Original Message-----
> From: habar...@googlegroups.com [mailto:habar...@googlegroups.com]

[Geoffrey Sneddon]

On 30 Mar 2008, at 22:00, Christian Mohn wrote:

> A while? I let mine run for over an hour. But, 55k sessions are insane
> anyway which may be indicative of other issues.

A while is hard to quantify :) It's really relative to the about of
content you have in the database, and not so much the number of rows
(e.g., one post with 5000 characters would be slower to convert than
1000 with two characters in each). But yeah, 55k sessions makes me
thing something else is wrong :)

[Christian Mohn]

Anyhow, should all my Habari tables now be UTF8? habari__themes is still
latin1_swedish_ci while the rest is either utf8_unicode_ci or binary

Christian

> -----Original Message-----
> From: habar...@googlegroups.com [mailto:habar...@googlegroups.com]
> On Behalf Of Geoffrey Sneddon
> Sent: 31. mars 2008 13:54
> To: habar...@googlegroups.com
> Subject: [habari-dev] Re: My next commit will delete your whole site
>
>
>

[Andy C]

I sync'ed up from SVN and I had a single error where a quotation mark
led to the truncation of a recent article.

Easily fixed but, rather than scan 769 posts and 1500 comments, does
the conversion utility has log any errors/warning or dire threats.

[Geoffrey Sneddon]

On 31 Mar 2008, at 20:16, Christian Mohn wrote:

> Anyhow, should all my Habari tables now be UTF8? habari__themes is
> still
> latin1_swedish_ci while the rest is either utf8_unicode_ci or binary

All should be. themes is not, because it no longer exists. So it
should really have been automagically deleted.

[Geoffrey Sneddon]

On 31 Mar 2008, at 20:58, Andy C wrote:

> I sync'ed up from SVN and I had a single error where a quotation mark
> led to the truncation of a recent article.
>
> Easily fixed but, rather than scan 769 posts and 1500 comments, does
> the conversion utility has log any errors/warning or dire threats.

I don't know how to have non-fatal error handling when changing
encoding while using MySQL. The only way any invalid data should get
in the database in the first place is through something (most likely
the browser) screwing up.

[Christian Mohn]

Ah, that explains why it isn't updated. :) I'll get rid of it manually.

> -----Original Message-----
> From: habar...@googlegroups.com [mailto:habar...@googlegroups.com]
> On Behalf Of Geoffrey Sneddon
> Sent: 1. april 2008 01:02
> To: habar...@googlegroups.com
> Subject: [habari-dev] Re: My next commit will delete your whole site
>
>
>

[Michael C. Harris]

On Mon, Mar 31, 2008 at 12:58:27PM -0700, Andy C wrote:
>
> I sync'ed up from SVN and I had a single error where a quotation mark
> led to the truncation of a recent article.

I've found this as well. A post truncated at an apostrophe.