Avatar

Problems with the upgrade procedure from 2.4.x to 2.5 (Bugs)

by Auge ⌂, Friday, September 23, 2022, 20:06 (12 days ago)

Hello

During the last weeks we saw a few error reports about issues during the upgrade from a 2.4.x version to the currently latest stable version 20220803.1 (2.5.4). Some of these issues are caused by previous tests of a development version of the 2.5-branch and a later downgrade to a 2.4-x version with orphaned but remaining tables for 2.5 in the database. That broke the upgrade script because we never thought about such a situation when writing the code. Some other issues are caused by our (mainly my) carelessness and for us unexpected server configuration.

I opened a pull request on Github as work in progress (unfinished at the time of writing) to solve the issues, where possible. To keep the reports at one place where i can find them easily I will tie them now to this thread.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Update from 2.4.20 to 20220803.1 failed, two issues so far

by Nico Hoffmann @, Sunday, September 04, 2022, 10:21 (31 days ago) @ Auge

Hello,

I tried the update like written in the subject and experienced two issues. Both issues break the update and leave the database in a state which is not working for the old version.

1. the update script printed out "Database error in line 452: Specified key was too long; max key length is 767 bytes" and the update did not finish.
I noticed that this issue was discussed already, and I also did some googeling. Probably this isuue is somehow related to an old SQL version, in my case it is "Server version: 5.5.62 MySQL Community Server (GPL)", i.e. I agree with the explanation from user Auge.

I did not really understand if there is already a bugfix. If yes, it looks like it did not work properly. Maybe the best solution is a SQL update.


2. just for fun i tried a quick-and-dirty hack (not recommended for others). I changed in the update script line 448 "VARCHAR(256)" to "VARCHAR(255)", because 3 * 255 < 768. And indeed, this error was gone, but another appeared: "Database error in line 452: Duplicate entry 'abcdef@domain.tld' for key 'key_user_email'". It turned out that there are two different usernames with the same email address.
Thats a case of a bad administration, in the first place :-) I will remove the double entry.

But I think it should checked before updating or at least mentioned in the update hints.


I installed the backup of my current forum, so there is no damage and I can prepare the next try without stress. While installing the database backup I noticed the new version added four tables. i.e. PREFIX-akismet_rating, PREFIX-b8_rating, PREFIX-b8_wordlist, and PREFIX-uploads.
It turned out that these tables remained in the database after installing the backup and caused errors at the next try.

I would suggest to offer some short fallback hints in the update description, maybe:
"Before update: back up database and php/html code.
If it did not work:
replace code with your backup
replace database with your backup
delete the newly added tables, if still present"

have a nice day!

Avatar

Update from 2.4.20 to 20220803.1 failed, two issues so far

by Auge ⌂, Sunday, September 04, 2022, 16:09 (31 days ago) @ Nico Hoffmann

Hello

Und ewig grüßt das Murmeltier … Groundhog Day again.

I tried the update like written in the subject and experienced two issues. Both issues break the update and leave the database in a state which is not working for the old version.

1. the update script printed out "Database error in line 452: Specified key was too long; max key length is 767 bytes" and the update did not finish.
I noticed that this issue was discussed already, and I also did some googeling. Probably this isuue is somehow related to an old SQL version, in my case it is "Server version: 5.5.62 MySQL Community Server (GPL)", i.e. I agree with the explanation from user Auge.

I did not really understand if there is already a bugfix. If yes, it looks like it did not work properly. Maybe the best solution is a SQL update.

Yes, I thought I fixed it. But when this problem encountered for you with the latest available version 20220803.1, it obviously doesn't work in every case.

2. just for fun i tried a quick-and-dirty hack (not recommended for others). I changed in the update script line 448 "VARCHAR(256)" to "VARCHAR(255)", because 3 * 255 < 768.

I will check, if this is an appropriate way, a.k.a. a not so quick-and-dirty hack as you think, to solve the issue. I solved the same issue with setting the charset to utf8 (without the appendix mb4). That way the maximum index length does not exceed because of the different charset. But it was obviously the wrong place to do this.

And indeed, this error was gone, but another appeared: "Database error in line 452: Duplicate entry 'abcdef@domain.tld' for key 'key_user_email'". It turned out that there are two different usernames with the same email address.
Thats a case of a bad administration, in the first place :-) I will remove the double entry.

I don't know, if this is really a case of a "bad administration". Until version 20220508.1 (2.5.0) it was simply possible to add more than one account with the same e-mail-address. I for myself used this possibility regulary for testing purposes.

It's a trap when updating to a version in the 2.5 branch.

But I think it should checked before updating or at least mentioned in the update hints.

Yes, you are right.

Once we changed the collation of the column username in the userdata table because the database found the name "Hans" and the fictive name "Häns" to be identical. The update script tested the database for such pseudoidentical values, to enforce the admin to solve the problem in collaboration with the affected users before the update.

This time (making the e-mail-address unique) we didn't checked the values for doublettes. That is a failure. I will have to completely rework the update script.

I installed the backup of my current forum, so there is no damage and I can prepare the next try without stress. While installing the database backup I noticed the new version added four tables. i.e. PREFIX-akismet_rating, PREFIX-b8_rating, PREFIX-b8_wordlist, and PREFIX-uploads.
It turned out that these tables remained in the database after installing the backup and caused errors at the next try.

Hmm. The update script doesn't check for already existing tables. Normally this will not happen, but in your case with a previoulsy partially failed upgrade, it will fail.

I would suggest to offer some short fallback hints in the update description, maybe:
"Before update: back up database and php/html code.
If it did not work:
replace code with your backup
replace database with your backup
delete the newly added tables, if still present"

ACK

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Update from 2.4.20 to 20220803.1 failed, two issues so far

by Nico Hoffmann @, Sunday, September 18, 2022, 17:06 (17 days ago) @ Nico Hoffmann

Well, the next try :-)

my provider offered a transfer to a new database with MYSQL 5.7 (also possible MYSQL 8, or MariaDB 10.6) and I tried the update. I got these errors:


Fehler!

    Database error in line 363: Incorrect datetime value: '0000-00-00 00:00:00' for column 'edited' at row 1
    Database error in line 378: Incorrect date value: '0000-00-00' for column 'birthday' at row 1
    Database error in line 381: Incorrect date value: '0000-00-00' for column 'birthday' at row 1

The forum itself seems to run, however it shown ugly PHP warnings:


Warning: mysqli_fetch_row() expects parameter 1 to be mysqli_result, boolean given in /PATH/includes/main.inc.php on line 259

Warning: mysqli_free_result() expects parameter 1 to be mysqli_result, boolean given in /PATH/forum/includes/main.inc.php on line 260

Warning: mysqli_fetch_row() expects parameter 1 to be mysqli_result, boolean given in /PATH/forum/includes/main.inc.php on line 262

Warning: mysqli_free_result() expects parameter 1 to be mysqli_result, boolean given in /PATH/forum/includes/main.inc.php on line 263

Warning: mysqli_fetch_row() expects parameter 1 to be mysqli_result, boolean given in /PATH/forum/includes/main.inc.php on line 267

Warning: mysqli_free_result() expects parameter 1 to be mysqli_result, boolean given in /PATH/forum/includes/main.inc.php on line 268

Warning: Cannot modify header information - headers already sent by (output started at /PATH/forum/includes/main.inc.php:259) in /PATH/forum/index.php on line 208

Warning: Cannot modify header information - headers already sent by (output started at /PATH/forum/includes/main.inc.php:259) in /PATH/forum/index.php on line 209

remarks:

- Before the update attempt today the forum was O.K., but I did a similar try two weeks ago. I cannot confirm that this try did not touch the databases. However this should be unlikely.

- In the "entries" table there are entries with "0000-00-00 00:00:00" in the edited row.

- in the "uderdata" table there are entries with "0000-00-00" in the borthday row.

Should I set these entries to NULL manually before updating?

Any suggestions?

Avatar

Update from 2.4.20 to 20220803.1 failed, two issues so far

by Auge ⌂, Monday, September 19, 2022, 09:19 (16 days ago) @ Nico Hoffmann

Hello

Well, the next try :-)

As first: Thank you for your unintentionally tests.

my provider offered a transfer to a new database with MYSQL 5.7 (also possible MYSQL 8, or MariaDB 10.6) and I tried the update. I got these errors:


Fehler!

Database error in line 363: Incorrect datetime value: '0000-00-00 00:00:00' for column 'edited' at row 1
Database error in line 378: Incorrect date value: '0000-00-00' for column 'birthday' at row 1
Database error in line 381: Incorrect date value: '0000-00-00' for column 'birthday' at row 1

That's the pitfalls of the configuration dependent, possible lax data model of MySQL and the default values for columns that was mostly set by Alex years ago (possibly wíthout the necessary knowledge about the corner cases, i.e. much more stringent configurations of MySQL-servers).

The forum itself seems to run, however it shown ugly PHP warnings:


Warning: mysqli_fetch_row() expects parameter 1 to be mysqli_result, boolean given in /PATH/includes/main.inc.php on line 259

Warning: mysqli_free_result() expects parameter 1 to be mysqli_result, boolean given in /PATH/forum/includes/main.inc.php on line 260

Warning: mysqli_fetch_row() expects parameter 1 to be mysqli_result, boolean given in /PATH/forum/includes/main.inc.php on line 262

Warning: mysqli_free_result() expects parameter 1 to be mysqli_result, boolean given in /PATH/forum/includes/main.inc.php on line 263

Warning: mysqli_fetch_row() expects parameter 1 to be mysqli_result, boolean given in /PATH/forum/includes/main.inc.php on line 267

Warning: mysqli_free_result() expects parameter 1 to be mysqli_result, boolean given in /PATH/forum/includes/main.inc.php on line 268

Hmmm, the error in line 259 is the cause of the following errors. This is the complete block in main.inc.php (version 20220803.1).


259 | $count_result = mysqli_query($connid, "SELECT COUNT(*) FROM " . $db_settings['userdata_table'] . " WHERE activate_code = ''");
260 | list($registered_users) = mysqli_fetch_row($count_result);
261 |
262 | if ($settings['count_users_online'] > 0) {
263 |  user_online($settings['count_users_online']);
264 |  $count_result = mysqli_query($connid, "SELECT COUNT(*) FROM ".$db_settings['useronline_table']." WHERE user_id > 0");
265 |  list($registered_users_online) = mysqli_fetch_row($count_result);
266 |  $count_result = mysqli_query($connid, "SELECT COUNT(*) FROM ".$db_settings['useronline_table']." WHERE user_id = 0");
267 |  list($unregistered_users_online) = mysqli_fetch_row($count_result);
268 |  $total_users_online = $unregistered_users_online + $registered_users_online;
269 | }
270 | mysqli_free_result($count_result);

But comparing the line numbers in your error messages with the code one can see, that the messages reports MySQL-errors in lines, where we don't see MySQL-related function calls in the code of version 20220803.1 (as shown above). But when we look into the code of version 2.4.20 (the version from where you started the update) we can see MySQL-related code with a query, that includes columns that are gone with version 2.5. The gone colum is part of the string in the variable $display_spam_query_and, that was generated in the previous block, beginning with line #239.


256 | } else {
257 |  // there are categories
258 |  $count_result = mysqli_query($connid, "SELECT COUNT(*) FROM ".$db_settings['forum_table']." WHERE pid = 0".$display_spam_query_and." AND category IN (".$category_ids_query.")");
259 |  list($total_threads) = mysqli_fetch_row($count_result);
260 |  mysqli_free_result($count_result);
261 |  $count_result = mysqli_query($connid, "SELECT COUNT(*) FROM ".$db_settings['forum_table']." WHERE category IN (".$category_ids_query.")".$display_spam_query_and);
262 |  list($total_postings) = mysqli_fetch_row($count_result);
263 |  mysqli_free_result($count_result);
264 | }
265 | // count spam:
266 | $count_spam_result = mysqli_query($connid, "SELECT COUNT(*) FROM ".$db_settings['forum_table']." WHERE spam = 1");
267 | list($total_spam) = mysqli_fetch_row($count_spam_result);
268 | mysqli_free_result($count_spam_result);

That said, we have to assume, that you forgot to upload the script file(s) (at least the main.inc.php).


Warning: Cannot modify header information - headers already sent by (output started at /PATH/forum/includes/main.inc.php:259) in /PATH/forum/index.php on line 208

Warning: Cannot modify header information - headers already sent by (output started at /PATH/forum/includes/main.inc.php:259) in /PATH/forum/index.php on line 209

These are subsequent errors because of the output of the previous error messages. This errors would not occur, if the error in main.inc.php, line 259 would not happen.

remarks:

- Before the update attempt today the forum was O.K., but I did a similar try two weeks ago. I cannot confirm that this try did not touch the databases. However this should be unlikely.
- In the "entries" table there are entries with "0000-00-00 00:00:00" in the edited row.
- in the "uderdata" table there are entries with "0000-00-00" in the borthday row.

Should I set these entries to NULL manually before updating?

Generally this is a proper idea, but I cannot recommend this without knowing the possible side effects (i.e. when a function relies on the format (0000-00-00 (00:00:00)) and does not take a NULL value into account). There are many more places in the database, where setting the default values to NULL would make IMHO much more sense than the current defaults does.

Any suggestions?

I had a few looks into the code of the update script and I have ideas to improve the script without breaking updates, that have aready been made. I will inspect the code with a focus on the date and datetime topics today and will give a report today in the evening or tomorrow.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Update from 2.4.24 to 20220803.1 failed

by Fritz ⌂ @, Tuesday, September 20, 2022, 15:02 (15 days ago) @ Auge

Hi,

update shows following errors:

Database error in line 298: Table 'mlf2_akismet_rating' already exists
Database error in line 301: Table 'mlf2_b8_rating' already exists
Database error in line 304: Unknown character set: 'utf8mb4'
Database error in line 307: Table 'mlf2_uploads' already exists
Database error in line 327: Table 'w10500-mlf2.mlf2_b8_wordlist' doesn't exist
Database error in line 330: Duplicate entry '1' for key 1
Database error in line 333: Duplicate entry '1' for key 1

Result: Forum no longer accessible.

Avatar

Update from 2.4.24 to 20220803.1 failed

by Auge ⌂, Tuesday, September 20, 2022, 20:34 (15 days ago) @ Fritz

Hello

update shows following errors:

Database error in line 298: Table 'mlf2_akismet_rating' already exists
Database error in line 301: Table 'mlf2_b8_rating' already exists

You had obviously installed a version from the 2.5-branch (2.4.99.x or a 2022-any-number) before the current update attempt. You might have downgraded the version to a stable version of the 2.4-branch but the then obsolete tables remained.

Database error in line 304: Unknown character set: 'utf8mb4'

What version of MySQL operates on your server?

Database error in line 307: Table 'mlf2_uploads' already exists

Same here for the case of a previous installed 2.5-branch version.

Database error in line 327: Table 'w10500-mlf2.mlf2_b8_wordlist' doesn't exist

The creation of the table failed because of the unknown charset utf8mb4 from the error in line 304.

Database error in line 330: Duplicate entry '1' for key 1
Database error in line 333: Duplicate entry '1' for key 1

If the tables would be new, they would be empty and there would be no entry 1, that could be duplicated. If this error occurs, the akismet rating table as well as the b8 rating table are not empty during the update process. So again: You had obviously installed a version from the 2.5-branch (2.4.99.x or a 2022-any-number) before the current update attempt.

So to confirm or deny my assumptions is the first step to get your forum running again. Beside from that I am quite confused because I test a clean installation and updates from a 2.4.20 and the previous version (in this case the version 20220529.1) before every release of a new version and I never experienced errors like yours or like the errors from other threads. ☹️

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Update from 2.4.24 to 20220803.1 failed

by Fritz ⌂ @, Wednesday, September 21, 2022, 08:45 (14 days ago) @ Auge

Hi Auge,

Thanks for the quick reply.
Correct, the tables 'mlf2_akismet_rating' and 'mlf2_b8_rating' as well as 'mlf2_uploads' exist in the database.

You had obviously installed a version from the 2.5-branch (2.4.99.x or a 2022-any-number) before the current update attempt. You might have downgraded the version to a stable version of the 2.4-branch but the then obsolete tables remained.

I don't remember doing that. But it doesn't matter how the tables got into the database.
My question: Can I simply delete these 3 tables, for example with phpMyAdmin?

Database error in line 304: Unknown character set: 'utf8mb4'

What version of MySQL operates on your server?

MySQL 5.0

So to confirm or deny my assumptions is the first step to get your forum running again.

My forum is currently running again in version 2.4.24 after a restore of the database, but of course I want to upgrade to 2.5 to support PHP 8.1.

Cheers
Fritz

Avatar

Update from 2.4.24 to 20220803.1 failed

by Auge ⌂, Wednesday, September 21, 2022, 16:18 (14 days ago) @ Fritz

Hello

Thanks for the quick reply.
Correct, the tables 'mlf2_akismet_rating' and 'mlf2_b8_rating' as well as 'mlf2_uploads' exist in the database.

You had obviously installed a version from the 2.5-branch (2.4.99.x or a 2022-any-number) before the current update attempt. You might have downgraded the version to a stable version of the 2.4-branch but the then obsolete tables remained.

I don't remember doing that. But it doesn't matter how the tables got into the database.
My question: Can I simply delete these 3 tables, for example with phpMyAdmin?

As you are running MLF in version 2.4.24 again, you can delete all tables, that are generated for version 2.5. These are the tables for uploads, B8-wordlist and rating and the akismet rating table.

Database error in line 304: Unknown character set: 'utf8mb4'

What version of MySQL operates on your server?

MySQL 5.0

I cannot say with certainty, in which version of MySQL the charset utf8mb4 was added but in case of MySQL 5.0 would be not able to handle the charset utf8mb4, you will not be able to upgrade the forum script to version 2.5 or any newer version. On the other hand MySQL 5.0 is a really ancient version. Please ask your hosting service/server admin to upgrade to a recent version of MySQL.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Update from 2.4.24 to 20220803.1 failed

by Fritz ⌂ @, Friday, September 23, 2022, 13:00 (12 days ago) @ Auge

Servus Auge,

thank you for your quick and comprehensive reply.

The database is now running MySQL 5.7

Just one more question:
in db_setting.php the following is declared:
$db_settings['b8_wordlist_table'] = 'mlf2_b8_wordlist';
$db_settings['b8_rating_table'] = 'mlf2_b8_rating';
$db_settings['akismet_rating_table'] = 'mlf2_akismet_rating';
$db_settings['uploads_table'] = 'mlf2_uploads';

Should I also delete these lines in db_setting.php prior to upgrading to 20220803.1

Cheers
Fritz

Avatar

Update from 2.4.24 to 20220803.1 failed

by Auge ⌂, Friday, September 23, 2022, 14:19 (12 days ago) @ Fritz

Hello

thank you for your quick and comprehensive reply.

The database is now running MySQL 5.7

Much better. 👍

Just one more question:
in db_setting.php the following is declared:
$db_settings['b8_wordlist_table'] = 'mlf2_b8_wordlist';
$db_settings['b8_rating_table'] = 'mlf2_b8_rating';
$db_settings['akismet_rating_table'] = 'mlf2_akismet_rating';
$db_settings['uploads_table'] = 'mlf2_uploads';

Should I also delete these lines in db_setting.php prior to upgrading to 20220803.1

You can delete these entries without a side effect when you are running a 2.4.x-version. But you can also leave these entries in the database config file because they simply will be ignored in a 2.4.x-version and the file itself will be overwritten when you upgrade to a newer version. So the content would be the same but the bytes would be "new". 😆

Long story short, it is irrelevant.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Update from 2.4.24 to 20220803.1 failed

by Fritz ⌂ @, Friday, September 23, 2022, 14:28 (12 days ago) @ Auge

... and one more question:
The tables created by version 2.4.24 use utf8_general_ci throughout, not utf8mb4
Example:
name varchar(255) utf8_general_ci
Do I have to change every collation to utf8mb4 in the database manually, or will this be done by update_2.4.19.1-2.5.php?

Cheers
Fritz

Avatar

Update from 2.4.24 to 20220803.1 failed, a warning

by Auge ⌂, Friday, September 23, 2022, 17:12 (12 days ago) @ Fritz

Hello

... and one more question:
The tables created by version 2.4.24 use utf8_general_ci throughout, not utf8mb4
Example:
name varchar(255) utf8_general_ci
Do I have to change every collation to utf8mb4 in the database manually, or will this be done by update_2.4.19.1-2.5.php?

This is one of the tasks, the update script is performing.

Unfortunately there are currently a few pitfalls in the update script, that are caused by MySQL-server-configurations we wasn't aware of and wasn't be able to test before the releases of the first versions of the 2.5-branch (20220508.1 and later). We have to solve these issues first, before we can recommend an upgrade to the 2-5-branch for forum operators, who are not really familiar with PHP, MySQL and the code of the forum script to inspect and possibly solve issues themselfes.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Update from 2.4.24 to 20220803.1 failed, a warning

by Fritz ⌂ @, Friday, September 23, 2022, 17:53 (12 days ago) @ Auge

Servus Auge,

OK, I can wait. Thanks for the warning.

Cheers
Fritz

Update from 2.4.24 to 20220803.1 failed, a warning

by Fritz ⌂ @, Saturday, October 01, 2022, 11:19 (4 days ago) @ Auge

OK. I ran the update. No risk - no fun. Everything worked fine. :-)

Avatar

I am having issues installing My little forum!

by monaco, Tuesday, September 20, 2022, 18:47 (15 days ago) @ Auge

It says some errors after finishing installing ! like "Missing Config/VERSION !

I would really want a solution for this thanks !

Avatar

I am having issues installing My little forum!

by Auge ⌂, Tuesday, September 20, 2022, 20:06 (15 days ago) @ monaco

Hello

It says some errors after finishing installing !

Which errors (beside the missing config/VERSION)?

like "Missing Config/VERSION !

Why didn't you upload this file? The readme says in section Installation, point 2. one has to upload the whole script package (including config/VERSION).

I would really want a solution for this thanks !

Here you go.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Avatar

I am having issues installing My little forum!

by Auge ⌂, Wednesday, September 21, 2022, 16:10 (14 days ago) @ monaco

Hello

Output:
"Error in line 42: Missing the file config/VERSION."

But:

https://asciinema.org/a/3mGTiEHRBKnIcpqyDgtVghzp3

Ok, you proved me wrong. I see the content of the file in your screencast.

Sorry for insisting, you are sure to run the installation script in the directory /home/shoqni/web/shoqni.com/public_html/forum/install/? The code in the line 41, which causes the error message in line 42, asks only for the existience of this file with a relative path from the installation script to the file in question (if(!file_exists('../config/VERSION')) …).

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Avatar

I am having issues installing My little forum!

by monaco, Wednesday, September 21, 2022, 16:28 (14 days ago) @ Auge

Hello

Output:
"Error in line 42: Missing the file config/VERSION."

But:

https://asciinema.org/a/3mGTiEHRBKnIcpqyDgtVghzp3


Ok, you proved me wrong. I see the content of the file in your screencast.

Sorry for insisting, you are sure to run the installation script in the directory /home/shoqni/web/shoqni.com/public_html/forum/install/? The code in the line 41, which causes the error message in line 42, asks only for the existience of this file with a relative path from the installation script to the file in question (if(!file_exists('../config/VERSION')) …).

Tschö, Auge

Well managed to install it and there is for demo https://shoqni.com/bbs .... but !

My session expires and seems that nothing that I want to apply to forum such as posting or adding categories is working does that have to do that I use nginx instead of Apache2 or !

RSS Feed of thread