A request to the audience and especially to forum owners

Hello

When we released the first versions of the 2.5-branch (beginning with version 20220508.1) I wrote the upgrade script and made a few mistakes, that can lead to a broken installation. Because of that and because I had lost track of it all I completely rewrote the upgrade script and wrote a complete set of instructions for every version group from one can start an upgrade instead splitting the process into several single steps for every next version like it was done before.

I made several tests myself. Starting from checking for the version of MySQL or MariaDB that is required to run the forum in the new versions over checking for E-mail addresses used in multiple user accounts and stopping the upgrade process if one of the requirements is not fulfilled to changing the database engine type of the tables to InnoDB in a loop (no change if the tables are already defined in the engine type InnoDB) I performed many single unit tests (testing only one function/step at once).

I also performed upgrades from several versions to the current stand. Starting versions was version 2.4.24 (as example for all supported stable versions of the 2.4-branch), version 2.4.99.1 and 2.4.99.3 as examples for the testing versions and versions 20220508.1 and 202205017.1 as stable versions of the 2.5-branch with different states of their database structure when a forum was installed or upgrades with the upgrade script as it was before.

Now I am at a point, where I need input from outside. I want to prevent to introduce new pitfalls and bugs to the upgrade process. Because of this I want to encourage people who are able to read PHP and SQL soucecode and who have a Github account to inspect the code of the PR (pull request) and to ask questions, to follow my thoughts and mention possible errors in code and thoughts.

I am also interested in upgrade tests of forum owners/operators with copies of their running forums. One can upgrade from one of the following versions.

- 2.4.19 (new, was not possible before)
- 2.4.19.1
- 2.4.20
- 2.4.21
- 2.4.22
- 2.4.23
- 2.4.24
- 2.4.99.1
- 2.4.99.3
- 20220508.1
- 20220509.1
- 20220517.1
- 20220529.1
- 20220803.1 (upgrade possible with the further changes made 2024-03-11)

The not mentioned testing versions 2.4.99.0 and 2.4.99.2 was never released to the public. Noone except me could have access to these versions. ;-)

[delete]It is also important to know that version 20220803.1 is not yet among those for which an upgrade is possible.[/delete]

The previous paragraph is outdated. With the changes, made to the code of the pull request on 2024-03-11 an upgrade from version 20220803.1 to the current development stand is in fact possible.

To do this upgrade one needs …

1. … access to a tool like phpMyAdmin, the MySQL Work Bench or similar tools to perform operations in and to create copies of the tables of the running forum (in example by copying all tables of the running forum from mlf2_table_name i.e. to mlf3_table_name, …

2. … to put a copy of the script files of the current forum to a new directory, …

3. … to adapt the table names in the file config/db_settings.php of the copied forum to the names of the new, copied tables …

4. … and to change the following settings in the table mlf3_settings

- forum_address to the URL with the new directory
- forum_name to distinguish it from the always running forum
- session_prefix to a different value than the one of the running instance (one will need to login again) to prevent a mixup of the user session of the administrator account

As last step of the preparations please download the branch with the fixed upgrade script from my fork of My Little Forum. Open the submenu for downloading with the button named "<> Code" (marked with 1 in the screenshot) and download the ZIP from the bottom of the overlay (marked with 2 in the screenshot).

[image]

After unzipping load up the files config/VERSION and the upgrade script update/update_2.4.19-2.5.php itself to the copy of the forum. Go to the admin area of the forum copy and run the upgrade with the offered script update_2.4.19-2.5.php. There might be further upgrade scripts depending of the current version of your forum (update_2.3.5-2.4.php or update_2.4.19.1-2.5.php). Make sure to use the correct script update_2.4.19-2.5.php for the upgrade. After performing the database changes in the first step load up the files, that are mentioned to be updated (and deleted) to have a working upgraded copy without a bunch of error messages on every page. You'll end up with the version 20240308.1 which is not an official release. This version includes a few changes that was made after the release of the version 20220803.1 but not a single one of the much discussed database performance fixes. The only fix is the one for the size of the column for storing IPs because this bug can lead to server errors.

As last go the the advanded settings page and change the following settings.

- delete_inactive_users = 0
- notify_inactive_users = 0

The two settings, set to value 0, prevents the notification of inactive users about the possible deletion of their accounts and the deletion in itself. That would be bad in the copy of the real forum. In the real forum it could be a judical necessity (i.e. in the EU).

Normally one has also to set the setting (forum_enabled) to the value 1. This would reactivate a forum after the upgrade but is not necessary in the case of an upgrade test in a copy of a real forum.

After performing the upgrade I would like to see the table definitions of the upgraded forum. Please perform an export with phpMyAdmin, MySQL WorkBench or a similar tool you used for the first step of the preparations. The export is done in phpMyAdmin in the database view (do not open it in the view of a single table, this would lead to unnecessary additional work) in the Export tab. Activate the option "Customised - display all possible options" (I translated it from german, I don't know th original option text). You'll see a table with the names of all database tables with checkboxes for structure and data. We need only the structure of the upgraded tables (table names beginning with the prefix mlf3_ (using my example names from above)). Activate the option "Show output as text" in the section Output. As last click the export button. Copy the result and insert it here in a new reply to this posting together with the information about the version from where you started the upgrade.

At that point one can delete the forum copy and also delete the corresponding copies of the forums database tables.

Very long story very short, the complete rewrite consists of over sixty individual changes and I need support for this really big task to prevent the introduction of new bugs while solving old ones.

Thank you in advance. If you have any questions about the tasks to do please ask here before you begin.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Tags:
development, testing

Hello

After nearly eight days without any response, I can't help but realise, that there seems to be no interest in co-operation to bring the project out of the current crisis. I hoped, that a few members of the small audience are willing to support especially the development of the new upgrade script because this is the stopping point for every further change. I asked the audience outside the development team, because Micha as the other active team member is currently not available to help.

There's no space for straight development of new features without a working upgrade script. Because the upgrade script was broken in the past, I have asked for input from other people in the hope, that they will find possible existing errors, that I have overlooked. This was to prevent to switch from one broken state to another one.

That said, I will commit the rework of the upgrade script, independent from possible existing hidden faults. We'll have to wait for response in form of error reports to learn about possible existing errors.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Tags:
development, testing, upgrade

I attempted to download the fix_upgrade branch from the link, but it does not appear to be there any more. Let me know where I can access it, and I'll report back with my findings. We're currently on 2.4.24, so that would also be my baseline attempt.

[EDIT: I see there are updated scripts now in the master branch, and suspect you've merged all fix_upgrade files into master. Would like to confirm I should use master now instead of fix_upgrade.]

I attempted to download the fix_upgrade branch from the link, but it does not appear to be there any more.

Yes, that's true ...

[EDIT: I see there are updated scripts now in the master branch, and suspect you've merged all fix_upgrade files into master. Would like to confirm I should use master now instead of fix_upgrade.]

... because I meanwhile merged the changes into the master branch. So, if you are willing to support the project by testing the changes, you can carry out the test with the master branch.

I tested myself an upgrade starting with version 2.4.20, 2.4.99.1, 2.4.99.3, 20220508.1 and 20220529.1. I installed a copy of my testing forum on version 2.4.20 in each case, upgraded it to one of the interim versions, that should be the starting point of the test and did the upgrade to the newest target version. In the end I found no issues with the changes in the database structure (after several corrections).

Nevertheless, I recommend to undertake the upgrade test with a copy of your live forum (script code and database). I fyou find issues please report them here or, if you have a Github account, as an issue.

Thank you for your effort.

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Our current forum is running on 2.4.24, so this was my baseline upgrade after making a copy of everything. I ran this upgrade out of the box without checking anything prior.

1) Our DB does have duplicate email addresses and the upgrade correctly flagged the culprits.

2) The one upgrade error I received was the following:
Error!
Database error in line 298: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ' KEY `b8_spam` (`spam`), KEY `B8_training_type` (`training_type`)) ENGINE=Inn...' at line 1

With a quick check of Line 298 in update/update_2.4.19-2.5.php, I see 2 commas after PRIMARY KEY (`eid’):
if (!@mysqli_multi_query($connid, "CREATE TABLE IF NOT EXISTS `" . $db_settings['b8_rating_table'] . "` (`eid` int(11) NOT NULL, `spam` tinyint(1) NOT NULL DEFAULT '0', `training_type` tinyint(1) NOT NULL DEFAULT '0', PRIMARY KEY (`eid`), , KEY `b8_spam` (`spam`), KEY `B8_training_type` (`training_type`)) ENGINE=InnoDB CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;")) $update['errors'][] = 'Database error in line '.__LINE__.': ' . mysqli_error($connid);

3) After changing the code in the script and running it again, the upgrade was successful.

4) I didn't see the same issue in any of the other CREATE TABLE scripts.

5) I will also run through a 2.4.24 --> 20220803.1 --> 20240308.1 upgrade, since I'm guessing many forums are currently running off 20220803.1. Will report back.

Hello

2) The one upgrade error I received was the following:
Error!
Database error in line 298: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ' KEY `b8_spam` (`spam`), KEY `B8_training_type` (`training_type`)) ENGINE=Inn...' at line 1

With a quick check of Line 298 in update/update_2.4.19-2.5.php, I see 2 commas after PRIMARY KEY (`eid’):
if (!@mysqli_multi_query($connid, "CREATE TABLE IF NOT EXISTS `" . $db_settings['b8_rating_table'] . "` (`eid` int(11) NOT NULL, `spam` tinyint(1) NOT NULL DEFAULT '0', `training_type` tinyint(1) NOT NULL DEFAULT '0', PRIMARY KEY (`eid`), , KEY `b8_spam` (`spam`), KEY `B8_training_type` (`training_type`)) ENGINE=InnoDB CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;")) $update['errors'][] = 'Database error in line '.__LINE__.': ' . mysqli_error($connid);

Thank you for spotting this. It was introduced by me with an addition I made after testing the specific section. I'll provide a fix.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

As noted in the previous post, our current forum is 2.4.24, so this one was a 2-step upgrade to get to 20240308.1.

I made a copy of the site and removed any duplicate user emails before proceeding, as I believe the upgrade to 20220803.1 would crash otherwise.
I then performed a successful upgrade to 20220803.1
I then performed a successful upgrade to 20240308.1, using the code located in GitHub/master.

Some Notes:
1) This script did not crash on the same CREATE TABLE step as the direct upgrade from 2.4.24. I am guessing this is because the b8_rating_table was already created during the upgrade to 20220803.1, but I do see several DROP TABLE commands prior to the CREATE TABLE in Lines 290-294. The comment on the DROP TABLE commands says it was for pre-release testing. Since Im not exactly sure if these lines (290-294) are working as designed, I'm just making a note to you.

2) In your original comments, you note that this upgrade doesn't include any much discussed database performance fixes. Our site DB is quite large (~500,000 posts), and runs very slowly on any 2.5 version of MLF due to the performance of some read queries which JOIN on the new Spam tables. I had previously dug into improving performance for our site when running 20220803.1, but never did come up with an acceptable solution. I am willing to look into this further if appropriate, and also wanted to see if you know of any potential improvements that I could test out on our site running 20220803.1 or 20240308.1.

Regards,

Joe

Hello

As noted in the previous post, our current forum is 2.4.24, so this one was a 2-step upgrade to get to 20240308.1.

I made a copy of the site and removed any duplicate user emails before proceeding, as I believe the upgrade to 20220803.1 would crash otherwise.

The script itself would not crash in the actual meaning but you would end with a broken forum. So yes, that was a sensible precautionary measure.

I then performed a successful upgrade to 20220803.1
I then performed a successful upgrade to 20240308.1, using the code located in GitHub/master.

Nice. :-)

Some Notes:
1) This script did not crash on the same CREATE TABLE step as the direct upgrade from 2.4.24. I am guessing this is because the b8_rating_table was already created during the upgrade to 20220803.1, but I do see several DROP TABLE commands prior to the CREATE TABLE in Lines 290-294. The comment on the DROP TABLE commands says it was for pre-release testing. Since Im not exactly sure if these lines (290-294) are working as designed, I'm just making a note to you.

Ok, I see there is room for clarifications.

Until version 20220803.1 the upgrade script worked this way: All upgrade steps are additions to previously performed steps (inside a script run). All steps, that names the starting version in its if-clause, get executed one after the other.

Due to errors in the compilation of the necessary changes (namely the uniqueness of the e-mail-addresses in the accounts, to big indexes in case of MySQL < 5.7.7 (or MariaDB < 10.2.2), sizes of specific columns and double indices) it was not possible for me to adapt the necessary changes and fixes to the correct version steps. I lost the track in the end and decided to reorganíse the upgrade script.

Now the upgrade script works by providing a complete upgrade process for every starting version (or groups of versions with an identical database structure). This includes all steps from the changes in the table structure (new, changed or removed columns, new tables) to new, changed or deleted values in table rows.

Additionally – and here we are at the deleting and recreation of the new tables – we got a few reports about broken upgrades because people once upgraded to one of the accessible testing versions (2.4.99.1, 2.4.99.3) but downgraded their forum to a stable version of the 2.4-branch (up to version 2.4.24) again. This resulted in a database structure with the matching tables for 2.4.x but also with the not deleted tables from the testing versions (2.4.99.1, 2.4.99.3). When upgrading now from 2.4.x to a version of the 2.5-branch, the upgrade script failes to create the new but already existing tables.

Because of this the script deletes any existing tables that it is supposed to create and creates them again afterwards when we can be sure, that the database is cleaned up. This step is only necessary, when it comes to an upgrade starting with a stable vesion of the 2.4-branch (2.4.19 to 2.4.24). All other versions, that are allowed to be upgraded with this script, already contain the tables in question, so there is no CREATE statement for one of these tables in these cases.

2) In your original comments, you note that this upgrade doesn't include any much discussed database performance fixes. Our site DB is quite large (~500,000 posts), and runs very slowly on any 2.5 version of MLF due to the performance of some read queries which JOIN on the new Spam tables. I had previously dug into improving performance for our site when running 20220803.1, but never did come up with an acceptable solution. I am willing to look into this further if appropriate, and also wanted to see if you know of any potential improvements that I could test out on our site running 20220803.1 or 20240308.1.

I re-read the old thread about this big issue a few days ago but also did not came to a good, solid solution until now.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Hello Joe,

I found another error in the upgrade script, which could have influence the performance of the queries, that reads the akismet rating table. In another thread from the last year someone (I thought it was you but I do not find the posting) reported, that the table mlf2_akismet_rating lacks two of the indexes, when the new version was an upgrade and not a fresh installation.

Because ot this I aaded code to check the presence of indexes in every case of an upgrade of this table [1] and to add the indexes when they are missing. But I copy'n'pasted the ALTER-TABLE-query and did not replace the table name with the correct one. So the check notices the missing indexes, but the ALTER-TABLE-query tries to create the index in the wrong table. Nothing happens there, as the column in question does not exist there, but the index is still missing in the correct place.

I corrected the upgrade script today but I have not a forum with enough content to make a proper query-speed-test. Can you please check for a possibly changed loading performance in your testing forum after applying the changes from the linked commit? Thank you in advance.

Tschö, Auge

[1]: every upgrade case except the upgrade starting with versions 2.4.19 to 2.4.24, where the table gets created

--
Trenne niemals Müll, denn er hat nur eine Silbe!

As noted previously, our site has a large DB, so it runs quite slow on any 2.5 code (9-13 seconds main page load, with the addition of the akismet and b8 tables). I revisited the (three) worst performing queries on our site and seem to have found a good solution for each that makes performance (< 1 second main page load) close to 2.4.24.

/includes/main.inc.php queries:
total_spam
total_postings

/includes/index.inc.php
latest_postings

I'd be glad to share these with you if you're interested. I still have some testing to do to make sure I didn't break the code for different configurations than our own, but so far they look solid.

Joe

Hi,

I'd be glad to share these with you if you're interested. I still have some testing to do to make sure I didn't break the code for different configurations than our own, but so far they look solid.

Sounds fine! If you like, you can create a new branch via github with you changed/improved code.

/Micha

--
applied-geodesy.org - OpenSource Least-Squares Adjustment Software for Geodetic Sciences

Hi Micha / Auge / friday-admin,

I'm now done with my code changes and extensive testing of the SQL performance modifications. Since it does not appear that I can write to My-Little-Forum, I created a fork and pull request for the changes. 4 total includes files changed: entry, index, main, and thread.

Performance Note: In all instances tested, page load time went from 10-15 seconds in base 20240308.1 down to sub 1 second with these modifications. I do see about a 40% increase in page load time over 2.4.24 (from 500ms to 800ms avg), but page load is still acceptably under 1 second, even within our forum of 500,000 posts / 40,000 threads. If these modifications work for you, it should hopefully eliminate a need to further customize SQL queries based on forum spam settings.

Joe

Hello Joe

I'm now done with my code changes and extensive testing of the SQL performance modifications. Since it does not appear that I can write to My-Little-Forum, I created a fork and pull request for the changes.

That's the normal way, even for us team members (partially). Also we must provide our changes with a pull request. I do this, for historic reasons, from my fork, Micha typically with a pull request from a branch in the original repository (because he – as a team member – has access to the original repository).

4 total includes files changed: entry, index, main, and thread.

Until now I did not dive into your code in the PR. The list of changed files tells me, that these are changes in the operation code of a running forum and no changes in the upgrade script. So there shouldn't be interferences between these two "sections".

Performance Note: In all instances tested, page load time went from 10-15 seconds in base 20240308.1 down to sub 1 second with these modifications.

😀👌

That's a very remarkable enhancement.

I do see about a 40% increase in page load time over 2.4.24 (from 500ms to 800ms avg), but page load is still acceptably under 1 second, even within our forum of 500,000 posts / 40,000 threads.

With increasing complexity of the database queries, a certain slowdown is to be expected. However, your measurements remain within the limits of what is initially acceptable.

If these modifications work for you, it should hopefully eliminate a need to further customize SQL queries based on forum spam settings.

At the very least, it would make it for a start unnecessary to split the queries according to registered and non-registered users, according to whether spam protection measures are switched on or not or according to other unnamed criteria.

If and when we see the necessity to split queries I would like to do this in stored procedures on the MySQL-server but not in the PHP-code. To do this, we need a reliable way of recognising whether the database user of a forum operator is allowed to create, change, delete and execute stored procedures. If not, an upgrade to the corresponding version must be prevented. With the complete conversion of the database engine to InnoDB, we have at least laid the foundation for this because we can now work with transactions which is a precondition to reliably create stored procedures with the mysqli-library (as far as I see in code examples).

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Performance Note: In all instances tested, page load time went from 10-15 seconds in base 20240308.1 down to sub 1 second with these modifications.

😀👌

That's a very remarkable enhancement.

I do see about a 40% increase in page load time over 2.4.24 (from 500ms to 800ms avg), but page load is still acceptably under 1 second, even within our forum of 500,000 posts / 40,000 threads.

With increasing complexity of the database queries, a certain slowdown is to be expected. However, your measurements remain within the limits of what is initially acceptable.

I had forgotten that I also made a modification to base 2.4.24 for our forum to significantly improve latest_postings performance for non-logged in users (with forum running on MariaDB and not MySql), so these page load numbers would be unique to us for that scenario. For logged in users, the above comparison is relevant and acceptable.

If these modifications work for you, it should hopefully eliminate a need to further customize SQL queries based on forum spam settings.

At the very least, it would make it for a start unnecessary to split the queries according to registered and non-registered users, according to whether spam protection measures are switched on or not or according to other unnamed criteria.

If and when we see the necessity to split queries I would like to do this in stored procedures on the MySQL-server but not in the PHP-code. To do this, we need a reliable way of recognising whether the database user of a forum operator is allowed to create, change, delete and execute stored procedures. If not, an upgrade to the corresponding version must be prevented. With the complete conversion of the database engine to InnoDB, we have at least laid the foundation for this because we can now work with transactions which is a precondition to reliably create stored procedures with the mysqli-library (as far as I see in code examples).

Yes agreed that Stored Procedures would be the way to go at some point. From last year, I do see Andreas Schneider had written a quick procedure to test out whether the forum admin had SP write capability, and something like that could be used to test before proceeding with an upgrade. I had also tested it out on our own forum with positive results. Hopefully / likely that very few forum administrators would be affected by this change.

FYI, I ran into performance issues related to most of the queries that use the display_spam_query_(and or where) variables. I've ended up refactoring a fair bit of php and sql code relating to those SQL statements for performance, comprehension and simplicity purposes.

I'm currently working through extensive testing of my code changes now, and hope to close this out fairly soon. Fingers crossed.

Hello

FYI, I ran into performance issues related to most of the queries that use the display_spam_query_(and or where) variables. I've ended up refactoring a fair bit of php and sql code relating to those SQL statements for performance, comprehension and simplicity purposes.

I'm currently working through extensive testing of my code changes now, and hope to close this out fairly soon. Fingers crossed.

I expect your comment in the Github-issue #709 about missing indexes is a direct consequence of your comments here. You named an obvious cause, the missing indexes for the table mlf2_akismet_rating in case of an upgrade. I found the issue myself and made a commit, that corrects the copy'n'paste error I made, for the pull request with the fixes of the upgrade script.

Did you find further problems? Oh, and can you say something about the seen running time of the upgrade script of roughly five minutes in a forum with around 600,000 entries? I would expect a similar running time of the upgrade script in your half-million-entries forum.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

I expect your comment in the Github-issue #709 about missing indexes is a direct consequence of your comments here. You named an obvious cause, the missing indexes for the table mlf2_akismet_rating in case of an upgrade. I found the issue myself and made a commit, that corrects the copy'n'paste error I made, for the pull request with the fixes of the upgrade script.

Hi Auge, once I noticed the missing indexes, I did add them manually. Unfortunately, adding these did not materially improve performance on base 20240308.1 code. I then proceeded to make my performance modifications with these indexes properly in place. Thank you for also catching this and creating the pull request for the correction.

Joe

FYI, the DB structure paste is too large to post in a response here (~14,200 char paste, 10,000 max). I’d be glad to send to you in a different manner.

Hi,

I’d be glad to send to you in a different manner.

You can try to open a new issue via github.

/Micha

--
applied-geodesy.org - OpenSource Least-Squares Adjustment Software for Geodetic Sciences

2 new issues created with DB structure export included fore each upgrade.

Hello

FYI, the DB structure paste is too large to post in a response here (~14,200 char paste, 10,000 max). I’d be glad to send to you in a different manner.

How do you like the idea to group the tables in different postings? This would make it possible to discuss and work on changes in specific tables/table groups in different branches of the thread.

… or that :-D

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Hello Auge and Micha,
Hello everyone,

thanks for the effort you put into development of MLF! :)

Like probably most other users, I only read here occasionally. I therefore apologize for only seeing the notice now.

I'm running 2.4.20 with about 600.000 posts.
I tried update/update_2.4.19-2.5.php. I downloaded it today from github.

Result:
I experienced the same issues as Joe I. So the update process is at least consistent over different systems. ;)

The update ran for 5 min, which may be relevant for admins who set a time out for running php scripts.

After 5 min, the following error occurs:
Database error in line 298: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ' KEY `b8_spam` (`spam`), KEY `B8_training_type` (`training_type`)) ENGINE=Inn...' at line 1

I already tried the new versions a few months ago and I came to the same conclusion as Joe I:

2) In your original comments, you note that this upgrade doesn't include any much discussed database performance fixes. Our site DB is quite large (~500,000 posts), and runs very slowly on any 2.5 version of MLF due to the performance of some read queries which JOIN on the new Spam tables.

My solution a few months ago was to manually remove all joins with spam tables, since I don't need the spam detection functions. But it was still slow and it's surely not the best solution to fork every release.

That's the reason why I'm still running on 2.4.20. ;)

If I can help in any way, please let me know!

Best regards

Florian

Hello

Like probably most other users, I only read here occasionally. I therefore apologize for only seeing the notice now.

You don't have to apologise. I was really disappointed by those who had just asked us for a solution to the new EU Digital Services Act legislation and who I assumed would help us with testing in the interest of a quick new release. I didn't made that clear enough in my "ranting" posting.

I'm running 2.4.20 with about 600.000 posts.
I tried update/update_2.4.19-2.5.php. I downloaded it today from github.

Result:
I experienced the same issues as Joe I. So the update process is at least consistent over different systems. ;)

Running the upgrade script from any version within the range from 2.4.19 to 2.4.24 runs the same section of the script. Consequently, the same errors occur. That's consistency. :-)

The update ran for 5 min, which may be relevant for admins who set a time out for running php scripts.

Wow, five minutes! In my testing instance, a copy of my forum for testing my theme with only nearly 100 postings, the upgrade ran within around a second. That the running time will be longer with more than a half million entries sounds reasonable, but five minutes!

@Joe: What's your experience with running time of the script?

Do we need mentions, perhaps? 🤔

After 5 min, the following error occurs:
Database error in line 298: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ' KEY `b8_spam` (`spam`), KEY `B8_training_type` (`training_type`)) ENGINE=Inn...' at line 1

That's the same error like Joe reported. Also he, like you, startet with a stable version of the 2.4-branch. I have a pull request in the pipeline, that solves also this bug.

2) In your original comments, you note that this upgrade doesn't include any much discussed database performance fixes. Our site DB is quite large (~500,000 posts), and runs very slowly on any 2.5 version of MLF due to the performance of some read queries which JOIN on the new Spam tables.

My solution a few months ago was to manually remove all joins with spam tables, since I don't need the spam detection functions. But it was still slow and it's surely not the best solution to fork every release.

It's an option to customise the queries based on the settings to prevent unnecessary joins. In your case it seems to be the solution because there is really no need to join unused tables. There are further cases of table joins, that are always executed, but which, strictly speaking, are settings-dependent.

But on the other hand alls this solves nothing for installations, where the spam prevention mechanisms or specific other settings are in use. And it makes the decision making tree in the code much more complex.

If I can help in any way, please let me know!

I think we really need thoughtful input from as many sides as possible. And we need more testing of changes in the database structue. Not only in this case but also in the future.

Thank you for your input.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

The update ran for 5 min, which may be relevant for admins who set a time out for running php scripts.

Wow, five minutes! In my testing instance, a copy of my forum for testing my theme with only nearly 100 postings, the upgrade ran within around a second. That the running time will be longer with more than a half million entries sounds reasonable, but five minutes!

@Joe: What's your experience with running time of the script?

Do we need mentions, perhaps? 🤔

Unfortunately, I did not time the upgrade process when I did either a single or double upgrade. My biggest takeaway was that it didn't crash due to a web or SQL timeout, so to me that was fine.

No harm to mention that it could take an extended time for bigger forums, but I'm not sure even 5 minutes is a huge issue at the moment.

Joe

Hello Joe

Thank you for sharing your experience.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

RSS Feed of thread

my little forum

A request to the audience and especially to forum owners (Development)

Tja, there is obviously no interest in co-operation 😒

Auge, I have time to go through the upgrade steps.

Auge, I have time to go through the upgrade steps.

2.4.24 --> 20240308.1 Upgrade Notes

2.4.24 --> 20240308.1 Upgrade Notes

2.4.24 --> 20220803.1 --> 20240308.1 Upgrade Notes

2.4.24 --> 20220803.1 --> 20240308.1 Upgrade Notes

2.4.24 --> 20220803.1 --> 20240308.1 Upgrade Notes

SQL Performance Tweaks

SQL Performance Tweaks

Fork / Pull Request Created

Fork / Pull Request Created

Quick Notes

Quick Update

Quick Update

Quick Update

DB Structure Note

DB Structure Note

Done.

DB Structure Note

2.4.20 --> 2.5 Upgrade Notes

2.4.20 --> 2.5 Upgrade Notes (with a question to Joe)

a question to Joe

a question to Joe