Avatar

Announcement: a new version will be released in the next days (Development)

by Auge ⌂, (153 days ago)

Hello

After a very long time, creating a completely overhauled upgrade script, fixing various bugs and making serious improvements to database performance in several places, a new version is due to be released in the next few days.

A very old design decision, that can cause database errors nowadays, still needs to be fixed. The new version will be released afterwards.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Announcement: a new version will be released in the next days

by Esbo, (153 days ago) @ Auge

We are not longer use it because of the slow Updates. Bye.

Avatar

Announcement: a new version will be released in the next days

by Auge ⌂, (153 days ago) @ Esbo

Hello

We are not longer use it because of the slow Updates. Bye.

I apologise for saying this so succinctly, but that's how it is. 🤷

If you, as someone who has never ever appeared here before under your current username (another sock puppet?), look back at the history of this project, it has never been any different. It was seldom, that the time between two releases was short. The only exceptions were urgent bug fixes or really simple enhancements. The reality is, without support from the already small community the speed of updates is and remains slow.

Bye

--
Trenne niemals Müll, denn er hat nur eine Silbe!

who needs today a message board??

by Berry(E), (153 days ago) @ Auge

Hey, sorry but who needs today a message board?? :confused:

Avatar

who needs today a message board??

by Micha ⌂, (153 days ago) @ Berry(E)

Hi,

Hey, sorry but who needs today a message board?? :confused:

No one, that's why we develop a forum instead of a board.

--
applied-geodesy.org - OpenSource Least-Squares Adjustment Software for Geodetic Sciences

who needs today a message board??

by guest9, (152 days ago) @ Micha

you can call it whatever you want but my little forum is a message board and not a forum

who needs today a message board??

by Mulischaf, (151 days ago) @ guest9

you can call it whatever you want but my little forum is a message board and not a forum

In the end it doesn't matter. Users can write entries/messages and can reply to each other in a kind of answer-stream. Forum likes phpBB, Xenforo, jkBB, multiple other discussion boards, blogs with comment functions, CMS standard websites with guestbook replies, Wiki engines with discussion pages, old school mailing lists.... the concept is more or less always the same. And so MLF. People write, people react, people write again.

And yes, I need and use it. For tiny, small projects with small group of people in closed space/area, but for example also for a small, new online project. The amount of users and feedback is 0 to low and will be low always in the future, but that's the same with blog comments and other social things today (aside of "likes" or troll postings). But they all still exist and live here and there, for the same reason, why a lot of people still play DOS games, build and work with retro or analog computers or programming the next window manager for linux or a cat feeding station, controlled by hand gestures. And somebody will read the posting(s) one day, because he/she is searching for something specific online. Or an AI is using it as source for better answers in the future.

It's just a matter of interest, fun, hobby, personal project and the right combination of people. Sometimes slow and a small amount of activity, but still alive and breathing for good reasons. Doesn't need always the big scope/need/impact - so my personal opinion.

who needs today a message board??

by Willi, (125 days ago) @ Berry(E)

We don't need it, but we wanna use it

This is great news, and much appreciated.

by Joe I, (146 days ago) @ Auge

I know you and others have worked hard over a long period to roll out a worthy upgrade. If you’re at a point where you want / need additional upgrade testing, please let me know.

Joe

Avatar

This is great news, and much appreciated.

by Auge ⌂, (146 days ago) @ Joe I

Hello

I know you and others have worked hard over a long period to roll out a worthy upgrade.

Not so modest. You have made a significant contribution to the changes in the new version.

If you’re at a point where you want / need additional upgrade testing, please let me know.

In fact, there was a problem that I found by upgrading an empty forum, freshly installed with version 2.4.21, to the current version. I cannot currently judge whether the error also occurs if the forum was installed with the current code version.

When I have completed the upgrade and returned to the admin overview, everything was displayed normally. Even calling up the (except for my own account) empty user list, the empty overviews of my own postings and bookmarks worked. Only when I called up the main forum page with the overview of threads, which was also empty at this time, the page displayed a database error. It concerned queries with ORDER-BY clauses in which columns are to be sorted that are not contained in the SELECT.

I don't know, why this happened (beside the reported syntax error). I suspect an overoptimisation in your performance improvements. However, I have absolutely no proof of this. It's just an assumption because the error did not occure during my tests in the process of the overhaul of the upgrade script I made before applying your optimisations to the repository.

I solved the issue by adding the columns we want to order by to the list of columns to select in the pull request #725. You can view the changes made in the list of changes. It worked with no and also with a few entries in two threads.

Can you please take a look on my work? In my tests, described above, it worked but until now I didn't perform any test with a fresh installation of the current code or with an upgraded forum with previously existing content (users, entries and so on).

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Let me check this out.

by Joe I, (145 days ago) @ Auge

- No text -

Avatar

Did you check this out?

by Auge ⌂, (144 days ago) @ Joe I

Hello Joe

Did you meanwhile took a look? I want to release the new version and it would be nice to be informed about a possible trap.

I tested myself to upgrade a forum from version 2.4.20 to the current code (I added a few further changes to the upgrade script yesterday). The forum comes with around 70 entries in 29 threads and with 15 entries in the userdata table, nothing big but also nota n empty forum. So the tests are not comparable with my first tests with an empty forum. The upgrade ran without any obvious error, the database structure (and content) looks ok to me and all pages, I tested, loaded without an error message or an obvious misbehaviour.

That is why I would like to publish the new version. A second, independent report of an error-free upgrade would give me additional certainty.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Code review comments.

by Joe I, (141 days ago) @ Auge
edited by Joe I,

Hi Auge,

Some comments here, and still need to dig into the testing.

Line 40/41 Update
Looks very good. ORDER BY Should generally work either with or without the “ft.”, however including "ft.time" Should eliminate any possible issue here, as you have done. See Line 109 for more info.

Line 88 Update
I Think this looks good, but need to test numerous combinations. The original format should not have caused any issue according to SQL standards, however there have been some SQL / MySQL implementations that required any ORDER BY also be in the SELECT. I need to make sure adding ft.sticky and ft.time in the SELECT DISTINCT won't change the # of output rows.

Line 109 Update
Looks good. The issue may stem from unsafe use of the variable name “time” in the SQL SELECT (Line 109), as this is a Reserved word.

Line 113 Update
Looks fine. This could have also been left as “ft.time” but no harm done.

Line 249 Update
Thanks. for cleaning that up.

Some questions as well:

Q1: To avoid potential row result issues arising from the change in Line 88, I'd like to know if you did already try changing Line 40/41 without changing Line 88.

From your upgrade notes:
"Only when I called up the main forum page with the overview of threads, which was also empty at this time, the page displayed a database error. It concerned queries with ORDER-BY clauses in which columns are to be sorted that are not contained in the SELECT."
Q2: Do you know if this error occurred on both Line 88 and 109?
Q3: Can you confirm whether you're using MySQL or MariaDB? Trying to wrap my head around these, as I can't duplicate the issues on my existing system (MariaDB, with data). Still need to go through upgrade using an empty DB, but curious if there may be an additional complicating factor between MariaDB and MySQL.


Testing factors I need to look at (particularly for Line 88 and system not crashing)
Categories (categories disabled, categories enabled but no categories, categories enabled and exist)
Spam (no spam entries, spam entries with view enabled & disabled)
MariaDB vs MySQL. I only have MariaDB 11, so won't be able to test out MySQL 8.

Avatar

Code review comments.

by Auge ⌂, (140 days ago) @ Joe I

Hello Joe,

Line 40/41 Update
Looks very good. ORDER BY Should generally work either with or without the “ft.”, however including "ft.time" Should eliminate any possible issue here, as you have done. See Line 109 for more info.

This was more to clarify, which column in which table is meant. In the query itself the relation is clear but this part of the query is generated to far away from the rest of the query to be self-explanatory.

Line 88 Update
I Think this looks good, but need to test numerous combinations. The original format should not have caused any issue according to SQL standards, however there have been some SQL / MySQL implementations that required any ORDER BY also be in the SELECT.

I would bet for a configuration issue. Be that as it may, on my webspace/database server the missing columns in the SELECT were reported as an error and adding it was the solution.

I need to make sure adding ft.sticky and ft.time in the SELECT DISTINCT won't change the # of output rows.

No, the change does not change the number of the lines in result. It doesn't because the thread-ID (`mlf2_entries.tid`), selected with DISTINCT in combination with WHERE ft.pid = 0 which ensure to select only the opening posting of a thread, is unique in every case and the additional selected columns doesn't change this in any case.

Line 109 Update
Looks good. The issue may stem from unsafe use of the variable name “time” in the SQL SELECT (Line 109), as this is a Reserved word.

I suspect that the conversion of the column time from the MySQL timestamp type to the Unix integer timestamp under the same name as the column itself makes sorting impossible. When adding the aliased column to the select and ordering by the alias (and so with the original values) the code works.

Some questions as well:

Q1: To avoid potential row result issues arising from the change in Line 88, I'd like to know if you did already try changing Line 40/41 without changing Line 88.

No, I didn't. I hope, my description above about DISTINCT in combination with the WHERE clause clarifies what happens.

From your upgrade notes:
"Only when I called up the main forum page with the overview of threads, which was also empty at this time, the page displayed a database error. It concerned queries with ORDER-BY clauses in which columns are to be sorted that are not contained in the SELECT."

Q2: Do you know if this error occurred on both Line 88 and 109?

Yes, it does. The first error message was for line #88. After solving the error, another error message came up, related to the query in line #109.

Q3: Can you confirm whether you're using MySQL or MariaDB? Trying to wrap my head around these, as I can't duplicate the issues on my existing system (MariaDB, with data). Still need to go through upgrade using an empty DB, but curious if there may be an additional complicating factor between MariaDB and MySQL.

I was able to reproduce the error on both systems, MySQL 8.x and MariaDB 10.5.x on the servers of one hosting company (with an obvioulsy strict configuration).

Testing factors I need to look at (particularly for Line 88 and system not crashing)
Categories (categories disabled, categories enabled but no categories, categories enabled and exist)
Spam (no spam entries, spam entries with view enabled & disabled)
MariaDB vs MySQL. I only have MariaDB 11, so won't be able to test out MySQL 8.

As mentioned in another posting I successfully upgraded a forum with several postings in several threads in different categories and with several users. The only scenario I did not test was to have spam postings in the data. But also these postings would only be postings. There was no change regarding their handling in an upgrade.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Avatar

Another case of ordering by a column, that is not in the SELECT

by Auge ⌂, (138 days ago) @ Joe I

Hello Joe

Line 88 Update
I Think this looks good, but need to test numerous combinations. The original format should not have caused any issue according to SQL standards, however there have been some SQL / MySQL implementations that required any ORDER BY also be in the SELECT. I need to make sure adding ft.sticky and ft.time in the SELECT DISTINCT won't change the # of output rows.

I found another case of that type. When reordering the list of threads on the main page by the date of the latest reply (in my testing installation the threads are ordered by the date of the opening posting by default), one gets an error message, I documented on Github in the issue #731.

The message states clearly "this is incompatible with DISTINCT".

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

I added a couple comments in Github.

by Joe I, (137 days ago) @ Auge

Github issue #731

[edit]: Added the link to the Github issue. Auge

Upgrade Test

by Joe I, (141 days ago) @ Auge

Upgrade from 2.4.24 --> 20240729, no entries in database, running on MariaDB 11.

Clean install worked fine, and no errors on any forum or admin pages, including changing display options.

Still to check out:
Results of query changes to make sure the results match across query options.

SQL Change Results

by Joe I, (141 days ago) @ Auge

After running the modified queries using different Spam entries in the DB (no Spam. Spam on child records, Spam on parent records, different Spam values in Akismet and B8), the query results match between the old and new code. And looking at the actual SQL, this makes perfect sense. So, I'd say the changes all look good.

Avatar

SQL Change Results, thank you for your tests

by Auge ⌂, (140 days ago) @ Joe I

Hello

After running the modified queries using different Spam entries in the DB (no Spam. Spam on child records, Spam on parent records, different Spam values in Akismet and B8), the query results match between the old and new code. And looking at the actual SQL, this makes perfect sense. So, I'd say the changes all look good.

Thank you very much for your effort. Your tests confirmed mine and makes it much easier for me to publish the update. In the past there have been errors from time to time that have been overlooked, so I really wanted to hear/read a second opinion from a competent person.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Upgrade Failed — 2.4.24 to 20240729.1

by Vinayy, (134 days ago) @ Auge
edited by Vinayy,

I tried to upgrade from 2.4.24 to 20240729.1 in Wampserver but it is not getting upgraded and I am repeatedly getting errors.

Using version: 2.4.24
phpMyAdmin: 5.2.1
MySQL: 8.3.0
PHP ver: 7.3.33
Database: utf8mb4_unicode_ci
Database: over 240 MB

I have uploaded the screenshot, which is all on one page, they are not separate images. Please check and advise.

[image]

Avatar

Upgrade Failed — 2.4.24 to 20240729.1

by Auge ⌂, (134 days ago) @ Vinayy

Hello

I tried to upgrade from 2.4.24 to 20240729.1 in Wampserver but it is not getting upgraded and I am repeatedly getting errors.

Using version: 2.4.24
phpMyAdmin: 5.2.1
MySQL: 8.3.0
PHP ver: 7.3.33
Database: utf8mb4_unicode_ci
Database: over 240 MB

Are you sure about the versions? Especially the version 8.3.0 of MySQL looks a bit odd (to me) in combination with the (fairly old) PHP version 7.3.33.

I have uploaded the screenshot, which is all on one page, they are not separate images. Please check and advise.

[image]

The first four messages are about the names of the tables, which was introduced with version 20220508.1 (2.5.0). The query, starting in version 20240729.1 in line #162, reads all table names and engines from the system data if the table engine is not InnoDB. In my tests not existing table names was not relevant but reading your report, I became aware of the cause. The error occures not in the query but in the PHP code that generates the query. At this time of code execution the namely table names are not known if one wants to upgrade from a 2.4.x-version. The table names are defined after this point in the lines #251 to #254.

I performed several tests with upgrading from versions 2.4.19, 2.4.20 and 2.4.24 and never ran into this specific error. Obviously the PHP-code generates empty strings instead throwing an error on my webserver. It seems, that PHP on your webserver is configured more strict with errors end their reporting. Anyway, I will code a fix for it by collecting the really existing table names with a join of $db_settings elements whose names ends with "_table" and inserting the result into the query.

The fifth error report in your screenshot was thrown in line #468, which is about changing the charset of the table mlf2_entries_cache. It is about exceeding the maximal execution time of three seconds for a script. The permitted three seconds of execution time may already have been used up in previous steps, except that they were finally exceeded in line #468. It's possible, that the fix for the first four errors will shorten the execution time enough to keep it under three seconds, it's also possible, it don't. We will see.

For comparision: On my webspace I run a subdomain specifically for testing purposes, where PHP runs in version 7.4. with a max_execution_time of 30 seconds (ten times your value) with a memory limit of 256MB. The values are the defaults, provided by my hosting company and were not set by me.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Upgrade Failed — 2.4.24 to 20240729.1

by Vinayy, (134 days ago) @ Auge
edited by Vinayy,

Before upgrading from 2.4.24 to 20240729.1, I changed the engine of all the tables to InnoDB (some were already InnoDB). I used PHP 7.3 for the upgrade.

After the upgrade, this error is appearing above the forum content but only up to PHP 7.4.

[image]

PHP versions greater 7.4 are only showing errors and no forum content. I need it to work on minimum PHP 8.1, which is what my host is providing.

Before I replaced & updated all the files and folders, immediately after the upgrade, the page was showing 6 errors, which reduced to just 2 (image shared above). I had taken the screenshot and here it is (in 3 parts).

[image]
[image]
[image]

Avatar

Upgrade Failed — 2.4.24 to 20240729.1

by Auge ⌂, (133 days ago) @ Vinayy

Before upgrading from 2.4.24 to 20240729.1, I changed the engine of all the tables to InnoDB (some were already InnoDB). I used PHP 7.3 for the upgrade.

After the upgrade, this error is appearing above the forum content but only up to PHP 7.4.

You talk about the first of your images with the error in lines #66 and #67 of includes/index.inc.php?

[image]

PHP versions greater 7.4 are only showing errors and no forum content. I need it to work on minimum PHP 8.1, which is what my host is providing.

Can you please check in phpMyAdmin, if the tables mlf2_akismet_rating, mlf2_b8_rating, mlf2_b8_wordlist and mlf2_uploads exist?

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Upgrade Failed — 2.4.24 to 20240729.1

by Vinayy, (133 days ago) @ Auge

Can you please check in phpMyAdmin, if the tables mlf2_akismet_rating, mlf2_b8_rating, mlf2_b8_wordlist and mlf2_uploads exist?


Yes, all the 4 tables mlf2_akismet_rating, mlf2_b8_rating, mlf2_b8_wordlist and mlf2_uploads exist in phpMyAdmin, in the upgraded version.

Avatar

Upgrade Failed — 2.4.24 to 20240729.1

by Auge ⌂, (133 days ago) @ Vinayy

Hello

Can you please check in phpMyAdmin, if the tables mlf2_akismet_rating, mlf2_b8_rating, mlf2_b8_wordlist and mlf2_uploads exist?


Yes, all the 4 tables mlf2_akismet_rating, mlf2_b8_rating, mlf2_b8_wordlist and mlf2_uploads exist in phpMyAdmin, in the upgraded version.

Ok, next question. Is your forum organised with categories? When I look into the code, the function mysqli_fetch_row, that expects its parameter to contain a query result, gets a bool (false) instead. That means, that the function mysqli_query (one line above) failed, what means, that the query itself, that is composed from different parts, is broken. One of the parts is the string of one category or a list of categories for the WHERE-clause of the query.

I myself currently performs several UI-tests in a forum, that I upgraded from 2.4.20 to 20240729.1 (with a few additional fixes). And this forum works like a charm. Now I search for differences and the first thing, that comes into my mind is "categories or no categories"?

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Upgrade Failed — 2.4.24 to 20240729.1

by Vinayy, (133 days ago) @ Auge

Is your forum organised with categories?

Yes, the forum has several categories, where some of the category names are using text other than English.

Avatar

Upgrade Failed — 2.4.24 to 20240729.1

by Auge ⌂, (133 days ago) @ Vinayy

Is your forum organised with categories?


Yes, the forum has several categories, where some of the category names are using text other than English.

IMHO the languages shouldn't make a difference. In my forum (and also here) some of the categories are named in english some other in german or different languages (but all in languages written in latin letters some with diacritics).

Please add the following codelines to includes/index.inc.php between the codelines 64 and 65 (directly before the line with mysqli_query($connid, $display_pid_result)).

echo "<pre>". print_r($connid, true) ."</pre>";
echo "<pre>". print_r($display_pid_result, true) ."</pre>";

This should display the query that seems to be broken. It will cause error messages about alredy sent headers and maybe more but it becomes visible, if the query is really broken or not.

Please copy the SQL-query and show it here in a reply. If there's some (unbroken) value, that should stay private, remove it before posting the query but best would be to show the complete query.

Thank you in advance.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Upgrade Failed — 2.4.24 to 20220803.1

by Vinayy, (134 days ago) @ Auge

I even tried to upgrade from 2.4.24 to 20220803.1 in Wampserver but it is not getting upgraded and I am repeatedly getting the error:

Error!

  • Database error in line 381: Could not change column 'user_email' of table 'mlf2_userdata'. The resulting size of index 'user_email' would exceed the max key length of 1000 bytes.

Using version: 2.4.24
phpMyAdmin: 5.2.1
MySQL: 8.3.0
PHP ver: 7.3.33
Database Collation: utf8mb4_unicode_ci and also tried utf8mb4_general_ci
Database: over 240 MB

Avatar

Upgrade Failed — 2.4.24 to 20220803.1

by Auge ⌂, (134 days ago) @ Vinayy

I even tried to upgrade from 2.4.24 to 20220803.1 in Wampserver but it is not getting upgraded and I am repeatedly getting the error:

Error!

  • Database error in line 381: Could not change column 'user_email' of table 'mlf2_userdata'. The resulting size of index 'user_email' would exceed the max key length of 1000 bytes.

Using version: 2.4.24
phpMyAdmin: 5.2.1
MySQL: 8.3.0
PHP ver: 7.3.33
Database Collation: utf8mb4_unicode_ci and also tried utf8mb4_general_ci
Database: over 240 MB

Also the configuration of your database server seems very strict to me. I know, that very old MySQL-versions prior to 5.7.7, that allowed the use of utf8mb4…, was only able to provide index sizes, that was to small to index char and varchar columns larger than 192 bytes. That led to errors like yours because the size of the column mlf2_userdata.user_email is 255 Bytes. The index size limit was removed with MySQL-version 5.7.7 (should be around 3.5kB now). Because of this the upgrade script of version 20240729.1 checks for the version of MySQL or MariaDB to be at least 5.7.7 respectively 10.2.2 for MariaDB. The upgrade script for MLF version 20220803.1 lacks this check but in your case it is generally irrelevant because you MySQL version of 8.3 would pass this test.

Again, are you sure about the version of your database server? Is this possibly the result of an upgrade from a WAMP version with a MySQL-server of an very old version (prior to 5.7.7)? I really don't know, what to do about such a special case like yours.

Very odd.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Upgrade Failed — 2.4.24 to 20220803.1

by Vinayy, (134 days ago) @ Auge

The upgrade from 2.4.24 to 20220803.1 was successful after I changed the "engine" of mlf2_userdata from MyISAM to InnoDB. Although I haven't run it afterwords in a live environment, as this was only in my Wampserver.

I will try to change the engine of all other tables in a similar manner and try the upgrade to the latest version.

Avatar

Upgrade Failed — 2.4.24 to 20220803.1

by Auge ⌂, (133 days ago) @ Vinayy

Hello

The upgrade from 2.4.24 to 20220803.1 was successful after I changed the "engine" of mlf2_userdata from MyISAM to InnoDB. Although I haven't run it afterwords in a live environment, as this was only in my Wampserver.

This error was foreseeable. The task of the query in which the four errors with the non-existent indexes occurred was to determine the tables for which the engine must be switched to InnoDB. If the errors had not occurred, the script could have changed the engine.

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Upgrade Failed — 20220803.1 to 20240729.1

by Willi, (125 days ago) @ Auge

Hi,

I tried to install this version on the page Katzenquatsch. But immediately after entering my Admin-Passwirt I got this error:

[image]

I think I followed all instructions. Where to look what to do?

I another post here you (Auge) asked if 4 tables doe exist. They do. And the tables are InnoDB. Probably a hint is that I changed the prefix of mlf2 to some other but somewhere in the config.

Thx and best regards,
Willi

Avatar

Upgrade Failed — 20220803.1 to 20240729.1

by Micha ⌂, (125 days ago) @ Willi

Hi Willi,

there is a copy & past error in the update script, solved within #736. Please use the modified update script from git.

/Micha

--
applied-geodesy.org - OpenSource Least-Squares Adjustment Software for Geodetic Sciences

Upgrade Failed — 20220803.1 to 20240729.1

by Willi, (125 days ago) @ Micha

Thx, that did the trick

RSS Feed of thread