Backup to big? (Technics)

by Nico Hoffmann @, Wednesday, August 14, 2019, 19:55 (1678 days ago)

Hello,

I make sometimes backups from my forum. For several years, all was fine.
Now, it seems that the entries backup is grown too big. I get following error message:

Fatal error: Out of memory (allocated 262656000) (tried to allocate 393216 bytes) in /.../forum/includes/classes/Backup.class.php on line 36

Some weeks ago I had the same problem. I contacted my provider, and I was told to write "memory_limit=512M" to ".user.ini". I did that, and it worked. The backup of the entries was +160MB in size. That's a usual size for my case, not really more than some weeks ago.

I also created a phpinfo.php and I could see a change of memory_limit from 256M to 512M. Everything seemed fine.


Now, I got this error again. memory_limit is still 512M. I tried 768M (and saw 768M in phpinfo), but i get still this error.

The entries increase very slowly, I would not more than 170M expect for a backup of all entries.

I run MLF 2.4.20. Any hints?

Nico Hoffmann

Avatar

Backup to big.

by Auge ⌂, Thursday, August 15, 2019, 07:02 (1678 days ago) @ Nico Hoffmann

Hello

I make sometimes backups from my forum. For several years, all was fine.
Now, it seems that the entries backup is grown too big. I get following error message:

Fatal error: Out of memory (allocated 262656000) (tried to allocate 393216 bytes) in /.../forum/includes/classes/Backup.class.php on line 36

As first, this is an individual server setting, that's not under the control of the forum script. Furthermore it has a different value on different servers.

Some weeks ago I had the same problem. I contacted my provider, and I was told to write "memory_limit=512M" to ".user.ini". I did that, and it worked. The backup of the entries was +160MB in size.

Please be aware, that the memory limit does not name the size of the resulting backup but the memory that is available for the PHP-script itself.

That includes the interpretation of the script code and the processed data. Even if the resulting backup has a size of "only" 160 or 170 Megabyte, the interim steps in the procession of these data beginning with reading it from the database over generating the strings that should be saved as the backup to the writing of the backup itself to the file system can allocate much more than the outcome of the script.

If you look into a backup file you will see the real data and also the part that is the SQL-query. Let's assume you have a backup file with a size of 100 Megabytes. The file contains the orders to restore the data and the data itself in a (fictive) ratio of 1/9 (10% (10MB) query strings, 90% (90MB) real data).

The backup script reads in a first step 90MB from the database. In the second step the script assembless a copy of these 90MB with additional 10MB for the queries. At that time we have 90MB a result of the database request and 90MB+10MB as the result of the string composition in the memory. Summed up it results in 190MB in our example at that point and we did not took the script interpretation and the following file writing into consideration until now.

Beside the fact that we forgot to add new tables to the backup process over the time for several times, we (Micha and me) think the backup script is more or less rudimentary and old in the meaning of it's concept. At the end we recommend to use tools like phpMyAdmin for the backup. These tools are (normally) much more advanced, robust and take cornercases into consideration which we don't know. Additionally I expect it to have automatisms to circumvent any memory limits (in case of phpMyAdmin the limits of PHP).

Tschö, Auge

--
Trenne niemals Müll, denn er hat nur eine Silbe!

Backup to big.

by Nico hoffmann @, Saturday, August 17, 2019, 10:02 (1676 days ago) @ Auge

Beside the fact that we forgot to add new tables to the backup process over the time for several times, we (Micha and me) think the backup script is more or less rudimentary and old in the meaning of it's concept. At the end we recommend to use tools like phpMyAdmin for the backup.

That is the most important part of your message. I have to do backups with third-party tools.

Thank you also for your technical explanations. Let me just for the sake of completness, make the point I was wondering about more clearly:

I made backups over the years. Last year, entries log was 160MB (say) and very slowly increasing. This year, it was 164MB. Then I had troubles, and doubled the memory_limit. It worked one or two times with a backup size of 164,5MB or 165MB, then it stopped again working. Again i increased the memory_limit, now to three times of the original value, but it seems still not sufficient...
It looks like that a rather small increase of the database size does outperforme a memory_limit of the factor three.

But we can stop the discussion here, since I have to change my tooling anyway.

Nico Hoffmann

RSS Feed of thread