On my server, the server where is profile it hosted, the profile picture does not load anymore.
It shows a fault:
Internal Server Error
Invalid photo with id 888.
#0 [internal function]: Friendica\Module\Photo::rawContent()
#1 /srv/http/src/App/Module.php(289): call_user_func()
#2 /srv/http/src/App.php(557): Friendica\App\Module->run()
#3 /srv/http/index.php(40): Friendica\App->runFrontend()
#4 {main}Link to profile photo should be this, according to html source:
https://friendica.utzer.de/photo/52959870015ac78226075e6200304821-4.jpg?ts=1523017271?rev=2018-04-06+14%3A21%3A11
Any idea what is wrong?
like this
Michael Vogel
•Michael Vogel
•select `backend-class`, `backend-ref` from photo where id = 888;Roland Häder likes this.
utzer [Friendica]
•Michael Vogel
•Please check if the "data" field contains a value there. If not then you have got a problem. The
backend-classshould only be empty when you still store your pictures in the database.Please check if you have got a field in the
storagetable with the id "670476". Also check if in the storage folder there is some file that contains this id. I don't know the exact format of the stored pictures, so I can't give you a hint for the exact filename.utzer [Friendica]
•Michael Vogel
•select id from storage where id=670476;utzer [Friendica]
•@Michael Vogel
Results of
select id from storage where id=670476;:Michael Vogel
•utzer [Friendica]
•@Michael Vogel
For
find ./ -type f -name "*670476*"I get many results. Should the name start with it?Michael Vogel
•utzer [Friendica]
•@Michael Vogel it will be a pleasure, meaning it is the least I can do.
By the way it is still moving some million of files it seems.
When I run
find ./ -type f -name "*"|grep -v "\.html"|wc -l, which counts lines of the output of find, which is equal to the number of files, I get 20.261.436 and wonder how it is possible that there is 20 million files.Michael Vogel
•update photo inner join storage on storage.id = photo.`backend-ref` set `backend-class` = 'Database' where photo.data = '' and `backend-class` = '' and `backend-ref` != '';utzer [Friendica]
•Michael Vogel
•utzer [Friendica]
•Michael Vogel
•Michael Vogel
•Michael Vogel
•update_active_contactsisn't set, then also every other known contact is checked every 6 month or 1 month if not failed.Michael Vogel
•Your case is really special and we yet haven't found the reason for this problem. So some manual updating won't fix the issue but would just disguise it.
Would be great if @Fabio could step in, since he wrote that class 😀
Fabio
•Michael Vogel
•Fabio
•backend-classset. Looking at the code, the only place where that is writtend in db is in\Friendica\Model\Photo::store(), and looks like it's correct. But maybe there is some corner case which isn't obvious...We should find a repeatable way to make the system write a wrong row in db to try to understand why it write a wrong row in db.
Fabio
•bin/console storage move?To which backend are you moving to?
Fabio
•mmh..
what
$ bin/console config storage namesays?can you try to change in
src/Core/StorageManager.phpin line 314:'backend-class' => $destinationto'backend-class' => $destination::getName()?utzer [Friendica]
•1. The move seems to be finished, it only moved a couple of thousand (4923) files and says it finished.
2. Mysql query of @Michael Vogel now finds 0 affected rows.
3. There is still over 20 million non .html files in the filesystem storage location it seems.
Fabio
•looks like a fix.
the files in filesystem storage dir should have been deleted while moving, but as long they have been copied in database, you could delete them, maybe....
Anyway, I'll do some tests and eventually send a pull request...
Which version of php are you using?
utzer [Friendica]
•@Fabio most of the files are 0B, over all size is only 90 MB, but that includes the html files again.
php 7.4.16
Fabio
•no, sorry, I've been drifted away by other things... I saw your commit, and from what we tested, looks like it's something needed for sure (and the code is more explicit, which imho is always a win).
I'm for a little pull request with that patch 👍
I saw in your screenshot the query
... show moreINSERT INTO storage (data) VALUES ('')which looks like is inserting empty data for photo. I don't know if the source file was empty, the log you get there is truncated because value of data was too big (but I don't think it's the case) or if the filesystemstorage backend failed inget()(which I don't think, otherwise it would have failed to fetch data for every images)... looking again at the code in
Friendica\Core\StorageManager::move()it could be that it was caused by the same bug: with the bug data was fetch from Filesystem, saved in Database and the backend was saved wrongly as "" (nothing). On the next run the same row has been processed again (as the backend saved is not the destination backend), but as now has empty 'backend-class' dano, sorry, I've been drifted away by other things... I saw your commit, and from what we tested, looks like it's something needed for sure (and the code is more explicit, which imho is always a win).
I'm for a little pull request with that patch 👍
I saw in your screenshot the query
INSERT INTO storage (data) VALUES ('')which looks like is inserting empty data for photo. I don't know if the source file was empty, the log you get there is truncated because value of data was too big (but I don't think it's the case) or if the filesystemstorage backend failed inget()(which I don't think, otherwise it would have failed to fetch data for every images)... looking again at the code in
Friendica\Core\StorageManager::move()it could be that it was caused by the same bug: with the bug data was fetch from Filesystem, saved in Database and the backend was saved wrongly as "" (nothing). On the next run the same row has been processed again (as the backend saved is not the destination backend), but as now has empty 'backend-class' data was loaded from the photo table, which was empty, and a new row is added in 'storage' with empty data.It could be that the photos you see missing are in reality in 'storage' table, but 'photo.backend-ref' is pointing to the row with empty data.
Problem is you probably never manage to find which "orphan" photo data in 'storage' goes with which row in 'photo' table, because there is no filename in 'storage' table, only data...
Fabio
•storagetable not referenced inphotoandattachtables, and we could get data and try to show it as images, but then someone should look at the images and find to which row inphotobelong, by hand.Because we have file metadata (filename, mimetype, ect) in
phototable with a link tostoragetable where there is the actual data, but we have no means to automatically get back fromstoragetophoto.utzer [Friendica]
•@Fabio hmm I think by hand is no an option, I should not look at users photos, maybe there is private photos there.
Also I am not sure how many files it would be, so it could be just a few or really alot.
Do you know why the move works in some case (social.yl.ms) and not in others (friendica.utzer.de) and I think @Michael Vogel also wrote he was moving storage (but maybe from database to filesystem.
Michael Vogel
•utzer [Friendica]
•Fabio
•I just tried on a new install, "rc" branch, without the patch on a ubuntu Bionic with php 7.4.7:
Upload a photo using "Legacy" backend;
Set Filesystem backend;
Move to Filesystem : no problems.
Set Database backend;
Move to Database: no problems.
While the patch imho is still good (I would add
::getName()also on line 308, same file, at the end of the log (otherwise the log showsSave data to new backend. {"newBackend":{}}instead ofSave data to new backend. {"newBackend":"Database"}which is what should say, imho ) looks like it should have worked anyway...utzer [Friendica]
•utzer [Friendica]
•Fabio
•utzer [Friendica] likes this.
utzer [Friendica]
•