Skip to main content

I deleted about 90 users, which gave me 180000 tasks in the queue. Didn't think about that before, but that will take a few hours now.

!Friendica Support #Friendica

#friendica !Friendica Support

reshared this

utzer [Friendica]
utzer [Friendica]  
•
The number of items in the queue is increasing and did now reach 207.000 items.
utzer [Friendica]
utzer [Friendica]  
•

from the log:

Load: 3.55/8 - processes: 1422/5/217062 - jpm: 78/43/56 (0:1, 20:2/36, 30:0/378, 40:1/157, 50:1/217913)

220.000

This entry was edited (3 years ago)
Michael Vogel
Michael Vogel  
•

If there are a lot of deffered posts, then you can decrease that with the configuration value "worker_defer_limit". I had set it to 6 on squeet.me.

Which kind of jobs are there the most?

utzer [Friendica]
utzer [Friendica]  
•

@Michael Vogel I think it is fine, it is just prio 50.

Load: 3.3/8 - processes: 1520/6/240430 - jpm: 45/49/59 (0:0, 20:3/33, 30:1/144, 40:1/323, 50:1/241450)

Right?

@Michael Vogel
This entry was edited (3 years ago)
utzer [Friendica]
utzer [Friendica]  
•

@Michael Vogel let me ask about this, is this "worker_defer_limit" about the first number in /admin (above 1284 in the picture)?

What I would like is to make the server more usable for the users on it, so the "every day" things should work faster, like delivery of posts and fetching of posts.

@Michael Vogel
Michael Vogel
Michael Vogel  
•
Well, your system should have no problems with the "every day" work, since the masses of tasks are having priority 50.
Michael Vogel
Michael Vogel  
•
Please execute: select command,count(*) from workerqueue where not done group by command;
utzer [Friendica]
utzer [Friendica]  
•
@Michael Vogel hmmm I think I see where this is going to.
+-------------------------+----------+
| command                 | count(*) |
+-------------------------+----------+
| APDelivery              |   294421 |
| CleanWorkerQueue        |        1 |
| ClearCache              |        1 |
| ContactDiscovery        |        9 |
| Delivery                |      749 |
| ForkHook                |        3 |
| Notifier                |       18 |
| PostUpdate              |        1 |
| PullDirectory           |        1 |
| RemoveUser              |       86 |
| SpoolPost               |        1 |
| UpdateContact           |      527 |
| UpdateContacts          |        1 |
| UpdateGServers          |        1 |
| UpdateServerDirectories |        1 |
+-------------------------+----------+
15 rows in set (0.756 sec)
@Michael Vogel
Michael Vogel
Michael Vogel  
•
Then execute this one:
select command,substr(parameter, 3, locate('"', parameter, 3)-3) as para, priority, count(*) from workerqueue where command = 'apdelivery' and not done group by para,priority;
utzer [Friendica]
utzer [Friendica]  
•
@Michael Vogel
select command,substr(parameter, 3, locate('"', parameter, 3)-3) as para, priority, count(*) from workerqueue where command = 'apdelivery' and not done group by para,priority;
+------------+----------+----------+----------+
| command    | para     | priority | count(*) |
+------------+----------+----------+----------+
| APDelivery | removeme |       50 |   319020 |
| APDelivery | wall-new |       20 |       10 |
| APDelivery | wall-new |       40 |        3 |
| APDelivery | wall-new |       50 |       38 |
+------------+----------+----------+----------+
4 rows in set (1.486 sec)
@Michael Vogel
utzer [Friendica]
utzer [Friendica]  
•

Now the JPM is increasing and the DB is using less CPU, the systems has a lower load.
Maybe the creation of new items has finished now?

Load: 2.32/8 - processes: 1739/9/318725 - jpm: 283/195/70 (0:1, 20:0/9, 30:0/2, 40:6/809, 50:2/319644)
Michael Vogel
Michael Vogel  
•
Okay. Some user removed their account. This will trigger this "removeme". But it shouldn't create that much jobs. I have to check. I will try to have a look at it in the evening.
utzer [Friendica]
utzer [Friendica]  
•
@Michael Vogel I think now the queue item number is decreasing and the system load is very low again, so the JPM is high as there is no load that reduces the JPM.
@Michael Vogel
utzer [Friendica]
utzer [Friendica]  
•
@Michael Vogel the JPM could be higher I think, maybe it is some bottleneck, but if so it is not memory, CPU or io.
2021-01-11T11:59:04Z worker [NOTICE]: Load: 2.97/8 - processes: 5134/7/296439 - jpm: 259/326/384 (0:1, 30:0/12, 50:6/301561) - maximum: 7/25 [] - {"file":"Worker.php","line":772,"function":"tooMuchWorkers","uid":"f1f78c","process_id":3588982}
@Michael Vogel
utzer [Friendica]
utzer [Friendica]  
•
@Ingo Jürgensmann @Michael Vogel yes, no prob, it is fine now, it will finish the queue after some time, the 300.000 remaining items are not a problem.
@Michael Vogel @Ingo Jürgensmann
utzer [Friendica]
utzer [Friendica]  
•
only down 55.000 items by now, so 266.000 to go.
⇧