Skip to main content
Upgraded from #pleroma BE to #rebased today, but haven't received any statuses for a few hours. Sending also didn't seem to work. For now I've restored a snapshot, but I am now again missing out on those nice features. Is there any documentation on how to troubleshoot Federation/why my messages are not working? @alex
#pleroma #rebased @Alex Gleason
Alex Gleason
Alex Gleason  
•
@corneIt really depends. First thing I’d check is logs with journalctl -fu pleroma and see what’s happening there.

Then I’d go into the database and check jobs with select count(*) from oban_jobs; Run the command several times. Are you in the hundreds, thousands, or tens of thousands? Is the number going up or down?

If you’ve been offline for a while, you could simply have backed up jobs, and you’d need to increase your resources to catch up. This can happen anytime you’ve been offline for a while.

If you have specific errors in your logs, we could address those.

Rebased doesn’t change much about Pleroma in regards to federation, so I suspect something is just wrong with the new configuration.
@Corné Dorrestijn
Alex Gleason
Alex Gleason  
•
@corneRegister on my server (or somewhere else) so you can ask me questions while yours is offline. I’ll help you figure it out.
@Corné Dorrestijn
corne
corne  
•
@corneAah, might have been that my postgres didn't accept enough connections (20). Bumped it to 40, but my cheapo EC2 instance doesn't like that. Might need to upgrade 😁
@Corné Dorrestijn
corne
corne  
•
@corneYeah, will be moving to another server tomorrow. These tiny instances can't handle a lot.
@Corné Dorrestijn
corne
corne  
•
@corneSo I've moved to another server, now I've increased the number of connections to 40, but I'm again getting the too many connections errors. 40 should be enough right?
@Corné Dorrestijn
corne
corne  
•
@corneThe oban_jobs table is slowly getting more items, so that seems to be working. However, they are not being processed.
@Corné Dorrestijn
Alex Gleason
Alex Gleason  
•
@corne@corneHas federation stopped? If the federation_incoming queue is backed up, you may need to increase the number of workers on that queue specifically.
@Corné Dorrestijn @corne
Alex Gleason
Alex Gleason  
•
@corne@cornePaste the actual errors from journalctl -u pleroma --since "1 minute ago"

You might need to increase the max_connections in Postgres itself

What does htop look like? Are you overloaded or still have room? Make sure you’re not using too much swap.
@Corné Dorrestijn @corne
Alex Gleason
Alex Gleason  
•
@corne@corneDid you have editing before you made this upgrade? If the answer is no, answer this question first.
@Corné Dorrestijn @corne
corne
corne  
•
@corneThe logs: https://gitlab.com/-/snippets/2481922. This machine is almost Idle. I could try increasing max_connections, but 20 was enough before the switch, so I can't imagine that more than 40 is needed.

> Did you have editing before you made this upgrade?

What editing?

pleroma last minute log ($2481922) · Snippets · Snippets · GitLab

GitLab.com
GitLab
@Corné Dorrestijn
Alex Gleason
Alex Gleason  
•
@corne@corneThe ability to edit posts
@Corné Dorrestijn @corne
Corné Dorrestijn
Corné Dorrestijn  
•
Well, increasing the max_connections (in postgresql) seems to be working. Might be useful to add to the documentation somewhere, since Pleroma recommends 20 connections.
Alex Gleason
Alex Gleason  
•
@corneThey’re talking about when using the pg_tune UI
@corne
Corné Dorrestijn
Corné Dorrestijn  
•
In the Linux OTP installation guide they link to that article.

> PgTune can be used to get recommended settings. Be sure to set "Number of Connections" to 20, otherwise it might produce settings hurtful to database performance.

That setting will result in max_connections being 20 in the generated configuration.

Thanks for helping out!
PGTune with settings prefilled. One if the settings is Number of Connections that results in max_connections=20
Alex Gleason
Alex Gleason  
•
@corneCrazy. @feld take a look. 👆
@feld @corne
Alex Gleason
Alex Gleason  
•
@corne@corneAre you editing pool_size in prod.secret.exs, or max_connections in postgresql.conf?

Because in postgres, you should have like 100-200 max_connections

In prod.secret.exs, 40 is not unreasonable. I have 30 on mine.
@Corné Dorrestijn @corne
⇧