Change Request/Notification: FAS server problems
I just noticed that fas was having issues and found that fas was using
over a GB of memory on fas2.fedora.phx.redhat.com. No problem, I
restarted fas and expected everything to be fine. 14 minutes later,
memory was over a GB again. Another restart took care of the problem
but I looked at the logs to see what's going on.
Apparently FAS is busy enough now that it's running into the database
connection limit we impose via SQLAlchemy and then requests are backing
up, causing the memory explosion. Since FAS is far and away our busiest
TG app I'm bumping the limits up so that we don't keep losing FAs service:
sqlalchemy.connection_pool 5 => 10
sqlalchemy.max_overflow 21 => 25
The connection_pool is the number of db connections each fas server will
hold open. Since it's busy, it makes sense to hold open more
connections. connection_pool + max_overflow = the total number of
connections that can be open when there's a lot of requests.
I've made these changes and pushed them live since it's causing FAS to
timeout and throw errors (which affects other services which auth
through fas as well.)
-Toshio