Postgres problems again

[Attila Heidrich]

Postgres database stopped again... I guess the problem was the enormous
number of open files...

We use 9.2, and we also use the postgres plugin - which still doesn't
really support 9.2 as far as I know.

Altogether Postgres (only the one storing the RHQ database) can only run
for a few days, than the number of open files and the open connections
raise really high, and finally I should restart it.

Usually I restart RHQ as well, I generally have no time to play with "what
to restart".

The log is practically endless, I can quote, but I think I should know
something to look for in it.

Regards,

Attila

[Heiko W.Rupp]

Attila,

Am 23.09.2013 um 10:52 schrieb Attila Heidrich:
> Postgres database stopped again... I guess the problem was the enormous number of open files...
>
> We use 9.2, and we also use the postgres plugin - which still doesn't really support 9.2 as far as I know.

Can you please disable that plugin to see if this may cause the issue?
To what value did you set max connections in postgresql.conf

max_connections = 100

John found an issue recently where the baseline calculation did not close
connections, but this should not create insane amounts of connections,
as the appserver is supposed to limit them.

Sorry for the inconvenience.

If that happens again, can you do a ps auxw| and grep for processes like these:

postgres        6087   0,0  0,1  2487132   5312   ??  Ss   11:28am   0:00.08 postgres: rhqadmin rhqdev 127.0.0.1(65248) idle
postgres        6081   0,0  0,1  2487132   4308   ??  Ss   11:28am   0:00.01 postgres: rhqadmin rhqdev 127.0.0.1(65243) idle

That may help us diagnose what is going wrong.

Heiko

[Attila Heidrich]

I have "uninventoried" the postgres plugin for most of the servers. By the
way there's only a single server which suffers from this issue, and this is
the one what is used by RHQ itself! The others - which are only monitored
by the agent, are quite fine. The only problem is with the 9.2 version that
some system tables has been changed with 9.2, so there's not current_query
anymore.

This is strange, that in postgres log there's still lines like this:
2013-09-23 13:43:04 CEST ERROR:  column "current_query" does not exist at
character 70
2013-09-23 13:43:04 CEST STATEMENT:  SELECT (SELECT COUNT(*) FROM
pg_stat_activity where usename = $1 AND current_query != '<IDLE>') AS
active,
          (SELECT COUNT(*) FROM pg_stat_activity WHERE usename = $2) AS
total
2013-09-23 13:43:44 CEST FATAL:  password authentication failed for user
"postgres"
2013-09-23 13:43:46 CEST FATAL:  password authentication failed for user
"postgres"

I guess uninventory is not the perfect way to disable the plugin...

max_connection is the default 100 for all the servers.

Whenever it happen again, I will collect more Pg statistics!

Regards,

Attila

2013/9/23 Attila Heidrich
> Postgres database stopped again... I guess the problem was the enormous
> number of open files...
>
> We use 9.2, and we also use the postgres plugin - which still doesn't
> really support 9.2 as far as I know.
>
> Altogether Postgres (only the one storing the RHQ database) can only run
> for a few days, than the number of open files and the open connections
> raise really high, and finally I should restart it.
>
> Usually I restart RHQ as well, I generally have no time to play with "what
> to restart".
>
> The log is practically endless, I can quote, but I think I should know
> something to look for in it.
>
> Regards,
>
> Attila

[Attila Heidrich]

And just one more interesting thing:

After I correct the agents authentication, and it can reach the system
tables, and report all the stuff, I still have got lines:
2013-09-23 14:28:56 CEST FATAL:  password authentication failed for user
"postgres"

Until I set up the access parameters, I have got only such lines, but after
setting it up, I have got this:

2013-09-23 14:28:17 CEST ERROR:  column "current_query" does not exist at
character 70
2013-09-23 14:28:17 CEST STATEMENT:  SELECT (SELECT COUNT(*) FROM
pg_stat_activity where usename = $1 AND current_query != '<IDLE>') AS
active,
          (SELECT COUNT(*) FROM pg_stat_activity WHERE usename = $2) AS
total

and less frequently this:
2013-09-23 14:28:56 CEST FATAL:  password authentication failed for user
"postgres"

Attila

2013/9/23 Attila Heidrich
> I have "uninventoried" the postgres plugin for most of the servers. By the
> way there's only a single server which suffers from this issue, and this is
> the one what is used by RHQ itself! The others - which are only monitored
> by the agent, are quite fine. The only problem is with the 9.2 version that
> some system tables has been changed with 9.2, so there's not current_query
> anymore.
>
> This is strange, that in postgres log there's still lines like this:
> 2013-09-23 13:43:04 CEST ERROR:  column "current_query" does not exist at
> character 70
> 2013-09-23 13:43:04 CEST STATEMENT:  SELECT (SELECT COUNT(*) FROM
> pg_stat_activity where usename = $1 AND current_query != '<IDLE>') AS
> active,
>           (SELECT COUNT(*) FROM pg_stat_activity WHERE usename = $2) AS
> total
> 2013-09-23 13:43:44 CEST FATAL:  password authentication failed for user
> "postgres"
> 2013-09-23 13:43:46 CEST FATAL:  password authentication failed for user
> "postgres"
>
> I guess uninventory is not the perfect way to disable the plugin...
>
> max_connection is the default 100 for all the servers.
>
> Whenever it happen again, I will collect more Pg statistics!
>
> Regards,
>
> Attila
>
> 2013/9/23 Attila Heidrich
> > Postgres database stopped again... I guess the problem was the enormous
> > number of open files...
> >
> > We use 9.2, and we also use the postgres plugin - which still doesn't
> > really support 9.2 as far as I know.
> >
> > Altogether Postgres (only the one storing the RHQ database) can only run
> > for a few days, than the number of open files and the open connections
> > raise really high, and finally I should restart it.
> >
> > Usually I restart RHQ as well, I generally have no time to play with
> > "what to restart".
> >
> > The log is practically endless, I can quote, but I think I should know
> > something to look for in it.
> >
> > Regards,
> >
> > Attila

[Attila Heidrich]

OS is Debian Wheezy (64bit)
Pg: 9.2.4-1.pgdg70+1
RHQ is 4.9

Other hosts running the very same versions, the agent-only configs have no
problem at all (only the agents' problem with Pg 9.2). There are also other
Linux versions and a few Windows config (XP and 2008 server), most of the
platform runs Pg as well.

Attila

[Thomas Segismont]

Le 24/09/2013 12:46, Attila Heidrich a écrit :
> OS is Debian Wheezy (64bit)
> Pg: 9.2.4-1.pgdg70+1
> RHQ is 4.9
>
> Other hosts running the very same versions, the agent-only configs have
> no problem at all (only the agents' problem with Pg 9.2). There are also
> other Linux versions and a few Windows config (XP and 2008 server), most
> of the platform runs Pg as well.
>
> Attila
>
> rhq-users mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/rhq-users

I was talking about the platform running your RHQ database, not the 
platforms running managed Postgres servers.

What is the amount reported by "ulimit -n"?

It is very easy to reach 10 000 open files (e.g. if you have 20 opened 
connections and each have 500 open files)

Thomas

[Attila Heidrich]

The version data below belongs to the server which runs the RHQ server and
the RHQ database.

I understand it's easy to reach the 10K open files, I can also accept if it
is normal, but what are the recommended settings then? Other Pg servers
which don't run the RHQ server works for a long time with higher "active
backend" amount and much smaller "open files" amount!

root@ct-front:/etc/postgresql/9.2/main# ulimit -n
1024

Attila

2013/9/24 Attila Heidrich
> OS is Debian Wheezy (64bit)
> Pg: 9.2.4-1.pgdg70+1
> RHQ is 4.9
>
> Other hosts running the very same versions, the agent-only configs have no
> problem at all (only the agents' problem with Pg 9.2). There are also other
> Linux versions and a few Windows config (XP and 2008 server), most of the
> platform runs Pg as well.
>
> Attila

[Attila Heidrich]

Currently I have to restart the RHQ server twice a day with a cron job.
Can I restart the server itself with a script triggered by some alert?

Attila

2013/9/24 Attila Heidrich
> The version data below belongs to the server which runs the RHQ server and
> the RHQ database.
>
> I understand it's easy to reach the 10K open files, I can also accept if
> it is normal, but what are the recommended settings then? Other Pg servers
> which don't run the RHQ server works for a long time with higher "active
> backend" amount and much smaller "open files" amount!
>
> root@ct-front:/etc/postgresql/9.2/main# ulimit -n
> 1024
>
> Attila
>
> 2013/9/24 Attila Heidrich
> > OS is Debian Wheezy (64bit)
> > Pg: 9.2.4-1.pgdg70+1
> > RHQ is 4.9
> >
> > Other hosts running the very same versions, the agent-only configs have
> > no problem at all (only the agents' problem with Pg 9.2). There are also
> > other Linux versions and a few Windows config (XP and 2008 server), most of
> > the platform runs Pg as well.
> >
> > Attila

[Thomas Segismont]

Hi Attila,

Sorry for the late reply.

Le 27/09/2013 10:34, Attila Heidrich a écrit :
> Currently I have to restart the RHQ server twice a day with a cron job.
> Can I restart the server itself with a script triggered by some alert?

Most probably. You said in previous email that opened files prevented 
the server from running, but did you have any error messages in RHQ 
server or Postgres logs?

You might be impacted by Bug 1009640 - "JDBC connections leaked during 
baseline calculations". I can help you getting RHQ 4.9 rebuilt with the 
necessary patch.
> Attila
>
> 2013/9/24 Attila Heidrich <[e-mail]
> >
> The version data below belongs to the server which runs the RHQ
> server and the RHQ database.
>
> I understand it's easy to reach the 10K open files, I can also
> accept if it is normal, but what are the recommended settings then?
> Other Pg servers which don't run the RHQ server works for a long
> time with higher "active backend" amount and much smaller "open
> files" amount!
>
> root@ct-front:/etc/postgresql/9.2/main# ulimit -n
> 1024

I would recommend to set this to 2048. Can you also check the system 
wide opened file limit ("sysctl -n fs.file-max")
> Attila

If your server stops working again, can you run this command and share 
the results:

"ps --no-headers -f -U postgres | awk '{print $2}' | while read pid ; do 
echo "Postgres Process: " ; ps --no-headers -f -p $pid ; echo "Opened 
files: " ; lsof -p $pid | wc -l ; echo ; echo  ; done"

Regards,
Thomas

[Attila Heidrich]

Running since yesterday morning. I have enabled the postgres plugins, since
the postgres log showed, that the agent was still active after
"uninventory" - but with wrong credentials.

I had to restart the rhq-server (only the server, neither the Pg, nor the
agent) this morning, since the number of open files raised above 10K.

The processes before restart:

root@ct-front:~# ps auxw|grep postgres|grep rhq
postgres  1631  0.0  0.1 121300 44740 ?        Ss   Sep23   0:05 postgres:
rhq rhq 127.0.0.1(46903) idle
postgres  1735  0.0  0.1 118792 40384 ?        Ss   Sep23   0:09 postgres:
rhq rhq 127.0.0.1(36888) idle
postgres  3669  0.4  0.1 118572 29676 ?        Ss   08:40   0:00 postgres:
rhq rhq 127.0.0.1(54749) idle
postgres  3670  0.3  0.1 119164 27232 ?        Ss   08:40   0:00 postgres:
rhq rhq 127.0.0.1(54750) idle
postgres  5669  0.0  0.0 102028  5644 ?        Ss   Sep23   0:00 postgres:
rhq rhq 127.0.0.1(56115) idle
postgres  5680  0.0  0.1 115072 39132 ?        Ss   Sep23   0:05 postgres:
rhq rhq 127.0.0.1(56125) idle
postgres  6025  0.0  0.0 103304 10648 ?        Ss   Sep23   0:27 postgres:
rhq rhq 127.0.0.1(56210) idle
postgres  6041  0.0  0.1 123280 48024 ?        Ss   Sep23   0:11 postgres:
rhq rhq 127.0.0.1(56215) idle
postgres  6060  0.0  0.1 119700 42168 ?        Ss   Sep23   0:17 postgres:
rhq rhq 127.0.0.1(56237) idle
postgres  7387  0.0  0.0 106948 16364 ?        Ss   Sep23   1:01 postgres:
postgres rhq 127.0.0.1(60327) idle
postgres  8535  0.0  0.1 114120 36108 ?        Ss   02:51   0:04 postgres:
rhq rhq 127.0.0.1(48651) idle
postgres 11169  0.0  0.1 118580 39900 ?        Ss   00:00   0:07 postgres:
rhq rhq 127.0.0.1(59908) idle
postgres 11409  0.0  0.1 123716 46292 ?        Ss   Sep23   0:11 postgres:
rhq rhq 127.0.0.1(46198) idle
postgres 11431  0.0  0.2 124624 52432 ?        Ss   Sep23   0:47 postgres:
rhq rhq 127.0.0.1(46289) idle
postgres 11568  0.0  0.1 116364 40484 ?        Ss   Sep23   0:05 postgres:
rhq rhq 127.0.0.1(34407) idle
postgres 12416  0.0  0.1 122788 49160 ?        Ss   Sep23   0:17 postgres:
rhq rhq 127.0.0.1(52744) idle
postgres 15707  0.0  0.0 103324 10892 ?        Ss   Sep23   0:25 postgres:
rhq rhq 127.0.0.1(60936) idle
postgres 19382  0.0  0.1 122916 48488 ?        Ss   Sep23   0:08 postgres:
rhq rhq 127.0.0.1(56676) idle
postgres 19868  0.1  0.1 118104 38276 ?        Ss   07:01   0:09 postgres:
rhq rhq 127.0.0.1(45391) idle
postgres 20554  0.0  0.1 118244 40764 ?        Ss   04:00   0:08 postgres:
rhq rhq 127.0.0.1(55602) idle
postgres 21534  0.0  0.2 125260 50348 ?        Ss   01:00   0:12 postgres:
rhq rhq 127.0.0.1(37533) idle
postgres 22480  0.0  0.1 105776 30264 ?        Ss   Sep23   0:03 postgres:
rhq rhq 127.0.0.1(58453) idle
postgres 23255  0.0  0.1 114344 43000 ?        Ss   Sep23   0:28 postgres:
rhq rhq 127.0.0.1(52114) idle
postgres 23435  0.1  0.1 120388 44036 ?        Ss   07:21   0:05 postgres:
rhq rhq 127.0.0.1(47287) idle
postgres 24113  0.0  0.1 123212 48060 ?        Ss   04:21   0:05 postgres:
rhq rhq 127.0.0.1(57681) idle
postgres 25861  0.0  0.1 119720 43260 ?        Ss   01:25   0:05 postgres:
rhq rhq 127.0.0.1(40007) idle
postgres 27396  0.0  0.1 124076 48696 ?        Ss   04:40   0:08 postgres:
rhq rhq 127.0.0.1(59573) idle
postgres 28973  0.0  0.1 116924 39768 ?        Ss   Sep23   0:05 postgres:
rhq rhq 127.0.0.1(43984) idle
postgres 30394  0.2  0.1 125260 39752 ?        Ss   08:01   0:06 postgres:
rhq rhq 127.0.0.1(51100) idle
postgres 30455  0.0  0.2 123336 50372 ?        Ss   Sep23   0:15 postgres:
rhq rhq 127.0.0.1(34681) idle
postgres 30576  0.2  0.1 124888 39200 ?        Ss   08:01   0:06 postgres:
rhq rhq 127.0.0.1(51154) idle
postgres 30841  0.1  0.1 121776 38844 ?        Ss   08:03   0:04 postgres:
rhq rhq 127.0.0.1(51340) idle
postgres 31674  0.2  0.1 125424 39604 ?        Ss   08:08   0:04 postgres:
rhq rhq 127.0.0.1(51823) idle
postgres 31702  0.0  0.2 122984 49660 ?        Ss   Sep23   0:07 postgres:
rhq rhq 127.0.0.1(40525) idle
postgres 32761  0.0  0.1 123000 47332 ?        Ss   Sep23   0:07 postgres:
rhq rhq 127.0.0.1(53885) idle

and after restart:

root@ct-front:~# ps auxw|grep postgres|grep rhq
postgres  4571  4.6  0.0 105016 15876 ?        Ss   08:43   0:01 postgres:
rhq rhq 127.0.0.1(55153) idle
postgres  4572  0.0  0.0 102052  5656 ?        Ss   08:43   0:00 postgres:
rhq rhq 127.0.0.1(55156) idle
postgres  7387  0.0  0.0 106948 16364 ?        Ss   Sep23   1:01 postgres:
postgres rhq 127.0.0.1(60327) idle

I have uploaded a screenshot here:
https://www.dropbox.com/s/ysmh9uq37xxacs0/RHQ-restart.png

Regards,

Attila

2013/9/23 Attila Heidrich
> Postgres database stopped again... I guess the problem was the enormous
> number of open files...
>
> We use 9.2, and we also use the postgres plugin - which still doesn't
> really support 9.2 as far as I know.
>
> Altogether Postgres (only the one storing the RHQ database) can only run
> for a few days, than the number of open files and the open connections
> raise really high, and finally I should restart it.
>
> Usually I restart RHQ as well, I generally have no time to play with "what
> to restart".
>
> The log is practically endless, I can quote, but I think I should know
> something to look for in it.
>
> Regards,
>
> Attila

[Thomas Segismont]

Hi,

What is the value of "max_files_per_process" in your Postgres databases?

Thanks,
Thomas

Le 24/09/2013 09:34, Attila Heidrich a écrit :
> Running since yesterday morning. I have enabled the postgres plugins,
> since the postgres log showed, that the agent was still active after
> "uninventory" - but with wrong credentials.
>
> I had to restart the rhq-server (only the server, neither the Pg, nor
> the agent) this morning, since the number of open files raised above 10K.
>
> The processes before restart:
>
> root@ct-front:~# ps auxw|grep postgres|grep rhq
> postgres  1631  0.0  0.1 121300 44740 ?        Ss   Sep23   0:05
> postgres: rhq rhq 127.0.0.1(46903) idle
> postgres  1735  0.0  0.1 118792 40384 ?        Ss   Sep23   0:09
> postgres: rhq rhq 127.0.0.1(36888) idle
> postgres  3669  0.4  0.1 118572 29676 ?        Ss   08:40   0:00
> postgres: rhq rhq 127.0.0.1(54749) idle
> postgres  3670  0.3  0.1 119164 27232 ?        Ss   08:40   0:00
> postgres: rhq rhq 127.0.0.1(54750) idle
> postgres  5669  0.0  0.0 102028  5644 ?        Ss   Sep23   0:00
> postgres: rhq rhq 127.0.0.1(56115) idle
> postgres  5680  0.0  0.1 115072 39132 ?        Ss   Sep23   0:05
> postgres: rhq rhq 127.0.0.1(56125) idle
> postgres  6025  0.0  0.0 103304 10648 ?        Ss   Sep23   0:27
> postgres: rhq rhq 127.0.0.1(56210) idle
> postgres  6041  0.0  0.1 123280 48024 ?        Ss   Sep23   0:11
> postgres: rhq rhq 127.0.0.1(56215) idle
> postgres  6060  0.0  0.1 119700 42168 ?        Ss   Sep23   0:17
> postgres: rhq rhq 127.0.0.1(56237) idle
> postgres  7387  0.0  0.0 106948 16364 ?        Ss   Sep23   1:01
> postgres: postgres rhq 127.0.0.1(60327) idle
> postgres  8535  0.0  0.1 114120 36108 ?        Ss   02:51   0:04
> postgres: rhq rhq 127.0.0.1(48651) idle
> postgres 11169  0.0  0.1 118580 39900 ?        Ss   00:00   0:07
> postgres: rhq rhq 127.0.0.1(59908) idle
> postgres 11409  0.0  0.1 123716 46292 ?        Ss   Sep23   0:11
> postgres: rhq rhq 127.0.0.1(46198) idle
> postgres 11431  0.0  0.2 124624 52432 ?        Ss   Sep23   0:47
> postgres: rhq rhq 127.0.0.1(46289) idle
> postgres 11568  0.0  0.1 116364 40484 ?        Ss   Sep23   0:05
> postgres: rhq rhq 127.0.0.1(34407) idle
> postgres 12416  0.0  0.1 122788 49160 ?        Ss   Sep23   0:17
> postgres: rhq rhq 127.0.0.1(52744) idle
> postgres 15707  0.0  0.0 103324 10892 ?        Ss   Sep23   0:25
> postgres: rhq rhq 127.0.0.1(60936) idle
> postgres 19382  0.0  0.1 122916 48488 ?        Ss   Sep23   0:08
> postgres: rhq rhq 127.0.0.1(56676) idle
> postgres 19868  0.1  0.1 118104 38276 ?        Ss   07:01   0:09
> postgres: rhq rhq 127.0.0.1(45391) idle
> postgres 20554  0.0  0.1 118244 40764 ?        Ss   04:00   0:08
> postgres: rhq rhq 127.0.0.1(55602) idle
> postgres 21534  0.0  0.2 125260 50348 ?        Ss   01:00   0:12
> postgres: rhq rhq 127.0.0.1(37533) idle
> postgres 22480  0.0  0.1 105776 30264 ?        Ss   Sep23   0:03
> postgres: rhq rhq 127.0.0.1(58453) idle
> postgres 23255  0.0  0.1 114344 43000 ?        Ss   Sep23   0:28
> postgres: rhq rhq 127.0.0.1(52114) idle
> postgres 23435  0.1  0.1 120388 44036 ?        Ss   07:21   0:05
> postgres: rhq rhq 127.0.0.1(47287) idle
> postgres 24113  0.0  0.1 123212 48060 ?        Ss   04:21   0:05
> postgres: rhq rhq 127.0.0.1(57681) idle
> postgres 25861  0.0  0.1 119720 43260 ?        Ss   01:25   0:05
> postgres: rhq rhq 127.0.0.1(40007) idle
> postgres 27396  0.0  0.1 124076 48696 ?        Ss   04:40   0:08
> postgres: rhq rhq 127.0.0.1(59573) idle
> postgres 28973  0.0  0.1 116924 39768 ?        Ss   Sep23   0:05
> postgres: rhq rhq 127.0.0.1(43984) idle
> postgres 30394  0.2  0.1 125260 39752 ?        Ss   08:01   0:06
> postgres: rhq rhq 127.0.0.1(51100) idle
> postgres 30455  0.0  0.2 123336 50372 ?        Ss   Sep23   0:15
> postgres: rhq rhq 127.0.0.1(34681) idle
> postgres 30576  0.2  0.1 124888 39200 ?        Ss   08:01   0:06
> postgres: rhq rhq 127.0.0.1(51154) idle
> postgres 30841  0.1  0.1 121776 38844 ?        Ss   08:03   0:04
> postgres: rhq rhq 127.0.0.1(51340) idle
> postgres 31674  0.2  0.1 125424 39604 ?        Ss   08:08   0:04
> postgres: rhq rhq 127.0.0.1(51823) idle
> postgres 31702  0.0  0.2 122984 49660 ?        Ss   Sep23   0:07
> postgres: rhq rhq 127.0.0.1(40525) idle
> postgres 32761  0.0  0.1 123000 47332 ?        Ss   Sep23   0:07
> postgres: rhq rhq 127.0.0.1(53885) idle
>
> and after restart:
>
> root@ct-front:~# ps auxw|grep postgres|grep rhq
> postgres  4571  4.6  0.0 105016 15876 ?        Ss   08:43   0:01
> postgres: rhq rhq 127.0.0.1(55153) idle
> postgres  4572  0.0  0.0 102052  5656 ?        Ss   08:43   0:00
> postgres: rhq rhq 127.0.0.1(55156) idle
> postgres  7387  0.0  0.0 106948 16364 ?        Ss   Sep23   1:01
> postgres: postgres rhq 127.0.0.1(60327) idle
>
> I have uploaded a screenshot here:
> https://www.dropbox.com/s/ysmh9uq37xxacs0/RHQ-restart.png
>
> Regards,
>
> Attila
>
> 2013/9/23 Attila Heidrich <[e-mail]
> >
> Postgres database stopped again... I guess the problem was the
> enormous number of open files...
>
> We use 9.2, and we also use the postgres plugin - which still
> doesn't really support 9.2 as far as I know.
>
> Altogether Postgres (only the one storing the RHQ database) can only
> run for a few days, than the number of open files and the open
> connections raise really high, and finally I should restart it.
>
> Usually I restart RHQ as well, I generally have no time to play with
> "what to restart".
>
> The log is practically endless, I can quote, but I think I should
> know something to look for in it.
>
>
> Regards,
>
> Attila
>
>
>
>
> rhq-users mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/rhq-users

[Attila Heidrich]

> What is the value of "max_files_per_process" in your Postgres databases?
>
> Thanks,
> Thomas

It is on the default value, maybe 1000?

root@ct-front:/etc/postgresql/9.2/main# grep max_files_per_process *
postgresql.conf:#max_files_per_process = 1000           # min 25

2013/9/23 Attila Heidrich
> Postgres database stopped again... I guess the problem was the enormous
> number of open files...
>
> We use 9.2, and we also use the postgres plugin - which still doesn't
> really support 9.2 as far as I know.
>
> Altogether Postgres (only the one storing the RHQ database) can only run
> for a few days, than the number of open files and the open connections
> raise really high, and finally I should restart it.
>
> Usually I restart RHQ as well, I generally have no time to play with "what
> to restart".
>
> The log is practically endless, I can quote, but I think I should know
> something to look for in it.
>
> Regards,
>
> Attila

[Thomas Segismont]

Le 24/09/2013 11:42, Attila Heidrich a écrit :
> > What is the value of "max_files_per_process" in your Postgres databases?
> >
> > Thanks,
> > Thomas
>
> It is on the default value, maybe 1000?
>
> root@ct-front:/etc/postgresql/9.2/main# grep max_files_per_process *
> postgresql.conf:#max_files_per_process = 1000           # min 25

Yes, if unset, the default is 1000. Which OS your database is running on?