Change in vdsm[master]: Multiply amount of open file descriptor for vdsm user

Yaniv Bronhaim has uploaded a new change for review.

Change subject: Multiply amount of open file descriptor for vdsm user
......................................................................

Multiply amount of open file descriptor for vdsm user

As part of fixing fds leak we increase the amount of available open fds
to allow stability in large scale of simultaneously operations.
Currently lv operations can't run simultaneously and
wait for pior operation to end. Simultaneously attach sd requests open
fds and close them only after end, when operation waits on lv operation
vdsm keeps the fds open for long time.

Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=920532
Signed-off-by: Yaniv Bronhaim 
---
M vdsm/limits.conf
1 file changed, 3 insertions(+), 2 deletions(-)

git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/66/13566/1

diff --git a/vdsm/limits.conf b/vdsm/limits.conf
index 287f70d..fc018a6 100644
--- a/vdsm/limits.conf
+++ b/vdsm/limits.conf
@@ -1,6 +1,7 @@
 # This limits are intended for medium VDSM hosts, for large hosts scale these
 # numbers appropriately.

-# nproc\nofile should be at least 150 + (3 * <The maximum amount of running VMs>)
+# nproc should be at least 150 + The maximum amount of running VMs (640) + The maximum amount of storage operations
+# nofile should be at least 3(stdin,stdour,stderr) * each external process. Each VM run or storage operation initiates external process.
 vdsm - nproc 4096
-vdsm - nofile 4096
+vdsm - nofile 8192

--
To view, visit http://gerrit.ovirt.org/13566
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: Multiply amount of open file descriptor for vdsm user
......................................................................

Patch Set 1:

Build Started http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/1798/ (1/2)

--
To view, visit http://gerrit.ovirt.org/13566
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim 
Gerrit-Reviewer: oVirt Jenkins CI Server

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: Multiply amount of open file descriptor for vdsm user
......................................................................

Patch Set 1:

Build Started http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/1849/ (2/2)

--
To view, visit http://gerrit.ovirt.org/13566
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim 
Gerrit-Reviewer: oVirt Jenkins CI Server

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: Multiply amount of open file descriptor for vdsm user
......................................................................

Patch Set 1:

Build Successful

http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/1798/ : SUCCESS

http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/1849/ : SUCCESS

--
To view, visit http://gerrit.ovirt.org/13566
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim 
Gerrit-Reviewer: Ayal Baron 
Gerrit-Reviewer: Barak Azulay 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Eduardo 
Gerrit-Reviewer: Saggi Mizrahi 
Gerrit-Reviewer: oVirt Jenkins CI Server

Eduardo has posted comments on this change.

Change subject: Multiply amount of open file descriptor for vdsm user
......................................................................

Patch Set 1: I would prefer that you didn't submit this

(1 inline comment)

We need to reduce the number of open procs and the operations opening fd's unnecessarily.

....................................................
File vdsm/limits.conf
Line 3: 
Line 4: # nproc should be at least 150 + The maximum amount of running VMs (640) + The maximum amount of storage operations
Line 5: # nofile should be at least 3(stdin,stdour,stderr) * each external process. Each VM run or storage operation initiates external process.
Line 6: vdsm - nproc 4096
Line 7: vdsm - nofile 8192
Please update the comment.

If you assume that each proc has at least 3 open sd's please change nofile to something sensible like nproc * 3

--
To view, visit http://gerrit.ovirt.org/13566
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim 
Gerrit-Reviewer: Ayal Baron 
Gerrit-Reviewer: Barak Azulay 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Eduardo 
Gerrit-Reviewer: Saggi Mizrahi 
Gerrit-Reviewer: oVirt Jenkins CI Server

Change in vdsm[master]: Increasing the amount of open file descriptor for vdsm user

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: Increasing the amount of open file descriptor for vdsm user
......................................................................

Patch Set 2:

Build Started http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/1810/ (1/2)

--
To view, visit http://gerrit.ovirt.org/13566
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim 
Gerrit-Reviewer: Ayal Baron 
Gerrit-Reviewer: Barak Azulay 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Eduardo 
Gerrit-Reviewer: Saggi Mizrahi 
Gerrit-Reviewer: Yaniv Bronhaim 
Gerrit-Reviewer: oVirt Jenkins CI Server

Change in vdsm[master]: Increasing the amount of open file descriptor for vdsm user

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: Increasing the amount of open file descriptor for vdsm user
......................................................................

Patch Set 2:

Build Started http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/1861/ (2/2)

--
To view, visit http://gerrit.ovirt.org/13566
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim 
Gerrit-Reviewer: Ayal Baron 
Gerrit-Reviewer: Barak Azulay 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Eduardo 
Gerrit-Reviewer: Saggi Mizrahi 
Gerrit-Reviewer: Yaniv Bronhaim 
Gerrit-Reviewer: oVirt Jenkins CI Server

Change in vdsm[master]: Increasing the amount of open file descriptor for vdsm user

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: Increasing the amount of open file descriptor for vdsm user
......................................................................

Patch Set 2:

Build Successful

http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/1810/ : SUCCESS

http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/1861/ : SUCCESS

--
To view, visit http://gerrit.ovirt.org/13566
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim 
Gerrit-Reviewer: Ayal Baron 
Gerrit-Reviewer: Barak Azulay 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Eduardo 
Gerrit-Reviewer: Saggi Mizrahi 
Gerrit-Reviewer: Yaniv Bronhaim 
Gerrit-Reviewer: oVirt Jenkins CI Server

Change in vdsm[master]: Increasing the amount of open file descriptor for vdsm user

[Dan Kenigsberg]

Dan Kenigsberg has posted comments on this change.

Change subject: Increasing the amount of open file descriptor for vdsm user
......................................................................

Patch Set 2: Looks good to me, approved

--
To view, visit http://gerrit.ovirt.org/13566
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim 
Gerrit-Reviewer: Ayal Baron 
Gerrit-Reviewer: Barak Azulay 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Eduardo 
Gerrit-Reviewer: Saggi Mizrahi 
Gerrit-Reviewer: Yaniv Bronhaim 
Gerrit-Reviewer: oVirt Jenkins CI Server

Change in vdsm[master]: Increasing the amount of open file descriptor for vdsm user

Yaniv Bronhaim has posted comments on this change.

Change subject: Increasing the amount of open file descriptor for vdsm user
......................................................................

Patch Set 2: Verified

--
To view, visit http://gerrit.ovirt.org/13566
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim 
Gerrit-Reviewer: Ayal Baron 
Gerrit-Reviewer: Barak Azulay 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Eduardo 
Gerrit-Reviewer: Saggi Mizrahi 
Gerrit-Reviewer: Yaniv Bronhaim 
Gerrit-Reviewer: oVirt Jenkins CI Server

Change in vdsm[master]: Increasing the amount of open file descriptor for vdsm user

[Dan Kenigsberg]

Dan Kenigsberg has submitted this change and it was merged.

Change subject: Increasing the amount of open file descriptor for vdsm user
......................................................................

Increasing the amount of open file descriptor for vdsm user

As part of fixing fds leak we increase the amount of available open fds
to allow stability in large scale of simultaneously operations.
Currently lv operations can't run simultaneously and
wait for pior operation to end. Simultaneously attach sd requests open
fds and close them only after end, when operation waits on lv operation
vdsm keeps the fds open for long time.

Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=920532
Signed-off-by: Yaniv Bronhaim 
---
M vdsm/limits.conf
1 file changed, 6 insertions(+), 2 deletions(-)

Approvals:
  Yaniv Bronhaim: Verified
  Dan Kenigsberg: Looks good to me, approved

--
To view, visit http://gerrit.ovirt.org/13566
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ide73b075c56c83d0298a9cd3a0892b0ce7cd46f6
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim 
Gerrit-Reviewer: Ayal Baron 
Gerrit-Reviewer: Barak Azulay 
Gerrit-Reviewer: Dan Kenigsberg 
Gerrit-Reviewer: Eduardo 
Gerrit-Reviewer: Saggi Mizrahi 
Gerrit-Reviewer: Yaniv Bronhaim 
Gerrit-Reviewer: oVirt Jenkins CI Server